Marine unmanned aerial vehicle emergency material throwing device and control strategy thereof

Abstract

The invention provides a marine unmanned aerial vehicle emergency material throwing device which comprises an unmanned aerial vehicle platform, and the unmanned aerial vehicle platform carries a navigation system, a sensor technology and a visual perception system. The throwing mechanism is connected with the unmanned aerial vehicle platform and is used for executing a throwing action; the material storage box is installed on the unmanned aerial vehicle platform and used for storing emergency materials. According to the marine unmanned aerial vehicle emergency material throwing device, through visual perception and deep reinforcement learning, the unmanned aerial vehicle can accurately calculate a throwing track and a drop point, and it is ensured that materials are accurately delivered to a target position. The invention further provides an emergency material throwing control strategy of the marine unmanned aerial vehicle, the emergency material throwing control strategy comprises the throwing device, deep reinforcement learning is carried out on the control strategy through a visual perception system, the optimal control strategy is learned through continuous trial and error, and accurate throwing is achieved. According to the marine unmanned aerial vehicle emergency material throwing control strategy, unmanned aerial vehicle autonomous accurate throwing based on visual driving is achieved, and errors caused by the environment in the throwing process are accommodated through a reinforcement learning method.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle emergency material throwing technology, and in particular to a marine unmanned aerial vehicle emergency material throwing device and a control strategy thereof. Background Art

[0002] In recent years, drone technology has made significant progress. It has not only been widely used in the military field, but also played an important role in many fields such as public safety and disaster relief. The high maneuverability, flexibility and remote operation capabilities of drones make them an ideal choice for emergency response missions. Especially in maritime rescue, drones can quickly reach the incident area and perform key tasks such as aerial reconnaissance and material delivery.

[0003] Existing technologies also have shortcomings in the autonomy and intelligence of drones. Traditional drones often require manual operation and monitoring when performing drop missions, which may lead to slow response and operational errors in emergency situations. In addition, drones have limited adaptability and decision-making capabilities when facing emergencies, making it difficult to achieve fast and effective autonomous drop. These shortcomings limit the potential and effectiveness of drones in maritime rescue operations. Summary of the invention

[0004] The present invention proposes a marine UAV emergency material throwing device and a control strategy thereof, which can ensure that the UAV emergency material throwing device can stably and accurately complete the throwing task.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A marine drone emergency material throwing device, comprising:

[0007] a. UAV platform, which is equipped with a navigation system, sensor technology and visual perception system;

[0008] b. A throwing mechanism connected to the UAV platform for performing the throwing action;

[0009] c. A material storage box, installed on the UAV platform, for storing emergency materials.

[0010] Furthermore, the visual perception system is used to implement a precise throwing strategy based on visual perception, including a state perception module, an accuracy perception module, a reinforcement learning module, an action selection module, a throwing environment module and a state update module.

[0011] A control strategy for dropping emergency materials from a marine unmanned aerial vehicle includes the dropping device described above. The control strategy utilizes the visual perception system to perform deep reinforcement learning, and learns the optimal control strategy through continuous trial and error to achieve accurate dropping.

[0012] Furthermore, the state perception module is used to obtain the attitude angle, speed and shaking direction of the drone when performing a throwing operation.

[0013] Further, the accuracy perception module is used to calculate the distance between the image target positions To assess target movement;

[0014] Where:

[0015] ΔP i : Indicates the change in the distance between the target position in consecutive frames of visual images; it is the Euclidean distance that measures the change in the target position between two consecutive frames of images, and is used to describe the movement of the target in the camera's field of view;

[0016] u i and v i : represent the horizontal and vertical coordinates of the target distance from the center of the image in the i-th frame of the visual image;

[0017] u i+1 and v i+1 : represent the horizontal and vertical coordinates of the target distance from the center of the image in the i+1th frame of the visual image;

[0018] (u i+1 -u i ) 2 +(v i+1 -v i ) 2 : Represents the square of the change in the target position on the horizontal and vertical axes respectively.

[0019] Furthermore, the reinforcement learning module is used to adjust the state of the drone to the next state and select the next throwing action by maximizing the state action value function H π (S i ,D i ) to achieve the highest accuracy of autonomous precision casting of UAVs based on vision-driven.

[0020] Furthermore, the action selection module is used to execute the throwing action and interact with the environment through the throwing environment module.

[0021] Furthermore, the state update module is used to store and update the state information of the drone for use by the reinforcement learning module.

[0022] Furthermore, the optimal state action value function H πT (S i ,D i ) is expressed as Used to find a strategy that maximizes future rewards among all possible strategies;

[0023] Where:

[0024] H πT (S i ,D i ): Among all possible strategies π, from state S i Select Action D i The best value;

[0025] maxπ: Take the maximum value among all possible strategies π;

[0026] R i : In state S i Next, select Action D i Immediate rewards after

[0027] S=S i :The current status is S i ;

[0028] D=D i :The action selected is D i ;

[0029] π: Strategy.

[0030] Beneficial effects of the technical solution provided by this application:

[0031] 1. The emergency material throwing device of the UAV at sea can accurately calculate the throwing trajectory and landing point through visual perception and deep reinforcement learning, ensuring that the materials are accurately delivered to the target location. With the assistance of the navigation system, sensor technology and visual perception system, the UAV can complete the throwing task autonomously, reducing dependence on operators.

[0032] 2. The control strategy for dropping emergency materials by UAVs at sea is based on the autonomous and precise dropping of UAVs driven by vision. It uses reinforcement learning methods to accommodate the errors caused by the environment during dropping, breaking the difficulties in traditional dropping methods. The autonomous decision-making ability and precise dropping ability of UAVs are further improved, solving the problem of precise dropping in uncertain environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1This is a schematic diagram of the emergency material throwing device of the marine drone of the present invention;

[0035] Figure 2 This is a schematic diagram of the architecture of the emergency material throwing control strategy of the marine UAV of the present invention;

[0036] Figure 3 Schematic diagram of the overall framework of the algorithm logic of the present invention.

[0037] In the picture: 10 UAV platforms; 20 launching mechanisms; 30 material storage boxes. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Reference Figure 1 A drone emergency material throwing device includes a drone platform 10, a throwing mechanism 20, and a material storage box 30; the drone platform 10 is responsible for carrying and transporting materials, the throwing mechanism 20 is responsible for releasing materials at a designated location, and the control system is responsible for the precise control of the entire throwing process.

[0040] Reference Figure 2-3 ,A control strategy for dropping emergency materials by UAV adopts a precise dropping strategy based on visual perception.,Firstly, the attitude angle, speed during dropping, and shaking direction (movement direction) of the UAV are obtained,and the optimal control strategy is learned through deep reinforcement learning and continuous trial and error.

[0041] The specific design ideas of the vision-driven UAV autonomous precision casting decision model include state perception module, accuracy perception module, reinforcement learning module, action module, casting environment module, and state module. Using strategy π, the UAV selects the casting action D i , get the state S when throwing operation is performed i , through the state perception module to perceive the state S i , through the accuracy perception module to perceive the throwing accuracy range R i , adjust the drone state to S through the reinforcement learning module i+1 , using strategy π i+1 , the drone selects the throwing action D i+1 , through the accuracy perception module to perceive the throwing accuracy range R i+1 , thereby learning the optimal control strategy through trial and error.

[0042] The drone moves to different positions for shooting. The target position in the camera field of view is different. Continuous frames of visual images are input, and the position coordinates P of the target from the center of the image on the continuous frame visual images are detected. * (u i ,v i ), according to the different time series t on the synthetic motion trajectory image 1 ,t 2 ,t 3 , compare the detected target image coordinates of each frame, and find the spacing of the image target position:

[0043]

[0044] Where:

[0045] ΔP i : Indicates the change in the distance between the target position in consecutive frames of visual images; it is the Euclidean distance that measures the change in the target position between two consecutive frames of images, and is used to describe the movement of the target in the camera's field of view;

[0046] u i and v i : represent the horizontal and vertical coordinates of the target distance from the center of the image in the i-th frame of the visual image;

[0047] u i+1 and v i+1 : represent the horizontal and vertical coordinates of the target distance from the center of the image in the i+1th frame of the visual image;

[0048] (u i+1 -u i ) 2 +(v i+1 -v i ) 2 : Represents the square of the change in the target position on the horizontal and vertical axes respectively.

[0049] The object is regarded as the target to be detected, and the aerial surface image is used as the background. The single-frame visual image of the throwing result of the drone in a certain state is sent to the target detection algorithm to solve the distance deviation Mc between the object landing point and the target center, and the distance deviation Mc is compared with the level Ra to obtain the accuracy, and the sample landing area is divided into 5 levels:

[0050]

[0051] Where:

[0052] r: represents the output result of the casting accuracy level;

[0053] g(Mc): is a function that maps the distance deviation Mc to different accuracy levels r;

[0054] Mc: indicates the distance deviation between the object landing point and the target center;

[0055] Ra1, Ra2, Ra3, Ra4, Ra5: represent different levels of accuracy thresholds, which are used to classify the distance deviation Mc into different levels.

[0056] r: In the formula, r is used as a threshold to determine in which interval Mc falls.

[0057] After throwing, we get the landing point deviation image, and through the state perception algorithm and calculation based on deep learning, we get r=5.

[0058] The drone is in state S based on strategy π i Select Action D i The value of the feedback given by the environment is r i+1 (S i ,D i ). The closer the landing point is to the target center area after throwing, the higher the score is, and vice versa. We start from state S under strategy π. i Select Action D i The value of H π (S i ,D i ):

[0059]

[0060] In the formula:

[0061] H π (S i ,D i ): state action value function, indicating that under the strategy π, from state S i Execute action D i The expected return that can be obtained in the future;

[0062] E π : Expected value operator, indicating the calculation of the expected value of subsequent rewards under strategy π;

[0063] The summation symbol represents the sum of all items from step 0 to step n;

[0064] γ k : The k-th power of the discount factor γ, which is used to calculate the current value of future rewards. The discount factor γ is a value between 0 and 1, which is used to weigh the importance of immediate rewards and future rewards;

[0065] r k+i+1 : The reward obtained at time k+i+1, that is, from the current state S iExecute action D i After that, the reward obtained after k steps.

[0066] Let the state action value function H π (S i ,D i ) is maximized to achieve the highest accuracy of autonomous precision casting of the UAV based on vision-driven, and the optimal state action value function H πT (S i ,D i ) is expressed as:

[0067]

[0068] Where: H πT (S i ,D i ): Among all possible strategies π, from state S i Select Action D i The best value;

[0069] maxπ: Take the maximum value among all possible strategies π;

[0070] R i : In state S i Next, select Action D i Immediate rewards after

[0071] S=S i :The current status is S i ;

[0072] D=D i :The action selected is D i ;

[0073] π: Strategy.

[0074] The drone selects strategy π by continuously improving its own behavior. When the drone learns the optimal strategy πt, it maximizes the expected future return value.

[0075] The pseudo code is as follows:

[0076] For any state S i ∈S,D i ∈D, initialize the action value function value H π (S i ,D i ), the drone starts exploring from the starting point.

[0077] for (for each batch):

[0078] ① Initialization state S;

[0079] ②for (for each step in the batch): use strategy π to select an action D from the action set D i Execute; after the action is executed, observe the reward value of the environment feedback and enter the new state S i+1 ; Update H value according to the formula; let the current state S = S i+1 (S i+1 Indicates D i The corresponding next state); until S terminates.

[0080] We continuously reinforce learning to find a suitable control strategy to maximize the state-action value function, and we iterate to find the optimal strategy πt.

[0081] The drone continuously uses visual perception to adjust state parameters such as attitude angle, shaking speed, and relative relationship with ground targets. Through deep reinforcement learning, it solves the impact of video acquisition and processing transmission delay and decision-making control delay, thereby solving the decision-making optimization problem and achieving autonomous and high-precision casting.

[0082] In summary, the control strategy for UAV emergency material delivery is based on the autonomous and precise delivery of UAVs driven by vision. It uses reinforcement learning methods to accommodate the errors caused by the environment during delivery, breaking the difficulties in traditional delivery methods. The autonomous decision-making ability and precise delivery ability of UAVs are further improved, solving the problem of precise delivery in uncertain environments. The ground control station is flexibly applied to monitor flight information in real time and transmit images in real time, thereby improving the accuracy of UAV route control, ensuring the stable flight and precise positioning of UAVs in complex environments, ensuring the safety and reliability of the system, and effectively improving the stability of delivery.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A marine drone emergency material throwing device, characterized in that: include: a. An unmanned aerial vehicle platform (10), wherein the unmanned aerial vehicle platform (10) is equipped with a navigation system, sensor technology and a visual perception system; b. a throwing mechanism (20), connected to the drone platform, for performing a throwing action; c. A material storage box (30), installed on the UAV platform, for storing emergency materials.

2. The throwing device according to claim 1, wherein the visual perception system is used to implement an accurate throwing strategy based on visual perception, including a state perception module, an accuracy perception module, a reinforcement learning module, an action selection module, a throwing environment module and a state update module.

3. A control strategy for dropping emergency supplies from a marine drone, comprising a dropping device as described in any one of claims 1-2, characterized in that: The control strategy utilizes the visual perception system to perform deep reinforcement learning, and learns the optimal control strategy through continuous trial and error to achieve accurate casting.

4. The control strategy according to claim 3, wherein the state perception module is used to obtain the attitude angle, speed and shaking direction state of the drone when performing a throwing operation.

5. The control strategy according to claim 3, wherein the accuracy perception module is used to calculate the distance between the image target positions. To assess target movement; Where: ΔP i : Indicates the change in the distance between the target position in consecutive frames of visual images; it is the Euclidean distance that measures the change in the target position between two consecutive frames of images, and is used to describe the movement of the target in the camera's field of view; u i and v i : represent the horizontal and vertical coordinates of the target distance from the center of the image in the i-th frame of the visual image; u i+1 and v i+1 : represent the horizontal and vertical coordinates of the target distance from the center of the image in the i+1th frame of the visual image; (u i+1 -u i ) 2 +(v i+1 -v i ) 2 : Represents the square of the change in the target position on the horizontal and vertical axes respectively.

6. The control strategy according to claim 3, wherein the reinforcement learning module is used to adjust the state of the drone to the next state and select the next throwing action by maximizing the state action value function H π (S i ,D i ) to achieve the highest accuracy of autonomous precision casting of UAVs based on vision-driven; H π (S i ,D i ): Under strategy π, from state S i Select Action D i value.

7. The control strategy according to claim 3, wherein the action selection module is used to perform the throwing action and interact with the environment through the throwing environment module.

8. The control strategy according to claim 3, wherein the state update module is used to store and update the state information of the drone for use by the reinforcement learning module.

9. The control strategy according to claim 6, wherein the optimal state action value function H πT (S i ,D i ) is expressed as Used to find a strategy that maximizes future rewards among all possible strategies; Where: H πT (S i ,D i ): Among all possible strategies π, from state S i Select Action D i The best value; maxπ: Take the maximum value among all possible strategies π; R i : In state S i Next, select Action D i Immediate rewards after S=S i :The current status is S i ; D=D i :The action selected is D i ; π: Strategy.

Images