Personnel falling-into-water intelligent identification and automatic throwing lifesaving method based on unmanned aerial vehicle

Abstract

The invention discloses an unmanned aerial vehicle-based personnel falling-into-water intelligent identification and automatic throwing lifesaving method, which comprises the following steps: acquiring a water area image acquired by an unmanned aerial vehicle, and carrying out real-time detection on a falling-into-water person on the water area image to obtain pixel coordinate data of the falling-into-water person; based on the position coordinate data of the unmanned aerial vehicle and the pixel coordinate data of the personnel falling into water, obtaining three-dimensional geographic coordinate data of the personnel falling into water; on the basis of the three-dimensional geographic coordinate data and the real-time environment information of the drowning person, predicting the putting position of the lifesaving device to obtain a target putting position; and based on the target putting position, controlling the unmanned aerial vehicle to put the lifesaving device to the position of the drowning person. According to the method, the falling position of the person can be automatically identified and positioned in real time, and the unmanned aerial vehicle autonomously completes accurate throwing of the lifesaving device, so that the rescue response time is shortened, manual operation dependence is reduced, and the rescue intelligence and success rate are improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) rescue, and more particularly to a method for intelligent identification and automatic throwing rescue of people falling into water based on UAVs. Background Technology

[0002] With the increasing demands for urban waterway management and public safety, improving safety patrols of open waterways such as rivers, lakes, and reservoirs has become crucial. Traditional waterway patrols rely heavily on shore-based cameras, manual monitoring, or patrol boats, which suffer from limited monitoring range, delayed discovery of people falling into the water, and slow rescue response.

[0003] In recent years, drones have been increasingly used in water patrol and emergency rescue scenarios due to their high mobility and wide coverage, enabling them to reach the scene quickly for monitoring and rescue assistance. However, existing drone-based water patrol and rescue technologies rely on manual identification and operation, resulting in problems such as untimely rescue, slow response speed, low deployment accuracy, and insufficient intelligence. Summary of the Invention

[0004] In view of the above problems, this application provides a method for intelligent identification and automatic throwing of rescue devices for people falling into water based on drones. It can automatically identify and locate the location of people falling into the water in real time, and the drone can autonomously complete the precise throwing of rescue devices, thereby shortening the rescue response time, reducing reliance on human operation, and improving the intelligence and success rate of rescue.

[0005] In a first aspect, this application provides a method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs). The method includes: acquiring water area images collected by the UAV and performing real-time detection of people falling into the water in the water area images to obtain pixel coordinate data of the people falling into the water; obtaining three-dimensional geographic coordinate data of the people falling into the water based on the position coordinate data of the UAV and the pixel coordinate data of the people falling into the water; predicting the deployment location of the rescue device based on the three-dimensional geographic coordinate data of the people falling into the water and real-time environmental information to obtain the target deployment location; and controlling the UAV to deploy the rescue device to the location of the people falling into the water based on the target deployment location. In the technical solution of this application embodiment, firstly, water area images collected by a drone are acquired, and real-time detection of people falling into the water is performed on the water area images to obtain pixel coordinate data of the people falling into the water. Then, based on the position coordinate data of the drone and the pixel coordinate data of the people falling into the water, the three-dimensional geographic coordinate data of the people falling into the water is obtained. Then, based on the three-dimensional geographic coordinate data of the people falling into the water and real-time environmental information, the deployment location of the rescue device is predicted to obtain the target deployment location. Finally, based on the target deployment location, the drone is controlled to deploy the rescue device to the location of the people falling into the water. This enables automatic identification and real-time positioning of the location of people falling into the water, and the drone autonomously completes the precise deployment of the rescue device, effectively shortening the rescue response time, reducing reliance on human operation, and improving the intelligence and success rate of rescue.

[0006] In some embodiments, the deployment location of the rescue device is predicted based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information to obtain the target deployment location. This includes: establishing a kinematic model of the rescue device based on the three-dimensional geographic coordinate data of the person who fell into the water, according to the initial altitude data and initial speed data of the UAV, to calculate the deployment trajectory of the rescue device and obtain the initial deployment location; and dynamically correcting the initial deployment location based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information to obtain the target deployment location. The real-time environmental information includes real-time wind speed information, UAV attitude information, and water drift information. In some embodiments, the initial deployment location is dynamically corrected based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information to obtain the target deployment location. This includes: if the initial deployment location meets a first preset condition, performing compensation displacement prediction based on real-time environmental information to dynamically correct the initial deployment location; and obtaining the target deployment location based on the difference between the three-dimensional geographic coordinate data of the person who fell into the water and the compensation displacement.

[0007] In some embodiments, obtaining the three-dimensional geographic coordinate data of the person who fell into the water based on the location coordinate data of the UAV and the pixel coordinate data of the person who fell into the water includes: converting the pixel coordinate data of the person who fell into the water into normalized planar coordinates of the camera coordinate system through a camera intrinsic parameter matrix according to a perspective projection model, to obtain the target direction vector of the camera coordinate system; performing a rotation transformation on the target direction vector of the camera coordinate system to the geographic coordinate system based on a rotation matrix, to obtain the target direction vector of the geographic coordinate system; and calculating the latitude and longitude position of the person who fell into the water through a coordinate transformation chain based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system, to obtain the three-dimensional geographic coordinate data of the person who fell into the water.

[0008] In some embodiments, the rotation matrix includes a first rotation matrix and a second rotation matrix; based on the rotation matrix, the target direction vector of the camera coordinate system is rotated and transformed into the geographic coordinate system to obtain the target direction vector of the geographic coordinate system, including: determining the first rotation matrix according to the fixed installation angle of the UAV, and transforming the target direction vector of the camera coordinate system to the target direction vector of the body coordinate system through the first rotation matrix; determining the second rotation matrix according to the UAV's inertial measurement device, and transforming the target direction vector of the body coordinate system to the target direction vector of the geographic coordinate system through the second rotation matrix, wherein the target direction vector of the geographic coordinate system represents the relative position of the UAV and the person who fell into the water.

[0009] In some embodiments, based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system, the latitude and longitude position of the person who fell into the water is calculated through a coordinate transformation chain to obtain the three-dimensional geographic coordinate data of the person who fell into the water. This includes: calculating the three-dimensional geographic distance between the UAV and the person who fell into the water through a ray equation based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system to obtain the three-dimensional geographic coordinate data of the person who fell into the water; wherein, the three-dimensional geographic coordinate data of the person who fell into the water includes altitude data, and the altitude data is related to the water surface height.

[0010] In some embodiments, real-time detection of people falling into the water is performed on water area images to obtain pixel coordinate data of the people falling into the water. This includes: scaling and normalizing the water area images to obtain a normalized image; extracting multi-level features from the normalized image using target detection and recognition algorithms to obtain water surface texture and edge information features, human contour features and motion features, and fusing the water surface texture and edge information features, human contour features and motion features at multiple scales to obtain a fused feature map; and performing prediction processing and filtering on the fused feature map to obtain pixel coordinate data of the people falling into the water.

[0011] In some embodiments, after controlling the drone to deploy the rescue device to the location of the person in the water based on the target deployment location, the method further includes: confirming whether the rescue device has reached the location of the person in the water, and feeding back the rescue result to the ground command center in real time.

[0012] On the other hand, this application provides a drone-based intelligent identification and automatic throwing rescue device for people falling into water. The device includes: an acquisition module for acquiring water area images collected by the drone and performing real-time detection of people falling into the water in the water area images to obtain pixel coordinate data of the people falling into the water; an acquisition module for obtaining three-dimensional geographic coordinate data of the people falling into the water based on the location coordinate data of the drone and the pixel coordinate data of the people falling into the water; a prediction module for predicting the deployment location of the rescue device based on the three-dimensional geographic coordinate data of the people falling into the water and real-time environmental information to obtain the target deployment location; and a deployment module for controlling the drone to deploy the rescue device to the location of the people falling into the water based on the target deployment location.

[0013] On the other hand, this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any of the above embodiments.

[0014] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a method for intelligent identification and automatic throwing rescue of people falling into water based on an embodiment of this application is shown; Figure 2 A block diagram of a drone-based intelligent identification and automatic throwing rescue device for people falling into water, according to an embodiment of this application, is shown. Figure 3 A schematic diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0016] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0018] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0021] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0022] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0023] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0024] With the increasing demands for urban waterway management and public safety, improving safety patrols of open waterways such as rivers, lakes, and reservoirs has become crucial. Traditional waterway patrols rely heavily on shore-based cameras, manual monitoring, or patrol boats, which suffer from limited monitoring range, delayed discovery of people falling into the water, and slow rescue response.

[0025] In recent years, drones have been increasingly used in water patrol and emergency rescue scenarios due to their high mobility and wide coverage, enabling them to reach the scene quickly for monitoring and rescue assistance. However, existing drone-based water patrol and rescue technologies rely on manual identification and operation, resulting in problems such as untimely rescue, slow response speed, low deployment accuracy, and insufficient intelligence.

[0026] The water patrol technology based on drones mainly includes the following parts: (1) Unmanned aerial vehicle (UAV) flight platform: usually a multi-rotor UAV, with functions such as hovering and flight path; (2) Monitoring device: mainly equipped with high-definition camera or infrared camera, used to collect images of water area; (3) Deployment mechanism: Some drones are equipped with simple payload mounting and release devices, which can release life rings or rescue packs through manual remote control; (4) Manual control system: Operators control the UAV and deploy rescue devices via ground remote controller or ground station; (5) Data transmission module: Video data is transmitted to the ground control terminal via a wireless link, and manual observation is conducted to determine if any personnel have fallen into the water.

[0027] Operation and Working Method: Drones are generally patrolled by pre-planning flight paths or remote control. When a suspected person has fallen into the water is detected, the operator makes a manual judgment based on video footage received from the ground. After confirmation, the operator then controls the drone to fly to the target location and manually controls the release mechanism to deploy a lifebuoy. This method relies on manual identification and operation, lacks intelligent identification and automated linkage mechanisms, and has the following shortcomings: (1) Recognition relies on manual labor: It requires personnel to make visual recognition through video images, which is prone to delays or misjudgments; (2) Operation depends on manual operation: The deployment of lifebuoys requires manual control, making it difficult to guarantee the accuracy of deployment; (3) Limited response speed: When the drone's coverage area is large or the operator's attention is distracted, the best rescue opportunity may be missed; (4) Low level of intelligence: lack of automatic identification, automatic positioning and autonomous rescue capabilities.

[0028] In view of this, this application proposes a method for intelligent identification and automatic throwing of rescue devices for people falling into water based on drones. This method can automatically identify and locate the location of people falling into the water in real time, and the drone can autonomously complete the precise throwing of rescue devices, thereby shortening the rescue response time, reducing reliance on human operation, and improving the intelligence and success rate of rescue.

[0029] In the technical solution of this application embodiment, firstly, water area images collected by a drone are acquired, and real-time detection of people falling into the water is performed on the water area images to obtain pixel coordinate data of the people falling into the water. Then, based on the position coordinate data of the drone and the pixel coordinate data of the people falling into the water, the three-dimensional geographic coordinate data of the people falling into the water is obtained. Then, based on the three-dimensional geographic coordinate data of the people falling into the water and real-time environmental information, the deployment location of the rescue device is predicted to obtain the target deployment location. Finally, based on the target deployment location, the drone is controlled to deploy the rescue device to the location of the people falling into the water. This enables automatic identification and real-time positioning of the location of people falling into the water, and the drone autonomously completes the precise deployment of the rescue device, effectively shortening the rescue response time, reducing reliance on human operation, and improving the intelligence and success rate of rescue.

[0030] It should be noted that the method of this application can be applied to various complex aquatic environments such as rivers, lakes, and reservoirs, as well as island areas. It also has the characteristics of all-weather and all-area coverage. For example, drones can be used to patrol within island areas, monitor the sea surface in real time, identify and locate people who have fallen into the water, and autonomously complete the precise throwing of life-saving devices.

[0031] Figure 1 A flowchart of a method for intelligent identification and automatic throwing rescue of people falling into water based on a drone, according to an embodiment of this application, is shown.

[0032] like Figure 1 As shown, the UAV-based intelligent identification and automatic throwing rescue method 100 for people falling into water provided in this application includes steps S110 to S140.

[0033] Step S110: Acquire water area images collected by the drone, and perform real-time detection of people falling into the water in the water area images to obtain pixel coordinate data of the people falling into the water.

[0034] For example, images of various water bodies such as rivers, lakes, and reservoirs can be acquired using cameras or other image acquisition devices mounted on drones. Artificial intelligence algorithms, such as target detection algorithms, can then be used to detect and identify people who have fallen into the water in real time, thereby obtaining the pixel coordinate data of the people who have fallen into the water.

[0035] Step S120: Based on the location coordinate data of the UAV and the pixel coordinate data of the person who fell into the water, obtain the three-dimensional geographic coordinate data of the person who fell into the water.

[0036] For example, the drone's position coordinate data can be, for instance, the drone's GPS (Global Positioning System) position coordinates. Alternatively, based on the drone's GPS position coordinates and the pixel coordinates of the person who fell into the water, the three-dimensional position of the person can be calculated using a perspective projection model and converted into geographic coordinates.

[0037] Step S130: Based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information, the deployment location of the rescue device is predicted to obtain the target deployment location.

[0038] For example, real-time environmental information may include real-time wind speed, drone attitude, and water drift. For instance, at the initial altitude of the drone, the trajectory of the deployment device can be calculated based on the three-dimensional geographic coordinates of the person in the water and environmental information such as current wind speed and drone speed. This allows for the prediction of the location where the drone can release the rescue device, i.e., the deployment location of the rescue device. The deployment location can be dynamically adjusted based on real-time environmental information to plan the drone's flight path, ensuring the accuracy of the deployment location and thus obtaining the target deployment location.

[0039] Step S140: Based on the target deployment location, control the drone to deploy the rescue device to the location of the person who has fallen into the water.

[0040] For example, a drone can be controlled to fly to the height of the target deployment location, hover stably, and then deploy a rescue device so that the rescue device falls near the person who has fallen into the water.

[0041] In the technical solution of this application embodiment, firstly, water area images collected by a drone are acquired, and real-time detection of people falling into the water is performed on the water area images to obtain pixel coordinate data of the people falling into the water. Then, based on the position coordinate data of the drone and the pixel coordinate data of the people falling into the water, the three-dimensional geographic coordinate data of the people falling into the water is obtained. Then, based on the three-dimensional geographic coordinate data of the people falling into the water and real-time environmental information, the deployment location of the rescue device is predicted to obtain the target deployment location. Finally, based on the target deployment location, the drone is controlled to deploy the rescue device to the location of the people falling into the water. This enables automatic identification and real-time positioning of the location of people falling into the water, and the drone autonomously completes the precise deployment of the rescue device, effectively shortening the rescue response time, reducing reliance on human operation, and improving the intelligence and success rate of rescue.

[0042] In one example, the execution of the UAV-based intelligent identification and automatic throwing rescue method for people falling into the water is based on a UAV rescue system. This system can collect water area images and videos in real time, and perform personnel falling into the water detection and location identification on the water area images, thereby planning the flight path of the UAV to automatically throw rescue devices to the people who have fallen into the water. The following is a detailed description.

[0043] The unmanned aerial vehicle (UAV) rescue system mainly includes the following components: UAV flight platform, image acquisition module, processing and recognition module, positioning and navigation module, rescue device deployment module, and communication module.

[0044] (1) Unmanned aerial vehicle (UAV) flight platform: It has functions such as hovering, flight path, and obstacle avoidance.

[0045] (2) Image acquisition module: sensors such as high-definition camera and infrared thermal imager are used to acquire water images and videos in real time; installed at the front or bottom of the UAV to ensure panoramic view and overhead monitoring.

[0046] (3) Processing and recognition module: equipped with artificial intelligence algorithms for detecting people falling into the water and recognizing their location in the images; placed in the airborne computing cabin and connected to the image acquisition module and the positioning and navigation module.

[0047] (4) Positioning and navigation module: GPS or inertial navigation system, used for UAV positioning and flight path planning.

[0048] (5) Life-saving device deployment module: life ring, flotation device or other rescue device and its deployment mechanism; installed under the fuselage, the life-saving device is released through an electronic control mechanism.

[0049] (6) Communication module: used to transmit data back to the command center in real time to realize remote monitoring and collaborative control; connected to the processing and identification module, it transmits the identification results, positioning information and deployment status to the ground command center in real time.

[0050] The following details how to acquire water images based on a drone rescue system to detect and locate people who have fallen into the water in real time, thereby automatically deploying rescue devices.

[0051] For example, an image acquisition module can be used to acquire images of various types of water bodies such as rivers, lakes, and reservoirs, and to detect people falling into the water in real time. This allows for the acquisition of pixel coordinate data of the people in the water. For instance, firstly, the water images are scaled and pixel normalized to obtain a normalized image. Then, a multi-level feature extraction is performed on the normalized image using target detection and recognition algorithms to obtain water surface texture and edge information features, human contour features, and motion features. Multi-scale feature fusion is then performed on the water surface texture and edge information features, human contour features, and motion features to obtain a fused feature map. Finally, the fused feature map is subjected to prediction processing and filtering to obtain the pixel coordinate data of the people in the water.

[0052] Specifically, target detection and recognition algorithms can be used to perform real-time detection on water images acquired by the UAV (image acquisition module). The detection process is as follows: (1) Image preprocessing The input image (water area image) is scaled to a fixed size and pixel normalized to obtain a normalized image that meets the input requirements of the network model.

[0053] (2) Feature extraction and fusion For example, the YOLOv8 deep learning model (object detection and recognition algorithm) can be used to extract and fuse features from water images for real-time detection. Normalized images are input into the YOLOv8 deep learning model, and multi-level features are extracted through the CNN (Convolutional Neural Network) backbone network. Shallow features extract water surface texture and edge information, while deep features extract human contour and motion features. Then, a neck network (FPN+PAN, Feature Pyramid Network and Pyramid Attention Network) is used to achieve multi-scale feature fusion, improving the accuracy of water fall detection for different body shapes and postures, thus obtaining a fused feature map.

[0054] (3) Dense forecasting Prediction is performed on the fused feature map, and each grid cell outputs the center point, width and height, target confidence and category (e.g., "normal water area", "floating object", "person who fell into the water").

[0055] (4) Post-processing screening For example, non-maximum suppression (NMS) is used to filter out redundant boxes, retaining only the candidate results with the highest confidence, and outputting the final detection box and location information of the person who fell into the water, thus obtaining the pixel coordinate data of the person who fell into the water.

[0056] In the technical solution of this application embodiment, the water area image is first scaled and pixel normalized to obtain a normalized image. Then, multi-level feature extraction is performed on the normalized image through target detection and recognition algorithms to obtain water surface texture and edge information features, human body contour features and action features. Multi-scale feature fusion is then performed on the water surface texture and edge information features, human body contour features and action features to obtain a fused feature map. The fused feature map is then predicted and filtered to obtain the pixel coordinate data of the person who fell into the water. Thus, an improved target detection and recognition algorithm mounted on a UAV is used, combined with image processing and deep learning models, and a linkage mechanism with the UAV flight control system to achieve intelligent identification of people who have fallen into the water in complex water environments. This improves the accuracy of identification of people who have fallen into the water in complex water environments and ensures the reliability of monitoring data.

[0057] Next, using the drone's GPS location, drone altitude, camera parameters, and the pixel coordinates of the target that fell into the water, the three-dimensional position of the target (i.e., the person who fell into the water) is calculated using a perspective projection model, and then converted into geographic coordinates. The specific process is explained below.

[0058] For example, based on the location coordinate data of the UAV and the pixel coordinate data of the person who fell into the water, the three-dimensional geographic coordinate data of the person who fell into the water is obtained. For example, firstly, according to the perspective projection model, the pixel coordinate data of the person who fell into the water is converted into normalized planar coordinates of the camera coordinate system through the camera intrinsic parameter matrix to obtain the target direction vector of the camera coordinate system; then, based on the rotation matrix, the target direction vector of the camera coordinate system is rotated and transformed into the geographic coordinate system to obtain the target direction vector of the geographic coordinate system; then, based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system, the latitude and longitude position of the person who fell into the water is calculated through the coordinate transformation chain to obtain the three-dimensional geographic coordinate data of the person who fell into the water.

[0059] Specifically, the process of converting pixel coordinate data of a person who has fallen into the water into three-dimensional geographic coordinate data for location tracking includes the following steps: (1) Pixel coordinates → camera coordinate system: The target pixel coordinates (pixel coordinate data of the person who fell into the water) are converted into normalized planar coordinates using the camera intrinsic parameter matrix (focal length, principal point).

[0060] If the drone's GPS location (the drone's position coordinate data) is ( X d , Y d , Z d The drone's altitude is H, and the camera intrinsic parameter matrix is ​​shown in formula (1): (1) in, fx , fyFor the camera's focal length, ( cx , cy () are the coordinates of the main point.

[0061] The pixel coordinate data of the person who fell into the water is converted into normalized planar coordinates through the camera intrinsic parameter matrix as shown in formula (2): (2) in,( u , v , 1) represents the pixel coordinate data of the person who fell into the water, ( xc , yc ,1) represents the direction vector of the person who fell into the water in the camera coordinate system, that is, the target direction vector.

[0062] For example, the rotation matrix includes a first rotation matrix and a second rotation matrix; based on the rotation matrix, the target direction vector of the camera coordinate system is rotated and transformed into the geographic coordinate system to obtain the target direction vector of the geographic coordinate system. For example, firstly, the first rotation matrix is ​​determined according to the fixed installation angle of the UAV, and the target direction vector of the camera coordinate system is transformed into the target direction vector of the body coordinate system through the first rotation matrix; then, the second rotation matrix is ​​determined according to the UAV's inertial measurement device, and the target direction vector of the body coordinate system is transformed into the target direction vector of the geographic coordinate system through the second rotation matrix, wherein the target direction vector of the geographic coordinate system represents the relative position of the UAV and the person who fell into the water.

[0063] Specifically, the process of converting the pixel coordinate data of a person who has fallen into the water into their three-dimensional geographic coordinate data for location tracking also includes the following steps: (2) Camera coordinate system → Body coordinate system: Rotation transformation is performed in combination with the UAV attitude angles (pitch, roll, yaw).

[0064] The camera and the drone's body coordinate system have a fixed installation angle, for example... Let X represent the rotation angles (pitch angle), Y represent the rotation angles (roll angle), and Z represent the rotation angles (yaw angle) around the UAV's X-axis, respectively. These can be represented by the first rotation matrix. The target direction vector in the camera coordinate system is transformed to the target direction vector in the body coordinate system using the first rotation matrix, as shown in formula (3). (3) in, Let be the target direction vector in the body coordinate system. R c2b This is the first rotation matrix (i.e., the installation error of the UAV). is the target direction vector in the camera coordinate system.

[0065] Then, based on the second rotation matrix, the target direction vector in the body coordinate system is transformed to the target direction vector in the geographic coordinate system. The second rotation matrix represents the external rotation error of the UAV, which can be obtained from the data of the UAV's inertial device sensors (e.g., inertial measurement units). The target direction vector in the geographic coordinate system is represented by formula (4): (4) in, This is the target direction vector in the geographic coordinate system. R b2n The second rotation matrix determines the relative direction (relative position) between the drone and the person who fell into the water based on the target direction vector in the geographic coordinate system.

[0066] For example, based on the UAV's position coordinate data and the target direction vector of the geographic coordinate system, the latitude and longitude of the person who fell into the water are calculated through a coordinate transformation chain to obtain the person's three-dimensional geographic coordinate data. For instance, based on the UAV's position coordinate data and the target direction vector of the geographic coordinate system, the three-dimensional geographic distance between the UAV and the person who fell into the water is calculated through a ray equation to obtain the person's three-dimensional geographic coordinate data. The person's three-dimensional geographic coordinate data includes altitude data, which is related to the water surface height.

[0067] Specifically, the process of converting the pixel coordinate data of a person who has fallen into the water into their three-dimensional geographic coordinate data for location tracking also includes the following steps: (3) Body coordinate system → geographic coordinate system: Combine the UAV's GPS position and altitude, and calculate the latitude and longitude of the person who fell into the water through a coordinate transformation chain (such as NED coordinate system, local geographic coordinate system).

[0068] If the water level is Z = Zw The drone's current altitude is Z d The location of the person who fell into the water can be obtained by calculating the intersection of the ray and the water surface plane using the ray equation (i.e., calculating the three-dimensional geographical distance between the UAV and the person who fell into the water based on the target direction vector), as shown in formula (5): (5) in, The starting point of the ray, i.e., the position coordinates of the UAV, λ As a scale factor, Let be the target direction vector in the geographic coordinate system. Z = Zw Solving for:

[0069] The final three-dimensional geographic coordinates of the person who fell into the water are as follows:

[0070] Among them, the height data of the people who fell into the water Z t water level Zw In related fields, the three-dimensional geographic coordinates of the person who fell into the water can be directly used for subsequent automatic drop calculations by drones.

[0071] In the technical solution of this application embodiment, firstly, based on the perspective projection model, the pixel coordinate data of the person who fell into the water is converted into normalized planar coordinates of the camera coordinate system through the camera intrinsic parameter matrix to obtain the target direction vector of the camera coordinate system. Then, the target direction vector of the camera coordinate system is converted to the target direction vector of the body coordinate system through the first rotation matrix, and the target direction vector of the body coordinate system is converted to the target direction vector of the geographic coordinate system through the second rotation matrix. Then, based on the position coordinate data of the UAV and the target direction vector of the geographic coordinate system, the three-dimensional geographic distance between the UAV and the person who fell into the water is calculated through the ray equation to obtain the three-dimensional geographic coordinate data of the person who fell into the water. Thus, the coordinate transformation formula is combined with the UAV flight control system, and the latitude and longitude of the person who fell into the water are accurately calculated using perspective projection and coordinate transformation chain to accurately locate the person who fell into the water.

[0072] For example, based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information, the deployment location of the rescue device is predicted to obtain the target deployment location. For example, firstly, based on the three-dimensional geographic coordinate data of the person who fell into the water, a kinematic model of the rescue device is established according to the initial altitude data and initial speed data of the UAV to calculate the deployment trajectory of the rescue device and obtain the initial deployment location; then, based on the three-dimensional geographic coordinate data of the person who fell into the water, the initial deployment location is dynamically corrected according to the real-time environmental information to obtain the target deployment location; wherein, the real-time environmental information includes real-time wind speed information, UAV attitude information and water flow drift information.

[0073] Specifically, the target deployment location is calculated based on the three-dimensional geographic coordinate data of the person who has fallen into the water, in order to adaptively deploy the rescue device controlled by the drone, including the following steps: (1) Calculation of delivery trajectory A fixed local inertial coordinate system on the ground (e.g., ENU: x-axis pointing east, y-axis pointing north, z-axis pointing upward) is used to perform data prediction calculations related to the rescue deployment trajectory.

[0074] In this coordinate system, the position vector is: The coordinates of the location at the moment of drone release are: ,in, z 0 represents the initial altitude data of the drone. At this point, the initial velocity of the release target (deployment device) relative to the ground (i.e., the initial velocity data of the drone) is: If the drone hovers and releases without applying additional velocity, the initial velocity is the drone's ground speed (0 if air resistance and wind are ignored). If the release mechanism (i.e., the life-saving device release module) has an initial velocity, the initial release velocity component is added. Based on the above data, a kinematic model of the life-saving device is established to calculate the parabolic trajectory of the release, thereby predicting and calculating the release point (the release location of the life-saving device).

[0075] (2) Location calculation If air resistance and wind are ignored (or considered to be included in the initial velocity), the trajectory of the rescue device can be calculated using classical projectile motion as shown in formulas (6) and (7): (6) (7) in, The velocity of the rescue device from its deployment position. The displacement of the rescue device from its deployment position, where g is the acceleration due to gravity, taken as 9.81, and the vector g = [0, 0, -g]. ⊤ .

[0076] Based on the displacement and velocity of the motion, the vertical landing time of the rescue device after deployment (contact surface z=0) is obtained as follows: .

[0077] Based on the initial velocity, the horizontal displacement of the life-saving device at the moment of release (dumping) (e.g., under no-resistance conditions) is shown in Equation (8): (8) Based on the horizontal displacement of the life-saving device at the moment of release (deployment) and the initial altitude data of the UAV, air resistance and wind field corrections are performed (according to the above calculation results, this is the ideal case of no resistance), thereby calculating the UAV release point (the deployment location of the life-saving device), as shown in formula (9): (9) in, Location of the person who fell into the water (three-dimensional geographic coordinate data of the person who fell into the water). The displacement compensation is based on the real-time wind speed and resistance at the time of deployment. In the ideal case of no resistance, the value is 0, that is, the position of the person falling into the water is equal to the position of the life-saving device deployment.

[0078] For example, the initial deployment position is predicted based on the initial altitude data and initial speed data of the drone (in the ideal case of no resistance or after initial displacement compensation). After the rescue device is deployed based on the initial deployment position, displacement compensation is predicted again based on real-time environmental information, such as real-time wind speed information, drone attitude information and water drift information, to prevent the initially deployed rescue device from deviating from the position of the person who fell into the water, thereby dynamically correcting the deployment position to obtain the target deployment position.

[0079] In the technical solution of this application embodiment, firstly, based on the three-dimensional geographic coordinate data of the person who fell into the water, a kinematic model of the rescue device is established according to the initial altitude data and initial speed data of the UAV to calculate the deployment trajectory of the rescue device and obtain the initial deployment position. Then, based on the three-dimensional geographic coordinate data of the person who fell into the water, the initial deployment position is dynamically corrected according to real-time environmental information to obtain the target deployment position. Through the UAV's autonomous flight path planning and real-time event processing, the deployment point is accurately calculated under the condition of air resistance and wind field correction, thereby automatically deploying the rescue device, significantly improving rescue efficiency and increasing rescue response speed.

[0080] For example, based on the three-dimensional geographic coordinate data of the person who fell into the water, the initial deployment position is dynamically corrected according to real-time environmental information to obtain the target deployment position. For example, firstly, if the initial deployment position meets the first preset condition, compensation displacement prediction is performed according to real-time environmental information to dynamically correct the initial deployment position; then, the target deployment position is obtained according to the difference between the three-dimensional geographic coordinate data of the person who fell into the water and the compensation displacement.

[0081] Specifically, after the initial deployment location of the rescue device is predicted through the above implementation method, the drone flies above the deployment point, hovers stably, and deploys the rescue device. If the sensor detects that the initial deployment location deviation exceeds a threshold (meeting the first preset condition), it indicates that the rescue device cannot reach the location of the person in the water after landing. Then, displacement compensation is performed again based on real-time wind speed, drone attitude, and water drift, thereby dynamically correcting the predicted target deployment location. See the above-mentioned content on reverse-calculating the deployment location for details. Based on this, the flight control system automatically adjusts the drone's position and deploys the rescue device again to ensure that the rescue device lands near the person in the water.

[0082] In the technical solution of this application embodiment, firstly, when the initial deployment position meets the first preset condition, compensation displacement prediction is performed based on real-time environmental information to dynamically correct the initial deployment position. Then, based on the difference between the three-dimensional geographic coordinate data of the person in the water and the compensation displacement, the target deployment position is obtained. Thus, adaptive deployment is achieved based on the linkage strategy of UAV and sensor, enabling rapid detection of people in the water and timely deployment of life-saving devices, significantly improving rescue efficiency. At the same time, the use of UAV autonomous flight, intelligent identification, and automatic deployment reduces human intervention, lowers manpower and time costs, improves the identification accuracy and rescue efficiency of UAV in water patrol and rescue, and enhances the intelligence level of water safety management and emergency rescue.

[0083] For example, after controlling the drone to deploy the rescue device to the location of the person who has fallen into the water based on the target deployment location, the rescue results can also be fed back, such as confirming whether the rescue device has reached the location of the person who has fallen into the water, and feeding back the rescue results to the ground command center in real time.

[0084] Specifically, drones can be controlled to use cameras to confirm whether the rescue device has landed near the person in the water, and the rescue results can be transmitted back to the ground command center in real time via onboard sensors. If the rescue is unsuccessful, the event is recorded and marked as requiring human intervention, thus realizing an adaptive deployment process of initial deployment—adaptive adjustment—feedback confirmation.

[0085] In the technical solution of this application embodiment, based on the target deployment location, after controlling the drone to deploy the rescue device to the location of the person who fell into the water, it confirms whether the rescue device has reached the location of the person who fell into the water, and feeds back the rescue result to the ground command center in real time, thereby realizing the closed-loop linkage between drone rescue and ground command system, forming a closed-loop rescue mechanism, and realizing remote monitoring and collaborative control.

[0086] It should be noted that the above-described method for intelligent identification and automatic throwing rescue of people falling into water based on drones is a specific implementation of this application. The algorithms and specific data calculation methods involved in this method can be replaced without affecting the rescue effect, as follows: (a) Perception and Recognition Substitution 1. Sensor combination replacement: Visible light cameras can be replaced with infrared thermal imaging, binocular / multi-view vision, radar (millimeter wave / liDAR) or any combination thereof to improve robustness in environments such as night, strong reflection, rain and fog; 2. Algorithm replacement: Object detection can be replaced by YOLO-based models with Transformer (self-attention network model) object detection (such as DETR-based models, end-to-end object detection architecture), two-stage detection (Faster R-CNN), or "detection + tracking" multi-object tracking (MOT) framework; pose / behavior discrimination can be achieved using spatiotemporal networks, optical flow methods, or few-shot anomaly detection models.

[0087] (ii) Positioning and ranging alternatives 1. Geometric method alternatives: In addition to monocular perspective projection, binocular / multi-view triangulation, structured light depth, lidar ground fitting + projection, or vision-inertial tight coupling (SLAM) can be used to obtain the water surface plane and the target landing point. 2. Geolocation Alternatives: In addition to GPS / IMU, RTK, UWB ranging (a real-time positioning technology), shore-based base station differential, or visual positioning (VIO) can be integrated to improve coordinate accuracy and resistance to obstruction.

[0088] (III) Substitution of delivery agencies and trajectory control 1. Replacement of deployment devices: Life rings can be replaced with automatic inflatable flotation devices, rope bags, self-floating beacons, and visual pyrotechnic / fluorescent buoys; 2. Trajectory control alternatives: In addition to pure throwing, miniature deceleration parachutes / folding wing deceleration parachutes, deployable wing surfaces (gliders), or micro-thrust pulse corrections can be used to reduce wind drift and correct the landing point; 3. Alternative release strategies: In addition to fixed-point hovering deployment, dynamic deployment with "forward / side-throwing", multi-point circular deployment in multiple stages, or closed-loop strategies of "deployment-observation-replenishment" can be used.

[0089] (iv) Adaptation and Decision Substitution 1. Adaptive Strategy Alternative: In addition to focal length, pitch, and flight altitude, lens adaptation can incorporate cascaded strategies such as yaw shift, side-view imaging, or switching to the infrared channel. 2. Wind field estimation alternative: Wind speed can be estimated by "aircraft airspeed meter + ground speed difference", back-calculated based on landing point observation, or estimated by shore-based meteorological station / coordinated UAVs; 3. Optimized solution alternatives: In addition to numerical integration and binary search, the release point back calculation can be solved quickly using table lookup interpolation, particle swarm / genetic algorithms, and MPC (model predictive control).

[0090] (v) Platform and Collaborative Substitution 1. Platform replacement: Multi-rotor aircraft can be replaced by vertical take-off and landing fixed-wing aircraft, tethered drones (long-term stay), or amphibious unmanned surface vessels (USVs) + drones in coordination, with USVs approaching for rescue and drones responsible for discovery and guidance; 2. Collaborative Intelligence: Multi-aircraft formation patrols and relay deployments in designated areas, with one aircraft identifying and locating the target while another carries more life-saving equipment to carry out the deployment.

[0091] (vi) Communication and Command Alternatives 1. Link alternatives: In addition to 4G / 5G, Mesh (wireless mesh network) self-organizing networks, satellite links, or shore-based relays can be used; 2. Command Alternative: A layered architecture for joint decision-making between local edge computing (airborne / shore-based) and the cloud, supporting offline / weak network emergency modes.

[0092] (vii) Human-computer interaction and safe alternatives 1. Manual takeover alternative: In low confidence or strong wind conditions, remote one-click takeover / confirmation and re-deployment are allowed; 2. Compliance Modes: Geofencing, no-fly zone self-inspection, and automatic return-to-home / forced landing strategies can be implemented as different methods; 3. Evidence chain replacement: Automatic capture / video recording before and after deployment, time and location watermarks and hash signatures to ensure evidence collection and post-event review.

[0093] (viii) Application process replacement 1. Cruise mode alternatives: fixed route, hotspot patrol, autonomous search (coverage-path planning) or event-driven (rapid deployment after receiving public / shore-based alarms); 2. Replacement of confirmation mechanism: The single-model confidence threshold can be replaced by multimodal consistency (visual + thermal imaging + acoustic distress signal) confirmation to reduce false alarms.

[0094] Figure 2 A block diagram of a drone-based intelligent identification and automatic throwing rescue device for people falling into water, according to an embodiment of this application, is shown.

[0095] like Figure 2 As shown, this application provides a drone-based intelligent identification and automatic throwing rescue device 200 for people falling into water. The device 200 includes: The acquisition module 210 is used to acquire water area images collected by the UAV and perform real-time detection of people falling into the water from the water area images to obtain the pixel coordinate data of the people falling into the water.

[0096] Module 220 is used to obtain the three-dimensional geographic coordinate data of the person who fell into the water based on the location coordinate data of the UAV and the pixel coordinate data of the person who fell into the water.

[0097] The prediction module 230 is used to predict the deployment location of the rescue device based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information, so as to obtain the target deployment location.

[0098] The delivery module 240 is used to control the drone to deliver the life-saving device to the location of the person who has fallen into the water, based on the target delivery location.

[0099] For example, the prediction module 230 is further configured to: establish a kinematic model of the rescue device based on the three-dimensional geographic coordinate data of the person who fell into the water, and calculate the deployment trajectory of the rescue device to obtain the initial deployment position based on the initial altitude data and initial speed data of the UAV; and dynamically correct the initial deployment position based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information to obtain the target deployment position; wherein, the real-time environmental information includes real-time wind speed information, UAV attitude information and water drift information. For example, based on the three-dimensional geographic coordinate data of the person who fell into the water, the initial deployment position is dynamically corrected according to real-time environmental information to obtain the target deployment position, including: if the initial deployment position meets the first preset condition, a compensation displacement prediction is performed according to real-time environmental information to dynamically correct the initial deployment position; the target deployment position is obtained according to the difference between the three-dimensional geographic coordinate data of the person who fell into the water and the compensation displacement.

[0100] For example, module 220 is further configured to: convert the pixel coordinate data of the person who fell into the water into normalized planar coordinates of the camera coordinate system through the camera intrinsic parameter matrix according to the perspective projection model, and obtain the target direction vector of the camera coordinate system; perform a rotation transformation on the target direction vector of the camera coordinate system to the geographic coordinate system based on the rotation matrix, and obtain the target direction vector of the geographic coordinate system; calculate the latitude and longitude position of the person who fell into the water through the coordinate transformation chain based on the position coordinate data of the UAV and the target direction vector of the geographic coordinate system, and obtain the three-dimensional geographic coordinate data of the person who fell into the water.

[0101] For example, the rotation matrix includes a first rotation matrix and a second rotation matrix; based on the rotation matrix, the target direction vector of the camera coordinate system is rotated and transformed into the geographic coordinate system to obtain the target direction vector of the geographic coordinate system, including: determining the first rotation matrix according to the fixed installation angle of the UAV, and transforming the target direction vector of the camera coordinate system to the target direction vector of the body coordinate system through the first rotation matrix; determining the second rotation matrix according to the UAV's inertial measurement device, and transforming the target direction vector of the body coordinate system to the target direction vector of the geographic coordinate system through the second rotation matrix, wherein the target direction vector of the geographic coordinate system represents the relative position of the UAV and the person who fell into the water.

[0102] For example, based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system, the latitude and longitude position of the person who fell into the water is calculated through a coordinate transformation chain to obtain the three-dimensional geographic coordinate data of the person who fell into the water. This includes: calculating the three-dimensional geographic distance between the UAV and the person who fell into the water through a ray equation based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system to obtain the three-dimensional geographic coordinate data of the person who fell into the water; wherein, the three-dimensional geographic coordinate data of the person who fell into the water includes altitude data, which is related to the water surface height.

[0103] For example, real-time detection of people falling into the water is performed on water area images to obtain the pixel coordinate data of the people falling into the water. This includes: scaling and normalizing the water area image to obtain a normalized image; extracting multi-level features from the normalized image using target detection and recognition algorithms to obtain water surface texture and edge information features, human body contour features and motion features; fusing the water surface texture and edge information features, human body contour features and motion features at multiple scales to obtain a fused feature map; and performing prediction processing and filtering on the fused feature map to obtain the pixel coordinate data of the people falling into the water.

[0104] For example, after controlling the drone to deploy the rescue device to the location of the person who has fallen into the water based on the target deployment location, the device 200 also includes a feedback module for: confirming whether the rescue device has reached the location of the person who has fallen into the water, and feeding back the rescue result to the ground command center in real time.

[0105] Figure 3 A schematic diagram of an electronic device according to an embodiment of this application is shown.

[0106] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0107] like Figure 3 As shown, for ease of understanding, embodiments of this application illustrate a specific electronic device 300.

[0108] Electronic device 300 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 300 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] like Figure 3As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0110] Multiple components in electronic device 300 are connected to I / O interface 305. These components include: input unit 306, such as a keyboard or mouse; output unit 307, such as various types of displays or speakers; storage unit 308, such as a disk or optical disk; and communication unit 309, such as a network interface card (NIC), modem, or wireless transceiver. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0111] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods described above. For example, in some embodiments, any one or more of the methods described above can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of any one or more of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform any one or more of the methods described above by any other suitable means (e.g., by means of firmware).

[0112] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0113] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0114] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for intelligent identification and automatic rescue of people falling into water based on unmanned aerial vehicles (UAVs), characterized in that, The method includes: The system acquires images of the water area collected by the drone and performs real-time detection of people who have fallen into the water on the images to obtain the pixel coordinate data of the people who have fallen into the water. Based on the location coordinate data of the UAV and the pixel coordinate data of the person who fell into the water, the three-dimensional geographic coordinate data of the person who fell into the water is obtained. Based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information, the deployment location of the rescue device is predicted to obtain the target deployment location; Based on the target deployment location, the drone is controlled to deploy the rescue device to the location of the person who has fallen into the water.

2. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The prediction of the deployment location of the rescue device based on the three-dimensional geographic coordinate data and real-time environmental information of the person who fell into the water, to obtain the target deployment location, includes: Based on the three-dimensional geographic coordinate data of the person who fell into the water, a kinematic model of the rescue device is established according to the initial altitude data and initial speed data of the UAV to calculate the deployment trajectory of the rescue device and obtain the initial deployment position; Based on the three-dimensional geographic coordinate data of the person who fell into the water, the initial placement location is dynamically corrected according to the real-time environmental information to obtain the target placement location; The real-time environmental information includes real-time wind speed information, UAV attitude information, and water flow drift information.

3. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to claim 2, characterized in that, The method of dynamically correcting the initial deployment location based on the three-dimensional geographic coordinate data of the person who fell into the water, according to the real-time environmental information, to obtain the target deployment location includes: If the initial deployment position meets the first preset condition, compensation displacement prediction is performed based on the real-time environmental information to dynamically correct the initial deployment position. The target deployment location is obtained based on the difference between the three-dimensional geographic coordinates of the person who fell into the water and the compensation displacement.

4. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to any one of claims 1-3, characterized in that, The three-dimensional geographic coordinate data of the person who fell into the water is obtained based on the location coordinate data of the UAV and the pixel coordinate data of the person, including: Based on the perspective projection model, the pixel coordinate data of the person who fell into the water is converted into normalized planar coordinates of the camera coordinate system through the camera intrinsic parameter matrix, and the target direction vector of the camera coordinate system is obtained. Based on the rotation matrix, the target direction vector of the camera coordinate system is rotated and transformed into the geographic coordinate system to obtain the target direction vector in the geographic coordinate system. Based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system, the latitude and longitude of the person who fell into the water are calculated through a coordinate transformation chain, and the three-dimensional geographic coordinate data of the person who fell into the water are obtained.

5. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, The rotation matrix includes a first rotation matrix and a second rotation matrix; The step of performing a rotation transformation on the target direction vector in the camera coordinate system based on a rotation matrix to obtain the target direction vector in the geographic coordinate system includes: The first rotation matrix is ​​determined based on the fixed installation angle of the UAV, and the target direction vector of the camera coordinate system is transformed to the target direction vector of the body coordinate system through the first rotation matrix. The second rotation matrix is ​​determined based on the UAV's inertial measurement device, and the target direction vector of the body coordinate system is transformed to the target direction vector of the geographic coordinate system through the second rotation matrix, wherein the target direction vector of the geographic coordinate system represents the relative position of the UAV and the person who fell into the water.

6. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to claim 5, characterized in that, The method of calculating the latitude and longitude of the person who fell into the water based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system through a coordinate transformation chain to obtain the three-dimensional geographic coordinate data of the person who fell into the water includes: Based on the location coordinate data of the UAV and the target direction vector of the geographic coordinate system, the three-dimensional geographic distance between the UAV and the person who fell into the water is calculated by the ray equation, and the three-dimensional geographic coordinate data of the person who fell into the water is obtained. The three-dimensional geographic coordinate data of the person who fell into the water includes height data, which is correlated with the water level.

7. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The real-time detection of people falling into the water from the water area image, to obtain the pixel coordinate data of the people falling into the water, includes: The water area image is scaled and pixel normalized to obtain a normalized image; The normalized image is subjected to multi-level feature extraction by target detection and recognition algorithms to obtain water surface texture and edge information features, human body contour features and motion features, and the water surface texture and edge information features, human body contour features and motion features are fused at multiple scales to obtain a fused feature map. The fused feature map is subjected to prediction processing and filtering to obtain the pixel coordinate data of the person who fell into the water.

8. The method for intelligent identification and automatic throwing rescue of people falling into water based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, After controlling the drone to deploy the rescue device to the location of the person who has fallen into the water based on the target deployment location, the method further includes: Confirm whether the rescue device has reached the location of the person who has fallen into the water, and report the rescue results to the ground command center in real time.

9. A drone-based intelligent identification and automatic throwing rescue device for people falling into water, characterized in that, The device includes: The acquisition module is used to acquire water area images collected by the UAV, and to perform real-time detection of people falling into the water in the water area images to obtain the pixel coordinate data of the people falling into the water. The module is used to obtain the three-dimensional geographic coordinate data of the person who fell into the water based on the location coordinate data of the UAV and the pixel coordinate data of the person who fell into the water. The prediction module is used to predict the deployment location of the rescue device based on the three-dimensional geographic coordinate data of the person who fell into the water and real-time environmental information, so as to obtain the target deployment location. The delivery module is used to control the drone to deliver the rescue device to the location of the person who has fallen into the water, based on the target delivery location.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-8.

Images