Personnel search and rescue identification method and system based on unmanned aerial vehicle cluster and airborne imaging equipment

Abstract

The invention discloses a personnel search and rescue identification method and system based on an unmanned aerial vehicle cluster and airborne imaging equipment. The method comprises the following steps that each unmanned aerial vehicle in the unmanned aerial vehicle cluster receives a corresponding inspection flight parameter; each unmanned aerial vehicle in the unmanned aerial vehicle cluster executes an inspection task according to the corresponding inspection flight parameter, and obtains a ground image in the inspection area in real time in the execution process of the inspection task; after the existence of the trapped person is determined, the plurality of unmanned aerial vehicles execute a cooperative search and rescue task; and the unmanned aerial vehicle returns to the material storage point, returns to the area where the trapped person is located according to the material delivery path, and delivers materials to the area where the trapped person is located. According to the invention, trapped persons can be detected and accurately identified in real time, the trapped persons can be tracked in real time, and multiple links such as emergency material delivery and guidance of ground search and rescue persons to a search and rescue place can be further completed through cooperative work of the unmanned aerial vehicle cluster, so that the unmanned aerial vehicle cluster can execute multiple tasks in a complex environment of a depopulated area.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a method and system for personnel search and rescue recognition based on an unmanned aerial vehicle (UAV) cluster and airborne imaging equipment. Background Art

[0002] Currently, due to factors such as harsh natural environments and complex terrains in uninhabited areas, personnel search and rescue operations face huge challenges. Traditional manual search and rescue methods are time-consuming, inefficient, and pose safety hazards. With the rapid development of UAV technology, its high mobility and versatility have broad application prospects in the field of emergency rescue. However, when using UAVs to perform personnel search and rescue in uninhabited areas in the prior art, the following problems exist:

[0003] The complex ground environment causes traditional vision algorithms to be prone to misjudgment and missed judgment in target detection, resulting in low target recognition accuracy;

[0004] The coverage area of a single UAV is limited, making it difficult to quickly conduct a comprehensive inspection of a large uninhabited area, resulting in insufficient search and rescue efficiency;

[0005] 3. For personnel in need of emergency rescue, the technology for accurately delivering emergency supplies is not yet mature, making it difficult to accurately and quickly deliver emergency supplies. Summary of the Invention

[0006] Aiming at the deficiencies of the prior art, the present invention provides a method and system for personnel search and rescue recognition based on a UAV cluster and airborne imaging equipment, which can detect and accurately identify trapped personnel in real time through the airborne imaging equipment, and perform real-time tracking on the trapped personnel. Further, through the collaborative work of the UAV cluster, multiple links such as emergency supply delivery and guiding ground search and rescue personnel to the search and rescue location are completed, enabling it to perform multiple tasks in the complex environment of uninhabited areas.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] On the one hand, a method for personnel search and rescue recognition based on a UAV cluster and airborne imaging equipment is provided, which includes the following steps:

[0009] Each UAV in the UAV cluster receives its corresponding inspection flight parameters;

[0010] Each UAV in the UAV cluster executes an inspection task according to its corresponding inspection flight parameters, and during the execution of the inspection task, each UAV obtains ground images within the inspection area in real time through the imaging equipment carried on the UAV, and detects trapped personnel based on the ground images;

[0011] After determining that there are trapped personnel, several UAVs execute collaborative search and rescue tasks;

[0012] Moreover, after the UAV returns to the material storage point, loads materials on the UAV, returns to the area where the trapped people are located according to the material delivery path, and delivers materials to the area where the trapped people are located.

[0013] Preferably, the inspection flight parameters include the UAV inspection altitude and the inspection route, and the inspection altitude is obtained by the following formula:

[0014]

[0015] Where H i is the cruising altitude of the i-th UAV; H min is the minimum safety altitude of the i-th UAV, and H i ≥H min ; A coverage is the inspection area of the i-th UAV; N is the total number of UAVs.

[0016] Preferably, the collaborative search and rescue tasks include:

[0017] Obtaining images of the trapped people from different angles through imaging devices carried on at least one UAV and sending them to the control terminal.

[0018] Preferably, the collaborative search and rescue tasks include: providing lighting for the area where the trapped people are located through lighting devices carried on at least one UAV.

[0019] Preferably, the collaborative search and rescue tasks include: performing voice interaction with the trapped people through voice devices carried on at least one UAV.

[0020] Preferably, the collaborative search and rescue tasks include: obtaining the position of the trapped people in real time through a target tracking unit carried on at least one UAV.

[0021] Preferably, the position of the trapped people is obtained according to the following formula:

[0022] T j (t) = {(X j (t), Y j (t))|t0 ≤ t ≤ t f}

[0023]

[0024] Where T j (t) is the movement trajectory of the trapped people; (X j (t), Y j (t)) is the position of the trapped people at the current moment t; t0 is the moment when the trapped people are first discovered; t fis the end moment of search and rescue; I(x, y) is the ground image, and (x, y) is the pixel value of any pixel point in the ground image; P j is the pixel set of the area where the j-th trapped person is located.

[0025] Preferably, the material delivery path is determined through the following steps:

[0026] Based on the following formula, the position of the trapped person is dynamically updated through the position estimation unit carried on at least one drone to obtain the predicted value of the position of the trapped person at the current moment:

[0027]

[0028] where is the predicted value of the position of the trapped person at time k; is the predicted value of the position of the trapped person at time k - 1; K k is the Kalman gain; z k is the actual measurement value; H is the observation matrix;

[0029] Determine the optimal flight path of the drone according to the following formula:

[0030]

[0031] where P fly (t) represents the optimal flight path of the drone at time t; P(t) is the current flight path trajectory of the drone at time t; argmin represents the variable value when the objective function takes the minimum value;

[0032] And determine the optimal material delivery path of the drone according to the following formula:

[0033]

[0034] where R resupply represents the optimal material delivery path of the drone, and t represents the time variable.

[0035] Preferably, the inspection flight parameters of at least 2 drones are completely different.

[0036] On the other hand, a personnel search and rescue recognition system is also provided, which includes:

[0037] Several drones, which form a drone cluster;

[0038] Several imaging devices, which are carried on the drones, and each drone is equipped with an imaging device;

[0039] A target detection unit, which is carried on a drone and is used to detect trapped persons based on the ground images in the inspection area obtained in real time by the imaging device to determine whether there are trapped persons;

[0040] A lighting device, which is carried on at least one drone and is used to provide lighting for the area where the trapped persons are located;

[0041] A voice device, which is carried on at least one drone and is used to conduct voice interaction with the trapped persons;

[0042] A target tracking unit, which is carried on at least one drone and is used to obtain the position of the trapped persons in real time;

[0043] A path optimization unit, which is used to determine the material delivery path of the drone according to the position of the trapped persons, so that the drone returns to the area where the trapped persons are located according to the material delivery path and delivers materials to the area where the trapped persons are located;

[0044] And a control terminal, which is connected to one or several of the imaging device, the target detection unit, the lighting device, the voice device, the target tracking unit and the path optimization unit.

[0045] The present invention can detect and accurately identify trapped persons in real time through the airborne imaging device, and track the trapped persons in real time. Further, through the collaborative work of the drone cluster, multiple links such as emergency material delivery and guiding ground search and rescue personnel to the search and rescue location are completed. Thus, the advantages of the drone cluster relative to a single drone are fully utilized, enabling it to perform multiple tasks (such as personnel identification, positioning, voice interaction, lighting, and material delivery, etc.) in the complex environment of the uninhabited area, so as to efficiently and accurately complete the personnel search and rescue task, and has a wide range of application prospects. Brief Description of the Drawings

[0046] Figure 1 It is a step flow chart of the personnel search and rescue identification method based on the drone cluster and the airborne imaging device in the present invention;

[0047] Figure 2 It is the identification result of the trapped persons in the ground image in the present invention;

[0048] Figure 3 It is a schematic structural diagram of the personnel search and rescue identification system in the present invention. Detailed Embodiment

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] Embodiment 1:

[0051] As Figure 1-2 shown, this embodiment provides a method for personnel search and rescue recognition based on an unmanned aerial vehicle (UAV) cluster and an airborne imaging device, which includes the following steps:

[0052] S1. Each UAV in the UAV cluster receives the corresponding inspection flight parameters. In this embodiment, the UAV cluster includes N UAVs (N is a positive integer greater than or equal to 2), and the inspection flight parameters include the UAV inspection altitude and the inspection route;

[0053] Specifically, the inspection altitude is obtained through formula (1):

[0054]

[0055] where, H i is the cruising altitude of the i-th UAV; H min is the minimum safety altitude of the i-th UAV, which can be preset, and H i ≥H min , for example, in this embodiment, the value range of H min is [20m, 30m]; A coverage is the inspection area of the i-th UAV, for example, 10 square kilometers, and the inspection areas of each UAV can be different; N is the total number of UAVs;

[0056] The inspection route can be a preset set of inspection point coordinate points R i , and R i ={(x i,1 , y i,1 ),...,(x i,j , y i,j )}, where, (x i,j , y i,j ) is the coordinate of the j-th inspection point. For example, the inspection route of the i-th UAV can be: (0,0) → (0,500) → (500,500) → (500,0);

[0057] In this embodiment, the inspection flight parameters of at least two drones are completely different. Thus, in this embodiment, the inspection tasks of the drones in the drone cluster can be planned through formula (1) and the inspection point coordinate point set to ensure that each drone covers its corresponding inspection area and avoid repeated inspections, thereby improving the search and rescue efficiency;

[0058] S2. Each drone in the drone cluster executes the inspection task according to its corresponding inspection flight parameters. During the execution of the inspection task, each drone uses the imaging device (such as an infrared imaging device) carried on the drone to obtain the ground image (the ground image can be an infrared image) I(x, y) in the inspection area in real time, and performs trapped person detection based on the ground image I(x, y) to determine whether there are trapped persons;

[0059] Specifically, in this embodiment, a target detection model based on a neural network (CNN) can be used for trapped person detection, and the target loss function L is used to optimize the parameters of the target detection model so that the target detection model can accurately identify and distinguish the trapped persons in the ground image (as shown in Figure 2 ), where the expression of the target loss function L is shown in formula (2):

[0060]

[0061] where N is the number of samples (i.e., the ground images used for training the target detection model); y i is the true value of the i-th pixel in the image (the true value is 0 or 1, representing the background or the target (i.e., the trapped person) respectively); p i is the predicted probability value;

[0062] S3. After determining that there are trapped persons, several drones execute collaborative search and rescue tasks;

[0063] Specifically, the collaborative search and rescue tasks include:

[0064] Using the imaging device carried on at least one drone to obtain images of the trapped persons from different angles and send them to the control end; where the imaging device includes a visible light camera and / or an infrared camera, correspondingly, the images include visible light images and / or infrared images, and the control end can be a smart terminal carried by search and rescue personnel, including smartphones, PC computers, tablets, smart watches, smart helmets, and other wearable devices, etc. Thus, the operator at the control end can judge the physical condition and mental state of the trapped persons, such as whether they are injured, etc., through the sent-back images of the trapped persons;

[0065] And / or, provide illumination to the area where the trapped person is located through the lighting equipment carried on at least one drone, so that the trapped person can see the surrounding environment, such as the surrounding terrain and landforms, in a dim environment (such as at night). At the same time, the illumination intensity of the lighting equipment can be determined by formula (3):

[0066]

[0067] Wherein, I is the illumination intensity, unit: lm (lumen); P is the power of the lighting equipment, unit: W (watt); d is the distance between the drone and the trapped person, which can be obtained in real time through equipment such as a distance sensor carried on the drone;

[0068] And / or, conduct voice interaction with the trapped person through the voice equipment (such as a megaphone, etc.) carried on at least one drone to obtain information about the trapped person (including the identity information, physical condition, and mental state of the trapped person) and / or information about the environment around the trapped person, etc., and send it to the control terminal to facilitate the operators at the control terminal to timely master the relevant information of the trapped person and the environment where they are located;

[0069] And / or, obtain the position of the trapped person in real time through the target tracking unit carried on at least one drone and send it to the control terminal; specifically, in this embodiment, the position of the trapped person is obtained in real time through formula (4):

[0070] T j (t) = {(X j (t), Y j (t))|t0 ≤ t ≤ t f} (4)

[0071] Wherein, T j (t) is the movement trajectory of the trapped person; (X j (t), Y j (t)) is the position of the trapped person at the current moment t; t0 is the moment when the trapped person is first discovered, for example, 10:30; t f is the end moment of the search and rescue, which can be preset, for example, 11:30; thus, the movement trajectory of the trapped person can be tracked in real time and the position of the trapped person can be timely mastered;

[0072] Furthermore, the position of the trapped person at the current moment t can be obtained through formula (5):

[0073]

[0074] Wherein, (X j , Y j ) is the position of the trapped person; I(x, y) is the ground image, and (x, y) is the pixel value of any pixel point in the ground image; Pj is the set of pixels in the area where the j-th trapped person is located;

[0075] S4. The drone returns to the supply storage point. After loading supplies on the drone, it returns to the area where the trapped person is located according to the supply delivery path and delivers supplies to the area where the trapped person is located;

[0076] Among them, the supply delivery path is determined through the following steps:

[0077] The position of the trapped person is dynamically updated through the position estimation unit carried on at least one drone to obtain the predicted value of the position of the trapped person at the current moment, and the predicted value of the position of the trapped person at the current moment is sent to the control terminal. Specifically, in this embodiment, the position of the trapped person is dynamically updated through formula (6):

[0078]

[0079] Among them, is the predicted value of the position of the trapped person at time k (that is, the predicted value of the position of the trapped person at the current moment k); is the predicted value of the position of the trapped person at time k - 1; K k is the Kalman gain; z k is the actual measurement value; H is the observation matrix;

[0080] According to the predicted value of the position of the trapped person at the current moment, the flight path of the drone is determined, which specifically includes the following steps:

[0081] Obtain the optimal flight path of the drone according to formula (7):

[0082]

[0083] Among them, P fly (t) represents the optimal flight path of the drone at time t, which is a function of time and is used to describe the spatial position coordinates of the drone changing with time during flight; P(t) is the current flight path trajectory of the drone at time t, which can be a set of several position coordinates; argmin represents the variable value when the objective function takes the minimum value; thus, the flight path of the drone is optimized through formula (1) to minimize the flight distance and flight time of the drone flying to the position of the trapped person;

[0084] And determine the optimal supply delivery path of the drone according to formula (8):

[0085]

[0086] Among them, R resupplyIt represents the optimal material delivery path of the UAV, and t represents the time variable. Thus, according to the real-time position change of the trapped person, the material delivery path of the UAV can be continuously optimized, and finally the shortest flight path can be obtained to ensure that the UAV can fly quickly to the position of the trapped person to complete the accurate delivery of materials.

[0087] Thus, in this embodiment, the Kalman filtering algorithm can be used to dynamically locate the trapped person and provide guidance for the ground search and rescue personnel, enabling the search and rescue personnel to quickly and accurately reach the position of the trapped person, improving the rescue efficiency. At the same time, the UAV can also accurately fly back to the area where the trapped person is located according to the optimal material delivery path determined by the dynamic positioning result of the trapped person to accurately deliver materials.

[0088] Embodiment 2:

[0089] This embodiment provides a personnel search and rescue recognition system, which can implement the personnel search and rescue recognition method described in Embodiment 1. As Figure 3 shown, the personnel search and rescue recognition system includes:

[0090] A number of UAVs 1, which form a UAV cluster;

[0091] A number of imaging devices 2, which are carried on the UAVs 1, and each UAV is equipped with an imaging device 2;

[0092] A target detection unit 3, which is carried on the UAV 1 and is used to detect trapped persons according to the ground images in the inspection area obtained in real time by the imaging device 2 to determine whether there are trapped persons, and the process is the same as step S2;

[0093] A lighting device 4, which is carried on at least one UAV 1 and is used to provide lighting for the area where the trapped person is located;

[0094] A voice device 5, which is carried on at least one UAV 1 and is used to conduct voice interaction with the trapped person;

[0095] A target tracking unit 6, which is carried on at least one UAV 1 and is used to obtain the position of the trapped person in real time;

[0096] A path optimization unit 7, which is used to determine the material delivery path of the UAV according to the position of the trapped person, so that the UAV returns to the area where the trapped person is located according to the material delivery path and delivers materials to the area where the trapped person is located, and the process is the same as step S4;

[0097] And a control terminal 8, which is connected to one or several of the imaging device 2, the target detection unit 3, the lighting device 4, the voice device 5, the target tracking unit 6, and the path optimization unit 7.

[0098] In summary, the present invention can detect and accurately identify trapped persons in real time through airborne imaging devices, and conduct real-time tracking of the trapped persons. Further, through the collaborative work of the UAV swarm, multiple links such as emergency material delivery and guiding ground search and rescue personnel to the search and rescue location are completed. Thus, the advantages of the UAV swarm over a single UAV are fully utilized, enabling it to perform multiple tasks (such as personnel identification, positioning, voice interaction, lighting, and material delivery, etc.) in the complex environment of uninhabited areas, so as to efficiently and accurately complete the personnel search and rescue task, and having broad application prospects.

[0099] It should be noted that the technical features in the above-mentioned Embodiments 1-2 can be combined arbitrarily, and the combined technical solutions all fall within the protection scope of the present application. In this text, terms such as "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the said element.

[0100] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for searching and identifying personnel based on drone clusters and airborne imaging equipment, characterized in that: The steps include: Each drone in the drone cluster receives the inspection flight parameters corresponding to it; Each drone in the drone cluster performs the inspection task according to its corresponding inspection flight parameters. During the inspection task execution, each drone obtains the ground image in the inspection area in real time through the imaging device carried on the drone, and detects trapped persons based on the ground image. After confirming the presence of trapped persons, several drones carried out coordinated search and rescue missions; In addition, the drone returns to the material storage point, and after the materials are mounted on the drone, it returns to the area where the trapped persons are located according to the material delivery path, and delivers the materials to the area where the trapped persons are located.

2. The method for searching and identifying personnel according to claim 1, characterized in that: The inspection flight parameters include the inspection height and inspection route of the drone, and the inspection height is obtained by the following formula: Among them, H i is the cruising altitude of the i-th UAV; H min is the minimum safe height of the i-th UAV, and H i ≥H min ; A coverage is the inspection area of ​​the i-th drone; N is the total number of drones.

3. The method for searching and identifying personnel according to claim 1, characterized in that: The collaborative search and rescue mission includes: Images of trapped persons are acquired from different angles through imaging equipment carried by at least one UAV and sent to a control terminal.

4. The method for searching and identifying personnel according to claim 1, characterized in that: The collaborative search and rescue mission includes: Provide lighting for the area where the trapped persons are located by using lighting equipment carried by at least one drone.

5. The method for searching and identifying personnel according to claim 1, characterized in that: The collaborative search and rescue mission includes: Voice interaction is carried out with the trapped persons through the voice device carried by at least one drone.

6. The method for searching and identifying personnel according to claim 1, characterized in that: The collaborative search and rescue mission includes: The location of the trapped person is obtained in real time through a target tracking unit carried by at least one UAV.

7. The method for searching and identifying personnel according to claim 7, characterized in that: The position of the trapped person is obtained according to the following formula: T j (t)={(X j (t),Y j (t))|t0≤t≤t f } Among them, T j (t) is the moving trajectory of the trapped person; (X j (t),Y j (t)) is the position of the trapped person at the current time t; t0 is the time when the trapped person is first found; t f is the end time of search and rescue; I(x,y) is the ground image, (x,y) is the pixel value of any pixel in the ground image; P j is the pixel set of the area where the jth trapped person is located.

8. The method for searching and identifying personnel according to claim 7, characterized in that: The material delivery path is determined by the following steps: The position estimation unit carried by at least one UAV dynamically updates the position of the trapped person based on the following formula to obtain the predicted value of the trapped person's position at the current moment: in, is the predicted value of the trapped person’s position at time k; is the predicted value of the trapped person’s position at time k-1; K k is the Kalman gain; z k is the actual measurement value; H is the observation matrix; The optimal flight path of the drone is determined according to the following formula: Among them, P fly (t) represents the optimal flight path of the UAV at time t; P(t) is the current flight path trajectory of the UAV at time t; argmin represents the variable value when the objective function takes the minimum value; And determine the optimal material delivery path of the drone according to the following formula: Among them, R resupply represents the optimal material delivery path of the UAV, and t represents the time variable.

9. The method for searching and identifying personnel according to claim 1, characterized in that: The inspection flight parameters of at least 2 drones were completely different.

10. A personnel search and rescue identification system, characterized in that: include: A number of drones, which form a drone swarm; A plurality of imaging devices are carried on the drones, and each drone is equipped with an imaging device; A target detection unit, which is carried on the UAV and is used to detect trapped persons based on the ground image in the inspection area acquired in real time by the imaging device to determine whether there are trapped persons; A lighting device, which is carried on at least one UAV and is used to provide lighting for the area where the trapped persons are located; A voice device, which is carried on at least one UAV and is used for voice interaction with the trapped person; A target tracking unit, which is carried on at least one UAV and is used to obtain the position of the trapped person in real time; A path optimization unit is used to determine the material delivery path of the UAV according to the location of the trapped person, so that the UAV returns to the area where the trapped person is located according to the material delivery path and delivers materials to the area where the trapped person is located; And a control end, which is connected to one or more of the imaging device, the target detection unit, the lighting device, the voice device, the target tracking unit and the path optimization unit.

Images