A UAV rescue method and system based on deep learning

Abstract

The method of the embodiment of the present invention proposes a drone rescue method and system based on deep learning, wherein the management server receives the data information of the missing person sent by the mobile terminal, and sends the target tracking task to the drone group; the drone group takes aerial photos along the preset route; the management server compares and identifies the aerial data, and sends the identified suspected image to the drone group; the drone group takes secondary aerial photos and locates the suspected target; the management server sends the data to the mobile terminal. The present invention generates the activity range of the missing person by setting the initial position assessment, and then completes the aerial coverage of the activity range through the coordination of the drone group, and then accurately identifies the target from the aerial image through the combination of recognition model and manual work, which is accurate, efficient and convenient, and avoids the problems of high cost, time consumption and low efficiency in manual search.

Description

[Technical field]

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a unmanned aerial vehicle rescue method and system based on deep learning. [Background technology]

[0002] Self-guided travel has been popular among travel enthusiasts. Since travel enthusiasts are autonomous and spontaneous, and most of them have not received professional training, they are also accompanied by huge risks while pursuing excitement in adventures, and loss of contact occurs frequently.

[0003] In the process of searching and rescuing missing persons in the wild, the rescue work is often greatly inconvenienced due to the large search area, harsh environment, and inconvenient transportation. In addition, the blanket search is inefficient, time-consuming, and costly. Therefore, the best rescue time is often missed due to untimely rescue. [Summary of the invention]

[0004] In view of this, an embodiment of the present invention provides a drone rescue method and system based on deep learning.

[0005] In a first aspect, an embodiment of the present invention provides a drone rescue method based on deep learning, the method comprising:

[0006] S1. The management server receives the data information of the missing person sent by the mobile terminal, generates an initial identification range and a final identification range, and sends a target tracking task to the drone group based on the final identification range. The data information includes the set initial position, target image and missing time;

[0007] S2, the drone group plans and performs aerial photography according to the target tracking task;

[0008] S3, the management server receives the aerial photography data sent by the drone group, pre-processes the aerial images in the aerial photography data, performs comparison and recognition, and sends the recognized suspected images to the drone group;

[0009] S4, the drone group performs secondary aerial photography and positioning of the suspected target in the suspected image within the initial identification range, and sends the secondary aerial photography image and positioning data back to the management server;

[0010] S5. The management server sends the secondary aerial image data, the positioning data, and the suspected image data within the non-overlapping identification range to the mobile terminal.

[0011] As described above, with respect to any possible implementation, there is further provided an implementation, wherein the data information also includes historical data information of missing persons at the place where contact was lost, and information on the age, physical, psychological and material condition of the target missing persons.

[0012] According to the above aspects and any possible implementation, an implementation is further provided, wherein generating the initial recognition range and the final recognition range in S1 specifically includes:

[0013] Calculate the first activity distance L1 of the missing person. The first activity distance L1 is calculated by the formula Calculate, where i is the time interval unit, N is the total time the missing person has been missing, and λ is the adjustment coefficient. is the average distance that different people walk on flat ground in each i time unit measured;

[0014] Calculate the second activity distance L2 of the missing person. The second activity distance L2 is calculated by the formula Calculate, where The average distance between the location where the historical missing persons were found and the initial location, i is the serial number of the historical missing persons in the lost place, m is the number of historical missing persons in the lost place, e is a natural constant, α, β, γ, δ are correction parameters, f1 is the age of the missing persons, if they are elderly, pregnant women or children, f1 is 1, otherwise it is 0, f2 is the physical condition of the missing persons, if they are not healthy, f2 is 1, otherwise it is 0, f3 is the psychological condition of the missing persons, if they are not healthy, f3 is 1, otherwise it is 0, f4 is the material condition of the missing persons, if they are in short supply, f4 is 1, otherwise it is 0;

[0015] Compare the first moving distance L1 and the second moving distance L2, and use the smaller one as the first target moving distance L3, and use the larger one as the second target moving distance L4;

[0016] With the set initial position as the center and the first target activity distance L3 as the radius, an initial recognition range is generated by radiating outward in a two-dimensional plane, and with the second target activity distance L4 as the radius, a final recognition range is generated by radiating outward in a two-dimensional plane.

[0017] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein S2 specifically includes:

[0018] The drone swarm leaves the base based on the constraints and reaches the initial identification range;

[0019] The initial identification range of the drone group is determined by the following function: Among them, v i (t) is the speed of the i-th UAV at time t, k c is the speed adjustment coefficient, Ω i is the recognition range of the i-th drone, h is the penalty function, C is the expected recognition range value, g(z0,q) is the recognition capability function, c0 is the recognition constant, q is the drone recognition range Ωi Points inside, γ i (q, t) is the identification value of point q at time t;

[0020] The drone swarm adjusts its position and uploads to the drone swarm network:

[0021] Defining the dispersion function in, is the dispersion value of UAV i and its neighboring UAV j at time k, is the position of UAV i at time k, is the position of drone j at time k, μ=N*π*C R 2 / A, N is the total number of drones, C R is the aerial photography range of the drone, i.e. the recognition range, and A is the final recognition range area. is the density value of the relationship area of ​​UAV i at time k, μ is the expected density value, S = {s1, s2, ..., s n}, m is the total number of drones in the relationship area, is the distance between drones,

[0022] in, in, is the total dispersion value between UAV i and its neighboring UAVs,

[0023] Define the set of drones adjacent to the drone Among them, the total scattered value of the drone is greater than the aggregate of other drones;

[0024] Relationship area dispersion value is the number of times UAV i is selected as the target UAV to move to time k,

[0025] Define relationship area density value

[0026] Define the relational region qualification function Among them, α is the adjustment constant, N is the total number of drones, and the drone with the highest qualified value in each relationship area is taken as the target drone;

[0027] The drone group updates its position until the drone relationship area density values ​​of all drone relationship areas converge to meet the preset requirements, completing the aerial photography coverage of the final identification range;

[0028] The drone swarm reaches the initial identification range until the aerial photography is covered, and aerial photography is carried out at a preset frequency.

[0029] According to the aspects described above and any possible implementation method, an implementation method is further provided, wherein the constraint conditions are set according to the target tracking task, and the constraint conditions include the number of drones in the drone swarm, the flight speed of the drones, the position coordinates of the drones, the maximum deflection angle of the drones, the maximum deflection angle change of the drones, the recognition range of the drones, and the initial recognition range parameters.

[0030] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the preprocessing of the aerial images in the aerial data in S3 specifically includes:

[0031] De-noising and extracting appearance features of aerial images;

[0032] The extracted appearance features are input into the pre-trained first-level SVM classifier to determine whether there is a distress sign based on the confidence level. If so, the corresponding aerial image is marked as a suspected image;

[0033] The extracted appearance features are input into the pre-trained secondary SVM classifier to determine whether there is a color target corresponding to the target image based on the confidence level. If so, it is determined whether the number of aerial images with color targets is greater than the quantity threshold. If it is less than the quantity threshold, the corresponding aerial image is marked as a suspected image.

[0034] According to the above aspects and any possible implementation, an implementation is further provided, wherein the comparison and identification in S3 specifically includes:

[0035] Aerial images are identified through key point detection algorithm models to determine whether there are human-shaped targets in the aerial images;

[0036] If there is a humanoid target, calculate the humanoid key points in the aerial image, obtain the maximum circumscribed rectangle of the humanoid key points, obtain the overall maximum circumscribed rectangle of all humanoid key points, and intercept the overall maximum circumscribed rectangle as the humanoid image;

[0037] Select the maximum circumscribed rectangle of the human figure image and the corresponding human figure key points of the preprocessed target image in turn for image comparison and calculate the similarity value S ij , where the similarity value S ij By formula Calculate, where N * is the number of key points of the human figure in the human figure image, M * The number of segments divided by the grayscale value interval in the statistical grayscale histogram of the maximum circumscribed rectangle of the humanoid key point, Q ij is the number of pixels in the jth grayscale value interval of the i-th human key point in the human image, F ijis the number of pixels in the jth grayscale value interval of the i-th human key point of the target image;

[0038] Determine the similarity value S of the aerial image ij Is it greater than the set similarity value threshold? If so, the aerial image is marked as a suspected image.

[0039] In a second aspect, an embodiment of the present invention provides a drone rescue system based on deep learning, the system comprising:

[0040] The mobile terminal is used to send the data information of the missing person, wherein the data information includes the set initial location, target image and missing time;

[0041] The drone swarm is used to plan and perform aerial photography according to the target tracking task; it is also used to perform secondary aerial photography and positioning of suspected targets in the suspected images within the initial identification range, and send the secondary aerial photography images and positioning data back to the management server;

[0042] Management server, including task generation module, image preprocessing module, target recognition module and transmission module,

[0043] The task generation module is used to generate an initial recognition range and a final recognition range, and send a target tracking task to the drone group based on the final recognition range;

[0044] The image preprocessing module is used to receive the aerial photography data sent by the drone fleet and preprocess the aerial images in the aerial photography data;

[0045] The target recognition module is used to compare and recognize the target image through the recognition model, and send the recognized suspected image to the drone group;

[0046] The transmission module is used to receive data information of missing persons sent by the mobile terminal, receive secondary aerial images and positioning data sent by the drone group, and is also used to send secondary aerial image data, positioning data and suspected image data within a non-overlapping identification range to the mobile terminal.

[0047] According to the above aspects and any possible implementation, an implementation is further provided, wherein the task generation module is used to generate an initial recognition range and a final recognition range, specifically including:

[0048] Calculate the first activity distance L1 of the missing person. The first activity distance L1 is calculated by the formula Calculate, where i is the time interval unit, N is the total time the missing person has been missing, and λ is the adjustment coefficient. is the average distance that different people walk on flat ground in each i time unit measured;

[0049] Calculate the second activity distance L2 of the missing person. The second activity distance L2 is calculated by the formula Calculate, where The average distance between the location where the historical missing persons were found and the initial location, i is the serial number of the historical missing persons in the lost place, m is the number of historical missing persons in the lost place, e is a natural constant, α, β, γ, δ are correction parameters, f1 is the age of the missing persons, if they are elderly, pregnant women or children, f1 is 1, otherwise it is 0, f2 is the physical condition of the missing persons, if they are not healthy, f2 is 1, otherwise it is 0, f3 is the psychological condition of the missing persons, if they are not healthy, f3 is 1, otherwise it is 0, f4 is the material condition of the missing persons, if they are in short supply, f4 is 1, otherwise it is 0;

[0050] Compare the first moving distance L1 and the second moving distance L2, and use the smaller one as the first target moving distance L3, and use the larger one as the second target moving distance L4;

[0051] With the set initial position as the center and the first target activity distance L3 as the radius, an initial recognition range is generated by radiating outward in a two-dimensional plane, and with the second target activity distance L4 as the radius, a final recognition range is generated by radiating outward in a two-dimensional plane.

[0052] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the image preprocessing module is used to preprocess the aerial images in the aerial photography data, specifically including:

[0053] De-noising and extracting appearance features of aerial images;

[0054] The extracted appearance features are input into the pre-trained first-level SVM classifier to determine whether there is a distress sign based on the confidence level. If so, the corresponding aerial image is marked as a suspected image;

[0055] The extracted appearance features are input into the pre-trained secondary SVM classifier to determine whether there is a color target corresponding to the target image based on the confidence level. If so, it is determined whether the number of aerial images with color targets is greater than the quantity threshold. If it is less than the quantity threshold, the corresponding aerial image is marked as a suspected image.

[0056] One of the above technical solutions has the following beneficial effects:

[0057] The method of the embodiment of the present invention proposes a drone rescue method and system based on deep learning, wherein the management server receives data information of the missing person sent by the mobile terminal, generates an initial recognition range and a final recognition range, and sends a target tracking task to the drone group based on the final recognition range; the drone group performs aerial photography planning according to the target tracking task and performs aerial photography; the management server receives the aerial photography data sent by the drone group, pre-processes the aerial images in the aerial photography data, performs comparison and recognition, and sends the identified suspected images to the drone group; the drone group performs secondary aerial photography and positioning on the suspected targets in the suspected images within the initial recognition range, and sends the secondary aerial images and positioning data back to the management server; the management server sends the secondary aerial image data, positioning data, and suspected image data within the non-overlapping recognition range to the mobile terminal. The present application utilizes drones to search for missing persons to set an initial position assessment to generate the activity range of the missing persons, and then completes aerial coverage of the activity range through the coordination of a drone swarm, and then accurately identifies the target from the aerial images through a combination of recognition models and manual work. Therefore, the search for missing persons is accurate, efficient, and convenient, avoiding the problems of high cost, time consumption, and low efficiency in manual search, which is conducive to improving the survival rate of missing persons.

Brief Description of the Drawings

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 is a flowchart of a deep learning-based drone rescue method provided by an embodiment of the present invention;

[0060] Figure 2 This is a functional block diagram of a deep learning-based drone rescue system provided in an embodiment of the present invention. [Specific implementation method]

[0061] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Please refer to Figure 1 , which is a flow chart of the deep learning-based UAV rescue method provided by an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0063] S1. The management server receives the data information of the missing person sent by the mobile terminal, generates an initial identification range and a final identification range, and sends a target tracking task to the drone group based on the final identification range. The data information includes the set initial position, target image and missing time;

[0064] S2, the drone group plans and performs aerial photography according to the target tracking task;

[0065] S3, the management server receives the aerial photography data sent by the drone group, pre-processes the aerial images in the aerial photography data, performs comparison and recognition, and sends the recognized suspected images to the drone group;

[0066] S4, the drone group performs secondary aerial photography and positioning of the suspected target in the suspected image within the initial identification range, and sends the secondary aerial photography image and positioning data back to the management server;

[0067] S5. The management server sends the secondary aerial image data, the positioning data, and the suspected image data within the non-overlapping identification range to the mobile terminal.

[0068] The method of the embodiment of the present invention proposes a drone rescue method based on deep learning, which uses drones to search for missing persons to set an initial position assessment to generate the activity range of the missing persons, and then through the coordination and cooperation of the drone group, the aerial coverage of the activity range is completed, and then the target is accurately identified from the aerial image through the recognition model and manual combination. Therefore, the search for missing persons is accurate, efficient and convenient, avoiding the problems of high cost, time consumption and low efficiency in manual search, which is conducive to improving the survival rate of missing persons.

[0069] Specifically, the data information also includes historical data information of missing persons in the place where contact was lost, as well as the age, physical, psychological and material condition information of the target missing persons.

[0070] Furthermore, the initial recognition range and the final recognition range are generated in S1 of the present invention, specifically including:

[0071] Calculate the first activity distance L1 of the missing person. The first activity distance L1 is calculated by the formula Calculate, where i is the time interval unit, N is the total time the missing person has been missing, and λ is the adjustment coefficient. is the average distance that different people walk on flat ground in each i time unit measured;

[0072] Calculate the second activity distance L2 of the missing person. The second activity distance L2 is calculated by the formula Calculate, where The average distance between the location where the missing persons were found and the initial location. i is the serial number of the missing persons in the lost place. m is the number of the missing persons in the lost place. e is a natural constant. α, β, γ, and δ are correction parameters. f1 is the age of the missing persons. If they are elderly, pregnant women, or children, f1 is 1. Otherwise, it is 0. f2 is the physical condition of the missing persons. If they are not healthy, f2 is 1. Otherwise, it is 0. f3 is the psychological condition of the missing persons. If they are not healthy, f3 is 1, such as depression. Otherwise, it is 0. f4 is the material condition of the missing persons. If they are in short supply, f4 is 1. Otherwise, it is 0.

[0073] Compare the first moving distance L1 and the second moving distance L2, and use the smaller one as the first target moving distance L3, and use the larger one as the second target moving distance L4;

[0074] With the set initial position as the center and the first target activity distance L3 as the radius, an initial recognition range is generated by radiating outward in a two-dimensional plane, and with the second target activity distance L4 as the radius, a final recognition range is generated by radiating outward in a two-dimensional plane.

[0075] The present invention evaluates the activity range of the missing person by setting the initial position and the information of the missing person, limits the search range, avoids waste of search resources and meaningless area search, and the initial identification range is the area where the missing person is most likely to be searched. Therefore, it is more valuable to conduct a secondary aerial photography confirmation, and the non-overlapping area between the initial identification range and the final identification range is the area where the missing person may appear.

[0076] Furthermore, S2 of the present invention specifically includes:

[0077] The drone swarm leaves the base based on the constraints and reaches the initial identification range;

[0078] The initial identification range of the drone group is determined by the following function: Among them, v i (t) is the speed of the i-th UAV at time t, k c is the speed adjustment coefficient, Ω i is the recognition range of the i-th drone, h is the penalty function, C is the expected recognition range value, g(z0,q) is the recognition capability function, c0 is the recognition constant, q is the drone recognition range Ω i Points inside, γ i (q, t) is the identification value of point q at time t;

[0079] The drone swarm adjusts its position and uploads to the drone swarm network:

[0080] Defining the dispersion function in, is the dispersion value of UAV i and its neighboring UAV j at time k, is the position of UAV i at time k, is the position of drone j at time k, μ=N*π*C R 2 / A, N is the total number of drones, C R is the aerial photography range of the drone, i.e. the recognition range, and A is the final recognition range area. is the density value of the relationship area of ​​UAV i at time k, μ is the expected density value, S = {s1, s2, ..., s n}, m is the total number of drones in the relationship area, is the distance between drones,

[0081] in, in, is the total dispersion value between UAV i and its neighboring UAVs,

[0082] Define the set of drones adjacent to the drone Among them, the total scattered value of the drone is greater than the aggregate of other drones;

[0083] Relationship area dispersion value is the number of times UAV i is selected as the target UAV to move to time k,

[0084] Define relationship area density value

[0085] Define the relational region qualification function Among them, α is the adjustment constant, N is the total number of drones, and the drone with the highest qualified value in each relationship area is taken as the target drone;

[0086] The drone group updates its position until the drone relationship area density values ​​of all drone relationship areas converge to meet the preset requirements, completing the aerial photography coverage of the final identification range;

[0087] The drone swarm reaches the initial identification range until the aerial photography is covered, and aerial photography is carried out at a preset frequency.

[0088] Specifically, the above-mentioned constraints are set according to the target tracking task, and the constraints include the number of drones in the drone swarm, the flight speed of the drones, the position coordinates of the drones, the maximum deflection angle of the drones, the maximum deflection angle change of the drones, the recognition range of the drones, and the initial recognition range parameters.

[0089] Furthermore, in S3 of the present invention, the aerial images in the aerial data are preprocessed, specifically including:

[0090] De-noising and extracting appearance features of aerial images;

[0091] The extracted appearance features are input into the pre-trained first-level SVM classifier to determine whether there is a distress sign based on the confidence level. If so, the corresponding aerial image is marked as a suspected image;

[0092] The extracted appearance features are input into the pre-trained secondary SVM classifier to determine whether there is a color target corresponding to the target image based on the confidence level. If so, it is determined whether the number of aerial images with color targets is greater than the quantity threshold. If it is less than the quantity threshold, the corresponding aerial image is marked as a suspected image.

[0093] The distress sign may be, for example, "help", "sos", "help", "110", etc., and the color target corresponding to the target image is generally selected from a color that is generally larger than the natural color, such as red.

[0094] It should be noted that the pre-trained SVM classifier can quickly perform image classification and identify suspected images, and the present invention uses a first-level SVM classifier to identify distress signs and a second-level SVM classifier to identify color targets, which can quickly and thoroughly screen out distress signals left intentionally or unintentionally by missing persons and track down the missing persons.

[0095] Furthermore, the comparison and identification in S3 of the present invention specifically includes:

[0096] Aerial images are identified through key point detection algorithm models to determine whether there are human-shaped targets in the aerial images;

[0097] If there is a humanoid target, calculate the humanoid key points in the aerial image, obtain the maximum circumscribed rectangle of the humanoid key points, obtain the overall maximum circumscribed rectangle of all humanoid key points, and intercept the overall maximum circumscribed rectangle as the humanoid image;

[0098] Select the maximum circumscribed rectangle of the human figure image and the corresponding human figure key points of the preprocessed target image in turn for image comparison and calculate the similarity value S ij , where the similarity value S ij By formula Calculate, where N * is the number of key points of the human figure in the human figure image, M * The number of segments divided by the grayscale value interval in the statistical grayscale histogram of the maximum circumscribed rectangle of the humanoid key point, Q ij is the number of pixels in the jth grayscale value interval of the i-th human key point in the human image, Fij is the number of pixels in the jth grayscale value interval of the i-th human key point of the target image;

[0099] Determine the similarity value S of the aerial image ij Is it greater than the set similarity value threshold? If so, the aerial image is marked as a suspected image.

[0100] The present invention identifies all human targets in the search area through a key point detection algorithm model without omission, and then compares similarity values ​​to search for missing persons.

[0101] The embodiments of the present invention further provide device embodiments for implementing the steps and methods in the above method embodiments.

[0102] Please refer to Figure 2 , which is a functional block diagram of a deep learning-based drone rescue system provided by an embodiment of the present invention, the system includes:

[0103] The mobile terminal is used to send the data information of the missing person, wherein the data information includes the set initial location, target image and missing time;

[0104] The drone swarm is used to plan and perform aerial photography according to the target tracking task; it is also used to perform secondary aerial photography and positioning of suspected targets in the suspected images within the initial identification range, and send the secondary aerial photography images and positioning data back to the management server;

[0105] Management server, including task generation module, image preprocessing module, target recognition module and transmission module,

[0106] The task generation module is used to generate an initial recognition range and a final recognition range, and send a target tracking task to the drone group based on the final recognition range;

[0107] The image preprocessing module is used to receive the aerial photography data sent by the drone fleet and preprocess the aerial images in the aerial photography data;

[0108] The target recognition module is used to compare and recognize the target image through the recognition model, and send the recognized suspected image to the drone group;

[0109] The transmission module is used to receive data information of missing persons sent by the mobile terminal, receive secondary aerial images and positioning data sent by the drone group, and is also used to send secondary aerial image data, positioning data and suspected image data within a non-overlapping identification range to the mobile terminal.

[0110] Specifically, the task generation module is used to generate an initial recognition range and a final recognition range, including:

[0111] Calculate the first activity distance L1 of the missing person. The first activity distance L1 is calculated by the formula Calculate, where i is the time interval unit, N is the total time the missing person has been missing, and λ is the adjustment coefficient. is the average distance that different people walk on flat ground in each i time unit measured;

[0112] Calculate the second activity distance L2 of the missing person. The second activity distance L2 is calculated by the formula Calculate, where The average distance between the location where the historical missing persons were found and the initial location, i is the serial number of the historical missing persons in the lost place, m is the number of historical missing persons in the lost place, e is a natural constant, α, β, γ, δ are correction parameters, f1 is the age of the missing persons, if they are elderly, pregnant women or children, f1 is 1, otherwise it is 0, f2 is the physical condition of the missing persons, if they are not healthy, f2 is 1, otherwise it is 0, f3 is the psychological condition of the missing persons, if they are not healthy, f3 is 1, otherwise it is 0, f4 is the material condition of the missing persons, if they are in short supply, f4 is 1, otherwise it is 0;

[0113] Compare the first moving distance L1 and the second moving distance L2, and use the smaller one as the first target moving distance L3, and use the larger one as the second target moving distance L4;

[0114] With the set initial position as the center and the first target activity distance L3 as the radius, an initial recognition range is generated by radiating outward in a two-dimensional plane, and with the second target activity distance L4 as the radius, a final recognition range is generated by radiating outward in a two-dimensional plane.

[0115] Furthermore, the mission generation module is also used for the drone swarm to leave the base and reach the initial identification range based on the constraints;

[0116] The initial identification range of the drone group is determined by the following function: Among them, v i (t) is the speed of the i-th UAV at time t, k c is the speed adjustment coefficient, Ω i is the recognition range of the i-th drone, h is the penalty function, C is the expected recognition range value, g(z0,q) is the recognition capability function, c0 is the recognition constant, q is the drone recognition range Ω i Points inside, γ i (q, t) is the identification value of point q at time t;

[0117] The drone swarm adjusts its position and uploads to the drone swarm network:

[0118] Defining the dispersion function in, is the dispersion value of UAV i and its neighboring UAV j at time k, is the position of UAV i at time k, is the position of drone j at time k, μ=N*π*C R 2 / A, N is the total number of drones, C R is the aerial photography range of the drone, i.e. the recognition range, and A is the final recognition range area. is the density value of the relationship area of ​​UAV i at time k, μ is the expected density value, S = {s1, s2, ..., s n}, m is the total number of drones in the relationship area, is the distance between drones,

[0119] in, in, is the total dispersion value between UAV i and its neighboring UAVs,

[0120] Define the set of drones adjacent to the drone Among them, the total scattered value of the drone is greater than the aggregate of other drones;

[0121] Relationship area dispersion value is the number of times UAV i is selected as the target UAV to move to time k,

[0122] Define relationship area density value

[0123] Define the relational region qualification function Among them, α is the adjustment constant, N is the total number of drones, and the drone with the highest qualified value in each relationship area is taken as the target drone;

[0124] The drone group updates its position until the drone relationship area density values ​​of all drone relationship areas converge to meet the preset requirements, completing the aerial photography coverage of the final identification range;

[0125] The drone swarm reaches the initial identification range until the aerial photography is covered, and aerial photography is carried out at a preset frequency.

[0126] Specifically, the image preprocessing module is used to preprocess the aerial images in the aerial photography data, specifically including:

[0127] De-noising and extracting appearance features of aerial images;

[0128] The extracted appearance features are input into the pre-trained first-level SVM classifier to determine whether there is a distress sign based on the confidence level. If so, the corresponding aerial image is marked as a suspected image;

[0129] The extracted appearance features are input into the pre-trained secondary SVM classifier to determine whether there is a color target corresponding to the target image based on the confidence level. If so, it is determined whether the number of aerial images with color targets is greater than the quantity threshold. If it is less than the quantity threshold, the corresponding aerial image is marked as a suspected image.

[0130] Specifically, the target recognition module is used to:

[0131] Aerial images are identified through key point detection algorithm models to determine whether there are human-shaped targets in the aerial images;

[0132] If there is a humanoid target, calculate the humanoid key points in the aerial image, obtain the maximum circumscribed rectangle of the humanoid key points, obtain the overall maximum circumscribed rectangle of all humanoid key points, and intercept the overall maximum circumscribed rectangle as the humanoid image;

[0133] Select the maximum circumscribed rectangle of the human figure image and the corresponding human figure key points of the preprocessed target image in turn for image comparison and calculate the similarity value S ij , where the similarity value S ij By formula Calculate, where N * is the number of key points of the human figure in the human figure image, M * The number of segments divided by the grayscale value interval in the statistical grayscale histogram of the maximum circumscribed rectangle of the humanoid key point, Q ij is the number of pixels in the jth grayscale value interval of the i-th human key point in the human image, F ij is the number of pixels in the jth grayscale value interval of the i-th human key point of the target image;

[0134] Determine the similarity value S of the aerial image ij Is it greater than the set similarity value threshold? If so, the aerial image is marked as a suspected image.

[0135] Since each unit module in this embodiment can execute Figure 1 For the method shown in the embodiment, the part not described in detail in this embodiment can be referred to Figure 1 Related instructions.

[0136] At the hardware level, the device may include a processor, and optionally an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. Of course, the device may also include hardware required for other services.

[0137] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0138] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0139] The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0140] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0141] For the convenience of description, the above device is described as various units or modules according to their functions. Of course, when implementing the present invention, the functions of each unit or module can be implemented in the same or multiple software and / or hardware.

[0142] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0148] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0149] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0150] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0151] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0152] Each embodiment of the present invention is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0153] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A drone rescue method based on deep learning, characterized in that: The method comprises: S1. The management server receives the data information of the missing person sent by the mobile terminal, generates an initial identification range and a final identification range, and sends a target tracking task to the drone group based on the final identification range. The data information includes the set initial position, target image and missing time; S2, the drone group plans and performs aerial photography according to the target tracking task; S3, the management server receives the aerial photography data sent by the drone group, pre-processes the aerial images in the aerial photography data to obtain suspected images, compares the aerial images with the target images to obtain suspected images, and sends the identified suspected images to the drone group; S4, the drone group performs secondary aerial photography and positioning of the suspected target in the suspected image within the initial identification range, and sends the secondary aerial photography image and positioning data back to the management server; S5. The management server sends the secondary aerial image data, the positioning data, and the suspected image data within the non-overlapping identification range of the initial identification range and the final identification range to the mobile terminal; The initial recognition range and the final recognition range are generated in S1, specifically including: Calculate the first activity distance L1 of the missing person. The first activity distance L1 is calculated by the formula Calculate, where i is the time interval unit, N is the total time of missing persons, λ is the adjustment coefficient, is the average distance walked on flat ground by different persons in each i time interval unit measured; Calculate the second activity distance L2 of the missing person. The second activity distance L2 is calculated by the formula Calculate, where The average distance between the location where the historical missing persons were found and the initial location. i is the serial number of the historical missing persons in the lost place. m is the number of historical missing persons in the lost place. e is a natural constant. α, β, γ, and δ are correction parameters. f1 is the age of the missing person. If the missing person is an elderly person, a pregnant woman, or a child, f1 is 1. Otherwise, it is 0. f2 is the physical condition of the missing person. If the missing person is not healthy, then f2 is 1, otherwise it is 0. f3 The psychological condition of the missing person. If he is not healthy, f3 is 1, otherwise it is 0. f4 The missing persons' material conditions. If they are in short supply, f4 is 1, otherwise it is 0; Compare the first moving distance L1 and the second moving distance L2, and use the smaller one as the first target moving distance L3, and use the larger one as the second target moving distance L4; With the set initial position as the center and the first target activity distance L3 as the radius, an initial recognition range is generated by radiating outward in a two-dimensional plane, and with the second target activity distance L4 as the radius, a final recognition range is generated by radiating outward in a two-dimensional plane.

2. The deep learning-based drone rescue method according to claim 1, characterized in that: In S3, the aerial images in the aerial photography data are preprocessed, specifically including: De-noising and extracting appearance features of aerial images; The extracted appearance features are input into the pre-trained first-level SVM classifier to determine whether there is a distress sign based on the confidence level. If so, the corresponding aerial image is marked as a suspected image; The extracted appearance features are input into the pre-trained secondary SVM classifier to determine whether there is a color target corresponding to the target image based on the confidence level. If so, it is determined whether the number of aerial images with color targets is greater than the quantity threshold. If it is less than the quantity threshold, the corresponding aerial image is marked as a suspected image.

3. A deep learning-based drone rescue system using the method of claim 1, characterized in that: The system comprises: The mobile terminal is used to send the data information of the missing person, wherein the data information includes the set initial location, target image and missing time; The drone swarm is used to plan and perform aerial photography according to the target tracking task; it is also used to perform secondary aerial photography and positioning of suspected targets in the suspected images within the initial identification range, and send the secondary aerial photography images and positioning data back to the management server; Management server, including task generation module, image preprocessing module, target recognition module and transmission module, The task generation module is used to generate an initial recognition range and a final recognition range, and send a target tracking task to the drone group based on the final recognition range; The image preprocessing module is used to receive the aerial photography data sent by the drone fleet and preprocess the aerial images in the aerial photography data; The target recognition module is used to compare and recognize the target image through the recognition model, and send the recognized suspected image to the drone group; The transmission module is used to receive data information of missing persons sent by the mobile terminal, receive secondary aerial images and positioning data sent by the drone group, and is also used to send secondary aerial image data, positioning data and suspected image data within a non-overlapping identification range to the mobile terminal.

4. The deep learning-based drone rescue system according to claim 3, characterized in that: The image preprocessing module is used to preprocess the aerial images in the aerial photography data, specifically including: De-noising and extracting appearance features of aerial images; The extracted appearance features are input into the pre-trained first-level SVM classifier to determine whether there is a distress sign based on the confidence level. If so, the corresponding aerial image is marked as a suspected image; The extracted appearance features are input into the pre-trained secondary SVM classifier to determine whether there is a color target corresponding to the target image based on the confidence level. If so, it is determined whether the number of aerial images with color targets is greater than the quantity threshold. If it is less than the quantity threshold, the corresponding aerial image is marked as a suspected image.

Images