A UAV disaster relief system and method based on deep learning and radar detection

Abstract

The invention discloses a UAV disaster relief system and method based on deep learning and radar detection, which uses a microwave radar to transmit an electromagnetic wave signal of a certain frequency and uses a receiving antenna to receive a human body echo signal; collects radar echo signals of a human body in a variety of different environments to increase the diversity of signal waveforms; selects and processes the received echo signals to obtain a required signal waveform; divides the collected effective signal bands into a training set and a test set according to a suitable ratio; inputs data of all training sets into a convolutional neural network, completes model training on a cloud server, and then uses the trained model to detect whether there are people trapped; uses a UAV to carry a tested radar detection device to detect the existence information of living bodies in the environment in real time; the detection device uses a WiFi module to transmit the collected environmental echo signals to a server, and obtains a live body detection result.

Description

Technical Field

[0001] The present invention relates to the field of through-wall radar human body detection and the field of unmanned aerial vehicle technology, and in particular to a unmanned aerial vehicle disaster relief system and method based on deep learning and radar detection. Background Art

[0002] Nowadays, drones are playing an increasingly important role in post-disaster rescue work. They are beginning to be used in various complex environments and have a role that cannot be replaced by traditional rescue methods. Although there are some disaster relief drones now and they have been put into post-disaster rescue work, most of these drones use cameras, infrared pyroelectrics and other equipment to detect the location of trapped people, which will have some limitations. When encountering some high temperature, cold or complex terrain and topography, the search and rescue efficiency of these rescue equipment will be greatly reduced. At the same time, the complex and changing environment will affect the detection results of these devices.

[0003] Taking all the above into consideration, traditional UAV search and rescue equipment has relatively stringent requirements on the environment. It is difficult to achieve the expected results in some special environments and it is difficult to adapt to complex and changing environments. Summary of the invention

[0004] In order to solve the shortcomings of the prior art, the liveness detection of human body is performed by microwave radar, and a deep learning algorithm is added to detect human body through walls, so as to quickly and accurately identify the position of trapped persons, and achieve accurate rescue without being affected by complex terrain, topography and changeable natural environment, so as to improve the rescue efficiency of search and rescue personnel. The present invention adopts the following technical solutions:

[0005] A UAV disaster relief system based on deep learning and radar detection includes: a radar detection device, a processing module, a neural network model and a UAV. The UAV is equipped with a radar detection device to obtain a human echo signal. The effective signal band is obtained through the processing module and input into a trained convolutional neural network model set up on a cloud server to detect trapped people. The detection results are fed back to the UAV to guide the UAV to adjust the search and rescue trajectory.

[0006] Furthermore, the transmitting signal of the radar detection device is a step signal:

[0007]

[0008] Wherein, N represents the number of operating frequencies of the signal, Re represents the imaginary part of the function, f0 represents the starting frequency of the stepped frequency radar, Δf represents the frequency interval, and t represents the time;

[0009] The echo signal of the radar detection device is:

[0010]

[0011] Among them, τ=2R / c represents the delay time of the echo signal, R represents the distance between the radar and the object being measured, and c represents the speed of light propagation.

[0012] Furthermore, in the processing module, when the electromagnetic wave emitted by the radar detection device encounters a stationary object, the frequency of its reflected wave does not change. If the object moves slightly or produces a small action, the frequency of the object echo signal received by the radar detection device will change. Therefore, in order to obtain a useful signal band, it is necessary to orthogonally demodulate the echo signal and the transmitted signal, and then perform filtering processing to filter out the high-frequency component of the signal and fix it in the baseband, and then obtain the baseband I and Q echo signals:

[0013]

[0014] Wherein, N represents the number of operating frequencies of the signal, j is the unit of imaginary number, τ represents the delay time of the echo signal, f0 represents the starting frequency of the stepped frequency radar, Δf represents the frequency interval, and t represents time.

[0015] Furthermore, after the drone detects vital signs, it hovers above the location and sounds an alarm to tell the rescue team the specific location of the trapped person. If no trapped person is detected, the drone will continue to move along a pre-set trajectory. The drone will hover briefly while running on the preset trajectory. At this time, the microwave radar module installed under the drone will continuously emit electromagnetic wave signals. At the same time, the receiving antenna will receive the echo signal in real time to determine whether there are people trapped in the shelter under the drone.

[0016] Furthermore, the UAV records and predicts the search and rescue route based on the feedback information of vital sign activities, and adjusts the route for search and rescue based on the continuity of the detected vital sign activity locations and the number of vital sign activities in the unit area. For example, if the UAV detects that the previous search and rescue location is in a zigzag or circular shape, which indicates that there may have been a staircase or corridor before the disaster, the UAV will continue to move forward or reverse along the zigzag or circular shape; if the UAV detects that the unit area is increasing or decreasing, which indicates that many people may have fled along this path during the disaster, the UAV will move forward or reverse in the direction of the area with the largest increase.

[0017] Furthermore, the classification labels of the neural network include stillness and vital signs activities. Vital signs activities include: body movement, limb movement, and small movements of the human body such as chest movement, facial movement, head movement and fingertip movement. Vital signs activities include body movement, local body movement and small movements of the human body. Drones usually move at a long distance. The first vital sign activity obtained is usually body movement. When the vital sign activity is obtained, the distance between the drone and the human body is shortened to further detect the category of local body activities. The state of the human body is judged by the classification of limb activities, such as the hand drives the body to move, the leg does not move, etc. According to the local activities The drone can also determine the position of the human body and the degree of injury to a certain extent. For example, the relative position of the moving limbs and torso, torso and head can be used to determine whether the human body has suffered a fracture, etc., which can be used as reference information for subsequent rescue. When the vital signs obtained by the drone are tiny movements of the human body, it usually indicates that the human body is alive but has suffered certain trauma. For example, when the chest rises and falls, it indicates that the human body has signs of life. By obtaining the subdivided categories of tiny movements of the human body, the human body posture can be determined as reference information for subsequent rescue. For example, if the ankle movement, chest rise and fall, and facial movement are detected, it can be determined whether the human body is lying or sitting. If the human body is buried, it can be determined whether the head is facing up or down.

[0018] A UAV disaster relief method based on deep learning and radar detection includes the following steps:

[0019] S1, emits electromagnetic wave signals through a radar detection device to collect echo signals of the human body in different environments, thereby increasing the diversity of signal waveforms;

[0020] S2, select and process the echo signal to obtain the effective signal band;

[0021] S3, divide the effective signal band into training set and test set in proportion;

[0022] S4, input the training set into the neural network model and complete the model training, and use the trained model to detect the echo signal to determine whether there are people trapped;

[0023] S5, uses a radar detection device mounted on a drone to detect the presence of living things in the environment in real time;

[0024] S6, the radar detection device uses the WiFi module to transmit the collected environmental echo signal to the trained neural network model on the cloud server, and obtains the human survival detection result;

[0025] S7: The drone adjusts the search and rescue trajectory in real time based on the feedback detection results.

[0026] Furthermore, the radar detection device in S1 emits electromagnetic waves to illuminate the human body in different movements under the wall ruins, collects the human body echo signal, and extracts the human body feature information from the Doppler frequency information of the echo signal, thereby increasing the diversity of the received human body echo signal and realizing the detection and identification of human targets.

[0027] Furthermore, in S3, the collected human echo signal is first extracted into effective bands, and then the effective signal waveform is randomly divided according to the same time interval, and then the divided signal bands are randomly selected. Finally, all the selected signal waveforms are used as the data set for deep learning, and the data set is divided into a training data set and a test data set in a ratio of 7:3.

[0028] Furthermore, in S4, the training set is input into the convolutional neural network, the waveform data in the training set is used to train the model, and the corresponding loss function and the size of each weight index are obtained. Then, the test data set is used to test the trained model, and the loss function of the test set and the size of the model accuracy are obtained. Then, according to the test results of the test set, the model is adjusted and optimized, and finally a trained neural network model with high accuracy is obtained. In the subsequent human body liveness detection process, the real-time collected environmental echo signal is input into the model for testing, and the final detection result is output.

[0029] The advantages and beneficial effects of the present invention are:

[0030] 1. Microwave radar is used to detect buried personnel through walls, avoiding the influence of the environment like cameras or infrared pyroelectrics. Using microwave radar for detection can avoid the influence of complex environments and bad weather, and better realize human liveness detection in multiple environments.

[0031] 2. Use deep learning algorithms to process the echo signals received by the radar module and train the model, which improves the accuracy of the detection equipment in identifying human living signals.

[0032] 3. The combination of drones and human body detection equipment can be used to search and rescue trapped people. This can be done without being affected by complex terrain and topography, and can be carried out freely in post-disaster scenes. This not only reduces the difficulty of rescue for rescuers, but also improves their efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the operation of the system of the present invention.

[0034] Figure 2 It is a schematic diagram of the signal processing of the radar echo signal in the present invention.

[0035] Figure 3 It is a flow chart of the monitoring device in the present invention for detecting whether a human body exists. DETAILED DESCRIPTION

[0036] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0037] like Figure 1 As shown, a UAV disaster relief system and method based on deep learning and radar detection, the system includes: a microwave radar, a processing module, a cloud server, and a UAV. The UAV is equipped with a microwave radar. A convolutional neural network is trained on the cloud server. The echo signal obtained by the microwave radar is filtered through the processing module, and an effective signal band is selected. Then, the trained convolutional neural network is used to identify whether there are people trapped. The UAV adjusts its motion trajectory in real time according to the feedback information identified.

[0038] The method comprises the following steps:

[0039] 1. Use the transmitting antenna of the microwave radar to transmit electromagnetic wave signals of a certain frequency, and use the receiving antenna to receive the human body echo signal; collect radar echo signals of the human body in a variety of different environments to increase the diversity of signal waveforms;

[0040] 2. Select and process the received echo signal to obtain the required signal band;

[0041] 3. Divide the collected effective signal bands into training sets and test sets according to appropriate proportions;

[0042] 4. Input all training set data into the convolutional neural network and complete the model training on the cloud server. Then use the trained model to detect whether there are people trapped;

[0043] 5. Use drones to carry tested radar detection devices to detect the presence of living things in the environment in real time;

[0044] 6. The detection device uses the WiFi module to transmit the collected environmental echo signal to the server and obtain the liveness detection result;

[0045] 7. The drone adjusts its motion trajectory in real time based on feedback information;

[0046] In this embodiment, the use of the transmitting antenna of the microwave radar to transmit the electromagnetic wave signal and the use of the receiving antenna to receive the human echo signal means that the radar transmitter generates a radar waveform that meets the requirements, and then radiates it from the transmitting antenna through the feeder and the transceiver switch. After the transmitted electromagnetic wave signal encounters the human target, a part of the electromagnetic wave is reflected and then received by the receiver after passing through the receiving antenna and the transceiver switch. Radar echo human feature extraction is a key link in radar human detection. When detecting the human body, the radar transmits electromagnetic waves to irradiate the human body, and extracts human feature information from the Doppler frequency information of the radar echo, so that the detection and identification of the human target can be realized.

[0047] In this embodiment, the diversity of signal waveforms is increased by collecting radar echo signals of human bodies in various environments, which means that various post-disaster scenarios are simulated according to the actual application environment, and microwave radar is used to collect echo signals of human bodies under the wall ruins. At the same time, the human body under the wall needs to make different movements or carry out activities of different magnitudes to simulate the various movements of trapped persons under the ruins, so as to increase the diversity of received human body echo signals. The waveform collection of echo signals is not static, and new waveform data will be continuously added according to the actual situation, and it will continue to grow with the increase of subsequent application scenarios.

[0048] In this embodiment, the received echo signal is selected and processed to obtain the required signal waveform, which means that when the electromagnetic wave emitted by the radar encounters a stationary object, the frequency of its reflected wave does not change. If the object moves slightly or produces a small movement (such as the movement of a person's body, or the movement of a person's fingers or arms, etc.), the frequency of the object echo signal received by the radar will change. Therefore, in order to obtain a useful signal band, the radar echo signal needs to be demodulated and filtered. After the waveform processing is completed, the useful band is selected and this signal band is used as the required signal waveform, and then further operations are performed.

[0049] The purpose of human feature extraction from radar echo is to extract these rich Doppler information from radar echo and use these features as reference to achieve the purpose of human detection and human state recognition. The transmission signal of the microwave radar module is a step frequency signal. If the step frequency signal is x(t), the formula of the radar transmission signal is as follows:

[0050]

[0051] Where x(t) represents the radar's transmitted signal, N represents the number of the signal's operating frequencies, Re represents the imaginary part of the function, f0 represents the starting frequency of the stepped frequency radar, Δf represents the frequency interval, and t represents time.

[0052] Assuming the delay time of the radar echo signal is τ, the formula of the echo signal is as follows:

[0053]

[0054] Where s(t) represents the radar echo signal, and τ represents the delay time of the echo signal.

[0055] The delay time of the radar echo signal is τ, which is expressed as:

[0056] τ=2R / c

[0057] Where R represents the distance between the radar and the object being measured, and c represents the speed of light propagation.

[0058] The echo signal is orthogonally demodulated with the transmission signal, and then filtered to remove the high-frequency components of the signal so that it is fixed in the baseband. Then, the baseband I and Q echo signals z(t) are obtained. The formula of the processed echo signal is:

[0059]

[0060] Wherein z(t) represents the echo signal after filtering, and j is the unit of imaginary number.

[0061] The process of signal processing of radar echo signal is as follows Figure 2 shown.

[0062] In this embodiment, dividing the collected effective signal bands into training sets and test sets in an appropriate ratio means first extracting the effective bands of the collected human echo signals, and then randomly dividing the effective signal waveforms according to the same time intervals, and then randomly selecting the divided signal bands, and finally using all the selected signal waveforms as the data set for deep learning, and dividing the data set into training data set and test data set in a ratio of 7:3, that is, using 70% of the data to train the network and using the remaining 30% to test the network.

[0063] In this embodiment, inputting all the training set data into the convolutional neural network, completing the model training, and using the trained model to detect whether a person is trapped means inputting the intercepted training data set into the convolutional neural network, using the waveform data in the training set to train the model, and obtaining the corresponding loss function and the size of each weight index, and then using the test data set to test the trained model, obtaining the loss function of the test set and the size of the model accuracy, and then adjusting and optimizing the model according to the test results of the test set, and finally obtaining a neural network model with a higher accuracy. In the subsequent human body liveness detection process, the real-time collected environmental echo signal is input into the model for testing, and the final detection result is output.

[0064] In this embodiment, the use of a drone to carry a tested radar detection device to detect the presence of living things in the environment in real time means using a drone to carry a tested radar detection device and fix it to the middle and lower part of the drone. At the same time, the direction of the transmitting and receiving antennas of the radar module should be adjusted vertically downward to better transmit and receive radar signals. By combining the drone with the radar detection device, real-time detection of living objects in the environment can be achieved.

[0065] In this embodiment, regarding the basic structure of the radar detection device, the ESP32 chip is selected as the main control chip, the radar module is used as a peripheral, and the device is also equipped with an alarm circuit. When the device receives feedback from the human body's living signal, it will issue an alarm prompt sound to tell the search and rescue team the location information of the trapped person.

[0066] In this embodiment, the detection device uses the WiFi module to transmit the collected environmental echo signal to the server and obtain the liveness detection result, which means that the detection equipment on the drone receives the reflected object echo signal through the radar module, and the echo signal is transmitted to the ESP32 chip through the serial port after signal processing, and then the ESP32 transmits it to the cloud server through the WiFi signal for waveform analysis, and then the server inputs the collected echo signal data into the pre-trained model for waveform detection, and finally the cloud server outputs the analysis result of the input signal. In this way, it can be concluded whether there are people buried in the ruins.

[0067] In this embodiment, the drone will adjust its motion trajectory in real time according to feedback information, which means that the radar detection device receives the results of human liveness detection returned from the cloud server, and the drone adjusts its motion trajectory in real time according to the feedback results. When a human liveness signal is detected in the ruins, the drone will stop moving and hover above the location. At the same time, the radar detection device will issue an alarm to tell the rescue team the specific location of the trapped person. If no trapped person is detected, the drone will continue to move along a pre-set trajectory. During the operation of the drone on the preset trajectory, the drone will perform short hovers many times. At this time, the microwave radar module installed under the drone will continuously emit electromagnetic wave signals. At the same time, the receiving antenna will receive the echo signal in real time to determine whether there are people trapped in the shelter under the drone. The target detection process is as follows: Figure 3 shown.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A UAV disaster relief system based on deep learning and radar detection, comprising: A radar detection device, a processing module, a neural network model and a drone, characterized in that: the drone is equipped with a radar detection device to obtain a human echo signal, obtains an effective signal band through a processing module, and inputs a trained neural network model to detect trapped personnel, and feeds back the detection results to the drone to guide the drone to adjust the search and rescue trajectory; after the drone detects vital signs, it hovers above the corresponding position and alarms to tell the rescue team the specific location of the trapped person. If no person is detected to be trapped, the drone continues to move along a pre-set trajectory. During the operation of the drone on the preset trajectory, it will hover briefly, and at this time, the radar detection device installed under the drone will continuously emit electromagnetic wave signals. At the same time, the receiving antenna will receive the echo signal in real time to determine whether there are people trapped in the shelter below the drone; the drone will record and predict the search and rescue path based on the feedback information of vital signs activities, and adjust the path for search and rescue based on the continuity of the detected vital signs activity locations and the number of vital signs activities per unit area. Specifically, the drone will determine that the shape of the previous search and rescue location may have been a pedestrian path before the disaster, and the drone will continue to move forward or reverse along the shape; if the drone detects that the number of vital signs activities per unit area is increasing or decreasing, it means that many people may have been fleeing along this path during the disaster, and the drone will move forward or reverse in the direction of the area with the largest increase.

2. The UAV disaster relief system based on deep learning and radar detection according to claim 1, characterized in that: The transmitting signal of the radar detection device is a step signal: Wherein, N represents the number of operating frequencies of the signal, Re represents the imaginary part of the function, f0 represents the starting frequency of the stepped frequency radar, Δf represents the frequency interval, and t represents the time; The echo signal of the radar detection device is: Among them, τ=2R / c represents the delay time of the echo signal, R represents the distance between the radar and the object being measured, and c represents the speed of light propagation.

3. The UAV disaster relief system based on deep learning and radar detection according to claim 1, characterized in that: The processing module performs orthogonal demodulation on the echo signal and the transmission signal, and then performs filtering processing to filter out the high-frequency component of the signal so that it is fixed in the baseband, and then obtains the baseband echo signal: Wherein, N represents the number of operating frequencies of the signal, j is the unit of imaginary number, τ represents the delay time of the echo signal, f0 represents the starting frequency of the stepped frequency radar, Δf represents the frequency interval, and t represents time.

4. The UAV disaster relief system based on deep learning and radar detection according to claim 1, characterized in that: The classification labels of the neural network include stillness and vital sign activities. Vital sign activities include body movement, local body activities and tiny movements of the human body. When the vital sign activities acquired by the drone are body movement, the distance between the drone and the human body is shortened to further detect the categories of the local body activities. The state of the human body is judged by the classification of limb activities, which serves as reference information for subsequent rescue. When the vital sign activities acquired by the drone are tiny movements of the human body, the human body posture is judged by acquiring the subdivided categories of tiny movements of the human body, which serves as reference information for subsequent rescue.

5. A disaster relief method of the UAV disaster relief system based on deep learning and radar detection according to claim 1, characterized in that: The steps include: S1, transmitting electromagnetic wave signals through a radar detection device to collect echo signals of the human body in different environments; S2, obtain the effective signal band according to the echo signal; S3, divide the effective signal band into training set and test set in proportion; S4, input the training set into the neural network model and complete the model training; S5, using a drone equipped with a radar detection device; S6, the radar detection device transmits the collected environmental echo signal to the trained neural network model and obtains the human survival detection result; S7: The drone adjusts the search and rescue trajectory in real time based on the feedback detection results.

6. The disaster relief method according to claim 5, characterized in that: In S1, the radar detection device emits electromagnetic waves to illuminate the human body in different movements under the wall ruins, collects the human body echo signal, and extracts the human body feature information from the Doppler frequency information of the echo signal.

7. The disaster relief method according to claim 5, characterized in that: In S3, the collected human body echo signal is firstly extracted for effective bands, and then the effective signal waveform is randomly segmented according to the same time intervals, and then the segmented signal bands are randomly selected, and finally the selected signal waveform is used as a data set for deep learning, which is divided into a training data set and a test data set.

8. The disaster relief method according to claim 5, characterized in that: In S4, the training set is input into the convolutional neural network, the waveform data in the training set is used to train the model, and the corresponding loss function and the size of each weight index are obtained. Then, the test data set is used to test the trained model, and the loss function and model accuracy of the test set are obtained. Then, according to the test result of the test set, the model is adjusted and optimized, and finally a trained neural network model is obtained.

Images