Improved penetration positioning device for unmanned aerial vehicle and use method of improved penetration positioning device

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses an unmanned aerial vehicle improved penetration positioning device, which comprises an unmanned aerial vehicle body, a millimeter wave radar system, a visual acquisition system, an edge server data processing system and a visualization system, and is characterized in that a high-precision RTK-GNSS positioning module and an I MU inertial navigation system are integrated on the unmanned aerial vehicle body; the millimeter wave radar system is installed on the unmanned aerial vehicle body, adopts a 77GHz frequency band, and works in an FMCW modulation mode. According to the improved penetration positioning device for the unmanned aerial vehicle and the use method of the improved penetration positioning device, the high-frequency signal of a millimeter-wave radar system and an FMCW modulation mode are utilized, strong penetrating power is achieved, a deeply-buried or shielded life body is effectively detected, the radar signal processing module is combined with a two-dimensional FFT and CFAR detection algorithm and a deep learning network Poi ntNet + +, life body characteristics are accurately recognized, and the positioning accuracy of the unmanned aerial vehicle is improved. High-precision RTK-GNSS and I MU navigation accurate positioning of an unmanned aerial vehicle body are combined, and the reliability and accuracy of positioning are improved through multi-sensor fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and particularly to an improved penetration positioning device for unmanned aerial vehicles and its usage method. Background Art

[0002] In complex environments such as collapsed object rescue, the rapid and accurate positioning of living bodies is the key to rescue work. Traditionally, unmanned aerial vehicles are equipped with infrared imaging devices, biosensors or acoustic wave sensors for detecting living bodies. The infrared imaging device captures the thermal radiation emitted by the human body or animals for imaging, but it is limited by the detection depth and environmental interference, and the effect is not good under conditions such as high temperature and smoke. The biosensor detects the vital sign signals in the air, such as the carbon dioxide concentration and the gas components generated by breathing. However, its detection range is limited and it is easily interfered by environmental factors such as chemical leakage and high temperature. The acoustic wave sensor locates the living body by listening to the sound signals in the environment, but its effectiveness will be greatly reduced in a noisy environment or when the living body cannot make a sound.

[0003] Although the above traditional positioning methods have certain application values under specific conditions, in complex environments such as collapsed object rescue, they generally face challenges such as inaccurate positioning, limited detection range and susceptibility to environmental interference. There are obvious deficiencies in the infrared imaging and biosensors in penetrating obstacles, and it is difficult to detect living bodies buried deep or blocked. The acoustic wave sensor is limited by environmental noise and detection distance, and often cannot provide reliable living body position information. In addition, most of these traditional positioning methods rely on a single sensing technology and lack the fusion processing of multi-source data, resulting in the reliability and accuracy of the positioning results to be improved. Therefore, there is an urgent need for a new type of unmanned aerial vehicle positioning device and its usage method that can penetrate obstacles and achieve accurate positioning in complex environments. Therefore, we have proposed an improved penetration positioning device for unmanned aerial vehicles and its usage method. Summary of the Invention

[0004] (1) Technical Problems to be Solved

[0005] In view of the deficiencies of the prior art, the present invention provides an improved penetration positioning device for unmanned aerial vehicles and its usage method, which has the advantages of strong penetration ability, high positioning accuracy and anti-environmental interference, and solves the problems of inaccurate positioning and limited detection range of traditional positioning methods in complex environments.

[0006] (2) Technical Solutions

[0007] To achieve the above purposes of strong penetration ability, high positioning accuracy and anti-environmental interference, the present invention provides the following technical solutions: An improved penetration positioning device for unmanned aerial vehicles, comprising an unmanned aerial vehicle body, a millimeter wave radar system, a visual acquisition system, an edge server data processing system and a visualization system:

[0008] The UAV body is integrated with a high-precision RTK-GNSS positioning module and an IMU inertial navigation system;

[0009] The millimeter-wave radar system is installed on the UAV body, operates in the 77GHz band, and uses the FMCW modulation method. The millimeter-wave radar system is equipped with a multi-channel MIMO antenna array to improve spatial resolution; the radar system also includes a radar signal processing module, which processes the radar echo signal using two-dimensional FFT to obtain target distance information, detects targets in complex backgrounds using the CFAR detection algorithm, and combines the deep learning network PointNet++ to perform vital feature recognition on the acquired point cloud data;

[0010] The vision acquisition system includes at least one high-resolution camera;

[0011] The edge server data processing system adopts a distributed architecture, including a high-performance computing cluster, which is used to process data from the millimeter-wave radar and the vision acquisition system. The point cloud data of consecutive frames is spatially aligned through the ICP point cloud registration algorithm, and voxel filtering is used for downsampling optimization. Combining the Kalman filter of the target tracking algorithm realizes the real-time tracking of the position of the living body. The processed data is stored in the time series database and pushed to the visualization system through the WebSocket protocol;

[0012] The visualization system is developed based on Unity and ARKit, can receive the data pushed by the edge server, and realizes the precise positioning of the wearer through SLAM technology. Combining gyroscope data to calculate the field of view angle, and real-time rendering of the dynamic point cloud effect of the living body position or displaying marker points on the three-dimensional map.

[0013] Preferably, the millimeter-wave radar system transmits the original data to the edge server data processing system in real time through the 5G network module, and at the same time packages the precise position information of the UAV into a standardized data frame and sends it together.

[0014] Preferably, the high-resolution camera of the vision acquisition system is stably installed on the gimbal of the UAV body to ensure clear images can be captured in different flight postures.

[0015] Preferably, the edge server data processing system also includes a data preprocessing module, which is used to preprocess the received radar and vision data, including operations such as removing low-quality data frames, image classification and sorting.

[0016] Preferably, the visualization system also includes a user interaction interface, through which the flight state of the UAV can be controlled, data acquisition parameters can be adjusted, and real-time rescue information can be viewed.

[0017] Preferably, it further includes a gesture control system. The gesture control system can capture the user's gesture actions, perform real-time analysis and classification on the user's gestures through ARFoundation, convert them into UAV control instructions, and transmit the instructions to the UAV end through Socket communication technology to achieve remote control of the UAV.

[0018] A usage method of an improved UAV penetration positioning device includes the improved UAV penetration positioning device and further includes the following steps:

[0019] S1. Control the UAV carrying the millimeter-wave radar and the vision acquisition system to fly over the disaster area;

[0020] S2. The millimeter-wave radar collects the vital sign data of the living body, and at the same time the vision acquisition system takes pictures of the environment image of the disaster area;

[0021] S3. Transmit the collected data to the edge server data processing system in real time through the 5G network module;

[0022] S4. The edge server data processing system preprocesses, identifies the vital body characteristics and tracks the position of the received data;

[0023] S5. Push the processed vital body position information, vital sign data and UAV position information to the visualization system through the WebSocket protocol;

[0024] S6. The visualization system renders the dynamic point cloud effect of the vital body position in real time or displays the marked points on the three-dimensional map for the reference of the rescue personnel;

[0025] S7. According to the information of the visualization system, the rescue personnel remotely control the UAV through the gesture control system, adjust its flight position and angle, and conduct precise rescue.

[0026] Preferably, it further includes the step of jointly debugging the UAV, the millimeter-wave radar system, the vision acquisition system, the edge server data processing system and the visualization system before the rescue to ensure the normal data communication and processing process between the systems.

[0027] (III) Beneficial effects

[0028] Compared with the prior art, the present invention provides an improved UAV penetration positioning device and its usage method, which have the following beneficial effects:

[0029] 1. The improved penetration positioning device for the drone and its usage method utilize the high-frequency signal and FMCW modulation method of the millimeter-wave radar system to achieve strong penetration ability, effectively detect living bodies buried deep or blocked, and the radar signal processing module combines two-dimensional FFT, CFAR detection algorithm and the deep learning network PointNet++ to accurately identify the characteristics of living bodies. Combined with the high-precision RTK-GNSS and IMU navigation of the drone body, precise positioning is achieved, and multi-sensor fusion improves the reliability and accuracy of positioning.

[0030] 2. The improved penetration positioning device for the drone and its usage method efficiently process the data of the millimeter-wave radar and the visual acquisition system through the edge server data processing system, and apply algorithms such as the point cloud registration algorithm ICP, voxel filtering, and Kalman filtering to track the position of the living body in real time. The visualization system renders the dynamic point cloud effect or map marker of the living body position in real time, providing intuitive information for rescue personnel, and the gesture control system realizes remote control, improving the rescue efficiency and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic structural diagram of the improved penetration positioning device for the drone of the present invention;

[0032] Figure 2 It is a flowchart of the usage method of the improved penetration positioning device for the drone of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments and drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] Please refer to Figure 1-2 , an improved penetration positioning device for a drone, comprising a drone body, a millimeter-wave radar system, a visual acquisition system, an edge server data processing system and a visualization system:

[0035] The drone body is integrated with a high-precision RTK-GNSS positioning module and an IMU inertial navigation system;

[0036] The millimeter-wave radar system is installed on the UAV body, operates in the 77GHz band, and uses the FMCW modulation method. The millimeter-wave radar system is equipped with a multi-channel MIMO antenna array to improve spatial resolution. The radar system also includes a radar signal processing module. The radar signal processing module processes the radar echo signal using two-dimensional FFT to obtain target distance information, detects targets in complex backgrounds using the CFAR detection algorithm, and combines the deep learning network PointNet++ to perform vital sign feature recognition on the acquired point cloud data.

[0037] The visual acquisition system includes at least one high-resolution camera.

[0038] The edge server data processing system adopts a distributed architecture, including a high-performance computing cluster for processing data from the millimeter-wave radar and the visual acquisition system. It spatially aligns the point cloud data of consecutive frames through the ICP point cloud registration algorithm, performs downsampling optimization using voxel filtering, and combines the Kalman filter target tracking algorithm to achieve real-time tracking of the vital sign position. The processed data is stored in a time-series database and pushed to the visualization system through the WebSocket protocol.

[0039] The visualization system is developed based on Unity and ARKit, can receive data pushed by the edge server, and realizes precise positioning of the wearer through SLAM technology. It calculates the field of view angle in combination with gyroscope data and renders the dynamic point cloud effect of the vital sign position in real time or displays marker points on a 3D map.

[0040] A usage method of an improved UAV penetration positioning device includes the improved UAV penetration positioning device and also includes the following steps:

[0041] S1. Control the UAV to carry the millimeter-wave radar and the visual acquisition system to fly over the disaster area.

[0042] S2. The millimeter-wave radar collects the vital sign data of the vital signs, and at the same time, the visual acquisition system takes pictures of the environment images of the disaster area.

[0043] S3. Transmit the collected data to the edge server data processing system in real time through the 5G network module.

[0044] S4. The edge server data processing system preprocesses, performs vital sign feature recognition, and tracks the position of the data received.

[0045] S5. Push the processed vital sign position information, vital sign data, and UAV position information to the visualization system through the WebSocket protocol.

[0046] S6. The visualization system renders the dynamic point cloud of the life form's position in real time or displays marker points on the 3D map for the rescue personnel to refer to;

[0047] S7. Based on the information from the visualization system, the rescue personnel remotely control the drone through the gesture control system, adjust its flight position and angle, and conduct precise rescue.

[0048] Example 1:

[0049] The present invention relates to an improved penetration positioning device for drones. This device mainly consists of a drone body, a millimeter-wave radar system, a visual acquisition system, an edge server data processing system, and a visualization system. The following is a detailed description of each part:

[0050] The drone body is integrated with a high-precision RTK-GNSS positioning module and an IMU inertial navigation system to ensure the precise positioning and stable flight of the drone in complex environments. The millimeter-wave radar system is installed on the drone body, operates in the 77GHz band, and uses the FMCW modulation method. It has strong penetration ability. This radar system is equipped with a multi-channel MIMO antenna array to improve the spatial resolution, thereby achieving precise detection of target objects. The radar signal processing module uses two-dimensional FFT to process the radar echo signal to obtain the target distance information, and combines the CFAR detection algorithm to detect targets in complex backgrounds. In addition, the deep learning network PointNet++ is introduced to identify the life form features of the acquired point cloud data, improving the accuracy of detection.

[0051] The visual acquisition system includes at least one high-resolution camera, which is stably installed on the gimbal of the drone body to ensure clear images can be captured in different flight postures. These image data provide an important basis for subsequent 3D reconstruction and scene understanding.

[0052] The edge server data processing system adopts a distributed architecture, including a high-performance computing cluster for processing data from the millimeter-wave radar and the visual acquisition system. This system spatially aligns the point cloud data of consecutive frames through the point cloud registration algorithm ICP and uses voxel filtering for downsampling optimization to improve the data processing efficiency. Combining with the target tracking algorithm Kalman filter, this system can achieve real-time tracking of the life form's position. The processed data is stored in the time series database and pushed to the visualization system through the WebSocket protocol.

[0053] The visualization system is developed based on Unity and ARKit. It can receive the data pushed by the edge server, accurately locate the wearer through SLAM technology, calculate the field of view angle in combination with gyroscope data, and can render the dynamic point cloud effect of the life form's position in real time or display the marker points on the 3D map, providing intuitive rescue information for the rescue personnel.

[0054] Embodiment 2:

[0055] The working process of the improved penetration positioning device of the drone of the present invention is as follows:

[0056] First, control the drone to carry the millimeter-wave radar and the visual acquisition system to fly over the disaster area. The millimeter-wave radar collects the vital signs data of the life form, such as life sign signals like breathing and heartbeat. At the same time, the visual acquisition system takes pictures of the environment of the disaster area, and these data are transmitted to the edge server data processing system in real time through the 5G network module.

[0057] The edge server data processing system preprocesses the received data, including operations such as removing low-quality data frames, image classification and sorting. Subsequently, the radar data is processed using the two-dimensional FFT and CFAR detection algorithms to obtain the target distance information. Combining with the deep learning network PointNet++, the life form features are identified from the point cloud data to achieve accurate detection of the life form. At the same time, the point cloud data of consecutive frames is spatially aligned through the point cloud registration algorithm ICP, and voxel filtering is used for downsampling optimization. Combining with the target tracking algorithm Kalman filter, the real-time tracking of the life form's position is realized. The processed data is stored in the time series database and pushed to the visualization system through the WebSocket protocol.

[0058] After the visualization system receives the data pushed by the edge server, according to the position and field of view angle of the wearer, it renders the dynamic point cloud effect of the life form's position in real time or displays the marker points on the 3D map. The rescue personnel can, according to the information provided by the visualization system, remotely control the drone through the gesture control system, adjust its flight position and angle, and conduct precise rescue.

[0059] Embodiment 3:

[0060] The present invention also includes a gesture control system, which can capture the user's gesture actions, perform real-time analysis and classification of the user's gestures through AR Foundation, convert them into drone control instructions, and transmit the instructions to the drone side through Socket communication technology to achieve remote control of the drone.

[0061] The implementation of the gesture control system relies on the cross-platform AR development technology provided by the AR Foundation. First, the system captures the user's gesture actions through the built-in camera or sensors of the Vision Pro, and then uses the AR Foundation to analyze and classify the user's gestures in real time. After recognizing a specific gesture, the system converts the gesture information into corresponding drone control instructions, such as takeoff, landing, forward, backward, turning, etc. These instructions are transmitted to the drone end through Socket communication technology, and the drone executes the corresponding actions after receiving the instructions.

[0062] The application of the gesture control system greatly simplifies the operation process of the drone, enabling rescue personnel to more conveniently control the drone for detection and rescue work. At the same time, combined with the intuitive rescue information provided by the visualization system, rescue personnel can more efficiently locate and rescue trapped personnel.

[0063] Example 4:

[0064] Before the rescue, it is necessary to conduct joint debugging on the drone, millimeter-wave radar system, visual acquisition system, edge server data processing system, and visualization system to ensure the normal data communication and processing process between the systems. During the debugging process, it is necessary to check whether the hardware connections, software configurations, and data communication protocols of each system are correct. At the same time, simulation tests are also required to verify the detection and positioning capabilities of the device in complex environments.

[0065] In practical applications, the improved drone penetration positioning device provided by the present invention can be widely used in complex environments such as collapsed object rescue. Through the penetration ability of the millimeter-wave radar and the feature recognition ability of the deep learning network, the device can achieve precise detection and positioning of buried or blocked living bodies. At the same time, combined with the intuitive rescue information provided by the visualization system, rescue personnel can carry out rescue work more efficiently, improving the rescue efficiency and success rate.

[0066] In summary, for the improved drone penetration positioning device and its usage method, by using the high-frequency signal and FMCW modulation method of the millimeter-wave radar system, strong penetration is achieved to effectively detect buried or blocked living bodies. The radar signal processing module combines two-dimensional FFT, CFAR detection algorithm, and the deep learning network PointNet++, accurately identifies the characteristics of living bodies, and combines the high-precision RTK-GNSS and IMU navigation of the drone body to achieve precise positioning. The multi-sensor fusion improves the reliability and accuracy of positioning.

[0067] Moreover, for the improved penetration positioning device of the drone and its usage method, the edge server data processing system efficiently processes the data from the millimeter-wave radar and the vision acquisition system. By using algorithms such as the Iterative Closest Point (ICP) point cloud registration algorithm, voxel filtering, and Kalman filtering, it can track the position of the living body in real time. The visualization system can render the dynamic point cloud effect or map markers of the living body position in real time, providing intuitive information for rescue personnel. The gesture control system enables remote operation, improving the rescue efficiency and flexibility, and solving the problems of inaccurate positioning and limited detection range of traditional positioning methods in complex environments.

[0068] All relevant modules involved in this system are hardware system modules or functional modules formed by combining computer software programs or protocols in the prior art with hardware. The computer software programs or protocols themselves involved in this functional module are all well-known technologies to those skilled in the art, and they are not the improvements of this system. The improvement of this system lies in the interaction relationship or connection relationship between the modules, that is, the overall structure of the system is improved to solve the corresponding technical problems to be solved by this system.

[0069] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An improved penetration positioning device for an unmanned aerial vehicle, comprising an unmanned aerial vehicle body, a millimeter-wave radar system, a visual acquisition system, an edge server data processing system, and a visualization system, characterized in that: The unmanned aerial vehicle body is integrated with a high-precision RTK-GNSS positioning module and an IMU inertial navigation system; The millimeter-wave radar system is installed on the unmanned aerial vehicle body, operates in the 77GHz band, and uses the FMCW modulation method. The millimeter-wave radar system is equipped with a multi-channel MIMO antenna array for improving spatial resolution; the radar system also includes a radar signal processing module, which uses two-dimensional FFT to process the radar echo signal to obtain target distance information, uses the CFAR detection algorithm to detect targets in a complex background, and combines the deep learning network PointNet++ to perform vital body feature recognition on the acquired point cloud data; The visual acquisition system includes at least one high-resolution camera; The edge server data processing system adopts a distributed architecture and includes a high-performance computing cluster for processing data from the millimeter-wave radar and the visual acquisition system. The point cloud data of continuous frames is spatially aligned through the point cloud registration algorithm ICP, and voxel filtering is used for downsampling optimization. Combining the target tracking algorithm Kalman filter to achieve real-time tracking of the vital body position. The processed data is stored in a time series database and pushed to the visualization system through the WebSocket protocol; The visualization system is developed based on Unity and ARKit, can receive data pushed by the edge server, and realizes precise positioning of the wearer through SLAM technology. Combining gyroscope data to calculate the field of view angle, and real-time rendering of the dynamic point cloud effect of the vital body position or displaying marker points on a three-dimensional map.

2. An improved penetration positioning device for an unmanned aerial vehicle according to claim 1, characterized in that, The millimeter-wave radar system transmits the original data to the edge server data processing system in real time through a 5G network module, and at the same time packages the precise position information of the unmanned aerial vehicle into a standardized data frame and sends it together.

3. An improved penetration positioning device for an unmanned aerial vehicle according to claim 1, characterized in that, The high-resolution camera of the visual acquisition system is stably installed on the gimbal of the unmanned aerial vehicle body to ensure clear images can be captured in different flight postures.

4. An improved penetration positioning device for an unmanned aerial vehicle according to claim 1, characterized in that, The edge server data processing system also includes a data preprocessing module for preprocessing the received radar and visual data, including operations such as removing low-quality data frames, image classification, and sorting.

5. An improved penetration positioning device for an unmanned aerial vehicle according to claim 1, characterized in that, The visualization system also includes a user interaction interface, through which the flight state of the unmanned aerial vehicle can be controlled, data acquisition parameters can be adjusted, and real-time rescue information can be viewed.

6. An improved penetration positioning device for a drone according to claim 1, characterized in that, It also includes a gesture control system, which can capture the user's gesture actions, perform real-time analysis and classification of the user's gestures through AR Foundation, convert them into unmanned aerial vehicle control commands, and transmit the commands to the unmanned aerial vehicle end through Socket communication technology to realize remote control of the unmanned aerial vehicle.

7. A method for using an improved penetration positioning device for an unmanned aerial vehicle, characterized in that, Including the improved penetration positioning device for an unmanned aerial vehicle described in claims 1-6, the following steps are also included: S1. Control the unmanned aerial vehicle equipped with a millimeter-wave radar and a visual acquisition system to fly over the disaster area; S2. The millimeter-wave radar collects the vital sign data of the living body, and at the same time, the visual acquisition system takes pictures of the environment image of the disaster area; S3. Transmit the collected data to the edge server data processing system in real time through the 5G network module; S4. The edge server data processing system preprocesses, identifies the vital signs, and tracks the position of the received data; S5. Push the processed vital body position information, vital sign data, and UAV position information to the visualization system through the WebSocket protocol; S6. The visualization system renders the dynamic point cloud effect of the vital body position in real time or displays the marked points on the 3D map for the reference of rescue personnel; S7. According to the information of the visualization system, the rescue personnel remotely control the UAV through the gesture control system, adjust its flight position and angle, and conduct precise rescue.

8. The usage method of an improved penetration positioning device for an unmanned aerial vehicle according to claim 1, characterized in that, It also includes the step of jointly debugging the UAV, millimeter-wave radar system, visual acquisition system, edge server data processing system, and visualization system before the rescue to ensure the normal data communication and processing process between the systems.

Images