Unmanned aerial vehicle visible light video and radar GMTI combined vehicle detection method

Abstract

The invention relates to the technical field of vehicle target detection, in particular to an unmanned aerial vehicle visible light video and radar GMTI combined vehicle detection method. Comprising the following steps: carrying out comprehensive moving target detection on a whole to-be-scanned area through a GMTI radar; the detected radar GMTI information is stored in a data recorder in the photoelectric radar comprehensive load; data in the data recorder are read, moving vehicle targets in radar GMTI information are automatically screened in real time, and the screened vehicle longitude and latitude height coordinates, the vehicle moving speed and the vehicle moving track are used for predicting the current position of the vehicle; a visible light camera is used for pointing the visual axis of visible light to the current position of the vehicle one by one through geographical coordinate guidance, and moving target detection is conducted on the vehicle; and when the vehicle is detected, a visible light tracking state is switched, and detailed information of the vehicle is confirmed. The method has the advantages that the calculation amount is small, the moving vehicle target in a large area can be quickly searched, and the accuracy and the reliability are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle target detection, and in particular to a vehicle detection method combining unmanned aerial vehicle visible light video and radar GMTI. Background Art

[0002] Traditional methods for detecting drone vehicles mainly rely on single sensor data, such as using only visible light cameras or radar systems for target detection. These methods may be effective under certain conditions, but in complex environments or lighting conditions, such as at night or in bad weather, the detection accuracy and reliability will be seriously affected.

[0003] Visible light cameras can provide high-resolution images, but their performance is limited in low light conditions or when light changes greatly. On the other hand, radar systems, while capable of working in all weather conditions and with good penetration, usually have lower spatial resolution than optical sensors and have limited ability to distinguish stationary targets.

[0004] In order to overcome the limitations of a single sensor and improve the performance of vehicle target detection, it is necessary to combine the advantages of different sensors. Although some existing fusion methods try to combine different sensor data, they often have problems such as low data processing efficiency, poor real-time performance or complex fusion algorithms. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a vehicle detection method combining UAV visible light video and radar GMTI, which specifically includes the following steps: S1. Install the photoelectric radar integrated payload on the UAV; determine the area to be scanned by the GMTI radar, and perform comprehensive moving target detection on the entire area to be scanned by the GMTI radar; store the detected radar GMTI information into a custom data segment in a specified data format, and store it in the data recorder in the photoelectric radar integrated payload; S2. Read the data in the data recorder, automatically screen the moving vehicle targets in the radar GMTI information in real time, and store the selected vehicle latitude and longitude coordinates, vehicle speed and vehicle movement trajectory in the data recorder of the photoelectric radar integrated load; S3. Predict the current position of the vehicle based on the vehicle's longitude and latitude coordinates, vehicle speed and vehicle trajectory; use a visible light camera to point the visible light axis to the current position of the vehicle one by one through the method of geographic coordinate guidance, and detect the moving target of the vehicle; when the vehicle is detected, switch to the visible light tracking state to confirm the detailed information of the vehicle.

[0006] Preferably, the radar GMTI information includes the flight altitude of the UAV, the azimuth angle of the photoelectric radar integrated load, the pitch angle of the photoelectric radar integrated load, the UTC time of the current scan, and the longitude, latitude, altitude, movement speed and movement direction of the moving target.

[0007] Preferably, the screening method in step S2 specifically comprises the following steps: S201 retrieves the road network information of the area and the radar GMTI information in the data recorder; S202. Determine the position of the moving target according to the custom data segment in the radar GMTI information; S203. Calculate the distance between the moving target position and each road in the road network, and determine the road closest to the moving target; determine whether the moving target is located on a road in the road network information according to the road type; S204. Calculate the vehicle's speed during the 30-second continuous scan. V and motion trajectory P ; S205. Determine vehicle movement speed V Is it uniform, movement trajectory P Whether it extends along the road network; if the vehicle movement speed V Uniform and moving trajectory P If it extends along the road network, the target is considered to be a vehicle.

[0008] Preferably, the road type is a rural road, a regular road or a highway; The determination method in step S203 is specifically as follows: Let the position of the moving target be point A ( x 1 , y 1 ), a point on the road is a point B ( x 2 , y 2 ), then the Euclidean distance between two points D Calculated by the following formula: ; if D Less than the preset threshold or , then the moving target is considered to be on the road; the preset threshold for judging whether the vehicle is on the road or Dynamic adjustment is made based on vehicle size and road type. The adjustment principles are as follows: If it is a rural road, or <20 meters; if it is an ordinary road, or <10 meters; if it is a highway, or <5 meters.

[0009] Preferably, the movement speed in step S204 V The calculation formula is as follows: V = ΔPosition / Δt; Among them, ΔPosition is the vehicle’s position in the time interval Δ t Distance moved within Vehicle movement trajectory P Determined by multi-point interpolation or curve fitting method.

[0010] Preferably, the vehicle's trajectory P It is determined by multi-point interpolation; the multi-point interpolation method is as follows: Use multi-point interpolation to fill in the areas where the vehicle is not observed in the GMTI continuous scan; Given a series of vehicle position points {( x 1 , y 1 ),( x 2 , y 2 ), ..., ( x n , y n )}; Connect all vehicle position points to form a trajectory line; Use Lagrange interpolation to calculate the intermediate points on the trajectory line; interpolation polynomial L ( x ) is expressed as: ; in, x and y are the horizontal and vertical coordinates of the vehicle position point, n is the number of known points.

[0011] Preferably, the determination method of step S205 is: Using Standard Deviation Calculate movement speed V Uniformity: ; Where N represents the number of scans of the vehicle within the detection time, T represents the detection time, It is i The vehicle speed of the scan, is the average speed; Calculate motion trajectory P Angle with the road direction ;if Less than the preset threshold , then the motion trajectory is consideredP Extend along the direction of the highway; The calculation formula is as follows: ; in, represents the motion trajectory vector, represents the road direction vector; Preset Threshold Dynamic adjustment is made based on vehicle size and road conditions (i.e. road type, rural road, ordinary road or expressway). The adjustment principle is as follows: If it is a rural road, <40°; if it is an ordinary road, <20°; if it is a highway, <10°.

[0012] Preferably, step S3 specifically includes the following sub-steps: S301. Read the vehicle's latitude and longitude coordinates and the corresponding timestamp t i ; S302. According to the vehicle's movement speed V and motion trajectory P , use the following formula to predict the vehicle at the next time stamp t i+1 Location: ; in, is the average radius of the earth, direction is the unit vector of the direction of vehicle motion; S303. Direct the visual axis of the visible light camera of the airborne optoelectronic turret to the current position of the vehicle, and use the following formula to calculate the pointing angle of the optoelectronic turret: ; ; Among them, A and E are the azimuth and elevation angles that the optoelectronic turret needs to rotate, respectively. and are the latitude and longitude differences between the vehicle's predicted position and the current line of sight, respectively. is the difference between the flying altitude of the UAV and the ground altitude of the vehicle, and distanceToVehicle is the horizontal distance between the vehicle and the optoelectronic turret; S304. Capture the vehicle image using a visible light camera and apply a target detection algorithm to detect moving targets on the vehicle; S305. When a vehicle is detected, the system switches to the visible light tracking state; the photoelectric turret is adjusted to stably track the vehicle.

[0013] Preferably, in step S305, the photoelectric turret is adjusted by the servo system of the photoelectric turret, and the calculation formula is as follows:

[0014] ; in, and are the azimuth and elevation angles that the optoelectronic turret needs to adjust, k p is the proportional control coefficient, and They are the differences between the current azimuth and pitch angles and the actual position angle of the vehicle.

[0015] Compared with the prior art, the present invention can achieve the following beneficial effects: The present invention provides a method for high-efficiency vehicle target detection in a large area by combining visible light video and radar GMTI (Ground Moving Target Indicator) information carried by a drone. The method utilizes the complementarity of optical and radar sensor data, significantly improving the accuracy and reliability of vehicle target detection, and is suitable for a variety of application scenarios such as traffic monitoring and disaster management. First, the radar GMTI is used to detect moving targets in the entire area to be scanned, and the longitude and latitude coordinates of the vehicle are recorded in the data recorder in the photoelectric radar integrated payload. The photoelectric radar integrated payload reads the vehicle target coordinates detected by the radar in the recorder, points the visual axis of the visible camera to the target, uses the visible light camera to adjust the field of view of the visible light camera, and performs secondary confirmation of the target vehicle.

[0016] The method of the present invention processes the visible light video and radar GMTI information of the UAV in real time on the UAV, and runs on the UAV onboard computer, with small computational complexity, and can quickly search for moving vehicle targets in a large area, thereby improving the accuracy and reliability of UAV moving vehicle target detection, and providing more powerful and flexible technical support for the application of UAVs in the fields of traffic monitoring, disaster management, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention provides a flow chart of a vehicle detection method combining UAV visible light video and radar GMTI according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are represented by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, the detailed description thereof will not be repeated.

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0020] The present invention provides a vehicle detection method combining UAV visible light video and radar GMTI, which specifically includes the following steps: S1. Install the photoelectric radar integrated payload on the UAV; determine the area to be scanned by the GMTI radar, perform comprehensive moving target detection on the entire area to be scanned by the GMTI radar, store the detected radar GMTI information into a custom data segment in a specified data format, and store it in the data recorder in the photoelectric radar integrated payload; The optoelectronic radar integrated payload is an integral device that contains radar and optoelectronic sensors; Radar GMTI information includes the flight altitude of the UAV, the azimuth angle of the photoelectric radar integrated load, the pitch angle of the photoelectric radar integrated load, the UTC time of the current scan, and the longitude, latitude, altitude, speed and direction of the moving target; The format of the custom data segment is as shown in Table 1: Table 1 Format of custom data segment

[0021] S2. The onboard computer of the UAV automatically screens the moving vehicle targets in the radar GMTI information in real time by reading the data in the data recorder, and stores the selected vehicle target coordinates, vehicle movement speed and vehicle movement trajectory in the data recorder of the photoelectric radar integrated load; the screening method specifically includes the following steps: S201. Retrieve the highway network information of the area and the radar GMTI information in the data recorder.

[0022] S202. Determine the position of the moving target according to the custom data segment in the radar GMTI information.

[0023] S203. Calculate the distance between the moving target position and each road in the road network, and determine the road closest to the moving target; determine whether the moving target is located on a road in the road network information according to the road type; the road type is a rural road, an ordinary road, or an expressway; the determination method is as follows: Let the position of the moving target be point A ( x 1 , y 1 ), a point on the road is a point B ( x 2 ,y 2 ), then the Euclidean distance between two points D It can be calculated by the following formula: ; if D Less than the preset threshold or , then the moving target is considered to be on the road; the preset threshold for judging whether the vehicle is on the road or Dynamic adjustment is made based on vehicle size and road conditions (i.e. road type, rural road, ordinary road or expressway). The adjustment principle is as follows: If it is a rural road, or <20 meters; if it is an ordinary road, or <10 meters; if it is a highway, or <5 m; The distance calculation adopts the Euclidean distance formula, which is used to calculate the distance between the moving target position and each road in the road network; determine the road with the closest Euclidean distance, query the type of road according to the attributes or number of the road, and determine whether the moving target is located on the road according to the distance.

[0024] S204. Calculate the vehicle's speed during the 30-second continuous scan. V and motion trajectory P ; Movement speed V The calculation formula is as follows: V = ΔPosition / Δt; Among them, ΔPosition is the vehicle’s position in the time interval Δ t Distance moved within Determine the trajectory by connecting the position points of the vehicle in consecutive scans P Specifically, the vehicle trajectory is determined by using multi-point interpolation or curve fitting methods. P ; The multi-point interpolation method is as follows: Use multi-point interpolation to fill in the areas where the vehicle is not observed in the GMTI continuous scan; Given a series of vehicle position points {( x 1 , y 1 ),( x 2 , y 2 ),...,( x n , y n )}; Connect all vehicle position points to form a trajectory line; Use Lagrange interpolation to calculate the intermediate points on the trajectory line; interpolation polynomial L (x ) is expressed as: ; in, x and y are the horizontal and vertical coordinates of the vehicle position point, n is the number of known points.

[0025] S205. Determine vehicle movement speed V Is it uniform, movement trajectory P Whether it extends along the road network; if the vehicle movement speed V Uniform and moving trajectory P If the target extends along the road network, it is considered to be a vehicle; the judgment method is: Using Standard Deviation Calculate movement speed V Uniformity: ; Where N represents the number of scans of the vehicle within the detection time, T represents the detection time, It is i The vehicle speed of the scan, is the average speed; Calculate motion trajectory P Angle with the road direction ;if Less than the preset threshold , then the motion trajectory is considered P Extend along the direction of the highway; The calculation formula is as follows: ; in, represents the motion trajectory vector, represents the road direction vector; Preset Threshold Dynamic adjustment is made based on vehicle size and road conditions (i.e. road type, rural road, ordinary road or expressway). The adjustment principle is as follows: If it is a rural road, <40°; if it is an ordinary road, <20°; if it is a highway, <10°.

[0026] S3. Predict the current position of the vehicle based on the longitude and latitude coordinates, the speed and trajectory of the vehicle; use a visible light camera to point the visual axis of the visible light to the current position of the vehicle one by one through the method of geographic coordinate guidance, and detect the moving target of the vehicle; when the vehicle is detected, switch to the visible light tracking state to confirm the detailed information of the vehicle; specifically, the following sub-steps are included: S301. Read the vehicle's latitude and longitude coordinates and the corresponding timestamp t i ; S302. According to the vehicle's movement speed V and motion trajectory P , use the following formula to predict the vehicle at the next time stamp t i+1 Location: ; in, is the average radius of the earth, direction is the unit vector of the direction of vehicle motion; S303. Direct the visual axis of the visible light camera of the airborne optoelectronic turret to the current position of the vehicle, and use the following formula to calculate the pointing angle of the optoelectronic turret: ; ; Among them, A and E are the azimuth and elevation angles that the optoelectronic turret needs to rotate, respectively. and are the latitude and longitude differences between the vehicle's predicted position and the current line of sight, respectively. is the difference between the flying altitude of the UAV and the ground altitude of the vehicle, and distanceToVehicle is the horizontal distance between the vehicle and the optoelectronic turret; S304. Capture the vehicle image using a visible light camera and apply a target detection algorithm to detect moving targets on the vehicle; S305. When a vehicle is detected, the visible light tracking state is switched; the servo system of the photoelectric turret uses the following formula to adjust the photoelectric turret to stably track the vehicle:

[0027] ; in, and are the azimuth and elevation angles that the optoelectronic turret needs to adjust, k p is the proportional control coefficient, and The azimuth and pitch angles are the difference between the current pointing angle and the actual position angle of the vehicle. Use proportional control or other control algorithms to adjust the azimuth and pitch angles of the turret to accurately point to the predicted position of the vehicle.

[0028] In summary, the present invention performs vehicle target detection by comprehensively utilizing the data of optical and radar GMTI, and processes the images of various sensors of the drone in real time on board. It has a small amount of computation and runs on the drone's onboard computer. It can quickly search for vehicle targets in a large area, thereby improving the accuracy and reliability of vehicle target detection, and providing more powerful and flexible technical support for the application of drones in the fields of traffic monitoring, disaster management, etc.

[0029] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0030] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle detection method combining UAV visible light video and radar GMTI, characterized in that: The specific steps include: S1. Install the photoelectric radar integrated payload on the UAV; determine the area to be scanned by the GMTI radar, and perform comprehensive moving target detection on the entire area to be scanned by the GMTI radar; store the detected radar GMTI information into a custom data segment in a specified data format, and store it in the data recorder in the photoelectric radar integrated payload; S2. Read the data in the data recorder, automatically screen the moving vehicle targets in the radar GMTI information in real time, and store the selected vehicle latitude and longitude coordinates, vehicle speed and vehicle movement trajectory in the data recorder of the photoelectric radar integrated load; S3. Predict the current position of the vehicle based on the vehicle's longitude and latitude coordinates, vehicle speed and vehicle trajectory; use a visible light camera to point the visible light axis to the current position of the vehicle one by one through the method of geographic coordinate guidance, and detect the moving target of the vehicle; when the vehicle is detected, switch to the visible light tracking state to confirm the detailed information of the vehicle.

2. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 1 is characterized by: The radar GMTI information includes the UAV flight altitude, the azimuth angle of the photoelectric radar integrated load, the pitch angle of the photoelectric radar integrated load, the UTC time of the current scan, and the longitude, latitude, altitude, movement speed and movement direction of the moving target.

3. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 1 is characterized by: The screening method in step S2 specifically comprises the following steps: S201 retrieves the road network information of the area and the radar GMTI information in the data recorder; S202. Determine the position of the moving target according to the custom data segment in the radar GMTI information; S203. Calculate the distance between the moving target position and each road in the road network, and determine the road closest to the moving target; Judging whether the moving target is located on a highway in the highway network information according to the highway type; S204. Calculate the vehicle's speed during the 30-second continuous scan. V and motion trajectory P ; S205. Determine vehicle movement speed V Is it uniform, movement trajectory P Whether it extends along the road network; If the vehicle's speed V Uniform and moving trajectory P If it extends along the road network, the target is considered to be a vehicle.

4. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 3 is characterized by: The type of road is a rural road, a regular road or an expressway; The determination method in step S203 is specifically as follows: Let the position of the moving target be point A ( x 1, y 1) A point on the road is a point B ( x 2, y 2), then the Euclidean distance between two points is D Calculated by the following formula: ； if D Less than the preset threshold η , then the moving target is considered to be on the road; the preset threshold for judging whether the vehicle is on the road η Dynamic adjustment is made based on vehicle size and road type. The adjustment principles are as follows: If it is a rural road, η <20 meters; if it is an ordinary road, η <10 meters; if it is a highway, η <5 meters.

5. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 4 is characterized by: The movement speed in step S204 V The calculation formula is as follows: V = ΔPosition / Δt; Among them, ΔPosition is the vehicle’s position in the time interval Δ t Distance moved within Vehicle trajectory P Determined by multi-point interpolation or curve fitting method.

6. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 5 is characterized by: Vehicle trajectory P It is determined by multi-point interpolation; the multi-point interpolation method is as follows: Use multi-point interpolation to fill in the areas where the vehicle is not observed in the GMTI continuous scan; Given a series of vehicle position points {( x 1, y 1), ( x 2, y 2), ..., ( x n , y n )}; Connect all vehicle position points to form a trajectory line; Use Lagrange interpolation to calculate the intermediate points on the trajectory line; interpolation polynomial L ( x ) is expressed as: ； in, x and y are the horizontal and vertical coordinates of the vehicle position point, n is the number of known points.

7. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 6 is characterized by: The determination method of step S205 is: Using Standard Deviation Calculate movement speed V Uniformity: ； Where N represents the number of scans of the vehicle within the detection time, T represents the detection time, It is i The vehicle speed of the scan, is the average speed; Calculate motion trajectory P Angle with the road direction ;if Less than the preset threshold , then the motion trajectory is considered P Extend along the direction of the highway; The calculation formula is as follows: ； in, represents the motion trajectory vector, represents the road direction vector; Preset Threshold Dynamic adjustment is made based on vehicle size and road conditions (i.e. road type, rural road, ordinary road or expressway). The adjustment principle is as follows: If it is a rural road, <40°; if it is an ordinary road, <20°; if it is a highway, <10°.

8. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 1 is characterized by: The step S3 specifically includes the following sub-steps: S301. Read the vehicle's latitude and longitude coordinates and the corresponding timestamp t i ; S302. According to the vehicle's movement speed V and motion trajectory P , use the following formula to predict the vehicle at the next time stamp t i+1 Location: ； in, is the average radius of the earth, direction is the unit vector of the direction of vehicle motion; S303. Direct the visual axis of the visible light camera of the airborne optoelectronic turret to the current position of the vehicle, and use the following formula to calculate the pointing angle of the optoelectronic turret: ； ； Among them, A and E are the azimuth and elevation angles that the optoelectronic turret needs to rotate, respectively. and are the latitude and longitude differences between the vehicle's predicted position and the current line of sight, respectively. is the difference between the flying altitude of the UAV and the ground altitude of the vehicle, and distanceToVehicle is the horizontal distance between the vehicle and the optoelectronic turret; S304. Capture the vehicle image using a visible light camera and apply a target detection algorithm to detect moving targets on the vehicle; S305. When a vehicle is detected, the system switches to the visible light tracking state; the photoelectric turret is adjusted to stably track the vehicle.

9. The vehicle detection method combining UAV visible light video and radar GMTI according to claim 8 is characterized by: In step S305, the photoelectric turret is adjusted by the servo system of the photoelectric turret, and the calculation formula is as follows: ； in, and are the azimuth and elevation angles that the optoelectronic turret needs to adjust, k p is the proportional control coefficient, and They are the differences between the current azimuth and pitch angles and the actual position angle of the vehicle.

Images