A shape-preserving fusion method for 3D obstacle data based on comprehensive vision

Abstract

The present invention belongs to the field of airborne optoelectronic reconnaissance and situational awareness, and discloses a conformal fusion method for three-dimensional obstacle data based on comprehensive vision, including the following steps: non-periodic communication reception of obstacle data; obstacle classification; analysis of obstacle data of different categories; generation of layers of comprehensive vision scenes; conformal fusion of three-dimensional models based on data analysis results. The present invention uses relevant technical means to achieve superimposition and fusion of conformal obstacle models in a three-dimensional geographic information system, and can provide three-dimensional reconstruction and identification of obstacles in a conformal manner; thereby providing a complete and accurate comprehensive situational awareness capability.

Description

Technical Field

[0001] The invention belongs to the field of airborne optoelectronic reconnaissance and situation awareness, and relates to a three-dimensional obstacle data conformal fusion method based on comprehensive vision. Background Art

[0002] The low-altitude collision avoidance radar for helicopters can provide obstacle detection capabilities including power lines, and can detect the latest obstacles outside the carrier in real time. The obstacle data obtained based on surveying and mapping technology can reflect the static obstacles in the area. The combination of static data and dynamic real-time detection equipment provides a basis for helicopters to have low-altitude obstacle avoidance and collision avoidance capabilities.

[0003] In order to effectively utilize the real-time external obstacle detection equipment and the static obstacle data prepared in advance, combined with the spatial situation display capability of the three-dimensional geographic information system, and improve the pilot's perception of the external environment and obstacles, it is necessary to establish a standard method for the conformal fusion of three-dimensional obstacle data of the comprehensive vision. Summary of the invention

[0004] (I) Purpose of the invention

[0005] The purpose of the present invention is to provide a three-dimensional obstacle data conformal fusion method based on comprehensive vision, which can perform standardized rapid conformal fusion of dynamic real-time radar obstacle data with pre-prepared static data in a three-dimensional geographic information system, and provide pilots with better situational awareness.

[0006] (II) Technical solution

[0007] In order to improve the ability to detect external obstacles and enhance the low-altitude obstacle avoidance capability of helicopters, the present invention proposes a method for conformal fusion of radar real-time obstacle fusion and static obstacle data in a three-dimensional comprehensive vision, which provides a more reliable obstacle display capability through the complementary characterization of external obstacles by the two data.

[0008] The method mainly includes the following parts: first, establish a non-periodic communication receiving process for obstacle data to receive various types of obstacle data reported by radar equipment in real time; second, classify the received obstacles according to the communication protocol, including vertical obstacles, horizontal obstacles, and multi-dimensional obstacles; third, further analyze the corresponding attribute data according to different types of obstacles; fourth, generate a comprehensive visual scene based on terrain data at the same time; fifth, conformally fuse the pre-designed three-dimensional model based on the data analysis results.

[0009] (III) Beneficial effects

[0010] According to the method provided by the invention, it can be designed as a software function, which can be used to achieve high-confidence loading and drawing of external obstacle data in the integrated vision system, thereby assisting in improving the external obstacle avoidance capability of the carrier aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the process composition of the method in the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.

[0013] like Figure 1 As shown, the carrier navigation and positioning method of the embodiment of the present invention includes the following steps: first, establishing a non-periodic communication receiving process for obstacle data, and receiving various types of obstacle data reported by the radar equipment in real time; second, classifying the received obstacles according to the communication protocol, including vertical obstacles, horizontal obstacles, and multi-dimensional obstacles; third, further parsing the corresponding attribute data according to different types of obstacles; fourth, generating a comprehensive visual scene based on the terrain data at the same time; fifth, conformally fusing the pre-designed three-dimensional model based on the data analysis results.

[0014] The following is a detailed description of each step in the above process:

[0015] S1: Establishing the non-periodic communication receiving process of obstacle data

[0016] Establish a software process for communicating with the radar device, encapsulate the Ethernet communication protocol, monitor the obstacle data reporting thread from the radar device in real time, and receive it in a non-periodic communication mode triggered by real-time obstacle data.

[0017] S2: Obstacle Data Classification

[0018] According to the radar data protocol, the received data is classified, mainly including vertical obstacles, horizontal obstacles, and multi-dimensional obstacles.

[0019] The data that needs to be analyzed for vertical obstacles include obstacle location, expressed in latitude and longitude; obstacle width; obstacle height; obstacle confidence;

[0020] The data that needs to be analyzed for horizontal obstacles include obstacle location, expressed as starting point longitude and latitude and longitude of the end point; obstacle height; obstacle confidence;

[0021] The data to be analyzed for multi-dimensional obstacles include the latitude and longitude of multiple locations; obstacle height; obstacle confidence;

[0022] S3: Analysis of different types of obstacle data

[0023] Parse obstacle attribute data, including:

[0024] The number of vertical obstacles is recorded as Num_ver. For each Num_ver vertical obstacle, its location is analyzed, including longitude and latitude, recorded as Lon_i, Lat_i, and height, recorded as Hei_i, and confidence level, recorded as Con_i.

[0025] The number of horizontal obstacles is recorded as Num_hor. For Num_hor horizontal obstacles, their starting and ending positions are analyzed respectively, including longitude and latitude, recorded as Lon_start_i, Lat_start_i, Lon_end_i, Lat_end_i; and height, recorded as Hei_hor_i; confidence, recorded as Con_hor_i.

[0026] The number of multi-dimensional obstacles is recorded as Num_mul. For Num_mul multi-dimensional obstacles, the positions of multiple corner points are analyzed respectively, including longitude and latitude, recorded as Lon_dim_i_j, Lat_dim_i_j, and height, recorded as Hei_mul_i, and confidence, recorded as Con_mul_i.

[0027] S4: 3D scene generation

[0028] The posture parameters of the carrier mainly include the position parameters and attitude parameters of the carrier. The position parameters include longitude, latitude, and altitude, which are recorded as L, B, and H respectively. The position data is based on the geographic coordinate system, and the longitude and latitude units are degrees. The attitude parameters include heading angle, pitch angle, and roll angle, which are recorded as a, p, and r respectively, and the unit is degrees. The angle is based on the northeast sky coordinate system. The electro-optical sight line attitude data includes the pitch angle and azimuth angle of the sight line, which are recorded as al os ,pl os , the angle is based on the carrier coordinate system.

[0029] A total of 8 data including the position, attitude, and aiming line attitude of the carrier aircraft are obtained as input for the subsequent dynamic continuous synthetic visual image generation step.

[0030] The spatial position transformation matrix is ​​denoted as M pos , the posture matrix M pos The following calculation process is used:

[0031]

[0032] Where n, u, v are basis vectors in the local transformation coordinate system, nx, ny, nz are the x, y, z components of vector n, ux, uy, uz are the x, y, z components of vector u, and vx, vy, (vz are the x, y, z components of vector v), which are calculated using the following formula:

[0033] n=(cosLcosB,sinLcosB,sinB)

[0034] vpx is the x-component of the carrier position vp in geocentric coordinates, vpy is the y-component of the carrier position vp in geocentric coordinates, and vpz is the z-component of the carrier position vp in geocentric coordinates. The calculation is based on the following formula:

[0035] vpx=(N+H)cosBcosL

[0036] vpy=(N+H)cosBsinL

[0037] vpz=[(N(1-e 2 )+H]sinB

[0038] Where L, B are the longitude and latitude of each frame of the aircraft position data collected in the above steps, N is the radius of the circle, e 2 is the first eccentricity, and the following calculation formulas are used:

[0039]

[0040]

[0041] In the above formula, a and c are the long radius and short radius of the earth ellipsoid model respectively.

[0042] a=6378137.0m

[0043] c=6356752.3142m.

[0044] The spatial attitude transformation matrix is ​​denoted as M atti

[0045] Posture matrix M atti The following calculation process is used. First, a quaternion is constructed based on the carrier's attitude data, denoted as q:

[0046]

[0047] Wherein, a, p, and r are respectively the heading angle, pitch angle, and roll angle of the carrier aircraft acquired in the above steps;

[0048]

[0049] Based on the terrain data of the geographical area where the aircraft is located, including elevation data and satellite texture image data, a three-dimensional geographic scene SCENE of the area is generated. stategraph ; Take the three-dimensional geographic scene SCENE of the area stategraph As input, the aircraft posture data M collected in the S1 stage pos , Matti , aiming line attitude M los is the input, where M los The line-of-sight space transformation matrix constructed for the line-of-sight attitude data is calculated to obtain the composite transformation matrix M composite , by the composite matrix M composite Driver-generated 3D static scene:

[0050] M composite =M los *M atti *M pos

[0051] A dynamic and continuous synthetic visual image can be generated, where the image of a certain frame is recorded as f svs (x,y,z,t)

[0052] S5: Conformal fusion of obstacles of various categories:

[0053] Based on the location and height attribute data of various types of obstacles obtained by the S3 stage analysis, the 3D models of different obstacles are superimposed on the 3D scene generated in the S5 stage:

[0054] For vertical obstacles, based on the number of Num_ver, the spatial horizontal 3D model with a fusion height of Hei_i is superimposed in the position (Lon_i, Lat_i). For this model, it is necessary to solve the transformation matrix converted to the world coordinates, recorded as Mat_local_to_world. The solution process of this matrix refers to the spatial position transformation matrix M in the S4 stage. pos , and then the nodes of the model are added to the transformation matrix to be conformally superimposed on the 3D GIS;

[0055] For horizontal obstacles, based on the number of Num_ver, the midpoint position (Lon_mid_i, Lat_mid_i) is calculated based on the starting position (Lon_start_i, Lat_start_i) and the end position (Lon_end_i, Lat_end_i), and the spatial horizontal three-dimensional model is superimposed and fused based on the midpoint position and height Hei_i. For this model, it is also necessary to solve the transformation matrix converted to the world coordinates, recorded as Mat_local_to_world. The solution process of this matrix refers to the spatial position conversion matrix M in the S4 stage pos , and then the nodes of the model are added to the transformation matrix to be conformally superimposed on the 3D GIS;

[0056] For multi-dimensional obstacles, based on the number of Num_ver, the centroid or midpoint position (Lon_mid_i, Lat_mid_i) is calculated based on the position of each corner point (Lon_dim_i_j, Lat_dim_i_j), and the spatial horizontal three-dimensional model is superimposed and fused based on the midpoint position and height Hei_i. For this model, it is also necessary to solve the transformation matrix converted to the world coordinates, recorded as Mat_local_to_world. The solution process of this matrix refers to the spatial position conversion matrix M in the S4 stage. pos , and then the nodes of the model are added to the transformation matrix to be conformally superimposed on the 3D GIS;

[0057] It can be seen from the above technical solutions that the present invention utilizes the spatial description capability of the geographic information system for large-scale three-dimensional scenes, and conformally superimposes and fuses the obstacle data detected by the radar in real time into the three-dimensional geographic space based on its geographic spatial position, thereby integrating real-time obstacles in the static three-dimensional geographic information space, and providing a more accurate and comprehensive three-dimensional situational awareness capability. This method combines the achievements in the field of surveying and mapping and information fusion processing, and implements a new obstacle fusion conformal fusion method in software. It has strong engineering application significance for airborne avionics systems, and can improve the target reconnaissance capability and multi-target reconnaissance capability of helicopters. Its tactical significance is worth further exploration to enhance the battlefield survivability of helicopters.

[0058] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A three-dimensional obstacle data shape-preserving fusion method based on comprehensive vision, characterized in that: The following steps are involved: S1: Obstacle data aperiodic communication reception; S2: Disorder classification; S3: Data analysis of different categories of obstacles; S4: Comprehensive visual scene generation; S5: Conformal fusion of obstacles of different categories to generate a 3D model; In the step S2, the received obstacle data is classified according to the radar data protocol, including vertical obstacles, horizontal obstacles, and multi-dimensional obstacles; The data that needs to be parsed for vertical obstacles include: obstacle location, expressed in longitude and latitude; obstacle width; obstacle height; obstacle confidence; the data that needs to be parsed for horizontal obstacles include: obstacle location, expressed in longitude and latitude of the starting point and the end point; obstacle height; obstacle confidence; the data that needs to be parsed for multi-dimensional obstacles include: longitude and latitude of multiple locations; obstacle height; obstacle confidence; In step S3, the obstacle attribute data analyzed includes the number of vertical obstacles, the number of horizontal obstacles, and the number of multi-dimensional obstacles; The number of vertical obstacles is recorded as Num_ver; for Num_ver vertical obstacles, their positions are analyzed respectively, including longitude and latitude, recorded as Lon_i, Lat_i; and height, recorded as Hei_i; and confidence, recorded as Con_i; The number of horizontal obstacles is recorded as Num_hor; for Num_hor horizontal obstacles, the starting and ending positions are analyzed respectively, including longitude and latitude, recorded as Lon_start_i, Lat_start_i, Lon_end_i, Lat_end_i; and height, recorded as Hei_hor_i; confidence, recorded as Con_hor_i; The number of multi-dimensional obstacles is recorded as Num_mul; for Num_mul multi-dimensional obstacles, the positions of multiple corner points thereof are analyzed respectively, including longitude and latitude, recorded as Lon_dim_i_j, Lat_dim_i_j; and height, recorded as Hei_mul_i; and confidence, recorded as Con_mul_i; In step S5, each type of obstacle is subjected to shape-preserving fusion: Based on the location and height of various types of obstacles obtained by the S3 analysis, the 3D models of different obstacles are superimposed on the 3D scene generated in the S4 stage: For vertical obstacles, based on the number of Num_ver, the spatial horizontal three-dimensional model with a fusion height of Hei_i is superimposed in the position (Lon_i, Lat_i). For the model, the transformation matrix converted to the world coordinate is solved, recorded as Mat_local_to_world, and then the nodes of the model are added to the transformation matrix to be conformally superimposed on the three-dimensional geographic information system; For horizontal obstacles, based on the number of Num_ver, the midpoint position (Lon_mid_i, Lat_mid_i) is calculated based on the starting position (Lon_start_i, Lat_start_i) and the end position (Lon_end_i, Lat_end_i), and the spatial horizontal three-dimensional model is superimposed and fused based on the midpoint position and the height Hei_i. For the model, the transformation matrix converted to the world coordinate is solved, recorded as Mat_local_to_world, and then the nodes of the model are added to the transformation matrix to be conformally superimposed on the three-dimensional geographic information system; For multi-dimensional obstacles, based on the number of Num_ver, the centroid or midpoint position (Lon_mid_i, Lat_mid_i) is calculated based on the position of each corner point (Lon_dim_i_j, Lat_dim_i_j), and the spatial horizontal three-dimensional model is superimposed and fused based on the midpoint position and height Hei_i. For this model, the transformation matrix converted to the world coordinate is solved, recorded as Mat_local_to_world. The solution process of this matrix refers to the spatial position transformation matrix M in the S4 stage pos , and then the nodes of the model are added to the transformation matrix to be conformally overlaid on the 3D GIS.

2. The method for shape-preserving fusion of three-dimensional obstacle data based on comprehensive vision according to claim 1, characterized in that: In step S1, the process of receiving obstacle data via non-periodic communication is as follows: establishing an obstacle data non-periodic communication receiving process, establishing a software process for communicating with the radar device, encapsulating the Ethernet communication protocol, monitoring the obstacle data reporting thread from the radar device in real time, and receiving obstacle data in an aperiodic communication mode triggered by real-time obstacle data.

3. The method for shape-preserving fusion of three-dimensional obstacle data based on comprehensive vision according to claim 2, characterized in that: In step S4, a three-dimensional synthetic visual scene is generated.

4. The method for shape-preserving fusion of three-dimensional obstacle data based on comprehensive vision according to claim 3, characterized in that: In step S4, the process of generating a three-dimensional synthetic visual scene is as follows: Calculate the spatial position transformation matrix, which is denoted as M pos : Where n, u, v are basis vectors in the transformed coordinate system, nx, ny, nz are the x, y, z components of vector n, ux, uy, uz are the x, y, z components of vector u, vx, vy, vz are the x, y, z components of vector v, respectively. The calculation is done using the following formula: n=(cos L cos B,sin L cos B,sin B) vpx is the x-component of the carrier position vp in geocentric coordinates, vpy is the y-component of the carrier position vp in geocentric coordinates, and vpz is the z-component of the carrier position vp in geocentric coordinates. The calculation is based on the following formula: vpx=(N+H)cos B cos L vpy=(N+H)cos B sin L vpz=[(N(1-e 2 )+H]sin B Where L, B are the longitude and latitude of each frame of the aircraft position data collected in the above steps, N is the radius of the circle, e 2 is the first eccentricity, and the following calculation formulas are used: In the above formula, a and c are the long radius and short radius of the earth ellipsoid model respectively. a=6378137.0m c=6356752.3142m The spatial attitude transformation matrix is ​​denoted as M atti The calculation process is: First, construct a quaternion based on the carrier's attitude data, denoted as q: Wherein, a, p, and r are respectively the heading angle, pitch angle, and roll angle of the carrier aircraft acquired in the above steps;

Images