A method for fusion of optoelectronic video and radar GMTI

Abstract

The present invention relates to a method for fusing optoelectronic video and radar GMTI, which is applied to the optical and radar collaborative search and reconnaissance working mode under optoelectronic radar integrated reconnaissance equipment. The technical scheme includes the following steps: using radar to perform GMTI search to generate tracks, collecting optical videos, obtaining the center distance of the video field of view through a laser rangefinder, and obtaining the frame angle information of the integrated navigation and optoelectronic radar integrated reconnaissance equipment; establishing a conversion model between the image coordinate system and the geographic coordinate system; solving the geographic coordinates of the GMTI track into image coordinates, and marking the track number, geographic location, distance, speed magnitude, speed direction and other information on the video. This method organically combines optical reconnaissance with radar reconnaissance, giving full play to the advantages of both, improving the readability of GMTI reconnaissance results, and improving reconnaissance efficiency.

Description

Technical Field

[0001] The invention relates to a method for fusing photoelectric video and radar GMTI, and belongs to the technical field of unmanned aerial vehicle photoelectric radar reconnaissance. Background Art

[0002] In order to improve reconnaissance capabilities, drones are generally equipped with both optoelectronic imaging systems and radar imaging systems. The two reconnaissance systems operate independently to achieve different functions.

[0003] Optical systems generally integrate high-definition visible light cameras, infrared thermal imagers, lasers and other payloads, and add a stable platform and image processing system to achieve day and night reconnaissance of targets. Lasers are mainly used to measure the distance and illuminate the target, thereby achieving functions such as accurate target positioning, target indication, and guided strikes. Radar imaging systems generally use synthetic aperture radar imaging, which is a new type of microwave imaging technology with the advantages of long distance, all-day, and all-weather, making up for the shortcomings of traditional optical images that are easily affected by visibility. Radar can perform strip imaging, beam imaging, and ground moving target indication (GMTI). Among them, ground moving target indication generally generates point tracks and tracks, which contain valid data such as number, target longitude, target latitude, target distance, target speed direction, and target speed size, which can usually be displayed on a map. Optoelectronic radar reconnaissance equipment integrates the above payloads together to achieve a significant improvement in reconnaissance capabilities.

[0004] The video output by the optical system has the advantages of being fast and intuitive, but it is greatly affected by visibility. When the radar system detects ground moving targets, visibility has little effect on its working performance, but the output GMTI track has poor readability and cannot provide operators with intuitive reconnaissance results. Summary of the invention

[0005] The technical problem solved by the present invention is: to overcome the deficiencies of the prior art, to propose a method for fusing optoelectronic video with radar GMTI, to organically combine the optical reconnaissance system with the radar reconnaissance system, and to improve the readability of GMTI reconnaissance results.

[0006] The solution of the present invention is:

[0007] A method for fusing optoelectronic video and radar GMTI, comprising:

[0008] Based on the photoelectric radar integrated reconnaissance equipment, the radar is used to search for GMTI and generate tracks, collect optical videos, obtain the center distance of the video field of view through the laser rangefinder, and obtain the combined navigation information and the frame angle information of the photoelectric radar integrated reconnaissance equipment;

[0009] Establish a conversion model between the image coordinate system and the geographic coordinate system;

[0010] The geographic coordinates of the GMTI track are resolved into image coordinates, and the track number, geographic location, distance, speed and direction information are marked on the video to achieve the fusion of the GMTI track and optical video.

[0011] Furthermore, the optoelectronic radar integrated reconnaissance equipment is used to realize the simultaneous loading of the optical system and the radar system, control the imaging area through a common servo system, and measure the distance from the center of the field of view to the equipment in real time through the loaded laser rangefinder; through the loaded combined navigation system, the radar is convenient for GMTI target positioning search and optical positioning.

[0012] Furthermore, the image coordinate system is the coordinate system of the output image after optical imaging, with the upper left corner of the image as the origin and positive values ​​to the right and downward; the geographic coordinate system is the earth's coordinate system, expressed in longitude and latitude, and each track contains the geographic location, movement speed direction and size.

[0013] Furthermore, the conversion model is determined based on the ranging distance, pixel size, focal length value, and positioning information, and the geographic coordinates of the GMTI track are converted into image coordinates, thereby achieving data fusion.

[0014] Furthermore, during optical positioning, the parallelism between the laser optical axis and the visible light optical axis is ≤0.2 mrad.

[0015] Furthermore, when the radar performs GMTI target positioning search, the parallelism between its beam direction and visible light is less than 0.5°.

[0016] Furthermore, the model conversion process between the image coordinate system and the geographic coordinate system is:

[0017] (1) Locate the center of the image and determine its geographical location;

[0018] (2) Perform virtual ortho transformation on the original image;

[0019] (3) According to the laser ranging distance L, the current focal length value f, and the pixel size s information, the geographic coordinates (lon, lat) are transformed into orthophotod image coordinates;

[0020] (4) Obtain the inverse M based on the orthogonal transformation obtained in step (2) -1 , acting on the image coordinates (ox, oy) after orthophotography, the original image coordinates P = (ox1, oy1) are calculated.

[0021] Furthermore, in step (1), the center of the image is located. Assuming that the azimuth angle of the photoelectric radar integrated reconnaissance equipment is α, the pitch angle is β, the laser ranging distance is L, the coordinate system of the photoelectric radar integrated reconnaissance equipment is a right-handed coordinate system, the origin is the intersection of the azimuth axis and the pitch axis, the X axis points to the right side of the aircraft nose direction, and the Y axis points to the aircraft nose direction, then the position of the target in the equipment coordinate system is:

[0022]

[0023] The device's geographic location is included in the combined navigation information, with longitude lon, latitude lat, altitude alt, roll angle γ, pitch angle θ, heading angle The device coordinate system to the geographic coordinate system rotates around the roll angle -γ, the pitch angle -θ, and the heading angle The corresponding rotation matrices are A4, A5, and A6, so the position of the target in the origin coordinate system is A6A5A4P0, and the rotation matrix from the geographic coordinate system to the earth coordinate system is

[0024]

[0025] The translation matrix from the geographic coordinate system with the device coordinate system as the origin to the earth coordinate system is A8, whose components

[0026]

[0027] Where R n is the radius of curvature of the cycloid corresponding to latitude lat, and e is the flattening of the earth;

[0028] Therefore, the position of the target in the earth's rectangular coordinate system is:

[0029]

[0030] Furthermore, in step (2), the transformation matrix is Where R represents the rotation matrix, T represents the translation vector, P represents the perspective matrix, and S represents the scaling factor.

[0031] Furthermore, in step (3), let the width of the transformed image be w, the height be h, the radius of the earth be R, the geographic coordinates of the center point of the image be (cX, cY), the geographic coordinates of the track be (locX, locY), and the orthogonal image coordinates (ox, oy) satisfy:

[0032]

[0033]

[0034] Where:

[0035] Intermediate variables

[0036] Intermediate variables

[0037] Intermediate variables

[0038] Furthermore, during positioning, the parallelism between the laser optical axis and the visible light optical axis is required to be maintained within 0.2 mrad.

[0039] The beneficial effects of the present invention compared with the prior art are:

[0040] (1) The present invention establishes a conversion relationship from the geographic coordinate system to the image coordinate system, fuses the GMTI reconnaissance results with the real-time collected images, thereby improving the readability of the GMTI reconnaissance results and facilitating operators to make judgments;

[0041] (2) The present invention gives full play to the advantages of the optical reconnaissance system and the radar reconnaissance system, organically combines them, and improves the reconnaissance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is the basic flow chart for realizing the fusion of optoelectronic video and radar GMTI in the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the embodiments.

[0044] like Figure 1 As shown, a method for fusing optoelectronic video and radar GMTI includes:

[0045] Based on the photoelectric radar integrated reconnaissance equipment, the radar is used to search for GMTI and generate tracks, collect optical videos, obtain the center distance of the video field of view through the laser rangefinder, and obtain the combined navigation information and the frame angle information of the photoelectric radar integrated reconnaissance equipment;

[0046] Establish a conversion model between the image coordinate system and the geographic coordinate system;

[0047] The geographic coordinates of the GMTI track are resolved into image coordinates, and the track number, geographic location, distance, speed and direction information are marked on the video to achieve the fusion of the GMTI track and optical video.

[0048] The optoelectronic radar integrated reconnaissance equipment is used to realize the simultaneous loading of the optical system and the radar system, control the imaging area through a shared servo system, and measure the distance from the center of the field of view to the equipment in real time through the installed laser rangefinder; the installed combined navigation system facilitates the radar to perform GMTI target positioning search and optical positioning.

[0049] The image coordinate system is the coordinate system of the output image after optical imaging. The upper left corner of the image is the origin, and the rightward and downward directions are positive. The geographic coordinate system is the earth's coordinate system, expressed in longitude and latitude. Each track contains the geographic location, movement speed direction and size.

[0050] The conversion model is determined according to the ranging distance, pixel size, focal length value, and positioning information, and the geographic coordinates of the GMTI track are converted into image coordinates to achieve data fusion.

[0051] When positioning the optics, the parallelism between the laser optical axis and the visible light optical axis is ≤0.2mrad.

[0052] When the radar performs GMTI target positioning search, the parallelism between its beam direction and visible light is less than 0.5°.

[0053] The model conversion process between the image coordinate system and the geographic coordinate system is:

[0054] (1) Locate the center of the image and determine its geographical location;

[0055] (2) Perform virtual ortho transformation on the original image;

[0056] (3) According to the laser ranging distance L, the current focal length value f, and the pixel size s information, the geographic coordinates (lon, lat) are transformed into orthophotod image coordinates;

[0057] (4) Obtain the inverse M based on the orthogonal transformation obtained in step (2) -1 , acting on the image coordinates (ox, oy) after orthophotography, the original image coordinates P = (ox1, oy1) are calculated.

[0058] In step (1), the center of the image is located. Assume that the azimuth angle of the photoelectric radar integrated reconnaissance equipment is α (the direction of the aircraft nose is 0°, and counterclockwise is positive), the pitch angle is β (horizontally forward is 0°, and vertically downward is -90°), the laser ranging distance is L, and the coordinate system of the photoelectric radar integrated reconnaissance equipment is a right-handed coordinate system. The origin is the intersection of the azimuth axis and the pitch axis. The X axis points to the right side of the aircraft nose direction, and the Y axis points to the direction of the aircraft nose. The position of the target in the equipment coordinate system is:

[0059]

[0060] The device's geographic location is obtained through inertial navigation information, with longitude lon, latitude lat, altitude alt, roll angle γ, pitch angle θ, and heading angle The device coordinate system to the geographic coordinate system rotates around the roll angle -γ, the pitch angle -θ, and the heading angle The corresponding rotation matrices are A4, A5, and A6, so the position of the target in the origin coordinate system is A6A5A4P0, and the rotation matrix from the geographic coordinate system to the earth coordinate system is

[0061]

[0062] The translation matrix from the geographic coordinate system with the device coordinate system as the origin to the earth coordinate system is A8, whose components

[0063]

[0064] Where R n is the radius of curvature of the cycloid corresponding to latitude lat, and e is the flattening of the earth;

[0065] So the position of the target in the earth's rectangular coordinate system is:

[0066]

[0067] In step (2), the transformation matrix is Where R represents the rotation matrix, T represents the translation vector, P represents the perspective matrix, and S represents the scaling factor.

[0068] In step (3), let the width of the transformed image be w, the height be h, the radius of the earth be R, the geographic coordinates of the center point of the image be (cX, cY), the geographic coordinates of the track be (locX, locY), and the orthogonal image coordinates (ox, oy) satisfy:

[0069]

[0070]

[0071] Where:

[0072] Intermediate variables

[0073] Intermediate variables

[0074] Intermediate variables

[0075] During positioning, the parallelism between the laser optical axis and the visible light optical axis is required to be kept within 0.2 mrad.

[0076] Example

[0077] In this specific embodiment, the visible light imaging resolution is 1920X1080, the pixel size is 3.2μm, the focal length is generally set to 200mm, the laser ranging is more than 10km, the parallelism of the laser optical axis and the visible light optical axis is maintained within 0.2mrad, and the parallelism of the radar beam direction and the visible light is less than 0.5°.

[0078] The radar performs GMTI search, generates a track and sends it to the image processing system. The image processing system locates the target in the center of the field of view based on the combined navigation information, the range value L and other information. Since there is a certain angle difference between the imaging target surface and the earth, the image needs to be virtually ortho-transformed. The ortho-transformation relationship M is determined by locating the center point, and its coordinates in the image after ortho-transformation are calculated according to the geographical location of the track. Then, the image is transformed by M. -1 Perform inverse transformation to obtain the coordinate position P of the track in the real-time image, and then mark the relevant information. This method greatly enhances the readability of the track, enables operators to understand the surrounding situation of the target more intuitively, and improves the efficiency of reconnaissance.

[0079] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for fusing optoelectronic video and radar GMTI, characterized in that: include: Use radar to search for GMTI and generate track information, which includes track number, geographic location, distance, speed and direction; Collect optical video and obtain the center distance of the video field of view through a laser rangefinder; Obtaining integrated navigation information and azimuth and elevation information of optoelectronic radar integrated reconnaissance equipment; Perform model conversion between image coordinate system and geographic coordinate system; The geographical coordinates of the GMTI track are resolved into image coordinates, and the track number, geographical location, distance, speed and direction information are marked on the optical video to achieve the fusion of the GMTI track and the optical video; Among them, the model conversion process between the image coordinate system and the geographic coordinate system is: (1) Locate the center of the image and determine its geographical location; (2) Perform virtual ortho transformation on the original image; (3) According to the laser ranging distance L, the current focal length value f, and the pixel size s information, the geographic coordinates (lon, lat) are transformed into orthophotod image coordinates; (4) Obtain the inverse M based on the orthogonal transformation obtained in step (2) -1 , acting on the image coordinates (ox, oy) after orthophotography, the original image coordinates P = (ox1, oy1) are calculated; In step (1), the center of the image is located. Assume that the azimuth angle of the photoelectric radar integrated reconnaissance equipment is α, the pitch angle is β, the laser ranging distance is L, the coordinate system of the photoelectric radar integrated reconnaissance equipment is a right-hand coordinate system, the origin is the intersection of the azimuth axis and the pitch axis, the X axis points to the right side of the aircraft nose direction, and the Y axis points to the aircraft nose direction. Then the position of the target in the equipment coordinate system is: The device's geographic location is obtained through inertial navigation information, with longitude lon, latitude lat, altitude alt, roll angle γ, pitch angle θ, and heading angle The device coordinate system to the geographic coordinate system rotates around the roll angle -γ, the pitch angle -θ, and the heading angle The corresponding rotation matrices are A4, A5, and A6, so the position of the target in the origin coordinate system is A6A5A4P0, and the rotation matrix from the geographic coordinate system to the earth coordinate system is The translation matrix from the geographic coordinate system with the device coordinate system as the origin to the earth coordinate system is A8, whose components Where R n is the radius of curvature of the cycloid corresponding to latitude lat, and e is the flattening of the earth; Therefore, the position of the target in the earth's rectangular coordinate system is: In step (2), the transformation matrix is In the formula, R represents the rotation matrix, T represents the translation vector, P represents the perspective matrix, and S represents the scaling factor; In step (3), let the width of the transformed image be w, the height be h, the radius of the earth be R, the geographic coordinates of the center point of the image be (cX, cY), the geographic coordinates of the track be (locX, locY), and the orthogonal image coordinates (ox, oy) satisfy: Where: Intermediate variables Intermediate variables Intermediate variables 2. The method for fusing optoelectronic video and radar GMTI according to claim 1, characterized in that: The image coordinate system is the coordinate system of the output image after optical imaging. The upper left corner of the image is the origin, and the rightward and downward directions are positive. The geographic coordinate system is the earth's coordinate system, expressed in longitude and latitude. Each track contains the geographic location, movement speed direction and size.

3. The method for fusing optoelectronic video and radar GMTI according to claim 1, characterized in that: The model conversion is determined according to the ranging distance, pixel size, focal length value, and positioning information, and the geographic coordinates of the GMTI track are converted into image coordinates to achieve data fusion.

4. The method for fusing optoelectronic video and radar GMTI according to claim 1, characterized in that: When positioning the optics, the parallelism between the laser optical axis and the visible light optical axis is ≤0.2mrad.

5. The method for fusing optoelectronic video and radar GMTI according to claim 1, characterized in that: When the radar performs GMTI target positioning search, the parallelism between its beam direction and visible light is less than 0.5°.

6. The method for fusing optoelectronic video and radar GMTI according to claim 1, characterized in that: During positioning, the parallelism between the laser optical axis and the visible light optical axis is required to be kept within 0.2 mrad.

Images