A multi-radar image fusion method suitable for ship navigation and shore-based monitoring

Abstract

The application relates to a multi-radar image fusion method suitable for ship navigation and shore-based monitoring, which comprises the following steps: 1, acquiring the position information and north information of different received radars, establishing a unified spatial coordinate system of the different radars, and calculating the position points of the different radars in the coordinate system; 2, respectively drawing the scanning lines of the received radars in the unified spatial coordinate system; 3, transmitting the radar echo data transmitted in real time, the fusion mode configuration parameters and the image of the scanning lines of the radars drawn in the unified spatial coordinate system to the GPU through OpenGL; 4, fusing the images of the scanning lines of all the radars drawn in the unified spatial coordinate system, and coloring the fused image in the GPU; and 5, displaying the fused image on a display interface. Through the OpenGL technology, the work of the CPU is transferred to the GPU, the work load of the CPU is reduced, seven fusion modes are provided, and the enhanced perception of the multi-radar image of the user is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of ship navigation and shore-based offshore monitoring, in particular to a multi-radar image fusion method suitable for ship navigation and shore-based monitoring. BACKGROUND

[0002] Marine navigation radar is a radar installed on a ship for navigation avoidance, ship positioning, and narrow waterway piloting, which provides necessary observation means for mariners in poor visibility.

[0003] Single radar detection can cause the problem of excessive blind area. Through radar information fusion technology, the navigation radar detection data of the ship distributed in front, back, left and right positions can be integrated to realize full coverage, reduce measurement error and positioning error, improve system fault tolerance and reliability, eliminate detection blind area, and reduce the impact of "false targets" during radar operation.

[0004] Secondly, with the development of VTS monitoring radar and shore-based radar network, the comprehensive solution of offshore radar network requires real-time observation and monitoring of the detection sea area in all-weather and all-automatic manner, and needs to realize multi-station networking, which can provide highly integrated real-time maritime situation awareness information for sea monitoring tasks. Through intelligent analysis and judgment of detection data information, it provides information means for maritime monitoring law enforcement, safety production, port monitoring, disaster reduction prediction, marine ecological environment protection and other observation and monitoring tasks, and improves management efficiency. The radar detection ranges of different positions of the shore-based radar overlap each other. Through multi-radar fusion technology, the radars at different positions of the shore-based radar can be fused to improve the reliability of the target.

[0005] However, in order to integrate with the chart system and other systems, modern radars generally adopt the mode of digital radar client. In this mode, the secondary signal processing and image rendering both use CPU as the core processor, which requires more computing resources. When multiple radars are superimposed, the CPU computing resources of the PC platform are seriously occupied, and it is basically difficult to meet the rendering requirements.

[0006] At the same time, the current common multi-radar fusion scheme only simply superimposes images. The existing technology, such as the invention patent with the publication number CN113608213A, discloses a marine target joint detection method based on ship radar information fusion. The polar coordinate system corresponding to the two radar videos obtained is converted into a Cartesian coordinate system with a consistent reference point CCRP. The two radar videos in the consistent reference point CCRP Cartesian coordinate system are fused by using a fusion strategy, and the fused image is output. Although the existing technology can fuse radars, it only provides a fusion means, which cannot be changed according to the actual scene of the user and cannot meet the actual use requirements of the user. SUMMARY

[0007] In order to solve the above technical problems, the present application provides a multi-radar image fusion method suitable for ship navigation and shore-based monitoring, which improves the efficiency of multi-radar video information fusion, reduces the fusion workload of the entire multi-radar system, improves the quality of radar fusion, and can meet the needs of users to change according to actual scenes.

[0008] The technical purpose of the present application is realized by the following technical solutions:

[0009] A multi-radar image fusion method suitable for ship navigation and shore-based monitoring, the method comprising:

[0010] Step 1, the multi-radar image fusion system receives echo data from at least two radars, obtains the position information and north information of the received different radars, establishes a unified spatial coordinate system of different radars, and calculates the position points of different radars in the coordinate system;

[0011] Step 2, the received scanning lines of the radars are respectively drawn in the unified spatial coordinate system;

[0012] Step 3, the real-time transmitted radar echo data, fusion mode configuration parameters and the image of the scanning lines of the radars drawn in the unified spatial coordinate system are transmitted to the GPU through OpenGL;

[0013] Step 4, the images of the scanning lines of all radars drawn in the unified spatial coordinate system are fused according to the fusion mode configuration parameters in the GPU, and the fused image is colored in the GPU;

[0014] Step 5, the fused image is displayed through a display interface.

[0015] Further, in step 3, the fusion mode includes maximum fusion, minimum fusion, mean fusion, different color fusion, XOR fusion, XNOR fusion and threshold fusion, and the user can choose one of the seven fusion modes according to the actual scene.

[0016] Further, in step 1, when establishing the unified spatial coordinate system of different radars, the following steps are included:

[0017] Step 1.1, a consistent reference point is established according to the application scene, and the application scene includes a ship navigation scene and a shore-based monitoring scene;

[0018] Step 1.2, a coordinate system is established according to the consistent reference point in the application scene, and the coordinate system is an X-Y two-dimensional coordinate system;

[0019] Step 1.3, the position points of different radars in the coordinate system are calculated.

[0020] Further, in step 1.1, the CCRP point set on the ship is taken as the reference point in the ship navigation scenario, and the geodetic origin is taken as the reference point in the shore-based monitoring scenario.

[0021] Further, in step 1.2, in the ship navigation scenario, the coordinate system is established with the north direction as the Y axis and the east direction as the X axis; in the shore-based monitoring scenario, the coordinate system is established with the bow direction as the Y axis and the starboard direction as the X axis.

[0022] Further, in step 1.3, in the ship navigation scenario, the position point of the radar in the coordinate system is obtained by querying the ship design manual, represented as (ant x , ant x ).

[0023] Further, in step 1.3, in the shore-based monitoring scenario, the position point of the radar in the coordinate system is calculated as follows:

[0024] d = R * arcos (sin (lon0) * sin (lon) + cos (lon0) * cos (lon) * cos (lat-lat0));

[0025]

[0026] wherein, lon0 and lat0 are the longitude and latitude of the geodetic origin respectively;

[0027] lon and lat are the longitude and latitude of the radar antenna respectively;

[0028] R is the radius of the earth;

[0029] d is the distance between two points;

[0030] is the bearing of the radar antenna relative to the coordinate origin;

[0031] According to the polar coordinate conversion rectangular coordinate formula, the position coordinates (ant x , ant x ) of the radar in the coordinate system are obtained:

[0032]

[0033]

[0034] Further, in step 2, the information of the radar scanning line is converted into two-dimensional data (x, y), calculated as follows:

[0035] x = i * cos (theta) + ant x ;

[0036] y = i * sin(theta) + ant y ;

[0037] Wherein, x and y respectively represent the horizontal coordinate position and the vertical coordinate position of the point on the scanning line in the coordinate system;

[0038] I is the position of the point on the scanning line, and theta is the angle of the point on the scanning line.

[0039] Further, each pixel point of the radar image has a gray value,

[0040] When taking large fusion, the pixel with the maximum gray value of the radar image at the same position is taken and colored;

[0041] When taking small fusion, the pixel with the minimum gray value of the radar image at the same position is taken and colored;

[0042] When mean value fusion, the pixel with the mean value of the gray values of all the radar images at the same position is taken and colored;

[0043] When different color fusion, the pixels with the gray values of all the radar images at the same position are superimposed and colored;

[0044] When XOR fusion, the pixel points outside the superimposed area of the radar image range are displayed in bright color;

[0045] When XNOR fusion, the pixel points in the superimposed area of the radar image range are displayed in dark color;

[0046] When threshold fusion, the pixels after eliminating the pixels with the gray values lower than the threshold value at the same position are superimposed and colored.

[0047] Compared with the prior art, the present application has the beneficial effects that:

[0048] 1、The present application transfers the work of CPU to GPU through OpenGL technology, reduces the work load of CPU, and solves the problem that large-scale multi-radar image fusion cannot be realized due to the limitation of CPU operation resources.

[0049] 2、The present application provides seven fusion modes, which are beneficial to enhance the user's perception of multi-radar images; among them, the taking large fusion is suitable for scenes with less clutter, which is beneficial to observe small targets; the taking small fusion is suitable for scenes with more clutter, which is beneficial to reduce clutter interference; the mean value fusion is suitable for general scenes; the different color fusion is suitable for when the user needs to observe different radar echoes; the XOR fusion and the XNOR fusion are both suitable for emphasizing the multi-radar overlapping area to the user, which is beneficial to the user to observe the difference of multi-radar echoes; the threshold fusion is a supplement to the mean value fusion, which helps to exclude the interference of small clutter in the scene of mean value fusion.

[0050] 3. The radar echo data transmitted in real time, the fusion mode configuration parameters and the image of the scanning line of the radar drawn in the unified space coordinate system are transmitted to the GPU by using the OpenGL technology, the data exchange between the CPU and the GPU is reduced, and the operation efficiency of the CPU and the GPU is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a multi-radar image fusion method flowchart suitable for ship navigation and shore-based monitoring of the application. DETAILED DESCRIPTION

[0052] The technical solutions of the application will be further described below in combination with specific embodiments:

[0053] A multi-radar image fusion method suitable for ship navigation and shore-based monitoring, based on the glsl language of OpenGL, seven image fusion modes are realized, and the image perception degree of the user to the multi-radar system is enhanced, as shown in the figure, the method comprises: Figure 1

[0054] Step 1, accept the opening request of the radar image fusion function of at least two or even more radars, check whether the echo data from the multiple radars is received, if the echo data is received, obtain the position information and the north information of the received different radars, establish the unified space coordinate system of the different radars, and calculate the position points of the different radars in the coordinate system;

[0055] Specifically, when establishing the unified coordinate system of the different radars, the following steps are included:

[0056] Step 1.1, a consistent reference point is established according to the application scene, the application scene includes a ship navigation scene and a shore-based monitoring scene; in the ship navigation scene, the CCRP point set on the ship is taken as the reference point, and in the shore-based monitoring scene, the geodetic origin is taken as the reference point, and the geodetic origin is subject to the geodetic origin of the local country.

[0057] Step 1.2, the coordinate system is established according to the consistent reference point in the application scene, and the coordinate system is an X-Y two-dimensional coordinate system; the establishment of the coordinate system is different in different application scenes:

[0058] In the ship navigation scene, the coordinate system is established with the north direction as the Y axis and the east direction as the X axis; in the shore-based monitoring scene, the coordinate system is established with the bow direction as the Y axis and the starboard direction as the X axis.

[0059] Step 1.3, the position points of the different radars in the coordinate system are calculated (ant x , ant x ​), in the ship navigation scene, since the position of the radar relative to the CCRP is determined when the radar is installed, the position point of the radar in the unified space coordinate system can be obtained by querying the ship design manual;

[0060] In the shore-based monitoring scene, the position point of the radar in the unified space coordinate system is calculated as follows:

[0061] d=R*arcos(sin(lon0)*sin(lon)+cos(lon0)*cos(lon)*cos(lat-lat0));

[0062]

[0063] Wherein, lon0, lat0 are the longitude and latitude of the geodetic origin respectively;

[0064] lon, lat are the longitude and latitude of the radar antenna respectively;

[0065] R is the radius of the earth;

[0066] d is the distance between two points;

[0067] is the azimuth of the radar antenna relative to the coordinate origin;

[0068] According to the polar coordinate conversion rectangular coordinate formula, the position coordinates of the radar in the coordinate system (ant x , ant x ) are obtained:

[0069]

[0070]

[0071] Step 2, draw the received scanning lines of the radar in the unified coordinate system respectively;

[0072] The scanning line is the analog information returned by the electromagnetic wave emitted by the antenna of the radar after sampling, which is arranged by distance. Generally, the radar will generate 1024 to 4096 scanning lines per rotation. The final image formed by the radar is a two-dimensional image. The scanning line information needs to be converted to two-dimensional data. Through the conversion formula of polar coordinates and rectangular coordinates, the position (x, y) of each radar in the unified space coordinate system needs to be increased, which is calculated as follows:

[0073] x=i*cos(theta)+ant x ;

[0074] y=i*sin(theta)+ant y ;

[0075] wherein x and y represent the horizontal coordinate position and the vertical coordinate position of the point on the scanning line in the coordinate system respectively;

[0076] i is the position of the point on the scanning line, and θ is the angle of the point on the scanning line.

[0077] Step 3, transmit the real-time transmitted radar echo data, the fusion mode configuration parameter and the image of the scanning line of the radar drawn in the unified coordinate system to the GPU through OpenGL;

[0078] Specifically, the following steps are included:

[0079] Step 3.1

[0080] The real-time transmitted radar echo data is transmitted to the GPU through OpenGL. The two-dimensional radar echo data needs to be implemented by creating a two-dimensional texture buffer area in the video memory through OpenGL. The OpenGL creates the texture buffer area by means of the glGenTextures function, and the specific implementation is as follows:

[0081] glGenTextures(GLsizei n, GLuint*textures)

[0082] glTexImage2D(GLenum target, GLint level, GLenum internalformat, GLsizei width, GLsizei height, GLint border, GLenum format, GLenum type, const void* pixels);

[0083] Similarly, a memory area with the same size as the radar echo data needs to be created in the memory to store the radar grayscale value image transmitted by the previous process and transmitted to the video memory through OpenGL. The method of transferring data from the OpenGL memory to the video memory is as follows:

[0084] void glTexSubImage2D(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLsizei width, GLsizei height, GLenum format, GLenum type, const GLvoid* data).

[0085] Step 3.2

[0086] When fusing the texture images transmitted by different radars, the fusing mode configuration parameters need to be formed according to the fusing mode selected by the user to perform the fusing. The fusing mode includes seven fusing modes, which are maximum fusing, minimum fusing, mean fusing, different color fusing, XOR fusing, XNOR fusing and threshold fusing. The user selects one of the seven fusing modes according to the application scene requirement.

[0087] The fusing mode configuration parameters need to be transmitted to the graphics card, and the position serial number of the uniform variable is returned by the GlintglGetUniformLocation(GLuint program, const GLchar*name) function in the GLSL language. The glUniform1f(GLint location, GLfloat v0) can assign values to the position serial number, so that the value of the specified variable in the glsl is changed.

[0088] Step 3.3

[0089] The images of different radars in the uniform coordinate system are transmitted to the GPU. The OpenGL glTexImage3D is used to obtain the texture data; and then the data is processed in the fragment shader texture3D, and the obtained data has been compressed to 0-1. After the data is imported by the OpenGL function glTexImage3D, the data is compressed in the GLSL. A three-dimensional texture image is specified by glTexImage3D. In this way, the texture data can be transmitted to the glsl, and the gray value of the specified position in the texture is obtained by texture in the glsl. In this way, the gray scale images of all radars can be transmitted to the texture buffer area in the glsl.

[0090] Step 4, the images of all radars in the uniform coordinate system are fused according to the fusing mode configuration parameters, and the fused image is colored in the GPU.

[0091] A1, A2, A3, …, A i ,…,A n are the gray values of the same position of different radars, and the conversion of the gray value to the RGB color can be realized by looking up the table, and the color table is represented as Table[n]

[256] .

[0092] When the maximum fusing is performed, the pixel with the maximum gray value of the radar image at the same position is taken and colored. Generally, the first color in the color table is used for pixel coloring by default:

[0093] Color=Table[1][max(A1,A2,A3,…,A i ,…,A n )];

[0094] When taking the minimum fusion, the pixel with the minimum gray value of the radar image at the same position is taken and colored, and the first color in the color table is generally used by default for pixel coloring:

[0095] Color = Table [1] [min (A1, A2, A3, …, A i , A n )];

[0096] When taking the mean fusion, the pixel with the mean value of the gray values of all radar images at the same position is taken and colored, and the first color in the color table is generally used by default for pixel coloring:

[0097]

[0098] When taking the different color fusion, the pixels of the gray values of all radar images at the same position are superimposed and colored:

[0099] Color = Table [i] [A i ];

[0100] When taking the XOR fusion, the image ranges of different radars can be different, and the pixel points outside the superimposed area of the radar image ranges are colored in bright color, and the first color in the color table is generally used by default for pixel coloring:

[0101] Color = Table [1] [A1] ⊕ Table [2] [A2] ⊕ … ⊕ Table [n] [A n ];

[0102] When taking the XNOR fusion, the image ranges of different radars can be different, and the pixel points in the superimposed area of the radar image ranges are colored in dark color, and the first color in the color table is generally used by default for pixel coloring:

[0103] Color = Table [1] [A1] ⊙ Table [2] [A2] ⊙ … ⊙ Table [n] [A n ];

[0104] When taking the threshold fusion, the pixels after removing the pixels with gray values lower than the threshold value at the same position are superimposed and colored, and the first color in the color table is generally used by default for pixel coloring:

[0105] Color = Table [1] [AVG (U)];

[0106] U = {A i |A i > δ}, where δ is an amplitude threshold value, and δ is generally selected as 30% of the mean value of the amplitudes after removing abnormal points.

[0107] Step 5, the fused image is displayed through a display interface.

[0108] The embodiment is only a further explanation of the present application, and is not a limitation of the present application. Those skilled in the art can make non-creative modifications to the embodiment according to the needs after reading the specification, but as long as the modifications are within the scope of the claims of the present application, they are protected by the Patent Law.

Claims

1. A multi-radar image fusion method suitable for ship navigation and shore-based monitoring, characterized in that, The method comprises: Step 1, a multi-radar image fusion system receives echo data from at least two radars, obtains position information and north information of the received radars, and establishes a unified spatial coordinate system of the radars, and the establishment of the unified spatial coordinate system comprises the following steps: Step 1.1, a consistent reference point is established according to an application scenario, and the application scenario comprises a ship navigation scenario and a shore-based monitoring scenario; Step 1.2, a coordinate system is established according to the consistent reference point in the application scenario, and the coordinate system is an X-Y two-dimensional coordinate system; Step 1.3, the position points of the radars in the coordinate system are calculated, and in the shore-based monitoring scenario, the position points of the radars in the coordinate system are calculated as follows: ； ； wherein , are the longitude and latitude of the geodetic origin, respectively; lon and lat are the longitude and latitude of the radar antenna respectively; R is the radius of the earth; d is the distance between two points; is the azimuth of the radar antenna relative to the coordinate origin; The position coordinates of the radar in the coordinate system are obtained using the formula for transforming polar coordinates to rectangular coordinates. ): ； ； Step 2, the received scanning lines of the radars are drawn in the unified spatial coordinate system respectively; Step 3, the real-time transmitted radar echo data, the fusion mode configuration parameters and the images of the scanning lines of the radars drawn in the unified spatial coordinate system are transmitted to the GPU through OpenGL; Step 4, the images of the scanning lines of all the radars drawn in the unified spatial coordinate system are fused according to the fusion mode configuration parameters in the GPU, and the fused images are colored in the GPU; Step 5, the fused images are displayed through a display interface.

2. The multi-radar image fusion method for ship navigation and shore-based monitoring according to claim 1, characterized in that, In the step 3, the fusion mode comprises maximum fusion, minimum fusion, mean fusion, different color fusion, XOR fusion, XNOR fusion and threshold fusion, and a user selects one according to an actual scenario.

3. The radar image fusion method for ship navigation and shore-based monitoring according to claim 1, characterized in that, In the step 1.1, in the ship navigation scenario, a CCRP point set on the ship is taken as the reference point, and in the shore-based monitoring scenario, a geodetic origin is taken as the reference point.

4. The radar image fusion method for ship navigation and shore-based monitoring according to claim 3, characterized in that, In the step 1.2, in the ship navigation scenario, the coordinate system is established with the north direction as the Y axis and the east direction as the X axis, and in the shore-based monitoring scenario, the coordinate system is established with the bow direction as the Y axis and the starboard direction as the X axis.

5. The radar image fusion method for ship navigation and shore-based monitoring according to claim 4, characterized in that, In the step 1.3, under the ship navigation scenario, the position point of the radar in the coordinate system is obtained by querying the ship design manual, denoted as (xR, yR). ).

6. The radar image fusion method for ship navigation and shore-based monitoring according to claim 1 or 5, characterized in that, In the step 2, the information of the radar scanning lines is converted into two-dimensional data (x, y), and the calculation is as follows: ； ； Wherein, x and y represent the horizontal coordinate position and the vertical coordinate position of the points on the scanning line in the coordinate system respectively; i is the position of the point on the scanning line, and θ is the angle of the point on the scanning line.

7. The multi-radar image fusion method for ship navigation and shore-based monitoring according to claim 2, characterized in that, Each pixel point of the radar image has a gray value, When the maximum fusion is taken, the pixel with the maximum gray value of the radar image at the same position is taken and colored; When the minimum fusion is taken, the pixel with the minimum gray value of the radar image at the same position is taken and colored; When the mean fusion is taken, the pixel with the mean value of the gray values of all the radar images at the same position is taken and colored; When the different color fusion is taken, the pixels of the gray values of all the radar images at the same position are superimposed and colored; When the XOR fusion is taken, the pixel points outside the superimposed area of the radar image range are brightly colored and displayed; When the XNOR fusion is taken, the pixel points of the superimposed area of the radar image range are darkly colored and displayed; When the threshold fusion is taken, the pixels after the pixels with the gray values lower than the threshold value at the same position are removed, and the superimposed pixels are colored and displayed.

Images