Image fusion method of solar blind and visible light binocular heterogenous wide-angle imaging system

Abstract

The invention discloses an image fusion method for a solar blind and visible light binocular heterogenous wide-angle imaging system, and the method can achieve the more accurate, efficient and direct measurement of an object image position relation through a solar blind ultraviolet target light source and a precision track with scales. Therefore, the first mapping relation between the physical position of the point target and the actual image point coordinate can be quickly determined, then the original distorted solar blind ultraviolet image can be corrected based on the first mapping relation and the second mapping relation, and the accuracy and efficiency of generating the corrected solar blind ultraviolet image are improved. Besides, fusion processing is carried out on the corrected solar blind ultraviolet image and the corrected visible light image based on the preset solar blind and visible light image point pixel coordinate relation, so that the problem that a traditional registration fusion method based on gray or geometric characteristics is not applicable due to different imaging principles and image characteristics of solar blind ultraviolet and visible light is solved; and the accuracy and efficiency of generating the fused image are improved.

Description

Technical Field

[0001] The present application relates to the field of optical technologies, and particularly to an image fusion method for a solar-blind and visible light binocular heterologous wide-angle imaging system. Background Art

[0002] In a solar-blind ultraviolet and visible light dual-band fusion detection system, due to the different characteristics of the target ultraviolet band and visible light band in the scene. For example, some targets in the same scene can autonomously emit solar-blind ultraviolet radiation, some targets only reflect the ultraviolet radiation from the sun, and some other targets basically do not reflect or radiate solar-blind ultraviolet signals. Moreover, whether it is autonomous radiation or reflection, it will be strongly absorbed by the atmosphere, resulting in very weak solar-blind ultraviolet signals of the targets in the scene. The above differences in band characteristics require different detection principles, different optical system parameters, as well as different detector types, target surface sizes, and image resolutions. It is a typical heterologous binocular detection, and the target characteristics and the differences between the two bands make it impossible to adopt a common optical path design.

[0003] Traditionally, when performing distortion correction and image fusion on a dual-band independent optical imaging camera based on parallel optical axes, although the fields of view of the two overlap, they are not the same. Coupled with the distortion problem of wide angles, the image registration and fusion of this binocular heterologous wide-angle imaging system are relatively special, seriously lacking feature backgrounds and feature points, and the gray-scale responses are also inconsistent. Therefore, conventional registration and fusion methods based on the similarity of feature points and gray-scale values cannot be used.

[0004] Therefore, how to achieve image fusion of a solar-blind and visible light binocular heterologous wide-angle imaging system is an urgent problem to be solved. Summary of the Invention

[0005] The present application aims to at least solve the technical problems existing in the prior art. To this end, in a first aspect of the present application, an image fusion method for a solar-blind and visible light binocular heterologous wide-angle imaging system is proposed. The method includes:

[0006] Based on the object-image position relationship between the solar-blind ultraviolet target light source and the precision track with scale for measuring the solar-blind ultraviolet camera, and obtaining a first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship;

[0007] Obtaining a second mapping relationship between the physical position of the point target and the theoretical coordinates of the preset image point;

[0008] Performing correction processing on the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image;

[0009] Based on the preset relationship between the pixel coordinates of the solar-blind and visible light image points, the corrected solar-blind ultraviolet image and the corrected visible light image are fused to generate a fused image.

[0010] In a possible implementation, based on the object-image position relationship between the solar-blind ultraviolet target light source and the precision track with scale for measuring the solar-blind ultraviolet camera, it includes:

[0011] Adjust the supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable diaphragm to the target values; wherein, the solar-blind ultraviolet camera is located on the central axis of the solar-blind ultraviolet target light source and faces the center of the adjustable diaphragm, the solar-blind ultraviolet target light source is fixed on the precision track with scale, and the optical axis of the solar-blind ultraviolet camera is perpendicular to the track;

[0012] At the target values, move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale to obtain multiple light source image data;

[0013] Based on the multiple light source image data, obtain the object-image position relationship of the solar-blind ultraviolet camera.

[0014] In a possible implementation, at the target values, move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale to obtain multiple light source image data, including:

[0015] At the target values, move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale with a preset moving distance to obtain multiple light source positions;

[0016] For each light source position, obtain the light source image data corresponding to the light source position to obtain multiple light source image data.

[0017] In a possible implementation, the process of correcting the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image includes:

[0018] Based on the first mapping relationship and the second mapping relationship, determine the third mapping relationship between the actual image point coordinates and the preset image point theoretical coordinates, and obtain the distortion correction matrix based on the third mapping relationship;

[0019] Based on the distortion correction matrix, perform correction processing on the original distorted solar-blind ultraviolet image to generate a corrected solar-blind ultraviolet image.

[0020] In a possible implementation, the generation process of the preset relationship between the solar-blind and visible light image point pixel coordinates includes:

[0021] Obtain the first collinearity relationship between the preset object point and the preset solar-blind ultraviolet image point;

[0022] Obtain the second collinearity relationship between the preset object point and the preset visible light image point;

[0023] Based on the first collinearity relationship and the second collinearity relationship, a fourth mapping relationship between the preset solar-blind ultraviolet image pixel points and the preset visible light image pixel points is constructed, and the fourth mapping relationship is used as the pixel coordinate relationship between the preset solar-blind and visible light pixel points.

[0024] In a possible implementation manner, obtaining the first collinearity relationship between the preset object point and the preset solar-blind ultraviolet image pixel point includes:

[0025] Taking the solar-blind ultraviolet camera as a reference, a world coordinate system is established, and the origin of the world coordinate system is used as the optical center of the solar-blind ultraviolet camera;

[0026] Obtain the preset first image reduction ratio, the preset pixel size of the solar-blind ultraviolet camera, the coordinates of the preset solar-blind ultraviolet image pixel point, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera;

[0027] Based on the preset first image reduction ratio, the preset pixel size of the solar-blind ultraviolet camera, the coordinates of the preset solar-blind ultraviolet image pixel point, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera, calculate the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel point;

[0028] Based on the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel point and the first preset collinearity equation, the first collinearity relationship is calculated.

[0029] In a possible implementation manner, obtaining the second collinearity relationship between the preset object point and the preset visible light image pixel point includes:

[0030] Based on the parallel relationship between the optical axis of the solar-blind ultraviolet camera and the optical axis of the visible light camera, the optical center of the visible light camera is determined;

[0031] Obtain the preset second image reduction ratio, the preset pixel size of the visible light camera, the coordinates of the preset visible light image pixel point, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera;

[0032] Based on the preset second image reduction ratio, the preset pixel size of the visible light camera, the coordinates of the preset visible light image pixel point, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera, calculate the image space coordinates corresponding to the preset visible light image pixel point;

[0033] Based on the image space coordinates corresponding to the preset visible light image pixel point and the second preset collinearity equation, the second collinearity relationship is calculated.

[0034] A second aspect of the present application proposes an image fusion device for a solar-blind and visible light binocular heterologous wide-angle imaging system, and the device includes:

[0035] A first acquisition module, configured to determine the object-image position relationship between a solar-blind ultraviolet target light source and a precision track with scale for measuring a solar-blind ultraviolet camera, and acquire a first mapping relationship between the physical position of a point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship;

[0036] A second acquisition module, configured to acquire a second mapping relationship between the physical position of the point target and the theoretical coordinates of a preset image point;

[0037] A first generation module, configured to perform correction processing on an original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image;

[0038] A second generation module, configured to perform fusion processing on the corrected solar-blind ultraviolet image and a corrected visible light image based on a preset relationship between the pixel coordinates of solar-blind and visible light image points to generate a fused image.

[0039] In a possible implementation manner, the above-mentioned first acquisition module is specifically configured to:

[0040] Adjust the power supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable diaphragm to target values; wherein, the solar-blind ultraviolet camera is located on the central axis of the solar-blind ultraviolet target light source and faces the center of the adjustable diaphragm, the solar-blind ultraviolet target light source is fixed on a precision track with scale, and the optical axis of the solar-blind ultraviolet camera is perpendicular to the track;

[0041] Move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale at the target values to obtain a plurality of light source image data;

[0042] Obtain the object-image position relationship of the solar-blind ultraviolet camera based on the plurality of light source image data.

[0043] In a possible implementation manner, the above-mentioned first acquisition module is further configured to:

[0044] Move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale at the target values by a preset moving distance to obtain a plurality of light source positions;

[0045] For each light source position, acquire the light source image data corresponding to the light source position to obtain a plurality of light source image data.

[0046] In a possible implementation manner, the above-mentioned first generation module is specifically configured to:

[0047] Determine a third mapping relationship between the actual image point coordinates and the theoretical coordinates of the preset image point based on the first mapping relationship and the second mapping relationship, and obtain a distortion correction matrix based on the third mapping relationship;

[0048] Perform correction processing on the original distorted solar-blind ultraviolet image based on the distortion correction matrix to generate a corrected solar-blind ultraviolet image.

[0049] In a possible implementation, the image fusion device of the above-mentioned solar-blind and visible light binocular heterologous wide-angle imaging system is further configured to:

[0050] Obtain a first collinearity relationship between a preset object point and a preset solar-blind ultraviolet image pixel point;

[0051] Obtain a second collinearity relationship between a preset object point and a preset visible light image pixel point;

[0052] Based on the first collinearity relationship and the second collinearity relationship, construct a fourth mapping relationship between the preset solar-blind ultraviolet image pixel point and the preset visible light image pixel point, and use the fourth mapping relationship as the pixel coordinate relationship between the preset solar-blind and visible light pixel points.

[0053] In a possible implementation, the image fusion device of the above-mentioned solar-blind and visible light binocular heterologous wide-angle imaging system is further configured to:

[0054] Taking the solar-blind ultraviolet camera as a reference, establish a world coordinate system, and use the origin of the world coordinate system as the optical center of the solar-blind ultraviolet camera;

[0055] Obtain a preset first image reduction ratio, a preset solar-blind ultraviolet camera pixel size, the coordinates of a preset solar-blind ultraviolet image pixel point, and the coordinates of the center point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera;

[0056] Based on the preset first image reduction ratio, the preset solar-blind ultraviolet camera pixel size, the coordinates of the preset solar-blind ultraviolet image pixel point, and the coordinates of the center point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera, calculate the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel point;

[0057] Based on the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel point and the first preset collinearity equation, calculate the first collinearity relationship.

[0058] In a possible implementation, the image fusion device of the above-mentioned solar-blind and visible light binocular heterologous wide-angle imaging system is further configured to:

[0059] Based on the parallel relationship between the optical axis of the solar-blind ultraviolet camera and the optical axis of the visible light camera, determine the optical center of the visible light camera;

[0060] Obtain a preset second image reduction ratio, a preset visible light camera pixel size, the coordinates of a preset visible light image pixel point, and the coordinates of the center point of the visible light image corresponding to the optical center of the visible light camera;

[0061] Calculate the image space coordinates corresponding to the preset visible light image pixel point based on the preset second image reduction ratio, the preset visible light camera pixel size, the coordinates of the preset visible light image pixel point, and the coordinates of the center point of the visible light image corresponding to the optical center of the visible light camera;

[0062] Calculate the second collinearity relationship based on the image space coordinates corresponding to the preset visible light image pixel point and the second preset collinearity equation.

[0063] A third aspect of the present application provides an electronic device, which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the image fusion method of the solar-blind and visible light binocular heterologous wide-angle imaging system as described in the first aspect.

[0064] A fourth aspect of the present application provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image fusion method of the solar-blind and visible light binocular heterologous wide-angle imaging system as described in the first aspect.

[0065] The embodiments of the present application have the following beneficial effects:

[0066] The image fusion method of the solar-blind and visible light binocular heterologous wide-angle imaging system provided by the embodiment of the present application includes: based on the object-image position relationship between the solar-blind ultraviolet target light source and the precise orbit measurement solar-blind ultraviolet camera with scales, and obtaining the first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship, obtaining the second mapping relationship between the physical position of the point target and the preset theoretical coordinates of the image point, correcting the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image, and performing fusion processing on the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset solar-blind and visible light image point pixel coordinate relationship to generate a fused image. Through the solar-blind ultraviolet target light source and the precise orbit with scales in this solution, the object-image position relationship can be measured more accurately and efficiently directly, so that the first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera can be quickly determined. Then, the original distorted solar-blind ultraviolet image can be corrected based on the first mapping relationship and the second mapping relationship, improving the accuracy and efficiency of generating the corrected solar-blind ultraviolet image; in addition, by performing fusion processing on the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset solar-blind and visible light image point pixel coordinate relationship, the problem that the traditional registration and fusion methods based on gray scale or geometric features are not applicable due to different imaging principles and image characteristics between solar-blind ultraviolet and visible light is solved, and the flexibility, accuracy and efficiency of generating the fused image are improved. Description of the Drawings

[0067] Figure 1 It is a block diagram of a computer device provided by an embodiment of the present application;

[0068] Figure 2 It is a flowchart of the steps of an image fusion method of a solar-blind and visible light binocular heterologous wide-angle imaging system provided by an embodiment of the present application;

[0069] Figure 3 It is a flowchart of the steps of measuring the object-image position relationship provided by an embodiment of the present application;

[0070] Figure 4 It is a schematic diagram of a test optical path provided by an embodiment of the present application;

[0071] Figure 5 It is a flowchart of the steps of obtaining multiple light source image data provided by an embodiment of the present application;

[0072] Figure 6 It is a flowchart of the steps of generating a corrected solar-blind ultraviolet image provided by an embodiment of the present application;

[0073] Figure 7A flowchart of steps for generating the relationship between the pixel coordinates of preset solar-blind and visible light image points provided by an embodiment of the present application;

[0074] Figure 8 A flowchart of steps for obtaining the first collinear relationship provided by an embodiment of the present application;

[0075] Figure 9 A flowchart of steps for obtaining the second collinear relationship provided by an embodiment of the present application;

[0076] Figure 10 A schematic diagram of the geometric positions of a solar-blind ultraviolet camera and a visible light camera provided by an embodiment of the present application;

[0077] Figure 11 A structural block diagram of an image fusion device for a solar-blind and visible light binocular heterologous wide-angle imaging system provided by an embodiment of the present application. Detailed implementation manners

[0078] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0079] In a solar-blind ultraviolet and visible light dual-band fusion detection system, due to the different characteristics of the target ultraviolet band and visible light band in the scene. For example, some targets in the same scene can autonomously emit solar-blind ultraviolet radiation, some targets only reflect the ultraviolet radiation from the sun, and some other targets basically do not reflect or radiate solar-blind ultraviolet signals. Moreover, whether it is autonomous radiation or reflection, it will be strongly absorbed by the atmosphere, resulting in very weak solar-blind ultraviolet signals of the targets in the scene, unclear geometric features of the ultraviolet targets, and poor signal-to-noise ratio. Such characteristics also have different requirements for indicators such as detection principles, detection technology systems, optical designs, system resolutions, field of view angles, and integration times.

[0080] The above differences in band characteristics require the use of different detection principles, different optical system parameters, as well as different detector types, target surface sizes, and image resolutions. It is a typical heterologous binocular detection. The target characteristics and the differences between the two bands make it impossible to adopt a common optical path design. The present invention performs distortion correction and image fusion based on a dual-band independent optical imaging camera with parallel optical axes. Although the fields of view of the two overlap, they are not the same. Coupled with the distortion problem of wide angles, the image registration and fusion of this binocular heterologous wide-angle imaging system are relatively special, seriously lacking feature backgrounds and feature points, and the gray-scale responses are also inconsistent. Therefore, it is impossible to adopt a conventional registration method based on the similarity of feature points and gray-scale values.

[0081] For the image fusion problem of the above wide-angle binocular heterologous imaging system, the distortion correction problems of the wide-angle solar-blind ultraviolet and wide-angle visible light cameras need to be solved first, and then the registration and fusion problems of the dual-band images need to be solved.

[0082] In the industry, the "Zhang's calibration method" is commonly used for the distortion correction of general visible light wide-angle cameras. That is, under good lighting conditions, a checkerboard with black and white squares is made using white paper or a whiteboard, placed at the working distance of the camera, and image acquisition is carried out at different positions in the center and edge of the camera's field of view. Then, the camera distortion correction parameters are obtained by solving according to Zhang's calibration algorithm. During image processing, the distortion correction parameters and the orthorectification formula are used to correct the distorted image, and finally an undistorted image is obtained. This is a simple and effective calibration method that can be operated by users themselves. Many professional image processing tools such as OpenCV and Matlab provide algorithms for calculating distortion parameters and distortion correction, which greatly promotes the application and popularization of computer vision products.

[0083] However, the distortion correction parameters of the "Zhang's calibration method" depend on the quality of the checkerboard image. When calibrating a visible light camera, the lighting conditions and image quality can meet the calibration requirements. However, for a wide-angle solar-blind ultraviolet camera, if the method of the visible light camera is used for distortion calibration, the following problems will occur:

[0084] The solar-blind ultraviolet camera only responds to the solar-blind band, and its signal-to-noise ratio is relatively low. Special solar-blind light sources are required for ambient light. The power of such light sources is limited, the light intensity reflected after irradiating the checkerboard is weak, and the coverage area is limited, and the lighting uniformity is very poor. Coupled with the weak edge response of the wide-angle camera, the quality of the solar-blind ultraviolet image of the checkerboard is significantly lower than that of the visible light image, and the recognition success rate of the corner points is not high, resulting in the failure of distortion calibration.

[0085] Since the field of view of the wide-angle solar-blind camera is large, in order to ensure that there are enough corner points participating in the operation, a large-sized checkerboard needs to be used to cover a large area of the field of view, and at the same time, higher requirements are put forward for the coverage and uniformity of the solar-blind light source. The "large solar-blind light source + large-sized checkerboard" scheme is generally not feasible due to the high cost of the light source and poor flexibility.

[0086] If the "small solar-blind light source + large-sized checkerboard" scheme is adopted, since the light source cannot cover the edge of the checkerboard well, especially when the checkerboard is located at the edge of the field of view, the insufficient image plane illumination leads to the failure of the solution during the subsequent "Zhang's calibration method" processing.

[0087] If the "small-sized solar-blind light source + small-sized checkerboard" solution is adopted, the light source can meet the coverage requirements. However, for a wide-angle field of view, the small-sized checkerboard has insufficient coverage. Due to insufficient calibration corner points, it is still impossible to calculate the distortion parameters after multiple attempts.

[0088] After the distortion correction of the wide-angle solar-blind ultraviolet camera and the wide-angle visible light camera, the registration and fusion of the dual-band images can be carried out. When the visible light camera images, basically all the targets in the scene can be imaged, and they have good geometric shapes. However, when using the solar-blind ultraviolet camera to image the targets in the same scene, if the target itself does not radiate solar-blind signals or the reflected solar-blind signals are weak, such targets will not be displayed in the final image. Even if they can be detected, their geometric shapes are not obvious. That is to say, the targets in the solar-blind image and the visible light image cannot correspond one by one, and the traditional image registration and fusion methods based on gray scale and geometric shape are not applicable.

[0089] In summary, for the distortion correction and image registration and fusion problems of wide-angle solar-blind ultraviolet and wide-angle visible light binocular heterologous imaging, the checkerboard distortion correction method and the conventional image registration and fusion method cannot solve them. New methods need to be adopted for distortion calibration and correction, as well as image registration and fusion.

[0090] Based on this, the present application proposes an image fusion method for a solar-blind and visible light binocular heterologous wide-angle imaging system. Through the solar-blind ultraviolet target light source and the precision track with scales, this solution can more accurately and efficiently directly measure the object-image position relationship, so as to quickly determine the first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after being imaged by the solar-blind ultraviolet camera. Then, based on the first mapping relationship and the second mapping relationship, the original distorted solar-blind ultraviolet image can be corrected, improving the accuracy and efficiency of generating the corrected solar-blind ultraviolet image. In addition, by fusing the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset pixel coordinate relationship between the solar-blind and visible light image points, the problem that the traditional registration and fusion methods based on gray scale or geometric features are not applicable due to different imaging principles and image characteristics between solar-blind ultraviolet and visible light is solved, and the flexibility, accuracy and efficiency of generating the fused image are improved.

[0091] Hereinafter, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "a plurality" is two or more. Additionally, the use of "based on" or "in accordance with" implies openness and inclusiveness, because a process, step, calculation, or other action based on or in accordance with one or more of the stated conditions or values may, in practice, be based on additional conditions or values beyond those stated.

[0092] The image fusion method of the solar-blind and visible light binocular heterologous wide-angle imaging system provided by this application can be applied to a computer device (electronic device). The computer device can be a server or a terminal. Among them, the server can be a single server or a server cluster composed of multiple servers. The embodiments of this application do not make specific limitations in this regard. The terminal can be, but is not limited to, various personal computers, laptop computers, smartphones, tablet computers, and portable wearable devices.

[0093] Taking the computer device as a server as an example, Figure 1 shows a block diagram of a server, as Figure 1 shown. The server can include a processor and a memory connected by a system bus. Among them, the processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, it realizes an image fusion method of a solar-blind and visible light binocular heterologous wide-angle imaging system.

[0094] Those skilled in the art can understand that Figure 1 the structure shown in

[0095] is only a block diagram of a part of the structure related to the solution of this application and does not constitute a limitation on the server to which the solution of this application is applied. Optionally, the server can include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout.

[0096] Figure 2 is a flowchart of the steps of an image fusion method of a solar-blind and visible light binocular heterologous wide-angle imaging system provided by the embodiments of this application. As Figure 2As shown, the method includes the following steps:

[0097] Step 202: Based on the relationship between the solar-blind ultraviolet target light source and the object-image position of the precision track-measuring solar-blind ultraviolet camera with scales, and obtain the first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship.

[0098] Among them, in the solar-blind ultraviolet and visible light dual-band fusion detection system, due to the different characteristics of the target ultraviolet band and visible light band in the scene, different detection principles, different optical system parameters, as well as different detector types, target surface sizes, and image resolutions are required. It is a typical heterologous binocular detection and requires image registration and fusion of the binocular heterologous wide-angle imaging system.

[0099] In some optional embodiments, as Figure 3 shown, Figure 3 is a flowchart of the steps for measuring the object-image position relationship provided by the embodiment of the present application, including:

[0100] Step 302: Adjust the supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable diaphragm to the target values.

[0101] Step 304: Move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale at the target values to obtain multiple light source image data.

[0102] Step 306: Obtain the object-image position relationship of the solar-blind ultraviolet camera based on the multiple light source image data.

[0103] Among them, an LED solar-blind ultraviolet light source can be used as the target source. To simulate a point target, an adjustable small-hole adjustable diaphragm is placed at the front end of the light source. Thus, the solar-blind ultraviolet camera is located on the central axis of the solar-blind ultraviolet target light source and is directly opposite the center of the adjustable diaphragm. The solar-blind ultraviolet target light source is fixed on a precision track with scales, and the optical axis of the solar-blind ultraviolet camera is perpendicular to the track.

[0104] As Figure 4 shown, Figure 4 is a schematic diagram of a test optical path provided by the embodiment of the present application. Among them, the solar-blind ultraviolet target light source is fixed on a precision track with scales. The track length is denoted as L D , which at least covers half of the camera's field of view. The 0 scale of the track is aligned with the optical axis of the solar-blind ultraviolet camera. The optical axis of the to-be-tested solar-blind imaging system is perpendicular to the track. The working distance of the solar-blind ultraviolet camera is denoted as Lc. The starting scale position of the track is denoted as A, the ending scale position is denoted as B, and the adjustable range of the aperture size of the adjustable diaphragm is denoted as Φ.

[0105] Initially, place the solar-blind ultraviolet target light source at the 0 scale of the track, and adjust the adjustable aperture to make the spot size on the image as small as possible. Optionally, the supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable aperture can be adjusted to the target values to make the spot as small as possible and the solar-blind ultraviolet camera can effectively image, and the spot gray level is 50-80% of the saturation value.

[0106] Next, the solar-blind ultraviolet target light source can be moved from the starting scale to the ending scale of the track at the target value to obtain multiple light source image data. In some optional embodiments, as Figure 5 shown, Figure 5 FIG. is a flowchart of steps for obtaining multiple light source image data provided by an embodiment of the present application, including:

[0107] Step 502: Move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale at a preset moving distance at the target value to obtain multiple light source positions.

[0108] Step 504: For each light source position, obtain the light source image data corresponding to the light source position to obtain multiple light source image data.

[0109] Among them, the preset moving distance can be customarily set in advance. Exemplarily, it can be set to 2 cm, so that the solar-blind ultraviolet target light source can be moved from the starting scale of the track to the ending scale at the preset moving distance to obtain multiple light source positions. Finally, the solar-blind ultraviolet target light source will be moved to the end of the track, and corresponding light source image data can be captured by the solar-blind ultraviolet camera at each light source position.

[0110] Therefore, during the process of moving the solar-blind ultraviolet target light source to the end of the track, for each light source position, obtain the light source image data corresponding to the light source position to obtain multiple light source image data, so that the object-image position relationship of the solar-blind ultraviolet camera can be obtained based on the multiple light source image data and the multiple light source positions.

[0111] Next, the first mapping relationship between the point target physical position corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after being imaged by the solar-blind ultraviolet camera can be obtained based on the object-image position relationship. Among them, the point target physical position is the light source position during the movement of the solar-blind ultraviolet target light source, the actual image point coordinates are the image point coordinates in the light source image data after being imaged by the solar-blind ultraviolet camera, and the first mapping relationship is the one-to-one correspondence between the point target physical position and the actual image point coordinates.

[0112] Step 204: Obtain the second mapping relationship between the point target physical position and the preset image point theoretical coordinates.

[0113] Among them, the preset theoretical coordinate of the image point is the theoretical coordinate of the image point after non-wide-angle imaging with equal focal length, that is, the ideal image point coordinate without distortion. The specific generation process can refer to the existing technology and will not be elaborated here. The second mapping relationship is the one-to-one correspondence between the physical position of the point target and the preset theoretical coordinate of the image point.

[0114] Step 206: Correct the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image.

[0115] Among them, in some optional embodiments, such as Figure 6 shown, Figure 6 is a flowchart of steps for generating a corrected solar-blind ultraviolet image provided by an embodiment of the present application, including:

[0116] Step 602: Determine the third mapping relationship between the actual image point coordinate and the preset theoretical coordinate of the image point based on the first mapping relationship and the second mapping relationship, and obtain a distortion correction matrix based on the third mapping relationship.

[0117] Step 604: Correct the original distorted solar-blind ultraviolet image based on the distortion correction matrix to generate a corrected solar-blind ultraviolet image.

[0118] Among them, through the first mapping relationship and the second mapping relationship, the actual image point coordinate and the preset theoretical coordinate of the image point can be related, so that the third mapping relationship between the actual image point coordinate and the preset theoretical coordinate of the image point can be determined. Optionally, the third mapping relationship between the actual image point coordinate and the preset theoretical coordinate of the image point can be established by a preset interpolation algorithm. The type of the preset interpolation algorithm is not specifically limited in the embodiments of the present application.

[0119] Based on the third mapping relationship, a distortion correction matrix at a specific imaging distance Lc can be obtained. When the solar-blind ultraviolet camera images, the original distorted solar-blind ultraviolet image can be corrected based on the distortion correction matrix, so as to generate a corrected solar-blind ultraviolet image.

[0120] Step 208: Fuse the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset relationship between the solar-blind and visible light image point pixel coordinates to generate a fused image.

[0121] Among them, the corrected visible light image can be obtained by using the "Zhang's calibration method" to obtain the corresponding distortion correction parameters at the same imaging distance, and the undistorted corrected visible light image can be calculated by using the standard functions in OpenCV.

[0122] In some optional embodiments, such as Figure 7 shown, Figure 7 is a flowchart of steps for generating the preset relationship between the solar-blind and visible light image point pixel coordinates provided by an embodiment of the present application, including:

[0123] Step 702: Obtain the first collinearity relationship between the preset object point and the image point of the preset solar-blind ultraviolet image.

[0124] Step 704: Obtain the second collinearity relationship between the preset object point and the image point of the preset visible light image.

[0125] Step 706: Based on the first collinearity relationship and the second collinearity relationship, construct the fourth mapping relationship between the image point of the preset solar-blind ultraviolet image and the image point of the preset visible light image, and use the fourth mapping relationship as the pixel coordinate relationship between the preset solar-blind and visible light image points.

[0126] Among them, in some optional embodiments, as Figure 8 shown, Figure 8 is a flowchart of steps for obtaining the first collinearity relationship provided by an embodiment of the present application, including:

[0127] Step 802: Taking the solar-blind ultraviolet camera as a reference, establish a world coordinate system, and use the origin of the world coordinate system as the optical center of the solar-blind ultraviolet camera.

[0128] Step 804: Obtain the preset first image scaling ratio, the preset pixel size of the solar-blind ultraviolet camera, the coordinates of the image point of the preset solar-blind ultraviolet image, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera.

[0129] Step 806: Based on the preset first image scaling ratio, the preset pixel size of the solar-blind ultraviolet camera, the coordinates of the image point of the preset solar-blind ultraviolet image, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera, calculate the image space coordinates corresponding to the image point of the preset solar-blind ultraviolet image.

[0130] Step 808: Based on the image space coordinates corresponding to the image point of the preset solar-blind ultraviolet image and the first preset collinearity equation, calculate the first collinearity relationship.

[0131] In some optional embodiments, as Figure 9 shown, Figure 9 is a flowchart of steps for obtaining the second collinearity relationship provided by an embodiment of the present application, including:

[0132] Step 902: Based on the parallel relationship between the optical axis of the solar-blind ultraviolet camera and the optical axis of the visible light camera, determine the optical center of the visible light camera.

[0133] Step 904: Obtain the preset second image scaling ratio, the preset pixel size of the visible light camera, the coordinates of the image point of the preset visible light image, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera.

[0134] Step 906: Calculate the corresponding image space coordinates of the preset visible light image point based on the preset second image reduction ratio, the preset visible light camera pixel size, the coordinates of the preset visible light image point, and the coordinates of the center point of the visible light image corresponding to the optical center of the visible light camera.

[0135] Step 908: Calculate the second collinearity relationship based on the image space coordinates corresponding to the preset visible light image point and the second preset collinearity equation.

[0136] Among them, the solar-blind ultraviolet camera and the visible light camera adopt an imaging system with parallel optical axes and independent optical channels, with different resolutions and fields of view, and different lens parameters and distortion degrees. When performing image registration and fusion, a spatial geometric model needs to be constructed based on the spatial position relationship between the two imaging cameras, the target distance, the image resolution, and the optical focal length and other internal and external orientation elements to establish the mathematical relationship of the dual-band image pixels. As Figure 10 shown, Figure 10 is a schematic diagram of the geometric positions of a solar-blind ultraviolet camera and a visible light camera provided by an embodiment of the present application. The ultimate goal is to establish the mapping relationship between P 0 and P 1 through the collinearity relationship of the same preset object point P and the two image points P 0 and P 1 .

[0137] Taking the solar-blind ultraviolet camera as the reference, establish the above OXYZ coordinate system, with the optical center of the solar-blind ultraviolet camera as the origin O(0, 0, 0), and the XY directions are the same as the XY directions of the solar-blind positive image plane respectively, establish a right-handed coordinate system, and determine the Z axis. And taking OXYZ as the world coordinate system, it should be noted that the world coordinate system here is the object space coordinate system.

[0138] In the case where the optical axes of the visible camera and the solar-blind camera are parallel, assume that the optical center of the visible camera is O1, and its world coordinates are (dx, dy, dz), that is, there is only a translation relationship between the positions of the two cameras and no rotation.

[0139] Assume that the world coordinates of the preset object point P are (P x , P y , P z ), the focal length of the solar-blind ultraviolet camera is f 0 , the focal length of the visible camera is f 1 , the distance between the object point P and the solar-blind ultraviolet camera is D, that is, P z = D, and the distance from the visible light camera is D - dz. The image point corresponding to the object point P in the solar-blind ultraviolet camera is P 0 , and the image point corresponding to it in the visible light camera is P 1 . The center point of the solar-blind ultraviolet image corresponding to the optical center O is C 0 (C 0m, C 0n ),optical center O 1 The corresponding center point of the visible light image is C 1 (C 1m , C 1n ),The pixel size of the preset solar-blind ultraviolet camera is denoted as Pitch 0 ,The ground sampling distance corresponding to the preset solar-blind ultraviolet camera is denoted as GSD 0 ,The pixel size of the preset visible light camera is denoted as Pitch 1 ,The ground sampling distance corresponding to the preset visible light camera is denoted as GSD 1 .

[0140] It should be noted that since the solar-blind ultraviolet camera uses an ICMOS camera, that is, through a light cone and CMOS coupling, the optical image plane of the solar-blind ultraviolet camera is located at the II cathode and does not overlap with the final detector target plane. The physical size of the solar-blind camera image needs to consider the light cone reduction ratio. Since both are based on the image plane, the physical size of the image plane = the pixel size of the preset solar-blind ultraviolet camera * the preset first image reduction ratio Mag

[0141] Assume that the coordinates of the preset solar-blind ultraviolet image pixel corresponding to the solar-blind image point P0 are (P 0m , P 0n ), where m and n are pixel column indices and row indices starting from 0. Then the image space coordinates of the solar-blind image point P0 are: P 0x = Mag * Pitch 0 * (P 0m + 0.5 - C 0m ), P 0y = Mag * Pitch 0 * (P 0n + 0.5 - C 0n ), P 0z = f 0 .

[0142] For the solar-blind ultraviolet camera, its photographic center coincides with the origin of the object space coordinate system. Since the image space auxiliary coordinate system takes the photographic center as the origin and its XYZ axis directions are the same as those of the object coordinate system, the object coordinate system, image space coordinate system and image space auxiliary coordinate system of the solar-blind ultraviolet camera are the same. Therefore, the first collinearity relationship can be calculated based on the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel and the first preset collinearity equation Among them, the first preset collinearity equation is a mathematical relationship expressing that three points, namely the preset object point, the solar-blind image point and the photographic center of the solar-blind ultraviolet camera, are on a straight line

[0143] Visible light image pixel P 1The coordinates of the corresponding preset solar-blind ultraviolet image pixel are (P 1m , P 1n ). Since there is only a translational relationship between the two cameras, their image space coordinate systems coincide with the auxiliary image space coordinate systems. Therefore, the image space coordinates and auxiliary image space coordinates of the pixel are: P 1x = Pitch 1 * (P 1m + 0.5 - C 1m ), P 1y = Pitch 1 * (P 1n + 0.5 - C 1n ), P 1z = f 1 .

[0144] Based on the image space coordinates corresponding to the preset visible light image pixel and the second preset collinearity equation, the object space coordinates of the projection center of the visible light camera are (dx, dy, dz). The following second collinearity relationship exists between the auxiliary image space coordinates of the visible light image pixel P 1 and the object space coordinates of the object point P: Among them, the second preset collinearity equation is a mathematical relationship expressing that the three points of the preset object point, the visible light image pixel, and the projection center of the visible light camera are located on a straight line.

[0145] Finally, based on the first collinearity relationship and the second collinearity relationship, a fourth mapping relationship between the preset solar-blind ultraviolet image pixel and the preset visible light image pixel can be constructed, and the fourth mapping relationship is used as the relationship between the preset solar-blind and visible light pixel coordinates in formulas (1) and (2).

[0146]

[0147] After obtaining the relationship between the preset solar-blind and visible light pixel coordinates, the solar-blind ultraviolet image can be mapped and transformed into an image with the same size and resolution as the visible light image. In the mapping calculation process, in fact, according to the solar-blind ultraviolet image pixels and other parameters, substituting them into the above formulas (1) and (2), the fused image is calculated. During the calculation process, if the coordinates of (P 1m , P 1n ) do not meet the integer requirements, an image with the same size corresponding to the visible light image pixels can be obtained by interpolation. Finally, the corrected solar-blind ultraviolet image and the corrected visible light image are fused according to the weighted calculation method.

[0148] The present application provides an image fusion method for a solar-blind and visible light binocular heterologous wide-angle imaging system. The method includes: based on the object-image position relationship between the solar-blind ultraviolet target light source and the precise track with scale for measuring the solar-blind ultraviolet camera, and obtaining a first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship, obtaining a second mapping relationship between the physical position of the point target and the preset theoretical coordinates of the image point, correcting the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image, and performing a fusion process on the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset pixel coordinate relationship between the solar-blind and visible light image points to generate a fused image. Through the solar-blind ultraviolet target light source and the precise track with scale in this solution, the object-image position relationship can be measured more accurately and efficiently directly, so that the first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera can be quickly determined. Then, the original distorted solar-blind ultraviolet image can be corrected based on the first mapping relationship and the second mapping relationship, improving the accuracy and efficiency of generating the corrected solar-blind ultraviolet image. In addition, by performing a fusion process on the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset pixel coordinate relationship between the solar-blind and visible light image points, the problem that the traditional registration and fusion methods based on gray scale or geometric features are not applicable due to different imaging principles and image characteristics between solar-blind ultraviolet and visible light is solved, and the flexibility, accuracy, and efficiency of generating the fused image are improved.

[0149] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0150] Figure 11 It is a structural block diagram of an image fusion device for a solar-blind and visible light binocular heterologous wide-angle imaging system provided by an embodiment of the present application.

[0151] As Figure 11 shown, the image fusion device 1100 of the solar-blind and visible light binocular heterologous wide-angle imaging system includes:

[0152] The first acquisition module 1102 is configured to measure the object-image position relationship of the solar-blind ultraviolet camera with a scale-based precision orbit based on the solar-blind ultraviolet target light source, and obtain a first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship.

[0153] The second acquisition module 1104 is configured to obtain a second mapping relationship between the physical position of the point target and the theoretical coordinates of the preset image point.

[0154] The first generation module 1106 is configured to perform correction processing on the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image.

[0155] The second generation module 1108 is configured to perform fusion processing on the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset relationship between the pixel coordinates of the solar-blind and visible light image points to generate a fused image.

[0156] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here. Each module in the above image fusion device for the solar-blind and visible light binocular heterologous wide-angle imaging system can be implemented in whole or in part by software, hardware, and their combinations. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations of the above modules.

[0157] In an embodiment of the present application, a computer device is provided. The computer device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0158] Measure the object-image position relationship of the solar-blind ultraviolet camera with a scale-based precision orbit based on the solar-blind ultraviolet target light source, and obtain a first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging by the solar-blind ultraviolet camera based on the object-image position relationship;

[0159] Obtain a second mapping relationship between the physical position of the point target and the theoretical coordinates of the preset image point;

[0160] Perform correction processing on the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image;

[0161] Perform fusion processing on the corrected solar-blind ultraviolet image and the corrected visible light image based on the preset relationship between the pixel coordinates of the solar-blind and visible light image points to generate a fused image.

[0162] In an embodiment of the present application, when the processor executes the computer program, the following steps are further implemented:

[0163] Adjust the supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable aperture to the target values; wherein, the solar-blind ultraviolet camera is located on the central axis of the solar-blind ultraviolet target light source and faces the center of the adjustable aperture, the solar-blind ultraviolet target light source is fixed on a precision track with scales, and the optical axis of the solar-blind ultraviolet camera is perpendicular to the track;

[0164] At the target values, move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale to obtain a plurality of light source image data;

[0165] Based on the plurality of light source image data, obtain the object-image position relationship of the solar-blind ultraviolet camera.

[0166] In an embodiment of the present application, when the processor executes the computer program, the following steps are further implemented:

[0167] At the target values, move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale by a preset moving distance to obtain a plurality of light source positions;

[0168] For each light source position, obtain the light source image data corresponding to the light source position to obtain a plurality of light source image data.

[0169] In an embodiment of the present application, when the processor executes the computer program, the following steps are further implemented:

[0170] Based on the first mapping relationship and the second mapping relationship, determine the third mapping relationship between the actual image point coordinates and the preset image point theoretical coordinates, and obtain the distortion correction matrix based on the third mapping relationship;

[0171] Based on the distortion correction matrix, perform correction processing on the original distorted solar-blind ultraviolet image to generate a corrected solar-blind ultraviolet image.

[0172] In an embodiment of the present application, when the processor executes the computer program, the following steps are further implemented:

[0173] Obtain the first collinear relationship between the preset object point and the preset solar-blind ultraviolet image point;

[0174] Obtain the second collinear relationship between the preset object point and the preset visible light image point;

[0175] Based on the first collinear relationship and the second collinear relationship, construct the fourth mapping relationship between the preset solar-blind ultraviolet image point and the preset visible light image point, and use the fourth mapping relationship as the preset solar-blind and visible light image point pixel coordinate relationship.

[0176] In an embodiment of the present application, when the processor executes the computer program, the following steps are further implemented:

[0177] Taking the solar-blind ultraviolet camera as a reference, establish a world coordinate system, and use the origin of the world coordinate system as the optical center of the solar-blind ultraviolet camera;

[0178] Obtain a preset first image scaling ratio, a preset solar-blind ultraviolet camera pixel size, the coordinates of a preset solar-blind ultraviolet image pixel point, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera;

[0179] Based on the preset first image scaling ratio, the preset solar-blind ultraviolet camera pixel size, the coordinates of the preset solar-blind ultraviolet image pixel point, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera, calculate the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel point;

[0180] Based on the image space coordinates corresponding to the preset solar-blind ultraviolet image pixel point and the first preset collinearity equation, calculate the first collinearity relationship.

[0181] In an embodiment of the present application, when the processor executes the computer program, the following steps are further implemented:

[0182] Based on the parallel relationship between the optical axis of the solar-blind ultraviolet camera and the optical axis of the visible light camera, determine the optical center of the visible light camera;

[0183] Obtain a preset second image scaling ratio, a preset visible light camera pixel size, the coordinates of a preset visible light image pixel point, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera;

[0184] Based on the preset second image scaling ratio, the preset visible light camera pixel size, the coordinates of the preset visible light image pixel point, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera, calculate the image space coordinates corresponding to the preset visible light image pixel point;

[0185] Based on the image space coordinates corresponding to the preset visible light image pixel point and the second preset collinearity equation, calculate the second collinearity relationship.

[0186] The computer device provided by the embodiment of the present application has the same implementation principle and technical effects as the above method embodiment, and will not be elaborated here.

[0187] In an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0188] Based on the object-image position relationship between the solar-blind ultraviolet target light source and the precise track with scales, measure the solar-blind ultraviolet camera, and based on the object-image position relationship, obtain the first mapping relationship between the physical position of the point target corresponding to the solar-blind ultraviolet target light source and the actual image point coordinates after imaging through the solar-blind ultraviolet camera;

[0189] Obtain the second mapping relationship between the physical position of the point target and the theoretical coordinates of the preset image point;

[0190] Perform correction processing on the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image;

[0191] Perform fusion processing on the corrected solar-blind ultraviolet image and the corrected visible light image based on the relationship between the pixel coordinates of the preset solar-blind and visible light image points to generate a fused image.

[0192] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:

[0193] Adjust the supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable diaphragm to the target values; wherein, the solar-blind ultraviolet camera is located on the central axis of the solar-blind ultraviolet target light source and faces the center of the adjustable diaphragm, the solar-blind ultraviolet target light source is fixed on a precision track with scales, and the optical axis of the solar-blind ultraviolet camera is perpendicular to the track;

[0194] Move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale at the target values to obtain multiple light source image data;

[0195] Obtain the object-image position relationship of the solar-blind ultraviolet camera based on the multiple light source image data.

[0196] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:

[0197] Move the solar-blind ultraviolet target light source from the starting scale of the track to the ending scale at the target values with a preset moving distance to obtain multiple light source positions;

[0198] For each light source position, obtain the light source image data corresponding to the light source position to obtain multiple light source image data.

[0199] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:

[0200] Determine the third mapping relationship between the actual image point coordinates and the theoretical coordinates of the preset image point based on the first mapping relationship and the second mapping relationship, and obtain the distortion correction matrix based on the third mapping relationship;

[0201] Perform correction processing on the original distorted solar-blind ultraviolet image based on the distortion correction matrix to generate a corrected solar-blind ultraviolet image.

[0202] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:

[0203] Obtain the first collinearity relationship between the preset object point and the image point of the preset solar-blind ultraviolet image;

[0204] Obtain the second collinearity relationship between the preset object point and the image point of the preset visible light image;

[0205] Based on the first collinearity relationship and the second collinearity relationship, construct the fourth mapping relationship between the image point of the preset solar-blind ultraviolet image and the image point of the preset visible light image, and use the fourth mapping relationship as the pixel coordinate relationship between the preset solar-blind and visible light image points.

[0206] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:

[0207] Taking the solar-blind ultraviolet camera as a reference, establish a world coordinate system, and use the origin of the world coordinate system as the optical center of the solar-blind ultraviolet camera;

[0208] Obtain the preset first image reduction ratio, the preset pixel size of the solar-blind ultraviolet camera, the coordinates of the image point of the preset solar-blind ultraviolet image, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera;

[0209] Based on the preset first image reduction ratio, the preset pixel size of the solar-blind ultraviolet camera, the coordinates of the image point of the preset solar-blind ultraviolet image, and the coordinates of the central point of the solar-blind image corresponding to the optical center of the solar-blind ultraviolet camera, calculate the image space coordinates corresponding to the image point of the preset solar-blind ultraviolet image;

[0210] Based on the image space coordinates corresponding to the preset solar-blind ultraviolet image point and the first preset collinearity equation, calculate the first collinearity relationship.

[0211] In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented:

[0212] Based on the parallel relationship between the optical axis of the solar-blind ultraviolet camera and the optical axis of the visible light camera, determine the optical center of the visible light camera;

[0213] Obtain the preset second image reduction ratio, the preset pixel size of the visible light camera, the coordinates of the image point of the preset visible light image, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera;

[0214] Based on the preset second image reduction ratio, the preset pixel size of the visible light camera, the coordinates of the image point of the preset visible light image, and the coordinates of the central point of the visible light image corresponding to the optical center of the visible light camera, calculate the image space coordinates corresponding to the image point of the preset visible light image;

[0215] Based on the image space coordinates corresponding to the preset visible light image point and the second preset collinearity equation, calculate the second collinearity relationship.

[0216] The computer-readable storage medium provided in this embodiment has the same implementation principle and technical effects as those of the above method embodiment, and will not be elaborated herein.

[0217] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. This computer program can be stored in a non-volatile computer-readable storage medium. When this computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0218] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0219] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image fusion method for solar blind and visible light binocular heterogeneous wide-angle imaging systems, characterized in that: The method comprises: Measuring the object-image position relationship of a day-blind ultraviolet target light source and a scaled precision track of a day-blind ultraviolet camera, and acquiring a first mapping relationship between the point target physical position corresponding to the day-blind ultraviolet target light source and the actual image point coordinates after imaging by the day-blind ultraviolet camera based on the object-image position relationship; Acquire a second mapping relationship between the physical position of the point target and the preset image point theoretical coordinates; Based on the first mapping relationship and the second mapping relationship, the original distorted solar-blind ultraviolet image is corrected to generate a corrected solar-blind ultraviolet image; The corrected solar-blind ultraviolet image and the corrected visible light image are fused based on a preset solar-blind and visible light image point pixel coordinate relationship to generate a fused image.

2. The method according to claim 1, characterized in that The object-image position relationship based on the solar-blind ultraviolet target light source and the precision track measurement solar-blind ultraviolet camera with scale includes: The power supply voltage of the solar-blind ultraviolet target light source and the size of the adjustable aperture are adjusted to target values; wherein the solar-blind ultraviolet camera is located on the central axis of the solar-blind ultraviolet target light source and directly faces the center of the adjustable aperture; the solar-blind ultraviolet target light source is fixed on a precision track with a scale; and the optical axis of the solar-blind ultraviolet camera is perpendicular to the track; Moving the solar-blind ultraviolet target light source from the starting scale to the ending scale of the track under the target value to obtain a plurality of light source image data; The object-image position relationship of the solar-blind ultraviolet camera is obtained based on the multiple light source image data.

3. The method according to claim 2, characterized in that The solar-blind ultraviolet target light source is moved from the starting scale of the track to the ending scale under the target value to obtain a plurality of light source image data, including: At the target value, the solar-blind ultraviolet target light source is moved from the starting scale of the track to the ending scale by a preset moving distance to obtain a plurality of light source positions; For each of the light source positions, light source image data corresponding to the light source position is acquired to obtain a plurality of light source image data.

4. The method according to any one of claims 1 to 3, characterized in that: The correcting process is performed on the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image, including: Determining a third mapping relationship between the actual image point coordinates and the preset image point theoretical coordinates based on the first mapping relationship and the second mapping relationship, and obtaining a distortion correction matrix based on the third mapping relationship; The original distorted solar-blind ultraviolet image is corrected based on the distortion correction matrix to generate a corrected solar-blind ultraviolet image.

5. The method according to any one of claims 1 to 3, characterized in that: The process of generating the preset relationship between the solar blindness and the pixel coordinates of the visible light image point includes: Acquiring a first collinear relationship between a preset object point and a preset solar-blind ultraviolet image point; Acquiring a second collinear relationship between the preset object point and the preset visible light image point; Based on the first collinear relationship and the second collinear relationship, a fourth mapping relationship between the preset day-blind ultraviolet image pixels and the preset visible light image pixels is constructed, and the fourth mapping relationship is used as the preset day-blind and visible light image pixel coordinate relationship.

6. The method according to claim 5, characterized in that The step of obtaining a first collinear relationship between a preset object point and a preset solar-blind ultraviolet image point includes: Taking the solar-blind ultraviolet camera as a reference, a world coordinate system is established, and the origin of the world coordinate system is used as the optical center of the solar-blind ultraviolet camera; Obtaining a preset first image scaling ratio, a preset solar-blind ultraviolet camera pixel size, coordinates of image points of the preset solar-blind ultraviolet image, and coordinates of a solar-blind image center point corresponding to the optical center of the solar-blind ultraviolet camera; Calculate the image space coordinates corresponding to the preset solar-blind ultraviolet image point based on the preset first image reduction ratio, the preset solar-blind ultraviolet camera pixel size, the coordinates of the preset solar-blind ultraviolet image point, and the coordinates of the solar-blind image center point corresponding to the optical center of the solar-blind ultraviolet camera; The first collinear relationship is calculated based on the image space coordinates corresponding to the preset solar-blind ultraviolet image points and the first preset collinearity equation.

7. The method according to claim 5, characterized in that The acquiring of a second collinear relationship between the preset object point and the preset visible light image point comprises: Determining the optical center of the visible light camera based on the parallel relationship between the optical axis of the solar-blind ultraviolet camera and the optical axis of the visible light camera; Obtaining a preset second image scaling ratio, a preset visible light camera pixel size, coordinates of image points of the preset visible light image, and coordinates of a center point of the visible light image corresponding to the optical center of the visible light camera; Calculating the image space coordinates corresponding to the preset visible light image point based on the preset second image reduction ratio, the preset visible light camera pixel size, the coordinates of the preset visible light image point, and the coordinates of the visible light image center point corresponding to the optical center of the visible light camera; The second collinear relationship is calculated based on the image space coordinates corresponding to the preset visible light image points and the second preset collinearity equation.

8. An image fusion device for solar blind and visible light binocular heterogeneous wide-angle imaging systems, characterized in that: The device comprises: A first acquisition module is used to measure the object-image position relationship of the day-blind ultraviolet target light source and the precision track with scale of the day-blind ultraviolet camera, and obtain a first mapping relationship between the point target physical position corresponding to the day-blind ultraviolet target light source and the actual image point coordinates after imaging by the day-blind ultraviolet camera based on the object-image position relationship; A second acquisition module, used to acquire a second mapping relationship between the physical position of the point target and the preset image point theoretical coordinates; A first generating module is used to perform correction processing on the original distorted solar-blind ultraviolet image based on the first mapping relationship and the second mapping relationship to generate a corrected solar-blind ultraviolet image; The second generating module is used to fuse the corrected solar-blind ultraviolet image and the corrected visible light image based on a preset solar-blind and visible light image point pixel coordinate relationship to generate a fused image.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the image fusion method of the day-blind and visible light binocular heterogeneous wide-angle imaging system as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the image fusion method of the day-blind and visible light binocular heterogeneous wide-angle imaging system as described in any one of claims 1 to 7.

Images