Image fusion method, system and related equipment for multispectral camera

Abstract

The present invention is applicable to the field of computer vision technology, and in particular, relates to an image fusion method, system and related equipment for a multispectral camera. The image fusion method for a multispectral camera obtains a calibration image through a multispectral camera; calculates the edge diffusion function of the degraded channel in the calibration image through a first preset rule; calculates the point diffusion function according to the edge diffusion function; calculates the distribution value within a preset pixel range according to the point diffusion function as a convolution kernel; uses the channel whose central wavelength interval of the calibration image is within the preset interval as a reference channel, and performs a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data; performs a multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image. Compared with the prior art, the present invention can achieve channel fusion with lower computational complexity while retaining accurate spectral information, and is suitable for imaging scenes with high immediacy requirements.

Description

Technical Field

[0001] The present invention is applicable to the field of computer vision technology, and in particular relates to an image fusion method, system and related equipment for a multi-spectral camera. Background Art

[0002] A multispectral camera is a device that can image multiple spectral channels simultaneously, and can obtain more spectral information than a traditional three-channel visible light camera. There are many ways to implement a multispectral camera, but due to hardware and software constraints, it is often difficult to control the imaging quality of each channel at the same level, which is reflected in the decrease in spatial resolution of some channels, which is more blurred than normal channels.

[0003] Existing multispectral camera image fusion methods can be divided into two categories: traditional methods and learning-based methods. Traditional image fusion methods use methods such as principal component analysis (PCA) and multi-resolution analysis (MRA) to extract different components of the channel and fuse them. Learning-based image fusion methods use neural networks to learn image features at different scales and fuse them within the network.

[0004] Multispectral image fusion is mainly used in the field of remote sensing imaging, using high spatial resolution panchromatic images to enhance the spatial information of images collected by multispectral cameras. Therefore, most multispectral image fusion methods focus on further improving the quality and spectral accuracy of fused images, and do not have high requirements for image processing speed. In recent years, multispectral cameras have important applications in security monitoring, drone agriculture, criminal investigation and identification, and other fields. These fields often place high demands on the immediacy of multispectral camera imaging. At the same time, the problem of inconsistent imaging quality of each channel of a multispectral camera still exists. Existing multispectral image fusion methods can achieve good results, but the computational complexity is high and cannot meet the immediacy requirements of the application.

[0005] Therefore, there is an urgent need for a new multispectral camera image fusion method, system and related equipment to solve the above technical problems. Summary of the invention

[0006] The present invention provides an image fusion method, system and related equipment for a multispectral camera, aiming to achieve instant multispectral image fusion with lower computational complexity and meet the demand for high frame rate video capture.

[0007] In a first aspect, the present invention provides an image fusion method for a multispectral camera, the image fusion method comprising the following steps:

[0008] S1, obtain calibration images through a multispectral camera;

[0009] S2, calculating the edge diffusion function of the degraded channel in the calibration image by a first preset rule;

[0010] S3, calculating a point spread function according to the edge spread function;

[0011] S4, calculating a distribution value within a preset pixel range according to the point spread function as a convolution kernel;

[0012] S5, selecting a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, performing a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data;

[0013] S6. Perform multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image.

[0014] Preferably, in step S1, a black and white line pair pattern is used as a target, and the target is photographed in the horizontal direction and the vertical direction respectively by the multispectral camera to obtain the calibration image.

[0015] Preferably, the edge diffusion function includes a first edge diffusion function in the horizontal direction and a second edge diffusion function in the vertical direction. The first preset rule is: the first edge diffusion function and the second edge diffusion function are calculated according to the pixel values ​​of the degraded channel in the calibration image in the horizontal direction and the vertical direction perpendicular to the stripe edge. The edge diffusion function is defined as ESF , which satisfies:

[0016] ;

[0017] in, Represents the mean value of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. Represents the standard deviation of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. represents all pixel values ​​perpendicular to the edge of the stripe in the horizontal direction or the vertical direction; the standard deviation and the mean respectively satisfy:

[0018] = , = ;

[0019] in, express The i pixel values, n express The total number of pixel values ​​in .

[0020] Preferably, the point spread function is a two-dimensional Gaussian distribution, and the point spread function is calculated according to the first edge spread function and the second edge spread function, and the point spread function is defined as PSF , which satisfies the following rules:

[0021] ;

[0022] in, They represent the mean of the two-dimensional Gaussian distribution in the horizontal direction and the mean of the two-dimensional Gaussian distribution in the vertical direction, respectively. and They represent the standard deviation of the two-dimensional Gaussian distribution in the horizontal direction and the standard deviation of the two-dimensional Gaussian distribution in the vertical direction respectively.

[0023] Preferably, the preset interval is 100 nm.

[0024] In a second aspect, the present invention further provides an image fusion system for a multispectral camera, comprising:

[0025] An image acquisition module, used for acquiring a calibration image through a multispectral camera;

[0026] An edge diffusion module, used to calculate the edge diffusion function of the degraded channel in the calibration image according to a first preset rule;

[0027] A point spread module, used for calculating a point spread function according to the edge spread function;

[0028] A convolution kernel module, used to calculate a distribution value within a preset pixel range as a convolution kernel according to the point spread function;

[0029] A transformation module, used for selecting a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, performing a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data;

[0030] The inverse transformation module is used to perform multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image.

[0031] Preferably, the edge diffusion function includes a first edge diffusion function in the horizontal direction and a second edge diffusion function in the vertical direction. The first preset rule is: the first edge diffusion function and the second edge diffusion function are calculated according to the pixel values ​​of the degraded channel in the calibration image in the horizontal direction and the vertical direction perpendicular to the stripe edge. The edge diffusion function is defined as ESF , which satisfies:

[0032] ;

[0033] in, Represents the mean value of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. Represents the standard deviation of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. represents all pixel values ​​perpendicular to the edge of the stripe in the horizontal direction or the vertical direction; the standard deviation and the mean respectively satisfy:

[0034] = , = ;

[0035] in, express The i pixel values, n express The total number of pixel values ​​in .

[0036] Preferably, the point spread function is a two-dimensional Gaussian distribution, and the point spread function is calculated according to the first edge spread function and the second edge spread function, and the point spread function is defined as PSF , which satisfies:

[0037] ;

[0038] in, They represent the mean of the two-dimensional Gaussian distribution in the horizontal direction and the mean of the two-dimensional Gaussian distribution in the vertical direction, respectively. and They represent the standard deviation of the two-dimensional Gaussian distribution in the horizontal direction and the standard deviation of the two-dimensional Gaussian distribution in the vertical direction respectively.

[0039] In a third aspect, the present invention further provides a computer device, comprising: a memory, a processor, and an image fusion program for a multispectral camera stored in the memory and executable on the processor, wherein when the processor executes the image fusion program for the multispectral camera, the steps in the image fusion method for the multispectral camera as described in any one of the above embodiments are implemented.

[0040] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which is stored an image fusion program for a multispectral camera, and when the image fusion program for the multispectral camera is executed by a processor, the steps in the image fusion method for a multispectral camera as described in any one of the above embodiments are implemented.

[0041] Compared with the prior art, the present invention obtains a calibration image through a multispectral camera; calculates the edge diffusion function of the degraded channel in the calibration image through a first preset rule; calculates the point diffusion function according to the edge diffusion function; calculates the distribution value within the preset pixel range according to the point diffusion function as a convolution kernel; uses the channel whose central wavelength interval of the calibration image is within the preset interval as a reference channel, performs a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data; performs a multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image. In this way, the present invention can achieve channel fusion with lower computational complexity while retaining accurate spectral information, and is suitable for imaging scenes with high immediacy requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be described in detail below in conjunction with the accompanying drawings. The above and other aspects of the present invention will become clearer and easier to understand through the detailed description made in conjunction with the following drawings. In the accompanying drawings:

[0043] Figure 1 is a flowchart of an image fusion method for a multi-spectral camera provided by an embodiment of the present invention;

[0044] Figure 2 is a structural schematic diagram of an image fusion system for a multispectral camera provided by an embodiment of the present invention;

[0045] Figure 3 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] Embodiment 1

[0048] Please refer to Figure 1 The present invention provides an image fusion method for a multi-spectral camera, the image fusion method comprising the following steps:

[0049] S1. Obtain calibration images through a multispectral camera.

[0050] In the embodiment of the present invention, in order to obtain the point spread function in the subsequent step S3 PSFA very small point light source is required, which is difficult to achieve. Therefore, in step S1, a black and white line pattern with straight and sharp edges is used as a target, and the target is photographed in the horizontal and vertical directions by the multispectral camera to obtain the calibration image. Specifically, the target refers to a specially designed target that can produce recognizable image patterns or features in multiple spectral bands. These patterns or features are captured by the multispectral camera and used to test important performance indicators such as the camera's resolution, alignment accuracy, and color reproduction ability.

[0051] S2. Calculate the edge spread function (ESF) of the degraded channel in the calibration image according to a first preset rule.

[0052] In an embodiment of the present invention, an edge diffusion function is a function that describes the degree of diffusion of an image edge, and is used to evaluate the performance of an imaging system, especially in an optical imaging system. A degraded channel refers to a channel in a calibration image that is affected by image degradation factors such as blur and has a significantly reduced imaging quality. The edge diffusion function includes a first edge diffusion function in the horizontal direction and a second edge diffusion function in the vertical direction. The first preset rule is: the first edge diffusion function and the second edge diffusion function are calculated based on the pixel values ​​of the degraded channel in the calibration image that are perpendicular to the stripe edge in the horizontal and vertical directions. The edge diffusion function is defined as ESF , which satisfies:

[0053] ;

[0054] in, Represents the mean value of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. Represents the standard deviation of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. represents all pixel values ​​perpendicular to the edge of the stripe in the horizontal direction or the vertical direction; the standard deviation and the mean respectively satisfy:

[0055] = , = ;

[0056] in, express The i pixel values, n express The total number of pixel values ​​in .

[0057] It should be noted that the edge diffusion function can calculate the pixel value perpendicular to the stripe edge in any direction of the degraded channel in the calibration image, and is not limited to the horizontal direction and the vertical direction.

[0058] S3. Calculate a point spread function (PSF) according to the edge spread function.

[0059] In an embodiment of the present invention, the point spread function is used to evaluate the diffusion degree of the imaging system for a point light source or a point object. The point spread function is calculated according to the first edge spread function and the second edge spread function. The point spread function is defined as: PSF , which satisfies the following rules:

[0060] ;

[0061] in, They represent the mean of the two-dimensional Gaussian distribution in the horizontal direction and the mean of the two-dimensional Gaussian distribution in the vertical direction, respectively. and They represent the standard deviation of the two-dimensional Gaussian distribution in the horizontal direction and the standard deviation of the two-dimensional Gaussian distribution in the vertical direction respectively.

[0062] It should be noted that the edge diffusion function in any different directions can be used to establish a two-dimensional Gaussian distribution as a point diffusion function, and is not limited to the vertical and horizontal directions.

[0063] S4. Calculate a distribution value within a preset pixel range according to the point spread function as a convolution kernel.

[0064] In the embodiment of the present invention, the size of the preset pixel range can be modified according to actual conditions.

[0065] S5, selecting a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, and performing a multi-resolution analysis (MRA) on the image of the reference channel according to the convolution kernel to obtain detail component data;

[0066] In an embodiment of the present invention, the preset interval is 100nm, and a normal image quality channel with a central wavelength interval of less than 100nm in the spectral channel to be fused is selected as the reference channel. At this time, the spatial information of the reference channel is basically consistent with that of the channel to be fused, and only differs in relative intensity. Therefore, when performing multi-resolution inverse transformation in the subsequent step S6, a better effect is achieved.

[0067] In the embodiment of the present invention, the image of the reference channel is subjected to multi-resolution transformation according to the convolution kernel, and detail component data and background component data in the image of the reference channel can be obtained. When performing multi-resolution transformation, the background component data is replaced by the image of the channel to be fused for transformation. Define the reference channel image as a, the convolution kernel as w, the background component data as a0 and the detail component data as d0, then:

[0068] { a0, d0} = AWT(a;w,1);

[0069] AWT stands for Multi-Resolution Transform.

[0070] Specifically, multi-resolution transforms include Discrete Wavelet Transform (DWT), Laplacian Pyramid (LP) and Atrous Wavelet Transform (AWT). Taking AWT as an example, the basic process of AWT is as follows:

[0071] 1. Take the reference channel image a0, the initial convolution kernel w, the total number of transformations N as input, and initialize the number of transformations j to 1;

[0072] 2. Upsample the initial convolution kernel w to 2 j -1 times the size to get w j .

[0073] 3. The background component data a of the jth transformation j It can be obtained from the following formula, where * is the convolution operation:

[0074] ;

[0075] 4. Detailed component data d of the jth transformation j It can be obtained by the following formula:

[0076] ;

[0077] 5. Repeat steps 2 to 4 until j>N to obtain the detail component data d of the image at different scales j And background component dataa N ;

[0078] The above AWT transformation of the reference channel image a0 can be recorded as:

[0079] .

[0080] S6. Perform multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image.

[0081] In the embodiment of the present invention, taking the atrous wavelet transform AWT as an example, the inverse transform of the AWT only needs to add the detail components at different scales to the background component data to restore the original image:

[0082] ;

[0083] Combined with the above processing flow, the background component data a N The fused image can be obtained by replacing the image with the channel to be fused and performing an inverse transform.

[0084] The detail component data of the normal channel is extracted by multi-resolution transformation, and then the detail component data and the image of the channel to be fused are subjected to multi-resolution inverse transformation for image fusion, which can effectively compensate for the distortion of the image of the channel to be fused. The calculation complexity is low and it can adapt to imaging scenarios with immediacy requirements.

[0085] Compared with the prior art, the present invention obtains a calibration image through a multispectral camera; calculates the edge diffusion function of the degraded channel in the calibration image through a first preset rule; calculates the point diffusion function according to the edge diffusion function; calculates the distribution value within the preset pixel range according to the point diffusion function as a convolution kernel; uses the channel whose central wavelength interval of the calibration image is within the preset interval as a reference channel, performs a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data; performs a multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image. In this way, the present invention can achieve channel fusion with lower computational complexity while retaining accurate spectral information, and is suitable for imaging scenes with high immediacy requirements.

[0086] Embodiment 2

[0087] The present invention also provides an image fusion system for a multi-spectral camera. Figure 2 , FIG. is a schematic diagram of the structure of an image fusion system 200 for a multi-spectral camera provided in an embodiment of the present invention, which includes:

[0088] 201. An image acquisition module, used for acquiring a calibration image through a multispectral camera;

[0089] In the embodiment of the present invention, in order to obtain the point spread function in the subsequent step S3 PSF An extremely small point light source is required, which is difficult to realize. Therefore, in step S1, a black and white line pair pattern with straight and sharp edges is used as a target, and the target is photographed in the horizontal direction and the vertical direction respectively by the multispectral camera to obtain the calibration image.

[0090] 202. An edge diffusion module, configured to calculate an edge diffusion function of a degraded channel in the calibration image according to a first preset rule;

[0091] In the embodiment of the present invention, the degraded channel refers to a channel in the calibration image that is affected by image degradation factors such as blur and has a significantly reduced imaging quality. The edge diffusion function includes a first edge diffusion function in the horizontal direction and a second edge diffusion function in the vertical direction. The first preset rule is: the first edge diffusion function and the second edge diffusion function are calculated based on the pixel values ​​of the degraded channel in the calibration image in the horizontal direction and the vertical direction perpendicular to the stripe edge. The edge diffusion function is defined as ESF , which satisfies:

[0092] ;

[0093] in, Represents the mean value of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. Represents the standard deviation of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. represents all pixel values ​​perpendicular to the edge of the stripe in the horizontal direction or the vertical direction; the standard deviation and the mean respectively satisfy:

[0094] = , = ;

[0095] in, express The i pixel values, n express The total number of pixel values ​​in .

[0096] 203. A point spread module, configured to calculate a point spread function according to the edge spread function;

[0097] In an embodiment of the present invention, the point spread function is used to evaluate the diffusion degree of the imaging system for a point light source or a point object. The point spread function is calculated according to the first edge spread function and the second edge spread function. The point spread function is defined as: PSF , which satisfies the following rules:

[0098] ;

[0099] in, They represent the mean of the two-dimensional Gaussian distribution in the horizontal direction and the mean of the two-dimensional Gaussian distribution in the vertical direction, respectively. and They represent the standard deviation of the two-dimensional Gaussian distribution in the horizontal direction and the standard deviation of the two-dimensional Gaussian distribution in the vertical direction respectively.

[0100] It should be noted that the edge diffusion function in any different directions can be used to establish a two-dimensional Gaussian distribution as a point diffusion function, and is not limited to the vertical and horizontal directions.

[0101] 204. A convolution kernel module, configured to calculate a distribution value within a preset pixel range according to the point spread function as a convolution kernel;

[0102] In the embodiment of the present invention, the size of the preset pixel range can be modified according to actual conditions.

[0103] 205. A transformation module, configured to select a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, and perform a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data;

[0104] In the embodiment of the present invention, the image of the reference channel is subjected to multi-resolution transformation according to the convolution kernel, and detail component data and background component data in the image of the reference channel can be obtained. When performing multi-resolution transformation, the background component data is replaced by the image of the channel to be fused for transformation. Define the reference channel image as a, the convolution kernel as w, the background component data as a0 and the detail component data as d0, then:

[0105] ;

[0106] AWT stands for Multi-Resolution Transform.

[0107] Specifically, multi-resolution transforms include Discrete Wavelet Transform (DWT), Laplacian Pyramid (LP) and Atrous Wavelet Transform (AWT). Taking AWT as an example, the basic process of AWT is as follows:

[0108] 1. Take the reference channel image a0, the initial convolution kernel w, the total number of transformations N as input, and initialize the number of transformations j to 1;

[0109] 2. Upsample the initial convolution kernel w to 2 j -1 times the size to get w j .

[0110] 3. The background component data a of the jth transformation j It can be obtained by the following formula, where For the convolution operation:

[0111] ;

[0112] 4. Detailed component data d of the jth transformation j It can be obtained by the following formula:

[0113] ;

[0114] 5. Repeat steps 2 to 4 until j>N to obtain the detail component data d of the image at different scales j And background component dataa N ;

[0115] The above AWT transformation of the reference channel image a0 can be recorded as:

[0116] .

[0117] 206. An inverse transformation module, configured to perform a multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image.

[0118] In the embodiment of the present invention, taking the atrous wavelet transform AWT as an example, the inverse transform of the AWT only needs to add the detail components at different scales to the background component data to restore the original image:

[0119] ;

[0120] Combined with the above processing flow, the background component data a N Replace with to be merged

[0121] The fused image can be obtained by performing inverse transform on the channel images.

[0122] The detail component data of the normal channel is extracted by multi-resolution transformation, and then the detail component data and the image of the channel to be fused are subjected to multi-resolution inverse transformation for image fusion, which can effectively compensate for the distortion of the image of the channel to be fused. The calculation complexity is low and it can adapt to imaging scenarios with immediacy requirements.

[0123] The image fusion system 200 for a multispectral camera can implement the steps in the image fusion method for a multispectral camera in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment, which will not be repeated here.

[0124] Embodiment 3

[0125] The embodiment of the present invention also provides a computer device, please refer to Figure 3 , Figure 3300 is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. The computer device 300 includes: a memory 302, a processor 301, and an image fusion program of a multispectral camera stored in the memory 302 and executable on the processor 301.

[0126] The processor 301 calls the image fusion program of the multi-spectral camera stored in the memory 302 to execute the steps of the image fusion method of the multi-spectral camera provided in the embodiment of the present invention. Figure 1 , specifically including the following steps:

[0127] S1, obtain calibration images through a multispectral camera;

[0128] S2, calculating the edge diffusion function of the degraded channel in the calibration image by a first preset rule;

[0129] S3, calculating a point spread function according to the edge spread function;

[0130] S4, calculating a distribution value within a preset pixel range according to the point spread function as a convolution kernel;

[0131] S5, selecting a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, performing a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data;

[0132] S6. Perform multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image.

[0133] The computer device 300 provided in the embodiment of the present invention can implement the steps in the method in the above embodiment and can achieve the same technical effects. Please refer to the description in the above embodiment and will not be repeated here.

[0134] Embodiment 4

[0135] An embodiment of the present invention further provides a computer-readable storage medium, on which an image fusion program for a multispectral camera is stored. When the image fusion program for the multispectral camera is executed by a processor, the various processes and steps in the image fusion method for a multispectral camera provided in an embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0137] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0139] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only the preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation manner. The above-mentioned specific implementation manner is only illustrative rather than restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. An image fusion method for a multispectral camera, characterized in that: The image fusion method comprises the following steps: S1, obtain calibration images through a multispectral camera; S2, calculating the edge diffusion function of the degraded channel in the calibration image by a first preset rule; S3, calculating a point spread function according to the edge spread function; S4, calculating a distribution value within a preset pixel range according to the point spread function as a convolution kernel; S5, selecting a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, performing a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data; S6, performing a multi-resolution inverse transform on the image of the channel to be fused and the detail component data to obtain a fused image; The edge diffusion function includes a first edge diffusion function in the horizontal direction and a second edge diffusion function in the vertical direction. The first preset rule is: the first edge diffusion function and the second edge diffusion function are calculated according to the pixel values ​​of the degraded channel in the calibration image in the horizontal direction and the vertical direction perpendicular to the stripe edge. The edge diffusion function is defined as ESF , which satisfies: ； in, Represents the mean value of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. Represents the standard deviation of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. represents all pixel values ​​perpendicular to the edge of the stripe in the horizontal direction or the vertical direction; the standard deviation and the mean respectively satisfy: = ， = ； in, express The i pixel values, n express The total number of pixel values ​​in .

2. The image fusion method of a multispectral camera as claimed in claim 1, characterized in that: In step S1, a black and white line pair pattern is used as a target, and the target is photographed in the horizontal direction and the vertical direction respectively by the multispectral camera to obtain the calibration image.

3. The image fusion method of a multispectral camera as claimed in claim 1, characterized in that: The point spread function is calculated according to the first edge spread function and the second edge spread function. The point spread function is a two-dimensional Gaussian distribution. The point spread function is defined as: PSF , which satisfies the following rules: ； in, They represent the mean of the two-dimensional Gaussian distribution in the horizontal direction and the mean of the two-dimensional Gaussian distribution in the vertical direction, respectively. and They represent the standard deviation of the two-dimensional Gaussian distribution in the horizontal direction and the standard deviation of the two-dimensional Gaussian distribution in the vertical direction respectively.

4. The image fusion method of a multispectral camera as claimed in claim 1, characterized in that: The preset interval is 100 nm.

5. An image fusion system for a multispectral camera, characterized in that: include: An image acquisition module, used for acquiring a calibration image through a multispectral camera; An edge diffusion module is used to calculate the edge diffusion function of the degraded channel in the calibration image according to a first preset rule; wherein the edge diffusion function includes a first edge diffusion function in the horizontal direction and a second edge diffusion function in the vertical direction, and the first preset rule is: the first edge diffusion function and the second edge diffusion function are calculated according to the pixel values ​​of the degraded channel in the calibration image in the horizontal direction and the vertical direction perpendicular to the stripe edge, and the edge diffusion function is defined as ESF , which satisfies: ； in, Represents the mean value of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. Represents the standard deviation of the pixel value distribution perpendicular to the edge of the stripe in the horizontal direction or vertical direction. represents all pixel values ​​perpendicular to the edge of the stripe in the horizontal direction or the vertical direction; the standard deviation and the mean respectively satisfy: = ， = ； in, express The i pixel values, n express The total number of pixel values ​​in ; A point spread module, used for calculating a point spread function according to the edge spread function; A convolution kernel module, used to calculate a distribution value within a preset pixel range as a convolution kernel according to the point spread function; A transformation module, used for selecting a channel within a preset interval from the central wavelength of the channel to be fused as a reference channel, performing a multi-resolution transformation on the image of the reference channel according to the convolution kernel to obtain detail component data; The inverse transformation module is used to perform multi-resolution inverse transformation on the image of the channel to be fused and the detail component data to obtain a fused image.

6. The image fusion system of a multi-spectral camera as claimed in claim 5, characterized in that: The point spread function is calculated according to the first edge spread function and the second edge spread function. The point spread function is a two-dimensional Gaussian distribution. The point spread function is defined as: PSF , which satisfies: ； in, They represent the mean in the horizontal direction of the two-dimensional Gaussian distribution and the mean in the vertical direction of the two-dimensional Gaussian distribution, respectively. and They represent the standard deviation of the two-dimensional Gaussian distribution in the horizontal direction and the standard deviation of the two-dimensional Gaussian distribution in the vertical direction respectively.

7. A computer device, characterized in that: include: A memory, a processor, and an image fusion program for a multispectral camera stored in the memory and executable on the processor, wherein the processor implements the steps in the image fusion method for a multispectral camera as claimed in any one of claims 1 to 4 when executing the image fusion program for the multispectral camera.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an image fusion program for a multispectral camera, and when the image fusion program for a multispectral camera is executed by a processor, the steps in the image fusion method for a multispectral camera according to any one of claims 1 to 4 are implemented.

Images