A focusing method, device and equipment of a multispectral imaging module and a storage medium

Abstract

The application discloses a focusing method and device of a multispectral imaging module, equipment and a storage medium, and relates to the optical field, which comprises the following steps: image acquisition is performed by using a camera to be focused to determine a target image; multispectral mixed energy of the target image is split to obtain each spectral energy, and filtering is performed to obtain each filtered energy; a first target operator and a second target operator are used to process each filtered energy to determine a target index curve corresponding to the target image; when the target index curve corresponding to the current image acquisition position is not higher than the target index curve corresponding to the last image acquisition position, the focal length corresponding to the last image acquisition position is determined as an optimal focal length; a new current image acquisition position is determined, and when the target curve corresponding to the current image acquisition position is not lower than the target index curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position is determined as a global optimal focal length. The application improves the focusing effect of the multispectral camera in a high-noise scene.

Description

Technical Field

[0001] This invention relates to the field of optics, and in particular to a focusing method, apparatus, device, and storage medium for a multispectral imaging module. Background Technology

[0002] Currently, when focusing on spatially spectral coupled mosaic cameras, multiple images at different distances relative to the observed object are often acquired. The image with the highest sharpness is used as a reference to acquire images at multiple positions closer to and further away from the object. Finally, the average signal strength of all acquired images is calculated, and the position corresponding to the image with the strongest average signal strength is used as the focus position. Alternatively, an automatic alignment algorithm combining YOLOv5 (You Only Look Once version 5) network intelligent ROI (Region of Interest) and an improved Laplacian autofocus algorithm is used to determine the ROI focus window. The sum of squared gradients after convolving this region with the Laplace operator is used as the sharpness evaluation value for focusing. However, both methods are sensitive to noise in the image. In high-noise scenes, the sharpness index obtained may not reflect the true image quality, resulting in poor focusing performance for multispectral cameras.

[0003] In conclusion, improving the focusing performance of multispectral cameras in high-noise scenarios is a pressing issue that needs to be addressed. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a focusing method, apparatus, device, and storage medium for a multispectral imaging module, which can improve the focusing performance of multispectral cameras in high-noise scenes. The specific solution is as follows:

[0005] In a first aspect, this application discloses a focusing method for a multispectral imaging module, including:

[0006] The current image acquisition position is determined based on the preset focusing direction. At the current image acquisition position, the camera to be focused is used to acquire an image of the standard chart to obtain the original image. The original image is then processed based on the preset target area to determine the target image. The camera to be focused is a mosaic camera.

[0007] The multispectral mixed energy corresponding to the target image is split to obtain the spectral energy corresponding to the target image, and Gaussian filtering and bilateral filtering are performed on each spectral energy in sequence to obtain the corresponding filtered energy.

[0008] A first target operator is used to determine the gradients in each direction corresponding to each filtered energy so as to determine the target average gradient based on the gradients in each direction. A second target operator is used to determine the target variances corresponding to each filtered energy. Based on the target average gradients and the target variances, the target index score corresponding to the target image is determined so as to determine the target index curve corresponding to the target image based on the target index score. The first target operator is an optimized Sobel operator for calculating gradients in multiple directions, and the second target operator is a Laplacian operator.

[0009] Determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and determine the focal length corresponding to the previous image acquisition position as the optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position.

[0010] The opposite direction of the preset focusing direction is taken as the new preset focusing direction. Based on the new preset focusing direction, a new current image acquisition position is determined. When the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position is determined as the global optimal focal length to complete the focusing of the camera to be focused.

[0011] Optionally, the step of acquiring an image of the standard image card using the camera to be focused at the current image acquisition position to obtain the original image includes:

[0012] At the current image acquisition location, images are acquired from the standard chart using the camera to be focused based on a preset exposure time to obtain the original image set.

[0013] Optionally, the step of splitting the multispectral mixture energy corresponding to the target image to obtain the individual spectral energies corresponding to the target image includes:

[0014] Determine the spectral response function of each channel corresponding to the camera to be focused, and determine the corresponding energy contribution ratio based on the area ratio of each channel spectral response function in the corresponding wavelength range;

[0015] The multispectral mixed energy is split based on the energy contribution ratio to obtain the spectral energy corresponding to the target image.

[0016] Optionally, the step of using the first target operator to determine the gradients in each direction corresponding to each of the filtered energies in order to determine the target average gradient based on each of the directional gradients includes:

[0017] The first target operator is used to determine the gradient of each filtered energy in each target direction; the target directions include 0°, 45°, 90° and 315°; the weight of the middle pixel in the first target operator is greater than the weight of the edge pixel;

[0018] The gradient sum of the target image is determined based on the gradients of each of the aforementioned directions, and the average gradient of the target is determined based on the gradient sum.

[0019] Optionally, determining the target index score corresponding to the target image based on the target average gradient and the target variance includes:

[0020] The channel index scores corresponding to each filtered energy are determined based on the target average gradient and the target variance, wherein the formula for determining the channel index scores is as follows:

[0021] ;

[0022] in, The channel index score corresponding to the i-th channel. The average gradient of the target is... Let Variance be the target variance. and These are the weighting coefficients;

[0023] The target index score corresponding to the target image is determined based on the scores of each of the aforementioned channel indexes, wherein the formula for determining the target index score is:

[0024] ;

[0025] in, The target metric score is given by t, where t is the total number of channels.

[0026] Optionally, the focusing method of the multispectral imaging module further includes:

[0027] If the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, then proceed to the step of determining the current image acquisition position based on the preset focusing direction.

[0028] Optionally, the focusing method of the multispectral imaging module further includes:

[0029] If the target index curve corresponding to the current image acquisition position meets the preset focus standard, then the corresponding focus adjustment completion prompt operation is executed; the preset focus standard includes the difference between the target index score corresponding to the current image acquisition position and the target index score corresponding to the optimal focal length being less than a preset threshold.

[0030] Secondly, this application discloses a focusing device for a multispectral imaging module, comprising:

[0031] The image acquisition module is used to determine the current image acquisition position based on a preset focusing direction, acquire an image of a standard chart using a camera to be focused at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera;

[0032] The filtering module is used to decompose the multispectral mixed energy corresponding to the target image to obtain the spectral energy corresponding to the target image, and to perform Gaussian filtering and bilateral filtering on each spectral energy in sequence to obtain the corresponding filtered energy.

[0033] The scoring determination module is used to determine the gradients of each direction corresponding to each of the filtered energies using a first target operator in order to determine the target average gradient based on the gradients of each direction; to determine the target variances of each of the filtered energies using a second target operator; and to determine the target index score corresponding to the target image based on the target average gradients and the target variances, so as to determine the target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating gradients of multiple directions, and the second target operator is a Laplacian operator;

[0034] The first focal length determination module is used to determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, the focal length corresponding to the previous image acquisition position is determined as the optimal focal length.

[0035] The second focal length determination module is used to take the opposite direction of the preset focusing direction as the new preset focusing direction, determine the new current image acquisition position based on the new preset focusing direction, and determine the focal length corresponding to the current image acquisition position as the global optimal focal length when the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, so as to complete the focusing of the camera to be focused.

[0036] Thirdly, this application discloses an electronic device, including:

[0037] Memory, used to store computer programs;

[0038] A processor is used to execute the computer program to implement the aforementioned focusing method of the multispectral imaging module.

[0039] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned focusing method of the multispectral imaging module.

[0040] In this application, when focusing on the camera to be focused, the current image acquisition position is determined based on a preset focusing direction. At the current image acquisition position, the camera to be focused acquires an image of a standard chart to obtain an original image. The original image is then processed based on a preset target region to determine a target image. The camera to be focused is a mosaic camera. The multispectral mixed energy corresponding to the target image is split to obtain the spectral energy corresponding to the target image. Gaussian filtering and bilateral filtering are then applied to each spectral energy sequentially to obtain the corresponding filtered energy. A first target operator is used to determine the gradients of each direction corresponding to each filtered energy so as to determine the target average gradient. A second target operator is used to determine the target variances corresponding to each filtered energy. The target index score corresponding to the target image is determined based on the target average gradient and the target variance. Finally, the target index score is used to determine the target index score. Define the target index curve corresponding to the target image; the first target operator is an optimized Sobel operator used to calculate gradients in multiple directions, and the second target operator is a Laplacian operator; determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, determine the focal length corresponding to the previous image acquisition position as the optimal focal length; take the opposite direction of the preset focusing direction as the new preset focusing direction, determine the new current image acquisition position based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, determine the focal length corresponding to the current image acquisition position as the global optimal focal length to complete the focusing of the camera to be focused. As can be seen, in the scenario of focusing fixture, this application acquires image data from a mosaic camera in real time. By splitting the multi-band multispectral mixed energy in the spatial spectrum coupled image into multispectral single energy, multiple single-band image data are obtained. After preprocessing each single-band image using Gaussian filtering and bilateral filtering, the composite imaging quality index is used to calculate the quality index of each single-band image. Combined with channel weights, a comprehensive imaging index is calculated. Production personnel adjust the focal length according to the changes in the comprehensive imaging quality index curve to achieve the globally optimal focal length of the camera to be focused. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 This is a flowchart of a focusing method for a multispectral imaging module disclosed in this application;

[0043] Figure 2 A schematic diagram of a four-channel spectral response function provided in this application;

[0044] Figure 3 This is a schematic diagram of a specific focusing method for a multispectral imaging module disclosed in this application.

[0045] Figure 4 This is a schematic diagram of a specific focusing method for a multispectral imaging module disclosed in this application.

[0046] Figure 5 This is a schematic diagram of the focusing result of a multispectral imaging module disclosed in this application;

[0047] Figure 6 A schematic diagram illustrating the result of focusing a mosaic camera using a conventional focusing method, as provided in this application;

[0048] Figure 7 This is a comparison image of the focusing results of the B channel after focusing, as disclosed in this application. Figure 7 (a) in the figure is the focusing result obtained by focusing using the focusing method disclosed in this application. Figure 7 (b) in the image shows the focusing result obtained using the traditional focusing method;

[0049] Figure 8 This is a comparison image of the focusing results of the G channel after focusing, as disclosed in this application. Figure 8 (a) in the figure is the focusing result obtained by focusing using the focusing method disclosed in this application. Figure 8 (b) in the image shows the focusing result obtained using the traditional focusing method;

[0050] Figure 9 This is a comparison image of the focusing results of the R channel after focusing, as disclosed in this application. Figure 9 (a) in the figure is the focusing result obtained by focusing using the focusing method disclosed in this application. Figure 9 (b) in the image shows the focusing result obtained using the traditional focusing method;

[0051] Figure 10This is a quantitative comparison table of focusing results between focusing a camera using a conventional method and focusing a camera using the focusing method disclosed in this application.

[0052] Figure 11 This is a schematic diagram of the focusing device structure of a multispectral imaging module disclosed in this application;

[0053] Figure 12 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Currently, when focusing on spatially coupled mosaic cameras, multiple images at different distances relative to the observed object are often acquired. The image with the highest sharpness is used as a reference to acquire images at multiple positions closer to and further away from the observed object. Finally, the average signal strength of all acquired images is calculated, and the position corresponding to the image with the strongest average signal strength is used as the focus position. Alternatively, an automatic alignment algorithm fusing YOLOv5 network intelligent regions of interest (ROIs) and an improved Laplacian autofocus algorithm is used to determine the ROI focus window, and the sum of squared gradients after convolving this region with the Laplace operator is used as the sharpness evaluation value for focusing. However, both methods are sensitive to noise in the image. In high-noise scenes, the sharpness index obtained may not reflect the true image quality, resulting in poor focusing performance for multispectral cameras. To address these technical problems, this application discloses a focusing method for a multispectral imaging module that can improve the focusing performance of multispectral cameras in high-noise scenes.

[0056] See Figure 1 As shown, this embodiment of the invention discloses a focusing method for a multispectral imaging module, including:

[0057] Step S11: Determine the current image acquisition position based on the preset focusing direction, and use the camera to be focused to acquire an image of the standard chart at the current image acquisition position to obtain the original image. Then, process the original image based on the preset target area to determine the target image. The camera to be focused is a mosaic camera.

[0058] In this embodiment, the camera to be focused is a mosaic camera. When focusing the camera in a tooling environment, the focusing direction must first be determined, i.e., a preset focusing direction. In practical applications, production personnel can randomly adjust the camera focal length in one direction. If the index curve decreases, the focus is adjusted in the opposite direction; if the index curve increases, the focus continues in that direction. Then, the current image acquisition position under the preset focusing direction is determined so that the camera to be focused can acquire the original image with the standard chart as the target at the current image acquisition position. In the tooling environment, the image data includes not only the chart content required for focusing but also relay lens edge information, etc. To eliminate interference, the original image can be cropped by setting the ROI area size to obtain effective image data, which serves as input for subsequent processing. That is, the original image can be processed based on the preset target area to determine the target image, where the target image is a part of the original image.

[0059] In this embodiment, images can be acquired from a standard image chart using a camera to be focused at the current image acquisition location based on a preset exposure time to obtain a set of original images. That is, the exposure times of multiple cameras can be set to simulate brightness variations in a real-world scene, such as 0.5 milliseconds, 1 millisecond, 1.5 milliseconds, 2 milliseconds, 2.5 milliseconds, 3 milliseconds, etc., while the gain can be set to 1. In practical applications, only one camera exposure time needs to be set, and the exposure time and gain can be adjusted as needed to avoid overexposure or underexposure.

[0060] Step S12: Decompose the multispectral mixed energy corresponding to the target image to obtain the spectral energy corresponding to the target image, and sequentially perform Gaussian filtering and bilateral filtering on each spectral energy to obtain the corresponding filtered energy.

[0061] In this embodiment, for the energy mixing of a mosaic multispectral camera, the multispectral mixed energy of the multispectral mosaic image can be decomposed into single energies of multiple spectra based on the camera's spectral response function, thereby obtaining multiple single-band image data. In other words, the specific process of decomposing the multispectral mixed energy corresponding to the target image to obtain the spectral energies corresponding to the target image can include: determining the spectral response function of each channel corresponding to the camera to be focused, and determining the corresponding energy contribution ratio based on the area ratio of each channel's spectral response function within the corresponding wavelength range; and decomposing the multispectral mixed energy based on the energy contribution ratio to obtain the spectral energies corresponding to the target image. The spectral response function mainly consists of three parts: optical lens transmittance, multi-band narrowband filter transmittance, and detector quantum efficiency. The calculation formula for the spectral response function of each channel is as follows:

[0062] ;

[0063] in, The transmittance of the optical lens. For a multi-bandpass narrowband filter, is the transmittance function. Let be the quantum efficiency function of the detector. These represent individual channels. The area ratio of the spectral response function of each channel within the corresponding wavelength range is calculated, such as... Figure 2 As shown, the energy contribution ratio of each channel can be obtained, and the calculation formula is as follows:

[0064] ;

[0065] in, for aisle The proportion of energy contribution in the band. Each represents a different channel. This represents the band number of a multi-bandpass narrowband filter. Taking blue, red, green, and infrared light as examples, the energy contribution ratio of each channel can be used. Solve the following system of multivariate linear equations:

[0066] ;

[0067] By solving this system of linear equations, the mixing energy of each channel can be determined. Single spectral energy split into multiple spectra ,in , , and It represents four different channels: blue light, red light, green light, and infrared light.

[0068] In this embodiment, as Figure 3 As shown, after obtaining the spectral energies, each spectral energy can be filtered to eliminate outliers and noise interference in the image, making the image smoother. Specifically, as... Figure 4 As shown, Gaussian filtering can be performed first, followed by bilateral filtering. In one specific implementation, after obtaining the size and data area of ​​the target image, the data area can be copied to a preset buffer, and then a buffer of size 5 can be selected. A Gaussian filter window of size 5 is created, and the pixel values ​​of each point within the window are iteratively obtained to calculate the center pixel value using the Gaussian filter formula. The Gaussian filter formula is as follows:

[0069] ;

[0070] in, This represents the pixel values ​​of the filtered image. This represents the pixel value at position (im, jn) in the original image. The weights of the Gaussian convolution kernel are represented by (m, n), the kernel coordinates are represented by (i, j), and the original row and column coordinates of the image are represented by (i, j). The convolution kernel is calculated as follows:

[0071] ;

[0072] Where (x, y) are the coordinates of the convolution kernel. The standard deviation is denoted as .

[0073] After Gaussian filtering, bilateral filtering can be performed to optimize edges and details in the image. Specifically, spatial domain weights and intensity domain weights can be calculated using the target image size obtained in the previous steps and the data area in the preset buffer. The filtering window size can also be 5. 5. Then, the total weight is determined using the spatial domain weight and the intensity domain weight, and the weighted pixel values ​​are accumulated as the target weight. The bilateral filtering formula is as follows:

[0074] ;

[0075] in Let be the pixel value at position (x, y) in the filtered image. Let be the pixel value at (x, y) in the target image, representing the normalized result. For spatial domain weights, The intensity domain weights are represented by the numerator in the formula, which calculates a weighted sum of all neighborhood pixels. This is to ensure that the pixel values ​​remain within a reasonable range after normalization, thus guaranteeing that the resulting filtered pixel values ​​are valid. By sequentially applying Gaussian and bilateral filtering to each spectral energy, the corresponding filtered energies can be obtained.

[0076] Step S13: Utilize a first target operator to determine the gradients in each direction corresponding to each filtered energy, so as to determine the target average gradient based on each gradient in each direction; utilize a second target operator to determine the target variances corresponding to each filtered energy; and determine the target index score corresponding to the target image based on the target average gradients and the target variances, so as to determine the target index curve corresponding to the target image based on the target index scores; the first target operator is an optimized Sobel operator used to calculate gradients in multiple directions, and the second target operator is a Laplacian operator.

[0077] In this embodiment, after obtaining the filtered energies corresponding to each spectral energy, the gradients in each direction corresponding to each filtered energy can be determined using the first target operator, and then the target average gradient can be determined using the gradients in each direction. The specific process of determining the gradients in each direction corresponding to each filtered energy using the first target operator to determine the target average gradient based on the gradients in each direction includes: determining the gradients in each direction corresponding to each filtered energy in each target direction using the first target operator; determining the gradient sum of the target image based on the gradients in each direction; and determining the target average gradient based on the gradient sum. It can be understood that since the edges of many objects in the image are not strictly horizontal or vertical, the first target operator is an optimized Sobel operator used to calculate gradients in multiple directions. The target directions include 0°, 45°, 90°, and 315°. Other angles can also be used as target directions according to the needs of image edge detection to improve the sensitivity and detection capability of oblique edges in the image, capture more details, and to some extent make up for the problem of poor anti-rotation capability of the traditional Sobel operator. Furthermore, the first target operator has a greater weight for the middle pixels than for the edge pixels. By increasing the weight of the middle pixels, it can provide better detection results than the Sobel operator in areas where the image edges change rapidly. Due to the larger weight, it is more sensitive to small changes in brightness (such as changes in focal length) in the image, making it more suitable for sharpness detection or fine-tuning of focal length. Especially in high-resolution images, edges are often more subtle and difficult to capture; increasing the weight can capture these details more accurately. The first target operator is shown below:

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in , , , These are used to detect directional gradients along four target directions: 0°, 45°, 90°, and 315°. The formulas for calculating the gradient in each direction are as follows:

[0083] ;

[0084] in, For the image at points gradient value at, For the image in Pixel value at that location, Indicates that the convolution kernel is in The weights of the positional elements are used to detect gradients at edges in specific directions within an image. The formula for the average gradient in the four directions is:

[0085] ;

[0086] in, For the image in The average gradient value at that point, , , , These represent the gradient values ​​of the image in the directions of 0° (horizontal), 90° (vertical), 45°, and 315°, respectively. The gradient of the entire target image is summed. The calculation formula is:

[0087] ;

[0088] Average gradient magnitude The calculation formula is:

[0089] ;

[0090] Where count refers to the total number of pixels in the target image.

[0091] In this embodiment, since the focus target is a standard image card, which contains edge information with varying degrees of detail and feature change in multiple directions, even the improved Sobel operator, as a first-order derivative, cannot meet the need to calculate gradient changes in all directions. The Laplacian operator, however, is a second-order derivative that captures high-frequency information (such as edges and details) in the image by calculating the brightness changes between a pixel and its neighboring pixels. This operator is rotation-invariant, satisfying edge detection requirements in different directions. Furthermore, as a second-order derivative, it can capture more complex edge information and subtle focal length changes. Therefore, using the Laplacian operator as the second target operator is effective for fine-tuning to the optimal focal length in the later stages of focusing. The kernel of the second target operator is shown below:

[0092] ;

[0093] The formula for calculating the target variance is as follows:

[0094] ;

[0095] in, To obtain the target variance, count is the total number of pixels in the target image. For the total variance, The average variance is denoted as .

[0096] In this embodiment, to evaluate the imaging quality of the lens at the current focal length, in one specific implementation, the target index score corresponding to the target image can be determined based on the target average gradient and the target variance. The specific process may include: determining the channel index score corresponding to each filtered energy based on the target average gradient and the target variance, and then determining the target index score corresponding to the target image based on the channel index scores. The formula for determining the channel index score is:

[0097] ;

[0098] The formula for determining the target indicator score is as follows:

[0099] ;

[0100] in, The channel index score corresponding to the i-th channel. The average gradient of the target is... Let Variance be the target variance. and These are the weighting coefficients; The target index score is given by t, where t is the total number of channels. After obtaining the target index score corresponding to the target image at the current image acquisition location, the target index score can be plotted on a chart to obtain the target index curve corresponding to the target image, such as... Figure 5 As shown in the figure, the red curve is the target index curve, and the vertical axis represents the value of the evaluation index normalized to the [0~1] interval. Each target index score corresponds to a point in the target index curve. The target index score of the target image corresponding to the current image acquisition position is the last point in the target index curve.

[0101] It is understandable that this embodiment is designed for focusing in complex and variable scenarios, assuming that all channels have the same level of energy importance, i.e., the scores of each channel are summed and the average is calculated. When a specific scenario is required, such as water bodies or vegetation, the weight of a certain channel can be increased to achieve better image quality in that scenario, facilitating subsequent spectral analysis.

[0102] Step S14: Determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and determine the focal length corresponding to the previous image acquisition position as the optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position.

[0103] In this embodiment, it can be understood that the process of determining the target index curve is a process of connecting points into a line, that is, marking the index values ​​corresponding to each image acquisition position on the coordinate system and connecting them. The target index curve corresponding to the current image acquisition position A is equivalent to marking the point (i.e., point B1) corresponding to the target index score of the target image of A on the coordinate system after obtaining the point (i.e., point A1) corresponding to the target index score of the previous image acquisition position B and the target index curve, and connecting point A1 with point B1 to obtain the target index curve corresponding to A. Therefore, by judging whether the target index curve corresponding to A is higher than the target index curve corresponding to B (i.e., judging whether point A1 is higher than point B1 in this coordinate system), it can be determined whether the target index score corresponding to A is greater than the target index score corresponding to B, thereby judging whether the focal length corresponding to B is the optimal focal length. In other words, after initially obtaining the target index score corresponding to B (the previous image acquisition position), it's impossible to determine whether the focal length b corresponding to B is the optimal focal length. Therefore, after obtaining the target index score corresponding to A (i.e., the current image acquisition position), if this target index score is less than or equal to the target index score corresponding to B, that is, the target index curve between B and A shows a downward trend, or the target index curve between A and B is a line segment parallel to the horizontal axis, then the focal length corresponding to position B is determined as the optimal focal length. Similarly, if the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, it indicates that the focal length corresponding to the previous image acquisition position is not the optimal focal length. In this case, we can proceed to the step of determining the current image acquisition position based on the preset focusing direction.

[0104] Step S15: Take the opposite direction of the preset focusing direction as the new preset focusing direction, determine the new current image acquisition position based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, determine the focal length corresponding to the current image acquisition position as the global optimal focal length to complete the focusing of the camera to be focused.

[0105] In this embodiment, it is understood that, considering the focusing process is a process of defocusing to focusing and then defocusing again, the globally optimal focal length (corresponding to the optimal target index score) can only be confirmed when the target index line passes through the peak and then begins to decline. The globally optimal focal length may be located between position B (i.e., the image acquisition position corresponding to the optimal focal length) and position A, or it may be located before position B. In other words, when position B is near the peak, if the index value corresponding to position A of the next image is not greater than the index value corresponding to position B when focusing continues in the current direction, it indicates that the process of defocusing again has begun. At this time, the globally optimal focal length is either before position B (i.e., the process of defocusing to focusing and then defocusing again has begun before reaching position B) or between position B and position A (i.e., the process of defocusing to focusing and then defocusing occurs during the process from B to A). Therefore, after obtaining the optimal focal length for the first time, the opposite direction of the preset focusing direction needs to be taken as the new preset focusing direction, and the new current image acquisition position needs to be determined based on the new preset focusing direction. If the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, then the focal length corresponding to the current image acquisition position can be determined as the global optimal focal length to complete the focusing of the camera to be focused. It is understood that the above focusing process can be repeated multiple times according to the required focusing accuracy to ensure that the final global optimal focal length meets the accuracy requirements, making the focusing result more reasonable and reliable.

[0106] In this embodiment, the change in the target index curve can be used to assist in determining the direction of focus adjustment. To further intuitively reflect the difference between the focal length corresponding to the current image acquisition position and the optimal focal length, and to assist production personnel in judging the deviation between the focal length corresponding to the current image acquisition position and the optimal focal length, and whether the deviation can be ignored, to determine whether to stop focusing, a corresponding focus completion prompt operation can be executed when the target index curve corresponding to the current image acquisition position meets the preset focus standard. The preset focus standard includes the difference between the target index score corresponding to the current image acquisition position and the target index score corresponding to the optimal focal length being less than a preset threshold. It is understood that if the preset threshold is set too high, the resulting image quality will be poor, and the current focal length will be far from the optimal focal length; if the preset threshold is set too low, the current focal length target value will oscillate around the optimal target value, never meeting the preset threshold. This preset threshold can be adjusted before each focus adjustment according to the focusing accuracy requirements, but when the focusing environment is fixed, i.e., when there is a stable light source and a focusing target, the optimal preset threshold is also relatively fixed. For example, during the initial focusing, the optimal preset threshold is unknown. The preset threshold can be adjusted to 0.1 for the first focusing attempt, then to 0.05 for the second, then to 0.02 for the third, and finally to 0.01 for the final focusing attempt. This final preset threshold is then considered the optimal one for the current focusing environment. Subsequent focusing attempts under this environment can directly use this preset threshold without needing multiple adjustments. Using a reliable preset threshold tested in a stable environment allows for efficient approximation of the globally optimal focal length during focusing.

[0107] In one specific implementation, the focus completion notification can be a signal light mechanism. For example, the signal light is green when the target index curve corresponding to the current image acquisition position meets the preset focus standard, and green when the target index curve does not meet the preset focus standard. Furthermore, this signal light mechanism can also be used during the focus direction determination process. If the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, a blue light can be displayed to inform the production personnel that focus adjustment should still be performed according to the current preset focus direction. If the target index curve corresponding to the current image acquisition position is not higher than the target index curve corresponding to the previous image acquisition position, a yellow light can be displayed to inform the production personnel that the preset focus direction needs to be reversed to approach the globally optimal focal length.

[0108] like Figure 5 The image shown is the focusing result of this embodiment. Figure 6 This is the result of focusing the mosaic camera using traditional focusing methods. Figure 7 This is a comparison image showing the focusing results of the B channel (blue channel) after focusing. Figure 7 (a) in the figure is the algorithm provided in this embodiment. Figure 7 (b) in the text represents the traditional method; Figure 8 This is a comparison chart of the focusing results of the G channel (green channel) after focusing. Figure 8 (a) in the figure is the algorithm provided in this embodiment. Figure 8 (b) in the text represents the traditional method; Figure 9 This is a comparison chart of the focusing results of the R channel (red channel) after focusing. Figure 9 (a) in the figure is the algorithm provided in this embodiment. Figure 9 (b) in the text represents the traditional method; Figure 10 A quantitative comparison between the traditional method and this embodiment reveals the differences in focusing results between the two methods. The gradient and high-frequency energy indicators in the table are important bases for evaluating image focusing performance; a detailed analysis and comparison are provided below:

[0109] First, looking at the gradient metrics, the gradient values ​​of our proposed method are significantly higher than those of the traditional method in both the B and R channels. For example, in the B channel, the gradient value of the traditional method is 24.231, while our method reaches 26.664, a significant improvement. Similarly, in the R channel, our method's gradient value is 26.265, also better than the traditional method's 23.854. This indicates that our proposed method can generate clearer image details in both the B and R channels compared to the traditional method.

[0110] Secondly, a larger absolute value of the high-frequency energy index indicates a greater number of high-frequency components in the image, which typically correspond to details, edges, and textures. Therefore, a larger absolute value generally indicates a clearer image. Looking at the high-frequency energy index, our method exhibits higher high-frequency energy in the B channel. In the B channel, the absolute value of the high-frequency energy of the traditional method is 2.70397e+08, while our method increases it to 2.72436e+08, indicating that our method better preserves high-frequency information and demonstrates superior high-frequency characteristics.

[0111] Comprehensive analysis shows that this method does not outperform traditional methods in terms of gradient indices in the G channel, nor does it have a significant advantage in high-frequency energy. However, after focusing using this method, the image data in the B and R channels better preserves image details and improves image clarity and contrast compared to traditional methods, thus achieving balanced optimization in image quality.

[0112] As can be seen, in the scenario of focusing equipment, this application acquires image data from a mosaic camera in real time. By splitting the multi-band multispectral mixed energy in the spatial spectrum coupled image into multispectral single energy, multiple single-band image data are obtained. After preprocessing each single-band image using Gaussian filtering and bilateral filtering, the composite imaging quality index is used to calculate the quality index of each single-band image. Combined with channel weights, a comprehensive imaging index is calculated. Production personnel adjust the focal length according to the changes in the comprehensive imaging quality index curve to achieve the optimal focal length.

[0113] See Figure 11 As shown, this application discloses a focusing device for a multispectral imaging module, comprising:

[0114] The image acquisition module 11 is used to determine the current image acquisition position based on a preset focusing direction, acquire an image of a standard chart using a camera to be focused at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera.

[0115] The filtering module 12 is used to split the multispectral mixed energy corresponding to the target image to obtain the spectral energy corresponding to the target image, and to perform Gaussian filtering and bilateral filtering on each spectral energy in sequence to obtain the corresponding filtered energy.

[0116] The scoring determination module 13 is used to determine the gradients of each direction corresponding to each of the filtered energies using a first target operator so as to determine the target average gradient based on each of the directional gradients; to determine the target variances of each of the filtered energies using a second target operator; and to determine the target index score corresponding to the target image based on the target average gradients and the target variances, so as to determine the target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating gradients in multiple directions, and the second target operator is a Laplacian operator;

[0117] The first focal length determination module 14 is used to determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, the focal length corresponding to the previous image acquisition position is determined as the optimal focal length.

[0118] The second focal length determination module 15 is used to take the opposite direction of the preset focusing direction as the new preset focusing direction, determine the new current image acquisition position based on the new preset focusing direction, and determine the focal length corresponding to the current image acquisition position as the global optimal focal length when the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, so as to complete the focusing of the camera to be focused.

[0119] As can be seen, in the scenario of focusing equipment, this application acquires image data from a mosaic camera in real time. By splitting the multi-band multispectral mixed energy in the spatial spectrum coupled image into multispectral single energy, multiple single-band image data are obtained. After preprocessing each single-band image using Gaussian filtering and bilateral filtering, the composite imaging quality index is used to calculate the quality index of each single-band image. Combined with channel weights, a comprehensive imaging index is calculated. Production personnel adjust the focal length according to the changes in the comprehensive imaging quality index curve to achieve the optimal focal length.

[0120] In one specific embodiment, the image acquisition module 11 may include:

[0121] The image group acquisition unit is used to acquire images of a standard image card at the current image acquisition position based on a preset exposure time using a camera to be focused, so as to obtain an original image group.

[0122] In one specific embodiment, the filtering module 12 may include:

[0123] The contribution ratio determination unit is used to determine the spectral response function of each channel of the camera to be focused, and to determine the corresponding energy contribution ratio based on the area ratio of each channel spectral response function in the corresponding wavelength range.

[0124] The spectral energy splitting unit is used to split the multispectral mixed energy based on the respective energy contribution ratios to obtain the respective spectral energies corresponding to the target image.

[0125] In one specific implementation, the score determination module 13 may include:

[0126] An orientation gradient determination unit is used to determine the orientation gradients of each filtered energy in each target direction using the first target operator; the target directions include 0°, 45°, 90° and 315°; the weight of the middle pixel in the first target operator is greater than the weight of the edge pixel;

[0127] An average gradient determination unit is used to determine the gradient sum of the target image based on each of the directional gradients, and to determine the average gradient of the target based on the gradient sum.

[0128] In one specific implementation, the score determination module 13 may include:

[0129] The channel score determination unit is used to determine the channel index score corresponding to each of the filtered energies based on the target average gradient and the target variance, wherein the formula for determining the channel index score is:

[0130] ;

[0131] in, The channel index score corresponding to the i-th channel. The average gradient of the target is... Let Variance be the target variance. and These are the weighting coefficients;

[0132] The target index score determination unit is used to determine the target index score corresponding to the target image based on the index scores of each channel, wherein the formula for determining the target index score is:

[0133] ;

[0134] in, The target metric score is given by t, where t is the total number of channels.

[0135] In one specific embodiment, the device may further include:

[0136] The acquisition position determination module is used to jump to the step of determining the current image acquisition position based on the preset focusing direction if the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position.

[0137] In one specific embodiment, the device may further include:

[0138] The focus completion prompt module is used to perform a corresponding focus completion prompt operation if the target index curve corresponding to the current image acquisition position meets the preset focus standard; the preset focus standard includes the difference between the target index score corresponding to the current image acquisition position and the target index score corresponding to the optimal focal length being less than a preset threshold.

[0139] Furthermore, embodiments of this application also disclose an electronic device, Figure 12 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0140] Figure 12This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the focusing method of the multispectral imaging module disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0141] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0142] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.

[0143] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the focusing method of the multispectral imaging module executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0144] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned focusing method of the multispectral imaging module. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0145] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0146] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0147] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0148] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A focusing method for a multispectral imaging module, characterized in that, include: The current image acquisition position is determined based on the preset focusing direction. At the current image acquisition position, the camera to be focused is used to acquire an image of the standard chart to obtain the original image. The original image is then processed based on the preset target area to determine the target image. The camera to be focused is a mosaic camera; The multispectral mixed energy corresponding to the target image is split to obtain the spectral energy corresponding to the target image, and Gaussian filtering and bilateral filtering are performed on each spectral energy in sequence to obtain the corresponding filtered energy. A first target operator is used to determine the gradients in each direction corresponding to each filtered energy so as to determine the target average gradient based on the gradients in each direction. A second target operator is used to determine the target variances corresponding to each filtered energy. Based on the target average gradients and the target variances, the target index score corresponding to the target image is determined so as to determine the target index curve corresponding to the target image based on the target index score. The first target operator is an optimized Sobel operator for calculating gradients in multiple directions, and the second target operator is a Laplacian operator. Determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and determine the focal length corresponding to the previous image acquisition position as the optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position. The opposite direction of the preset focusing direction is taken as the new preset focusing direction. Based on the new preset focusing direction, a new current image acquisition position is determined. When the target index curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position is determined as the global optimal focal length to complete the focusing of the camera to be focused. The step of determining the target index score corresponding to the target image based on the target average gradient and the target variance includes: The channel index scores corresponding to each filtered energy are determined based on the target average gradient and the target variance, wherein the formula for determining the channel index scores is as follows: ； in, The channel index score corresponding to the i-th channel. The average gradient of the target is... Let Variance be the target variance. and These are the weighting coefficients; The target index score corresponding to the target image is determined based on the scores of each of the aforementioned channel indexes, wherein the formula for determining the target index score is: ； in, The target metric score is given by t, where t is the total number of channels.

2. The focusing method of the multispectral imaging module according to claim 1, characterized in that, The step of acquiring an image of the standard image card using the camera to be focused at the current image acquisition position to obtain the original image includes: At the current image acquisition location, images are acquired from the standard chart using the camera to be focused based on a preset exposure time to obtain the original image set.

3. The focusing method of the multispectral imaging module according to claim 1, characterized in that, The step of splitting the multispectral mixed energy corresponding to the target image to obtain the individual spectral energies corresponding to the target image includes: Determine the spectral response function of each channel corresponding to the camera to be focused, and determine the corresponding energy contribution ratio based on the area ratio of each channel spectral response function in the corresponding wavelength range; The multispectral mixed energy is split based on the energy contribution ratio of each to obtain the spectral energy corresponding to the target image.

4. The focusing method of the multispectral imaging module according to claim 1, characterized in that, The step of determining the gradients in each direction corresponding to each of the filtered energies using the first target operator in order to determine the target average gradient based on each of the directional gradients includes: The first target operator is used to determine the gradient of each filtered energy in each target direction; the target directions include 0°, 45°, 90° and 315°; the weight of the middle pixel in the first target operator is greater than the weight of the edge pixel; The gradient sum of the target image is determined based on the gradients of each of the aforementioned directions, and the average gradient of the target is determined based on the gradient sum.

5. The focusing method of the multispectral imaging module according to claim 1, characterized in that, Also includes: If the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, then proceed to the step of determining the current image acquisition position based on the preset focusing direction.

6. The focusing method of the multispectral imaging module according to any one of claims 1 to 5, characterized in that, Also includes: If the target index curve corresponding to the current image acquisition position meets the preset focus standard, then the corresponding focus adjustment completion prompt operation will be executed. The preset focusing standard includes a difference between the target index score corresponding to the current image acquisition position and the target index score corresponding to the optimal focal length being less than a preset threshold.

7. A focusing device for a multispectral imaging module, characterized in that, include: The image acquisition module is used to determine the current image acquisition position based on a preset focusing direction, acquire an image of a standard chart using a camera to be focused at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera; The filtering module is used to decompose the multispectral mixed energy corresponding to the target image to obtain the spectral energy corresponding to the target image, and to perform Gaussian filtering and bilateral filtering on each spectral energy in sequence to obtain the corresponding filtered energy. The scoring determination module is used to determine the gradients of each direction corresponding to each of the filtered energies using a first target operator in order to determine the target average gradient based on the gradients of each direction; to determine the target variances of each of the filtered energies using a second target operator; and to determine the target index score corresponding to the target image based on the target average gradients and the target variances, so as to determine the target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating gradients of multiple directions, and the second target operator is a Laplacian operator; The first focal length determination module is used to determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, the focal length corresponding to the previous image acquisition position is determined as the optimal focal length. The second focal length determination module is used to take the opposite direction of the preset focusing direction as the new preset focusing direction, determine the new current image acquisition position based on the new preset focusing direction, and determine the focal length corresponding to the current image acquisition position as the global optimal focal length when the target index curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length in order to complete the focusing of the camera to be focused. The score determination module specifically includes: The channel score determination unit is used to determine the channel index score corresponding to each of the filtered energies based on the target average gradient and the target variance, wherein the formula for determining the channel index score is: ； in, The channel index score corresponding to the i-th channel. The average gradient of the target is... Let Variance be the target variance. and These are the weighting coefficients; The target index score determination unit is used to determine the target index score corresponding to the target image based on the index scores of each channel, wherein the formula for determining the target index score is: ； in, The target metric score is given by t, where t is the total number of channels.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the focusing method of the multispectral imaging module as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the focusing method of the multispectral imaging module as described in any one of claims 1 to 6.

Images