Focusing method and device of multispectral imaging module, equipment and storage medium

Abstract

The invention discloses a focusing method, device and equipment for a multispectral imaging module and a storage medium, and relates to the field of optics, and the method comprises the steps: carrying out the image collection through a to-be-focused camera, so as to determine a target image; splitting the multispectral mixed energy of the target image to obtain each spectral energy, and filtering to obtain each filtered energy; processing the filtered energy by using a first target operator and a second target operator to determine a target index curve corresponding to the target image; when the target index curve corresponding to the current image acquisition position is not higher than the target index curve corresponding to the previous image acquisition position, determining the focal length corresponding to the previous image acquisition position as the optimal focal length; and determining a new current image acquisition position, and determining the focal length corresponding to the current image acquisition position as the global optimal focal length when the target curve corresponding to the current image acquisition position is not lower than the target index curve corresponding to the optimal focal length. According to the invention, the focusing effect of the multispectral camera in a high-noise scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of optics, and in particular to a focusing method, device, equipment and storage medium of a multi-spectral imaging module. Background Art

[0002] At present, when focusing on a spatial-spectral coupled mosaic camera, multiple images at different distances from the observed object are often obtained, and the image corresponding to the position with the highest clarity is used as a reference to obtain multiple images at positions close to the observed object and multiple images at positions far from the observed object. Finally, the average signal strength of all the acquired images is calculated, and the position corresponding to the image with the strongest average signal strength is used as the focus position; or the ROI focus window is determined by the automatic alignment algorithm of the fusion YOLOv5 (You Only Look Once version 5) network intelligent ROI (Region Of Interest) and the improved Laplacian autofocus algorithm, and the gradient square sum after the convolution operation of the region and the Laplace operator is used as the clarity evaluation value for focusing. However, both methods are sensitive to noise in the image, and the clarity index obtained in a high-noise scene may not reflect the true quality of the image, resulting in poor focusing effect for multispectral cameras.

[0003] In summary, how to improve the focusing effect of multispectral cameras in high-noise scenes is a problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a focusing method, device, equipment and storage medium for a multispectral imaging module, which can improve the focusing effect of a multispectral camera in a high-noise scene. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a focusing method for a multispectral imaging module, comprising:

[0006] Determine the current image acquisition position based on the preset focusing direction, use the camera to be focused to acquire an image of the standard image card at the current image acquisition position to obtain an original image, and process the original image based on the preset target area to determine the target image; the camera to be focused is a mosaic camera;

[0007] Splitting the multi-spectral mixed energy corresponding to the target image to obtain the spectral energies corresponding to the target image, and sequentially performing Gaussian filtering and bilateral filtering on the spectral energies to obtain the corresponding filtered energies;

[0008] Using a first target operator to determine each directional gradient corresponding to each filtered energy so as to determine a target average gradient based on each directional gradient, using a second target operator to determine each target variance corresponding to each filtered energy, and determining a target index score corresponding to the target image based on the target average gradient and the target variance, so as to determine a target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator;

[0009] Determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and determine the focal length corresponding to the previous image acquisition position as the optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position;

[0010] The opposite direction of the preset focusing direction is used as a new preset focusing direction, a new current image acquisition position is determined based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target indicator curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position is determined as the global optimal focal length to complete the focusing of the camera to be focused.

[0011] Optionally, the step of acquiring an image of a standard image card using a camera to be focused at a current image acquisition position to obtain an original image includes:

[0012] At the current image acquisition position, the standard image card is imaged using the to-be-focused camera based on a preset exposure time to obtain an original image group.

[0013] Optionally, splitting the multi-spectral mixed energy corresponding to the target image to obtain each spectral energy corresponding to the target image includes:

[0014] Determine the spectral response function of each channel corresponding to the camera to be focused, and determine the corresponding energy contribution ratio based on the area proportion of each channel spectral response function within the corresponding wavelength range;

[0015] The multi-spectral mixed energy is split based on each of the energy contribution ratios to obtain each of the spectral energies corresponding to the target image.

[0016] Optionally, the using the first target operator to determine each directional gradient corresponding to each filtered energy so as to determine a target average gradient based on each directional gradient includes:

[0017] Determine each of the directional gradients corresponding to each of the filtered energies in each target direction using the first target operator; the target directions include 0°, 45°, 90° and 315°; the intermediate pixel weight of the first target operator is greater than the edge pixel weight;

[0018] The gradient sum of the target image is determined based on each of the directional gradients, and the target average gradient is determined based on the gradient sum.

[0019] Optionally, determining a target index score corresponding to the target image based on the target average gradient and the target variance includes:

[0020] The channel index scores corresponding to the filtered energies are determined based on the target average gradient and the target variance, wherein the channel index scores are determined by the formula:

[0021] ;

[0022] in, is the channel index score corresponding to the i-th channel, is the target average gradient, is the target variance, and is the weight coefficient;

[0023] The target index score corresponding to the target image is determined based on each of the channel index scores, wherein the formula for determining the target index score is:

[0024] ;

[0025] in, is the target indicator score, and t is the total number of channels.

[0026] Optionally, the focusing method of the multispectral imaging module further includes:

[0027] If the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, the process jumps to the step of determining the current image acquisition position based on the preset focusing direction.

[0028] Optionally, the focusing method of the multispectral imaging module further includes:

[0029] If the target index curve corresponding to the current image acquisition position meets the preset focus standard, the corresponding focus completion prompt operation is performed; the preset focus standard includes that the difference between the target index score corresponding to the current image acquisition position and the target index score corresponding to the optimal focal length is less than a preset threshold.

[0030] In a second aspect, the present application discloses a focusing device for a multispectral imaging module, comprising:

[0031] An image acquisition module, used to determine a current image acquisition position based on a preset focusing direction, acquire an image of a standard image card using a camera to be focused at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera;

[0032] A filtering module, used for splitting the multi-spectral mixed energy corresponding to the target image to obtain the spectral energies corresponding to the target image, and sequentially performing Gaussian filtering and bilateral filtering on the spectral energies to obtain the corresponding filtered energies;

[0033] A score determination module, used to determine each directional gradient corresponding to each filtered energy using a first target operator so as to determine a target average gradient based on each directional gradient, determine each target variance corresponding to each filtered energy using a second target operator, and determine a target index score corresponding to the target image based on the target average gradient and the target variance, so as to determine a target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator;

[0034] A first focal length determination module is used to determine whether a target index curve corresponding to a current image acquisition position is lower than or equal to a target index curve corresponding to a previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, determine the focal length corresponding to the previous image acquisition position as the optimal focal length;

[0035] A second focal length determination module is used to take the opposite direction of the preset focusing direction as a new preset focusing direction, determine a new current image acquisition position based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target indicator curve corresponding to the optimal focal length, determine the focal length corresponding to the current image acquisition position as the global optimal focal length to complete the focusing of the camera to be focused.

[0036] In a third aspect, the present application discloses an electronic device, comprising:

[0037] Memory, used to store computer programs;

[0038] The processor is used to execute the computer program to implement the focusing method of the multi-spectral imaging module.

[0039] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned focusing method of the multi-spectral imaging module when executed by a processor.

[0040] In the present application, when focusing the camera to be focused, the current image acquisition position is determined based on a preset focusing direction, the camera to be focused is used to acquire an image of a standard chart at the current image acquisition position to obtain an original image, and the original image is processed based on a preset target area to determine the target image; the camera to be focused is a mosaic camera; the multi-spectral mixed energy corresponding to the target image is split to obtain each spectral energy corresponding to the target image, and each spectral energy is sequentially Gaussian filtered and bilaterally filtered to obtain the corresponding filtered energies; the first target operator is used to determine each directional gradient corresponding to each filtered energy so as to determine a target average gradient based on each directional gradient, the second target operator is used to determine each target variance corresponding to each filtered energy, and the target index score corresponding to the target image is determined based on the target average gradient and the target variance, so as to determine the target index score based on the target index score. The target index curve corresponding to the target image is determined; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator; it is determined whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, the focal length corresponding to the previous image acquisition position is determined as the optimal focal length; the opposite direction of the preset focusing direction is used as a new preset focusing direction, a new current image acquisition position is determined based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position is determined as the global optimal focal length to complete the focusing of the camera to be focused. It can be seen that in the scenario of focusing tooling, the present application collects image data of the mosaic camera in real time, and obtains multiple single-band image data by splitting the multi-band and multi-spectral mixed energy in the spatial spectral coupling image into multi-spectral single energy. After pre-processing each single-band image using Gaussian filtering and bilateral filtering, the composite imaging quality index is used to calculate the quality index of each single-band image, and the comprehensive imaging index is calculated in combination with the channel weight. The production personnel adjust the focal length according to the changes in the comprehensive imaging quality index curve to achieve the global optimal focal length of the camera to be focused. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0042] Figure 1 A flow chart of a focusing method of a multi-spectral imaging module disclosed in this application;

[0043] Figure 2 A schematic diagram of a four-channel spectral response function provided for this application;

[0044] Figure 3 A schematic diagram of a focusing method flow of a specific multispectral imaging module disclosed in this application;

[0045] Figure 4 A schematic diagram of a focusing method flow of a specific multispectral imaging module disclosed in this application;

[0046] Figure 5 A schematic diagram of focusing results of a multi-spectral imaging module disclosed in this application;

[0047] Figure 6 A schematic diagram of the result of focusing a mosaic camera using a traditional focusing method provided in the present application;

[0048] Figure 7 This is a comparison diagram of the focusing results of the B channel after focusing disclosed in this application, where Figure 7 (a) is a focusing result obtained by focusing using the focusing method disclosed in the present application, Figure 7 (b) is the focusing result obtained by using the traditional focusing method;

[0049] Figure 8 This is a comparison diagram of the focusing results of the G channel after focusing disclosed in this application, where Figure 8 (a) is a focusing result obtained by focusing using the focusing method disclosed in the present application, Figure 8 (b) is the focusing result obtained by using the traditional focusing method;

[0050] Fig. 9 This is a comparison diagram of the focusing results of the R channel after focusing disclosed in this application, where Fig. 9 (a) is a focusing result obtained by focusing using the focusing method disclosed in the present application, Fig. 9 (b) is the focusing result obtained by using the traditional focusing method;

[0051] Fig.10A quantitative comparison table of focusing results of focusing a focus camera using a traditional method disclosed in the present application and focusing a focus camera using the focusing method disclosed in the present application;

[0052] Fig.11 A schematic diagram of the structure of a focusing device of a multi-spectral imaging module disclosed in this application;

[0053] Fig.12 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] At present, when focusing on a spatial-spectral coupled mosaic camera, multiple images at different distances from the observed object are often obtained, and the images of multiple positions close to the observed object and multiple positions away from the observed object are obtained based on the position corresponding to the image with the highest clarity. Finally, the average signal strength of all acquired images is calculated, and the position corresponding to the image with the strongest average signal strength is used as the focus position; or the ROI focus window is determined by the automatic alignment algorithm of the intelligent region of interest (ROI) of the YOLOv5 network and the improved Laplacian autofocus algorithm, and the gradient square sum after the convolution operation of the region and the Laplace operator is used as the clarity evaluation value for focusing. However, both methods are sensitive to the noise in the image, and the clarity index obtained in a high-noise scene may not reflect the true quality of the image, resulting in poor focusing effect on the multispectral camera. In order to solve the above technical problems, the present application discloses a focusing method for a multispectral imaging module, which can improve the focusing effect of a multispectral camera in a high-noise scene.

[0056] See also Figure 1 As shown, an embodiment of the present invention discloses a focusing method of a multi-spectral imaging module, comprising:

[0057] Step S11, determining the current image acquisition position based on a preset focusing direction, acquiring an image of a standard image card using a camera to be focused at the current image acquisition position to obtain an original image, and processing the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera.

[0058] In this embodiment, the camera to be focused is a mosaic camera. When focusing the camera to be focused in a tooling scene, the focusing direction, that is, the preset focusing direction, must first be determined. In actual applications, production personnel can randomly adjust the focal length of the camera in one direction. If the index curve drops, focus in the opposite direction. If the index curve rises, continue to focus in this direction. Then determine the current image acquisition position in the preset focusing direction, so that the camera to be focused can be used to acquire the original image with the standard chart as the target at the current image acquisition position. In a tooling environment, the image data includes not only the chart content required for focusing, but also the edge information of the relay lens, etc. In order to eliminate the interference, the original image can be intercepted by setting the ROI area size to obtain valid image data as the input of the subsequent processing flow. In other words, the original image can be processed based on the preset target area to determine the target image, where the target image is a part of the original image.

[0059] In this embodiment, the standard image card can be captured by the camera to be focused at the current image capture position based on the preset exposure time to obtain the original image group. That is to say, the exposure time of multiple groups of cameras can be set to simulate the light and dark changes in the actual scene, such as 0.5 milliseconds, 1 millisecond, 1.5 milliseconds, 2 milliseconds, 2.5 milliseconds, 3 milliseconds, etc., and the gain can be set to 1. In actual applications, only one camera exposure time needs to be set, and the exposure time and gain can also be adjusted as needed to avoid overexposure or overdarkness.

[0060] Step S12, splitting the multi-spectral mixed energy corresponding to the target image to obtain each spectral energy corresponding to the target image, and performing Gaussian filtering and bilateral filtering on each spectral energy in turn to obtain corresponding filtered energies.

[0061] In this embodiment, for the energy mixing of the mosaic multispectral camera, the multispectral mixed energy of the multispectral mosaic image can be split into single energies of multiple spectra based on the spectral response function of the camera, thereby obtaining multiple single-band image data. In other words, the specific process of splitting the multispectral mixed energy corresponding to the target image to obtain the spectral energies corresponding to the target image may include: determining the spectral response functions of each channel corresponding to the camera to be focused, and determining the corresponding energy contribution ratios based on the area proportion of the spectral response function of each channel in the corresponding wavelength range; splitting the multispectral mixed energy based on the energy contribution ratios to obtain the spectral energies corresponding to the target image. The spectral response function is mainly composed of three parts: the transmittance of the optical lens, the transmittance of the multi-band narrowband filter, and the quantum efficiency of the detector. The calculation formula of the spectral response function of each channel is as follows:

[0062] ;

[0063] in, is the transmittance of the optical lens, is the transmittance function of the multi-bandpass narrowband filter, is the detector quantum efficiency function, Represent each channel respectively. By calculating the area ratio of each channel spectral response function in the corresponding wavelength range, such as Figure 2 As shown, the energy contribution ratio of each channel can be obtained, and the calculation formula is as follows:

[0064] ;

[0065] in, for aisle The energy contribution ratio of the band, Represents each channel, Represents the band number of the multi-band narrowband filter. Taking the four spectra of blue light, red light, green light, and infrared light as examples, the energy contribution ratio of each channel can be used Solve the following multivariate linear equations:

[0066] ;

[0067] By solving the linear equations, the mixing energy of each channel can be Single spectral energy split into multiple spectra ,in , , and Indicates four different channels: blue light, red light, green light and infrared light.

[0068] In this embodiment, Figure 3 As shown in , after obtaining each spectral energy, each spectral energy can be filtered to eliminate the interference of abnormal points and noise on the image, making the image smoother. Specifically, as Figure 4 As shown, Gaussian filtering can be performed first and then bilateral filtering. In a specific implementation, after obtaining the size and data area of ​​the target image, the data area can be copied to a preset buffer area, and then a Gaussian filtering window of size 5*5 is selected, and the pixel values ​​of each point within the window size are obtained cyclically to calculate the pixel value of the center point using the Gaussian filtering formula. The Gaussian filtering formula is as follows:

[0069] ;

[0070] in, represents the pixel value of the image after filtering, Represents the pixel value of the original image at position (im, jn), represents the weight of the Gaussian convolution kernel, (m, n) represents the coordinates of the convolution kernel, (i, j) represents the original row and column coordinates of the image, and the convolution kernel is calculated as follows:

[0071] ;

[0072] Among them, (x, y) is the convolution kernel coordinate, is the standard deviation.

[0073] After completing Gaussian filtering, bilateral filtering can be performed to optimize the edges and details in the image. Specifically, the target image size obtained in the previous step and the data area in the preset buffer area can be used to calculate the spatial domain weight and the intensity domain weight. The filter window size can also be 5*5, and then the spatial domain weight and the intensity domain weight are used to determine the total weight, and the weighted pixel values ​​are accumulated as the target weight. The bilateral filtering formula is as follows:

[0074] ;

[0075] in is the pixel value of the filtered image at position (x, y), is the pixel value of the target image at (x, y), indicating the normalized result. is the spatial domain weight, is the intensity domain weight, and the target weight is the numerator in the formula, which calculates the weighted sum of all domain pixels to ensure that the pixel value is still within a reasonable range after normalization and that the pixel value obtained after filtering is a legal pixel value. The corresponding filter energy can be obtained by completing Gaussian filtering and bilateral filtering for each spectral energy in turn.

[0076] Step S13, using the first target operator to determine each directional gradient corresponding to each of the filtered energies so as to determine the target average gradient based on each of the directional gradients, using the second target operator to determine each target variance corresponding to each of the filtered energies, and determining the target indicator score corresponding to the target image based on the target average gradient and the target variance, so as to determine the target indicator curve corresponding to the target image based on the target indicator score; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator.

[0077] In this embodiment, after obtaining each filtered energy corresponding to each spectral energy, the first target operator can be used to determine each directional gradient corresponding to each filtered energy, and then the target average gradient can be determined using each directional gradient. The specific process of using the first target operator to determine each directional gradient corresponding to each filtered energy so as to determine the target average gradient based on each directional gradient includes: using the first target operator to determine each directional gradient corresponding to each filtered energy in each target direction; determining the gradient sum of the target image based on each directional gradient, and determining the target average gradient based on the gradient sum. It can be understood that since the edges of many objects in the image are not strictly horizontal or vertical, the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the target directions include 0°, 45°, 90° and 315°. Other angles can also be used as target directions according to the needs of image edge detection to improve the sensitivity and detection ability of oblique edges in the image, capture more details, and to a certain extent make up for the problem of poor anti-rotation ability of the traditional Sobel operator. In addition, the weight of the middle pixel of the first target operator is greater than the weight of the edge pixel. By increasing the weight of the middle pixel, it can provide better detection results than the Sobel operator in some areas where the edge of the image changes rapidly. Due to the larger weight, it is more sensitive to small brightness changes in the image (such as focal length changes), and is more suitable for sharpness detection or focus fine-tuning. Especially in high-resolution images, the edges are often more subtle and difficult to capture. Amplifying the weight can capture these details more accurately. The first target operator is as follows:

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in , , , They are used to detect the directional gradients in the four target directions of 0°, 45°, 90° and 315°. The calculation formulas for the directional gradients are as follows:

[0083] ;

[0084] in, For the image at point The gradient value at For images in The pixel value at Indicates that the convolution kernel is The weight value of the position element is used to detect the gradient of the edge in a specific direction in the image. The formula for the mean of the gradient in four directions is:

[0085] ;

[0086] in, For images in The average gradient value at , , , Represent the gradient values ​​of the image at 0° (horizontal), 90° (vertical), 45° and 315° respectively. The gradient and The calculation formula is:

[0087] ;

[0088] Average gradient magnitude The calculation formula is:

[0089] ;

[0090] Where count refers to the total number of pixels in the target image.

[0091] In this embodiment, since the focus target is a standard chart, the chart has edge information in multiple directions and with different degrees of detail feature changes. Even the improved Sobel operator as a first-order derivative cannot meet the needs of calculating gradient changes in all directions. The Laplacian operator is a second-order derivative that captures high-frequency information (such as edges and details) in the image by calculating the brightness changes of pixels and surrounding adjacent pixels. The operator has rotation invariance and can meet the edge detection needs in different directions. At the same time, because it is a second-order derivative, it can capture more complex edge information and slight focal length changes. Therefore, using the Laplacian operator as the second target operator has a better effect for fine-tuning to the optimal focal length in the later stage of focusing. The operator kernel of the second target operator is as follows:

[0092] ;

[0093] The target variance is calculated as follows:

[0094] ;

[0095] in, To obtain the target variance, count is the total number of pixels in the target image. is the total variance, is the mean variance.

[0096] In this embodiment, in order to evaluate the imaging quality of the lens at the current focal length, in a specific implementation, the target index score corresponding to the target image can be determined based on the target average gradient and the target variance. The specific process may include: determining each channel index score corresponding to each filtered energy based on the target average gradient and the target variance, and then determining the target index score corresponding to the target image based on each channel index score, wherein the channel index score is determined by the formula:

[0097] ;

[0098] The formula for determining the target indicator score is:

[0099] ;

[0100] in, is the channel index score corresponding to the i-th channel, is the target average gradient, is the target variance, and is the weight coefficient; is the target index score, and t is the total number of channels. After obtaining the target index score corresponding to the target image at the current image acquisition position, the target index score can be plotted in a chart to obtain a target index curve corresponding to the target image, such as Figure 5 As shown in the figure, the red curve is the target indicator curve, the ordinate represents the value of the evaluation indicator normalized to the [0~1] interval, each target indicator score corresponds to a point in the target indicator curve, and the target indicator score of the target image corresponding to the current image acquisition position is the last point in the target indicator curve.

[0101] It is understandable that this embodiment focuses on complex and non-fixed scenes, and assumes that the importance of all channel energies is the same, that is, the scores of each channel index are summed and the mean is calculated. When it is necessary to focus on a specific scene, such as water bodies, vegetation, etc., the weight of a channel can be increased to achieve better imaging quality in the scene, which is convenient for subsequent spectral analysis.

[0102] Step S14, determining whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and determining the focal length corresponding to the previous image acquisition position as the optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position.

[0103] In the present embodiment, it can be understood that the process of determining the target indicator curve is the process of connecting points into lines, that is, the process of marking the indicator values ​​corresponding to each image acquisition position on the coordinate system and connecting them. The target indicator curve corresponding to the current image acquisition position A is equivalent to obtaining the point corresponding to the target indicator score corresponding to the previous image acquisition position B on the coordinate system (that is, point B1) and the target indicator curve, marking the point corresponding to the target indicator score of the target image of A on the coordinate system (that is, point A1), and connecting point A1 with point B1 to obtain the target indicator curve corresponding to A. Therefore, it is possible to judge whether the target indicator score corresponding to A is greater than the target indicator score corresponding to B by judging whether the target indicator curve corresponding to A is higher than the target indicator curve corresponding to B (that is, judging whether point A1 is higher than point B1 in the coordinate system), thereby judging whether the focal length corresponding to B is the optimal focal length. In other words, after first obtaining the target indicator score corresponding to B (the previous image acquisition position), it is impossible to determine whether the focal length b corresponding to B is the optimal focal length. Therefore, after obtaining the target indicator score corresponding to A (i.e., the current image acquisition position), if the target indicator score is less than or equal to the target indicator score corresponding to B, that is, the target indicator curve between B and A shows a downward trend, or the target indicator curve between A and B is a line segment parallel to the horizontal axis, then the focal length corresponding to position B is determined as the optimal focal length. Similarly, if the target indicator curve corresponding to the current image acquisition position is higher than the target indicator curve corresponding to the previous image acquisition position, it means that the focal length corresponding to the previous image acquisition position is not the optimal focal length. At this time, you can jump to the step of determining the current image acquisition position based on the preset focusing direction.

[0104] Step S15: taking the opposite direction of the preset focusing direction as a new preset focusing direction, determining a new current image acquisition position based on the new preset focusing direction, and determining the focal length corresponding to the current image acquisition position as the global optimal focal length when the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length to complete the focusing of the camera to be focused.

[0105] In this embodiment, it can be understood that, considering that the focusing process is a process from defocusing to focusing to re-defocusing, the global optimal focal length (corresponding to the optimal target index score) can only be confirmed when the target index line passes through the peak and then starts to decline. The global optimal focal length may be between position B (that is, the image acquisition position corresponding to the optimal focal length) and position A, or it may be before position B. In other words, when position B is near the peak, the index value corresponding to position A of the next image when focusing continues along the current direction is not greater than the index value corresponding to position B, indicating that the re-defocusing process has been entered. At this time, the global optimal focal length is before position B (that is, the re-defocusing process has been entered before reaching position B) or between position B and position A (that is, the process from defocusing to focusing and then defocusing occurs in the process from B to A). Therefore, after obtaining the optimal focal length for the first time, it is necessary to use the opposite direction of the preset focusing direction as the new preset focusing direction, and determine the new current image acquisition position based on the new preset focusing direction. If the target curve corresponding to the current image acquisition position is higher than or equal to the target index curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position can be determined as the global optimal focal length to complete the focus of the camera to be focused. It can be understood that the above focusing process can be repeated multiple times according to the need for focusing accuracy so that the final global optimal focal length meets the accuracy requirements, making the focusing result more reasonable and reliable.

[0106] In this embodiment, the change of the target index curve can be used to assist in determining the direction of focal length adjustment. In order to further intuitively reflect the difference between the focal length corresponding to the current image acquisition position and the optimal focal length to assist production personnel in determining the deviation between the focal length corresponding to the current image acquisition position and the optimal focal length, whether the deviation can be ignored, etc. to determine whether to stop focusing, the corresponding focusing completion prompt operation can be performed when the target index curve corresponding to the current image acquisition position meets the preset focusing standard. Among them, the preset focusing standard includes that the difference between the target index score corresponding to the current image acquisition position and the target index score corresponding to the optimal focal length is less than the preset threshold. It can be understood that if the preset threshold is set too large, the image quality obtained is poor, and the current focal distance is far from the optimal focal length; if the preset threshold is set too small, the current focal length target value will oscillate around the optimal target value, and the preset threshold will never be met. The preset threshold can be adjusted before each focusing according to the focus accuracy requirements, but when the focus environment is fixed, that is, when there is a stable light source and focus target, the optimal preset threshold is also relatively fixed. For example, when focusing for the first time, the optimal preset threshold is unknown. You can first adjust the preset threshold to 0.1 for the first focus, then adjust the preset threshold to 0.05 for the second focus, then adjust the preset threshold to 0.02 for the third focus, and finally adjust the preset threshold to 0.01 for the last focus. This preset threshold is used as the optimal preset threshold in the current focus environment. If you need to focus in this focus environment later, you can directly use this preset threshold for focusing without adjusting the preset threshold multiple times. Using a reliable preset threshold tested in a stable environment can efficiently approach the global optimal focal length during the focusing process.

[0107] In a specific embodiment, the focus completion prompt operation can be a signal light mechanism, such as when the target index curve corresponding to the current image acquisition position meets the preset focus standard, the signal light is green, and when the target index curve corresponding to the current image acquisition position does not meet the preset focus standard, the signal light is green. In addition, the signal light mechanism can also be used in the process of determining the focus direction. If the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, a blue light can be turned on to inform the production personnel that they still need to focus according to the current preset focus direction. If the target index curve corresponding to the current image acquisition position is not higher than the target index curve corresponding to the previous image acquisition position, a yellow light can be turned on to inform the production personnel that they need to reverse the preset focus direction to approach the global optimal focal length.

[0108] like Figure 5 The figure shown is the focusing result diagram of this embodiment. Figure 6 This is the result of focusing the mosaic camera using the traditional focusing method. Figure 7 This is a comparison chart of the focusing results of the B channel (blue channel) after focusing. Figure 7 (a) is the algorithm provided in this embodiment, Figure 7 (b) the traditional method; Figure 8 This is a comparison chart of the focusing results of the G channel (green channel) after focusing. Figure 8 (a) is the algorithm provided in this embodiment, Figure 8 (b) the traditional method; Fig. 9 This is a comparison chart of the focusing results of the R channel (red channel) after focusing. Fig. 9 (a) is the algorithm provided in this embodiment, Fig. 9 (b) the traditional method; Fig.10 The quantitative comparison between the traditional method and this embodiment shows the difference in focusing results between the traditional method and this method. The two indicators of gradient and high-frequency energy in the table are important bases for evaluating the image focusing effect. The following is a detailed analysis and comparison of them:

[0109] First, from the perspective of gradient indicators, the gradient values ​​of this method on channels B and R are significantly higher than those of the traditional method. For example, in the B channel, the gradient value of the traditional method is 24.231, while this method reaches 26.664, which is a significant improvement. Similarly, in the R channel, the gradient value of this method is 26.265, which is also better than the 23.854 of the traditional method. This shows that compared with the traditional method, this method can generate clearer image details on the B and R channels.

[0110] Secondly, the larger the absolute value of the high-frequency energy index, the more high-frequency components the image has, and high-frequency components usually correspond to details, edges, and textures in the image. Therefore, the larger the absolute value, the clearer the image can be considered. From the perspective of high-frequency energy index, this method shows higher high-frequency energy on the B channel. In the B channel, the absolute value of high-frequency energy of the traditional method is 2.70397e+08, while this method increases it to 2.72436e+08, indicating that this method can better retain high-frequency information and show better high-frequency characteristics.

[0111] Comprehensive analysis shows that this method is not superior to the traditional method in terms of the gradient index of the G channel, and has no obvious advantage over the traditional method in terms of high-frequency energy. However, compared with the traditional method, the image data of the B and R channels after focusing by this method can better retain image details and improve image clarity and contrast, thereby achieving balanced optimization in image quality.

[0112] It can be seen that in the scenario of focusing tooling, the present application collects image data of the mosaic camera in real time, and obtains multiple single-band image data by splitting the multi-band and multi-spectral mixed energy in the spatial spectral coupling image into multi-spectral single energy. After pre-processing each single-band image using Gaussian filtering and bilateral filtering, the composite imaging quality index is used to calculate the quality index of each single-band image, and the comprehensive imaging index is calculated in combination with the channel weight. The production personnel adjust the focal length according to the changes in the comprehensive imaging quality index curve to achieve the optimal focal length.

[0113] See also Fig.11 As shown, the present application discloses a focusing device for a multi-spectral imaging module, comprising:

[0114] The image acquisition module 11 is used to determine the current image acquisition position based on a preset focusing direction, acquire an image of a standard image card using a camera to be focused at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera;

[0115] A filtering module 12 is used to split the multi-spectral mixed energy corresponding to the target image to obtain each spectral energy corresponding to the target image, and sequentially perform Gaussian filtering and bilateral filtering on each spectral energy to obtain corresponding filtered energies;

[0116] The score determination module 13 is used to determine each directional gradient corresponding to each filtered energy using a first target operator so as to determine a target average gradient based on each directional gradient, determine each target variance corresponding to each filtered energy using a second target operator, and determine a target index score corresponding to the target image based on the target average gradient and the target variance so as to determine a target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator;

[0117] A first focal length determination module 14 is used to determine whether a target index curve corresponding to a current image acquisition position is lower than or equal to a target index curve corresponding to a previous image acquisition position, and to determine a focal length corresponding to the previous image acquisition position as an optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position;

[0118] The second focal length determination module 15 is used to take the opposite direction of the preset focusing direction as a new preset focusing direction, determine a new current image acquisition position based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target indicator curve corresponding to the optimal focal length, determine the focal length corresponding to the current image acquisition position as the global optimal focal length to complete the focusing of the camera to be focused.

[0119] It can be seen that in the scenario of focusing tooling, the present application collects image data of the mosaic camera in real time, and obtains multiple single-band image data by splitting the multi-band and multi-spectral mixed energy in the spatial spectral coupling image into multi-spectral single energy. After pre-processing each single-band image using Gaussian filtering and bilateral filtering, the composite imaging quality index is used to calculate the quality index of each single-band image, and the comprehensive imaging index is calculated in combination with the channel weight. The production personnel adjust the focal length according to the changes in the comprehensive imaging quality index curve to achieve the optimal focal length.

[0120] In a specific implementation, the image acquisition module 11 may specifically include:

[0121] The image group acquisition unit is used to acquire images of a standard image card using a camera to be focused based on a preset exposure time at a current image acquisition position to obtain an original image group.

[0122] In a specific implementation, the filtering module 12 may specifically include:

[0123] A contribution ratio determination unit, used to determine the spectral response function of each channel corresponding to the camera to be focused, and determine the corresponding energy contribution ratio based on the area proportion of each channel spectral response function within the corresponding wavelength range;

[0124] The spectral energy splitting unit is used to split the multi-spectral mixed energy based on each energy contribution ratio to obtain each spectral energy corresponding to the target image.

[0125] In a specific implementation, the score determination module 13 may specifically include:

[0126] a directional gradient determination unit, configured to determine, using the first target operator, the directional gradients corresponding to the filtered energies in the target directions; the target directions include 0°, 45°, 90° and 315°; the weight of the intermediate pixels of the first target operator is greater than the weight of the edge pixels;

[0127] The average gradient determination unit is used to determine the gradient sum of the target image based on each of the directional gradients, and determine the target average gradient based on the gradient sum.

[0128] In a specific implementation, the score determination module 13 may specifically include:

[0129] A channel score determination unit is used to determine each channel index score corresponding to each filtered energy based on the target average gradient and the target variance, wherein the channel index score is determined by:

[0130] ;

[0131] in, is the channel index score corresponding to the i-th channel, is the target average gradient, is the target variance, and is the weight coefficient;

[0132] A target indicator score determination unit is used to determine the target indicator score corresponding to the target image based on each of the channel indicator scores, wherein the determination formula of the target indicator score is:

[0133] ;

[0134] in, is the target indicator score, and t is the total number of channels.

[0135] In a specific embodiment, the device may further include:

[0136] The acquisition position determination module is used to jump to the step of determining the current image acquisition position based on the preset focusing direction if the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position.

[0137] In a specific embodiment, the device may further include:

[0138] A focus completion prompt module is used to perform a corresponding focus completion prompt operation if the target indicator curve corresponding to the current image acquisition position meets a preset focus standard; the preset focus standard includes that the difference between the target indicator score corresponding to the current image acquisition position and the target indicator score corresponding to the optimal focal length is less than a preset threshold.

[0139] Furthermore, the present application also discloses an electronic device. Fig.12 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0140] Fig.12A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the focusing method of the multi-spectral imaging module disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0141] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0142] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0143] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the focusing method of the multispectral imaging module performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0144] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the focusing method of the multispectral imaging module disclosed above is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0145] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0146] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0148] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0149] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A focusing method for a multispectral imaging module, characterized in that: include: Determine a current image acquisition position based on a preset focusing direction, acquire an image of a standard image card using a to-be-focused camera at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; The camera to be focused is a mosaic camera; Splitting the multi-spectral mixed energy corresponding to the target image to obtain the spectral energies corresponding to the target image, and sequentially performing Gaussian filtering and bilateral filtering on the spectral energies to obtain the corresponding filtered energies; Using a first target operator to determine each directional gradient corresponding to each filtered energy so as to determine a target average gradient based on each directional gradient, using a second target operator to determine each target variance corresponding to each filtered energy, and determining a target index score corresponding to the target image based on the target average gradient and the target variance, so as to determine a target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator; Determine whether the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, and determine the focal length corresponding to the previous image acquisition position as the optimal focal length when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position; The opposite direction of the preset focusing direction is used as a new preset focusing direction, a new current image acquisition position is determined based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target indicator curve corresponding to the optimal focal length, the focal length corresponding to the current image acquisition position is determined as the global optimal focal length to complete the focusing of the camera to be focused.

2. The focusing method of the multispectral imaging module according to claim 1, characterized in that: The method of acquiring an image of a standard image card by using a camera to be focused at a current image acquisition position to obtain an original image includes: At the current image acquisition position, the standard image card is imaged using the to-be-focused camera based on a preset exposure time to obtain an original image group.

3. The focusing method of the multispectral imaging module according to claim 1, characterized in that: The step of splitting the multi-spectral mixed energy corresponding to the target image to obtain each spectral energy corresponding to the target image includes: Determine the spectral response function of each channel corresponding to the camera to be focused, and determine the corresponding energy contribution ratio based on the area proportion of each channel spectral response function within the corresponding wavelength range; The multi-spectral mixed energy is split based on each of the energy contribution ratios to obtain each of the spectral energies corresponding to the target image.

4. The focusing method of the multispectral imaging module according to claim 1, characterized in that: The method of using the first target operator to determine the directional gradients corresponding to the filtered energies so as to determine the target average gradient based on the directional gradients includes: Determine each of the directional gradients corresponding to each of the filtered energies in each target direction using the first target operator; the target directions include 0°, 45°, 90° and 315°; the intermediate pixel weight of the first target operator is greater than the edge pixel weight; The gradient sum of the target image is determined based on each of the directional gradients, and the target average gradient is determined based on the gradient sum.

5. The focusing method of the multi-spectral imaging module according to claim 1, characterized in that: The determining the target index score corresponding to the target image based on the target average gradient and the target variance includes: The channel index scores corresponding to the filtered energies are determined based on the target average gradient and the target variance, wherein the channel index scores are determined by the formula: ； in, is the channel index score corresponding to the i-th channel, is the target average gradient, is the target variance, and is the weight coefficient; The target index score corresponding to the target image is determined based on each of the channel index scores, wherein the formula for determining the target index score is: ； in, is the target indicator score, and t is the total number of channels.

6. The focusing method of the multi-spectral imaging module according to claim 1, characterized in that: Also includes: If the target index curve corresponding to the current image acquisition position is higher than the target index curve corresponding to the previous image acquisition position, the process jumps to the step of determining the current image acquisition position based on the preset focusing direction.

7. The focusing method of the multi-spectral imaging module according to any one of claims 1 to 6, characterized in that: Also includes: If the target index curve corresponding to the current image acquisition position meets the preset focus standard, the corresponding focus completion prompt operation is performed; The preset focus standard includes that a difference between a target index score corresponding to a current image acquisition position and a target index score corresponding to the optimal focal length is less than a preset threshold.

8. A focusing device for a multi-spectral imaging module, characterized in that: include: An image acquisition module, used to determine a current image acquisition position based on a preset focusing direction, acquire an image of a standard image card using a camera to be focused at the current image acquisition position to obtain an original image, and process the original image based on a preset target area to determine a target image; the camera to be focused is a mosaic camera; A filtering module, used for splitting the multi-spectral mixed energy corresponding to the target image to obtain the spectral energies corresponding to the target image, and sequentially performing Gaussian filtering and bilateral filtering on the spectral energies to obtain the corresponding filtered energies; A score determination module, used to determine each directional gradient corresponding to each filtered energy using a first target operator so as to determine a target average gradient based on each directional gradient, determine each target variance corresponding to each filtered energy using a second target operator, and determine a target index score corresponding to the target image based on the target average gradient and the target variance, so as to determine a target index curve corresponding to the target image based on the target index score; the first target operator is an optimized Sobel operator for calculating multiple directional gradients, and the second target operator is a Laplacian operator; A first focal length determination module is used to determine whether a target index curve corresponding to a current image acquisition position is lower than or equal to a target index curve corresponding to a previous image acquisition position, and when the target index curve corresponding to the current image acquisition position is lower than or equal to the target index curve corresponding to the previous image acquisition position, determine the focal length corresponding to the previous image acquisition position as the optimal focal length; A second focal length determination module is used to take the opposite direction of the preset focusing direction as a new preset focusing direction, determine a new current image acquisition position based on the new preset focusing direction, and when the target curve corresponding to the current image acquisition position is higher than or equal to the target indicator curve corresponding to the optimal focal length, determine the focal length corresponding to the current image acquisition position as the global optimal focal length to complete the focusing of the camera to be focused.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the focusing method of the multi-spectral imaging module as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the focusing method of the multi-spectral imaging module as described in any one of claims 1 to 7 is implemented.

Images