Imaging quality optimization method of multispectral sensor and related device

Abstract

The invention discloses an imaging quality optimization method of a multispectral sensor and a related device, and relates to the technical field of image processing, and the method comprises the steps: carrying out the focusing parameter analysis of the multispectral sensor on a target object, and obtaining a target focusing parameter; performing imaging influence analysis of ambient light intensity on the multispectral sensor to match target exposure parameters; performing correction processing on the original image data based on the non-uniformity correction matrix; performing white balance processing on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis; and carrying out dead pixel correction, linear conversion and fusion processing on first decomposed image data and second decomposed image data obtained by decomposing the original image data after the white balance processing to obtain a target imaging image. According to the method, the color expressive force of the imaged image is improved, and the imaging quality of the multispectral sensor is more effectively optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to an imaging quality optimization method and related device for a multispectral sensor. Background Art

[0002] Since a multispectral sensor can capture a variety of spectral information in the environment to provide more real and accurate color reproduction, most enterprises have applied multispectral sensors in camera products. With the improvement of people's requirements for imaging quality, how to optimize the imaging quality of multispectral sensors has become the research focus of enterprises. In the imaging of a multispectral sensor, the exposure parameter is related to the brightness of the imaging image. Currently, the exposure parameter is usually analyzed by histogram statistics of the test image, but the exposure parameter obtained by this method cannot well adapt to the on-site environment, resulting in insufficient brightness of the obtained imaging image. Since a multispectral sensor has more channels than a general image sensor, there will be certain non-uniformity errors in the generated image, but currently most lack correction for non-uniformity errors, resulting in insufficient authenticity of the obtained imaging image. For the imaging of a multispectral sensor, performing white balance processing on the image is also a very important step. Currently, the white balance processing of the image is usually performed by the gray world algorithm. However, if there is a large area of a single-color object in the image, this method will result in inaccurate color correction, affecting the imaging quality, resulting in insufficient color expressiveness of the imaging image, and making it impossible to effectively improve the optimization effect of the imaging quality of the multispectral sensor. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides an imaging quality optimization method and related device for a multispectral sensor, which improves the color expressiveness of the imaging image and enables the imaging quality of the multispectral sensor to be more effectively optimized.

[0004] To solve the above technical problems, the present invention provides an imaging quality optimization method for a multispectral sensor, and the method includes:

[0005] Analyze the focusing parameters of the multispectral sensor for the target object to obtain the target focusing parameters;

[0006] Analyze the imaging influence of the ambient light intensity on the multispectral sensor to obtain imaging influence data, and match the target exposure parameter based on the imaging influence data;

[0007] Construct a non-uniformity correction matrix, and perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focusing parameters and target exposure parameters based on the non-uniformity correction matrix to obtain the corrected original image data;

[0008] Perform white balance processing on the original image data after correction processing based on the target gains of each color channel obtained from the scene light source confidence analysis to obtain the original image data after white balance processing;

[0009] Perform dead pixel correction, linear conversion, and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain the target imaging image.

[0010] Optionally, the analysis of the focusing parameters of the multispectral sensor for the target object to obtain the target focusing parameters includes:

[0011] Calculate the first phase difference between the first initial image and the second initial image of the target object collected by the multispectral sensor at each position based on the pixel brightness curve;

[0012] Calculate the second phase difference between the first phase map generated from the first initial image and the phase map generated from the second initial image;

[0013] Perform analysis of the focusing parameters of the multispectral sensor based on the first phase difference and the second phase difference to obtain the target focusing parameters.

[0014] Optionally, the analysis of the imaging impact of the ambient light intensity on the multispectral sensor to obtain the imaging impact data and match the target exposure parameters based on the imaging impact data includes:

[0015] Integrate the features of the historical ambient light impact imaging record set to obtain the target feature data;

[0016] Analyze the correlation degree between different ambient light intensities and different imaging parameters based on the historical ambient light impact imaging record set;

[0017] Perform node adjustment on the initial imaging impact analysis model based on the target feature data and the correlation degree to obtain the target imaging impact analysis model, and perform analysis of the imaging impact of the ambient light intensity on the multispectral sensor based on the target imaging impact analysis model to obtain the imaging impact data;

[0018] Match the exposure parameters based on the imaging impact data and the mapping relationship to obtain the target exposure parameters.

[0019] Optionally, the performing node adjustment on the initial imaging impact analysis model based on the target feature data and the correlation degree to obtain the target imaging impact analysis model includes:

[0020] Determine the data label based on the correlation degree and the weight constraint of the target feature data, and perform network node evaluation using the evaluation function based on the data label to obtain the network node evaluation coefficient;

[0021] Adjust the nodes of the initial imaging impact analysis model based on the network node evaluation coefficient to obtain the target imaging impact analysis model.

[0022] Optionally, the constructing the non-uniformity correction matrix, and correcting the original image data of the target object collected by the multispectral sensor according to the target focusing parameter and the target exposure parameter based on the non-uniformity correction matrix to obtain the corrected original image data, includes:

[0023] Determine a reference region in the calibrated multispectral image based on the multi-channel response curve, and calculate the average pixel data of each channel in the reference region;

[0024] Construct a non-uniformity correction matrix based on the pixel data of each channel in the calibrated multispectral image and the average pixel data of each channel in the reference region;

[0025] Perform non-uniformity correction processing on the original image data based on the non-uniformity correction matrix to obtain the non-uniformity-corrected original image data;

[0026] Perform spectral correction processing on the non-uniformity-corrected original image data based on adaptive instance normalization and spline fitting to obtain the corrected original image data.

[0027] Optionally, the performing white balance processing on the corrected original image data based on the target gain of each color channel obtained from the scene light source confidence analysis to obtain the white balance-processed original image data, includes:

[0028] Perform clustering analysis on the spectra of each scene light source to obtain the corresponding light source categories, and perform model training based on the light source categories and the pixel received light intensity of each scene light source to obtain a light source prediction model;

[0029] Determine the light source prediction information and the corresponding confidence based on the real-time scene spectral data using the light source prediction model;

[0030] Determine the initial color temperature corresponding to the light source prediction information based on the tristimulus value curve, calculate the target color temperature using the confidence corresponding to the light source prediction information based on the initial color temperature, and determine the target gain of each color channel based on the target color temperature;

[0031] Perform transformation processing on the pixel values of each color channel in the corrected original image data based on the target gain of each color channel to obtain the white balance-processed original image data.

[0032] Optionally, the performing dead pixel correction, linear conversion and fusion processing on the first decomposition image data and the second decomposition image data obtained by decomposing the white balance-processed original image data to obtain the target imaging image, includes:

[0033] Determine the analysis region of the point to be detected in the historical output image based on the neighborhood window, perform vertical direction difference analysis by combining the adjacent pixel values based on the analysis region of the point to be detected, and obtain the vertical direction difference analysis result;

[0034] Construct a bad pixel judgment table based on the vertical direction difference analysis data by combining the threshold judgment method, and analyze the corresponding bad pixel in the first decomposition image data and the second decomposition image data based on the bad pixel judgment table;

[0035] Determine the correction pixel value corresponding to the bad pixel in the first decomposition image data and the second decomposition image data based on color component analysis and distance analysis, and perform bad pixel correction on the first decomposition image data and the second decomposition image data based on the correction pixel value to obtain the first decomposition image data and the second decomposition image data after bad pixel correction;

[0036] Perform channel correction on the first decomposition image data after bad pixel correction to obtain the first decomposition image data after channel correction, and perform linear conversion on the second decomposition image data after bad pixel correction to obtain the second decomposition image data after linear conversion;

[0037] Perform fusion processing on the first decomposition image data after channel correction and the second decomposition image data after linear conversion to obtain the target imaging image.

[0038] In addition, the present invention also provides an imaging quality optimization device for a multispectral sensor, and the device includes:

[0039] Focus parameter analysis module: used to analyze the focus parameters of the multispectral sensor for the target object to obtain the target focus parameters;

[0040] Exposure parameter analysis module: used to analyze the imaging influence of the ambient light intensity on the multispectral sensor to obtain imaging influence data, and match the target exposure parameters based on the imaging influence data;

[0041] Image correction module: used to construct a non-uniformity correction matrix, and perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameters and the target exposure parameters based on the non-uniformity correction matrix to obtain the original image data after correction processing;

[0042] White balance processing module: used to perform white balance processing on the original image data after correction processing based on the target gain of each color channel obtained by scene light source confidence analysis to obtain the original image data after white balance processing;

[0043] Fusion and optimization module: It is used to perform dead pixel correction, linear conversion, and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing, so as to obtain the target imaging image.

[0044] In addition, the present invention also provides an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the above-mentioned imaging quality optimization method for the multispectral sensor.

[0045] In addition, the present invention provides a computer-readable storage medium, which stores computer instructions. When the computer instructions run on an electronic device, the electronic device is enabled to execute the above-mentioned imaging quality optimization method for the multispectral sensor.

[0046] In the embodiment of the present invention, by analyzing the imaging influence of the ambient light intensity on the multispectral sensor and obtaining imaging influence data to match the target exposure parameters, more accurate exposure parameters can be obtained, so that the obtained exposure parameters can better adapt to the on-site environment. Constructing a non-uniformity correction matrix and performing correction processing on the original image data based on the non-uniformity correction matrix can improve the correction accuracy of the image and the authenticity of the final imaging image. Based on the target gains of each color channel obtained from the scene light source confidence analysis, white balance processing is performed on the corrected original image data. Even if there are large areas of monochromatic objects in the image, the reliability of image color correction can still be improved. Dead pixel correction, linear conversion, and fusion processing are performed on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing, so that the obtained target imaging image further reduces the gap with the real color, improves the color expressiveness of the imaging image, and effectively optimizes the imaging quality of the multispectral sensor. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0048] Figure 1 It is a flowchart of the imaging quality optimization method for the multispectral sensor in the embodiment of the present invention;

[0049] Figure 2 It is a flowchart of the imaging quality optimization method for the multispectral sensor in another embodiment of the present invention;

[0050] Figure 3 It is a schematic structural diagram of an imaging quality optimization device for a multispectral sensor in an embodiment of the present invention;

[0051] Figure 4 It is a schematic structural diagram of an electronic device in an embodiment of the present invention. Specific embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0053] Embodiment 1

[0054] Please refer to Figure 1 , Figure 1 It is a schematic flowchart of a method for optimizing the imaging quality of a multispectral sensor in an embodiment of the present invention, and the method includes:

[0055] S11: Analyze the focusing parameters of the multispectral sensor for the target object to obtain the target focusing parameters;

[0056] In the specific implementation process of the present invention, the analyzing the focusing parameters of the multispectral sensor for the target object to obtain the target focusing parameters includes: calculating the first phase difference between the first initial image and the second initial image of the target object collected by the multispectral sensor at each position based on the pixel brightness curve; calculating the second phase difference between the first phase map generated from the first initial image and the phase map generated from the second initial image; and analyzing the focusing parameters of the multispectral sensor based on the first phase difference and the second phase difference to obtain the target focusing parameters.

[0057] Specifically, the multispectral sensor is a multispectral image sensor. Compared with other general image sensors, it can better capture various spectral information in the environment to provide more real and accurate color restoration. Therefore, selecting a multispectral sensor can better improve the quality of the results. Calculate the first phase difference between the first initial image and the second initial image of the target object collected by the multispectral sensor at each position based on the pixel brightness curve. The multispectral sensor collects one image of the target object at each position. The image is respectively represented as the first initial image and the second initial image through a spectroscopic system. Taking the brightness values of the pixels of the first initial image and the second initial image as the ordinate and the imaging width value of the multispectral sensor as the abscissa to plot a graph, forming a pixel brightness curve. Calculate the offset between the pixel brightness curve of the first initial image and the pixel brightness curve of the second initial image, and take the offset at each position as the first phase difference. Calculate the second phase difference between the first phase map generated from the first initial image and the phase map generated from the second initial image. According to the pixel information of several selected pixel points in the first initial image, a first phase map is formed. The generation method of the second phase map is the same as that of the first phase map. Calculate the phase difference between the first phase map and the second phase map as the second phase difference. Analyze the focusing parameters of the multispectral sensor based on the first phase difference and the second phase difference to obtain the target focusing parameters. Calculate the average phase difference according to the first phase difference and the second phase difference at each position, and perform operations based on the average phase difference combined with the corresponding correction value to obtain the focusing position information of the multispectral sensor, that is, obtain the target focusing parameters.

[0058] S12: Analyze the imaging influence of the ambient light intensity on the multispectral sensor to obtain imaging influence data, and match the target exposure parameters based on the imaging influence data;

[0059] In the specific implementation process of the present invention, the step of analyzing the imaging influence of the ambient light intensity on the multispectral sensor to obtain imaging influence data and matching the target exposure parameters based on the imaging influence data includes: integrating the features of the historical ambient light influence imaging record set to obtain target feature data; analyzing the correlation degree between different ambient light intensities and different imaging parameters based on the historical ambient light influence imaging record set; adjusting the nodes of the initial imaging influence analysis model based on the target feature data and the correlation degree to obtain a target imaging influence analysis model, and analyzing the imaging influence of the ambient light intensity on the multispectral sensor based on the target imaging influence analysis model to obtain imaging influence data; matching the exposure parameters based on the imaging influence data and the mapping relationship to obtain the target exposure parameters.

[0060] Further, adjusting nodes of the initial imaging impact analysis model based on the target feature data and the degree of association to obtain a target imaging impact analysis model, including: determining data labels based on the degree of association and the weight constraint of the target feature data, and using an evaluation function based on the data labels to evaluate network nodes to obtain a network node evaluation coefficient; adjusting nodes of the initial imaging impact analysis model based on the network node evaluation coefficient to obtain a target imaging impact analysis model.

[0061] Specifically, perform feature integration on the historical environmental light impact imaging record set. The historical environmental light impact imaging record set includes record data of the imaging impacts of different past environmental light intensities on a multispectral sensor, such as the impacts of different environmental light intensities on exposure parameters and imaging brightness, etc. Use a feature extractor to extract features from the historical environmental light impact imaging record set to obtain several feature table sets, and perform feature integration on the several feature table sets. Feature integration can be carried out in a horizontal merging manner to obtain target feature data. Analyze the degree of association between different environmental light intensities and different imaging parameters based on the historical environmental light impact imaging record set. Analyze the degree of association between different environmental light intensities and different imaging parameters through an association rule algorithm, and analyze the occurrence frequency of the same imaging parameter among different environmental light intensities. Determine data labels based on the degree of association and the weight constraint of the target feature data. Assign a weight constraint to the target feature data, and the weight constraint has been preset in the database. Determine the corresponding data labels in the label database according to the degree of association and the weight constraint of the target feature data. The data labels can be determined, and use an evaluation function based on the data labels to evaluate network nodes, that is, use the evaluation function to evaluate the nodes of the preset initial topology map according to the data labels, evaluate the importance of each node, and obtain a network node evaluation coefficient. Adjust nodes of the initial imaging impact analysis model based on the network node evaluation coefficient. The initial imaging impact analysis model is a topology map. Adjusting nodes of the initial imaging impact analysis model can adjust the distance between nodes or remove / add nodes, etc., thereby obtaining a target imaging impact analysis model. And perform imaging impact analysis of environmental light intensity on the multispectral sensor based on the target imaging impact analysis model. Input the real-time environmental light intensity information into the target imaging impact model to obtain the exposure impact of the current real-time environmental light intensity information on the imaging of the multispectral sensor, that is, obtain imaging impact data. Perform exposure parameter matching based on the imaging impact data and the mapping relationship. Use the corresponding mapping relationship according to the imaging impact data to perform exposure parameter matching in the database to obtain target exposure parameters.

[0062] S13: Construct a non-uniformity correction matrix, and perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focusing parameter and the target exposure parameter based on the non-uniformity correction matrix to obtain the corrected original image data;

[0063] In the specific implementation process of the present invention, constructing the non-uniformity correction matrix, and performing correction processing on the original image data of the target object collected by the multispectral sensor according to the target focusing parameters and target exposure parameters based on the non-uniformity correction matrix to obtain the corrected original image data, including: determining a reference region in the calibrated multispectral image based on the multi-channel response curve, and calculating the average pixel point data of each channel in the reference region; constructing a non-uniformity correction matrix based on the pixel point data of each channel in the calibrated multispectral image and the average pixel point data of each channel in the reference region; performing non-uniformity correction processing on the original image data based on the non-uniformity correction matrix to obtain the original image data after non-uniformity correction processing; performing spectral correction processing on the original image data after non-uniformity correction processing based on adaptive instance normalization and spline fitting to obtain the corrected original image data.

[0064] Specifically, the multi-spectral sensor is controlled according to the target focus parameter and the target exposure parameter to collect an image of the target object, obtaining the original image data, and the original image data is transmitted to an image processor for processing. Based on the multi-channel response curve, a reference region in the calibrated multi-spectral image is determined, the first response curve of each channel to the wide-band spectrum without non-uniformity error is obtained, the second response curve of each channel to the wide-band spectrum during the actual use of the multi-spectral sensor is obtained, the coincidence degree between the two is calculated through the first response curve and the second response curve, the response curve with the highest coincidence degree in each channel is taken as the required response curve, the required response curves are combined into a multi-channel response curve, the ideal pixel points in the curve are obtained according to the multi-channel response curve, the reference region is determined according to the ideal pixel points, and the average pixel point data of each channel in the reference region is calculated, that is, the average value of the pixel point values of each channel in the reference region is calculated. An inhomogeneity correction matrix is constructed based on the pixel point data of each channel in the calibrated multi-spectral image and the average pixel point data of each channel in the reference region, the pixel point data of each channel is divided by the average pixel point data of each channel in the reference region to obtain the corresponding inhomogeneity correction matrix. The original image data is subjected to inhomogeneity correction processing based on the inhomogeneity correction matrix, that is, the data points of each channel in the original image data are divided by the inhomogeneity correction matrix corresponding to each channel to obtain the corrected data of each channel, thereby obtaining the original image data after inhomogeneity correction processing. The original image data after inhomogeneity correction processing is subjected to spectral correction processing based on adaptive instance normalization and spline fitting, the scaling and translation coefficients are obtained through a learnable radiometric transformation, the per-channel re-correction of the original image data after inhomogeneity correction processing is performed according to the scaling and translation coefficients to obtain the original image data after per-channel re-correction, that is, adaptive instance normalization is realized, the spectral distribution data of each channel in the original image data after per-channel re-correction is segmented to obtain the corresponding several segmented spectral data, the several segmented spectral data are subjected to spline fitting through a preset cubic polynomial to obtain several segmented spectral data after spline fitting, and the spectral ratio correction of the image is performed based on the light source model in combination with the several segmented spectral data after spline fitting to ensure the true spectral characteristics of the image, obtaining the original image data after correction processing.

[0065] S14: Perform white balance processing on the original image data after correction processing based on the target gain of each color channel obtained from the scene light source confidence analysis to obtain the original image data after white balance processing;

[0066] In the specific implementation process of the present invention, performing white balance processing on the corrected original image data based on the target gains of each color channel obtained from the confidence analysis of the scene light source, to obtain the white balance processed original image data, includes: performing clustering analysis on the spectra of each scene light source, obtaining the corresponding light source categories, and performing model training based on the light source categories and the pixel received light intensities of each scene light source, to obtain a light source prediction model; determining light source prediction information and the corresponding confidence based on the light source prediction model using real-time scene spectral data; determining the initial color temperature corresponding to the light source prediction information based on the tristimulus value curve, calculating the target color temperature using the confidence corresponding to the light source prediction information based on the initial color temperature, and determining the target gain of each color channel based on the target color temperature; performing transformation processing on the pixel values of each color channel in the corrected original image data based on the target gains of each color channel, to obtain the white balance processed original image data.

[0067] Specifically, for performing clustering analysis on the spectra of each scene light source, using the k-means clustering algorithm to perform clustering analysis on the spectra of each scene light source, the spectral similarity value between any two clustering centers is greater than the preset classification standard, and within each cluster, the spectral similarity value between any two light sources is less than the preset classification standard, to obtain the corresponding light source categories, and performing model training based on the light source categories and the pixel received light intensities of each scene light source. Taking the light source categories and the pixel received light intensities of each scene light source as the data set, where the pixel received light intensity is the light intensity of the light received by each pixel, to train a deep neural network, to obtain a light source prediction model. Determining light source prediction information and the corresponding confidence based on the light source prediction model using real-time scene spectral data, inputting the real-time scene spectral data into the light source prediction model, and the output layer of the light source prediction model outputs the light source prediction type and the corresponding confidence. Determining the initial color temperature corresponding to the light source prediction information based on the tristimulus value curve, determining the corresponding tristimulus values in the tristimulus value curve according to the light source prediction information, calculating the corresponding color coordinates according to the tristimulus values, calculating the corresponding initial color temperature according to the color coordinates, calculating the target color temperature using the confidence corresponding to the light source prediction information based on the initial color temperature, performing weighted summation on the initial color temperature using the confidence to obtain the target color temperature, and determining the target gain of each color channel based on the target color temperature, matching the target gain of each color channel in the database according to the target color temperature, that is, the gain value of each color channel. Performing transformation processing on the pixel values of each color channel in the corrected original image data based on the target gains of each color channel, that is, multiplying the pixel values of each color channel in the corrected original image data by the target gains of each color channel respectively, to obtain the white balance processed original image data.

[0068] S15: Perform dead pixel correction, linear conversion, and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain a target imaging image.

[0069] In the specific implementation process of the present invention, the performing dead pixel correction, linear conversion, and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain a target imaging image includes: determining an analysis region of a point to be detected in a historical output image based on a neighborhood window, performing vertical direction difference analysis by combining adjacent pixel values based on the analysis region of the point to be detected to obtain a vertical direction difference analysis result; constructing a dead pixel judgment table based on the vertical direction difference analysis data in combination with a threshold judgment method, and analyzing corresponding dead pixel pixels in the first decomposed image data and the second decomposed image data based on the dead pixel judgment table; determining correction pixel values corresponding to the dead pixel pixels in the first decomposed image data and the second decomposed image data based on color component analysis and distance analysis, and performing dead pixel correction on the first decomposed image data and the second decomposed image data based on the correction pixel values to obtain the first decomposed image data and the second decomposed image data after dead pixel correction; performing channel correction on the first decomposed image data after dead pixel correction to obtain the first decomposed image data after channel correction, and performing linear conversion on the second decomposed image data after dead pixel correction to obtain the second decomposed image data after linear conversion; performing fusion processing on the first decomposed image data after channel correction and the second decomposed image data after linear conversion to obtain a target imaging image.

[0070] Specifically, based on the neighborhood window, determine the analysis region of the point to be detected in the historical output image. The size of the neighborhood window is preset. Determine the corresponding analysis region in the point to be detected according to the neighborhood window. Based on the analysis region of the point to be detected and combined with the adjacent pixel values, perform vertical direction difference analysis. Perform vertical direction difference analysis on the point to be detected by combining the vertically adjacent pixel values in the same column of the analysis region, and obtain the upper pixel difference value and the lower pixel difference value. Analyze the magnitude relationship between the upper pixel difference value and the lower pixel difference value and the preset difference threshold respectively, that is, obtain the vertical direction difference analysis result. Based on the vertical direction difference analysis data, construct a bad pixel judgment table by combining the threshold judgment method. In the vertical direction difference analysis result, when both the upper pixel difference value and the lower pixel difference value are less than the preset difference threshold, perform horizontal direction difference analysis on the point to be detected by combining the horizontally adjacent pixel values in the same column, determine the column average pixel value of the adjacent columns in the analysis region of the point to be detected, and calculate the difference between the column average pixel value and the pixel value of the point to be detected, that is, obtain the horizontal analysis data. Determine the corresponding minimum value in the horizontal analysis data, compare the minimum value with the preset judgment threshold. If the minimum value is greater than the preset judgment threshold, the point to be detected is a bad pixel, otherwise the point to be detected is not a bad pixel. Thus, use a preset counter to combine the bad pixel judgment result to construct a bad pixel judgment table, and analyze the corresponding bad pixel in the first decomposition image data and the second decomposition image data based on the bad pixel judgment table, that is, judge the bad pixel existing in the first decomposition image data and the second decomposition image data according to the bad pixel judgment table. Determine the correction pixel value corresponding to the bad pixel in the first decomposition image data and the second decomposition image data based on color component analysis and distance analysis. In the first decomposition image data and the second decomposition image data, determine the first pixel point that is closest to the bad pixel and has the same color component. Determine several pairs of second pixel points that are symmetric about the first pixel point and have the same color component as the bad pixel. Obtain the target pixel values of several adjacent valid pixel points of the bad pixel according to the pixel values of the first pixel point and the second pixel point. Calculate the pixel gradients in the horizontal direction, vertical direction, 45-degree direction, and 135-degree direction respectively according to the target pixel values. Determine several target adjacent valid pixel points according to the pixel gradients in the horizontal direction, vertical direction, 45-degree direction, and 135-degree direction. Match the corresponding correction pixel value according to the average value of the pixel values corresponding to several target adjacent valid pixel points, and perform bad pixel correction on the first decomposition image data and the second decomposition image data based on the correction pixel value, that is, correct the bad pixel in the first decomposition image data and the second decomposition image data according to the correction pixel value, and obtain the first decomposition image data and the second decomposition image data after bad pixel correction.Perform channel correction on the first decomposed image data after dead pixel correction. Construct a channel correction matrix through the preset residual color compensation parameters. Perform channel correction on the first decomposed image data after dead pixel correction according to the channel correction matrix to obtain the first decomposed image data after channel correction. Perform linear conversion on the second decomposed image data after dead pixel correction, perform demosaicing on the second decomposed image data after dead pixel correction to obtain the second decomposed image data after demosaicing, and perform conversion and resampling on the second decomposed image data after demosaicing based on the residual mapping model to obtain the second decomposed image data after linear conversion. Perform fusion processing on the first decomposed image data after channel correction and the second decomposed image data after linear conversion. Add the pixel values of the pixel points with the same position in the first decomposed image data after channel correction and the second decomposed image data after linear conversion to obtain a preliminary fused image. Perform masking processing on the preliminary fused image according to the first mask to obtain a first input image. Perform masking operation on the first decomposed image data after channel correction according to the second mask to obtain a second input image. Add the pixel values of the pixel points with the same position in the first input image and the second input image to obtain a target imaging image. By adjusting the channel gain of the split image, the gap with the true color is further reduced, the color expressiveness of the image is improved, and the imaging quality is further improved.

[0071] In the embodiment of the present invention, analyze the imaging influence of the ambient light intensity on the multispectral sensor to obtain imaging influence data to match the target exposure parameters, which can obtain more accurate exposure parameters and make the obtained exposure parameters better adapt to the on-site environment. Construct a non-uniformity correction matrix, and perform correction processing on the original image data based on the non-uniformity correction matrix to improve the correction accuracy of the image and the authenticity of the final imaging image. Perform white balance processing on the original image data after correction processing based on the target gain of each color channel obtained from the scene light source confidence analysis. Even if there are large areas of monochromatic objects in the image, the reliability of image color correction can still be improved. Perform dead pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing, so that the obtained target imaging image further reduces the gap with the true color, improves the color expressiveness of the imaging image, and effectively optimizes the imaging quality of the multispectral sensor.

[0072] Embodiment 2

[0073] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a method for optimizing the imaging quality of a multispectral sensor in another embodiment of the present invention. The method includes:

[0074] S201: Analyze the focusing parameters of the multispectral sensor for the target object to obtain the target focusing parameters;

[0075] S202: Analyze the imaging impact of the ambient light intensity on the multispectral sensor to obtain imaging impact data, and match the target exposure parameters based on the imaging impact data;

[0076] S203: Construct a non-uniformity correction matrix, and perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focusing parameters and target exposure parameters based on the non-uniformity correction matrix to obtain the corrected original image data;

[0077] S204: Perform white balance processing on the corrected original image data based on the target gain of each color channel obtained from the scene light source confidence analysis to obtain the white balance processed original image data;

[0078] S205: Determine the analysis area of the point to be detected in the historical output image based on the neighborhood window, and perform vertical direction difference analysis by combining the adjacent pixel values based on the analysis area of the point to be detected to obtain the vertical direction difference analysis result;

[0079] S206: Construct a bad pixel judgment table based on the vertical direction difference analysis data combined with the threshold judgment method, and analyze the corresponding bad pixel in the first decomposed image data and the second decomposed image data based on the bad pixel judgment table;

[0080] S207: Determine the correction pixel values corresponding to the bad pixels in the first decomposed image data and the second decomposed image data based on color component analysis and distance analysis, and perform bad pixel correction on the first decomposed image data and the second decomposed image data based on the correction pixel values to obtain the first decomposed image data and the second decomposed image data after bad pixel correction;

[0081] S208: Perform channel correction on the first decomposed image data after bad pixel correction to obtain the first decomposed image data after channel correction, and perform linear conversion on the second decomposed image data after bad pixel correction to obtain the second decomposed image data after linear conversion;

[0082] S209: Perform fusion processing on the first decomposed image data after channel correction and the second decomposed image data after linear conversion to obtain the target imaging image.

[0083] In the embodiments of the present invention, by analyzing the imaging influence of ambient light intensity on a multispectral sensor to obtain imaging influence data for matching target exposure parameters, more accurate exposure parameters can be obtained, enabling the obtained exposure parameters to better adapt to the on-site environment. An inhomogeneity correction matrix is constructed, and based on the inhomogeneity correction matrix, the original image data is corrected to improve the correction accuracy of the image and enhance the authenticity of the final imaging image. Based on the target gains of each color channel obtained from the scene light source confidence analysis, white balance processing is performed on the corrected original image data. Even if there are large areas of monochromatic objects in the image, the reliability of image color correction can still be improved. Bad pixel correction, linear conversion, and fusion processing are performed on the first decomposed image data and the second decomposed image data obtained by decomposing the white balance-processed original image data, further narrowing the gap between the obtained target imaging image and the true color, enhancing the color expressiveness of the imaging image, and more effectively optimizing the imaging quality of the multispectral sensor.

[0084] Embodiment III

[0085] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of an imaging quality optimization device for a multispectral sensor in the embodiments of the present invention. The device includes:

[0086] Focus parameter analysis module 31: used to analyze the focus parameters of a multispectral sensor for a target object to obtain target focus parameters;

[0087] Exposure parameter analysis module 32: used to analyze the imaging influence of ambient light intensity on a multispectral sensor to obtain imaging influence data, and match target exposure parameters based on the imaging influence data;

[0088] Image correction module 33: used to construct an inhomogeneity correction matrix, and based on the inhomogeneity correction matrix, correct the original image data of the target object collected by the multispectral sensor according to the target focus parameters and target exposure parameters to obtain the corrected original image data;

[0089] White balance processing module 34: used to perform white balance processing on the corrected original image data based on the target gains of each color channel obtained from the scene light source confidence analysis to obtain the white balance-processed original image data;

[0090] Fusion optimization module 35: used to perform bad pixel correction, linear conversion, and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the white balance-processed original image data to obtain a target imaging image.

[0091] In the specific implementation process of the present invention, for the specific implementation manner of the device item, reference may be made to the implementation manner of the above method item, which will not be elaborated herein.

[0092] In an embodiment of the present invention, an imaging influence analysis of the ambient light intensity of a multispectral sensor is performed to obtain imaging influence data for matching target exposure parameters, so as to obtain more accurate exposure parameters, and make the obtained exposure parameters better adapt to the on-site environment. A non-uniformity correction matrix is constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the correction accuracy of the image and the authenticity of the final imaging image. Based on the target gain of each color channel obtained from the scene light source confidence analysis, white balance processing is performed on the corrected original image data. Even if there are large areas of monochromatic objects in the image, the reliability of image color correction can still be improved. Bad pixel correction, linear conversion, and fusion processing are performed on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing, so that the obtained target imaging image further reduces the gap with the true color, improves the color expressiveness of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor.

[0093] A computer-readable storage medium provided by an embodiment of the present invention stores a computer program on the computer-readable storage medium. When the program is executed by a processor, it implements the method for optimizing the imaging quality of a multispectral sensor in any one of the above embodiments. Among them, the computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card, or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (such as a computer, mobile phone), and can be a read-only memory, a magnetic disk, or an optical disk, etc.

[0094] Embodiment 4

[0095] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device in an embodiment of the present invention.

[0096] An embodiment of the present invention further provides an electronic device, such as Figure 4As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. Those skilled in the art can understand that Figure 3 the illustrated electronic device does not constitute a limitation on all devices and may include more or fewer components than shown, or combine certain components. The memory 41 can be used to store the computer program 42 and various functional modules. The processor 43 runs the computer program 42 stored in the memory 41 to execute various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include both an internal memory and an external memory. The internal memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory can include a hard disk, a floppy disk, a ZIP disk, a USB flash drive, a magnetic tape, etc. The processor 43 can be a central processing unit (CPU), or can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or the processor 43 can also be any conventional processor, etc. The processors and memories disclosed in the present invention include, but are not limited to, these types of processors and memories. The processors and memories disclosed in the present invention are only examples and not limitations.

[0097] As an embodiment, the electronic device includes: one or more processors 43, a memory 41, one or more computer programs 42, where the one or more computer programs 42 are stored in the memory 41 and are configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to execute the imaging quality optimization method of the multispectral sensor in any of the above embodiments. For the specific implementation process, please refer to the above embodiments and will not be elaborated here.

[0098] In the embodiments of the present invention, an imaging impact analysis of the ambient light intensity of the multispectral sensor is performed to obtain imaging impact data for matching target exposure parameters, so as to obtain more accurate exposure parameters, and enable the obtained exposure parameters to better adapt to the on-site environment. A non-uniformity correction matrix is constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the correction accuracy of the image and the authenticity of the final imaging image. Based on the target gains of each color channel obtained from the scene light source confidence analysis, white balance processing is performed on the corrected original image data. Even if there are large areas of monochromatic objects in the image, the reliability of image color correction can still be improved. Bad pixel correction, linear conversion, and fusion processing are performed on the first decomposition image data and the second decomposition image data obtained by decomposing the original image data after white balance processing, so that the obtained target imaging image further reduces the gap with the true color, improves the color expressiveness of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor.

[0099] In addition, the above has introduced in detail an imaging quality optimization method and related device for a multispectral sensor provided by the embodiments of the present invention. Specific examples should have been used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for optimizing the imaging quality of a multispectral sensor, characterized in that: The method comprises: Performing a focus parameter analysis of the multispectral sensor on the target object to obtain the target focus parameter; Performing imaging impact analysis of ambient light intensity on a multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data; Constructing a non-uniformity correction matrix, and performing correction processing on raw image data of a target object collected by a multispectral sensor according to target focus parameters and target exposure parameters based on the non-uniformity correction matrix to obtain raw image data after correction processing; Performing white balance processing on the original image data after the correction processing based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain the original image data after the white balance processing; The first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion and fusion processing to obtain a target imaging image.

2. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The step of analyzing the focus parameters of the multispectral sensor on the target object to obtain the target focus parameters includes: Calculate a first phase difference between a first initial image and a second initial image of the target object captured by the multispectral sensor at each position based on the pixel brightness curve; calculating a second phase difference between a first phase map generated by the first initial image and a phase map generated by the second initial image; Focus parameter analysis of the multispectral sensor is performed based on the first phase difference and the second phase difference to obtain target focus parameters.

3. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The step of performing imaging impact analysis of ambient light intensity on the multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data, includes: Integrate the features of the historical ambient light impact imaging record set to obtain target feature data; Based on the historical ambient light impact imaging record set, the correlation between different ambient light intensities and different imaging parameters is analyzed; Based on the target feature data and the degree of association, an initial imaging impact analysis model is adjusted at nodes to obtain a target imaging impact analysis model, and based on the target imaging impact analysis model, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor to obtain imaging impact data; Exposure parameters are matched based on imaging impact data and mapping relationships to obtain target exposure parameters.

4. The imaging quality optimization method of a multispectral sensor according to claim 3, characterized in that: The step of adjusting nodes of the initial imaging impact analysis model based on the target feature data and the degree of association to obtain the target imaging impact analysis model includes: Determine the data label based on the association degree and the weight constraint of the target feature data, and evaluate the network node using the evaluation function based on the data label to obtain the network node evaluation coefficient; The initial imaging impact analysis model is adjusted based on the network node evaluation coefficient to obtain a target imaging impact analysis model.

5. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The constructing of the non-uniformity correction matrix, and performing correction processing on the original image data of the target object collected by the multi-spectral sensor according to the target focus parameter and the target exposure parameter based on the non-uniformity correction matrix to obtain the corrected original image data, include: Determine the reference area in the calibration multispectral image based on the multi-channel response curve, and calculate the average pixel data of each channel in the reference area; A non-uniformity correction matrix is ​​constructed based on the pixel point data of each channel in the calibrated multispectral image and the average pixel point data of each channel in the reference area; Performing non-uniformity correction processing on the original image data based on the non-uniformity correction matrix to obtain the original image data after the non-uniformity correction processing; Based on adaptive instance normalization and spline fitting, spectral correction processing is performed on the original image data after non-uniform correction processing to obtain the original image data after correction processing.

6. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The step of performing white balance processing on the original image data after correction based on the target gain of each color channel obtained by analyzing the confidence of the scene light source to obtain the original image data after white balance processing includes: Perform cluster analysis on the spectrum of each scene light source to obtain the corresponding light source category, and perform model training based on the light source category and the light intensity received by the pixels of each scene light source to obtain a light source prediction model; Determining light source prediction information and corresponding confidence using real-time scene spectrum data based on the light source prediction model; Determine an initial color temperature corresponding to the light source prediction information based on the tristimulus value curve, calculate a target color temperature based on the initial color temperature using the confidence corresponding to the light source prediction information, and determine a target gain of each color channel based on the target color temperature; The pixel values ​​of each color channel in the original image data after the correction processing are transformed based on the target gain of each color channel to obtain the original image data after the white balance processing.

7. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The step of performing bad pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain the target imaging image includes: Determine the analysis area of ​​the point to be detected in the historical output image based on the neighborhood window, perform vertical difference analysis based on the analysis area of ​​the point to be detected combined with adjacent pixel values, and obtain a vertical difference analysis result; Constructing a bad pixel judgment table based on the vertical difference analysis data combined with a threshold judgment method, and analyzing corresponding bad pixel pixels in the first decomposed image data and the second decomposed image data based on the bad pixel judgment table; Determine the correction pixel value corresponding to the bad pixel pixel in the first decomposed image data and the second decomposed image data based on the color component analysis and the distance analysis, and perform bad pixel correction on the first decomposed image data and the second decomposed image data based on the correction pixel value to obtain the first decomposed image data and the second decomposed image data after the bad pixel correction; Performing channel correction on the first decomposed image data after bad pixel correction to obtain the first decomposed image data after channel correction, and performing linear transformation on the second decomposed image data after bad pixel correction to obtain the second decomposed image data after linear transformation; The first decomposed image data after channel correction and the second decomposed image data after linear transformation are fused to obtain a target imaging image.

8. An imaging quality optimization device for a multispectral sensor, characterized in that: The device comprises: Focus parameter analysis module: used to analyze the focus parameters of the multispectral sensor on the target object and obtain the target focus parameters; Exposure parameter analysis module: used to analyze the imaging impact of ambient light intensity on the multispectral sensor, obtain imaging impact data, and match target exposure parameters based on the imaging impact data; Image correction module: used to construct a non-uniformity correction matrix, and based on the non-uniformity correction matrix, perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameter and the target exposure parameter to obtain the corrected original image data; White balance processing module: used for performing white balance processing on the original image data after correction processing based on the target gain of each color channel obtained by the scene light source confidence analysis, so as to obtain the original image data after white balance processing; Fusion optimization module: used for performing bad pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain the target imaging image.

9. An electronic device, comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the imaging quality optimization method of the multi-spectral sensor according to any one of claims 1 to claim 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the imaging quality optimization method for a multi-spectral sensor according to any one of claims 1 to 7.

Images