A method and related device for optimizing imaging quality of multispectral sensor

Abstract

The present invention discloses a method and related device for optimizing the imaging quality of a multispectral sensor, relating to the field of image processing technology. The method comprises: analyzing the focus parameters of the multispectral sensor on a target object to obtain the target focus parameters; analyzing the imaging impact of ambient light intensity on the multispectral sensor to match the target exposure parameters; correcting the original image data based on a non-uniformity correction matrix; performing white balance processing on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis; and performing bad pixel correction, linear conversion, and fusion processing on first and second decomposed image data obtained by decomposing the white-balanced original image data to obtain the target image. The present invention improves the color expression of the imaged image and more effectively optimizes the imaging quality of the multispectral sensor.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for optimizing the imaging quality of a multispectral sensor and related devices. Background Art

[0002] Because multispectral sensors can capture multiple spectral information in the environment, providing more realistic and accurate color reproduction, most companies have already incorporated them into their camera products. However, with increasing demands for image quality, optimizing multispectral sensor imaging quality has become a key research focus. In multispectral sensor imaging, exposure parameters are crucial to image brightness. Currently, exposure parameters are typically analyzed using histogram statistics of test images. However, this approach does not adapt well to the actual environment, resulting in insufficient brightness in the resulting images. Because multispectral sensors have more channels than typical image sensors, their generated images exhibit certain non-uniformity errors. Currently, most approaches lack methods to correct for this non-uniformity error, resulting in insufficient fidelity in the resulting images. White balancing is also a crucial step for multispectral sensor imaging. Currently, grayscale algorithms are typically used for white balancing. However, if large monochromatic objects are present in the image, this approach can lead to inaccurate color correction, compromising image quality and rendering the image's color rendition ineffectively. This ineffectiveness in optimizing multispectral sensor imaging quality is ineffective. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a method and related devices for optimizing the imaging quality of a multispectral sensor, which improves the color expression of the imaged image and enables the imaging quality of the multispectral sensor to be more effectively optimized.

[0004] In order to solve the above technical problems, the present invention provides a method for optimizing the imaging quality of a multispectral sensor, the method comprising:

[0005] Performing focus parameter analysis of the multispectral sensor on the target object to obtain the target focus parameters;

[0006] Performing an imaging impact analysis of ambient light intensity on a multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data;

[0007] Constructing a non-uniformity correction matrix, and performing correction processing on raw image data of a target object collected by a multispectral sensor according to target focus parameters and target exposure parameters based on the non-uniformity correction matrix to obtain corrected raw image data;

[0008] performing white balance processing on the corrected raw image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain raw image data after white balance processing;

[0009] Defective pixel correction, linear conversion and fusion processing are performed on first decomposed image data and second decomposed image data obtained by decomposing the original image data after white balance processing to obtain a target imaging image.

[0010] Optionally, performing a focus parameter analysis of a multispectral sensor on a target object to obtain a target focus parameter includes:

[0011] Calculating a first phase difference between a first initial image and a second initial image of the target object captured by the multispectral sensor at each position based on the pixel brightness curve;

[0012] calculating a second phase difference between a first phase map generated by the first initial image and a phase map generated by the second initial image;

[0013] Focus parameter analysis of the multispectral sensor is performed based on the first phase difference and the second phase difference to obtain target focus parameters.

[0014] Optionally, performing imaging impact analysis of ambient light intensity on the multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data, includes:

[0015] Perform feature integration on the historical ambient light impact imaging record set to obtain target feature data;

[0016] Analyze the correlation between different ambient light intensities and different imaging parameters based on the historical ambient light impact imaging record set;

[0017] Performing node adjustments on the initial imaging impact analysis model based on the target feature data and the degree of correlation to obtain a target imaging impact analysis model, and performing imaging impact analysis of ambient light intensity on the multispectral sensor based on the target imaging impact analysis model to obtain imaging impact data;

[0018] Exposure parameters are matched based on imaging impact data and mapping relationships to obtain target exposure parameters.

[0019] Optionally, performing node adjustment on the initial imaging impact analysis model based on the target feature data and the correlation degree to obtain the target imaging impact analysis model includes:

[0020] Determine data labels based on the degree of association and the weight constraint of target feature data, and evaluate network nodes using an evaluation function based on the data labels to obtain network node evaluation coefficients;

[0021] The initial imaging impact analysis model is adjusted for nodes based on the network node evaluation coefficients to obtain a target imaging impact analysis model.

[0022] Optionally, constructing a non-uniformity correction matrix and performing correction processing on raw image data of a target object captured by a multispectral sensor according to target focus parameters and target exposure parameters based on the non-uniformity correction matrix to obtain the corrected raw image data includes:

[0023] Determine the reference area in the calibration multispectral image based on the multi-channel response curve, and calculate the average pixel data of each channel in the reference area;

[0024] A non-uniformity correction matrix is ​​constructed based on the pixel data of each channel in the calibrated multispectral image and the average pixel data of each channel in the reference area;

[0025] Performing non-uniformity correction processing on the original image data based on the non-uniformity correction matrix to obtain the original image data after the non-uniformity correction processing;

[0026] Based on adaptive instance normalization and spline fitting, spectral correction processing is performed on the original image data after non-uniform correction processing to obtain the corrected original image data.

[0027] Optionally, performing white balance processing on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain the white balanced original image data includes:

[0028] Perform cluster analysis on the spectrum of each scene light source to obtain the corresponding light source category, and perform model training based on the light source category and the pixel received light intensity of each scene light source to obtain a light source prediction model;

[0029] Determining light source prediction information and corresponding confidence levels using real-time scene spectrum data based on the light source prediction model;

[0030] determining an initial color temperature corresponding to the light source prediction information based on a tristimulus value curve, calculating a target color temperature based on the initial color temperature using a confidence level corresponding to the light source prediction information, and determining a target gain for each color channel based on the target color temperature;

[0031] The pixel values ​​of each color channel in the corrected original image data are transformed based on the target gain of each color channel to obtain the original image data after white balance processing.

[0032] Optionally, performing bad pixel correction, linear conversion, and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain the target imaging image includes:

[0033] Determine the analysis area of ​​the point to be detected in the historical output image based on the neighborhood window, perform vertical difference analysis based on the analysis area of ​​the point to be detected and combined with adjacent pixel values ​​to obtain a vertical difference analysis result;

[0034] Constructing a bad pixel judgment table based on the vertical difference analysis data in combination with a threshold judgment method, and analyzing corresponding bad pixel pixels in the first decomposed image data and the second decomposed image data based on the bad pixel judgment table;

[0035] Determining corrected pixel values ​​corresponding to bad pixel pixels in the first decomposed image data and the second decomposed image data based on color component analysis and distance analysis, and performing bad pixel correction on the first decomposed image data and the second decomposed image data based on the corrected pixel values ​​to obtain first decomposed image data and second decomposed image data after bad pixel correction;

[0036] Performing channel correction on the first decomposed image data after bad pixel correction to obtain first decomposed image data after channel correction, and performing linear transformation on the second decomposed image data after bad pixel correction to obtain second decomposed image data after linear transformation;

[0037] The first decomposed image data after channel correction and the second decomposed image data after linear transformation are fused to obtain a target imaging image.

[0038] In addition, the present invention also provides an imaging quality optimization device for a multispectral sensor, the device comprising:

[0039] Focus parameter analysis module: used to analyze the focus parameters of the multispectral sensor on the target object and obtain the target focus parameters;

[0040] Exposure parameter analysis module: used to analyze the imaging impact of ambient light intensity on the multispectral sensor, obtain imaging impact data, and match target exposure parameters based on the imaging impact data;

[0041] An image correction module is configured to construct a non-uniformity correction matrix, and to perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameters and the target exposure parameters based on the non-uniformity correction matrix to obtain the corrected original image data;

[0042] White balance processing module: used to perform white balance processing on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis, to obtain the original image data after white balance processing;

[0043] Fusion optimization module: used to perform bad pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain the target imaging image.

[0044] In addition, the present invention also provides an electronic device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the above-mentioned imaging quality optimization method of the multispectral sensor.

[0045] In addition, the present invention provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned imaging quality optimization method of the multispectral sensor.

[0046] In an embodiment of the present invention, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor to obtain imaging impact data to match target exposure parameters, thereby obtaining more accurate exposure parameters that can better adapt to the on-site environment. A non-uniformity correction matrix is ​​constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the image correction accuracy and the authenticity of the final image. White balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis. Even if there are large-area monochrome objects in the image, the reliability of image color correction can still be improved. The first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion, and fusion processing, so that the obtained target imaging image further narrows the gap with the true color, improves the color expression of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 1 is a flow chart of a method for optimizing imaging quality of a multispectral sensor according to an embodiment of the present invention;

[0049] Figure 2 is a flow chart of a method for optimizing imaging quality of a multispectral sensor according to another embodiment of the present invention;

[0050] Figure 3 Schematic diagram of the structure of the imaging quality optimization device of the multispectral sensor in an embodiment of the present invention;

[0051] Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] Example 1

[0054] See also Figure 1 , Figure 1 : is a flow chart of a method for optimizing imaging quality of a multispectral sensor according to an embodiment of the present invention, the method comprising:

[0055] S11: performing focus parameter analysis of the multispectral sensor on the target object to obtain target focus parameters;

[0056] In a specific implementation of the present invention, performing a focus parameter analysis of a multispectral sensor on a target object to obtain a target focus parameter includes: calculating a first phase difference between a first initial image and a second initial image of the target object captured by the multispectral sensor at each position based on a pixel brightness curve; calculating a second phase difference between a first phase map generated by the first initial image and a phase map generated by the second initial image; and performing a focus parameter analysis of the multispectral sensor based on the first phase difference and the second phase difference to obtain a target focus parameter.

[0057] Specifically, the multispectral sensor is a multispectral image sensor. Compared to other general image sensors, it can better capture multiple spectral information in the environment to provide more realistic and accurate color reproduction. Therefore, selecting a multispectral sensor can better improve the quality of the results. Based on the pixel brightness curve, the first phase difference between the first initial image and the second initial image of the target object captured by the multispectral sensor at each position is calculated. The multispectral sensor captures an image of the target object at each position, and the images are represented as the first initial image and the second initial image respectively using a spectroscopic system. The images are plotted with the brightness values ​​of the pixels in the first and second initial images as the vertical axis and the imaging width value of the multispectral sensor as the horizontal axis to form a pixel brightness curve. The offset between the pixel brightness curve of the first initial image and the pixel brightness curve of the second initial image is calculated, and the offset at each position is used as the first phase difference. The second phase difference between the first phase map generated from the first initial image and the phase map generated from the second initial image is calculated. The first phase map is constructed based on the pixel information of several pixels selected from the first initial image. The second phase map is generated in the same manner as the first phase map. The phase difference between the first phase map and the second phase map is calculated as the second phase difference. Based on the first phase difference and the second phase difference, a focus parameter analysis of the multispectral sensor is performed to obtain a target focus parameter. The average phase difference is calculated according to the first phase difference and the second phase difference at each position. The focus position information of the multispectral sensor is obtained by performing an operation based on the average phase difference combined with the corresponding correction value, that is, the target focus parameter is obtained.

[0058] S12: performing imaging impact analysis of ambient light intensity on the multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data;

[0059] In the specific implementation process of the present invention, the imaging impact analysis of ambient light intensity on the multispectral sensor is performed to obtain imaging impact data, and the target exposure parameters are matched based on the imaging impact data, including: feature integration of historical ambient light impact imaging record sets to obtain target feature data; analyzing the degree of correlation between different ambient light intensities and different imaging parameters based on the historical ambient light impact imaging record sets; adjusting the nodes of the initial imaging impact analysis model based on the target feature data and the degree of correlation to obtain a target imaging impact analysis model, and performing imaging impact analysis of ambient light intensity on the multispectral sensor based on the target imaging impact analysis model to obtain imaging impact data; and matching exposure parameters based on the imaging impact data and the mapping relationship to obtain target exposure parameters.

[0060] Furthermore, the node adjustment of the initial imaging impact analysis model based on the target feature data and the degree of association to obtain the target imaging impact analysis model includes: determining data labels based on the weight constraints of the degree of association and the target feature data, and evaluating network nodes using an evaluation function based on the data labels to obtain network node evaluation coefficients; and adjusting the nodes of the initial imaging impact analysis model based on the network node evaluation coefficients to obtain the target imaging impact analysis model.

[0061] Specifically, feature integration is performed on the historical ambient light impact imaging record set. The historical ambient light impact imaging record set includes recorded data on the impact of different ambient light intensities on the imaging of multispectral sensors in the past, such as the impact of different ambient light intensities on exposure parameters and imaging brightness. Feature extraction is performed on the historical ambient light impact imaging record set through a feature extractor to obtain several feature table sets. Feature integration of the several feature table sets can be performed in a horizontal merging manner to obtain target feature data. Based on the historical ambient light impact imaging record set, the correlation between different ambient light intensities and different imaging parameters is analyzed. The correlation between different ambient light intensities and different imaging parameters is analyzed through an association rule algorithm, and the frequency of occurrence of the same imaging parameters under different ambient light intensities is analyzed. A data label is determined based on the degree of association and the weight constraint of the target feature data. The target feature data is assigned a weight constraint, which is pre-set in a database. The corresponding data label is determined in the label database based on the degree of association and the weight constraint of the target feature data. The data label can be determined using the data label. Network node evaluation is performed based on the data label using an evaluation function. Specifically, a node evaluation is performed on a pre-set initial topology graph using the evaluation function based on the data label, evaluating the importance of each node and obtaining a network node evaluation coefficient. Node adjustments are performed on an initial imaging impact analysis model based on the network node evaluation coefficient. The initial imaging impact analysis model is a topology graph. Node adjustments to the initial imaging impact analysis model can include adjusting distances between nodes or removing / adding nodes, thereby obtaining a target imaging impact analysis model. Based on the target imaging impact analysis model, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor. Real-time ambient light intensity information is input into the target imaging impact model to obtain the exposure impact of the current real-time ambient light intensity information on the multispectral sensor imaging, i.e., imaging impact data. Exposure parameter matching is performed based on the imaging impact data and a mapping relationship. Exposure parameters are matched in the database using the corresponding mapping relationship based on the imaging impact data to obtain target exposure parameters.

[0062] S13: constructing a non-uniformity correction matrix, and performing correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameter and the target exposure parameter based on the non-uniformity correction matrix to obtain the corrected original image data;

[0063] In a specific implementation of the present invention, constructing a non-uniformity correction matrix and performing correction processing on raw image data of a target object captured by a multispectral sensor according to target focus parameters and target exposure parameters based on the non-uniformity correction matrix to obtain corrected raw image data include: determining a reference area in a calibrated multispectral image based on a multi-channel response curve, and calculating average pixel data of each channel in the reference area; constructing a non-uniformity correction matrix based on the pixel data of each channel in the calibrated multispectral image and the average pixel data of each channel in the reference area; performing non-uniformity correction processing on the raw image data based on the non-uniformity correction matrix to obtain non-uniformity corrected raw image data; and performing spectral correction processing on the raw image data after the non-uniformity correction based on adaptive instance normalization and spline fitting to obtain corrected raw image data.

[0064] Specifically, a multispectral sensor is controlled to capture an image of a target object based on target focus parameters and target exposure parameters, obtaining raw image data, and transmitting the raw image data to an image processor for processing. A reference region in the calibration multispectral image is determined based on the multichannel response curves. A first response curve of each channel to a wide-band spectrum when there is no non-uniformity error is obtained. A second response curve of each channel to a wide-band spectrum when the multispectral sensor is actually in use is obtained. The degree of overlap between the first and second response curves is calculated. The response curve with the highest degree of overlap in each channel is defined as the desired response curve. The desired response curves are combined into a multichannel response curve. Ideal pixels in the curves are obtained based on the multichannel response curves. A reference region is determined based on the ideal pixels, and average pixel data for each channel in the reference region is calculated, i.e., the average value of the pixel values ​​for each channel in the reference region is calculated. A non-uniformity correction matrix is ​​constructed based on the pixel data for each channel in the calibration multispectral image and the average pixel data for each channel in the reference region. The pixel data for each channel is divided by the average pixel data for each channel in the reference region to obtain the corresponding non-uniformity correction matrix. The original image data is subjected to non-uniformity correction based on a non-uniformity correction matrix. The data points of each channel in the original image data are divided by the corresponding non-uniformity correction matrix to obtain the corrected data for each channel, thereby obtaining the original image data after non-uniformity correction. The original image data after non-uniformity correction is subjected to spectral correction based on adaptive instance normalization and spline fitting. Scaling and translation coefficients are obtained through a learnable radiometric transformation. The non-uniformity correction is then recalibrated channel by channel based on the scaling and translation coefficients to obtain the recalibrated original image data channel by channel, thus achieving adaptive instance normalization. The spectral distribution data of each channel in the recalibrated original image data channel by channel is segmented to obtain a number of corresponding segmented spectral data. Spline fitting is performed on the segmented spectral data using a preset cubic polynomial to obtain a number of segmented spectral data after spline fitting. The spectral ratio of the image is corrected based on the light source model combined with the segmented spectral data after spline fitting to ensure the true spectral characteristics of the image, thereby obtaining the corrected original image data.

[0065] S14: performing white balance processing on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain white balanced original image data;

[0066] In a specific implementation of the present invention, the white balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain the original image data after white balance processing, including: clustering analysis of the spectrum of each scene light source to obtain the corresponding light source category, and model training based on the light source category and the pixel received light intensity of each scene light source to obtain a light source prediction model; based on the light source prediction model, light source prediction information and the corresponding confidence are determined using real-time scene spectrum data; an initial color temperature corresponding to the light source prediction information is determined based on a tristimulus value curve, a target color temperature is calculated based on the initial color temperature using the confidence corresponding to the light source prediction information, and a target gain of each color channel is determined based on the target color temperature; and pixel values ​​of each color channel in the corrected original image data are transformed based on the target gain of each color channel to obtain the original image data after white balance processing.

[0067] Specifically, cluster analysis is performed on the spectra of each scene light source using a k-means clustering algorithm. The light source spectral similarity between any two cluster centers is greater than a preset classification standard, and within each cluster, the spectral similarity between any two light sources is less than the preset classification standard. The corresponding light source category is obtained, and a model is trained based on the light source category and the pixel received light intensity of each scene light source. The light source category and the pixel received light intensity of each scene light source are used as a data set, and the pixel received light intensity is the light intensity of the light received by each pixel. A deep neural network is trained to obtain a light source prediction model. Based on the light source prediction model, light source prediction information and corresponding confidence levels are determined using real-time scene spectral data. The real-time scene spectral data is input into the light source prediction model, and the output layer of the light source prediction model outputs the light source prediction type and corresponding confidence level. An initial color temperature corresponding to the light source prediction information is determined based on a tristimulus value curve. Corresponding tristimulus values ​​are determined in the tristimulus value curve based on the light source prediction information. Corresponding color coordinates are calculated based on the tristimulus values. The corresponding initial color temperature is calculated based on the color coordinates. A target color temperature is calculated based on the initial color temperature using a confidence level corresponding to the light source prediction information. The initial color temperature is weighted and summed using the confidence level to obtain a target color temperature. A target gain for each color channel is determined based on the target color temperature. The target gain, i.e., the gain value of each color channel, is matched in a database based on the target color temperature. The pixel values ​​of each color channel in the corrected original image data are transformed based on the target gain of each color channel. Specifically, the pixel values ​​of each color channel in the corrected original image data are multiplied by the target gain of each color channel to obtain white-balanced original image data.

[0068] S15: performing bad pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after the white balance processing to obtain a target imaging image.

[0069] In the specific implementation process of the present invention, the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion and fusion processing to obtain the target imaging image, including: determining the analysis area of ​​the point to be detected in the historical output image based on the neighborhood window, performing vertical direction difference analysis based on the analysis area of ​​the point to be detected in combination with adjacent pixel values ​​to obtain the vertical direction difference analysis result; constructing a bad pixel judgment table based on the vertical direction difference analysis data in combination with a threshold judgment method, and analyzing the corresponding bad pixel pixels in the first decomposed image data and the second decomposed image data based on the bad pixel judgment table; analyzing the bad pixel pixels based on the color component analysis The method comprises the following steps: determining the corrected pixel values ​​corresponding to the bad pixel pixels in the first decomposed image data and the second decomposed image data by performing bad pixel correction based on the corrected pixel values ​​to obtain the first decomposed image data and the second decomposed image data after the bad pixel correction; performing channel correction on the first decomposed image data after the bad pixel correction to obtain the first decomposed image data after the channel correction; and performing linear transformation on the second decomposed image data after the bad pixel correction to obtain the second decomposed image data after the linear transformation; and performing fusion processing on the first decomposed image data after the channel correction and the second decomposed image data after the linear transformation to obtain the target imaging image.

[0070] Specifically, an analysis area of ​​a point to be detected in a historical output image is determined based on a neighborhood window, the size of the neighborhood window is preset, and the analysis area corresponding to the point to be detected is determined according to the neighborhood window. A vertical difference analysis is performed based on the analysis area of ​​the point to be detected in combination with adjacent pixel values. The point to be detected is combined with the upper and lower adjacent pixel values ​​in the same column in the analysis area to perform a vertical difference analysis to obtain an upper pixel difference value and a lower pixel difference value. The relationship between the upper pixel difference value and the lower pixel difference value and the preset difference threshold is analyzed respectively, and the vertical difference analysis result is obtained. A bad pixel judgment table is constructed based on the vertical difference analysis data in combination with a threshold judgment method. In the vertical difference analysis results, when the upper pixel difference value and the lower pixel difference value are both less than a preset difference threshold, the point to be detected is combined with the left and right adjacent pixel values ​​in the same column for horizontal difference analysis, the column average pixel value of the adjacent columns in the analysis area of ​​the point to be detected is determined, and the difference between the column average pixel value and the pixel value of the point to be detected is calculated, that is, horizontal analysis data is obtained. The corresponding minimum value is determined in the horizontal analysis data, and the minimum value is compared with a preset judgment threshold. If the minimum value is greater than the preset judgment threshold, the point to be detected is a bad pixel; otherwise, the point to be detected is not a bad pixel. Therefore, a preset counter is used in combination with the bad pixel judgment results to construct a bad pixel judgment table, and the corresponding bad pixel pixels in the first decomposed image data and the second decomposed image data are analyzed based on the bad pixel judgment table, that is, the bad pixel pixels existing in the first decomposed image data and the second decomposed image data are judged according to the bad pixel judgment table. Based on color component analysis and distance analysis, correction pixel values ​​corresponding to bad pixels in the first decomposed image data and the second decomposed image data are determined. A first pixel point that is closest to the bad pixel and has the same color component as the bad pixel is determined in the first decomposed image data and the second decomposed image data. Several pairs of second pixels that are symmetrical with the first pixel point and have the same color component as the bad pixel are determined. Target pixel values ​​of several adjacent valid pixels of the bad pixel are obtained based on the pixel values ​​of the first pixel point and the second pixel point. Pixel gradients in the horizontal, vertical, 45-degree, and 135-degree directions are calculated based on the target pixel values. Several target adjacent valid pixels are determined based on the pixel gradients in the horizontal, vertical, 45-degree, and 135-degree directions. Corresponding correction pixel values ​​are matched based on the average values ​​of the pixel values ​​corresponding to the several target adjacent valid pixels. Bad pixel correction is performed on the first decomposed image data and the second decomposed image data based on the correction pixel values. That is, the bad pixel pixels in the first decomposed image data and the second decomposed image data are corrected based on the correction pixel values ​​to obtain the first decomposed image data and the second decomposed image data after bad pixel correction.Channel correction is performed on the first decomposed image data after bad pixel correction, a channel correction matrix is ​​constructed using preset residual complementary color parameters, channel correction is performed on the first decomposed image data after bad pixel correction according to the channel correction matrix to obtain the first decomposed image data after channel correction, linear conversion is performed on the second decomposed image data after bad pixel correction, demosaicing is performed on the second decomposed image data after bad pixel correction to obtain the second decomposed image data after demosaicing, and the second decomposed image data after demosaicing is converted and resampled based on the residual mapping model to obtain the second decomposed image data after linear conversion. The first decomposed image data after channel correction and the second decomposed image data after linear transformation are fused, the pixel values ​​of the pixels at the same position in the first decomposed image data after channel correction and the second decomposed image data after linear transformation are added to obtain a preliminary fused image, the preliminary fused image is masked according to the first mask to obtain a first input image, the first decomposed image data after channel correction is masked according to the second mask to obtain a second input image, the pixel values ​​of the pixels at the same position in the first input image and the second input image are added to obtain a target imaging image, and by adjusting the channel gain of the split image, the gap with the true color is further narrowed, the color expression of the image is improved, and the imaging quality is further improved.

[0071] In an embodiment of the present invention, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor to obtain imaging impact data to match target exposure parameters, thereby obtaining more accurate exposure parameters that can better adapt to the on-site environment. A non-uniformity correction matrix is ​​constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the image correction accuracy and the authenticity of the final image. White balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis. Even if there are large-area monochrome objects in the image, the reliability of image color correction can still be improved. The first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion, and fusion processing, so that the obtained target imaging image further narrows the gap with the true color, improves the color expression of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor.

[0072] Example 2

[0073] See also Figure 2 , Figure 2 FIG. 5 is a flow chart of a method for optimizing imaging quality of a multispectral sensor according to another embodiment of the present invention, wherein the method comprises:

[0074] S201: Analyzing the focus parameters of the multispectral sensor on the target object to obtain the target focus parameters;

[0075] S202: performing imaging impact analysis of ambient light intensity on the multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data;

[0076] S203: constructing a non-uniformity correction matrix, and performing correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameter and the target exposure parameter based on the non-uniformity correction matrix to obtain corrected original image data;

[0077] S204: performing white balance processing on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain white balanced original image data;

[0078] S205: determining an analysis area of ​​the point to be detected in the historical output image based on the neighborhood window, performing vertical difference analysis based on the analysis area of ​​the point to be detected and combining adjacent pixel values ​​to obtain a vertical difference analysis result;

[0079] S206: constructing a bad pixel judgment table based on the vertical difference analysis data in combination with a threshold judgment method, and analyzing corresponding bad pixel pixels in the first decomposed image data and the second decomposed image data based on the bad pixel judgment table;

[0080] S207: Determine, based on color component analysis and distance analysis, correction pixel values ​​corresponding to bad pixel pixels in the first decomposed image data and the second decomposed image data, and perform bad pixel correction on the first decomposed image data and the second decomposed image data based on the correction pixel values ​​to obtain first decomposed image data and second decomposed image data after bad pixel correction;

[0081] S208: performing channel correction on the first decomposed image data after bad pixel correction to obtain first decomposed image data after channel correction, and performing linear transformation on the second decomposed image data after bad pixel correction to obtain second decomposed image data after linear transformation;

[0082] S209: performing fusion processing on the first decomposed image data after channel correction and the second decomposed image data after linear transformation to obtain a target imaging image.

[0083] In an embodiment of the present invention, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor to obtain imaging impact data to match target exposure parameters, thereby obtaining more accurate exposure parameters that can better adapt to the on-site environment. A non-uniformity correction matrix is ​​constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the image correction accuracy and the authenticity of the final image. White balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis. Even if there are large-area monochrome objects in the image, the reliability of image color correction can still be improved. The first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion, and fusion processing, so that the obtained target imaging image further narrows the gap with the true color, improves the color expression of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor.

[0084] Example 3

[0085] See also Figure 3 , Figure 3 : is a schematic diagram of the structural composition of an imaging quality optimization device for a multispectral sensor in an embodiment of the present invention, the device comprising:

[0086] Focus parameter analysis module 31: used to analyze the focus parameters of the multispectral sensor on the target object and obtain the target focus parameters;

[0087] Exposure parameter analysis module 32: configured to analyze the imaging impact of ambient light intensity on the multispectral sensor, obtain imaging impact data, and match target exposure parameters based on the imaging impact data;

[0088] Image correction module 33: used to construct a non-uniformity correction matrix, and perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameters and the target exposure parameters based on the non-uniformity correction matrix to obtain the corrected original image data;

[0089] White balance processing module 34: configured to perform white balance processing on the corrected raw image data based on the target gain of each color channel obtained by the scene light source confidence analysis, to obtain raw image data after white balance processing;

[0090] The fusion optimization module 35 is used to perform bad pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after the white balance processing, so as to obtain the target imaging image.

[0091] In the specific implementation process of the present invention, the specific implementation method of the device item can refer to the implementation method of the above-mentioned method item, which will not be repeated here.

[0092] In an embodiment of the present invention, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor to obtain imaging impact data to match target exposure parameters, thereby obtaining more accurate exposure parameters that can better adapt to the on-site environment. A non-uniformity correction matrix is ​​constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the image correction accuracy and the authenticity of the final image. White balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis. Even if there are large-area monochrome objects in the image, the reliability of image color correction can still be improved. The first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion, and fusion processing, so that the obtained target imaging image further narrows the gap with the true color, improves the color expression of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor.

[0093] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the imaging quality optimization method for a multispectral sensor according to any of the above-described embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, a storage device includes any medium that can store or transmit information in a readable form by a device (e.g., a computer or a mobile phone), and can be a read-only memory, a disk, or an optical disk.

[0094] Example 4

[0095] See also Figure 4 , Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0096] The embodiment of the present invention further provides an electronic device, such as Figure 4As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. It will be understood by those skilled in the art that Figure 3 The electronic devices shown do not constitute a limitation on all devices and may include more or fewer components than shown, or combinations of certain components. The memory 41 can be used to store the computer program 42 and various functional modules, and the processor 43 runs the computer program 42 stored in the memory 41, thereby executing various functional applications and data processing of the device. The memory can be internal memory or external memory, or include both internal memory and external memory. The internal memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. The external memory can include a hard disk, floppy disk, ZIP disk, USB flash drive, magnetic tape, etc. The processor 43 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, a single-chip microcomputer, or processor 43, or any conventional processor. The processor and memory disclosed in the present invention include but are not limited to these types of processors and memories. The processor and memory disclosed in the present invention are only examples and not limitations.

[0097] As an embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and are configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to execute the imaging quality optimization method of the multispectral sensor in any of the above-mentioned embodiments. For the specific implementation process, please refer to the above-mentioned embodiments and will not be repeated here.

[0098] In an embodiment of the present invention, an imaging impact analysis of ambient light intensity is performed on a multispectral sensor to obtain imaging impact data to match target exposure parameters, thereby obtaining more accurate exposure parameters that can better adapt to the on-site environment. A non-uniformity correction matrix is ​​constructed, and the original image data is corrected based on the non-uniformity correction matrix to improve the image correction accuracy and the authenticity of the final image. White balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by scene light source confidence analysis. Even if there are large-area monochrome objects in the image, the reliability of image color correction can still be improved. The first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing are subjected to bad pixel correction, linear conversion, and fusion processing, so that the obtained target imaging image further narrows the gap with the true color, improves the color expression of the imaging image, and more effectively optimizes the imaging quality of the multispectral sensor.

[0099] In addition, the above describes in detail the imaging quality optimization method and related devices of a multispectral sensor provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for optimizing the imaging quality of a multispectral sensor, characterized in that: The method comprises: Performing focus parameter analysis of the multispectral sensor on the target object to obtain the target focus parameters; Performing an imaging impact analysis of ambient light intensity on a multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data; Constructing a non-uniformity correction matrix, and performing correction processing on raw image data of a target object collected by a multispectral sensor according to target focus parameters and target exposure parameters based on the non-uniformity correction matrix to obtain corrected raw image data; performing white balance processing on the corrected raw image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain raw image data after white balance processing; Defective pixel correction, linear conversion and fusion processing are performed on first decomposed image data and second decomposed image data obtained by decomposing the original image data after white balance processing to obtain a target imaging image.

2. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The step of analyzing the focus parameters of the multispectral sensor on the target object to obtain the target focus parameters includes: Calculating a first phase difference between a first initial image and a second initial image of the target object captured by the multispectral sensor at each position based on the pixel brightness curve; calculating a second phase difference between a first phase map generated by the first initial image and a phase map generated by the second initial image; Focus parameter analysis of the multispectral sensor is performed based on the first phase difference and the second phase difference to obtain target focus parameters.

3. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The performing imaging impact analysis of the ambient light intensity on the multispectral sensor to obtain imaging impact data, and matching target exposure parameters based on the imaging impact data, includes: Perform feature integration on the historical ambient light impact imaging record set to obtain target feature data; Analyze the correlation between different ambient light intensities and different imaging parameters based on the historical ambient light impact imaging record set; Performing node adjustments on the initial imaging impact analysis model based on the target feature data and the degree of correlation to obtain a target imaging impact analysis model, and performing imaging impact analysis of ambient light intensity on the multispectral sensor based on the target imaging impact analysis model to obtain imaging impact data; Exposure parameters are matched based on imaging impact data and mapping relationships to obtain target exposure parameters.

4. The imaging quality optimization method of a multispectral sensor according to claim 3, characterized in that: The step of adjusting nodes of the initial imaging impact analysis model based on the target feature data and the degree of association to obtain the target imaging impact analysis model includes: Determine data labels based on the degree of association and the weight constraint of target feature data, and evaluate network nodes using an evaluation function based on the data labels to obtain network node evaluation coefficients; The initial imaging impact analysis model is adjusted for nodes based on the network node evaluation coefficients to obtain a target imaging impact analysis model.

5. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The constructing of the non-uniformity correction matrix and performing correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameter and the target exposure parameter based on the non-uniformity correction matrix to obtain the corrected original image data include: Determine the reference area in the calibration multispectral image based on the multi-channel response curve, and calculate the average pixel data of each channel in the reference area; A non-uniformity correction matrix is ​​constructed based on the pixel data of each channel in the calibrated multispectral image and the average pixel data of each channel in the reference area; Performing non-uniformity correction processing on the original image data based on the non-uniformity correction matrix to obtain the original image data after the non-uniformity correction processing; Based on adaptive instance normalization and spline fitting, spectral correction processing is performed on the original image data after non-uniform correction processing to obtain the corrected original image data.

6. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The white balance processing is performed on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis to obtain the white balanced original image data, including: Perform cluster analysis on the spectrum of each scene light source to obtain the corresponding light source category, and perform model training based on the light source category and the pixel received light intensity of each scene light source to obtain a light source prediction model; Determining light source prediction information and corresponding confidence levels using real-time scene spectrum data based on the light source prediction model; determining an initial color temperature corresponding to the light source prediction information based on a tristimulus value curve, calculating a target color temperature based on the initial color temperature using a confidence level corresponding to the light source prediction information, and determining a target gain for each color channel based on the target color temperature; The pixel values ​​of each color channel in the corrected original image data are transformed based on the target gain of each color channel to obtain the original image data after white balance processing.

7. The imaging quality optimization method of a multispectral sensor according to claim 1, characterized in that: The step of performing bad pixel correction, linear conversion, and fusion processing on first decomposed image data and second decomposed image data obtained by decomposing the original image data after white balance processing to obtain a target imaging image includes: Determine the analysis area of ​​the point to be detected in the historical output image based on the neighborhood window, perform vertical difference analysis based on the analysis area of ​​the point to be detected and combined with adjacent pixel values ​​to obtain a vertical difference analysis result; Constructing a bad pixel judgment table based on the vertical difference analysis result in combination with a threshold judgment method, and analyzing corresponding bad pixel pixels in the first decomposed image data and the second decomposed image data based on the bad pixel judgment table; Determining corrected pixel values ​​corresponding to bad pixel pixels in the first decomposed image data and the second decomposed image data based on color component analysis and distance analysis, and performing bad pixel correction on the first decomposed image data and the second decomposed image data based on the corrected pixel values ​​to obtain first decomposed image data and second decomposed image data after bad pixel correction; Performing channel correction on the first decomposed image data after bad pixel correction to obtain first decomposed image data after channel correction, and performing linear transformation on the second decomposed image data after bad pixel correction to obtain second decomposed image data after linear transformation; The first decomposed image data after channel correction and the second decomposed image data after linear transformation are fused to obtain a target imaging image.

8. An imaging quality optimization device for a multispectral sensor, characterized in that: The device comprises: Focus parameter analysis module: used to analyze the focus parameters of the multispectral sensor on the target object and obtain the target focus parameters; Exposure parameter analysis module: used to analyze the imaging impact of ambient light intensity on the multispectral sensor, obtain imaging impact data, and match target exposure parameters based on the imaging impact data; An image correction module is configured to construct a non-uniformity correction matrix, and to perform correction processing on the original image data of the target object collected by the multispectral sensor according to the target focus parameters and the target exposure parameters based on the non-uniformity correction matrix to obtain the corrected original image data; White balance processing module: used to perform white balance processing on the corrected original image data based on the target gain of each color channel obtained by the scene light source confidence analysis, to obtain the original image data after white balance processing; Fusion optimization module: used to perform bad pixel correction, linear conversion and fusion processing on the first decomposed image data and the second decomposed image data obtained by decomposing the original image data after white balance processing to obtain the target imaging image.

9. An electronic device comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the imaging quality optimization method of the multispectral sensor according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the imaging quality optimization method for a multispectral sensor according to any one of claims 1 to 7.

Images