Method and device for determining number of channels of Gaussian color filter type multispectral imaging system, electronic equipment and five-channel multispectral imaging system

Abstract

The invention relates to the technical field of multispectral imaging, and provides a channel number determination method and device of a Gaussian color filter type multispectral imaging system, electronic equipment and a five-channel multispectral imaging system. According to the technology, the problem of certainty of the optimal channel number is solved, and the five-channel multispectral imaging system which is small in channel number and good in spectrum reconstruction and color reproduction performance is provided. The method comprises the following steps of: firstly, calculating a reflectivity vector according to a selected spectrum transmittance vector quantum set, and scoring to obtain an optimal color filter of each channel number in a channel number set; and then, scoring the optimal color filters of different channel numbers to obtain an optimal target channel number. The device comprises a channel number optimal color filter obtaining unit and an optimal channel number determining unit. An electronic device includes a processor, a memory, and a computer program executing a channel number determination method. The five-channel multispectral imaging system can be applied to the field of multispectral color imaging, and has the advantages of excellent performance, low cost, few channels and the like.

Description

Technical Field

[0001] The present application relates to the field of multispectral imaging image technology, and in particular to a method, device, electronic device, and five-channel multispectral imaging system for determining the number of channels of a Gaussian filter type multispectral imaging system. Background Art

[0002] Multispectral color imaging aims to obtain the spectral reflectance of objects within the imaging scene within the visible light range using a small number of spectral channels, ensuring high-fidelity color reproduction. Research has found that the transmittance curve of an optimal filter combination composed of broadband filters exhibits a Gaussian curve. In narrowband systems, in addition to employing relatively mature interference filters, the spectral channels of lightweight multispectral color cameras are often implemented using components such as pre-filters (LCTFs) and on-chip process-compatible MSFAs. Their spectral transmittance can also be modeled using a Gaussian curve. In addition to the number of spectral channels limiting spectral reconstruction and color reproduction performance, the number of spectral channels also significantly impacts overall system performance, including system complexity, data load, temporal resolution, and spatial resolution. Therefore, whether designing a broadband or narrowband system, the number of spectral channels is a primary consideration in multispectral color camera design. However, the number of spectral channels employed in multispectral color camera construction varies widely. Burns and Cosentino used interference filter prefilters with 7 and 12 channels; Hardeberg, Wang, and LA et al. constructed multispectral color imaging systems using LCTFs, with the number of spectral channels ranging from 4 to 17, 20, and 5. With the development of on-chip multispectral image sensor technology, multispectral image acquisition is now achieved simultaneously through an array of "color filter mosaic basic units" containing different spectral channels, arranged spatially periodically within the MSFA. Common spectral channel numbers include 3, 4, 6, 9, and 16. In recent years, despite advances in ultra-flat image sensors and subwavelength optical device technology, spectral reconstruction accuracy and color fidelity in the visible light range remain key factors hindering their practical application. Determining the optimal number of spectral channels is also an unavoidable key issue. Summary of the Invention

[0003] The present invention provides a method, device, electronic device, and five-channel multispectral imaging system for determining the number of channels in a Gaussian color filter multispectral imaging system. These methods address the unique determination of the optimal number of channels. The resulting five-channel multispectral imaging system offers the advantages of a small number of channels, excellent spectral reconstruction, and good color reproduction performance. The technical solution is as follows: In a first aspect, an embodiment of the present invention provides a method for determining the number of channels of a Gaussian filter-based multispectral imaging system, comprising two steps: The first step is to obtain the channel number set Number of channels The optimal color filter for the Gaussian color filter type multispectral imaging system is: Obtain the spectral transmittance vectors corresponding to each Gaussian filter, and select a target number M of spectral transmittance vector subsets from each of the spectral transmittance vectors, each spectral transmittance vector subset includes spectral transmittance vectors, each subset of spectral transmittance vectors corresponds to Each initial color filter is composed of a Gaussian filter; Based on the target number of The spectral transmittance vector subset and the multispectral camera response function are used to calculate the multispectral camera response vector of each imaging target under each initial color filter; Calculating a reconstructed reflectivity vector of each imaging target under each initial color filter based on a multispectral camera response vector of each imaging target under each initial color filter; Based on the measured reflectivity vector corresponding to each of the imaging targets, calculating the error value corresponding to each of the reconstructed reflectivity vectors and each of the measured reflectivity vectors under each preset error parameter; Based on the error values, each of the initial color filters is scored, and the initial color filter with the highest score is used as the channel number Optimal color filter for According to the above method, traverse , get the corresponding common The number of channels is The optimal color filter for Gaussian color filter type multispectral imaging system; The second step is based on the Different channel numbers, the channel numbers are The error value corresponding to the optimal color filter of each channel is used to score the optimal color filter corresponding to each channel, and the channel number of the color filter with a higher score and the smallest channel number is used as the target channel number of the system, specifically: Based on the The number of channels is sorting the color filters according to the error values ​​corresponding to the optimal color filters; Scoring the optimal color filter of each channel of the color filter based on the type of the error value, the color filter sorting and the scoring rule; Obtaining a sum of scores of the color filter based on scores of each error type of the color filter; Based on the score sum, retaining a subsequence of the color filter sorting sequence with a higher score sum and a smaller number of channels to obtain a retained color filter set; Based on the color difference tolerance in the error and the interception filter set, the channel number of the interception filter that is not greater than the color difference tolerance and has the smallest corresponding channel number is selected as the optimal channel number.

[0004] In a second aspect, an embodiment of the present application provides a device for determining the number of channels of a multispectral imaging system, the device comprising: The first optimal color filter acquisition unit with different channel numbers acquires a channel number set Number of channels The optimal color filter of the Gaussian color filter type multispectral imaging system includes five submodules: A first acquisition submodule is configured to obtain spectral transmittance vectors corresponding to respective Gaussian color filters, and select a target number of spectral transmittance vector subsets from each of the spectral transmittance vectors, each spectral transmittance vector subset including spectral transmittance vectors of no less than channels, and the Gaussian color filters corresponding to each spectral transmittance vector subset constitute each initial color filter; The second calculation submodule is used to calculate the target number based on the target number. Spectral transmittance vector subset and multispectral camera response function, calculate the multispectral camera response vector of each imaging target under each initial filter; A third calculation submodule is configured to calculate a reconstructed reflectivity vector of each imaging target under each initial color filter based on the multispectral camera response vector of each imaging target under each initial color filter; a fourth calculation submodule, configured to calculate, based on the measured reflectivity vector corresponding to each of the imaging targets, an error value corresponding to each of the reconstructed reflectivity vectors and each of the measured reflectivity vectors under each of the preset error parameters; The fifth calculation submodule is configured to score each of the initial color filters based on the error values, and use the initial color filter with the highest score as the filter with the highest number of channels. Optimal color filter for The second optimal channel number determination unit is based on the Different channel numbers, the channel numbers are The error value corresponding to the optimal color filter of each channel is used to score the optimal color filter corresponding to each channel, and the channel number of the color filter with a higher score and the smallest channel number is used as the target channel number of the system, which includes five sub-modules: The first calculation submodule is used to calculate the The number of channels is sorting the color filters according to the error values ​​corresponding to the optimal color filters; a second calculation submodule, configured to score the optimal color filter of each channel of the color filter based on the type of the error value, the color filter sorting and the scoring rule; A third calculation submodule for obtaining a total score of the color filter based on the score of each error type of the color filter; a fourth calculation submodule, based on the score sum, retaining a subsequence of the color filter sorting sequence with a higher score sum and a smaller number of channels to obtain a retained color filter set; The fifth calculation submodule selects, based on the color difference tolerance in the error and the interception filter set, the one with the smallest number of channels and no greater than the color difference tolerance in the interception filter as the optimal number of channels.

[0005] In a third aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps of the above-mentioned first aspect.

[0006] In a fourth aspect, an embodiment of the present application provides a five-channel Gaussian filter multispectral imaging system, comprising: The spectral transmittance of each channel of the color filter can be expressed by Gaussian function Description, where: is the center peak wavelength, is the bandwidth of the spectral channel (i.e., peak half-maximum width), The first The wavelength coordinates of the sampling points; the five channels correspond to and They are (410, 40), (460, 80), (540, 60), (620, 80) and (685, 60), and the units are all nanometers (nm). The control error is no more than 2.5 nanometers; The imaging system light source adopts the spectral power distribution of the CIE D65 standard light source; the image sensor adopts, but is not limited to, a silicon-based monochrome area array image sensor; The reflectivity reconstruction algorithm is: ,in is the column vector of the multispectral image sensor response, , is the principal component column vector of the spectral reflectance vectors of 1269 Munsell color charts, is the column vector of light source spectral distribution, is the spectral sensitivity vector of the detector, Represents a diagonal matrix with the corresponding vector elements as diagonal elements, with the superscript Represents the transpose of a matrix; The color filter is a matrix composed of the transmittance column vectors of the five-channel color filters.

[0007] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least: The present application obtains a large number of Gaussian filters by changing the central wavelength and spectral bandwidth in the transmittance curve of the Gaussian filter. The method of obtaining the optimal number of channels by scoring the error value of the reconstructed spectrum in the optimal filters with different channel numbers solves the problem of uniqueness in determining the optimal number of channels of the Gaussian filter type multispectral imaging system, and improves the high performance of the multispectral imaging system in many aspects such as spectral reconstruction and color reproduction; the five-channel multispectral imaging system of the embodiment of the present application has the advantages of few channels, low cost and good performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0009] Figure 1 A schematic diagram of a workflow for determining the number of channels for a Gaussian color filter multispectral imaging system provided in this embodiment; Figure 2 A schematic diagram of multispectral imaging parameters provided in this embodiment; Figure 3 A schematic diagram showing the comparison of the integral of the optimal color filter and the color filter number obtained by adding the scores of the first 90 color filters sorted according to the error and taking the highest integral for the 29 channels from 3 to 31 provided in this example; Figure 4 The DE2000 color difference and maximum color difference corresponding to the system reflectance reconstruction performance under the optimal color filter corresponding to the number of channels provided in this embodiment; Figure 5 Schematic diagram of the integral obtained by sorting and assigning scores based on 8 error indicators within the range of 29 optimal color filters with a channel number of 3-31 provided in this embodiment; Figure 6 The Gaussian color filter type 9-channel optimal color filter provided in this embodiment; Figure 7 The Gaussian color filter type 5-channel optimal color filter provided in this embodiment; Figure 8 A schematic structural diagram of a Gaussian color filter type 5-channel multispectral imaging system provided in this embodiment; FIG9 (1) is a comparative schematic diagram of a 5-channel Gaussian color filter under the D65 light source and the A light source provided in this embodiment; Figure 9 (2) is a schematic diagram comparing the reflectance reconstruction performance of a 5-channel Gaussian filter under D65 light source and A light source; Figure 10 A schematic diagram of the structure of a device for determining the number of channels of a Gaussian color filter multispectral imaging system provided in an embodiment of the present application; Figure 11 A schematic diagram of the structure of an electronic device for determining the number of channels of a Gaussian color filter multispectral imaging system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] To make the purpose, features, and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0011] Throughout this application, it should be understood that the terms "first," "second," and so on are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Throughout this application, it should be noted that, unless expressly specified or limited otherwise, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or device. A person of ordinary skill in the art will understand the specific meanings of these terms in this application. Furthermore, throughout this application, unless otherwise specified, "plurality" refers to two or more. "And / or" describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exists simultaneously, or B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0012] The embodiments of the present application provide a method, device, electronic device, and five-channel multispectral imaging system for determining the number of channels of a Gaussian filter-type multispectral imaging system, which solve the problem of unique determination of the optimal number of channels. The five-channel multispectral imaging system proposed thereby has the advantages of a small number of channels, good spectral reconstruction, and color reproduction performance.

[0013] like Figure 1 FIG. 1 is a schematic diagram of a workflow for determining the number of channels of a Gaussian color filter multispectral imaging system provided in this embodiment. Figure 1 The method described in this embodiment may include the following two major steps:

[0014] S1, given the target channel number range, the filter unit spectral transmittance vector set and other imaging parameters, obtain the optimal filter corresponding to each channel number within the target channel number range, that is, obtain the channel number set Number of channels The optimal color filter for the Gaussian filter-type multispectral imaging system is determined by the following steps:

[0015] S101, obtaining the spectral transmittance vectors corresponding to each Gaussian filter, and selecting a target number of spectral transmittance vector subsets from each of the spectral transmittance vectors, each spectral transmittance vector subset includes spectral transmittance vectors, each subset of spectral transmittance vectors corresponds to Each initial color filter is composed of a Gaussian filter.

[0016] Among them, the Gaussian filter refers to a spectral spectrometer whose transmittance curve can be described by a Gaussian function. The form of the Gaussian function is Description, where: λ0 is the central peak wavelength, W is the bandwidth of the spectral channel (i.e., peak half-height width), λ i is the wavelength coordinate of the i-th sampling point of the spectral transmittance curve. The purpose of taking the value of 0.367 in the denominator of the formula exponent is to numerically satisfy the requirement that the wavelength dimension spacing of the left and right half-peak heights of the Gaussian curve is the spectral bandwidth W. In this embodiment, λ0 is taken at intervals of 5 nm from 400 to 700 nm, i.e., the number of central wavelength values ​​is 61, and the bandwidth W is taken at intervals of 5 nm from 10 to 80 nm, i.e., the number of spectral bandwidths is 16. Thus, a transmittance vector set consisting of 915 Gaussian vectors is available for selection, which can be seen in [1]. Figure 2 (2), Figure 2 (2) is a schematic diagram of some corresponding curves among the 915 Gaussian spectral transmittance vectors; Furthermore, the physical form of the Gaussian color filter described in the embodiment of the present application includes but is not limited to discrete components or on-chip integrated components, and its manufacturing process includes but is not limited to precipitation coating, ion beam etching, and hot pressing molding.

[0017] Furthermore, a target number of spectral transmittance vector subsets are selected from each spectral transmittance vector, that is, a target number of spectral transmittance vector subsets are selected from each spectral transmittance vector, and the Gaussian filter corresponding to each spectral transmittance vector subset is a combination of multiple Gaussian filters, and each combination can form an initial filter, and each initial filter includes the same number of Gaussian filters as the number of channels, and the number of channels refers to the number of channels of multi-spectral imaging. In the embodiment of the present application, the target number can be set to 915, and the specific selection method can be: traverse the first channel in the preset channels, and select the first spectral transmittance vector from the 915 spectral transmittance vectors as the preferred spectral transmittance vector for the first channel. The preset channels can include the first channel, the second channel... and the first channel. aisle, is a natural number greater than 3, and each channel includes 915 spectral transmittance vectors; traverse the second channel, based on the first spectral transmittance vector, select the second spectral transmittance vector that meets the "maximum irrelevant" condition from the 915 spectral transmittance vectors except the first spectral transmittance vector as the preferred spectral transmittance vector of the second channel; traverse the next channel in the above manner: when traversing to the first spectral transmittance vector, Channel, based on the first spectral transmittance vector to the -1 spectral transmittance vector, select the column vector matrix from the 915 spectral transmittance vectors except the first spectral transmittance vector to the The spectral transmittance vector that satisfies the "maximum linear independence" of the matrix other than the -1 spectral transmittance vector is taken as the first The preferred spectral transmittance vector of the channel. According to the above selection method, for each first spectral transmittance vector, the corresponding second spectral transmittance vector... and the corresponding Nth spectral transmittance vector can be selected, and the first spectral transmittance vector, the second spectral transmittance vector... and the Nth spectral transmittance vector can form a spectral transmittance vector subset. Since there are 915 choices for the first spectral transmittance vector, 915 different spectral transmittance vector subsets can be obtained, that is, 915 initial color filters are optimized and selected. It should be noted that when the preset channel includes For example, there are 6 channels, such as 1 channel, 2 channels, 3 channels, 4 channels, 5 channels and 6 channels. The first channel described above can be any one of the above 6 channels, not limited to 1 channel. It should be noted that the "maximum linear independence" means the minimum condition number, which is explained in detail in relevant literature and will not be repeated here.

[0018] Since the spectral channels of a multispectral imaging system should be relatively small integer values, the channel number variable in this embodiment takes values ​​within the largest possible dynamic range of 3-31, that is, the channel number set The elements of are 29 integer values ​​in the range of 3 to 31. Thus, it can be obtained that the preferred spectral transmittance vectors corresponding to the 29 channels with a channel number of 3 to 31 each constitute the target number of spectral transmittance vector subsets.

[0019] S102: Calculate a multispectral camera response vector for each imaging target under each initial color filter based on the spectral transmittance vector subset of the number of targets and the multispectral camera response function.

[0020] Specifically, in the embodiment of the present application, the response function of the multispectral camera, i.e., the multispectral imaging system, refers to the formula: C = R in diag(L)diag(S)T+N, where R in is the measured reflectivity vector corresponding to each imaging target, diag(*) represents the diagonal matrix composed of the corresponding vector elements, L is the column vector of the light source spectral distribution, S is the spectral sensitivity vector of the image detector, N is the Gaussian additive noise, and C is the response vector of the multispectral camera that stores the imaging response values ​​of each channel in columns. Substituting the above parameters into the formula corresponding to the multispectral camera response function, the multispectral camera corresponding vector of each imaging target under each initial color filter can be obtained. For the parameters in the formula, as for R in The embodiment of the present application uses a sample set of 1269 spectral reflectance vectors corresponding to the 1269 colors of the Munsell color chart, which can be obtained by Figure 2 (1); as for L, the spectral distribution of the CIE D65 light source is used, which can be seen as follows Figure 2 (3); as for S, the spectral response vector of Basler302fCCD is used, which can be seen in Figure 2 (4) is shown in the schematic diagram; as for N, 10 noises with signal-to-noise ratios of 200dB, 50dB, 47dB, 43dB, 40dB, 37dB, 33dB, 30dB, 27dB, and 23dB are used.

[0021] S103 , calculating a reconstructed reflectivity vector of each imaging target under each initial color filter based on the multispectral camera response vector of each imaging target under each initial color filter.

[0022] Specifically, in the embodiment of the present application, the reconstructed reflectivity of each imaging target under each initial color filter is calculated using the following formula: ,in , is the transmittance matrix of the ideal color filter, at which point the system noise is eliminated. Can be ignored, is a matrix composed of the principal component column vectors of the spectral reflectance of each imaging target. The principal component vectors can be calculated by the SVD function of Matlab using 1269 spectral reflectance samples. is the multispectral camera response vector of each imaging target under each initial color filter calculated in step S102. It is worth noting that when calculating By using the candidate initial color filters in the target number of spectral transmittance vector subsets obtained in S101 , the reconstructed reflectances of the corresponding 1269 spectral reflectance samples can be calculated in this step.

[0023] S104 , calculating, based on the measured reflectivity vector corresponding to each of the imaging targets, an error value corresponding to each of the reconstructed reflectivity vectors and each of the measured reflectivity vectors under each of the preset error parameters.

[0024] Specifically, in the embodiments of the present application, the preset error parameters may include three evaluation indicators: peak signal-to-noise ratio (PSNR), CIE2000 color difference formula (CIEDE2000 / DE2000), and mean square error (MSE). The calculation methods for the above error parameters are well known to those skilled in the art and will not be elaborated here. It is understood that for each initial color filter, 1269 reconstructed reflectance vectors of the Munsell color chart can be obtained. Therefore, the error values ​​between 1269 pairs of measured reflectance vectors and reconstructed reflectance under each preset error parameter can be calculated. These error values ​​may include the mean, standard deviation, and maximum value of these errors, all of which can be used as evaluation indicators for system performance and are directly meaningful for describing system performance or error control.

[0025] S105 , scoring each of the initial color filters based on the error values, and taking the initial color filter with the highest score as the optimal color filter with the number of channels.

[0026] Specifically, in an embodiment of the present application, the scoring method may be as follows: First, the mean error values ​​of all imaging targets in each initial color filter are calculated under each preset error parameter. That is, for each initial color filter, 1269 error values ​​of the Munsell color chart are obtained under each preset parameter, and therefore the mean error values ​​of these 1269 error values ​​under each preset parameter can be calculated. Second, based on the order of the preset error parameters and the mean error values, a score is calculated for each initial color filter under each preset error parameter. Because the definitions of each preset error parameter are inconsistent, a higher PSNR value indicates a higher score, while a lower CIEDE2000 and MSE value indicates a higher score. Therefore, each initial color filter will have three mean error values ​​corresponding to each of the three preset error parameters. These three mean error values ​​are scored according to the scoring strategies corresponding to the preset error parameters, resulting in a score for each initial color filter for each of the three preset error parameters. Finally, the sum of the scores for each initial color filter under each preset error parameter is calculated, and the initial color filter with the maximum sum of the scores is selected as the target color filter. That is to say, the score values ​​of the three preset error parameters obtained for each initial color filter are summed up, and the initial color filter with the largest sum is found, then this initial color filter is the optimal color filter.

[0027] Furthermore, for MSE and CIEDE2000, the mean, standard deviation, and upper error limit (maximum error) are considered, with smaller values ​​assigned higher scores. For PSNR, both the mean and standard deviation are considered, with larger means assigned higher scores and smaller standard deviations assigned higher scores. Thus, for a color filter, the score is ranked based on a total of eight indicators. Figure 3 A schematic diagram comparing the score and filter number of the optimal filter obtained by adding the scores of the first 90 filters sorted by the error and taking the highest score for each of the 29 channels (channels 3-31) provided in this example. Specifically, the scoring rule for each of the 90 filters in the 29 channels is to assign scores ranging from 90 to 1, based on the evaluation criteria from best to worst.

[0028] So far, the embodiment of the present application has completed the selection of the optimal color filter for the same channel with a channel number of 3-31. Figure 4 ( Figure 4 The optimal color filter corresponding to the number of channels provided in this embodiment shows the corresponding DE2000 color difference and maximum color difference of the system reflectance reconstruction performance. It can be seen that as the number of channels increases, the performance indicators of the system generally improve. However, on the one hand, a large number of channels is not the purpose of the present invention; on the other hand, Figure 3 The comparison between the comprehensive performance indicators of the best color filters with different channel numbers shown cannot be Figure 3The integrals shown in the figure are not logically comparable, as these integrals are derived from the error indices of the 915 color filters within their own channels. Therefore, the determination of the optimal number of channels awaits completion of the second major step of this embodiment. Specifically,

[0029] S2, based on the color filters and corresponding error values ​​of each channel within the target channel number range, the optimal channel number is obtained, that is, according to the Different channel numbers, the channel numbers are The error value corresponding to the optimal color filter of each channel is used to score the optimal color filter corresponding to each channel, and the channel number of the color filter with the highest score and the smallest channel number is used as the target channel number of the system. The specific steps include the following:

[0030] S201, sorting the color filters based on the error values ​​corresponding to the optimal color filters within the target channel number range, i.e., sorting the color filters based on the error values ​​corresponding to the optimal color filters within the target channel number range. The number of channels is The color filters are sorted according to the error values ​​corresponding to the optimal color filters.

[0031] Specifically, the 29 color filters are sorted based on the error values ​​of the optimal color filters with 29 channels and 3-31 channels as described in S104 and S05. It can be understood that since there are 8 indicators involved in the evaluation, there are 8 corresponding color filter sequences.

[0032] S202 : Score the optimal color filter of each channel of the color filter based on the type of the error value, the color filter order and the scoring rule.

[0033] Specifically, based on the 8 color filter sequences described in S201, the scoring rule for the optimal color filters corresponding to 29 channels with a number of channels of 3-31 is to assign scores ranging from 29 to 1 according to the 8 evaluation indicators from good to bad.

[0034] S203 : Calculate a total score of the color filter based on the scores of each error type of the color filter.

[0035] Specifically, based on the 8 scoring sequences of the 29 color filters described in S202, the 8 scores corresponding to each of the 29 color filters are summed to obtain the following: Figure 5 The integrals shown are obtained by sorting and scoring the 8 error indicators within the range of 29 optimal color filters with a channel number of 3-31.

[0036] S204 , based on the score sum in S203 , retain a subsequence of the color filter sorting sequence with a higher score sum and a smaller number of channels to obtain a retained color filter set.

[0037] Specifically, based on Figure 5 The corresponding integrals of the 29 optimal color filters are shown. It can be seen that when the number of filter channels is no greater than 16, the 5-, 9-, and 12-channel filters perform well, with the 9-channel filter being the best. When the number of filter channels is no greater than 8, the 4-, 5-, and 7-channel filters perform well, with the 5-channel filter being the best. Therefore, the 5-, 9-, and 12-channel filters or the 4-, 5-, and 7-channel filters can be used as the interception filter set. Figure 6 and Figure 7 They are schematic diagrams of the Gaussian color filtering type 9-channel optimal color filter and the 5-channel optimal color filter respectively.

[0038] S205 , based on the color difference tolerance of the error and the interception filter set, selecting the interception filter with the smallest channel number and no greater than the color difference tolerance as the optimal channel number.

[0039] Specifically, according to the interception filter set described in S204, Figure 4 It can be seen that the maximum DE2000 color difference of the optimal color filter with 5 channels is no more than 3, while that of the optimal color filter with 9 channels is less than 0.5. Both are at the best level of performance indicators among the optimal color filters with smaller adjacent channel numbers. Although the 4-channel rate color filter has the smallest number of channels, its maximum color difference is much larger than that of other color filters. It can be understood that the maximum color difference tolerance of DE2000 color difference no more than 3 can meet the technical requirements of many practical applications. Therefore, the number of channels corresponding to the 5-channel color filter can be selected as the optimal channel number, and thus the optimal channel number determined in this embodiment is 5.

[0040] When the solution of the embodiment of the present application is executed, first, by giving a target channel number range, a filter unit spectral transmittance vector set, and other imaging parameters, the optimal color filter corresponding to each channel number within the target channel number range is obtained. That is, by obtaining the spectral transmittance vector corresponding to each Gaussian filter, a target number of spectral transmittance vector subsets are selected from each spectral transmittance vector. Based on the target number of spectral transmittance vector subsets and the multispectral camera response function, the multispectral camera response vector of each imaging target under each initial color filter is calculated. Based on the multispectral camera response vector of each imaging target under each initial color filter, the reconstructed reflectivity vector of each imaging target under each initial color filter is calculated. The measured reflectivity vector corresponding to each imaging target is obtained, and the error value corresponding to each reconstructed reflectivity vector and each measured reflectivity vector under each preset error parameter is calculated. Based on each error value, each initial color filter is scored, and the initial color filter with the highest score is selected as the optimal color filter for the corresponding number of channels. The invention relates to a method for obtaining a multispectral imaging system comprising: firstly, a color filter and a filter group, and secondly, an optimal number of channels based on the color filters corresponding to each channel within the target channel number range and the corresponding error values, that is, sorting the color filters based on the error values ​​corresponding to the optimal color filters within the target channel number range, scoring the optimal color filter of each channel of the color filters based on the type of the error value, the color filter sorting and the scoring rule, obtaining a total score of the color filters based on the scores of each error type of the color filters, and intercepting a subsequence of the color filter sorting sequence with a higher total score and a smaller number of channels based on the total score to obtain a retained color filter set, and based on the color difference tolerance in the error and the retained color filter set, selecting the channel number of the retained color filter that is not greater than the color difference tolerance and has the smallest corresponding channel number as the optimal channel number, thereby solving the problem of unique determination of the optimal channel number, taking into account performance evaluation while adopting a smaller number of channels. Therefore, the multispectral imaging system with a channel number determined by the present scheme has the advantages of good spectral reconstruction and color reproduction performance, simple system structure and low cost.

[0041] Please see Figure 8 Based on the optimal channel number determination method provided in the embodiment of the present application, the following can be obtained: Figure 8 The Gaussian filter type 5-channel multispectral imaging system shown is a structural diagram, including: an imaging unit U801, an algorithm unit U802, a storage unit U803, a control unit U804 and a communication bus unit U805.

[0042] The imaging unit U801 is used to acquire multispectral images and includes a five-channel color filter module M8011, an image sensor module M8012, a light source module M8013, and an imaging target module M8013. The five-channel color filter module M8011 constitutes five spectral channels for imaging, and the spectral transmittance of each channel can be expressed using a Gaussian function. Description, where: is the center peak wavelength, is the bandwidth of the spectral channel (i.e., peak half-maximum width), The first The wavelength coordinates of the sampling points; the five channels correspond to and They are (410, 40), (460, 80), (540, 60), (620, 80) and (685, 60), and the units are all nanometers (nm). The control error is no more than 2.5 nanometers; the spectral response curve of the image sensor module M8012 adopts the spectral response curve of the Basler 302f CCD image sensor, and the light source module M8013 adopts the spectral power distribution of the CIE D65 standard light source. These imaging parameters are as follows: Figure 2 It is worth emphasizing that, in principle, the imaging target module M8013 can be any imaging target. It is understandable that the spectrum sampling interval of all data in the calculation process of this embodiment is 5nm. According to the sampling theorem, the center peak wavelength of the color filter is A control error of no more than 2.5 nanometers is reasonable. It is also understood that, because silicon-based monochrome area array image sensors have similar spectral response characteristics, the image sensor module M8012 is not limited to the Basler 302f CCD image sensor. Furthermore, the color filter module M8011, image sensor M8011, and light source module M1013 can be combined in a discrete, integrated, or integrated manner.

[0043] The algorithm unit U802 is used to perform reflectance reconstruction calculation on the multispectral image acquired by the imaging unit U801. The algorithm has been disclosed in detail in S103 and will not be repeated here.

[0044] The storage unit U803 is used to store the multispectral image acquired by the imaging unit U801, the calculation program of the algorithm unit U802, and the reflectivity reconstructed image after the algorithm unit U802 executes the reflectivity reconstruction algorithm.

[0045] Among them, the control unit U804 is used for system control, and the communication bus unit U805 is used to provide a transmission channel for data and instructions of each unit in the system.

[0046] When the scheme of the embodiment of the present application is executed, when the imaging target M8103 adopts the Munsell color card, the 5-channel multispectral imaging system is compared with the system composed of the 5-channel optimal color filter in the embodiment of the patent application published with application publication number CN114241068 A, and the result of comparative advantage is achieved. Because the difference in the imaging parameters of the two is only the use of different light sources, the former uses the CIE D65 light source, and the latter uses the CIE A light source. The processes of calculating multispectral imaging, reflectivity reconstruction, and the error evaluation index are exactly the same for both. No further details will be given here. Only the color filters and performance evaluation indicators of the two are compared, as follows: As shown in FIG9 (1), a comparison diagram of the 5-channel Gaussian color filter under the D65 light source and the A light source provided in this embodiment is shown. It can be seen from FIG9 (1) that, compared with the A light source, the bandwidth of the optimal color filter under the D65 light source in this embodiment is much larger, and the spectral bandwidth of two of the five spectral channels is 80nm. The concept that "broadband" color filters are usually not Gaussian filters has been updated; as shown in Figure 9 (2), a comparison diagram of the reflectance reconstruction performance of 5-channel Gaussian filters under D65 light source and A light source, it can be seen from Figure 9 (2) that under the D65 light source condition of this embodiment, the DE2000 color difference index is reduced from 3.85 to 1.16, the MSE mean square error index is reduced from 9.15e-4 to 3.46e-4, and the PSNR peak signal-to-noise ratio is increased from 39.82dB to 43.0dB, and the system performance has been greatly improved. Please refer to S102. Since the noise is fully considered in the imaging simulation calculation, and the universality of the natural color reflectance spatial distribution of the Munsell color card used in the human eye is not difficult to understand the practicality of the five-channel multispectral imaging system proposed in the application embodiment.

[0047] Please see Figure 10 , is a schematic diagram of the structure of a channel number determination device for a Gaussian color filter multispectral imaging system provided in an embodiment of the present application. The channel number determination device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. Figure 10 As shown, the channel number determination device of the Gaussian color filter multispectral imaging system of the embodiment of the present application includes: The first different channel number optimal color filter acquisition unit 101 is used to acquire a channel number set Number of channels The optimal color filter of the Gaussian color filter type multispectral imaging system includes five submodules: A first acquisition submodule 1011 is configured to obtain spectral transmittance vectors corresponding to respective Gaussian color filters, and select a target number of spectral transmittance vector subsets from each of the spectral transmittance vectors, each spectral transmittance vector subset including spectral transmittance vectors of no less than channels, and the Gaussian color filters corresponding to each spectral transmittance vector subset constitute each initial color filter; The second calculation submodule 1012 is used to calculate the target number based on the target number. The spectral transmittance vector subset and the multispectral camera response function are used to calculate the multispectral camera response vector of each imaging target under each initial color filter; A third calculation submodule 1013 is configured to calculate a reconstructed reflectivity vector of each imaging target under each initial color filter based on the multispectral camera response vector of each imaging target under each initial color filter; A fourth calculation submodule 1014 is configured to calculate, based on the measured reflectivity vector corresponding to each of the imaging targets, an error value corresponding to each of the reconstructed reflectivity vectors and each of the measured reflectivity vectors under each of the preset error parameters; The fifth calculation submodule 1015 is configured to score each of the initial color filters based on the error values, and use the initial color filter with the highest score as the filter with the highest number of channels. Optimal color filter for The second optimal channel number determination unit 102 is used to determine the optimal channel number according to the Different channel numbers, the channel numbers are The error value corresponding to the optimal color filter of each channel is used to score the optimal color filter corresponding to each channel, and the channel number of the color filter with a higher score and the smallest channel number is used as the target channel number of the system, which includes five sub-modules: The first calculation submodule 1021 is used to calculate the The number of channels is sorting the color filters according to the error values ​​corresponding to the optimal color filters; A second calculation submodule 1022 is configured to score the optimal color filter of each channel of the color filter based on the type of the error value, the color filter sorting and the scoring rule; The third calculation submodule 1023 is configured to obtain a total score of the color filter based on the score of each error type of the color filter; A fourth calculation submodule 1024 is configured to retain, based on the score sum, a subsequence of the color filter sorting sequence having a higher score sum and a smaller number of channels to obtain a retained color filter set; The fifth calculation submodule 1025 selects the channel number of the interception filter that is not greater than the color difference tolerance and has the smallest corresponding channel number as the optimal channel number based on the color difference tolerance in the error and the interception filter set.

[0048] When the solution of the embodiment of the present application is executed, first, by giving a target channel number range, a filter unit spectral transmittance vector set, and other imaging parameters, the optimal color filter corresponding to each channel number within the target channel number range is obtained. That is, by obtaining the spectral transmittance vector corresponding to each Gaussian filter, a target number of spectral transmittance vector subsets are selected from each spectral transmittance vector. Based on the target number of spectral transmittance vector subsets and the multispectral camera response function, the multispectral camera response vector of each imaging target under each initial color filter is calculated. Based on the multispectral camera response vector of each imaging target under each initial color filter, the reconstructed reflectivity vector of each imaging target under each initial color filter is calculated. The measured reflectivity vector corresponding to each imaging target is obtained, and the error value corresponding to each reconstructed reflectivity vector and each measured reflectivity vector under each preset error parameter is calculated. Based on each error value, each initial color filter is scored, and the initial color filter with the highest score is selected as the optimal color filter for the corresponding number of channels. The invention relates to a method for obtaining a multispectral imaging system comprising: firstly, a color filter and a filter group, and secondly, an optimal number of channels based on the color filters corresponding to each channel within the target channel number range and the corresponding error values, that is, sorting the color filters based on the error values ​​corresponding to the optimal color filters within the target channel number range, scoring the optimal color filter of each channel of the color filters based on the type of the error value, the color filter sorting and the scoring rule, obtaining a total score of the color filters based on the scores of each error type of the color filters, and intercepting a subsequence of the color filter sorting sequence with a higher total score and a smaller number of channels based on the total score to obtain a retained color filter set, and based on the color difference tolerance in the error and the retained color filter set, selecting the channel number of the retained color filter that is not greater than the color difference tolerance and has the smallest corresponding channel number as the optimal channel number, thereby solving the problem of unique determination of the optimal channel number. Since the performance evaluation is taken into account while a smaller number of channels is adopted, the multispectral imaging system with a channel number determined by the scheme has the advantages of good spectral reconstruction and color reproduction performance, simple system structure and low cost.

[0049] Please refer to Figure 12, which illustrates the structure of an electronic device for determining the number of channels in a Gaussian color filter multispectral imaging system according to an embodiment of the present application. As shown in Figure 12, the terminal device may include: at least one processor, at least one network interface, at least one input / output interface, at least one display unit, at least one memory, and at least one communication bus.

[0050] Among them, the communication bus is used to realize the connection and communication between these components.

[0051] The user interface may include a display screen and a camera. Optional user interfaces may also include a standard wired interface and a wireless interface.

[0052] The network interface may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0053] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the electronic device. It executes instructions, programs, code sets, or instruction sets stored in memory, and accesses data stored in memory to perform various functions of the electronic device and process data. Optionally, the processor may be implemented in hardware using at least one of the following: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs. It is understood that the modem may not be integrated into the processor and may be implemented as a separate chip.

[0054] The memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory may also optionally be at least one storage device located remotely from the aforementioned processor.

[0055] like Figure 11 As shown, the memory as a computer storage medium may include an operating system, a network communication module, a user interface module, and a program for determining the number of channels of a Gaussian color filter multispectral imaging system.

[0056] exist Figure 11 In the electronic device shown, the user interface is primarily used to provide an input interface for the user and obtain user input data; and the processor can be used to call a program for determining the number of channels of a Gaussian color filter multispectral imaging system stored in the memory and execute the operations of the method for determining the number of channels of the Gaussian color filter multispectral imaging system. The specific operational details are not repeated here.

[0057] Furthermore, those skilled in the art will appreciate that the structures of the electronic devices shown in the above figures do not limit the electronic devices. The electronic devices may include more or fewer components than shown, or may combine certain components or arrange the components differently. For example, the electronic devices may also include RF circuits, audio circuits, Wi-Fi components, power supplies, Bluetooth components, and other components, which will not be described in detail here.

[0058] An embodiment of the present application further provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method for determining the number of channels of a multispectral imaging system as described in the above embodiments.

[0059] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media includes any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. Method for determining the number of channels of Gaussian filter type multispectral imaging system, It is characterized in that The method comprises two steps: The first step is to obtain the channel number set Number of channels in The optimal color filter of the Gaussian color filter type multispectral imaging system is: Obtain the spectral transmittance vectors corresponding to each Gaussian filter, and select a target number M of spectral transmittance vector subsets from each of the spectral transmittance vectors, each spectral transmittance vector subset includes spectral transmittance vectors, each subset of spectral transmittance vectors corresponds to Each initial color filter is composed of a Gaussian filter; Based on the target quantity The spectral transmittance vector subset and the multispectral camera response function are used to calculate the multispectral camera response vector of each imaging target under each initial color filter; Calculating a reconstructed reflectivity vector of each imaging target under each initial color filter based on a multispectral camera response vector of each imaging target under each initial color filter; Obtaining a measured reflectivity vector corresponding to each of the imaging targets, and calculating an error value corresponding to each of the reconstructed reflectivity vectors and each of the measured reflectivity vectors under each preset error parameter; Based on the error values, each of the initial color filters is scored, and the initial color filter with the highest score is used as the channel number Optimal color filter for According to the above method, traverse , and obtain the corresponding common The number of channels is The optimal color filter for Gaussian color filter type multispectral imaging system; The second step is based on the The number of channels is different. The error value corresponding to the optimal color filter of each channel is used to score the optimal color filter corresponding to each channel, and the channel number of the color filter with a higher score and the smallest channel number is taken as the target channel number of the system.

2. The method according to claim 1, It is characterized in that The step of obtaining the spectral transmittance vectors corresponding to the Gaussian filters includes: Acquire each central wavelength within a preset spectral value range, and determine a first central wavelength from the central wavelengths; Acquire each spectral bandwidth within a preset spectral bandwidth value range, and determine a first spectral bandwidth from the spectral bandwidths; The first central wavelength is used as the central wavelength in the transmittance curve function of the Gaussian filter, and the first spectral bandwidth is used as the spectral bandwidth in the transmittance curve function of the Gaussian filter, to obtain the spectral transmittance vectors corresponding to each Gaussian filter.

3. The method according to claim 1, It is characterized in that The step of selecting a target number of spectral transmittance vector subsets from each of the spectral transmittance vectors comprises: Traversing the first channel in the preset channels, selecting a first spectral transmittance vector from the target number of spectral transmittance vectors as the preferred spectral transmittance vector corresponding to the first channel, the preset channels including channels, is not less than A natural number, each channel includes the target number of spectral transmittance vectors; Traversing a second channel in the preset channels, based on the first spectral transmittance vector, selecting a second spectral transmittance vector that satisfies a "maximum irrelevant" condition except the first spectral transmittance vector from the target number of spectral transmittance vectors as a preferred spectral transmittance vector corresponding to the second channel; According to the above method, traverse the next channel in the preset channels; When the first channel, based on the first spectral transmittance vector to the spectral transmittance vector, selecting from the target number of spectral transmittance vectors excluding the first spectral transmittance vector to the first spectral transmittance vector The spectral transmittance vector The spectral transmittance vector is used as the The preferred spectral transmittance vector corresponding to the channel; The target number of spectral transmittance vector subsets are composed of the preferred spectral transmittance vectors corresponding to each channel.

4. The method according to claim 1, It is characterized in that The step of scoring each of the initial color filters based on the error values ​​and taking the initial color filter with the highest score as the target color filter comprises: Calculating the mean and maximum error values ​​of all imaging targets in each initial color filter under each preset error parameter, wherein each preset error parameter includes at least one of a peak signal-to-noise ratio, a spectral fit index, a color difference formula, and a mean square error; Calculate the score of each initial color filter under each preset error parameter based on the ascending order of the mean value and the maximum value of each preset error parameter; The sum of the scores of each of the initial color filters at the preset error parameters is calculated, and the initial color filter corresponding to the maximum value of the sum of the scores is used as the target color filter.

5. The method according to claim 1, It is characterized in that According to the The number of channels is different. The error value corresponding to the optimal color filter of each channel is used to score the optimal color filter corresponding to each channel, and the number of channels of the color filter with a higher score and the smallest number of channels is used as the target number of channels of the system. The specific steps are: Based on the The number of channels is sorting the color filters according to the error values ​​corresponding to the optimal color filters; Scoring the best color filter for each channel of the color filter based on the type of the error value, the color filter ranking and the scoring rule; Obtaining a total score of the color filter based on the scores of each error type of the color filter; Based on the score sum, retaining a subsequence of the color filter sorting sequence with a higher score sum and a smaller number of channels to obtain a retained color filter set; Based on the color difference tolerance in the error and the interception filter set, the channel number of the interception filter that is not greater than the color difference tolerance and has the smallest corresponding channel number is selected as the optimal channel number.

6. The method according to claim 1, It is characterized in that The physical form of the Gaussian color filter includes but is not limited to discrete components or on-chip integrated components, and its manufacturing process includes but is not limited to precipitation coating, ion beam etching, and hot pressing molding.

7. A device for determining the number of channels of a Gaussian filter type multispectral imaging system, It is characterized in that The device comprises: The first optimal color filter acquisition unit with different channel numbers acquires a channel number set Number of channels in The optimal filter of the Gaussian filter type multispectral imaging system includes five sub-modules: The first acquisition submodule is used to obtain the spectral transmittance vectors corresponding to each Gaussian filter, and select a target number from each of the spectral transmittance vectors. spectral transmittance vector subsets, each spectral transmittance vector subset includes no less than The spectral transmittance vectors of the channels, wherein the Gaussian color filters corresponding to each subset of the spectral transmittance vectors constitute each initial color filter; A second calculation submodule is used to calculate the multispectral camera response vector of each imaging target under each initial color filter based on the spectral transmittance vector subset of the target number and the multispectral camera response function; A third calculation submodule is used to calculate the reflectivity vector of each imaging target under each initial color filter based on the multispectral camera response vector of each imaging target under each initial color filter; The fourth calculation submodule is used to obtain the measured reflectivity vector corresponding to each of the imaging targets, and calculate the error value corresponding to each of the reconstructed reflectivity vectors and each of the measured reflectivity vectors under each preset error parameter: A fifth calculation submodule is used to score each of the initial color filters based on the error values, and take the initial color filter with the highest score as the target number. Target Optimal Color Filter The second optimal channel number determination unit determines the optimal channel number according to the color filters of different channels obtained by the first unit, and includes five submodules: The first calculation submodule is used to calculate the The number of channels is sorting the color filters according to the error values ​​corresponding to the optimal color filters; A second calculation submodule, configured to score the optimal color filter of each channel of the color filter based on the type of the error value, the color filter sorting and the scoring rule; A third calculation submodule, for obtaining a total score of the color filter based on the score of each error type of the color filter; A fourth calculation submodule, configured to retain, based on the score sum, a subsequence of the color filter sorting sequence with a higher score sum and a smaller number of channels, to obtain a retained color filter set; The fifth calculation submodule is used to select, based on the color difference tolerance in the error and the interception filter set, the channel number of the interception filter that is not greater than the color difference tolerance and has the smallest corresponding channel number as the optimal channel number.

8. An electronic device, It is characterized in that include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 5.

9. Five-channel Gaussian filter type multispectral imaging system, It is characterized in that include: The spectral transmittance of each channel of the color filter can be expressed by Gaussian function Description, where: is the center peak wavelength, is the bandwidth of the spectral channel (i.e., the peak half-height width), The spectral transmittance curve is The wavelength coordinates of the sampling points; the corresponding wavelength coordinates of the five channels and They are (410, 40), (460, 80), (540, 60), (620, 80) and (685, 60) respectively, and the units are all nanometers (nm). The control error is no more than 2.5 nanometers; The imaging system light source adopts the spectral power distribution of CIE D65 standard light source; The image sensor includes but is not limited to a silicon-based monochrome area array image sensor.

10. The five-channel multispectral imaging system according to claim 9, Its characteristics are: The reflectivity reconstruction algorithm is: ,in is the column vector of the multispectral image sensor response, , is the principal component column vector of the spectral reflectance vectors of 1269 Munsell color charts, is the column vector of light source spectral distribution, is the spectral sensitivity vector of the detector, represents a diagonal matrix with the corresponding vector elements as diagonal elements, with the superscript Represents the transpose of a matrix; The color filter is a matrix composed of the transmittance column vectors of the five-channel color filters.

Images