Filter, camera module, camera apparatus, and electronic device

Abstract

The present application provides a filter, a camera module, a camera apparatus, and an electronic device. The camera apparatus comprises a first camera module, wherein the first camera module comprises a lens, a filter, and an image sensor which are sequentially arranged along an optical axis. The filter is used for allowing light within a target wavelength range to pass through to acquire, on the image sensor, spectral information of a target object; and the filter has a first transmittance curve acquired under a normal incidence condition and a second transmittance curve acquired under an oblique incidence condition. On the first transmittance curve, a wavelength corresponding to 50% transmittance comprises a first wavelength, and the first wavelength is within the range of 600 nm-660 nm; and on the second transmittance curve, a wavelength corresponding to 50% transmittance comprises a second wavelength, and the difference between the first wavelength and the second wavelength is within the range of 0-20 nm. The described technical solution can improve the color calibration accuracy of the camera module while ensuring the color reproduction capability of the camera module.

Description

Filters, camera modules, camera devices and electronic equipment

[0001] This application claims priority to Chinese Patent Application No. 202411203726.5, filed with the China National Intellectual Property Administration on August 29, 2024, entitled "Filter, Camera Module, Imaging Device and Electronic Device", the entire contents of which are incorporated herein by reference; and Chinese Patent Application No. 202511071731.X, filed with the China National Intellectual Property Administration on July 31, 2025, entitled "Filter, Camera Module, Imaging Device and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of camera technology, and more specifically, to a filter, camera module, camera device, and electronic device. Background Technology

[0003] Multispectral camera modules can obtain information on the radiation or reflection of multiple spectral bands of the target through spectral splitting technology, thereby obtaining the characteristic spectrum of the target. This characteristic spectrum can be used to restore the color of the target.

[0004] Compared to existing three-channel camera modules, multispectral camera modules have stronger spectral sensing capabilities, resulting in superior color reproduction and images that more closely resemble true colors. Therefore, to improve the quality of images output by a three-channel camera module, images acquired by a multispectral camera module can be used to perform color calibration on the images acquired by the three-channel camera module.

[0005] However, the color reproduction capability of a multispectral camera module is also affected by two key optical components: the optical lens and the filter. The optical response of the module needs to be designed by integrating the chip, optical lens, and filter together. A poor optical response design will result in poor accuracy for color calibration in the multispectral camera module. Even if the multispectral camera module can capture true colors, the three-channel camera module cannot accurately obtain color information from the multispectral camera module, leading to a significant deviation between the color representation of the three-channel camera module and the colors seen by the human eye, resulting in poor realism. Summary of the Invention

[0006] This application provides a filter, a camera module, a camera device, and an electronic device that can improve the accuracy of color calibration of the camera module while ensuring the color reproduction capability of the camera module, thereby improving the color authenticity and consistency of the image.

[0007] In a first aspect, a camera device is provided, including a first camera module. The first camera module includes an optical lens, a filter, and an image sensor arranged sequentially along an optical axis. The optical lens is used to receive light from a target object, the filter is used to allow light within a target wavelength range to pass through, and the image sensor is used to receive light within the target wavelength range to obtain spectral information of the target object. The image sensor has more than 3 channels. The filter has a first transmittance curve and a second transmittance curve, the first transmittance curve representing light normally incident on the filter. The first transmittance curve is a curve showing the relationship between wavelength and transmittance of light incident on the filter at a first angle greater than 0° and less than or equal to 35°. On the first transmittance curve, the wavelength corresponding to 50% transmittance includes the first wavelength, which is greater than or equal to 600nm and less than or equal to 660nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes the second wavelength, and the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20nm.

[0008] In this embodiment, the filter with the aforementioned parameters ensures a high degree of similarity between the spectral response curve of the first camera module and the spectral response curve of the human eye. Consequently, after the spectral response curve of the first camera module is mapped from a high dimension to a low dimension, it can approach or be equivalent to the spectral response curve of the human eye. Therefore, the images captured by the first camera module are closer to the colors seen by the human eye, thus improving the color reproduction capability of the first camera module. Furthermore, based on the first camera module, color correction can be performed on other camera modules, reducing or avoiding color cast problems that occur in images captured by other camera modules, improving the color fidelity of images from other camera modules, and thereby enhancing the user experience.

[0009] Specifically, under normal incidence, the first wavelength corresponding to a transmittance of 50% on the filter is in the range of 600nm-660nm. This limits the overall spectral response range, which helps to make the spectral response curve of the first camera module approximate the spectral response curve of the human eye after dimensionality reduction from a high-dimensional position. Furthermore, the difference between the first and second wavelengths is within the range of 0-20nm, ensuring that within the incident light range, the transmittance curve of the filter makes the spectral response curve of the first camera module highly similar to the spectral response curve of the human eye, thereby reducing or avoiding color cast issues and improving the color fidelity of the output image.

[0010] In addition, the difference between the first wavelength and the second wavelength is in the range of 0-20nm, which can limit the angular offset of the transmittance curve of the filter, improve the color reproduction consistency between the image center and the image edge, and reduce the color difference between the image center and the image edge.

[0011] In applications where the first camera module performs color correction on other camera modules, these other camera modules can be three-channel, four-channel, or other types of camera modules. Under normal incidence, the first wavelength range corresponding to 50% transmittance of the aforementioned filter is 600nm-660nm. This first wavelength range is close to the corresponding wavelength range of conventional filters in other camera modules, making the spectral response curve of the first camera module similar to that of other camera modules. This improves the realism of color correction performed by the first camera module on other camera modules.

[0012] It should be noted that the "difference between the first wavelength and the second wavelength" mentioned in the embodiments of this application refers to the numerical difference between the first wavelength and the second wavelength. This difference is a scalar and can be expressed as the absolute value of the value obtained by subtracting the second wavelength from the first wavelength, or as the absolute value of the value obtained by subtracting the first wavelength from the second wavelength.

[0013] In conjunction with the first aspect, in one possible implementation, on the first transmittance curve, the wavelength corresponding to 50% transmittance includes a third wavelength, which is greater than or equal to 400 nm and less than or equal to 440 nm; on the second transmittance curve, the wavelength corresponding to 50% transmittance includes a fourth wavelength, the difference between the third wavelength and the fourth wavelength being greater than or equal to 0 and less than or equal to 20 nm.

[0014] When incident at an oblique angle, the filter corresponds to a third wavelength range of 400nm-440nm at a transmittance of 50%. This limits the overall spectral response range, resulting in a high degree of similarity between the spectral response curve of the first camera module and the spectral response curve of the human eye. This improves the color reproduction capability of the first camera module and the accuracy of color correction for other camera modules, reducing or avoiding color cast issues and enhancing the color fidelity of images captured by other camera modules. The wavelength range corresponding to this filter is close to that of conventional filters in other camera modules, and the spectral response curve of the first camera module is similar to that of other camera modules, further improving the fidelity of color correction performed by the first camera module on other camera modules.

[0015] The difference between the third and fourth wavelengths is in the range of 0-20nm, which can limit the angular offset of the transmittance curve of the filter, improve the color reproduction consistency between the image center and the image edge, and reduce the color difference between the image center and the image edge.

[0016] It should be noted that the "difference between the third wavelength and the fourth wavelength" mentioned in the embodiments of this application refers to the numerical difference between the third wavelength and the fourth wavelength. This difference is a scalar and can be expressed as the absolute value of the value obtained by subtracting the fourth wavelength from the third wavelength, or as the absolute value of the value obtained by subtracting the third wavelength from the fourth wavelength.

[0017] In conjunction with the first aspect, in one possible implementation, the camera device further includes at least one second camera module, which is an M-channel camera, where M is greater than or equal to 3; the first camera module is a C-channel camera and C is greater than 3; and the similarity v between the spectral response curve of the first camera module and the spectral response curve of the human eye is... 0 If the similarity v′ between the spectral response curve of the first camera module and the spectral response curve of the second camera module is greater than or equal to 0.93, then...

[0018] Q N×C Let P{Q} be the spectral response matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the first camera module. N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T ;

[0019] This is the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the human eye.

[0020] X′ N×M Let P{X′ be the spectral response matrix obtained by sampling N points in each of the M channels based on the spectral response curve of the second camera. N×M}=X′ N×M [X′ N×M T X′ N×M ] -1 X′ N×M T ;

[0021] Tr() represents the trace.

[0022] In conjunction with the first aspect, in one possible implementation, the number of channels of the image sensor of the first camera module is greater than or equal to the number of channels of the image sensor of the second camera module.

[0023] In a second aspect, a camera device is provided, including a first camera module. The first camera module includes an optical lens, a filter, and an image sensor arranged sequentially along the optical axis. The optical lens is used to receive light from a target object, the filter is used to allow visible light in the light to pass through, and the image sensor is used to receive the visible light to obtain spectral information of the target object. The number of channels of the image sensor is greater than 3. The filter has a first transmittance curve and a second transmittance curve. The first transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter and its transmittance. The second transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter at a first angle and its transmittance. The first angle is greater than 0° and less than or equal to 35°. On the first transmittance curve, the wavelength corresponding to 50% transmittance includes a first wavelength, which is greater than or equal to 600 nm and less than or equal to 660 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes a second wavelength, and the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20 nm.

[0024] In this embodiment, the filter with the aforementioned parameters ensures a high degree of similarity between the spectral response curve of the first camera module and the spectral response curves of the human eye and the three-channel camera module, respectively. Consequently, after the spectral response curve of the first camera module is mapped from high dimension to low dimension, it can approximate or be equivalent to the spectral response curves of the human eye and the three-channel camera module, respectively. Therefore, it can improve the color reproduction capability of the first camera module and the accuracy of color correction for the three-channel camera module, reduce or avoid color cast problems, improve the color fidelity of the images from the three-channel camera module, and thus enhance the user experience.

[0025] Specifically, under normal incidence, the first wavelength range corresponding to 50% transmittance of this filter is 600nm-660nm, which is roughly equivalent to the corresponding wavelength range of the filter in the three-channel camera module. This limits the overall spectral response range, helping to make the spectral response curve of the first camera module, after dimensionality reduction, approximate the spectral response curve of the human eye and the three-channel camera module. Furthermore, the difference between the first and second wavelengths is within the range of 0-20nm, ensuring that within the incident light range, the transmittance curve of the filter maintains a high degree of similarity between the spectral response curve of the first camera module and the spectral response curves of the human eye and the three-channel camera module. This reduces or avoids color cast issues and improves the color fidelity of the output image.

[0026] In addition, the difference between the first wavelength and the second wavelength is in the range of 0-20nm, which can limit the angular offset of the transmittance curve of the filter, improve the color reproduction consistency between the image center and the image edge, and reduce the color difference between the image center and the image edge.

[0027] In conjunction with the second aspect, in one possible implementation, on the first transmittance curve, the wavelength corresponding to 50% transmittance includes a third wavelength, which is greater than or equal to 400 nm and less than or equal to 440 nm; on the second transmittance curve, the wavelength corresponding to 50% transmittance includes a fourth wavelength, and the difference between the third wavelength and the fourth wavelength is greater than or equal to 0 and less than or equal to 20 nm.

[0028] When incident at an oblique angle, the third wavelength corresponding to the filter at a transmittance of 50% is in the range of 400nm-440nm, which is roughly equivalent to the corresponding wavelength range of the filter in the three-channel camera module. This can limit the overall spectral response range, making the spectral response curve of the first camera module highly similar to the spectral response curves of the human eye and the three-channel camera module, respectively. This is beneficial to improving the color reproduction capability of the first camera module and the accuracy of color correction for the three-channel camera module, reducing or avoiding color cast problems, and improving the color authenticity of the images from the three-channel camera module.

[0029] The difference between the third and fourth wavelengths is in the range of 0-20nm, which can limit the angular offset of the transmittance curve of the filter, improve the color reproduction consistency between the image center and the image edge, and reduce the color difference between the image center and the image edge.

[0030] In conjunction with the first or second aspect, in one possible implementation, the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 10 nm.

[0031] In conjunction with the first or second aspect, in one possible implementation, on the first transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 3%, and the average transmittance in the 700nm–780nm band is less than or equal to 2%; on the second transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 4%, and the average transmittance in the 700nm–780nm band is less than or equal to 3%.

[0032] In this embodiment, the cutoff wavelength of the filter corresponds to the cutoff wavelength position of the human eye's visual response curve. By filtering out light waves that exceed the human eye's sensing band, the effective information can have a better signal-to-noise ratio, which is beneficial to improving image quality.

[0033] In conjunction with the first or second aspect, in one possible implementation, on the first transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%; on the second transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%.

[0034] The 440nm–580nm range represents a stable range for filter transmittance. By constraining the minimum transmittance within this range, significant fluctuations in filter transmittance can be prevented, ensuring consistent color reproduction. By constraining the average transmittance value within this range, the overall sensitivity of the multispectral camera module can be guaranteed, thereby improving image quality.

[0035] In one possible implementation, in conjunction with the first or second aspect, the first wavelength and the second wavelength belong to the red light band, and the third wavelength and the fourth wavelength belong to the blue light band.

[0036] In conjunction with the first or second aspect, in one possible implementation, the camera device further includes at least one second camera module, which is a three-channel camera; the first camera module is a C-channel camera where C is greater than 3, and the similarity v between the spectral response curve of the first camera module and the spectral response curve of the human eye is... 0 If the similarity v′ between the spectral response curve of the first camera module and the spectral response curve of the second camera module is greater than or equal to 0.93, then...

[0037] Q N×C Let P{Q} be the spectral response matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the first camera module. N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T ;

[0038] This is the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the human eye.

[0039] X′ N×3 Let P{X′ be the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the second camera.N×3}=X′ N×3 [X′ N×3 T X′ N×3 ] -1 X′ N×3 T ;

[0040] Tr() represents the trace.

[0041] When v 0 When the value is greater than or equal to 0.93, the spectral response curve of the first camera module can be as close as possible to the spectral response curve of the human eye after dimensionality reduction from high dimension. This can improve the color reproduction capability of the multispectral camera module, reduce or avoid color cast problems, and improve the color authenticity or color parameter accuracy of the output image of the multispectral camera module.

[0042] When v′ is greater than or equal to 0.93, the spectral response curve of the first camera module can be as close as possible to the spectral response curve of the second camera module after dimensionality reduction from high dimension. This can improve the accuracy of the transfer of color parameters from the multispectral camera module to the second camera module, reduce or avoid color cast caused by transfer error, and improve the color authenticity of the output image of the second camera module.

[0043] In conjunction with the first or second aspect, in one possible implementation, the spectral response matrix Q of the first camera module... N×C =M N×C ⊙T N×1 ⊙L N×1

[0044] M N×C This is a matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the image sensor;

[0045] T N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the filter;

[0046] L N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the optical lens.

[0047] Thirdly, a filter is provided for use in a camera module, which further includes an optical lens and an image sensor. The filter is disposed between the optical lens and the image sensor. The optical lens is used to receive light from a target object, and the filter allows light within a target wavelength range to pass through. The image sensor is used to receive light within the target wavelength range to obtain spectral information of the target object, and the image sensor has more than 3 channels. The filter has a first transmittance curve and a second transmittance curve, wherein the first transmittance curve is the transmittance curve at normal incidence on the filter. The second transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter and its transmittance at a first angle greater than 0° and less than or equal to 35°. On the first transmittance curve, the wavelength corresponding to 50% transmittance includes the first wavelength, which is greater than or equal to 600nm and less than or equal to 660nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes the second wavelength, and the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20nm.

[0048] In conjunction with the third aspect, in one possible implementation, the target wavelength range mentioned above is the visible light wavelength range.

[0049] Fourthly, a filter is provided for use in a camera module, which further includes an optical lens and an image sensor. The filter is disposed between the optical lens and the image sensor. The optical lens is used to receive light from a target object, the filter allows visible light in the light to pass through, and the image sensor is used to receive the visible light to obtain spectral information of the target object. The image sensor has more than 3 channels. The filter has a first transmittance curve and a second transmittance curve, the first transmittance curve being the transmittance of light normally incident on the filter. The wavelength-transmittance curve is a curve showing the relationship between wavelength and transmittance of light incident on the filter at a first angle, where the first angle is greater than 0° and less than or equal to 35°. On the first transmittance curve, the wavelength corresponding to 50% transmittance includes the first wavelength, which is greater than or equal to 600nm and less than or equal to 660nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes the second wavelength, where the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20nm.

[0050] In conjunction with the third or fourth aspect, in one possible implementation, on the first transmittance curve, the wavelength corresponding to 50% transmittance includes a third wavelength, which is greater than or equal to 400 nm and less than or equal to 440 nm; on the second transmittance curve, the wavelength corresponding to 50% transmittance includes a fourth wavelength, the difference between the third wavelength and the fourth wavelength being greater than or equal to 0 and less than or equal to 20 nm.

[0051] In conjunction with the third or fourth aspect, in one possible implementation, the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 10 nm.

[0052] In conjunction with the third or fourth aspect, in one possible implementation, on the first transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 3%, and the average transmittance in the 700nm–780nm band is less than or equal to 2%; on the second transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 4%, and the average transmittance in the 700nm–780nm band is less than or equal to 3%.

[0053] In conjunction with the third or fourth aspect, in one possible implementation, on the first transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%; on the second transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%.

[0054] In conjunction with the third or fourth aspect, in one possible implementation, the first wavelength and the second wavelength belong to the red light band, and the third wavelength and the fourth wavelength belong to the blue light band.

[0055] Fifthly, a camera module is provided, including an optical lens, an image sensor, and a filter as described in the third aspect and any implementation thereof. The filter is disposed between the optical lens and the image sensor. The optical lens is used to receive light from a target object, the filter is used to allow light of a target wavelength range to pass through, and the image sensor is used to receive light of the target wavelength range to obtain spectral information of the target object. The number of channels of the image sensor is greater than 3.

[0056] A sixth aspect provides a camera module including an optical lens, an image sensor, and a filter as described in the fourth aspect and any implementation thereof. The filter is disposed between the optical lens and the image sensor. The optical lens is used to receive light from a target object, the filter is used to allow visible light in the light to pass through, and the image sensor is used to receive the visible light to obtain spectral information of the target object. The number of channels of the image sensor is greater than 3.

[0057] In a seventh aspect, an electronic device is provided, including an image processing chip and a camera module as described in the fifth or sixth aspect above, wherein the image processing chip is used to process images acquired by the camera module.

[0058] Eighthly, an electronic device is provided, including an image processing chip and a camera device according to the first or second aspect and any implementation thereof, wherein the image processing chip is used to process images acquired by the camera device.

[0059] In conjunction with the eighth aspect, in one possible implementation, the camera device further includes at least one second camera module, which is a three-channel camera, and the image processing chip is used to correct the color of the target object acquired by the second camera module based on the spectral information of the target object acquired by the first camera module.

[0060] In conjunction with the eighth aspect, in one possible implementation, the camera device further includes at least one second camera module, which is an M-channel camera where M is greater than or equal to 3. The image processing chip is used to correct the color of the target object acquired by the second camera module based on the spectral information of the target object acquired by the first camera module.

[0061] In conjunction with the eighth aspect, in one possible implementation, the second camera module is a wide-angle camera, a telephoto camera, or an ultra-wide-angle camera.

[0062] The beneficial effects of the apparatus described in the third to eighth aspects above are similar to those described in the first and second aspects above, and will not be repeated here. Attached Figure Description

[0063] Figure 1 is a schematic structural diagram of an electronic device applicable to an embodiment of this application.

[0064] Figure 2 is a schematic exploded view of a camera module provided in an embodiment of this application.

[0065] Figure 3 is a schematic cross-sectional view of a camera module provided in an embodiment of this application.

[0066] Figure 4 is a schematic diagram of the transmittance curve of a filter provided in an embodiment of this application.

[0067] Figure 5 is a schematic diagram of the transmittance curve of another filter provided in an embodiment of this application.

[0068] Figure 6 is a schematic diagram of the transmittance curve of another filter provided in an embodiment of this application.

[0069] Figure 7 is a schematic diagram of the transmittance curve of another filter provided in the embodiments of this application.

[0070] Figure 8 is a schematic diagram of the transmittance curve of another filter provided in the embodiments of this application.

[0071] Figure 9 is a schematic diagram of the transmittance curve of another filter provided in an embodiment of this application.

[0072] Figure 10 is a schematic diagram of the transmittance curve of another filter provided in an embodiment of this application.

[0073] Figure 11 is a schematic diagram of the transmittance curve of a filter in a second camera module provided in an embodiment of this application.

[0074] Figure 12 is a schematic diagram of the transmittance curve of another filter provided in the embodiments of this application.

[0075] Figure 13 is a schematic diagram of the transmittance curve of another filter provided in the embodiments of this application.

[0076] Figure 14 is a schematic flowchart of a camera module design method provided in an embodiment of this application. Detailed Implementation

[0077] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0078] It should be noted that, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this article is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0079] In the embodiments of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more, and "at least one" and "one or more" refer to one, two, or more than two. The singular expressions "a," "an," "the," "the," "this," and "this" are intended to also include expressions such as "one or more," unless the context explicitly indicates otherwise.

[0080] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0081] In the description of the embodiments of this application, the terms "upper," "lower," "inner," "outer," "vertical," and "horizontal," etc., indicate orientations or positional relationships relative to the indicated placement of components in the accompanying drawings. It should be understood that these directional terms are relative concepts, used for relative description and clarification, and not to indicate or imply a specific orientation that the device or component must have, or its construction and operation in a specific orientation. They can change accordingly depending on the orientation of the components in the accompanying drawings, and therefore should not be construed as limiting this application. Furthermore, "vertical" in this application is not strictly vertical, but within the allowable error range. "Parallel" is not strictly parallel, but within the allowable error range.

[0082] In the embodiments of this application, the same reference numerals are used to denote the same component or part. Furthermore, the parts in the drawings are not drawn to scale, and the dimensions and sizes of the parts shown are merely exemplary and should not be construed as limiting this application.

[0083] To facilitate understanding, the technical terms used in this application will be explained and described below.

[0084] The optical axis is an imaginary line in an optical system, which can be understood as the direction in which light rays travel through the system. For a symmetrical transmission system, the optical axis generally coincides with the rotation center line of the optical system. If a ray of light coincides with the optical axis, it will travel along the optical axis within the optical system.

[0085] Auto focus (AF) is a technique that uses the principle of light reflection from the subject. The light reflected from the subject passes through the lens and is imaged and received on the image sensor. After being processed by a computer, the image sensor drives the focusing device to focus.

[0086] Optical image stabilization (OIS) refers to the use of optical components in imaging instruments such as mobile phones or cameras to avoid or reduce camera shake during the capture of optical signals, thereby improving image quality. A common approach is to use a gyroscope for shake detection, and then use an OIS motor to translate or rotate the entire lens in the opposite direction to compensate for image blur caused by camera shake during exposure.

[0087] A color space, also known as a color model, is a mathematical model used to represent color and quantify it. Common color spaces include Lab, Luv, LCh, Yxy, XYZ, CMYK, RGB, Hex, and YUV color spaces.

[0088] The parameters of a filter include: center wavelength, transmittance, peak transmittance, bandwidth, cutoff band, and incident angle.

[0089] The center wavelength (CWL) is the midpoint between wavelengths where the peak transmittance is 50%, also known as the midpoint of the full width at half maximum (FWHM). Generally, the center wavelength represents the peak transmission wavelength of a bandpass or narrowband filter, or the peak reflection wavelength of a notch filter.

[0090] Transmittance refers to the ratio of light allowed to pass through a filter to incident light, usually expressed as a percentage. It generally represents the light loss that occurs after the light enters the filter. When the transmittance is 10% or below, it is considered a cutoff.

[0091] Peak transmittance refers to the highest transmittance value in the bandpass, that is, the highest value that can be transmitted after the filter has been damaged. In a spectral curve, the area through which light actually passes is usually called the passband.

[0092] Bandwidth (full width at half maximum, FWHM), also known as half-peak full width, is a wavelength range that represents the difference between the spectral bands where the peak transmittance is located. Filters with a half-width of less than 20 nm are called narrowband filters, while filters with a half-width greater than 20 nm are called bandpass filters.

[0093] The cut-off wavelength refers to the range of wavelengths that a filter can cut off.

[0094] The angle of incidence (AOI) is the angle between the incident light and the normal to the surface of the filter. When the light is incident perpendicularly, the angle of incidence is 0°.

[0095] An operator is a mapping that transforms one mathematical object into another. An operator can be understood as an operation that processes input according to certain rules and produces output.

[0096] The trace of a matrix is ​​the sum of its eigenvalues. For a square matrix, the trace represents the sum of all elements along the main diagonal (from the top left to the bottom right).

[0097] It should be noted that the above-described terms and concepts are for illustrative purposes only and should not be construed as limiting the embodiments of this application.

[0098] Figure 1 shows a schematic structural diagram of an electronic device to which an embodiment of this application applies.

[0099] In this application, the electronic devices involved are those with imaging capabilities, such as mobile phones, personal digital assistants (PDAs), tablet computers, laptop computers, cameras, video recorders, smartwatches, smart wristbands, point-of-sale (POS) terminals, in-vehicle systems, televisions (e.g., smart screens), and wearable devices. This application does not impose any special limitations on the specific form of the electronic device. For ease of explanation and understanding, the following description uses a mobile phone as an example.

[0100] For example, Figures 1(a) and (b) schematically show the front and back of the electronic device 100, respectively. As shown in Figure 1, the electronic device 100 may include a housing 101, a display panel (DP) 102, and a camera array 103.

[0101] The housing 101 has a receiving space for accommodating the components of the electronic device 100. The housing 101 also serves to protect the electronic device 100 and support the entire device. The display screen 102 and the camera array 103 are disposed within the receiving space of the housing 101 and connected to the housing 101. In some embodiments, the housing 101 may include a rear cover opposite to the display screen 102 and a mid-frame disposed between the rear cover and the display screen 102; the display screen 102 and the camera array 103 may be fixed to the mid-frame. The housing 101 may be made of metal, plastic, ceramic, or glass, etc.

[0102] The display screen 102 is used to display images, such as images captured by the camera array 103. The display screen 102 can be a liquid crystal display (LCD) screen, an organic light emitting diode (OLED) screen, etc. The display screen 102 can be a regular screen, or an irregularly shaped screen, a foldable screen, etc. The display screen 102 can be located on the front and / or back of the electronic device 100. Here, the front of the electronic device 100 can be understood as the side facing the user when using the electronic device 100, and the back of the electronic device 100 can be understood as the side facing away from the user when using the electronic device 100.

[0103] Camera array 103 is used to capture still images or videos. Camera array 103 includes multiple camera compact modules (CCMs) (or simply cameras). For example, camera array 103 includes at least one multispectral camera module (also referred to as a first camera module in this application) and at least one second camera module, wherein the second camera module is a telephoto camera, wide-angle camera, ultra-wide-angle camera, or depth-sensing camera, etc. Here, the ultra-wide-angle camera includes a fisheye lens. When camera array 103 includes multiple multispectral camera modules (or multiple second camera modules), the multiple multispectral camera modules (or multiple second camera modules) can be identical or different; for example, the multiple multispectral camera modules (or multiple second camera modules) may have different focal lengths, different optical structures, or different optical parameters, etc.

[0104] In some embodiments, each of the at least one multispectral camera modules is capable of independent imaging, and the captured images can be directly presented to the user for viewing.

[0105] In some embodiments, each of the at least one second camera module is capable of independent imaging, and the captured images can be directly presented to the user for viewing. One or more of the at least one second camera module can serve as the main camera. Typically, the main camera is responsible for the primary shooting task, usually has the highest pixel count, and can provide higher resolution and a more powerful sensor, thereby meeting the user's photography needs in different scenarios.

[0106] In some embodiments, the images acquired by the multispectral camera module can be used to perform color calibration on the images acquired by the second camera module. For example, the M-channel image of the first scene acquired by the second camera module can be used as a reference to perform color calibration on the first scene's M-channel image acquired by the second camera module, where M is greater than or equal to 3. In some embodiments, the second camera module can be a three-channel camera, and the images it acquires are three-channel images, such as red-green-blue (RGB) images, red-green-green-blue (RGGB) images, red-yellow-blue (RYB) images, or red-yellow-yellow-blue (RYYB) images, etc. In other embodiments, the second camera module can be a four-channel camera, and the images it acquires are four-channel images, such as red-green-blue-white (RGBW) images, or red-green-blue-yellow (RGBY) images, etc. This disclosure does not specifically limit this.

[0107] The second camera module is an M-channel camera module, where M is greater than or equal to 3. Due to material limitations, its corresponding spectral response curve differs somewhat from that of the human eye, and it can perceive a smaller number of spectral bands, resulting in weaker color reproduction and a tendency for color casts in the acquired images. In contrast, the multispectral camera module can minimize the difference between its spectral response curve and that of the human eye, and it possesses stronger spectral perception capabilities (i.e., it can perceive more than 3 spectral bands). Therefore, it has superior color reproduction capabilities, and the colors in the acquired images are more closely approximating to true colors, where true colors refer to the colors perceived by the human eye. By using the images acquired by the multispectral camera module as a benchmark, color calibration of the second camera module can improve the color accuracy of the output images.

[0108] In some embodiments, the number of channels of the image sensor of the multispectral camera module (also referred to as the first camera module in this application) is greater than or equal to M. In other words, the number of channels of the image sensor of the first camera module is greater than or equal to the number of channels of the image sensor of the second camera module. It should be noted that when the number of channels of the first camera module and the second camera module is the same, color correction can also be performed on the image of the second camera module based on the image of the first camera module.

[0109] In some embodiments, the images acquired by the multispectral camera module can be used to perform color calibration on the images acquired by the second camera module. For example, the three-channel images (such as red-green-blue (RGB) images or red-yellow-blue (RYB) images of the first scene acquired by the second camera module) can be color-calibrated based on the multispectral image of the first scene acquired by the multispectral camera module.

[0110] The second camera module is a three-channel camera module. Due to material limitations, its spectral response curve differs somewhat from that of the human eye, and it can perceive a smaller number of spectral bands, resulting in weaker color reproduction and a tendency for color casts in the captured images. In contrast, the multispectral camera module can minimize the difference between its spectral response curve and that of the human eye, and it possesses stronger spectral perception capabilities (i.e., it can perceive more than three spectral bands). Therefore, it has superior color reproduction capabilities, and the colors in the captured images are more closely approximating to true colors after restoration. By using images captured by the multispectral camera module as a benchmark, color calibration of the second camera module can improve the color accuracy of the output images.

[0111] In some embodiments, the multispectral camera module or the second camera module can be a vertical module or a foldable module (or periscope camera module). A vertical camera module can be understood as light entering the camera module directly hitting the image sensor without bending the light path. A foldable camera module can be understood as light entering the camera module needing to pass through optical elements such as mirrors and prisms before hitting the image sensor, resulting in a folded light path.

[0112] In some embodiments, the camera array 103 can be disposed on the front and / or back of the electronic device 100. When the camera array 103 is disposed on the front of the electronic device 100, it can also be referred to as a front-facing camera. The front-facing camera may also include a telephoto camera, a wide-angle camera, or an ultra-wide-angle camera, etc. The first camera module (or multispectral camera module) in this embodiment can also be used in combination with the front-facing camera. The image captured by the multispectral camera module can also correct the image captured by the front-facing camera, improving the color accuracy of the front-facing camera. When the camera array 103 is disposed on the back of the electronic device 100, it can also be referred to as a rear-facing camera. In some embodiments, when the display screen 102 can be folded, the camera array 103 can function as a front-facing camera or a rear-facing camera as the display screen 102 folds. It is understood that the placement position of the camera array 103 can be determined according to actual needs, and the installation position shown in Figure 1 is merely illustrative.

[0113] In some embodiments, the electronic device 100 may further include a protective lens 104 for protecting the camera modules in the camera array 103. The protective lens 104 is disposed on the housing 101 and covers the camera modules. For example, when the protective lens 104 is used to protect the front-facing camera, the protective lens 104 may cover only the front-facing camera module or cover the entire front of the electronic device 100. When the protective lens 104 covers the entire front of the electronic device 100, it can also be used to protect the display screen 102, in which case the protective lens 104 is the cover glass (CG). As another example, when the protective lens 104 is used to protect the rear-facing camera, the protective lens 104 may cover the entire back of the electronic device 100, or it may be disposed only at the position corresponding to the rear-facing camera module.

[0114] In some embodiments, the protective lens 104 may be made of glass, sapphire, ceramic, etc., and this application does not impose any special limitations on it. For example, the protective lens 104 is transparent, and light from outside the electronic device 100 can enter the camera module through the protective lens 104.

[0115] In some embodiments, the electronic device 100 may further include a circuit board and an image processor (not shown), located within a receiving space formed by the housing 101. The image processor is fixed to and electrically connected to the circuit board. The image processor is communicatively connected to a camera module (e.g., a multispectral camera module or a second camera module) in the camera array 103. The image processor is used to acquire and process image data from the camera module. The communication connection between the camera module and the image processor may include data transmission via electrical connections such as wiring, or data transmission via coupling or other means. It is understood that the camera module and the image processor may also communicate via other methods capable of data transmission.

[0116] In some embodiments, the electronic device 100 may further include an analog-to-digital converter (also known as an A / D converter, not shown in the figure). The analog-to-digital converter is connected between the camera module and the image processor. The analog-to-digital converter is used to convert the signal generated by the camera module into a digital image signal and transmit it to the image processor, which then processes the digital image signal and finally displays the image or video on the display screen 102.

[0117] In some embodiments, the electronic device 100 may further include a memory (not shown) communicatively connected to an image processor. The image processor processes the digital image signal and then transmits the image to the memory, so that the image can be retrieved from the memory and displayed on the display screen 102 at any time when it is needed to view the image later. In some embodiments, the image processor may also compress the processed digital image signal before storing it in the memory to save memory space.

[0118] It should be understood that the structure shown in Figure 1 does not constitute a specific limitation on the electronic device 100. The electronic device 100 may include more or fewer components than shown in the figure. For example, the electronic device 100 may also include one or more of the following components: battery, flash, earpiece, buttons, sensors, etc. Alternatively, the electronic device 100 may not include the display screen 102, or the electronic device 100 may have a different component arrangement than shown in the figure.

[0119] Figures 2 and 3 show schematic diagrams of a camera module according to an embodiment of this application. Figure 2 is a schematic exploded view of the camera module 200, and Figure 3 is a schematic cross-sectional view of the camera module 200. The camera module 200 in Figure 2 can be an exemplary structure of the multispectral camera module or the second camera module in Figure 1. The structure of the camera module 200 will be briefly described below with reference to Figures 2 and 3.

[0120] For ease of description, the optical axis direction of the camera module 200 is defined as the Z direction, and the two directions perpendicular to the optical axis are the X direction and the Y direction, with the X direction perpendicular to the Y direction. In the Z direction, the side facing the object being photographed is the front side, and the side facing away from the object is the rear side. In the X and Y directions, the direction closer to the optical axis is the inner side, and the direction facing away from the optical axis is the outer side. In this embodiment, the optical axis direction is the direction in which the optical system transmits light.

[0121] Here, the definitions of X, Y, Z directions and front, back, inside, and outside also apply to the various figures described below. It should be noted that the above definitions of X, Y, Z directions and front, back, inside, and outside are merely for the convenience of describing the positional, connection, or motion relationships between the components in the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0122] As shown in Figures 2 and 3, the camera module 200 may include a housing 210, a lens assembly 220, a lens actuator 230, and a light sensing component 240.

[0123] The housing 210 has a receiving space for accommodating the lens assembly 220, the lens actuator 230, the light sensing assembly 240, etc. Additionally, the housing 210 also serves a protective and support function. It is understood that the structure of the housing 210 shown in Figures 2 and 3 is merely exemplary and should not be construed as limiting the scope of this application. Those skilled in the art can design the shape of the housing 210 according to actual needs.

[0124] The lens assembly 220 (or optical lens) mainly includes a lens group 221 and a lens barrel 222, with the lens group 221 housed within the receiving space formed by the lens barrel 222. The lens assembly 220 is used to image the scene on the object side onto the image plane on the image side. In some embodiments, the lens assembly 220 can also perform certain processing on the received imaging beam, such as aberration correction and chromatic aberration elimination. Here, the imaging beam refers to the beam formed by the light incident on the camera module 200.

[0125] Lens group 221 may include at least one lens. The lenses in lens group 221 may be different or at least partially the same. This application embodiment does not specifically limit the number of lenses included in lens group 221. Those skilled in the art can set the number of lenses according to actual needs, such as 1, 2, 3, 5, 8 or more.

[0126] The focal length of lens group 221 can be fixed, and correspondingly, lens assembly 220 is a prime lens. Alternatively, the focal length of lens group 221 can be adjusted, and correspondingly, lens assembly 220 is a zoom lens. For example, the focal length of lens group 221 can be adjusted by changing the relative positions of the lenses within it.

[0127] The lens barrel 222 has a receiving space, primarily for accommodating the lens assembly 221. In some embodiments, the lens barrel 222 can be a single unit, with the lens assembly 221 housed within this single unit. In other embodiments, the lens barrel 222 may also comprise multiple lens barrel sections, with the lenses of the lens assembly 221 grouped and disposed within these multiple lens barrel sections, wherein each lens barrel section and the lenses housed therein can be referred to as a lens group. Exemplarily, the relative positions between these multiple lens barrel sections can be adjusted, enabling optical zoom by adjusting the relative positions of the lenses.

[0128] It is understood that the structure of the lens barrel 222 and the connection method between the lens group 221 and the lens barrel 222 in Figures 2 and 3 are merely exemplary and do not impose any limitations on the embodiments of this application.

[0129] The lens actuator 230 is used to move the lens assembly 220 to achieve autofocus and / or optical image stabilization. In some embodiments, the lens actuator 230 may also be referred to as a lens motor, or simply a motor.

[0130] As shown in Figure 3, the lens actuator 230 may include a motor (hereinafter referred to as the AF motor) 231 for moving the lens assembly 220 for AF and / or a motor (hereinafter referred to as the OIS motor) 232 for moving the lens assembly 220 for OIS. Specifically, the AF motor 231 is used to move the lens assembly 220 for autofocus in the Z direction, and the OIS motor 232 is used to move the lens assembly 220 for optical image stabilization in the X and / or Y directions. In some embodiments, the AF motor 231 and the OIS motor 232 may be two independent components, which independently drive the lens assembly 220 for AF and OIS respectively. Alternatively, the AF motor 231 and the OIS motor 232 may be integrated into one unit, with one motor driving the lens assembly 220 for AF and OIS. Figure 3 exemplarily illustrates that the lens actuator 230 includes independent AF motor 231 and OIS motor 232, but it should be understood that the embodiments of this application are not limited thereto.

[0131] In some embodiments, the AF motor 231 or the OIS motor 232 can be used to move the entire lens assembly 220, or to move a portion of the lens assembly 220. For example, if a portion of the lens assembly 220 (such as the first lens group) is relatively fixed and another portion (such as the second lens group) is movable, the AF motor 231 or the OIS motor 232 can drive the movable portion to move, thereby changing the optical path to achieve the desired function.

[0132] In some embodiments, the AF motor 231 or the OIS motor 232 may be a voice coil motor (VCM), a shape memory alloy (SMA) motor, a stepping motor, a piezoelectric motor, etc. It should be understood that the specific structure of the AF motor 231 or the OIS motor 232 may be designed and selected according to the selected driving method, and the embodiments of this application do not limit this.

[0133] The light sensing component 240 is disposed on the rear side of the lens assembly 220 and is mainly used for imaging. For example, the light sensing component 240 may include a filter 241, an image sensor 242, and a circuit board 243.

[0134] A filter 241 is disposed between the lens assembly 220 and the image sensor 242. The filter 241 eliminates unwanted light projected onto the image sensor 242, preventing ghosting, stray light, and color cast during image formation. For example, the filter 241 can be an infrared cut-off filter (filtering out long-wavelength light other than visible light), a bandpass filter (allowing light of a specific wavelength to pass through while blocking light outside the passband), or a filter for filtering out other wavelengths of light. The filter 241 can have different effective ranges depending on the application scenario.

[0135] Image sensor 242 is a semiconductor chip used to convert collected external light signals into electrical signals. Specifically, the surface of image sensor 242 contains hundreds of thousands to millions of photodiodes. These photodiodes generate charges when illuminated, thereby converting the light signals collected by lens assembly 220 into electrical signals. For example, image sensor 242 may be a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) device.

[0136] The image sensor 242 includes multiple photosensitive units, which can convert light signals into electrical charges to form an electronic image corresponding to the scene. Each photosensitive unit corresponds to one pixel; the more pixels, the clearer the image.

[0137] In the image sensor used in a three-channel camera module, each photosensitive unit includes three different filter channels, such as red, green, and blue filter channels, or red, yellow, and blue filter channels. For example, different filter channels can be formed by placing three different colored filters on the photosensitive unit. For instance, a three-channel camera module may include a red-green-blue (RGB) camera module, a red-green-green-blue (RGGB) camera module, a red-yellow-blue (RYB) camera module, or a red-yellow-yellow-blue (RYYB) camera module, etc.

[0138] In a four-channel camera module, each photosensitive unit includes four different filter channels. For example, placing four different colored filters on the photosensitive unit can form different filter channels, such as red, green, blue, and white filter channels, thus constituting a red-green-blue-white (RGBW) camera module. Alternatively, a four-channel camera module can also be a red-green-blue-yellow (RGBY) camera module. Similarly, in an M-channel camera module, each photosensitive unit includes M different filter channels. For example, placing M different colored filters on the photosensitive unit can form different filter channels.

[0139] In the image sensor used in the multispectral camera module, each photosensitive unit includes more than three filter channels. For example, the multispectral image sensor may include four, five, six, seven, eight, nine or more filter channels.

[0140] In the image sensor used in a multispectral camera module (hereinafter referred to as a multispectral image sensor for ease of description), each photosensitive unit includes more than three filter channels. Different filter channels can be formed by creating differentiated color filter arrays (CFAs) through methods such as chemical dyes, coating interference, and metasurface micro / nano structures. Therefore, a multispectral image sensor can extract spectral information from multiple channels to acquire images. Because the multispectral image sensor has more filter channels and stronger spectral sensing capabilities, the color fidelity of the perceived multispectral image is higher. The multispectral image sensor can acquire multispectral images through multiple filter channels and can transmit the acquired image information to the color restoration module for analysis, achieving color imaging or color information extraction.

[0141] For example, a multispectral image sensor may include a spectral modulation region and a photoelectric conversion region. The spectral modulation region uses multiple materials or optical structures to form multiple filter channels to split the incident light. The photoelectric conversion region converts the split light signal into an electrical signal, and then outputs a digital signal or encoding through analog-to-digital conversion. A color restoration module (e.g., an image signal processor) is electrically connected to the multispectral image sensor. The color restoration module can calculate spectral information or color information based on the light signal and pixel position information input from the multispectral image sensor. Based on the color transformation matrix from multiple channels to the color space, the color restoration module can also convert the multispectral signal acquired by the multispectral image sensor into the color space information of the image (e.g., RGB information, XYZ tristimulus values, or YUV information). Here, the color transformation matrix from multiple channels to the color space is established based on the visual response curve of the human eye (e.g., the spectral tristimulus value curve, including the response curve of the human eye to red, green, and blue light) and the spectral response curve of the multispectral camera module (including the spectral response curve of the multispectral camera module to different wavelengths). In some embodiments, the color restoration module can also use the spectral information acquired by the multispectral camera module to perform color calibration on other cameras. For example, based on the transformation relationship between the spectral response curve of the multispectral camera module and the spectral response curve of other camera modules, the color space information corresponding to the multispectral camera module can be mapped onto other camera modules to correct their colors, making the image colors of other camera modules more accurate.

[0142] In some embodiments, camera modules with different focal lengths may have their own image sensors, or camera modules with different focal lengths may share the same image sensor.

[0143] In some embodiments, the light sensing component 240 may include an image signal processing (ISP) module. The ISP module is used to process the signals acquired by the image sensor 242, such as performing linear correction, noise reduction, automatic white balance, automatic exposure control, color correction, etc., converting the raw data acquired by the image sensor 242 into a format supported by the algorithm. For example, the image signal processing module may be integrated with the image sensor 242.

[0144] Circuit board 243 is used to transmit electrical signals, and it can be a flexible printed circuit (FPC) or a printed circuit board (PCB). Image sensor 242 can be electrically connected to circuit board 243 via wires to extract signals.

[0145] In some embodiments, the light sensing component 240 may further include a microelectromechanical system (MEMS) actuator 244, which drives the image sensor 242 to move along the optical axis and / or perpendicular to the optical axis, thereby achieving autofocus and / or optical image stabilization. The MEMS actuator 244 can be driven by electrostatic force, magnetoelectric force, piezoelectric force, thermoelectric force, etc. It should be understood that the specific structure of the MEMS actuator 244 can be designed and selected according to the chosen driving method, and this application does not limit it in this regard.

[0146] It should be understood that the structures illustrated in Figures 2 and 3 do not constitute a specific limitation on the camera module 200. The camera module 200 may include more or fewer components than illustrated. For example, the camera module 200 may also include connectors and peripheral electronic components, or the camera module 200 may not include the lens actuator 230, which will not be described in detail here.

[0147] As mentioned earlier, multispectral camera modules have stronger spectral sensing and color reproduction capabilities than other camera modules. Therefore, the spectral information acquired by multispectral camera modules can be used to perform color calibration on images acquired by other camera modules. However, the accuracy of current multispectral camera modules for color calibration of other camera modules still needs improvement. The following description will use a three-channel camera module as an example. It should be noted that the following description also applies to cases where the other camera module is a four-channel or more-channel camera module; this application will not elaborate on these cases.

[0148] Multispectral camera modules have stronger spectral sensing and color reproduction capabilities than three-channel camera modules. Therefore, the spectral information acquired by multispectral camera modules can be used to perform color calibration on images acquired by three-channel camera modules. However, the accuracy of using multispectral camera modules for color calibration of three-channel camera modules still needs improvement.

[0149] This is because the accuracy of multispectral camera modules used for color calibration depends not only on their own spectral sensing and color reproduction capabilities, but also on the similarity (or equivalence) between their spectral response curves and those of the three-channel camera module. Specifically, because multispectral camera modules sense more spectral bands, their spectral sensing function can be dimensionality-reduced and transformed into the spectral sensing function of the three-channel camera module. The higher the equivalence between the spectral response curves of the multispectral and three-channel camera modules, the higher the transformation accuracy. In other words, the more accurate the dimensionality reduction and fitting process from the multispectral camera module's spectral sensing function to the three-channel camera module's spectral sensing function, the more precise the three-channel camera module can obtain more accurate color information from the multispectral camera module based on the more precise dimensionality reduction transformation matrix between the two spectral sensing functions.

[0150] For example, if the spectral response curve of a multispectral camera module is completely equivalent to that of a three-channel camera module, then the spectral response curve of the multispectral camera module, after dimensionality reduction from a high dimension, is exactly the same as that of the three-channel camera module. In other words, the color perception capability of the multispectral camera module is completely compatible with that of the three-channel camera module. Therefore, the spectral perception function of the multispectral camera module can be linearly transformed into the spectral perception function of the three-channel camera module. Correspondingly, based on the dimensionality reduction transformation matrix between the two, a direct conversion of the color space can be performed through linear transformation. The color information obtained by the three-channel camera module is highly consistent with the color perceived by the multispectral camera module. When the color of the multispectral camera module is accurate, the three-channel camera module can obtain the true color information.

[0151] Conversely, the lower the similarity between the spectral response curve of the multispectral camera module and the spectral response curve of the three-channel camera module, the greater the difference between the spectral response curve of the multispectral camera module after dimensionality reduction from high dimension and the spectral response curve of the three-channel camera module. In other words, the lower the accuracy of dimensionality reduction fitting from the spectral perception function of the multispectral camera module to the spectral perception function of the three-channel camera module, the less accurate the three-channel camera module can obtain the color information of the multispectral camera module based on the dimensionality reduction transformation matrix between the two.

[0152] Similarly, for a multispectral camera module to achieve accurate color reproduction, its spectral response curve must be as equivalent as possible to the spectral response curve of the human eye (such as the tristimulus value curve). For example, when the spectral response curve of the multispectral camera module, after dimensionality reduction from a high dimension, is completely identical to the tristimulus value curve of the human eye—meaning the color perception capability of the multispectral camera module is perfectly adapted to the color perception capability of the human eye—then the spectral perception function of the multispectral camera module can be linearly transformed into the tristimulus value curve of the human eye. Correspondingly, based on the dimensionality reduction transformation matrix between the two, a direct conversion of the color space can be performed through linear transformation to obtain completely accurate color information.

[0153] However, the color reproduction capability of a multispectral camera module is affected by the filters in the module. The difference in the transmittance of the filters to light causes the spectral information acquired by the multispectral camera module to be prone to deviation when reproducing real image information in the later stage, resulting in a large deviation between the final image and the color seen by the human eye, and poor image authenticity.

[0154] In view of this, embodiments of this application provide a filter that, when applied to a multispectral camera module, can improve the accuracy of color calibration of the multispectral camera module while ensuring the color reproduction capability of the multispectral camera module, thereby improving the color authenticity and consistency of the image.

[0155] The filter provided in this embodiment is applied to a first camera module, which is a multispectral camera module. The first camera module also includes an optical lens and an image sensor, with the filter disposed between the optical lens and the image sensor. The optical lens receives light from a target object. The filter filters out light outside the target wavelength range, allowing light within the target wavelength range to pass through. For example, if the target wavelength range is the visible light wavelength range, the filter filters out light outside the visible light range. The image sensor is a multispectral image sensor with more than three channels.

[0156] In some embodiments, the target wavelength range can be from 380 nm to 750 nm. In other embodiments, the target wavelength range can be from 380 nm to 2526 nm, etc. This application does not specifically limit the target wavelength range.

[0157] The filter with the above-mentioned parameters provided in this application, when applied to a multispectral camera module, can make the spectral response curve of the multispectral camera module highly equivalent to the spectral response curve of the human eye. Accordingly, after the spectral response curve of the multispectral camera module is mapped from high dimension to low dimension, it can approach or be equivalent to the spectral response curve of the human eye. The image captured by the first camera module is closer to the color of the image seen by the human eye, thus improving the color reproduction capability of the multispectral camera module. Based on the image captured by the first camera module, color correction can also be performed on the image captured by the second camera module to improve the realism of the image captured by the second camera module.

[0158] In the application scenario of color correction of the second camera module based on the first camera module, the first wavelength range corresponding to the first transmittance curve of the above-mentioned filter at 50% transmittance is 600nm-660nm. This first wavelength range is close to the corresponding wavelength range of conventional filters in the second camera module. The spectral response curve of the first camera module can be close to the spectral response curve of the second camera module, which further helps to improve the accuracy of color correction of the second camera module by the first camera module, reduce or avoid color cast problems, improve the color authenticity of the output image of the second camera module, and enhance the user experience.

[0159] Specifically, under normal incidence, the range of the first wavelength corresponding to 50% transmittance of the filter is roughly equivalent to the corresponding wavelength range of the filter in the second camera module. This limits the overall spectral response range, which helps the spectral response curve of the multispectral camera module, after dimensionality reduction from high-dimensionality, approximate the spectral response curve of the human eye and the second camera module. Furthermore, the difference between the first and second wavelengths is within the range of 0-20nm, ensuring that within the incident light range, the transmittance curve of the filter maintains a high degree of similarity between the spectral response curve of the first camera module and the spectral response curves of both the human eye and the second camera module. This reduces or avoids color cast issues and improves the color fidelity of the output image.

[0160] It should be noted that the "difference between the first wavelength and the second wavelength" mentioned in the embodiments of this application refers to the numerical difference between the first wavelength and the second wavelength. This difference is a scalar and can be expressed as the absolute value of the value obtained by subtracting the second wavelength from the first wavelength, or as the absolute value of the value obtained by subtracting the first wavelength from the second wavelength.

[0161] Furthermore, compared to normal incidence, when light incident on the filter surface has an angle, the optical path length of the light within the filter film increases. This causes a change in the transmittance of light of the same wavelength within the filter, resulting in a transmittance curve angular shift (shifting towards shorter wavelengths). In other words, the change in the angle of incident light causes a shift in the filter's transmittance curve. Generally, the larger the angle of incident light, the greater the shift in the transmittance curve.

[0162] In this embodiment, the difference between the first wavelength under normal incidence and the second wavelength under oblique incidence is within the range of 0-20 nm. This ensures that even if the transmittance curve of the filter shifts when the incident angle of the light changes within the first angle (e.g., 35°), the spectral response curve of the multispectral camera module maintains a high degree of similarity to the spectral response curves of the human eye and the second camera module. In other words, a shift of less than 20 nm in the transmittance curve of the filter can still ensure a high degree of similarity between the spectral response curve of the multispectral camera module and the spectral response curves of the human eye and the second camera module, reducing or avoiding color cast issues and improving the color reproduction consistency between the image center and edges.

[0163] In the application scenario of correcting the image of the second camera module based on the image captured by the first camera module, the corresponding wavelength ranges of the filters in the first camera module and the filters in the second camera module are roughly equivalent. This can limit the spectral response range as a whole, making the spectral response curves of the first camera module and the second camera module highly similar. This is beneficial to improving the accuracy of the first camera module in color correction of the second camera module, reducing or avoiding color cast problems, and improving the color authenticity of the second camera module image.

[0164] The effects of the filter provided in this application can be verified through experimental measurement, simulation and other methods.

[0165] For example, in an experimental measurement-based approach, the spectral response curves of the multispectral camera module (including the filter provided in this application) (hereinafter referred to as the first spectral response curve for ease of description) and the second camera module (hereinafter referred to as the second spectral response curve for ease of description) can be measured experimentally. The spectral response curve of the human eye (hereinafter referred to as the third spectral response curve for ease of description) can be obtained using existing spectral tristimulus value curves. Based on the transformation relationship between the first and third spectral response curves, the color information of the target object captured by the multispectral camera module can be obtained, and the human eye can judge the color reproduction capability of the multispectral camera module. Based on the transformation relationship between the first and second spectral response curves, the color space information corresponding to the multispectral camera module can be mapped onto the second camera module, and the human eye can judge whether the image presented by the second channel camera module has color cast issues, which can be used to determine the accuracy of color correction performed by the multispectral camera module.

[0166] For example, in a simulation-based approach, a first spectral response curve can be obtained through simulation, and a second spectral response curve can be obtained through simulation or experimentation. The third spectral response curve can be the spectral tristimulus value curve. Computer simulation can be used to calculate the dimensionality reduction fitting accuracy between the first and second spectral response curves, as well as the dimensionality reduction fitting accuracy between the first and third spectral response curves. Based on the dimensionality reduction fitting accuracy between the first and third spectral response curves, the color reproduction capability of the multispectral camera module can be determined; based on the dimensionality reduction fitting accuracy between the first and second spectral response curves, the accuracy of the multispectral camera module in color correction for a second camera module can be determined.

[0167] The filter provided in this embodiment is applied to a first camera module, which is a multispectral camera module. The first camera module also includes an optical lens and an image sensor, with the filter disposed between the optical lens and the image sensor. The optical lens receives light from a target object. The filter filters out light other than visible light, allowing visible light to pass through. The image sensor is a multispectral image sensor with more than three channels.

[0168] The filter has a first transmittance curve and a second transmittance curve. The first transmittance curve is the curve showing the relationship between the wavelength of light incident on the filter and the transmittance. The second transmittance curve is the curve showing the relationship between the wavelength of light incident on the filter at a first angle greater than 0° and less than or equal to 35°.

[0169] On the first transmittance curve, the wavelength corresponding to 50% transmittance includes a first wavelength, which is greater than or equal to 600 nm and less than or equal to 660 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes a second wavelength, and the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20 nm.

[0170] It is understood that the first transmittance curve is obtained under normal incidence conditions, where the incident angle of the light rays incident on the filter is 0°. The second transmittance curve is obtained under oblique incidence conditions, where the incident angle of the light rays incident on the filter is the first angle. Both the first and second transmittance curves of the filter can be obtained through optical experiments or simulation experiments, and this application does not impose specific limitations on this.

[0171] It is understandable that the filters installed in camera modules typically have high transmittance for visible light. Based on this characteristic, the transmittance curve of the filter (with wavelength on the horizontal axis and transmittance on the vertical axis) usually has the following common characteristics: as the wavelength moves from the ultraviolet band to the visible light band, at least part of the transmittance curve shows an upward trend; for ease of description, this part of the curve is called the rising edge of the transmittance curve. Conversely, as the wavelength moves from the visible light band to the infrared band, at least part of the transmittance curve shows a downward trend; for ease of description, this part of the curve is called the falling edge of the transmittance curve. It should be noted that there may be small local fluctuations within the rising or falling edges of the transmittance curve; such small fluctuations do not affect the overall trend of the rising or falling edges.

[0172] In this embodiment, the first wavelength is greater than or equal to 600 nm and less than or equal to 660 nm, and the first wavelength is located at the falling edge of the first transmittance curve. The difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20 nm. The first wavelength and the second wavelength are close to each other, and the second wavelength is located at the falling edge of the second transmittance curve. Therefore, it can also be understood that the first wavelength corresponding to 50% transmittance at the falling edge of the first transmittance curve is greater than or equal to 600 nm and less than or equal to 660 nm. The difference between the first wavelength corresponding to 50% transmittance at the falling edge of the first transmittance curve and the second wavelength corresponding to 50% transmittance at the falling edge of the second transmittance curve is greater than or equal to 0 and less than or equal to 20 nm.

[0173] The filter with the above-mentioned parameters provided in this application, when applied to a multispectral camera module, can make the spectral response curve of the multispectral camera module highly equivalent to the spectral response curves of the human eye and the three-channel camera module, respectively. Correspondingly, after the spectral response curve of the multispectral camera module is mapped from high dimension to low dimension, it can approach or be equivalent to the spectral response curves of the human eye and the three-channel camera module, respectively. Therefore, it can improve the color reproduction capability of the multispectral camera module and the accuracy of color correction for the three-channel camera module, reduce or avoid color cast problems, improve the color fidelity of the output image of the three-channel camera module, and enhance the user experience.

[0174] Specifically, under normal incidence, the range of the first wavelength corresponding to 50% transmittance of the filter is roughly equivalent to the corresponding wavelength range of the filter in the three-channel camera module. This limits the overall spectral response range, which helps the spectral response curve of the multispectral camera module, after dimensionality reduction from high-dimensionality, approximate the spectral response curve of the human eye and the spectral response curve of the three-channel camera module. Furthermore, the difference between the first and second wavelengths is within the range of 0-20nm, ensuring that within the incident light range, the transmittance curve of the filter maintains a high degree of similarity between the spectral response curve of the first camera module and the spectral response curves of the human eye and the three-channel camera module. This reduces or avoids color cast issues and improves the color fidelity of the output image.

[0175] Furthermore, compared to normal incidence, when the light incident on the filter surface has an angle, the optical path length of the light in the filter film increases, causing a change in the transmittance of light of the same wavelength in the filter, resulting in a transmittance curve angle drift (drift towards shorter wavelengths). That is, due to the change in the angle of incident light, the transmittance curve of the filter shifts. Generally, the larger the angle of incident light, the greater the shift in the transmittance curve. In the embodiments of this application, the difference between the first wavelength under normal incidence and the second wavelength under oblique incidence is in the range of 0-20 nm, ensuring that even if the transmittance curve of the filter shifts when the incident angle of light varies within the first angle (e.g., 35°), it still maintains a high degree of similarity between the spectral response curve of the multispectral camera module and the spectral response curves of the human eye and the three-channel camera module. In other words, a shift of less than 20nm in the transmittance curve of the filter can make the spectral response curve of the multispectral camera module highly similar to the spectral response curves of the human eye and the three-channel camera module, respectively, reducing or avoiding color cast problems and improving the color reproduction consistency of the image center and edges.

[0176] Generally, infrared light forms a virtual image on the target surface of visible light imaging, affecting the color and quality of the image. In the embodiments of this application, the difference between the first wavelength and the second wavelength is in the range of 0-20nm, ensuring that the filter has a good cutoff effect on infrared light within the incident angle range, which is beneficial to improving imaging quality and reducing the risk of false colors.

[0177] Furthermore, the difference between the first and second wavelengths is within the range of 0-20, which constrains the angular offset of the transmittance curve and can reduce the color shading of the module. Specifically, the incident angles of light at the center and edge of the filter are different, causing the transmittance curve at the edge of the filter to shift compared to the transmittance curve at the center. Consequently, the spectral response curves at the center and edge of the multispectral camera module deviate, resulting in color difference between the image center and edge. Compared to ordinary three-channel camera modules, multispectral camera modules have more spectral channels, making additional correction more difficult. However, the filter provided in this application has a small angular offset in its transmittance curve, which can reduce the deviation of the spectral response curves at the center and edge of the camera module at the source, thereby reducing the color difference between the image center and edge. Therefore, without additional correction, the color shading of the multispectral camera module can be reduced to ensure the accuracy of color reproduction throughout the image and the accuracy used for color correction.

[0178] In some embodiments, on the first transmittance curve, the wavelength corresponding to 50% transmittance includes a third wavelength, which is greater than or equal to 400 nm and less than or equal to 440 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes a fourth wavelength, and the difference between the third wavelength and the fourth wavelength is greater than or equal to 0 and less than or equal to 20 nm.

[0179] In this embodiment, the third wavelength is greater than or equal to 400 nm and less than or equal to 440 nm, and the third wavelength is located at the rising edge of the first transmittance curve, while the fourth wavelength is located at the rising edge of the second transmittance curve. Therefore, it can also be understood that the third wavelength corresponding to 50% transmittance at the rising edge of the first transmittance curve is greater than or equal to 400 nm and less than or equal to 440 nm. The difference between the third wavelength corresponding to 50% transmittance at the rising edge of the first transmittance curve and the fourth wavelength corresponding to 50% transmittance at the rising edge of the second transmittance curve is greater than or equal to 0 and less than or equal to 20 nm.

[0180] When incident at an oblique angle, the third wavelength corresponding to the filter at a transmittance of 50% is in the range of 400nm-440nm, which is roughly equivalent to the corresponding wavelength range of the filter in the three-channel camera module. This can limit the overall spectral response range, making the spectral response curve of the first camera module highly similar to the spectral response curves of the human eye and the three-channel camera module, respectively. This is beneficial to improving the color reproduction capability of the first camera module and the accuracy of color correction for the three-channel camera module, reducing or avoiding color cast problems, and improving the color authenticity of the images from the three-channel camera module.

[0181] The difference between the third and fourth wavelengths is in the range of 0-20nm, which can limit the angular offset of the transmittance curve of the filter, improve the color reproduction consistency between the image center and the image edge, and reduce the color difference between the image center and the image edge.

[0182] It should be noted that the "difference between the third wavelength and the fourth wavelength" mentioned in the embodiments of this application refers to the numerical difference between the third wavelength and the fourth wavelength. This difference is a scalar and can be expressed as the absolute value of the value obtained by subtracting the fourth wavelength from the third wavelength, or as the absolute value of the value obtained by subtracting the third wavelength from the fourth wavelength.

[0183] In this embodiment, the incident angle refers to the angle between the light rays incident on the filter surface and the normal to the filter surface. 50% transmittance refers to the position on the transmittance curve where the transmittance is 50%. The rising edge is a segment on the transmittance curve where the transmittance generally increases with increasing wavelength, for example, the segment where the transmittance increases from the lowest value to the highest value with increasing wavelength, or the segment where the transmittance increases from 5% to 80%. The falling edge is a segment on the transmittance curve where the transmittance generally decreases with increasing wavelength, for example, the segment where the transmittance decreases from the highest value to the lowest value with increasing wavelength, or the segment where the transmittance decreases from 80% to 5%. The wavelength band corresponding to the rising edge overlaps with the blue light band (approximately 435nm to 450nm), and the wavelength band corresponding to the falling edge overlaps with the red light band (approximately 622nm to 760nm).

[0184] In some embodiments, the first wavelength and the second wavelength belong to the red light band.

[0185] In some embodiments, the third and fourth wavelengths belong to the blue light band.

[0186] In some embodiments, the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 10 nm. That is, the difference between the wavelength corresponding to 50% transmittance at the falling edge of the first transmittance curve and the wavelength corresponding to 50% transmittance at the falling edge of the second transmittance curve is greater than or equal to 0 and less than or equal to 10 nm.

[0187] In some embodiments, the difference between the third wavelength and the fourth wavelength is greater than or equal to 0 and less than or equal to 10 nm. That is, the difference between the wavelength corresponding to 50% transmittance at the rising edge of the first transmittance curve and the wavelength corresponding to 50% transmittance at the rising edge of the second transmittance curve is greater than or equal to 0 and less than or equal to 10 nm.

[0188] As mentioned earlier, the transmittance curve of the filter shifts when the angle of incident light changes. Limiting the difference between the first and second wavelengths to within 10nm ensures that even when the incident angle varies within the first angle range (e.g., 35°), the transmittance curve of the filter maintains a high degree of similarity to the spectral response curves of the multispectral camera module and the human eye and three-channel camera modules, further improving the color reproduction consistency at the image center and edges. Furthermore, limiting this difference range also reduces color shading in the module, further enhancing the accuracy of overall image color reproduction and color correction.

[0189] In some embodiments, on the first transmittance curve, the average transmittance in the 350nm to 395nm band is less than or equal to 3%, and the average transmittance in the 700nm to 780nm band is less than or equal to 2%.

[0190] In some embodiments, on the second transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 4%, and the average transmittance in the 700nm–780nm band is less than or equal to 3%.

[0191] Color reproduction in multispectral camera modules is based on the human eye. The wavelength range perceived by the human eye is generally between 400nm and 700nm. Light wavelengths outside this range are meaningless signals to multispectral camera modules. In this embodiment, the cutoff wavelength of the filter corresponds to the cutoff wavelength position of the human eye's visual response curve. Filtering out light waves outside the human eye's perception band allows for a better signal-to-noise ratio of the effective information, thus improving image quality.

[0192] In some embodiments, on the first transmittance curve, the minimum transmittance in the 440nm-580nm band is greater than or equal to 70%, and the average transmittance in the 440nm-580nm band is greater than or equal to 80%.

[0193] In some embodiments, on the second transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%.

[0194] The 440nm–580nm range represents a stable transmittance range for filters. By constraining the minimum transmittance within this range, significant fluctuations in filter transmittance can be prevented. Large transmittance fluctuations in the stable range can lead to inconsistent transmittance across different filters in actual production, affecting the consistency of the multispectral camera module's output and consequently, the consistency of color reproduction. Constraining the average transmittance value within the 440nm–580nm range improves image quality. Excessively low transmittance affects the overall sensitivity of the multispectral camera module, requiring longer exposure times under the same conditions, potentially resulting in image blurring and motion blur, thus impacting image quality.

[0195] In some embodiments, the filter material can be resin or glass. The filters described in the foregoing embodiments of this application can be obtained by adjusting the filter thickness, the type and / or thickness of the coating, or by adding organic materials with specific spectral absorption characteristics to the filter. This application does not limit the specific implementation of the filter.

[0196] For further understanding, Figures 4 to 10, and Figures 12 and 13 respectively show schematic diagrams of transmittance curves of several filters provided in the embodiments of this application.

[0197] Referring to Figures 4 to 10, and Figures 12 and 13, λ1 represents the first wavelength corresponding to 50% transmittance at the falling edge of the first transmittance curve, and λ3 represents the third wavelength corresponding to 50% transmittance at the rising edge of the first transmittance curve. λ2 represents the second wavelength corresponding to 50% transmittance at the falling edge of the second transmittance curve, and λ4 represents the fourth wavelength corresponding to 50% transmittance at the rising edge of the second transmittance curve.

[0198] For example, referring to Figures 4 to 10, as well as Figures 12 and 13, the approximate values ​​of the corresponding parameters are shown in Table 1 below.

[0199] Table 1

[0200] It is understandable that on the first transmittance curve, if the falling edge fluctuates around 50% transmittance, the wavelength corresponding to 50% transmittance may include multiple wavelengths in the range of 600nm to 660nm. The first wavelength λ1 can be any one of these multiple wavelengths in the range of 600nm to 660nm, for example, the shortest / longest / approximately middle wavelength among these multiple wavelengths in the range of 600nm to 660nm. On the first transmittance curve, if the rising edge fluctuates around 50% transmittance, the wavelength corresponding to 50% transmittance may include multiple wavelengths in the range of 400nm to 440nm. The third wavelength λ3 can be any one of these multiple wavelengths in the range of 400nm to 440nm, for example, the shortest / longest / approximately middle wavelength among these multiple wavelengths in the range of 400nm to 440nm.

[0201] Similarly, on the second transmittance curve, if the falling edge fluctuates around 50% transmittance, the wavelength corresponding to 50% transmittance can include multiple wavelengths whose difference from the first wavelength λ1 meets the requirement. The second wavelength λ2 can be any one of these wavelengths whose difference from the first wavelength λ1 meets the requirement, for example, the one furthest / closest to the first wavelength λ1. On the second transmittance curve, if the rising edge fluctuates around 50% transmittance, the wavelength corresponding to 50% transmittance can include multiple wavelengths whose difference from the third wavelength λ3 meets the requirement. The fourth wavelength λ4 can be any one of these wavelengths whose difference from the third wavelength λ3 meets the requirement, for example, the one furthest / closest to the third wavelength λ3.

[0202] Verification has shown that the filters shown in Figures 4 to 10, as well as Figures 12 and 13, can make the spectral response curve of the multispectral camera module highly similar to that of the human eye and the spectral response curve of the three-channel camera module.

[0203] The effects of the filter provided in this application can be verified through experimental measurement, simulation and other methods.

[0204] For example, in an experimental measurement-based approach, the spectral response curve of the multispectral camera module (including the filter provided in this application) (hereinafter referred to as the first spectral response curve for ease of description) and the spectral response curve of the three-channel camera module (hereinafter referred to as the second spectral response curve for ease of description) can be measured experimentally. The spectral response curve of the human eye (hereinafter referred to as the third spectral response curve for ease of description) can be obtained using existing spectral tristimulus value curves. Based on the transformation relationship between the first and third spectral response curves, the color information of the target object captured by the multispectral camera module can be obtained, and the human eye can judge the color reproduction capability of the multispectral camera module. Based on the transformation relationship between the first and second spectral response curves, the color space information corresponding to the multispectral camera module can be mapped onto the three-channel camera module, and the human eye can judge whether the image presented by the three-channel camera module has color cast issues, which can be used to determine the accuracy of color correction performed by the multispectral camera module.

[0205] For example, in a simulation-based approach, a first spectral response curve can be obtained through simulation, and a second spectral response curve can be obtained through simulation or experimentation. The third spectral response curve can be the spectral tristimulus value curve. Computer simulation can be used to calculate the dimensionality reduction fitting accuracy between the first and second spectral response curves, as well as the dimensionality reduction fitting accuracy between the first and third spectral response curves. Based on the dimensionality reduction fitting accuracy between the first and third spectral response curves, the color reproduction capability of the multispectral camera module can be determined; based on the dimensionality reduction fitting accuracy between the first and second spectral response curves, the accuracy of the multispectral camera module used for color correction in a three-channel camera module can be determined.

[0206] To facilitate understanding, the effect of the filter provided in this application can also be verified through formula derivation, which will be described in detail below.

[0207] In some embodiments, the Vora-Value can be used to describe the equivalence (or similarity) of the spectral response curves of two camera modules (the human eye can be considered a special type of three-channel camera module). The Vora-Value is a number between 0 and 1. When the Vora-Value is 1, it means that the spectral response curves of the two camera modules are completely equivalent, and their spectral sensing functions (or spectral response matrices) can be linearly transformed. For ease of quantification and expression, the Vora-Value is defined as v below. The superscript or subscript of v varies depending on the comparison object.

[0208] In some embodiments, the similarity between the spectral response curve of the multispectral camera module and the spectral response curve of the human eye is expressed as v.0 Specifically, it can be expressed as:

[0209] In formula (1):

[0210] Q N×C This is the spectral response matrix of the multispectral camera module, which is obtained based on the spectral response curve of the multispectral camera module;

[0211] It is the spectral response matrix of the human eye, which is obtained based on the spectral response curve of the human eye;

[0212] P{} is an operator; for any matrix A, P{A} = A[A T A] -1 A T ;

[0213] P{Q N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T Its order is N×N;

[0214] Its order is N×N;

[0215] Tr() is the trace operation of a square matrix, which is the sum of all elements on the main diagonal of the square matrix;

[0216] N equals the number of sampling points, and C equals the number of channels in the multispectral camera module.

[0217] Multispectral camera modules have more channels, enabling them to reduce the spectral response curve from high-dimensional to near that of the human eye. When the spectral response curve of a multispectral camera module is appropriate, it can make v 0 The color accuracy is as close to 1 as possible. Consequently, during the color reproduction process, the conversion accuracy between the multispectral camera module and the human eye is increasingly higher. This means that the multispectral camera module has better color reproduction capabilities and can reduce or avoid color cast problems.

[0218] When v 0 When = 1, it indicates that the spectral response curve of the multispectral camera module is completely equivalent to the spectral response curve of the human eye. The color perception capability of the multispectral camera module is completely adapted to the color perception capability of the human eye. The color space can be directly converted through linear transformation (such as matrix addition and multiplication). Accordingly, the multispectral camera module can accurately reproduce the colors seen by the human eye without color distortion.

[0219] For existing second camera modules, due to the limitations of the film material, their spectral response curve cannot be exactly the same as the spectral response curve of the human eye. Therefore, the similarity between the spectral response curve of existing second camera modules and the spectral response curve of the human eye is usually less than 0.9. This means that the color reproduction capability of existing second camera modules is weak and they are prone to color cast.

[0220] In this embodiment of the application, v can be calculated using the above formula (1). 0 via v 0 The value of v can determine the color reproduction capability of a multispectral camera module. For example, if v 0 A value greater than or equal to a certain threshold (e.g., 0.93) indicates that the spectral response curve of the multispectral camera module, after being reduced from high dimension, can be as close as possible to the spectral response curve of the human eye, which means that the color reproduction capability of the multispectral camera module is high.

[0221] In some embodiments, the second camera module is an M-channel camera module, where M is greater than or equal to 3. The similarity between the spectral response curve of the multispectral camera module and the spectral response curve of the M-channel camera module is represented by v′, which can be specifically:

[0222] In formula (2-1):

[0223] Q N×C This is the spectral response matrix of the multispectral camera module, which is obtained based on the spectral response curve of the multispectral camera module;

[0224] ′

[0225] X N×M This is the spectral response matrix of the M-channel camera module, which is obtained based on the spectral response curve of the M-channel camera module;

[0226] P{} is an operator; for any matrix A, P{A} = A[A T A] -1 A T ;

[0227] P{Q N×C}=Q N×C [Q N×C T Q TN×C ] -1 Q N×C T , T Its order is N×N;

[0228] P{X′ N×M}=X′ N×M[X′ N×M X′ N×M ] -1 X′ N×M Its order is N×N;

[0229] Tr() is the trace operation of a square matrix, which is the sum of all elements on the main diagonal of the square matrix;

[0230] N equals the number of sampling points, and C equals the number of channels in the multispectral camera module.

[0231] For example, if a camera device or electronic device includes i M-channel camera modules, each M-channel camera module has its own spectral response curve and spectral response matrix. To distinguish the spectral response matrices of different M-channel camera modules, for i = 1, 2, 3, ..., n, where n is an integer greater than or equal to 1, the spectral response matrices of the M-channel camera modules can be expressed as follows: The different values ​​of 'i' represent different M-channel camera modules. For example, in a triple-camera scenario, 'i' = 1, 2, 3 can represent the wide-angle camera module, the telephoto camera module, and the ultra-wide-angle camera module, respectively, and the corresponding spectral response matrices are respectively... Similarly, the similarity v′ between the spectral response curve of the multispectral camera module and the spectral response curve of each M-channel camera module can be expressed as v′. 1 v 2 v 3 , ..., v n In the calculation, X′ in formula (2-1) N×M The corresponding replacements are as follows:

[0232] When the spectral response curve of the multispectral camera module is appropriate, v′ can be made as close to 1 as possible. Accordingly, during the color correction process, the multispectral camera module can be mapped more accurately to the M-channel camera module, enabling the M-channel camera module to accurately acquire true color information and avoid color cast issues.

[0233] When v′=1, it indicates that the spectral response curve of the multispectral camera module is completely equivalent to that of the M-channel camera module. The color perception capability of the multispectral camera module is completely compatible with that of the M-channel camera module. The color space can be directly converted through linear transformation. Accordingly, the M-channel camera module can accurately acquire the color of the multispectral camera module. When the color of the multispectral camera module is accurate, the M-channel camera module will not have color cast.

[0234] In this embodiment, v′ can be calculated using the above formula (2-1), and the value of v′ can be used to determine the color correction capability of the multispectral camera module. For example, if v′ is greater than or equal to a certain threshold (e.g., 0.93), it indicates that the spectral response curve of the multispectral camera module is as close as possible to the spectral response curve of the M-channel camera module, and it also indicates that the multispectral camera module has a high accuracy in color correction of the M-channel camera module.

[0235] For Q in formulas (1) and (2-1) N×C It can be obtained in the following ways.

[0236] In one example, the spectral response curve of the multispectral camera module can be obtained first. This spectral response curve is obtained by directly measuring or simulating the entire module, and includes the spectral response curves of C channels. The spectral response curve of each channel takes into account the influence of the optical lens, filter, and image sensor. Then, based on the spectral response curve of the multispectral camera module, N points are sampled from the spectral response curve of each of the C channels to obtain Q. N×C .

[0237] In another example, instead of obtaining the spectral response curve of the multispectral camera module, the response value of each channel to light of each wavelength can be obtained directly by measuring or simulating the entire module using discrete wavelengths of light, thereby obtaining the Q value. N×C .

[0238] In yet another example, Q can be obtained based on the following formula (3). N×C Q N×C =M N×C ⊙T N×1 ⊙L N×1 (3)

[0239] In formula (3):

[0240] M N×C This is a matrix obtained by sampling N points in each of the C channels of the spectral response curve based on the image sensor.

[0241] T N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the filter;

[0242] L N×1 This is a matrix obtained by sampling N points based on the transmittance curve of an optical lens;

[0243] ⊙ represents the dot product.

[0244] For X′ in formula (2-1) N×M It can be obtained in the following ways.

[0245] In one example, the spectral response curve of an M-channel camera module can be obtained first. This curve is obtained through direct measurement or simulation of the entire module and includes the spectral response curves of M channels. Each channel's spectral response curve takes into account the influence of the optical lens, filters, and image sensor within the module. Then, based on the spectral response curve of this M-channel camera module, N points are sampled from the spectral response curve of each of the M channels to obtain X′. N×M .

[0246] In another example, instead of obtaining the spectral response curve of the M-channel camera module, the response value of each channel to light of each wavelength can be obtained directly by measuring or simulating the entire module using discrete wavelengths of light, thus obtaining X′. N×M .

[0247] In yet another example, similar to formula (3), X′ N×M It can be obtained by dot multiplication of the transmittance matrix of the optical lens in the M-channel camera module (obtained by sampling N points on the transmittance curve of the optical lens), the transmittance matrix of the filter in the M-channel camera module (obtained by sampling N points on the transmittance curve of the filter), and the spectral response matrix of the image sensor in the M-channel camera module (obtained by sampling N points on each channel of the spectral response curve of the image sensor). The transmittance curves of the optical lens, the filter, and the spectral response curve of the image sensor in the M-channel camera module are measured separately.

[0248] It can be understood that the spectral response curve of the image sensor in an M-channel camera module includes the spectral response curve corresponding to each of the M channels; in other words, its spectral response curve is a set of the spectral response curves of the M channels. Furthermore, when discretely sampling the corresponding curves, the N sampling points selected are the same, meaning the sampling positions are identical.

[0249] The human eye can be equated to a special three-channel camera module. By discretely sampling the spectral response curve corresponding to the human eye, the spectral response matrix of the human eye can be obtained. Its order is N×3 (i.e., N rows and 3 columns). It can also be represented as X 0 . Specifically, This can be achieved by sampling N points in each of the three channels of the human eye's spectral response curve. In other words, the human eye's spectral response curve includes the spectral response curves corresponding to each of the three channels. This was obtained by sampling N points on each of the three curves.

[0250] It should be noted that the sampling locations selected when sampling the spectral response curves of the M-channel camera module and the human eye are the same as those selected when sampling the multispectral camera module. For example, within the same wavelength range, a sampling point is used every 5 nanometers.

[0251] In some embodiments, the M-channel camera module is a three-channel camera module. For existing three-channel camera modules, due to the limitations of the film material, their spectral response curve cannot be exactly the same as the spectral response curve of the human eye. Therefore, the similarity between the spectral response curve of existing three-channel camera modules and the spectral response curve of the human eye is usually less than 0.9. This means that the color reproduction capability of existing three-channel camera modules is weak and they are prone to color casts.

[0252] In this embodiment of the application, v can be calculated using the above formula (1). 0 via v 0 The value of v can determine the color reproduction capability of a multispectral camera module. For example, if v 0 A value greater than or equal to a certain threshold (e.g., 0.93) indicates that the spectral response curve of the multispectral camera module, after being reduced from high dimension, can be as close as possible to the spectral response curve of the human eye, which means that the color reproduction capability of the multispectral camera module is high.

[0253] In some embodiments, the similarity between the spectral response curve of the multispectral camera module and the spectral response curve of the three-channel camera module is represented by v′, which can specifically be:

[0254] In formula (2-2):

[0255] Q N×C This is the spectral response matrix of the multispectral camera module, which is obtained based on the spectral response curve of the multispectral camera module;

[0256] X′ N×3 This is the spectral response matrix of the three-channel camera module, which is obtained based on the spectral response curves of the three-channel camera module.

[0257] P{} is an operator; for any matrix A, P{A} = A[A T A] -1 A T ;

[0258] P{Q N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T Its order is N×N;

[0259] P{X′ N×3}=X′ N×3 [X′ N×3 T X′ N×3 ] -1 X′ N×3 T Its order is N×N;

[0260] Tr() is the trace operation of a square matrix, which is the sum of all elements on the main diagonal of the square matrix;

[0261] N equals the number of sampling points, and C equals the number of channels in the multispectral camera module.

[0262] For example, if a camera device or electronic device includes i three-channel camera modules, each three-channel camera module has its own spectral response curve and spectral response matrix. To distinguish the spectral response matrices of different three-channel camera modules, for i = 1, 2, 3, ..., n, where n is an integer greater than or equal to 1, the spectral response matrices of the three-channel camera modules can be expressed as follows: Different values ​​of 'i' represent different three-channel camera modules. For example, in a three-camera scenario, 'i' = 1, 2, 3 can represent a wide-angle camera module, a telephoto camera module, and an ultra-wide-angle camera module, respectively, and the corresponding spectral response matrices are respectively... Similarly, the similarity v′ between the spectral response curve of the multispectral camera module and the spectral response curve of each three-channel camera module can be expressed as v′. 1 v 2 v 3 , ..., v n In the calculation, X′ in formula (2-2) N×3 The corresponding replacements are as follows:

[0263] Multispectral camera modules have more channels, enabling them to reduce the spectral response curve from a high dimension to approximate that of a three-channel camera module. When the spectral response curve of a multispectral camera module is appropriate, v′ can be made as close to 1 as possible. Consequently, during color correction, the multispectral camera module can more accurately map from a high dimension to a three-channel camera module, allowing the three-channel camera module to accurately acquire true color information and avoid color cast issues.

[0264] When v′=1, it indicates that the spectral response curve of the multispectral camera module is completely equivalent to that of the three-channel camera module. The color perception capability of the multispectral camera module is completely compatible with that of the three-channel camera module. The color space can be directly converted through linear transformation. Correspondingly, the three-channel camera module can accurately acquire the color of the multispectral camera module. When the color of the multispectral camera module is accurate, the three-channel camera module will not have color cast.

[0265] In this embodiment, v′ can be calculated using the above formula (2-2), and the value of v′ can be used to determine the color correction capability of the multispectral camera module. For example, if v′ is greater than or equal to a certain threshold (e.g., 0.93), it indicates that the spectral response curve of the multispectral camera module, after dimensionality reduction from high dimension, can be as close as possible to the spectral response curve of the three-channel camera module, and it also indicates that the multispectral camera module has high accuracy in color correction of the three-channel camera module.

[0266] For Q in formulas (1) and (2-2) N×C It can be obtained in the following ways.

[0267] In one example, the spectral response curve of the multispectral camera module can be obtained first. This spectral response curve is obtained by directly measuring or simulating the entire module, and includes the spectral response curves of C channels. The spectral response curve of each channel takes into account the influence of the optical lens, filter, and image sensor. Then, based on the spectral response curve of the multispectral camera module, N points are sampled from the spectral response curve of each of the C channels to obtain Q. N×C .

[0268] In another example, instead of obtaining the spectral response curve of the multispectral camera module, the response value of each channel to light of each wavelength can be obtained directly by measuring or simulating the entire module using discrete wavelengths of light, thereby obtaining the Q value. N×C .

[0269] In yet another example, Q can be obtained based on the following formula (3). N×C Q N×C=M N×C ⊙T N×1 ⊙L N×1 (3)

[0270] In formula (3):

[0271] M N×C This is a matrix obtained by sampling N points in each of the C channels of the spectral response curve based on the image sensor.

[0272] T N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the filter;

[0273] L N×1 This is a matrix obtained by sampling N points based on the transmittance curve of an optical lens;

[0274] ⊙ represents the dot product.

[0275] In this embodiment, the multispectral camera module includes an optical lens, a filter, and an image sensor. The optical lens focuses light from a target object onto the image sensor, and the filter, located between the optical lens and the image sensor, filters the light. The image sensor is a multispectral image sensor with multiple response channels (i.e., multiple filter channels, or simply channels), capable of responding to light within a continuous wavelength range. Each channel yields a corresponding spectral response curve, and the set of spectral response curves from all channels constitutes the spectral response curve corresponding to the image sensor. Both the optical lens and the filter are optical elements that affect the spectrum reaching the image sensor. Therefore, the spectral response characteristics of the multispectral camera module need to comprehensively consider the transmittance curves of the optical lens, the transmittance curves of the filter, and the spectral response curves of the image sensor.

[0276] In this embodiment, the transmittance curve of the optical lens is a curve showing the relationship between the transmittance of the optical lens and wavelength; the transmittance curve of the filter is a curve showing the relationship between the transmittance of the filter and wavelength; and the spectral response curve of the image sensor is a curve showing the relationship between the spectral response of the image sensor and wavelength. The transmittance curves of the optical lens, the filter, and the image sensor are obtained through separate measurements. The spectral response curve of the multispectral camera module can be indirectly obtained through the transmittance curves of the optical lens, the filter, and the image sensor.

[0277] It can be understood that the spectral response curve of an image sensor includes the spectral response curve corresponding to each of the C channels; in other words, the spectral response curve of an image sensor is a set of the spectral response curves of the C channels.

[0278] In some embodiments, by performing N-point discrete sampling on the transmittance curve of the optical lens, the transmittance matrix L of the optical lens can be obtained. Its order is N×1, that is, N rows and 1 column, which is also L in formula (3). N×1 By performing N-point discrete sampling on the transmittance matrix of the filter, the transmittance matrix T of the filter can be obtained, which has an order of N×1, i.e., T in formula (3). N×1 By performing discrete sampling of the spectral response curves of each of the C channels of the image sensor at N points, the spectral response matrix M of the image sensor can be obtained. Its order is N×C (i.e., N rows and C columns), which is also M in formula (3). N×C .

[0279] Here, the N sampling points selected when discretely sampling the transmittance curve of the filter, the N sampling points selected when discretely sampling the transmittance curve of the optical lens, and the N sampling points selected when discretely sampling each channel of the image sensor are the same; that is, the sampling positions are the same. For example, within a preset wavelength range, every 5 nanometers is a sampling point.

[0280] For X′ in formula (2-2) N×3 It can be obtained in the following ways.

[0281] In one example, the spectral response curve of a three-channel camera module can be obtained first. This spectral response curve is obtained through direct measurement or simulation of the entire module, and includes the spectral response curves of three channels. The spectral response curve of each channel takes into account the influence of the optical lens, filter, and image sensor in the module. Then, based on the spectral response curve of this three-channel camera module, N points are sampled from the spectral response curve of each of the three channels to obtain X′. N×3 .

[0282] In another example, instead of obtaining the spectral response curve of the three-channel camera module, the response value of each channel to light of each wavelength can be obtained directly by measuring or simulating the entire module using discrete wavelengths of light, thus obtaining X′. N×3 .

[0283] In yet another example, similar to formula (3), X′ N×3It can be obtained by dot multiplication of the transmittance matrix of the optical lens in the three-channel camera module (obtained by sampling at N points on the transmittance curve of the optical lens), the transmittance matrix of the filter in the three-channel camera module (obtained by sampling at N points on the transmittance curve of the filter), and the spectral response matrix of the image sensor in the three-channel camera module (obtained by sampling at N points on each channel of the spectral response curve of the image sensor). The transmittance curves of the optical lens, the filter, and the spectral response curve of the image sensor in the three-channel camera module are measured separately.

[0284] It can be understood that the spectral response curve of the image sensor in a three-channel camera module includes the spectral response curve corresponding to each of the three channels; in other words, its spectral response curve is a set of the spectral response curves of the three channels. Furthermore, when discretely sampling the corresponding curves, the N sampling points selected are the same, meaning the sampling positions are identical.

[0285] The human eye can be equated to a special three-channel camera module. By discretely sampling the spectral response curve corresponding to the human eye, the spectral response matrix of the human eye can be obtained. Its order is N×3 (i.e., N rows and 3 columns). It can also be represented as X 0 . Specifically, This can be achieved by sampling N points in each of the three channels of the human eye's spectral response curve. In other words, the human eye's spectral response curve includes the spectral response curves corresponding to each of the three channels. This was obtained by sampling N points on each of the three curves.

[0286] It should be noted that the sampling positions selected when sampling the spectral response curves of the three-channel camera module and the spectral response curve of the human eye are the same as those selected when sampling the multispectral camera module. For example, within the same wavelength range, a sampling point is used every 5 nanometers.

[0287] Referring again to Figures 4 to 9, the similarity between the spectral response curve of the multispectral camera module and the spectral response curve of the human eye and the three-channel camera module is calculated according to formulas (1) and (2-2) as follows.

[0288] The transmittance curves of the filters shown in Figures 4 to 9 all include transmittance curves at AOI of 0° and AOI of 35°. Taking i = 1, 2, and 3 in formula (2-2) as wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module respectively as examples, the corresponding spectral response matrices are respectively... The similarity calculated according to formula (2-2) can be expressed as v 1 v 2 v 3 The spectral response matrix of the human eye is as follows: The similarity calculated according to formula (1) can be expressed as v 0 .

[0289] As verified, in the embodiment where M equals 3, the transmittance curve of the filter (made of glass) shown in Figure 4 allows the Vora values ​​between the multispectral camera module and the human eye, wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module to be as follows:

[0290] When AOI = 0°, v 0 =0.9933; v 1 =0.9935; v 2 =0.9960; v 3 =0.9926.

[0291] When AOI = 35°, v 0 =0.9835; v 1 =0.9950; v 2 =0.9891; v 3 =0.9882.

[0292] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.98.

[0293] Verification showed that the transmittance curves of the filters shown in Figure 5 allow for the following Vora values ​​between the multispectral camera module and the human eye, wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module:

[0294] When AOI = 0°, v 0 =0.9799; v 1 =0.9958; v 2 =0.9866; v 3 =0.9893.

[0295] When AOI = 35°, v 0 =0.9743; v 1 =0.9933; v 2 =0.9809; v 3 =0.9774.

[0296] It can be seen that within the incident angle range of the filter, v0 v 1 v 2 v 3 All are greater than 0.97.

[0297] Verification showed that the transmittance curves of the filters shown in Figure 6 allow the following Vora values ​​between the multispectral camera module and the human eye, wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module:

[0298] When AOI = 0°, v 0 =0.9786; v 1 =0.9911; v 2 =0.9904; v 3 =0.9939.

[0299] When AOI = 35°, v 0 =0.9697; v 1 =0.9941; v 2 =0.9798; v 3 =0.9839.

[0300] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.96.

[0301] Verification showed that the transmittance curves of the filter (made of resin) shown in Figure 7 allow the following Vora values ​​between the multispectral camera module and the human eye, wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module:

[0302] When AOI = 0°, v 0 =0.9841; v 1 =0.9865; v 2 =0.9942; v 3 =0.9899.

[0303] When AOI = 35°, v 0 =0.9768; v 1 =0.9952; v 2 =0.9845; v 3 =0.9880.

[0304] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.97.

[0305] Verification showed that the transmittance curves of the filters shown in Figure 8 allow for the following Vora values ​​between the multispectral camera module and the human eye, wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module:

[0306] When AOI = 0°, v 0 =0.9824; v 1 =0.9872; v 2 =0.9933; v 3 =0.9907.

[0307] When AOI = 35°, v 0 =0.9744; v 1 =0.9947; v 2 =0.9832; v 3 =0.9828.

[0308] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.97.

[0309] Verification showed that the transmittance curves of the filters shown in Figure 9 allow for the following Vora values ​​between the multispectral camera module and the human eye, wide-angle camera module, telephoto camera module, and ultra-wide-angle camera module:

[0310] When AOI = 0°, v 0 =0.9799; v 1 =0.9958; v 2 =0.9866; v 3 =0.9893.

[0311] When AOI = 35°, v 0 =0.9727; v 1 =0.9892; v 2 =0.9768; v 3 =0.9690.

[0312] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.96.

[0313] According to v 0The value indicates that the spectral response curve of the multispectral camera module, after being reduced from high dimension, can be as close as possible to the spectral response curve of the human eye. Therefore, it can improve the color reproduction capability of the multispectral camera module, reduce or avoid color cast problems, and thus improve the color authenticity of the output image of the multispectral camera module.

[0314] According to v 1 v 2 v 3 The value indicates that the spectral response curve of the multispectral camera module, after being reduced from high dimension, can be as close as possible to the spectral response curve of the three-channel camera module. Therefore, it can improve the accuracy of color correction of the three-channel camera module by the multispectral camera module, reduce or avoid color cast problems, and improve the color authenticity of the output image of the three-channel camera module.

[0315] Existing three-channel camera modules struggle to achieve consistent spectral response curves, resulting in color differences in images output from different modules for the same scene, leading to poor color consistency and impacting the switching experience. In this application, different three-channel camera modules can all correspond to a good Vora-Value, meaning that the spectral response curves of the multispectral camera modules, after dimensionality reduction from high-dimensionality, can be as close as possible to the spectral response curve of each three-channel camera module. Thus, when the color information of the multispectral camera modules is uniformly mapped onto different three-channel camera modules, it ensures that the different three-channel camera modules conform to the same standard, i.e., they all obtain color information from the multispectral camera modules, improving the color consistency of the output images from multiple three-channel camera modules.

[0316] Furthermore, within the incident angle range of the filter, even if the transmittance curve of the filter drifts, a high Vora value can still be maintained. This reduces the deviation between the spectral response curve at the center and the spectral response curve at the edge of the multispectral camera module, ensuring the accuracy of color reproduction throughout the image.

[0317] Figure 10 is a transmittance curve of the filter in a first camera module of an exemplary embodiment of this application. The first camera module is a multi-channel multispectral camera module. Figure 11 is a transmittance curve of the filter in a second camera module of an exemplary embodiment of this application. The second camera module uses a four-channel image sensor, such as an RGBY image sensor or an RGBW image sensor.

[0318] It has been verified that the transmittance curve of the filter shown in Figure 10 allows the multispectral camera module to transmit light to the human eye at different frequencies. 0 Wide-angle camera module v 1 Telephoto camera module v 2 and ultra-wide-angle camera module v3 The Vora value between them is shown below, at which point the second camera module uses an RGBW image sensor.

[0319] When AOI = 0°, v 0 =0.997; v 1 =0.992; v 2 =0.987; v 3 =0.989.

[0320] When AOI = 35°, v 0 =0.997; v 1 =0.983; v 2 =0.972; v 3 =0.982.

[0321] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.96.

[0322] Figure 12 is a transmittance curve of the filter in the first camera module of another exemplary embodiment of this application. The first camera module is a multi-channel multispectral camera module.

[0323] The transmittance curve of the filter shown in Figure 12 allows the multispectral camera module to transmit light to the human eye respectively. 0 Wide-angle camera module v 1 Telephoto camera module v 2 and ultra-wide-angle camera module v 3 The Vora value between them is shown below, at which point the second camera module uses an RGBY image sensor.

[0324] When AOI = 0°, v 0 =0.997; v 1 =0.997; v 2 =0.991; v 3 =0.997.

[0325] When AOI = 35°, v 0 =0.997; v 1 =0.996; v 2 =0.992; v 3 =0.996.

[0326] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3All are greater than 0.96.

[0327] The transmittance curve of the filter shown in Figure 13 allows the multispectral camera module to transmit light to the human eye respectively. 0 Wide-angle camera module v 1 Telephoto camera module v 2 and ultra-wide-angle camera module v 3 The Vora values ​​are shown below, at which point the second camera module uses an RGBY image sensor:

[0328] When AOI = 0°, v 0 =0.997; v 1 =0.997; v 2 =0.991; v 3 =0.997.

[0329] When AOI = 35°, v 0 =0.997; v 1 =0.996; v 2 =0.991; v 3 =0.996.

[0330] It can be seen that within the incident angle range of the filter, v 0 v 1 v 2 v 3 All are greater than 0.96.

[0331] According to v 0 The value indicates that the spectral response curve of the multispectral camera module, after being reduced from high dimension, can be as close as possible to the spectral response curve of the human eye. Therefore, it can improve the color reproduction capability of the multispectral camera module, reduce or avoid color cast problems, and thus improve the color authenticity of the output image of the multispectral camera module.

[0332] According to v 1 v 2 v 3 The value indicates that the spectral response curve of the multispectral camera module, after being reduced from high dimension, can be as close as possible to the spectral response curve of the M-channel camera module. Therefore, it can improve the accuracy of color correction of the M-channel camera module by the multispectral camera module, reduce or avoid color cast problems, and improve the color authenticity of the output image of the M-channel camera module.

[0333] Existing M-channel camera modules struggle to achieve uniform spectral response curves, resulting in color differences between images output by different camera modules for the same scene. This poor color consistency negatively impacts the switching experience. In this application, different M-channel camera modules can all correspond to a good Vora-Value, meaning that the spectral response curves of the multispectral camera modules, after dimensionality reduction from high-dimensionality, can be as close as possible to the spectral response curve of each M-channel camera module. Thus, when the color information of the multispectral camera modules is uniformly mapped onto different M-channel camera modules, it ensures that the different M-channel camera modules conform to the same standard, i.e., they all obtain color information from the multispectral camera modules, thereby improving the color consistency of the output images from multiple M-channel camera modules.

[0334] This application embodiment also provides a camera device, which includes a first camera module. The first camera module includes an optical lens, a filter, and an image sensor arranged sequentially along the optical axis. The optical lens is used to receive light from a target object, the filter allows visible light to pass through, and the image sensor is used to receive visible light to obtain spectral information of the target object. The image sensor has more than 3 channels. The filter is the same as the filter in the aforementioned embodiment.

[0335] In some embodiments, the camera device further includes at least one second camera module, which is an M-channel camera, and the first camera module is a C-channel camera where C is greater than 3. The similarity v between the spectral response curve of the first camera module and the spectral response curve of the human eye is... 0 If the similarity v′ between the spectral response curve of the first camera module and the spectral response curve of the second camera module is greater than or equal to 0.93, then...

[0336] Q N×C Let P{Q} be the spectral response matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the first camera module. N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T ;

[0337] This is the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the human eye.

[0338] X′ N×MLet P{X′ be the spectral response matrix obtained by sampling N points in each of the M channels based on the spectral response curve of the second camera. N×M}=X′ N×M [X′ N×M T X′ N×M ] -1 X′ N×M T ;

[0339] Tr() represents the trace.

[0340] In some embodiments, the camera device further includes at least one second camera module, which is an M-channel camera module, wherein M is greater than or equal to 3. For example, the second camera module is a three-channel camera module, a four-channel camera module, or a camera module with more than one number of channels. The image processing chip is used to correct the color of the target object acquired by the second camera module based on the spectral information of the target object acquired by the multispectral camera module.

[0341] This application embodiment also provides a camera module, which includes an optical lens, an image sensor, and the aforementioned filter. The filter is disposed between the optical lens and the image sensor. The optical lens is used to receive light from a target object, the filter is used to allow visible light in the light to pass through, and the image sensor is used to receive visible light to obtain spectral information of the target object, wherein the number of channels of the image sensor is greater than 3.

[0342] This camera module is a multispectral camera module, also known simply as a multispectral module.

[0343] It is understandable that the number of channels in a camera module is determined based on the number of channels in the image sensor. Specifically, both the optical lens and the filter have one channel, so the number of channels in the camera module is the same as the number of channels in the image sensor. Correspondingly, if the number of channels in the image sensor is greater than 3, the number of channels in the camera module is also greater than 3. For example, if the number of channels is C, then the camera module is a C-channel camera.

[0344] In some embodiments, the filter is coated with an infrared cutoff film.

[0345] This application embodiment also provides a camera device, which includes a first camera module. The first camera module includes an optical lens, a filter, and an image sensor arranged sequentially along the optical axis. The optical lens is used to receive light from a target object, the filter allows visible light to pass through, and the image sensor is used to receive visible light to obtain spectral information of the target object. The image sensor has more than 3 channels. The filter is the same as the filter in the aforementioned embodiment.

[0346] In some embodiments, the camera device further includes at least one second camera module, which is a three-channel camera, and the first camera module is a C-channel camera where C is greater than 3. The similarity v between the spectral response curve of the first camera module and the spectral response curve of the human eye is... 0 If the similarity v′ between the spectral response curve of the first camera module and the spectral response curve of the second camera module is greater than or equal to 0.93, then...

[0347] Q N×C Let P{Q} be the spectral response matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the first camera module. N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T ;

[0348] This is the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the human eye.

[0349] X′ N×3 Let P{X′ be the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the second camera. N×3}=X′ N×3 [X′ N×3 T X′ N×3 ] -1 X′ N×3 T ;

[0350] Tr() represents the trace.

[0351] Regarding v 0 For details on the specific calculation methods of v′ and the related content of each parameter, please refer to the descriptions of formulas (1), (2-1), and (2-2). For the sake of brevity, they will not be elaborated here.

[0352] In some embodiments, Q N×C It can be obtained by measuring the spectral response curve of the entire first camera module and then performing discrete sampling on the spectral response curve of the module.

[0353] In other embodiments, Q N×C =M N×C ⊙T N×1 ⊙LN×1 ,in:

[0354] M N×C This is a matrix obtained by sampling N points in each of the C channels of the spectral response curve based on the image sensor.

[0355] T N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the filter;

[0356] L N×1 This is a matrix obtained by sampling N points based on the transmittance curve of an optical lens.

[0357] In other words, after measuring the spectral response curve of the image sensor, the transmittance curve of the filter, and the transmittance curve of the optical lens respectively, M can be obtained by discrete sampling of each. N×C L N×1 T N×1 Then, Q is obtained by multiplying the three factors. N×C .

[0358] Regarding Q N×C The specific calculation method and related information of each parameter can be found in the description of formula (3). For the sake of brevity, they will not be described in detail here.

[0359] In some embodiments, X′ N×3 X′ can be obtained by measuring the spectral response curve of the entire second camera module and then performing discrete sampling on the spectral response curve of the module. Alternatively, the spectral response curve of the image sensor, the transmittance curve of the filter, and the transmittance curve of the optical lens in the second camera module can be measured separately, and the corresponding matrices can be obtained by discrete sampling of each, and then X′ can be obtained by dot product of the three matrices. N×3 .

[0360] In some embodiments, It can be obtained by discretely sampling the spectral response curve of the human eye.

[0361] In some embodiments, the second camera module can be a telephoto camera module, a wide-angle camera module, or an ultra-wide-angle camera module. If the camera device includes multiple second camera modules, the focal lengths of the multiple second camera modules can be the same or different. For example, the multiple second camera modules can achieve full focal length coverage.

[0362] In this embodiment of the application, the second camera module includes a filter, for example, the filter is used to filter out other wavelengths besides visible light, and correspondingly, the second camera module can output a visible light image.

[0363] This application also provides an electronic device, which includes an image processing chip and the multispectral image head module (i.e., the aforementioned first camera module) involved in the foregoing embodiments. The image processing chip is used to process the images acquired by the multispectral camera module.

[0364] This application also provides an electronic device, which includes an image processing chip and a camera device involved in the foregoing embodiments. The image processing chip is used to process images acquired by the camera device.

[0365] For example, the image processing chip may be the image processor mentioned in the foregoing embodiments.

[0366] In some embodiments, the camera device further includes at least one second camera module, which is a three-channel camera, and the image processing chip is used to correct the color of the target object acquired by the second camera module based on the spectral information of the target object acquired by the multispectral camera module.

[0367] This application also provides a design method for optimizing a camera module. This method is used to optimize the color reproduction capability of a multispectral camera module (hereinafter also referred to as a first camera module), making the images captured by the multispectral camera module closer to the colors seen by the human eye. The design method specifically includes the following steps:

[0368] S401, obtain the initial spectral response matrix of the multispectral camera module and the spectral response matrix of the human eye.

[0369] S402, Based on the optimization objective, optimize the initial spectral response matrix of the multispectral camera module to obtain the target spectral response matrix of the multispectral camera module.

[0370] In some embodiments, the above design method can be used to further optimize the ability of a multispectral camera module to perform image correction for other camera modules.

[0371] Figure 14 shows a schematic flowchart of a camera module design method provided in an embodiment of this application. The number of color channels sensed by the multispectral camera module (hereinafter also referred to as the first camera module) designed by method 400 shown in Figure 14 is greater than or equal to the number of color channels sensed by the second camera module. For example, the second camera module is a three-channel camera module, and the multispectral camera module has more than 3 channels. Method 400 includes steps S410 to S420, which will be described below with reference to the accompanying drawings.

[0372] S410, acquire the initial spectral response matrix of the multispectral camera module, the spectral response matrix of the human eye, and the spectral response matrix of at least one second camera module.

[0373] S420, based on the optimization objective, optimizes the initial spectral response matrix of the multispectral camera module to obtain the target spectral response matrix of the multispectral camera module.

[0374] In this embodiment, the second camera module is an M-channel camera, where M is greater than or equal to 3. For example, an M-channel camera is a three-channel camera.

[0375] In this embodiment of the application, similar to the multispectral camera module, the human eye and the second camera module also have corresponding spectral response curves (or spectral response characteristics). The spectral response curve corresponding to the human eye includes the response curves of the human eye to red, green and blue light, and the spectral response curve corresponding to the second camera module includes its response curves to three different wavelengths of light (such as red, green and blue light).

[0376] This method 400 treats the human eye as a special type of three-channel camera module. The values ​​i = 0, 1, 2, 3, ..., n, where n is an integer greater than or equal to 1. When i is 0, it represents the human eye; when i is 1, 2, 3, ..., n, it represents different second camera modules. Based on the spectral response curves of the M-channel camera module, the spectral response matrix X of the M-channel camera module can be discretely sampled. i Its order is N×M (i.e., N rows and M columns). In some embodiments, X i It can also be expressed as In other words, the spectral response matrix of the human eye is represented as The spectral response matrices of different second camera modules are respectively expressed as follows:

[0377] In the embodiments of this application, There are multiple ways to obtain X′, such as referring to the formula (2-1) above. N×M and This application does not limit the method of obtaining it.

[0378] Based on the above introduction, in step S410, the initial spectral response matrix Q′ of the multispectral camera module can be obtained. N×C Spectral response matrix of the human eye and the second camera module Among them, Q′ N×C Specifically, through the transmittance matrix L of the optical lens N×1 The transmittance matrix T of the filter N×1 The spectral response matrix M of the image sensor N×C It is obtained by dot product.

[0379] For example, the L obtained separately N×1 T N×1 MN×C , The input is directly given to the device executing method 400. For example, L... N×1 M N×C T N×1 or (i>0) can be obtained from the data recorded in the specifications corresponding to the corresponding hardware, or from the manufacturer's test data, or from the data obtained through self-testing, etc. It can be obtained by sampling the tristimulus curve of the human eye.

[0380] For example, the spectral response curve of the human eye, the spectral response curve of the second camera module, the transmittance curve of the optical lens in the multispectral image sensor module, the transmittance curve of the filter, and the spectral response curve of the image sensor can be input into the device executing method 400, and the device can perform discrete sampling to obtain the corresponding... L N×1 T N×1 M N×C And by L N×1 T N×1 M N×C The dot product of the three elements gives Q′. N×C In other words, the device that executes method 400 is used both to sample and obtain the corresponding matrix and to execute the optimization process of method 400.

[0381] In some embodiments, the similarity between the spectral response matrix of the multispectral camera module and the spectral response matrix of the M-channel camera module is expressed as v. i Specifically, it can be expressed as:

[0382] The operator P{} is specifically:

[0383] P{Q N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T Its order is N×N, Q N×C =M N×C ⊙T N×1 ⊙L N×1

[0384] Its order is N×N.

[0385] In formula (4), i = 0, 1, 2, 3, ..., n, where n is an integer greater than or equal to 1. When i is 0, it represents the human eye, and the corresponding spectral response matrix is... When i is 1, 2, 3, ..., n, it represents different second camera modules, and the corresponding spectral response matrices are respectively

[0386] The above v i The value can be greater than or equal to a preset value; for example, the preset value can be greater than or equal to 0.93.

[0387] In this embodiment of the application, in order to enable the multispectral camera module to assist the second camera module in achieving an effect most similar to that of the human eye, v i It should be as close to 1 as possible, for example, v i The value needs to be increased to at least 0.93.

[0388] Based on the above formula (4-1), it can be seen that before optimization, the initial spectral response matrix Q′ of the multispectral camera module is... N×C Substituting into formula (4-1), we can obtain the equivalence between the spectral response curves of the multispectral camera module and those of the human eye and the second camera module, respectively. This can be achieved by adjusting the spectral response curve of the image sensor (corresponding to M...). N×C ), the transmittance curve of the filter (corresponding to T) N×1 ) and the transmittance curve of the optical lens (corresponding to L) N×1 At least one of them can be used for Q′ N×C Optimization can correspondingly improve v i Changes occur. When v i When the value satisfies the optimization objective, the obtained spectral response matrix is ​​the target spectral response matrix Q″ of the multispectral camera module. N×C The target spectral response matrix Q″ N×C Spectral response matrix of human eye and second camera module Compatible.

[0389] Figure 14 shows a schematic flowchart of a camera module design method provided in an embodiment of this application. The multispectral camera module (hereinafter also referred to as the first camera module) designed by method 400 shown in Figure 14 has a greater number of color channels sensed than the three-channel camera module (hereinafter also referred to as the second camera module). For example, the multispectral camera module has more than 3 channels. Method 400 includes steps S410 to S420, which will be described below with reference to the accompanying drawings.

[0390] S410, acquire the initial spectral response matrix of the multispectral camera module, the spectral response matrix of the human eye, and the spectral response matrix of at least one second camera module.

[0391] S420, based on the optimization objective, optimizes the initial spectral response matrix of the multispectral camera module to obtain the target spectral response matrix of the multispectral camera module.

[0392] In this embodiment, the multispectral camera module is a C-channel camera, where C is an integer greater than 3. For example, in the image sensor of the multispectral camera module, each photosensitive unit includes C different filter channels, where C > 3. That is, the image sensor of the multispectral camera module has more than 3 channels.

[0393] In this embodiment, the multispectral camera module includes an optical lens, a filter, and an image sensor. Both the optical lens and the filter are optical elements that affect the spectrum reaching the image sensor. Therefore, the spectral response characteristics of the multispectral camera module need to comprehensively consider the transmittance curves of the optical lens, the filter, and the image sensor.

[0394] The spectral response characteristics of the entire multispectral camera module can be expressed by the spectral response matrix Q. N×C (It can also be represented as Q) specifically, it can be represented as Q N×C =M N×C ⊙T N×1 ⊙L N×1 .

[0395] M N×C The matrix is ​​obtained by sampling N points in each of the C channels of the spectral response curve based on the image sensor; T N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the filter; L N×1 is a matrix obtained by sampling N points based on the transmittance curve of the optical lens; ⊙ represents the dot product.

[0396] It can be understood that the spectral response curve of an image sensor includes the spectral response curve corresponding to each of the C channels; in other words, the spectral response curve of an image sensor is a set of the spectral response curves of the C channels.

[0397] It can be seen that by adjusting the spectral response curve of the image sensor (corresponding to M...) N×C ), the transmittance curve of the filter (corresponding to T) N×1 ) and the transmittance curve of the optical lens (corresponding to L) N×1 At least one of these can adjust the spectral response characteristics (corresponding to Q) of the multispectral camera module. N×C ).

[0398] For example, taking the optimization of the transmittance curve of a filter as an example, before optimization, the filter has an initial transmittance matrix T′. N×1 The multispectral camera module has an initial spectral response matrix Q′N×C , where Q′ N×C =M N×C ⊙T′ N×1 ⊙L N×1 After optimizing the transmittance curve of the filter, the filter has a target transmittance matrix T″. N×1 The multispectral camera module has a target spectral response matrix Q″. N×C , where Q″ N×C =M N×C ⊙T″ N×1 ⊙L N×1 The spectral response matrix of the image sensor and the transmittance matrix of the optical lens remain unchanged.

[0399] In this embodiment, the second camera module is a three-channel camera. This can be understood as follows: each photosensitive unit on the image sensor of the second camera module includes three different filter channels, such as red, green, and blue filter channels or red, yellow, and blue filter channels. Each filter channel corresponds to a spectral response curve. That is, the image sensor of the second camera module has three channels.

[0400] In this embodiment of the application, similar to the multispectral camera module, the human eye and the second camera module also have corresponding spectral response curves (or spectral response characteristics). The spectral response curve corresponding to the human eye includes the response curves of the human eye to red, green and blue light, and the spectral response curve corresponding to the second camera module includes its response curves to three different wavelengths of light (such as red, green and blue light).

[0401] This method 400 treats the human eye as a special type of three-channel camera module. The values ​​i = 0, 1, 2, 3, ..., n, where n is an integer greater than or equal to 1. When i is 0, it represents the human eye; when i is 1, 2, 3, ..., n, it represents different second camera modules. Based on the spectral response curve of the three-channel camera module, the spectral response matrix X of the three-channel camera module can be discretely sampled. i Its order is N×3 (i.e., N rows and 3 columns). In some embodiments, X i It can also be expressed as In other words, the spectral response matrix of the human eye is represented as The spectral response matrices of different second camera modules are respectively expressed as follows:

[0402] In the embodiments of this application, There are multiple ways to obtain X′, such as referring to the formula (2-2) above. N×3 and This application does not limit the method of obtaining it.

[0403] Based on the above introduction, in step S410, the initial spectral response matrix Q′ of the multispectral camera module can be obtained. N×C Spectral response matrix of the human eye and the second camera module Among them, Q′ N×C Specifically, through the transmittance matrix L of the optical lens N×1 The transmittance matrix T of the filter N×1 The spectral response matrix M of the image sensor N×C It is obtained by dot product.

[0404] For example, the L obtained separately N×1 T N×1 M N×C , The input is directly given to the device executing method 400. For example, L... N×1 M N×C T N×1 or (i>0) can be obtained from the data recorded in the specifications corresponding to the corresponding hardware, or from the manufacturer's test data, or from the data obtained through self-testing, etc. It can be obtained by sampling the tristimulus curve of the human eye.

[0405] For example, the spectral response curve of the human eye, the spectral response curve of the second camera module, the transmittance curve of the optical lens in the multispectral image sensor module, the transmittance curve of the filter, and the spectral response curve of the image sensor can be input into the device executing method 400, and the device can perform discrete sampling to obtain the corresponding... L N×1 T N×1 M N×C And by L N×1 T N×1 M N×C The dot product of the three elements gives Q′. N×C In other words, the device that executes method 400 is used both to sample and obtain the corresponding matrix and to execute the optimization process of method 400.

[0406] In this embodiment of the application, the optimization objectives involved in step S420 include:

[0407] The similarity between the spectral response matrix of the multispectral camera module and the spectral response matrix of the human eye is greater than or equal to a first preset threshold.

[0408] The similarity between the spectral response matrix of the multispectral camera module and the spectral response matrix of each second camera module is greater than or equal to a second preset threshold.

[0409] For example, the first preset threshold is greater than or equal to 0.93. For instance, the first preset threshold can be 0.95, 0.96, or 0.97, etc.

[0410] For example, the second preset threshold is greater than or equal to 0.93. For instance, the second preset threshold can be 0.95, 0.96, or 0.97, etc.

[0411] The first preset threshold and the second preset threshold can be the same or different. They can be determined as needed in practical applications, and this application does not limit them.

[0412] It can be understood that the similarity between the spectral response matrix of a multispectral camera module and the spectral response matrix of the human eye is used to characterize the degree of similarity or equivalence between the spectral response curve of the multispectral camera module and the spectral response curve of the human eye. Specifically, it is used to characterize the degree of closeness between the spectral response matrix (or spectral response curve) of the multispectral camera module after dimensionality reduction from high dimension and the spectral response matrix (or spectral response curve) of the human eye.

[0413] It can be understood that the similarity between the spectral response matrix of the multispectral camera module and the spectral response matrix of each second camera module is used to characterize the degree of similarity or equivalence between the spectral response curve of the multispectral camera module and the spectral response curve of the second camera module. Specifically, it is used to characterize the degree of closeness between the spectral response matrix (or spectral response curve) of the multispectral camera module after dimensionality reduction from high dimension and the spectral response matrix (or spectral response curve) of the second camera module.

[0414] In some embodiments, the similarity between the spectral response matrix of the multispectral camera module and the spectral response matrix of the three-channel camera module is expressed as v. i Specifically, it can be expressed as:

[0415] The operator P{} is specifically:

[0416] P{Q N×C}=Q N×C [Q N×C T Q N×C ] -1 Q N×C T Its order is N×N, Q N×C =M N×C ⊙T N×1 ⊙L N×1

[0417] Its order is N×N.

[0418] In formula (4-2), i = 0, 1, 2, 3, ..., n, where n is an integer greater than or equal to 1. When i is 0, it represents the human eye, and the corresponding spectral response matrix is... When i is 1, 2, 3, ..., n, it represents different second camera modules, and the corresponding spectral response matrices are respectively

[0419] The above v i The value can be greater than or equal to a preset value; for example, the preset value can be greater than or equal to 0.93.

[0420] In this embodiment of the application, in order to enable the multispectral camera module to assist the second camera module in achieving an effect most similar to that of the human eye, v i It should be as close to 1 as possible, for example, v i The value needs to be increased to at least 0.93.

[0421] Based on the above formula (4-2), it can be seen that before optimization, the initial spectral response matrix Q′ of the multispectral camera module is... N×C Substituting into formula (4-2), we can obtain the equivalence between the spectral response curves of the multispectral camera module and the spectral response curves of the human eye and the second camera module, respectively. This can be achieved by adjusting the spectral response curve of the image sensor (corresponding to M...). N×C ), the transmittance curve of the filter (corresponding to T) N×1 ) and the transmittance curve of the optical lens (corresponding to L) N×1 At least one of them can be used for Q′ N×C Optimization can correspondingly improve v i Changes occur. When v i When the value satisfies the optimization objective, the obtained spectral response matrix is ​​the target spectral response matrix Q″ of the multispectral camera module. N×C The target spectral response matrix Q″ N×C Spectral response matrix of a three-channel camera module (including the human eye and the second camera module) Compatible.

[0422] For example, taking the optimization of a filter's transmittance curve as an example, by adjusting the filter's transmittance curve, the filter's transmittance matrix can be adjusted. Correspondingly, the spectral response matrix of the multispectral camera module changes, thereby affecting v. i Changes occur. When v i When the value meets the optimization target, for example, when it is greater than or equal to 0.93, the optimization design of the multispectral camera module can be considered complete. The corresponding filter parameter settings can then improve the color reproduction capability and color correction accuracy of the multispectral camera module.

[0423] In some embodiments, for different v i You can also set the corresponding weight F. i For example, v 0 v 1 v 2 v 3 , ..., v n The corresponding weights are F. 0 F 1 F 2 F 3 F n Among them, F i ≥0, The optimization objectives in step S420 also include: The value is greater than or equal to the third preset threshold.

[0424] For example, the third preset threshold is greater than or equal to 0.93. For instance, the third preset threshold can be 0.95, 0.96, or 0.97, etc.

[0425] v is determined by setting weights. i The value can ultimately be used to select a multispectral camera module that meets actual needs (such as the importance of each camera module, color balance strategy, etc.).

[0426] Understandable. Expanded as F 0 / m*v 0 +F 1 / m*v 1 +…+F n / m*v n For example, taking m=1 as an example, the above optimization objective can be concretized as:

[0427] In some embodiments, the weight allocation principle may be determined based on the importance of each second camera module and / or color balance strategy.

[0428] For example, the more important the second camera module, the greater its corresponding weight value. For instance, if a second camera module is the main camera module, the weight corresponding to the v value calculated based on the spectral response matrix of that camera module could be F. 0 F 1 F 2 F 3 F n The maximum value in.

[0429] For example, if the color balance strategy emphasizes the color reproduction capability of the multispectral camera module, then the weight corresponding to the v value calculated from the spectral response matrix of the human eye can be set to the maximum.

[0430] For example, if certain second camera modules are not considered in the color balance strategy, their corresponding weights can be set to 0.

[0431] For example, if the various second camera modules do not have different priorities, then the weights of the various second camera modules can be set to be equal.

[0432] In some embodiments, in step S420, based on the optimization objective, a nonlinear optimization algorithm can be used to optimize the initial spectral response matrix of the multispectral camera module.

[0433] For example, nonlinear optimization algorithms include, but are not limited to: the dividing rectangles (DIRECT) algorithm, the controlled random search (CRS) algorithm, the multi-level single-linkage (MLSL) algorithm, the improved stochastic ranking evolution strategy (ISRES), and the evolutionary algorithm (ESCH).

[0434] As mentioned earlier, in the design of a multispectral camera module, by adjusting T... N×1 M N×C and L N×1 At least one of them can achieve the above optimization objective, thereby obtaining the target spectral response matrix of the multispectral camera module.

[0435] Generally, optical lenses and image sensors involve more design considerations. For example, designing an optical lens requires consideration of depth of field, focal length, field of view, optical resolution, maximum aperture, tolerance accuracy, and aberration control. Similarly, designing an image sensor requires consideration of pixel size, target area size, resolution, and the feasibility of filter material manufacturing processes. Therefore, the design of optical lenses and image sensors should prioritize basic design requirements. This results in limited room for adjustment when optimizing the spectral response curve of a multispectral camera module, while still meeting these basic design requirements. Furthermore, such adjustments are costly and time-consuming.

[0436] The main function of a filter is to filter light to improve color reproduction accuracy and the utilization rate of the effective visible light band. Therefore, by adjusting the parameters of the filter to optimize the spectral response curve of the multispectral camera module, there is a large adjustment range and flexible adjustment methods. Therefore, based on method 400, by adjusting T... N×1It can optimize the spectral response curve of multispectral camera modules.

[0437] In this embodiment of the application, T N×1 The transmittance value is a relative value, only constraining the relative relationship between transmittance at different wavelengths. In practical applications, depending on the actual manufacturing capabilities of the filter, T can be adjusted... N×1 Multiplying by a constant yields the true transmittance matrix. The transmittance matrix of the filter can be converted into a transmittance curve.

[0438] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

A camera device, characterized in that, The system includes a first camera module, which comprises an optical lens, a filter, and an image sensor arranged sequentially along the optical axis. The optical lens is used to receive light from a target object, the filter is used to allow light within the target wavelength range to pass through, and the image sensor is used to receive light within the target wavelength range to obtain spectral information of the target object. The image sensor has more than 3 channels. The filter has a first transmittance curve and a second transmittance curve. The first transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter and the transmittance. The second transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter at a first angle and the transmittance. The first angle is greater than 0° and less than or equal to 35°. On the first transmittance curve, the wavelength corresponding to 50% transmittance includes the first wavelength, which is greater than or equal to 600 nm and less than or equal to 660 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes the second wavelength, and the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20 nm. The camera device according to claim 1 is characterized in that, The target wavelength range is the visible light wavelength range. The camera device according to claim 1 or 2 is characterized in that, On the first transmittance curve, the wavelength corresponding to 50% transmittance includes a third wavelength, which is greater than or equal to 400 nm and less than or equal to 440 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes a fourth wavelength, and the difference between the third wavelength and the fourth wavelength is greater than or equal to 0 and less than or equal to 20 nm. The camera device according to any one of claims 1 to 3 is characterized in that, The difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 10 nm. The camera device according to any one of claims 1 to 4 is characterized in that, On the first transmittance curve, the average transmittance in the 350nm-395nm band is less than or equal to 3%, and the average transmittance in the 700nm-780nm band is less than or equal to 2%. On the second transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 4%, and the average transmittance in the 700nm–780nm band is less than or equal to 3%. The camera device according to any one of claims 1 to 5 is characterized in that, On the first transmittance curve, the minimum transmittance in the 440nm-580nm band is greater than or equal to 70%, and the average transmittance in the 440nm-580nm band is greater than or equal to 80%. On the second transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%. The camera device according to any one of claims 3 to 6, characterized in that, The first wavelength and the second wavelength belong to the red light band, while the third wavelength and the fourth wavelength belong to the blue light band. The camera device according to any one of claims 1 to 7 is characterized in that, The camera device further includes at least one second camera module, wherein the second camera module is an M-channel camera, where M is greater than or equal to 3, and the first camera module is a C-channel camera, where C is greater than 3. The similarity v between the spectral response curve of the first camera module and the spectral response curve of the human eye is... 0 The similarity v′ between the spectral response curves of the first camera module and the second camera module is greater than or equal to 0.

93. Q N×C Let P{Q} be the spectral response matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the first camera module. N×C }=Q N×C [Q N×C T Q N×C ] -1 Q N×C T ; The spectral response matrix is ​​obtained by sampling N points in each of the three channels based on the spectral response curve of the human eye. X′ N×M Let P{X′ be the spectral response matrix obtained by sampling N points in each of the M channels based on the spectral response curve of the second camera. N×M }=X′ N×M [X′ N×M T X′ N×M ] -1 X′ N×M T ; Tr() represents the trace. The camera device according to claim 8 is characterized in that, The number of channels of the image sensor in the first camera module is greater than or equal to the number of channels of the image sensor in the second camera module. The camera device according to any one of claims 1 to 7 is characterized in that, The camera device further includes at least one second camera module, which is a three-channel camera. The first camera module is a C-channel camera where C is greater than 3. The similarity v between the spectral response curve of the first camera module and the spectral response curve of the human eye is... 0 The similarity v′ between the spectral response curves of the first camera module and the second camera module is greater than or equal to 0.

93. Q N×C Let P{Q} be the spectral response matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the first camera module. N×C }=Q N×C [Q N×C T Q N×C ] -1 Q N×C T ; The spectral response matrix is ​​obtained by sampling N points in each of the three channels based on the spectral response curve of the human eye. X′ N×3 Let P{X′ be the spectral response matrix obtained by sampling N points in each of the three channels based on the spectral response curve of the second camera. N×3 }=X′ N×3 [X′ N×3 T X′ N×3 ] -1 X′ N×3 T ; Tr() represents the trace. The camera device according to any one of claims 8 to 10, characterized in that, Spectral response matrix of the first camera module Q N×C =M N×C ⊙T N×1 ⊙L N×1 M N×C This is a matrix obtained by sampling N points in each of the C channels based on the spectral response curve of the image sensor; T N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the filter; L N×1 This is a matrix obtained by sampling N points based on the transmittance curve of the optical lens. A filter, characterized in that, The camera module is used in a camera module, which also includes an optical lens and an image sensor. The filter is disposed between the optical lens and the image sensor. The optical lens is used to receive light from the target object. The filter is used to pass light within the target wavelength range. The image sensor is used to receive light within the target wavelength range to obtain the spectral information of the target object. The number of channels of the image sensor is greater than 3. The filter has a first transmittance curve and a second transmittance curve. The first transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter and the transmittance. The second transmittance curve is a curve showing the relationship between the wavelength of light incident on the filter at a first angle and the transmittance. The first angle is greater than 0° and less than or equal to 35°. On the first transmittance curve, the wavelength corresponding to 50% transmittance includes the first wavelength, which is greater than or equal to 600 nm and less than or equal to 660 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes the second wavelength, and the difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 20 nm. The filter according to claim 12 is characterized in that, The target wavelength range is the visible light wavelength range. The filter according to claim 12 or 13 is characterized in that, On the first transmittance curve, the wavelength corresponding to 50% transmittance includes a third wavelength, which is greater than or equal to 400 nm and less than or equal to 440 nm. On the second transmittance curve, the wavelength corresponding to 50% transmittance includes a fourth wavelength, and the difference between the third wavelength and the fourth wavelength is greater than or equal to 0 and less than or equal to 20 nm. The filter according to claim 14 is characterized in that, The difference between the first wavelength and the second wavelength is greater than or equal to 0 and less than or equal to 10 nm. The filter according to any one of claims 12 to 15 is characterized in that, On the first transmittance curve, the average transmittance in the 350nm-395nm band is less than or equal to 3%, and the average transmittance in the 700nm-780nm band is less than or equal to 2%. On the second transmittance curve, the average transmittance in the 350nm–395nm band is less than or equal to 4%, and the average transmittance in the 700nm–780nm band is less than or equal to 3%. The filter according to any one of claims 12 to 16 is characterized in that, On the first transmittance curve, the minimum transmittance in the 440nm-580nm band is greater than or equal to 70%, and the average transmittance in the 440nm-580nm band is greater than or equal to 80%. On the second transmittance curve, the minimum transmittance in the 440nm–580nm band is greater than or equal to 70%, and the average transmittance in the 440nm–580nm band is greater than or equal to 80%. The filter according to any one of claims 14 to 17 is characterized in that, The first wavelength and the second wavelength belong to the red light band, while the third wavelength and the fourth wavelength belong to the blue light band. A camera module, characterized in that, The device includes an optical lens, an image sensor, and a filter as described in any one of claims 12 to 18, the filter being disposed between the optical lens and the image sensor, the optical lens being used to receive light from a target object, the filter being used to allow light within a target wavelength range to pass through, the image sensor being used to receive light within the target wavelength range to obtain spectral information of the target object, and the image sensor having more than 3 channels. An electronic device, characterized in that, It includes an image processing chip and a camera device as described in any one of claims 1 to 11, wherein the image processing chip is used to process images acquired by the camera device. The electronic device according to claim 20, characterized in that, The camera device further includes at least one second camera module, which is a three-channel camera. The image processing chip is used to correct the color of the target object acquired by the second camera module based on the spectral information of the target object acquired by the first camera module. The electronic device according to claim 20, characterized in that, The camera device further includes at least one second camera module, which is an M-channel camera where M is greater than or equal to 3. The image processing chip is used to correct the color of the target object acquired by the second camera module based on the spectral information of the target object acquired by the first camera module. The electronic device according to claim 21 or 22 is characterized in that, The second camera module is a wide-angle camera, a telephoto camera, or an ultra-wide-angle camera.

Images