Method, apparatus and electronic device for camera color calibration

Abstract

The application provides a method for camera color calibration applied to an electronic device, the electronic device comprising a camera array (including a first camera) and a high-resolution multispectral camera, the method comprising: synchronously collecting original images of a first scene by the first camera and the high-resolution multispectral camera, wherein the first camera collects a three-channel image (a first image), and the high-resolution multispectral camera synchronously collects an M-channel image (a second image), M being a positive integer greater than or equal to 3; converting the second image from the M-channel image to a three-channel image by a first conversion function; taking the converted three-channel image as a reference to perform color calibration on the first image to obtain first color parameters for color adjustment of the first camera. In the method, the color of the array camera is aligned with the color perceived by the high-resolution multispectral camera, and the authenticity and consistency of the color of the image captured by the camera in the camera array can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of camera color calibration, and more particularly, to a camera color calibration method, apparatus and electronic device. BACKGROUND

[0002] Taking a mobile phone as an example, the camera of a smart phone is generally in the form of a camera array, which includes multiple cameras. The main difference between these cameras is the focal length, which is mainly divided into ultra-wide-angle cameras, wide-angle cameras (main cameras) and long-focus cameras. Different mobile phones may have different cameras, but at least two cameras, which can independently image and have a photographing function, are included. The images or videos captured by these cameras are directly presented to the user for viewing. Since there are obvious color differences in images captured by different cameras under the same scene, the user's shooting experience is seriously affected. SUMMARY

[0003] The present application provides a camera color calibration method, apparatus and electronic device. The method, apparatus and electronic device use an image captured by a high-resolution multispectral camera as a color reference to adjust the color of an image captured by a camera in a camera array. The color of the array camera is aligned with the color perceived by the high-resolution multispectral camera, which can improve the authenticity of the color of the image captured by the camera in the camera array.

[0004] In a first aspect, a camera color calibration method is provided. The method is applied to an electronic device, which includes a camera array and a high-resolution multispectral camera. The method includes: under the condition that a first camera and the high-resolution multispectral camera are turned on at the same time, collecting a first image by the first camera, the first image being an original image of a first scene, the first image being a three-channel image, and the camera array including the first camera; synchronously collecting a second image by the high-resolution multispectral camera, the second image being a multispectral original image of the first scene, the second image being an M-channel image, where M is a positive integer greater than 3; converting the second image from an M-channel image to a three-channel image by a first conversion function; taking the three-channel image converted from the second image as a reference to perform color calibration on the first image to obtain a first color parameter; and issuing the first color parameter to an image signal processor (ISP) channel of the first camera, so that the first camera adjusts the image collected by the first camera according to the first color parameter.

[0005] In some embodiments, the first color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0006] In the embodiments of the present application, the image captured by the high-resolution multi-spectrum camera is used as a color reference to adjust the colors of the images captured by the cameras in the camera array, and the colors of the cameras in the array are aligned to the colors perceived by the high-resolution multi-spectrum camera, thereby improving the authenticity of the colors of the images captured by the cameras in the camera array.

[0007] In combination with the first aspect, in a possible implementation, the camera array further includes a second camera, and the method further includes: collecting, by the second camera, a third image, the third image being an original image of the first scene, the third image being a three-channel image; performing color calibration on the third image based on the three-channel image converted from the second image to obtain a second color parameter; and delivering the second color parameter to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on the image collected by the second camera according to the second color parameter.

[0008] In some embodiments, the first camera, the second camera, and the high-resolution multi-spectrum camera synchronously collect images of the first scene.

[0009] In yet some embodiments, the first camera, the second camera, and the high-resolution multi-spectrum camera asynchronously collect images of the first scene within a certain time threshold.

[0010] In the embodiments of the present application, the image captured by the high-resolution multi-spectrum camera is used as a color reference to adjust the colors of the images captured by the cameras in the camera array, and the colors of the cameras in the array are aligned to the colors perceived by the high-resolution multi-spectrum camera, thereby improving the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0011] In combination with the first aspect, in a possible implementation, the camera array further includes a second camera, and the method further includes: in a case where the second camera and the high-resolution multi-spectrum camera are simultaneously turned on, collecting, by the second camera, a third image, the third image being an original image of the second scene, the third image being a three-channel image; synchronously collecting, by the high-resolution multi-spectrum camera, a fourth image, the fourth image being a multi-spectrum original image of the second scene, the fourth image being an M-channel image; converting, by a second conversion function, the fourth image from the M-channel image to a three-channel image; performing color calibration on the third image based on the three-channel image converted from the fourth image to obtain a second color parameter; and delivering the second color parameter to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on the image collected by the second camera according to the second color parameter.

[0012] In some embodiments, the second color parameter includes any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0013] In the embodiments of the present application, the image captured by the high-resolution multispectral camera is used as a color reference, and the colors of the images captured by each camera in the camera array are adjusted respectively. The colors of the array cameras are all aligned to the color perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0014] In combination with the first aspect, in a possible implementation, the camera array further includes a third camera, and the method further includes: in a case where the third camera and the high-resolution multispectral camera are turned on at the same time, collecting a fifth image by the third camera, the fifth image being an original image of the first scene, the fifth image being a three-channel image; synchronously collecting a sixth image by the high-resolution multispectral camera, the sixth image being a multispectral original image of the first scene, the sixth image being an M-channel image, where M is a positive integer greater than 3; converting the sixth image from the M-channel image to a three-channel image by a third conversion function; taking the three-channel image converted from the sixth image as a reference, performing color calibration on the fifth image to obtain a third color parameter; and delivering the third color parameter to an image signal processor (ISP) channel of the third camera, so that the third camera performs color adjustment on the image collected by the third camera according to the third color parameter.

[0015] In some embodiments, the third color parameter includes any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0016] In some embodiments, the first camera is one of a main camera, an ultra-wide-angle camera, and a long-focus camera, the second camera is one of a main camera, an ultra-wide-angle camera, and a long-focus camera, and the third camera is one of a main camera, an ultra-wide-angle camera, and a long-focus camera, where the first camera, the second camera, and the third camera are different from each other.

[0017] In some embodiments, the first camera is a main camera (wide-angle camera), the second camera is an ultra-wide-angle camera, and the third camera is a long-focus camera.

[0018] In the embodiments of the present application, the image captured by the high-resolution multispectral camera is used as a color reference, and the colors of the images captured by each camera in the camera array are adjusted respectively. The colors of the array cameras are all aligned to the color perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0019] With reference to the first aspect, in a possible implementation, the converting the second image from the M-channel image to the three-channel image by the first conversion function comprises: converting the second image from the M-channel image to the three-channel image by downsampling from M channels of the second image to three channels of the first image according to the first conversion function, wherein the first conversion function is a function of performing matrix multiplication on the second image using a downsampling matrix with a size of Mx3.

[0020] In some embodiments, further explanation on the "converting the fourth image from the M-channel image to the three-channel image by the second conversion function" is similar to the above explanation on the "converting the second image from the M-channel image to the three-channel image by the first conversion function".

[0021] In some embodiments, further explanation on the "converting the sixth image from the M-channel image to the three-channel image by the third conversion function" is similar to the above explanation on the "converting the second image from the M-channel image to the three-channel image by the first conversion function".

[0022] In the embodiments of the present application, the M channels of the second image are downsampled to 3 channels by the downsampling matrix to establish the mapping relationship between the second image and the color space in which the image obtainable by the first camera is located. Since the second image contains M color channels, the spectral information it contains is more abundant than the spectral information corresponding to the first image. Therefore, the three-channel image obtained by mapping the second image to the color space in which the image obtainable by the first camera is located is closer to the true color of the first scene, and thus the image after down-sampling of the second image channel can be used to adjust the color of the first image, improving the color authenticity and consistency of the image captured by the first camera.

[0023] With reference to the first aspect, in a possible implementation, before converting the second image from the M-channel image to the three-channel image by the first conversion function, the method further comprises: performing field of view (FOV) alignment on the first image and the second image.

[0024] In some embodiments, before converting the fourth image from the M-channel image to the three-channel image by the second conversion function, the method further comprises: performing field of view (FOV) alignment on the third image and the fourth image.

[0025] In some embodiments, before converting the sixth image from the M-channel image to the three-channel image by the third conversion function, the method further comprises: performing field of view (FOV) alignment on the fifth image and the sixth image.

[0026] In the embodiments of the present application, before converting the second image from an M-channel image to a three-channel image (i.e., establishing a mapping relationship between the second image and a color space in which the image obtainable by the first camera is located), the first image and the second image are first subjected to FOV alignment, so that the authenticity and consistency of the color adjustment of the first image can be avoided from being affected by the difference in the viewing angle between the first image and the second image.

[0027] In combination with the first aspect, in a possible implementation manner, the method further includes: determining the first conversion function based on the FOV-aligned second image.

[0028] In some embodiments, the method further includes: determining the second conversion function based on the FOV-aligned fourth image.

[0029] In some embodiments, the method further includes: determining the third conversion function based on the FOV-aligned sixth image.

[0030] In some embodiments, the estimation method of the downsampling matrix used by the first conversion function can be any one of a Bayesian method, a least square method, a deep learning method, or other estimation methods.

[0031] In combination with the first aspect, in a possible implementation manner, converting the second image from an M-channel image to a three-channel image by using the first conversion function includes: performing spectral band super-resolution on the second image according to a spectral response curve corresponding to the second image to obtain an hyperspectral image corresponding to the second image, the hyperspectral image corresponding to the second image being an N-channel image, where N is a positive integer much larger than M; and converting the hyperspectral image corresponding to the second image from an N-channel image to a three-channel image by downsampling from N channels of the hyperspectral image corresponding to the second image to three channels of the first image according to the first conversion function, where the first conversion function is a function of performing matrix multiplication on the second image by using a downsampling matrix with a size of N×3.

[0032] In some embodiments, the further explanation of "converting the fourth image from an M-channel image to a three-channel image by using the second conversion function" is similar to the above explanation of "converting the second image from an M-channel image to a three-channel image by using the first conversion function".

[0033] In some embodiments, the further explanation of "converting the sixth image from an M-channel image to a three-channel image by using the third conversion function" is similar to the above explanation of "converting the second image from an M-channel image to a three-channel image by using the first conversion function".

[0034] In the embodiments of the present application, the N channels of the hyperspectral image of the second image are down-sampled to 3 channels by a down-sampling matrix, and a mapping relationship between the hyperspectral image of the second image and the color space in which the image obtainable by the first camera is established. Since the hyperspectral image of the second image contains N color channels, the spectral information contained is more abundant than the spectral information corresponding to the first image, so that the three-channel image obtained by mapping the hyperspectral image of the second image to the color space in which the image obtainable by the first camera is closer to the true colors of the first scene, and then the image obtained by down-sampling the hyperspectral image channels of the second image can be used to adjust the colors of the first image, thereby improving the color authenticity and consistency of the image captured by the first camera.

[0035] In combination with the first aspect, in a possible implementation manner, the method further includes: determining the first conversion function according to the hyperspectral image corresponding to the second image and the spectral response curve corresponding to the first image.

[0036] In some embodiments, the method further includes: determining the second conversion function according to the hyperspectral image corresponding to the fourth image and the spectral response curve corresponding to the third image.

[0037] In some embodiments, the method further includes: determining the third conversion function according to the hyperspectral image corresponding to the sixth image and the spectral response curve corresponding to the fifth image.

[0038] In some embodiments, the estimation method of the down-sampling matrix used by the first conversion function / second conversion function / third conversion function can be any one of a Bayesian method, a least squares method, and a deep learning method, and can also be other estimation methods.

[0039] The second aspect provides an electronic device, which includes a camera array, a high-resolution multi-spectral camera, and a processor. The camera array includes a first camera. The first camera is configured to capture a first image under the condition that the first camera and the high-resolution multi-spectral camera are turned on at the same time. The first image is an original image of a first scene, and the first image is a three-channel image. The high-resolution multi-spectral camera is configured to synchronously capture a second image. The second image is a multi-spectral original image of the first scene, and the second image is an M-channel image, where M is a positive integer greater than 3. The processor is configured to convert the second image from an M-channel image to a three-channel image by using a first conversion function. The processor is further configured to perform color calibration on the first image based on the three-channel image converted from the second image, to obtain a first color parameter. The processor is further configured to send the first color parameter to an image signal processor (ISP) channel of the first camera, so that the first camera can perform color adjustment on the image captured by the first camera according to the first color parameter.

[0040] In some embodiments, the first color parameter comprises any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, a 3D lookup table.

[0041] In the embodiments of the present application, the image captured by the high-resolution multispectral camera is used as a color reference to adjust the color of the images captured by the cameras in the camera array, and the color of the array cameras is aligned to the color perceived by the high-resolution multispectral camera, which can improve the authenticity of the color of the images captured by the cameras in the camera array.

[0042] With reference to the second aspect, in a possible implementation manner, the camera array further includes a second camera, and the second camera is configured to capture a third image, the third image being an original image of the first scene, and the third image being a three-channel image; the processor is further configured to perform color calibration on the third image based on the three-channel image converted from the second image to obtain a second color parameter; and the processor is further configured to send the second color parameter to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on the image captured by the second camera according to the second color parameter.

[0043] In some embodiments, the first camera, the second camera and the high-resolution multispectral camera synchronously capture images of the first scene.

[0044] In yet some embodiments, the first camera, the second camera and the high-resolution multispectral camera asynchronously capture images of the first scene within a certain time threshold.

[0045] In the embodiments of the present application, the image captured by the high-resolution multispectral camera is used as a color reference to adjust the color of the images captured by the cameras in the camera array, and the color of the array cameras is aligned to the color perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the color of the images captured by the cameras in the camera array.

[0046] With reference to the second aspect, in a possible implementation manner, the camera array further includes a second camera, and the second camera is configured to: in a case where the second camera and the high-resolution multi-spectrum camera are simultaneously turned on, collect a third image through the second camera, the third image being an original image of a second scene, and the third image being a three-channel image; the high-resolution multi-spectrum camera is further configured to: synchronously collect a fourth image, the fourth image being a multi-spectrum original image of the second scene, and the fourth image being an M-channel image; the processor is further configured to: convert the fourth image from the M-channel image to a three-channel image through a second conversion function; perform color calibration on the third image based on the three-channel image converted from the fourth image, to obtain a second color parameter; and send the second color parameter to an image signal processor (ISP) channel of the second camera, so that the second camera can perform color adjustment on an image collected by the second camera according to the second color parameter.

[0047] In some embodiments, the second color parameter includes any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0048] In the embodiments of the present application, the image captured by the high-resolution multi-spectrum camera is used as a color reference, and the colors of the images captured by each camera in the camera array are adjusted respectively, so that the colors of the images captured by the cameras in the camera array are consistent with the colors perceived by the high-resolution multi-spectrum camera, and the authenticity and consistency of the colors of the images captured by the cameras in the camera array are improved.

[0049] With reference to the second aspect, in a possible implementation manner, the camera array further includes a third camera, and the third camera is configured to: in a case where the third camera and the high-resolution multi-spectrum camera are simultaneously turned on, collect a fifth image through the third camera, the fifth image being an original image of a first scene, and the fifth image being a three-channel image; the high-resolution multi-spectrum camera is further configured to: synchronously collect a sixth image, the sixth image being a multi-spectrum original image of the first scene, and the sixth image being an M-channel image, where M is a positive integer greater than 3; the processor is further configured to: convert the sixth image from the M-channel image to a three-channel image through a third conversion function; perform color calibration on the fifth image based on the three-channel image converted from the sixth image, to obtain a third color parameter; and send the third color parameter to an image signal processor (ISP) channel of the third camera, so that the third camera can perform color adjustment on an image collected by the third camera according to the third color parameter.

[0050] In some embodiments, the third color parameter comprises any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, a 3D lookup table.

[0051] In some embodiments, the first camera is one of a main camera, an ultra-wide-angle camera, and a long-focus camera, the second camera is one of a main camera, an ultra-wide-angle camera, and a long-focus camera, and the third camera is one of a main camera, an ultra-wide-angle camera, and a long-focus camera, wherein the first camera, the second camera, and the third camera are different from each other.

[0052] In some embodiments, the first camera is a main camera (a wide-angle camera), the second camera is an ultra-wide-angle camera, and the third camera is a long-focus camera.

[0053] In the embodiments of the present application, the image captured by the high-resolution multispectral camera is used as a color reference, and the colors of the images captured by each camera in the camera array are adjusted respectively, so that the colors of the images captured by the cameras in the camera array are consistent with the colors perceived by the high-resolution multispectral camera, thereby improving the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0054] With reference to the second aspect, in a possible implementation manner, the processor is specifically configured to: convert the second image from an M-channel image to a three-channel image by down-sampling the M channels of the second image to three channels according to the first conversion function, wherein the first conversion function is a function of performing matrix multiplication on the second image using a down-sampling matrix with a size of Mx3.

[0055] In the embodiments of the present application, the M channels of the second image are down-sampled to 3 channels by a down-sampling matrix, and a mapping relationship between the second image and the color space in which the image obtainable by the first camera is located is established. Since the second image contains M color channels, the spectral information included is more abundant than the spectral information corresponding to the first image, so that when the second image is mapped to the color space in which the image obtainable by the first camera is located, the three-channel image obtained is closer to the true colors of the first scene, and then the image after down-sampling of the channels of the second image can be used to adjust the colors of the first image, thereby improving the color authenticity and consistency of the image captured by the first camera.

[0056] With reference to the second aspect, in a possible implementation manner, the processor is specifically configured to: perform field of view (FOV) alignment on the first image and the second image before converting the second image from an M-channel image to a three-channel image by the first conversion function.

[0057] In the embodiments of the present application, before the second image is converted from the M-channel image to the three-channel image (i.e., the mapping relationship between the second image and the color space in which the image obtainable by the first camera is established), the first image and the second image are first subjected to view angle alignment, so that the influence of the color adjustment of the first image caused by the view angle difference between the first image and the second image on the authenticity and consistency of the color adjustment of the first image can be avoided.

[0058] With reference to the second aspect, in a possible implementation manner, the processor is further configured to determine the first conversion function based on the second image after the FOV alignment.

[0059] In some embodiments, the estimation method of the downsampling matrix used by the first conversion function can be any one of a Bayesian method, a least square method, a deep learning method, or other estimation methods.

[0060] With reference to the second aspect, in a possible implementation manner, the processor is specifically configured to perform spectral band super-resolution on the second image according to a spectral response curve corresponding to the second image to obtain an hyperspectral image corresponding to the second image, the hyperspectral image corresponding to the second image being an N-channel image, where N is a positive integer much larger than M; and convert the hyperspectral image corresponding to the second image from the N-channel image to the three-channel image by downsampling, from N channels of the hyperspectral image corresponding to the second image, to three channels of the first image according to the first conversion function, where the first conversion function is a function of performing matrix multiplication on the second image by using a downsampling matrix with a size of N*3.

[0061] In the embodiments of the present application, the N channels of the hyperspectral image of the second image are downsampld to 3 channels by using the downsampling matrix, and the mapping relationship between the hyperspectral image of the second image and the color space in which the image obtainable by the first camera is established. Since the hyperspectral image of the second image includes a color channel number with a size of N, the spectral information included is more abundant than the spectral information corresponding to the first image, so that the three-channel image obtained by mapping the hyperspectral image of the second image to the color space in which the image obtainable by the first camera is closer to the real color of the first scene, and the image after the channel downsampling of the hyperspectral image of the second image can be used to adjust the color of the first image, thereby improving the color authenticity and consistency of the image captured by the first camera.

[0062] With reference to the second aspect, in a possible implementation manner, the processor is further configured to determine the first conversion function according to the hyperspectral image corresponding to the second image and the spectral response curve corresponding to the first image.

[0063] In a third aspect, an electronic device is provided, which includes a memory and a processor, wherein the memory is configured to store computer program code, and the processor is configured to execute the computer program code stored in the memory to implement the method in the first aspect or any possible implementation manner of the first aspect.

[0064] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program or instructions, and when the computer program or instructions are executed, the method in the first aspect or any possible implementation manner of the first aspect is implemented.

[0065] In a fifth aspect, a chip is provided, which stores instructions, and when the instructions are executed on a device, the chip performs the method in the first aspect or any possible implementation manner of the first aspect.

[0066] In a sixth aspect, a computer program product is provided, which stores a computer program or instructions, and when the computer program or instructions are executed, the method in the first aspect or any possible implementation manner of the first aspect is implemented. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 FIG. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0068] Figure 2 FIG. 2 is a software structural block diagram of an electronic device provided by an embodiment of the present application;

[0069] Figure 3 FIG. 3 is a schematic diagram of an array camera provided by an embodiment of the present application;

[0070] Figure 4 FIG. 4 is a schematic flow diagram of a camera color calibration method;

[0071] Figure 5 FIG. 5 is a schematic flow diagram of another camera color calibration method;

[0072] Figure 6 FIG. 6 is a schematic diagram of another array camera provided by an embodiment of the present application;

[0073] Figure 7 FIG. 7 is a structural schematic diagram of a single-point multi-spectrum camera provided by an embodiment of the present application;

[0074] Figure 8 FIG. 8 is a flow schematic diagram of a camera color calibration method using a single-point multi-spectrum camera provided by an embodiment of the present application;

[0075] Figure 9 FIG. 9 is a schematic diagram of an RGGB sensor mode provided by an embodiment of the present application;

[0076] Figure 10 is a spectral sensing schematic diagram of a high-resolution multispectral device provided by an embodiment of the present application;

[0077] Figure 11 is a schematic flowchart of a camera color calibration method provided by an embodiment of the present application;

[0078] Figure 12 is a schematic flowchart of another camera color calibration method provided by an embodiment of the present application;

[0079] Figure 13 is a schematic diagram of a system framework corresponding to a camera color calibration method provided by an embodiment of the present application;

[0080] Figure 14 is a schematic flowchart corresponding to another camera color calibration method provided by an embodiment of the present application;

[0081] Figure 15 is a schematic flowchart corresponding to another camera color calibration method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0082] The technical solutions in the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.

[0083] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; in this document, "and / or" only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "plurality" or "multiple" means two or more than two.

[0084] Hereinafter, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. In the description of the embodiments, unless otherwise specified, the meaning of "multiple" is two or more than two.

[0085] The terminology used in the following description merely for the purpose of describing particular embodiments and is not intended to limit the application. As used in this description and the accompanying claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "or" as used herein is used to denote exclusive "or", unless otherwise indicated herein.

[0086] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment, but can refer to different embodiments. The terms "comprises", "comprising", "including", and "having" as used in this specification are specifically intended to encompass the presence of stated features, structures, or characteristics, but do not preclude the presence or addition of one or more other features, structures, or characteristics. Furthermore, although individually listed, a plurality of features, structures, or characteristics can be implemented in one or more embodiments.

[0087] The method provided by the embodiments of the present application can be applied to electronic devices with time display function or time recognition function, for example, can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), smart home devices, etc. The embodiments of the present application do not make any limitation on the specific type of electronic device.

[0088] Exemplarily, Figure 1A structural diagram of the electronic device 100 is shown. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0089] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0090] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0091] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0092] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or is recycling. If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. This avoids repeated access and reduces the latency of the processor 110, thus improving the efficiency of the system.

[0093] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0094] The USB interface 130 is an interface that conforms to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect earphones to play audio through the earphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0095] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0096] The charging management module 140 is configured to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some embodiments with wired charging, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some embodiments with wireless charging, the charging management module 140 can receive wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 and power the electronic device through the power management module 141.

[0097] The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0098] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0099] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be configured to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0100] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer the same to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor, and radiate the same as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the same device as at least part of the modules of the processor 110.

[0101] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the microphone 170B, etc.), or displays an image or a video through the display screen 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110, and disposed in the same device as the mobile communication module 150 or other functional modules.

[0102] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and radiate it as an electromagnetic wave via the antenna 2.

[0103] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0104] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0105] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0106] The electronic device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0107] The ISP is configured to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0108] The camera 193 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0109] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0110] The video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0111] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognition applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0112] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize data storage functions. For example, music, video, etc. Files are saved in the external memory card.

[0113] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one App required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phonebook, etc.), etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), etc.

[0114] The electronic device 100 can realize audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0115] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some of the functions of the audio module 170 can be disposed in the processor 110.

[0116] The speaker 170A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0117] The receiver 170B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 100 receives a call or a voice message, the user can listen to the voice through the receiver 170B close to the ear.

[0118] The microphone 170C, also referred to as a "microphone", "sound collector", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can make a sound through the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, noise reduction functions can also be realized. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, and can also identify the source of the sound, realize directional recording function, etc.

[0119] The earphone interface 170D is configured to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0120] The keys 190 include a power key, a volume key, etc. The keys 190 can be mechanical keys. They can also be touch keys. The electronic device 100 can receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0121] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations acting on different regions of the display screen 194. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0122] The indicator 192 can be an indicator light, which can be used to indicate a charging state, a power change, and can also be used to indicate a message, a missed call, a notification, etc.

[0123] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 195 to realize contact and separation with the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, and N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external storage cards. The electronic device 100 interacts with a network through a SIM card to realize functions such as calling and data communication. In some embodiments, the electronic device 100 uses an embedded SIM (eSIM) card, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0124] It should be understood that the phone card in the embodiments of the present application includes but is not limited to a SIM card, an eSIM card, a universal subscriber identity module (USIM), a universal integrated circuit card (UICC), etc.

[0125] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0126] Figure 2Figure 1 is a software structure block diagram of an electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer. The application layer can include a series of application packages.

[0127] As shown in Figure 2, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. Figure 2

[0128] The application framework layer provides the application layer with application programming interfaces (APIs) and programming frameworks for the applications. The application framework layer includes some pre-defined functions.

[0129] As shown in Figure 3, the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc. Figure 2

[0130] The window manager is used to manage the window program. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and take a screenshot, etc.

[0131] The content provider is used to store and obtain data, and make the data accessible to the application. The data can include video, image, audio, dialed and received calls, browsing history and bookmarks, phonebook, etc.

[0132] The view system includes visual controls, such as controls that display text, controls that display pictures, etc. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface that includes a short message notification icon can include a view that displays text and a view that displays a picture.

[0133] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including connection, hang up, etc.).

[0134] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.

[0135] ​​The notification manager enables an application to display notification information in the status bar, which can be used to convey a message of the informing type, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform the completion of downloading, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the system top status bar, such as a notification of a background running application, and can also be a notification in the form of a dialogue window appearing on the screen. For example, the status bar prompts text information, emits a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0136] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0137] The core library includes two parts: one part is a function function that the java language needs to call, and the other part is the core library of Android.

[0138] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection, etc.

[0139] The system library can include multiple functional modules. For example: a surface manager, media libraries, a three-dimensional graphics processing library (for example: OpenGL ES), a 2D graphics engine (for example: SGL), etc.

[0140] The surface manager is used to manage the display subsystem, and provides a fusion of 2D and 3D layers for multiple applications.

[0141] The media library supports multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0142] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0143] The 2D graphics engine is a drawing engine for 2D drawing.

[0144] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.

[0145] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Harmony, etc.

[0146] The technical solution of the embodiments of the present application can be applied to the image processing scene of multiple cameras. Exemplarily, it can be applied to the color calibration scene of the image collected by the array camera.

[0147] The electronic device can be a television, a desktop computer, a notebook computer, and can also be a portable electronic device, such as a mobile phone, a folding screen, a tablet computer, a camera, a video camera, a video recorder, and can also be a smart home device, such as a refrigerator, a washing machine, a sweeper, and any electronic device with multiple cameras, and can also be an electronic device in a 5G network or an electronic device in a future evolved public land mobile network (PLMN) and the like.

[0148] Exemplarily, Figure 3 A schematic diagram of a camera setting provided by an embodiment of the present application is shown.

[0149] As Figure 3 shown, the electronic device 300 is provided with a camera array, which includes a camera 310, a camera 320, and a camera 330.

[0150] In some embodiments, the camera 310 is an ultra-wide-angle camera, the camera 320 is a wide-angle camera, which is also a main camera, and the camera 330 is a long-focus camera.

[0151] Taking a mobile phone as an example, the cameras of a smart phone are generally in the form of a camera array, which includes multiple cameras. The main difference between these cameras is the focal length, which is mainly divided into ultra-wide-angle cameras, wide-angle cameras (main cameras), and long-focus cameras. Different mobile phones may have different cameras, but at least two cameras, which can all independently image and have a photographing function. The images or videos captured by the cameras are directly presented to the user for viewing. Since the images captured by different cameras in the same scene are likely to have obvious color differences, this will seriously affect the user's shooting experience. Therefore, the user has a high requirement for the color consistency of the images captured by each camera in the camera array.

[0152] Exemplarily, Figure 4 A schematic flowchart of a camera color calibration method 400 is shown. The method is a multi-camera simultaneous start method, as Figure 4 shown, the method 400 includes:

[0153] S401: Taking one of the multiple cameras (usually the main camera) provided on the first device as a reference, real-time calibration is performed on the other cameras in the multiple cameras.

[0154] The plurality of cameras are all three-channel cameras, i.e., the images captured by the plurality of cameras are all three-channel images.

[0155] Specifically, the real-time calibration can be performed on each of the other cameras in the plurality of cameras based on one camera in the plurality of cameras arranged on the first device.

[0156] The color of the image captured by the camera is calibrated in real time.

[0157] In the method, there are many differences between different camera modules, such as shooting field of view (FOV), shooting brightness, shooting contrast, shooting dynamic range, and sensor spectral response consistency. One camera in the plurality of cameras is used as a reference to calibrate the other cameras. Since these differences between the cameras are not well considered, the accuracy of color calibration of the images captured by the cameras is relatively low. In addition, the calibration process needs to start at least two cameras at the same time, which will generate a large power consumption.

[0158] Exemplarily, Figure 5 Another method 500 for calibrating the color of a camera is shown in the schematic flowchart. The method is an offline calibration method, as shown in Figure 5 As shown in the method 500 includes:

[0159] S501: simultaneously starting a first camera and a second camera of an electronic device, wherein the first camera is used as a reference camera for calibrating the second camera.

[0160] The first camera and the second camera are both three-channel cameras, i.e., the images captured by the first camera and the second camera are both three-channel images.

[0161] S502: determining a transfer matrix / transfer function between the first camera and the second camera according to the color perception distribution of the first camera and the color perception distribution of the second camera.

[0162] It can be understood that S501 and S502 are a process of calibrating the second camera based on the first camera. After the calibration is completed (i.e., after the transfer matrix / transfer function between the first camera and the second camera is obtained), the first camera and the second camera are closed.

[0163] S503: when a first object is photographed by the first camera, the first camera is started, and a first image corresponding to the first object is captured by the first camera.

[0164] S504: determining a second image corresponding to the second camera according to the migration matrix / migration function between the first camera and the second camera and the first image.

[0165] Specifically, when a first object is photographed by the first camera, the first camera is turned on, and the first camera determines corresponding module migration parameters based on the migration matrix / migration function between the first camera and the second camera by estimating the spectrum of the current scene; and then determines the second image corresponding to the second camera according to the module migration parameters and the first image.

[0166] In this method, it is difficult to determine appropriate module migration parameters due to the difficulty in estimating the spectrum of the scene and the inability to finely perceive the color of the scene, resulting in relatively low accuracy of camera color calibration.

[0167] As can be seen from the above, the current camera color calibration methods all have the problems of low accuracy of camera color calibration and high calibration cost (labor cost, power consumption cost), and the images photographed by each camera in the camera array have inconsistent and unrealistic colors.

[0168] Therefore, the embodiments of the present application provide a camera color calibration method, device and electronic device, in which a high-resolution multispectral camera is arranged on the electronic device, and the cameras in the camera array arranged on the electronic device are color calibrated based on the image collected by the high-resolution multispectral camera, which can avoid the problem of inaccurate calibration caused by calibration between array cameras, improve the accuracy and stability of camera color calibration, make the colors of the images output by the multiple cameras in the camera array more consistent and more realistic, and reduce the degree of architecture redundancy of the electronic device due to the small size of the high-resolution multispectral camera. The power consumption generated during the calibration process is also lower, which can save the calibration cost and further improve the user's shooting experience.

[0169] Exemplarily, Figure 6 A schematic diagram of another camera arrangement provided by the embodiments of the present application is shown.

[0170] As Figure 6 shown, the electronic device 600 is provided with a camera array and a high-resolution multispectral camera 610, and the camera array includes a camera 310, a camera 320 and a camera 330.

[0171] In some embodiments, the camera 310 is a super wide-angle camera, the camera 320 is a wide-angle camera and is also a main camera, and the camera 330 is a long-focus camera.

[0172] Among them, the spectral sensing capability of the high-resolution multispectral camera is far stronger than that of the ultra-wide-angle camera, wide-angle camera and telephoto camera. In some embodiments, the solution of this application embodiment can be understood as follows: taking the multispectral image of the first scene acquired by the high-resolution multispectral camera 610 as a reference, the red-green-blue (RGB) images of the first scene acquired by the ultra-wide-angle camera 310, wide-angle camera 320 and telephoto camera 330 are respectively color calibrated.

[0173] It should be understood that this embodiment is only an illustrative description of the camera setup. The high-resolution multispectral camera 610 can be set independently of the camera array or it can be set in the camera array. This application does not limit the setup method of the high-resolution multispectral camera 610 on the electronic device.

[0174] To more clearly understand the advantages of using high-resolution multispectral cameras for camera color calibration, exemplarily, combined with Figure 7 and Figure 8 This paper provides a detailed introduction to the process of camera color calibration using a single-point multispectral camera.

[0175] Figure 7 The structural diagram of a single-point multispectral camera is shown, such as... Figure 7 As shown, a single-point multispectral camera includes a homogenizing film and a single-point multispectral sensor arranged in parallel. Light rays incident from different directions pass through the homogenizing film. Due to the averaging effect of the homogenizing film, the amplitude and exit angle of the light rays exiting the homogenizing film are the same. The exiting light rays then pass through the single-point multispectral sensor to perceive the average reflection spectrum of the scene. Multiple light rays exiting from the homogenizing film, after passing through the single-point multispectral sensor, output a single-point spectral perception result with a dimension of 1×M, i.e., a vector with a dimension of 1×M. Since the 1×M single-point spectral perception result has only one point (i.e., a single point) that is globally averaged in space and has spectral resolution, spatial correspondence cannot be performed, and the specific spatial location of each array camera corresponding to this single point cannot be determined. Therefore, it is impossible to estimate the spectral band downsampling mapping function (that is, it is impossible to convert multi-channel images to three-channel images).

[0176] Furthermore, Figure 8 A flowchart illustrating a method 800 for color calibration using a single-point multispectral camera is shown, as follows: Figure 8 As shown, the method 800 includes:

[0177] S801: The main camera / wide-angle / telephoto camera captures RAW images of the current scene.

[0178] S802: The single-point camera synchronously acquires the single-point spectral perception result of the current scene. The single-point spectral perception result is a single-point spectral perception result with a dimension of 1×M.

[0179] S803: performing light source classification based on the single-point spectral sensing result, i.e., determining a light source category of the current scene.

[0180] However, the light source classification based on the single-point spectral sensing result and the determined light source category have the problems of low accuracy and limited scene.

[0181] S804: determining a color parameter based on the light source category of the current scene according to a pre-calibrated color parameter-light source category association table.

[0182] The pre-calibrated color parameter-light source category association table refers to a color parameter-light source category association table pre-calibrated for mapping wide-angle / telephoto to main camera.

[0183] S805: issuing the determined color parameter to an ISP channel of the main camera / wide-angle / telephoto camera.

[0184] S806: performing color adjustment on the wide-angle / telephoto camera according to the color parameter.

[0185] In the method, the light source classification based on the single-point spectral sensing result and the determined light source category have the problems of low accuracy and limited scene. Therefore, the camera color calibration using the single-point multispectral camera is discrete, inaccurate and not robust, and is easily disturbed by mixed light sources, so it cannot accurately perform color calibration and cannot achieve multi-camera color consistency.

[0186] Therefore, to solve the above technical problems of the single-point multispectral camera, the embodiments of the present application perform camera color calibration using a high-resolution multispectral camera (a multi-point multispectral camera). To more clearly understand the method of camera color calibration provided by the embodiments of the present application, the high-resolution multispectral camera (or high-resolution multispectral device) is described in detail as follows.

[0187] Currently, the structure of the image sensor used on electronic devices is basically a Bayer pattern structure. A basic color pixel includes 2x2=4 pixels, and 3 different color filters (such as red, green and blue RGB, red, yellow and blue RYB) are placed thereon to form an RGGB sensor or an RYYB sensor, or to form a color perception unit similar to RGGB or RYYB. The entire image sensor is composed of a large number of rows and columns of color perception units.

[0188] RGGB is a color mode, which is a mosaic color filter array formed by arranging RGB color filters on the light sensing component grid, in which arrangement, 50% is green, 25% is red, and the other 25% is blue; the RYYB sensor is to change the arrangement of the RGGB array bottom filter, replace two green pixels (G) with two yellow pixels (Y) to form the RYYB sensor.

[0189] The high-resolution multispectral device is mainly different from the RGGB device and the RYYB device in that the color perception unit of the high-resolution multispectral device has more spectral perception functions (that is, more spectral bands). Specifically, the number of spectral perception functions (that is, the number of spectral bands) of the RGGB device and the RYYB device is 3 (red, green, and blue or red, yellow, and blue), and the number of spectral perception functions of the high-resolution multispectral device is greater than 3, for example, can be 3*3=9 spectral bands. Therefore, the high-resolution multispectral device can obtain a spectral perception capability much stronger than 3 channels while not significantly reducing the spatial resolution, that is, compared with the three-channel spectral perception capability of the RGGB device and the RYYB device, the spectral perception capability of the high-resolution multispectral device is stronger, and the perceived multispectral image has higher color authenticity.

[0190] Exemplarily, taking the RGGB device as an example, Figure 9 A spectral perception schematic diagram of a three-channel spectral device provided by an embodiment of the present application is shown.

[0191] As Figure 9 shown, in the RGGB device, RGB color filters are arranged on the light sensing component grid to form a mosaic color filter array, in which arrangement, 50% is green, 25% is red, and the other 25% is blue. The corresponding spectral perception capability is three-channel (red, green, and blue) spectral perception capability.

[0192] Exemplarily, Figure 10 A spectral perception schematic diagram of a high-resolution multispectral device provided by an embodiment of the present application is shown.

[0193] As Figure 10 shown in (a) in FIG. 6, in an embodiment, the number of spectral bands perceived by the high-resolution multispectral device is 5, for example, can include red, orange, yellow, blue, and purple. The corresponding multispectral response curve can be as shown in (c) in FIG. 6. Figure 10 The multispectral bands perceived by the high-resolution multispectral device as shown in (a) in FIG. 6 can be converted into the hyperspectral bands as shown in (b) in FIG. 6. The corresponding hyperspectral response curve can be as shown in (d) in FIG. 6. Figure 10 The multispectral bands perceived by the high-resolution multispectral device as shown in (a) in FIG. 6 can be converted into the hyperspectral bands as shown in (b) in FIG. 6. The corresponding hyperspectral response curve can be as shown in (d) in FIG. 6. Figure 10 The multispectral bands perceived by the high-resolution multispectral device as shown in (a) in FIG. 6 can be converted into the hyperspectral bands as shown in (b) in FIG. 6. The corresponding hyperspectral response curve can be as shown in (d) in FIG. 6. Figure 10 The multispectral bands perceived by the high-resolution multispectral device as shown in (a) in FIG. 6 can be converted into the hyperspectral bands as shown in (b) in FIG. 6. The corresponding hyperspectral response curve can be as shown in (d) in FIG. 6.

[0194] It can be seen that, compared with the 3-channel spectrum sensed by the three-channel spectral device, the high-resolution multi-spectral device has stronger spectral perception capability, and the color of the multi-spectral image collected by the high-resolution multi-spectral device is closer to the real color.

[0195] Specifically,

[0196] (1) The three-channel camera is limited by the material capability, and the corresponding spectral perception curve is greatly different from the spectral perception curve corresponding to the human eye, the color restoration capability of the collected image is weak, and color deviation of some colors is prone to occur, for example, when collecting images of real red-orange, green-blue and blue-violet, the corresponding colors in the output image deviate from red-orange, green-blue and blue-violet.

[0197] The corresponding multi-spectral perception curve and hyper-spectral perception curve of the high-resolution multi-spectral camera are less different from the spectral perception curve corresponding to the human eye, and the color restoration capability of the collected image is strong, so that the problem of color deviation basically does not occur.

[0198] In the embodiment of the application, the high-resolution multi-spectral camera can reduce the corresponding spectral perception function from high dimension, that is, based on the perceived multi-spectral band, the band is down-sampled to obtain a three-channel image with more real colors.

[0199] The spectral perception curve can also be described as a spectral response curve.

[0200] (2) The human eye and brain have color adaptation capability, which is mainly related to the color of the environmental light source. It is difficult to accurately estimate the light source color from the three-channel color information collected by the three-channel (for example: RGB / RYB) camera, and the output picture is prone to white balance inaccuracy problem, thereby causing the whole picture to be color deviated; the high-resolution multi-spectral camera has more channels, which greatly increases the spatial color information corresponding to the RAW image collected by the high-resolution multi-spectral camera, and the color of the output picture is closer to the real color of the collected object. Correspondingly, the white balance capability is also significantly enhanced, which can solve the problem of image color deviation to a great extent, wherein the white balance refers to the reference color for determining the color of the image. When the white balance is inaccurate (that is, the reference color is inaccurate), the whole picture will naturally be color deviated.

[0201] (3) The high-resolution multispectral camera can be reduced in dimension from the spectral perception function to the spectral perception function corresponding to the human eye, the spectral perception function corresponding to the RGB camera, and the spectral perception function corresponding to the RYB camera (it can also be understood that the spectral perception function corresponding to the high-resolution multispectral camera is fitted with the spectral perception function corresponding to the human eye, the spectral perception function corresponding to the RGB camera, and the spectral perception function corresponding to the RYB camera, respectively). That is, the multi-channel image collected by the high-resolution multispectral camera can be converted into a three-channel image. Since the multi-channel image collected by the high-resolution multispectral camera is highly authentic, the converted three-channel image is also more authentic than the three-channel image collected by the three-channel camera. The dimensionality reduction fitting of the spectral band from high dimension to low dimension is very accurate. Since the high-resolution multispectral camera can perform the above spectral band dimensionality reduction, it is equivalent to that the high-resolution multispectral camera can simultaneously capture the color graph perceived by the human eye, the color graph perceived by the ultra-wide-angle camera, the color graph perceived by the wide-angle (main camera) camera, and the color graph perceived by the long-focus camera. Moreover, there is no difference between the captured color graphs in FOV, brightness, contrast, dynamic range, overexposure, etc. This can achieve better color consistency and higher authenticity of the images collected by the multiple cameras in the camera array.

[0202] Therefore, the color perceived by the high-resolution multispectral camera is accurate, and the original image (RAW image) distribution of the ultra-wide-angle camera, the wide-angle (main camera), and the long-focus camera with the same information source can be simulated by spectral band dimensionality reduction (i.e., converting the perceived multi-light channel image into a three-channel image by spectral band dimensionality reduction). There is no FOV difference between the simulated multiple RAW image distributions, and there is no problem of inaccurate white balance. Therefore, the high-resolution multispectral camera can be used as a color global observer. Different types of cameras (including ultra-wide-angle cameras, wide-angle (main cameras), and long-focus cameras, including one-for-two and other modules) align with the color perceived by the high-resolution multispectral camera, so that the color performance is highly consistent and highly authentic.

[0203] Exemplarily, Figure 11 A schematic flowchart of a camera color calibration method 1100 provided by an embodiment of the present application is shown. As Figure 11 shown, the method 1100 includes:

[0204] S1101: simultaneously turn on the first camera and the high-resolution multispectral camera of the electronic device, and the first camera of the electronic device collects a first image, which is a RAW image of a first scene.

[0205] The first image is a three-channel image, for example, an RGB image, or an RYB image.

[0206] The first camera of the electronic device can include one or more of a wide-angle camera, an ultra-wide-angle camera, and a long-focus camera.

[0207] S1102: While the first camera is collecting the first image, a high-resolution multispectral camera of the electronic device collects a second image, which is a multispectral raw image of the first scene.

[0208] The second image is an M-channel image, where M is a positive integer greater than 3.

[0209] It can also be understood that the first image collected by the first camera corresponds to spectral information of three frequency bands, and the second image collected by the high-resolution multispectral camera corresponds to spectral information of M frequency bands. The second image has much richer spatial color information than the first image, that is, the high-resolution multispectral camera perceives more spectral information than the first camera, which is closer to the true color of the first scene.

[0210] S1103: Convert the second image from a multi-channel image to a three-channel image by using a first conversion function, where the first conversion function is a conversion function between the high-resolution multispectral camera and the first camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the first camera.

[0211] In a specific embodiment, the second image is converted from a multi-channel image to a three-channel image by performing spectral band down-sampling on the second image according to the first conversion function.

[0212] The first conversion function is a function of using a down-sampling matrix to perform matrix multiplication on the second image.

[0213] In some embodiments, the estimation method of the down-sampling matrix used by the first conversion function can be any one of a Bayesian method, a least squares method, and a deep learning method, or other estimation methods.

[0214] Down-sampling, also known as decimation or extraction, is one of the basic contents of digital signal processing. It refers to taking a sample once every few samples of a sample sequence, and the new sequence obtained is the down-sampling of the original sequence. In image processing, the main purpose of down-sampling is to reduce the image size to meet the size of the display area or to generate a thumbnail image corresponding to the image. If considering the matrix form of the image, the image in the original s x s window is changed into a pixel, and the value of this pixel point is the average value or maximum value of all pixels in the window, etc.

[0215] In image processing, the principle of downsampling is as follows: for an image I with a size of MxN, s-fold downsampling is performed, so as to obtain a resolution image with a size of (M / s)x(N / s), where s should be a common divisor of M and N.

[0216] S1104: Color calibration is performed on the first image based on the three-channel image converted from the second image, so as to obtain first color parameters.

[0217] S1105: The first color parameters are sent to an ISP channel corresponding to the first camera, so that the first camera performs color adjustment on the image collected by the first camera according to the first color parameters.

[0218] In some embodiments, the electronic device further includes a second camera, which is a three-channel camera. The second camera can be color calibrated based on the high-resolution multispectral camera, and the color calibration process is the same as the color calibration process of the first camera.

[0219] In the embodiments of the present application, a camera array color calibration method based on a high-resolution multispectral camera is provided. In the method, the image collected by the high-resolution multispectral camera is a multi-channel image, the spatial color information is richer, and the authenticity of the corresponding image color is higher. Taking the image shot by the high-resolution multispectral camera as a color reference, the colors of the images shot by each camera in the camera array are calibrated, which can improve the authenticity and consistency of the colors of the images shot by the multiple cameras in the camera array. At the same time, the high-resolution multispectral camera has a small size, and the power consumption generated in the calibration process is low, which can reduce the architecture redundancy and calibration cost of the electronic device in the color management development process. The method provides a unified reference line for each camera and improves the debugging efficiency.

[0220] Exemplarily, Figure 12 A schematic flowchart of another camera color calibration method 1200 provided by the embodiments of the present application is shown. As Figure 12 shown, the electronic device includes a camera array, which includes a main camera, an ultra-wide-angle camera, and a telephoto camera. The method 1200 includes the following steps.

[0221] S1201 to S1205 are the process of color calibration of the main camera of the electronic device based on the high-resolution multispectral camera.

[0222] S1201: The main camera and the high-resolution multispectral camera of the electronic device are started at the same time, and the main camera of the electronic device collects a first image, which is an original image of a first scene.

[0223] The first image is a three-channel image, for example, an RGB image or an RYB image.

[0224] S1202: While the main camera is capturing the first image, the high-resolution multispectral camera of the electronic device captures a second image, which is a multispectral raw image of the first scene.

[0225] The second image is an M-channel image, where M is a positive integer greater than 3.

[0226] It can also be understood that the first image captured by the main camera is a three-channel image, and the second image captured by the high-resolution multispectral camera is a multi-channel image. The spatial color information of the second image is much richer than that of the first image. That is, the spectral information perceived by the high-resolution multispectral camera is richer than that perceived by the main camera, and is closer to the true color of the first scene.

[0227] S1203: Convert the second image from a multi-channel image to a three-channel image by using a first conversion function, where the first conversion function is a conversion function between the high-resolution multispectral camera and the main camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the main camera.

[0228] In a specific embodiment, the second image is spectrally down-sampled according to the first conversion function, so as to convert the second image from a multi-channel image to a three-channel image.

[0229] In an implementation, the second image is down-sampled from M (or k x k) channels to three channels according to the first conversion function, so as to convert the second image from a multi-channel image to a three-channel image. The first conversion function can be a function of matrix multiplication of a down-sampling matrix with a size of M x 3 (or k x 3) and the second image. 2

[0230] S1204: Color calibrate the first image based on the three-channel image converted from the second image, to obtain a first color parameter.

[0231] In some embodiments, the first color parameter can include any one or more of an auto white balance (AWB) parameter, a color adjustment (CA) parameter, a color calibration (CC) parameter, and a 3D look up table (3DLUT). In addition, the first color parameter can also include other color-related parameters, which are not limited in the present application. ​

[0232] S1205: The first color parameter is sent to the ISP channel corresponding to the main camera to enable the main camera to perform color adjustment on the image collected by the main camera according to the first color parameter.

[0233] In some embodiments, S1201 to S1204 are an offline calibration process. When the first color parameter corresponding to the main camera is obtained through the offline calibration process, when the main camera collects an image, the collected image is color-adjusted according to the first color parameter when the collected image is processed by the ISP corresponding to the main camera, so that the image output by the main camera is more realistic.

[0234] In yet some embodiments, S1201 to S1204 are an online calibration process. When a user collects an image through the main camera, the high-resolution multispectral camera is turned on synchronously when the main camera is turned on, and the high-resolution multispectral camera collects an image of the same scene at the same time when the main camera collects an image. The first color parameter is obtained through S1201 to S1204, and the collected image is color-adjusted according to the first color parameter when the collected image is processed by the ISP corresponding to the main camera, so that the image output by the main camera is more realistic.

[0235] In some embodiments, the color standard process of the main camera can occur in each process of collecting an image through the main camera, or can occur in response to the calibration operation of the user, or can be offline calibration (or online calibration) once after a certain number of images are shot, or can be calibration once every certain time length, or can be other calibration time, and the application does not limit the occurrence time of the color calibration of the camera, nor the calibration scene, which can be offline calibration or online calibration.

[0236] In the embodiments of the application, the second image contains color channels with a size of M, and the spectral information included is more abundant than the spectral information corresponding to the first image, so that the three-channel image obtained by mapping the second image to the color space in which the main camera can obtain images is closer to the true color of the first scene, and then the first image can be color-adjusted using the image obtained by down-sampling the second image channel, thereby improving the color accuracy and consistency of the image shot by the main camera.

[0237] S1206 to S1210 are a process of color calibration of the ultra-wide-angle camera of the electronic device based on the high-resolution multispectral camera.

[0238] S1206: simultaneously turn on the ultra-wide-angle camera and the high-resolution multispectral camera of the electronic device, and the ultra-wide-angle camera of the electronic device collects a third image, which is a raw image of the second scene.

[0239] The third image is a three-channel image, for example, an RGB image or an RYB image.

[0240] S1207: while the ultra-wide-angle camera collects the third image, the high-resolution multispectral camera of the electronic device collects a fourth image, which is a multispectral raw image of the second scene.

[0241] The fourth image is an M-channel image, where M is a positive integer greater than 3.

[0242] It can also be understood that the third image collected by the ultra-wide-angle camera is a three-channel image, and the fourth image collected by the high-resolution multispectral camera is a multi-channel image, and the spatial color information of the fourth image is much richer than that of the third image, that is, the spectral information perceived by the high-resolution multispectral camera is more abundant than that perceived by the ultra-wide-angle camera, and is closer to the true color of the second scene.

[0243] S1208: convert the fourth image from a multi-channel image to a three-channel image by using a second conversion function, where the second conversion function is a conversion function between the high-resolution multispectral camera and the ultra-wide-angle camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the ultra-wide-angle camera.

[0244] In a specific embodiment, the fourth image is spectrally down-sampled according to the second conversion function, so as to convert the fourth image from a multi-channel image to a three-channel image.

[0245] In some embodiments, the estimation method of the down-sampling matrix used by the second conversion function can be any one of a Bayesian method, a least squares method, a deep learning method, or other estimation methods.

[0246] In an implementation, the fourth image is down-sampled from M (or k x k) channels to three channels according to the second conversion function, so as to convert the fourth image from a multi-channel image to a three-channel image, where the second conversion function can be a function of matrix multiplication of a down-sampling matrix with a size of M x 3 (or k x 3) and the fourth image. 2

[0247] S1209: color calibrate the third image based on the three-channel image converted from the fourth image, to obtain a second color parameter.

[0248] ​Wherein, the explanation of the second color parameter is the same as the above-mentioned explanation of the first color parameter, and for the sake of brevity, it will not be repeated here.

[0249] S1210: issuing the second color parameter to the ISP channel corresponding to the ultra-wide-angle camera, so that the ultra-wide-angle camera performs color adjustment on the image collected by the ultra-wide-angle camera according to the second color parameter.

[0250] In some embodiments, the second scene and the first scene are the same, and the fourth image and the second image are the same image. That is, in some implementations, when the ultra-wide-angle camera and the main camera camera both collect (which can be synchronous collection or asynchronous collection within a certain time threshold) the original image of the first scene, S1207 can not be performed, and the high-resolution multispectral camera only needs to collect the multispectral original image of the first scene once, and the color calibration of the ultra-wide-angle camera and the main camera camera is realized through the multispectral original image of the first scene.

[0251] In some embodiments, S1206 to S1209 are an offline calibration process. When the second color parameter corresponding to the ultra-wide-angle camera is obtained through the offline calibration process, when the ultra-wide-angle camera collects an image, the collected image is color-adjusted according to the second color parameter when the collected image is processed by the ISP corresponding to the ultra-wide-angle camera, so that the authenticity of the image output by the ultra-wide-angle camera is higher.

[0252] In yet some embodiments, S1206 to S1209 are an online calibration process. When a user collects an image through the ultra-wide-angle camera, the high-resolution multispectral camera is turned on synchronously when the ultra-wide-angle camera is turned on, and the high-resolution multispectral camera collects an image of the same scene at the same time when the ultra-wide-angle camera collects an image. The second color parameter is obtained through the above-mentioned S1206 to S1209, and the collected image is color-adjusted according to the second color parameter when the collected image is processed by the ISP corresponding to the ultra-wide-angle camera, so that the authenticity of the image output by the ultra-wide-angle camera is higher.

[0253] In some embodiments, the color standard process of the ultra-wide-angle camera can occur in the process of collecting an image through the ultra-wide-angle camera each time, or in response to the calibration operation of the user, or offline calibration (or online calibration) once after shooting a certain number of images, or calibration once every certain time length; or other calibration timing. The application does not limit the timing of the color calibration of the camera, and does not limit the calibration scene, which can be offline calibration or online calibration.

[0254] In the embodiments of the present application, the fourth image contains a color channel number of M, and the included spectral information is more abundant than the spectral information corresponding to the third image. Therefore, when the fourth image is mapped to the color space in which the image captured by the ultra-wide-angle camera is located, the obtained three-channel image is closer to the real color of the second scene, and then the image after down-sampling of the fourth image channel can be used to adjust the color of the third image, thereby improving the color accuracy and consistency of the image captured by the ultra-wide-angle camera.

[0255] S1211 to S1215 are processes of color calibration of the long-focus camera of the electronic device based on the high-resolution multispectral camera.

[0256] S1211: simultaneously turn on the long-focus camera and the high-resolution multispectral camera of the electronic device, and the long-focus camera of the electronic device captures a fifth image, which is an original image of the third scene.

[0257] The fifth image is a three-channel image, for example, it can be an RGB image, and it can also be an RYB image.

[0258] S1212: while the long-focus camera is capturing the fifth image, the high-resolution multispectral camera of the electronic device captures a sixth image, which is a multispectral original image of the third scene.

[0259] The sixth image is an M-channel image, where M is a positive integer greater than 3.

[0260] It can also be understood that the fifth image captured by the long-focus camera is a three-channel image, and the sixth image captured by the high-resolution multispectral camera is a multi-channel image, and the spatial color information possessed by the sixth image is much richer than the spatial color information possessed by the fifth image, that is, the spectral information perceived by the high-resolution multispectral camera is more abundant than the spectral information perceived by the long-focus camera, and it is closer to the real color of the third scene.

[0261] S1213: convert the sixth image from a multi-channel image to a three-channel image by using a third conversion function, where the third conversion function is a conversion function between the high-resolution multispectral camera and the long-focus camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the long-focus camera.

[0262] In a specific embodiment, according to the third conversion function, the spectral band down-sampling is performed on the sixth image, so as to convert the sixth image from a multi-channel image to a three-channel image.

[0263] In some embodiments, the estimation method of the down-sampling matrix used by the third conversion function can be any one of Bayesian method, least square method, deep learning method, and other estimation methods.

[0264] In some embodiments, the first / second / third conversion function described above can be understood as a first / second / third mapping parameter.

[0265] In an implementation, the third conversion function is a function of performing matrix multiplication between a down-sampling matrix with a size of Mx3 (or kx3) and the sixth image. 2

[0266] S1214: Color calibration is performed on the fifth image based on the three-channel image converted from the sixth image, to obtain a third color parameter.

[0267] Wherein, the explanation of the third color parameter is the same as the explanation of the first color parameter described above, and for brevity, will not be repeated here.

[0268] S1215: The third color parameter is issued to the ISP channel corresponding to the long-focus camera, so that the long-focus camera performs color adjustment on the image collected by the long-focus camera according to the third color parameter.

[0269] In some embodiments, the third scene and the first scene are the same, and the sixth image and the second image are the same image. That is, in some implementations, when the long-focus camera and the main camera both collect (may be synchronous collection, or may be asynchronous collection within a certain time threshold) the original image of the first scene, S1212 can not be performed, and the high-resolution multispectral camera only needs to collect the multispectral original image of the first scene once, and the color calibration of the long-focus camera and the main camera is realized through the multispectral original image of the first scene.

[0270] In yet some embodiments, the third scene and the second scene are the same, and the sixth image and the fourth image are the same image. That is, in some implementations, when the long-focus camera and the ultra-wide-angle camera both collect (may be synchronous collection, or may be asynchronous collection within a certain time threshold) the original image of the second scene, S1212 can not be performed, and the high-resolution multispectral camera only needs to collect the multispectral original image of the second scene once, and the color calibration of the long-focus camera and the ultra-wide-angle camera is realized through the multispectral original image of the second scene.

[0271] ​In yet some embodiments, the third scene, the second scene and the first scene are all the same, and the sixth image, the fourth image and the second image are all the same image. That is, in some implementations, when the ultra-wide-angle camera, the long-focus camera and the main camera all capture (which can be synchronous capture or asynchronous capture within a certain time threshold) the original image of the first scene, S1207 and S1212 can not be performed, and the high-resolution multispectral camera only needs to capture the multispectral original image of the first scene once, and the color calibration of the long-focus camera, the ultra-wide-angle camera and the main camera is realized through the multispectral original image of the first scene.

[0272] In some embodiments, S1211 to S1214 are an offline calibration process. When the third color parameter corresponding to the long-focus camera is obtained through the offline calibration process, when the long-focus camera captures an image, the captured image is color-adjusted according to the third color parameter when the captured image is processed by the ISP corresponding to the long-focus camera, so that the authenticity of the image output by the long-focus camera is higher.

[0273] In yet some embodiments, S1211 to S1214 are an online calibration process. When the user captures an image through the long-focus camera, the high-resolution multispectral camera is turned on synchronously when the long-focus camera is turned on, and the high-resolution multispectral camera captures an image of the same scene at the same time when the long-focus camera captures an image. The third color parameter is obtained through S1211 to S1214, and the captured image is color-adjusted according to the third color parameter when the captured image is processed by the ISP corresponding to the long-focus camera, so that the authenticity of the image output by the long-focus camera is higher.

[0274] In some embodiments, the color standard process of the long-focus camera can occur in the process of capturing an image through the long-focus camera each time, or in response to the user's calibration operation, or offline calibration (or online calibration) once after shooting a certain number of images, or calibration once every certain time interval; or other calibration timing. The application does not limit the timing of the color calibration of the camera, nor does it limit the calibration scene, which can be offline calibration or online calibration.

[0275] In some embodiments, the color standard process of the long-focus camera can occur in the process of capturing an image through the long-focus camera each time, or in response to the user's calibration operation, or offline calibration (or online calibration) once after shooting a certain number of images, or calibration once every certain time interval; or other calibration timing. The application does not limit the timing of the color calibration of the camera, nor does it limit the calibration scene, which can be offline calibration or online calibration.

[0276] In the embodiment of the present application, since the image collected by the high-resolution multispectral camera is a multi-channel image, the spatial color information is richer, and the authenticity of the corresponding image color is higher. Taking the image shot by the high-resolution multispectral camera as a color reference, the colors of the images shot by each camera in the camera array are calibrated respectively, so that the colors of the images collected by each camera are aligned with the color perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images shot by the multiple cameras in the camera array.

[0277] Exemplarily, Figure 13 A system framework schematic diagram corresponding to the camera color calibration method provided in the embodiment of the present application is shown.

[0278] As Figure 13 shown, the electronic device includes a main camera, a high-resolution multispectral camera, a color parameter determination module, and an image signal processor (ISP). The system framework is used to perform the following steps:

[0279] S1301: The main camera and the high-resolution multispectral camera of the electronic device are started at the same time, and the main camera collects a first RAW image, which is a RAW image of a first scene.

[0280] The first RAW image is a three-channel image, for example, it can be an RGB image or an RYB image.

[0281] S1302: While the main camera is collecting the first RAW image, the high-resolution multispectral camera collects a second RAW image, which is a RAW image of the first scene.

[0282] The second RAW image is an M-channel image, where M is a positive integer greater than 3.

[0283] S1303: The color parameter determination module determines a first color parameter according to the first RAW image and the second RAW image, and delivers the first color parameter to the ISP channel corresponding to the main camera.

[0284] Specifically, the color parameter determination module converts the second RAW image from an M-channel image to a three-channel image through a first conversion function, where the first conversion function is a conversion function between the high-resolution multispectral camera and the main camera, used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the main camera; then the three-channel image converted from the second RAW image is taken as a reference to perform color calibration on the first RAW image, and the first color parameter is obtained.

[0285] The explanation of the first color parameter has been given in Figure 11The embodiments shown are described in detail, and for brevity, will not be repeated here.

[0286] S1304: The main camera adjusts the color of the image collected by the main camera according to the first color parameter.

[0287] In some embodiments, the color calibration of the high-resolution multispectral camera to the main camera is online calibration. In the process of processing the first RAW image through the ISP channel corresponding to the main camera, the first RAW image is color adjusted according to the first color parameter, and the color-adjusted image, i.e., the main camera image, is output.

[0288] In yet some embodiments, the color calibration of the high-resolution multispectral camera to the main camera is offline calibration. After obtaining the first color parameter, when an image is collected by the main camera, in the process of processing the collected image through the ISP channel corresponding to the main camera, the collected image is color adjusted according to the first color parameter, and the color-adjusted image, i.e., the main camera image, is output.

[0289] Further, the electronic device further includes a super-wide-angle camera, and the color calibration of the super-wide-angle camera of the electronic device can be performed based on the high-resolution multispectral camera. The system architecture diagram corresponding to the color calibration process is similar to the system architecture diagram of the color calibration of the main camera of the electronic device based on the high-resolution multispectral camera, and for brevity, will not be repeated here.

[0290] Further, the electronic device further includes a long-focus camera, and the color calibration of the long-focus camera of the electronic device can be performed based on the high-resolution multispectral camera. The system architecture diagram corresponding to the color calibration process is similar to the system architecture diagram of the color calibration of the main camera of the electronic device based on the high-resolution multispectral camera, and for brevity, will not be repeated here. In the embodiments of the present application, the image captured by the high-resolution multispectral camera is taken as the color reference, and the colors of the images captured by each camera in the camera array are calibrated respectively, so that the colors of the images captured by each camera are aligned to the color perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by multiple cameras.

[0291] Exemplarily, Figure 14 A flowchart of another camera color calibration method 1400 provided by the embodiments of the present application is shown. As Figure 14 shown, the method 1400 includes:

[0292] S1401 and S1402 are the same as S1301 and S1302, Figure 13S1301 and S1302 in the illustrated embodiment are the same, and for brevity, will not be repeated here.

[0293] S1403: FOV alignment is performed on the first RAW image and the second RAW image.

[0294] In an implementation, the FOV alignment is performed on the first RAW image and the second RAW image in a manner of calibration.

[0295] S1404: based on the second RAW image after FOV alignment, a first conversion function is estimated, which can also be understood as a spectral band mapping parameter between the high-resolution multispectral camera and the main camera.

[0296] In some embodiments, the first conversion function is a down-sampling matrix for down-sampling from the second RAW image after FOV alignment to the first RAW image after FOV alignment.

[0297] Wherein, the method of using the down-sampling matrix to estimate the first conversion function has been described in detail in the foregoing embodiments, and for brevity, will not be repeated here.

[0298] 1405: through the first conversion function, the second RAW image after FOV alignment is converted from a multi-channel image to a three-channel image.

[0299] It should be understood that the three-channel image converted from the second RAW image has higher color authenticity than the three-channel image captured by the main camera.

[0300] In an implementation, M=k x k, according to the first conversion function, spectral band down-sampling is performed from k x k channels corresponding to the second RAW image to three channels, thereby realizing the conversion of the second RAW image from a multi-channel image to a three-channel image, wherein the first conversion function is a function of using a down-sampling matrix with a size of k 2 x 3 to perform matrix multiplication on the second image, and the value of k is a positive integer greater than or equal to 3.

[0301] S1406: based on the three-channel image converted from the second RAW image after FOV alignment, statistical correction is performed on the first RAW image after FOV alignment, and a first color parameter corresponding to the main camera is obtained.

[0302] Wherein, the explanation of the first color parameter has been described in detail in the foregoing embodiments, and for brevity, will not be repeated here. Figure 11

[0303] ​In some embodiments, the first RAW image after alignment is statistically corrected, and the statistical correction method used can include white point correction, etc.

[0304] S1407: The first color parameter is sent to the ISP channel corresponding to the main camera lens.

[0305] S1408: In the ISP of the first camera lens, the first RAW image after FOV alignment is color adjusted according to the first color parameter, and the color-adjusted image, i.e., the main camera lens image, is output.

[0306] In some embodiments, after obtaining the first color parameter in the ISP of the first camera lens, the main camera lens performs color adjustment on the image collected by the main camera lens according to the first color parameter.

[0307] Further, the electronic device also includes a super-wide-angle camera, and the color calibration of the super-wide-angle camera of the electronic device can be performed based on the high-resolution multispectral camera. The color calibration process can be similar to the process of color calibration of the main camera lens of the electronic device based on the high-resolution multispectral camera (S1401 to S1408) described above. For brevity, details are not repeated here.

[0308] Further, the electronic device also includes a long-focus camera, and the color calibration of the long-focus camera of the electronic device can be performed based on the high-resolution multispectral camera. The color calibration process is similar to the process of color calibration of the main camera lens of the electronic device based on the high-resolution multispectral camera (S1401 to S1408) described above. For brevity, details are not repeated here.

[0309] In the embodiments of the present application, the mapping relationship between the high-spectral-band resolution image obtained by the high-resolution multispectral camera and the low-spectral-band resolution (RGB three-channel) image of each array camera can be constructed respectively. Based on the mapping relationship, the high-spectral-band resolution image is down-sampled to a low-spectral-band resolution image, a three-channel image with higher color authenticity corresponding to the low-spectral-band resolution image is obtained, the low-spectral-band resolution image is calibrated through the three-channel image with higher color authenticity, the color parameter corresponding to each array camera can be obtained respectively, and then each array camera adjusts the color of the captured image according to the corresponding color parameter, so that the colors of the images captured by each camera are aligned with the colors perceived by the high-resolution multispectral camera, and the authenticity and consistency of the colors of the images captured by the multiple cameras in the camera array can be improved.

[0310] Exemplarily, Figure 15 A flowchart corresponding to another camera color calibration method 1500 provided by the embodiments of the present application is shown. As Figure 15As shown, the method 1500 includes:

[0311] S1501: simultaneously turn on the main camera and the high-resolution multispectral camera, and the main camera collects a first RAW image, the first RAW image being a RAW image of a first scene.

[0312] Wherein, the explanation of this step is the same as that of S1301 in the embodiment shown in Figure 13 The explanation of S1301 in the embodiment shown is the same as that in the embodiment shown in the embodiment shown, and for the sake of brevity, will not be repeated here.

[0313] S1502: generate a first response curve corresponding to the first RAW image.

[0314] Wherein, the first response curve can be understood as a spectral perception curve corresponding to the first RAW image, and for details, please refer to the embodiment shown in Figure 10

[0315] S1503: while the main camera collects the first RAW image, the high-resolution multispectral camera collects a second RAW image, the second RAW image being a RAW image of the first scene.

[0316] Wherein, the explanation of this step is the same as that of S1302 in the embodiment shown in Figure 13 The explanation of S1302 in the embodiment shown is the same as that in the embodiment shown, and for the sake of brevity, will not be repeated here.

[0317] S1504: generate a second response curve corresponding to the second RAW image.

[0318] Wherein, the second response curve can be understood as a spectral perception curve corresponding to the second RAW image, and for details, please refer to the embodiment shown in Figure 10

[0319] S1505: based on the second response curve, perform spectral band super-resolution on the second RAW image to obtain a hyperspectral image corresponding to the second RAW image.

[0320] It should be understood that the number of spectral bands corresponding to the hyperspectral image corresponding to the second RAW image is much larger than the number of spectral bands corresponding to the second RAW image.

[0321] In some embodiments, the second RAW image corresponds to M spectral bands, and the hyperspectral image corresponding to the second RAW image corresponds to N spectral bands, wherein N is much larger than M, and when M takes a value of k 2 , N>>k 2 .

[0322] ​​In some embodiments, the method of spectral band super-resolution of the second RAW image comprises a compressive sensing based super-resolution reconstruction method and / or a neural network based spectral band super-resolution method, and can also be other spectral band super-resolution methods, which are not limited in the present application.

[0323] S1506: Determine a first conversion function according to the hyperspectral image corresponding to the second RAW image and the first response curve, which can also be understood as a spectral band mapping parameter between the high-resolution multi-spectral camera and the main camera.

[0324] In some embodiments, the first conversion function is a function of matrix multiplication of a down-sampling matrix for down-sampling from the hyperspectral image corresponding to the second RAW image to the first RAW image and the second image.

[0325] S1507: Convert the second RAW image into a three-channel image through the first conversion function.

[0326] It should be understood that the three-channel image converted from the second RAW image has higher color authenticity than the three-channel image captured by the main camera.

[0327] In an implementation manner, based on a down-sampling matrix with a size of N x 3, spectral band down-sampling is performed from N channels corresponding to the hyperspectral image corresponding to the second RAW image to three channels, so as to convert the hyperspectral image corresponding to the second RAW image into a three-channel image, thereby realizing conversion of the second RAW image from a multi-channel image into a three-channel image, and N is much larger than M.

[0328] S1508: Take the converted three-channel image as a reference to statistically correct the first RAW image, and obtain the first color parameter corresponding to the main camera.

[0329] Wherein, the explanation of the first color parameter has been described in detail in the embodiment shown in Figure 11 For the sake of brevity, it will not be repeated here.

[0330] S1509: Issue the first color parameter to the ISP channel corresponding to the main camera.

[0331] S1510: In the ISP of the first camera, color adjust the first RAW image according to the first color parameter, and output the color adjusted image, that is, the main camera image.

[0332] In some embodiments, after obtaining the first color parameter in the ISP of the main camera, the main camera color adjusts the image captured by the main camera according to the first color parameter.

[0333] Further, the electronic device further comprises a super wide-angle camera, and the color calibration of the super wide-angle camera of the electronic device can be performed based on the high-resolution multi-spectrum camera. The color calibration process can be similar to the color calibration of the main camera of the electronic device based on the high-resolution multi-spectrum camera (S1501 to S1510), and details are not repeated here for brevity.

[0334] Further, the electronic device further comprises a long-focus camera, and the color calibration of the long-focus camera of the electronic device can be performed based on the high-resolution multi-spectrum camera. The color calibration process can be similar to the color calibration of the main camera of the electronic device based on the high-resolution multi-spectrum camera (S1501 to S1510), and details are not repeated here for brevity.

[0335] In an implementation, the main camera is calibrated online in the first shooting process to obtain a first color parameter, and the image obtained in the first shooting is color adjusted based on the first color parameter. Thereafter, when the main camera shoots an image, the image is directly color adjusted based on the first color parameter.

[0336] In another implementation, the main camera obtains a first color parameter through online calibration or offline calibration in a first time period, and the images shot in the first time period are color adjusted based on the first color parameter. The main camera obtains a second color parameter through online calibration or offline calibration in a second time period, and the images shot in the second time period are color adjusted based on the second color parameter.

[0337] In another implementation, the main camera obtains a first color parameter through offline calibration in advance, and when the main camera shoots an image, the image is color adjusted based on the first color parameter.

[0338] It should be understood that the multi-spectrum camera (or the high-resolution multi-spectrum camera described above, or the second camera described above) in the embodiments of the present application can be a color filter array (CFA) that is differentiated by chemical dye, or a CFA that is differentiated by film coating interference, or a CFA that is differentiated by a super surface winnower structure, or other sensor function implementation, which is not limited in the present application.

[0339] It should also be understood that the multispectral camera (or the high-resolution multispectral camera described above, or the second camera described above) in the embodiments of the present application can be independent of the existing camera (for example: wide-angle, ultra-wide-angle, long-focus, etc.) of the electronic device in terms of module function implementation, and can also be in a certain structural appearance with the existing camera, for example, a hybrid structural appearance. It can also be other module function implementation manners, which are not limited by the present application.

[0340] The color parameter described above can include one or more parameters corresponding to a color module in the ISP channel of the camera, and can also include one or more parameters corresponding to a brightness module in the ISP channel of the camera.

[0341] One or more of the modules or units described herein can be implemented in software, hardware, or a combination of both. When any of the above modules or units are implemented in software, the software exists in the form of computer program instructions and is stored in a memory, and a processor can be used to execute the program instructions and implement the above method flow. The processor can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, and various computing devices running software, each of which can include one or more cores for executing software instructions to perform operations or processing. The processor can be built into a SoC (system on chip) or an application specific integrated circuit (ASIC), or can be a separate semiconductor chip. In addition to the core for executing software instructions to perform operations or processing, the processor can further include necessary hardware accelerators, such as a field programmable gate array (FPGA), a PLD (programmable logic device), or a logic circuit implementing special logic operations.

[0342] When the modules or units described herein are implemented in hardware, the hardware can be any one or any combination of a CPU, a microprocessor, a DSP, an MCU, an artificial intelligence processor, an ASIC, a SoC, an FPGA, a PLD, a dedicated digital circuit, a hardware accelerator, or a non-integrated discrete device, which can run necessary software or be independent of software to execute the above method flow.

[0343] When the modules or units described in the specification are implemented by using software, the modules or units can be implemented in a form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded on a computer, the whole or part of the flow or function described in the embodiments of the present application is generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

[0344] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0345] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0346] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be realized by other ways. For example, the above described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0347] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0348] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0349] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0350] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for camera color calibration, the method comprising: The method is applied to an electronic device including a camera array and a high-resolution multispectral camera, and the method includes: In a case where the first camera and the high-resolution multispectral camera are turned on at the same time, a first image is captured by the first camera, the first image is an original image of a first scene, and the first image is a three-channel image, and the camera array includes the first camera; A second image is synchronously captured by the high-resolution multispectral camera, the second image is a multispectral original image of the first scene, and the second image is an M-channel image, where M is a positive integer greater than 3; The second image is converted from the M-channel image to a three-channel image by a first conversion function; Color calibration is performed on the first image based on the three-channel image converted from the second image, to obtain a first color parameter; The first color parameter is sent to an image signal processor (ISP) channel of the first camera, so that the first camera performs color adjustment on an image captured by the first camera according to the first color parameter.

2. The method of claim 1, wherein, The camera array further includes a second camera, and the method further includes: A third image is captured by the second camera, the third image is an original image of the first scene, and the third image is a three-channel image; Color calibration is performed on the third image based on the three-channel image converted from the second image, to obtain a second color parameter; The second color parameter is sent to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on an image captured by the second camera according to the second color parameter.

3. The method of claim 1, wherein, The camera array further includes a second camera, and the method further includes: In a case where the second camera and the high-resolution multispectral camera are turned on at the same time, a third image is captured by the second camera, the third image is an original image of a second scene, and the third image is a three-channel image; A fourth image is synchronously captured by the high-resolution multispectral camera, the fourth image is a multispectral original image of the second scene, and the fourth image is an M-channel image; The fourth image is converted from the M-channel image to a three-channel image by a second conversion function; Color calibration is performed on the third image based on the three-channel image converted from the fourth image, to obtain a second color parameter; The second color parameter is sent to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on an image captured by the second camera according to the second color parameter.

4. The method according to claim 2 or 3, characterized in that, The first camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera, the second camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera, and the first camera and the second camera are different.

5. The method according to any one of claims 1 to 3, characterized in that, The conversion of the second image from the M-channel image to the three-channel image by the first conversion function includes: The second image is converted from the M-channel image to the three-channel image by a first conversion function, wherein the first conversion function is a function of matrix multiplication of the second image using a down-sampling matrix with a size of M*3.

6. The method of claim 5, wherein, Before the second image is converted from the M-channel image to the three-channel image by the first conversion function, the method further comprises: aligning a field of view (FOV) of the first image and the second image.

7. The method of claim 6, wherein, The method further comprises: determining the first conversion function based on the second image after the FOV alignment.

8. The method according to any one of claims 1 to 3, characterized in that, The conversion of the second image from the M-channel image to the three-channel image by the first conversion function comprises: performing spectral band super-resolution on the second image according to a spectral response curve corresponding to the second image to obtain a hyperspectral image corresponding to the second image, the hyperspectral image corresponding to the second image being an N-channel image, wherein N is a positive integer much larger than M; the hyperspectral image corresponding to the second image is converted from the N-channel image to the three-channel image by a first conversion function, wherein the first conversion function is a function of matrix multiplication of the hyperspectral image corresponding to the second image using a down-sampling matrix with a size of N*3.

9. The method of claim 8, wherein, The method further comprises: determining the first conversion function based on the hyperspectral image corresponding to the second image and the spectral response curve corresponding to the first image.

10. The method according to any one of claims 1 to 3, characterized in that, The first color parameter includes any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

11. An electronic device, comprising: The electronic device includes a camera array, a high-resolution multi-spectral camera, and a processor, the camera array including a first camera, wherein the first camera is configured to capture a first image when the first camera and the high-resolution multi-spectral camera are both turned on, the first image being a raw image of a first scene, the first image being a three-channel image; the high-resolution multi-spectral camera is configured to synchronously capture a second image, the second image being a multi-spectral raw image of the first scene, the second image being an M-channel image, wherein M is a positive integer greater than 3; the processor is configured to convert the second image from the M-channel image to the three-channel image by a first conversion function; the processor is further configured to perform color calibration on the first image based on the three-channel image converted from the second image to obtain a first color parameter; the processor is further configured to send the first color parameter to an image signal processor (ISP) channel of the first camera, so that the first camera can perform color adjustment on an image captured by the first camera according to the first color parameter.

12. The electronic device of claim 11, wherein, The camera array further includes a second camera, wherein: the second camera is configured to capture a third image, the third image being a raw image of the first scene, the third image being a three-channel image; The processor is further configured to perform color calibration on the third image based on the three-channel image converted from the second image, to obtain a second color parameter. The processor is further configured to distribute the second color parameter to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on an image captured by the second camera according to the second color parameter.

13. The electronic device of claim 11, wherein, The camera array further includes a second camera, wherein: The second camera is configured to capture a third image by the second camera when the second camera and the high-resolution multi-spectral camera are both turned on, the third image being an original image of a second scene, and the third image being a three-channel image. The high-resolution multi-spectral camera is further configured to synchronously capture a fourth image, the fourth image being a multi-spectral original image of the second scene, and the fourth image being an M-channel image. The processor is further configured to convert the fourth image from the M-channel image to a three-channel image by using a second conversion function. The processor is further configured to perform color calibration on the third image based on the three-channel image converted from the fourth image, to obtain a second color parameter. The processor is further configured to distribute the second color parameter to an image signal processor (ISP) channel of the second camera, so that the second camera performs color adjustment on an image captured by the second camera according to the second color parameter.

14. The electronic device of claim 12 or 13, wherein, The first camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera, the second camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera, and the first camera and the second camera are different.

15. The electronic device of any of claims 11-13, wherein, The processor is specifically configured to: convert the second image from the M-channel image to a three-channel image by down-sampling from M channels of the second image to three channels of the first image according to the first conversion function, wherein the first conversion function is a function of performing matrix multiplication on the second image by using a down-sampling matrix with a size of M*3.

16. The electronic device of claim 15, wherein, The processor is further specifically configured to: align a field of view (FOV) of the first image and the second image before converting the second image from the M-channel image to a three-channel image by using the first conversion function.

17. The electronic device of claim 16, wherein, The processor is further configured to: determine the first conversion function based on the second image after the FOV alignment.

18. The electronic device of any of claims 11-13, wherein, The processor is specifically configured to: perform spectral band super-resolution on the second image according to a spectral response curve corresponding to the second image, to obtain a hyperspectral image corresponding to the second image, the hyperspectral image corresponding to the second image being an N-channel image, wherein N is a positive integer much larger than M; convert the hyperspectral image corresponding to the second image from the N-channel image to a three-channel image by down-sampling from N channels of the hyperspectral image corresponding to the second image to three channels of the first image according to the first conversion function, wherein the first conversion function is a function of performing matrix multiplication on the second image by using a down-sampling matrix with a size of N*3.

19. The electronic device of claim 18, wherein, The processor is further configured to: determine the first conversion function according to a hyperspectral image corresponding to the second image and a spectral response curve corresponding to the first image.

20. The electronic device of any of claims 11-13, wherein, The first color parameter includes any one or more of an auto white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D look-up table.

21. An electronic device, comprising: comprise: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 1-10.

22. A computer-readable storage medium, characterized in that, The storage medium has stored therein a program or instructions, which, when executed by a processor, implement the method of any one of claims 1-10.

23. A chip, characterized by The chip includes a circuit configured to perform the method of any one of claims 1-10.

24. A computer program product, characterised in that, The computer program product has stored therein a program or instructions, which, when executed by a processor, implement the method of any one of claims 1-10.

Images