Camera color calibration method and apparatus, and electronic device

Abstract

Provided in the present application is a camera color calibration method applied to an electronic device, wherein the electronic device comprises a camera array (comprising a first camera) and a high-resolution multi-spectral camera. The method comprises: synchronously collecting original images of a first scene by means of a first camera and a high-resolution multi-spectral camera, wherein the first camera collects a three-channel image (a first image), and the high-resolution multi-spectral camera synchronously collects an M-channel image (a second image), M being a positive integer greater than or equal to 3; by means of a first conversion function, converting the second image from the M-channel image into a three-channel image; and with the three-channel image obtained by means of conversion as a reference, performing color calibration on the first image, so as to obtain a first color parameter used by the first camera to perform color adjustment. In the method, the colors captured by array cameras are all aligned with colors perceived by a high-resolution multi-spectral camera, such that the authenticity and consistency of the colors of images captured by the cameras in the camera array can be improved.

Description

Camera color calibration method, device and electronic device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 29, 2024, with application number 202410390097.5 and application name “Camera color calibration method, device and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Embodiments of the present application relate to the field of camera color calibration, and more specifically, to a method, apparatus, and electronic device for camera color calibration. Background Art

[0003] Taking mobile phones as an example, smartphone cameras are typically presented in the form of a camera array, which includes multiple cameras. These cameras differ primarily in their focal lengths, primarily categorized as ultra-wide-angle, wide-angle (main), and telephoto. While different phones may have different camera configurations, they all have at least two cameras, capable of independent imaging and recording. The captured images or videos are directly presented to the user. However, images captured by different cameras in the same scene can easily exhibit significant color differences, significantly impacting the user's shooting experience. Summary of the Invention

[0004] The present application provides a method, device and electronic device for camera color calibration. Through this method, device and electronic device, the colors of images captured by cameras in a camera array are adjusted using images captured by a high-resolution multispectral camera as a color reference. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity of the colors of images captured by the cameras in the camera array.

[0005] In a first aspect, a method for camera color calibration is provided, which is applied to an electronic device, the electronic device including a camera array and a high-resolution multispectral camera, the method comprising: when a first camera and the high-resolution multispectral camera are simultaneously turned on, capturing a first image through the first camera, the first image being an original image of a first scene, the first image being a three-channel image, and the camera array including the first camera; synchronously capturing a second image through the high-resolution multispectral camera, the second image being a multispectral original image of the first scene, the second image being an M-channel image, wherein M is a positive integer greater than 3; converting the second image from an M-channel image to a three-channel image through a first conversion function; color calibrating the first image based on the three-channel image converted from the second image to obtain first color parameters; and sending the first color parameters to an image signal processor (ISP) path of the first camera, so that the first camera performs color adjustment on the image captured by the first camera according to the first color parameters.

[0006] In some embodiments, the first color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0007] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as a color reference to adjust the color of the image captured by the cameras in the camera array, so that the color of the array camera is aligned with the color perceived by the high-resolution multispectral camera, which can improve the authenticity of the color of the image captured by the cameras in the camera array.

[0008] In combination with the first aspect, in a possible implementation, the camera array also includes a second camera, and the method further includes: capturing a third image through the second camera, where the third image is the original image of the first scene, and the third image is a three-channel image; performing color calibration on the third image based on the three-channel image converted from the second image to obtain second color parameters; and sending the second color parameters to the image signal processor ISP channel of the second camera, so that the second camera adjusts the color of the image captured by the second camera according to the second color parameters.

[0009] In some embodiments, the first camera, the second camera, and the high-resolution multispectral camera synchronously capture images of the first scene.

[0010] In some further embodiments, the first camera, the second camera, and the high-resolution multispectral camera asynchronously capture images of the first scene within a certain time threshold.

[0011] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference, and the color of the image captured by each camera in the camera array is adjusted separately. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0012] In combination with the first aspect, in a possible implementation, the camera array also includes a second camera, and the method also includes: when the second camera and the high-resolution multispectral camera are turned on at the same time, collecting a third image through the second camera, the third image is the original image of the second scene, and the third image is a three-channel image; synchronously collecting a fourth image through the high-resolution multispectral camera, the fourth image is the multispectral original image of the second scene, and the fourth image is an M-channel image; converting the fourth image from an M-channel image to a three-channel image through a second conversion function; color calibrating the third image based on the three-channel image converted from the fourth image to obtain second color parameters; and sending the second color parameters to the image signal processor ISP channel of the second camera, so that the second camera adjusts the color of the image collected by the second camera according to the second color parameters.

[0013] In some embodiments, the second color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0014] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference, and the color of the image captured by each camera in the camera array is adjusted separately. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0015] In combination with the first aspect, in a possible implementation, the camera array also includes a third camera, and the method also includes: when the third camera and the high-resolution multispectral camera are turned on at the same time, collecting a fifth image through the third camera, the fifth image is the original image of the first scene, and the fifth image is a three-channel image; synchronously collecting a sixth image through the high-resolution multispectral camera, the sixth image is the multispectral original image of the first scene, and the sixth image is an M-channel image, where M is a positive integer greater than 3; converting the sixth image from an M-channel image to a three-channel image through a third conversion function; color calibrating the fifth image based on the three-channel image converted from the sixth image to obtain third color parameters; and sending the third color parameters to the image signal processor ISP channel of the third camera, so that the third camera adjusts the color of the image collected by the third camera according to the third color parameters.

[0016] In some embodiments, the third color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0017] In some embodiments, the first camera is one of the main camera, the ultra-wide-angle camera, and the telephoto camera; the second camera is one of the main camera, the ultra-wide-angle camera, and the telephoto camera; and the third camera is one of the main camera, the ultra-wide-angle camera, and the telephoto camera, wherein the first camera, the second camera, and the third camera are different from each other.

[0018] In some embodiments, the first camera is a main camera (wide-angle camera), the second camera is an ultra-wide-angle camera, and the third camera is a telephoto camera.

[0019] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference, and the color of the image captured by each camera in the camera array is adjusted separately. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0020] In combination with the first aspect, in one possible implementation, the second image is converted from an M-channel image to a three-channel image through a first conversion function, including: downsampling from the M channels of the second image to the three channels of the first image according to the first conversion function, converting the second image from an M-channel image to a three-channel image, wherein the first conversion function is a function that uses a downsampling matrix of size M×3 to perform matrix multiplication with the second image.

[0021] In some embodiments, the further explanation of “converting the fourth image from an M-channel image to a three-channel image through a second conversion function” is similar to the above explanation of “converting the second image from an M-channel image to a three-channel image through a first conversion function”.

[0022] In some embodiments, the further explanation of “converting the sixth image from an M-channel image to a three-channel image through a third conversion function” is similar to the above explanation of “converting the second image from an M-channel image to a three-channel image through the first conversion function”.

[0023] In an embodiment of the present application, the M channels of the second image are downsampled to 3 channels through a downsampling matrix, and a mapping relationship is established between the second image and the color space where the image can be obtained by the first camera. Since the second image contains a number of color channels of size M, the spectral information included is richer than the spectral information corresponding to the first image, so it is mapped to the color space where the image can be obtained by the first camera. The obtained three-channel image is closer to the true color of the first scene, and then the image after the downsampling of the second image channel can be used to adjust the color of the first image, thereby improving the color authenticity and consistency of the image taken by the first camera.

[0024] In combination with the first aspect, in a possible implementation, before converting the second image from an M-channel image to a three-channel image through a first conversion function, the method further includes: aligning the perspective FOV of the first image and the second image.

[0025] In some embodiments, before converting the fourth image from an M-channel image to a three-channel image using the second conversion function, the method further includes: aligning the third image and the fourth image by performing a field of view (FOV).

[0026] In some embodiments, before converting the sixth image from an M-channel image to a three-channel image using a third conversion function, the method further includes: aligning the fifth image and the sixth image by performing a field of view (FOV).

[0027] In an embodiment of the present application, before converting the second image from an M-channel image to a three-channel image (i.e., establishing a mapping relationship between the second image and the color space in which the image can be obtained by the first camera), the first image and the second image are first aligned in perspective, thereby avoiding the impact on the authenticity and consistency of the color adjustment of the first image due to the perspective difference between the first image and the second image.

[0028] In combination with the first aspect, in a possible implementation manner, the method further includes: determining the first conversion function based on the second image after FOV alignment.

[0029] In some embodiments, the method further includes determining the second conversion function based on the FOV-aligned fourth image.

[0030] In some embodiments, the method further includes determining the third conversion function based on the FOV-aligned sixth image.

[0031] In some embodiments, the estimation method of the downsampling matrix used by the first conversion function can be any one of the Bayesian method, the least squares method, the deep learning method, or other estimation methods.

[0032] In combination with the first aspect, in one possible implementation method, the second image is converted from an M-channel image to a three-channel image through a first conversion function, including: performing spectral band super-resolution on the second image according to the spectral sensing curve corresponding to the second image to obtain a hyperspectral image corresponding to the second image, where the hyperspectral image corresponding to the second image is an N-channel image, where N is a positive integer much larger than M; downsampling from the N channels of the hyperspectral image corresponding to the second image to the three channels of the first image according to the first conversion function, converting the hyperspectral image corresponding to the second image from an N-channel image to a three-channel image, where the first conversion function is a function that performs matrix multiplication on the second image using a downsampling matrix of size N×3.

[0033] In some embodiments, the further explanation of “converting the fourth image from an M-channel image to a three-channel image through a second conversion function” is similar to the above explanation of “converting the second image from an M-channel image to a three-channel image through a first conversion function”.

[0034] In some embodiments, the further explanation of “converting the sixth image from an M-channel image to a three-channel image through a third conversion function” is similar to the above explanation of “converting the second image from an M-channel image to a three-channel image through the first conversion function”.

[0035] In an embodiment of the present application, the N channels of the hyperspectral image of the second image are downsampled to 3 channels through a downsampling matrix, and a mapping relationship is established between the hyperspectral image of the second image and the color space where the image can be obtained by the first camera. Since the hyperspectral image of the second image contains a number of color channels of size N, the spectral information included is richer than the spectral information corresponding to the first image, so it is mapped to the color space where the image can be obtained by the first camera. The obtained three-channel image is closer to the true color of the first scene, and then the image after the downsampling of the hyperspectral image channel of the second image can be used to adjust the color of the first image, thereby improving the color authenticity and consistency of the image taken by the first camera.

[0036] In combination with the first aspect, in a possible implementation manner, the method further includes: determining the first conversion function according to a hyperspectral image corresponding to the second image and a spectral sensing curve corresponding to the first image.

[0037] In some embodiments, the method further includes: determining a second conversion function according to a hyperspectral image corresponding to the fourth image and a spectral sensing curve corresponding to the third image.

[0038] In some embodiments, the method further includes: determining a third conversion function according to the hyperspectral image corresponding to the sixth image and the spectral sensing curve corresponding to the fifth image.

[0039] In some embodiments, the estimation method of the downsampling matrix used by the first conversion function / the second conversion function / the third conversion function can be any one of the Bayesian method, the least squares method, the deep learning method, or other estimation methods.

[0040] In a second aspect, an electronic device is provided, which includes a camera array, a high-resolution multispectral camera and a processor, wherein the camera array includes a first camera, wherein the first camera is used to capture a first image when the first camera and the high-resolution multispectral camera are turned on at the same time, the first image being an original image of a first scene, and the first image being a three-channel image; the high-resolution multispectral camera is used to synchronously capture a second image, the second image being a multispectral original image of the first scene, and the second image being an M-channel image, wherein M is a positive integer greater than 3; the processor is used to convert the second image from an M-channel image to a three-channel image through a first conversion function; the processor is further used to perform color calibration on the first image based on the three-channel image converted from the second image to obtain first color parameters; the processor is further used to send the first color parameters to the image signal processor (ISP) path of the first camera, so that the first camera can adjust the color of the image captured by the first camera according to the first color parameters.

[0041] In some embodiments, the first color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0042] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as a color reference to adjust the color of the image captured by the cameras in the camera array, so that the color of the array camera is aligned with the color perceived by the high-resolution multispectral camera, which can improve the authenticity of the color of the image captured by the cameras in the camera array.

[0043] In combination with the second aspect, in one possible implementation, the camera array also includes a second camera, wherein: the second camera is used to capture a third image, the third image is the original image of the first scene, and the third image is a three-channel image; the processor is also used to perform color calibration on the third image based on the three-channel image converted from the second image to obtain second color parameters; the processor is also used to send the second color parameters to the image signal processor ISP channel of the second camera, so that the second camera adjusts the color of the image captured by the second camera according to the second color parameters.

[0044] In some embodiments, the first camera, the second camera, and the high-resolution multispectral camera synchronously capture images of the first scene.

[0045] In some further embodiments, the first camera, the second camera, and the high-resolution multispectral camera asynchronously capture images of the first scene within a certain time threshold.

[0046] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference, and the color of the image captured by each camera in the camera array is adjusted separately. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0047] In combination with the second aspect, in a possible implementation, the camera array also includes a second camera, wherein: the second camera is used to capture a third image through the second camera when the second camera and the high-resolution multispectral camera are turned on at the same time, and the third image is the original image of the second scene, and the third image is a three-channel image; the high-resolution multispectral camera is also used to synchronously capture a fourth image, and the fourth image is the multispectral original image of the second scene, and the fourth image is an M-channel image; the processor is also used to convert the fourth image from an M-channel image to a three-channel image through a second conversion function; the processor is also used to perform color calibration on the third image based on the three-channel image converted from the fourth image to obtain second color parameters; the processor is also used to send the second color parameters to the image signal processor ISP channel of the second camera, so that the second camera can adjust the color of the image captured by the second camera according to the second color parameters.

[0048] In some embodiments, the second color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0049] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference, and the color of the image captured by each camera in the camera array is adjusted separately. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0050] In combination with the second aspect, in a possible implementation, the camera array further includes a third camera, wherein: the third camera is used to capture a fifth image through the third camera when the third camera and the high-resolution multispectral camera are simultaneously turned on, the fifth image being the original image of the first scene, and the fifth image being a three-channel image; the high-resolution multispectral camera is also used to synchronously capture a sixth image, the sixth image being the multispectral original image of the first scene, and the sixth image being an M-channel image, where M is a positive integer greater than 3; the processor is further used to convert the sixth image from an M-channel image to a three-channel image through a third conversion function; the processor is further used to perform color calibration on the fifth image based on the three-channel image converted from the sixth image to obtain third color parameters; the processor is further used to send the third color parameters to the image signal processor (ISP) path of the third camera, so that the third camera can adjust the color of the image captured by the third camera according to the third color parameters.

[0051] In some embodiments, the third color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

[0052] In some embodiments, the first camera is one of the main camera, the ultra-wide-angle camera, and the telephoto camera; the second camera is one of the main camera, the ultra-wide-angle camera, and the telephoto camera; and the third camera is one of the main camera, the ultra-wide-angle camera, and the telephoto camera, wherein the first camera, the second camera, and the third camera are different from each other.

[0053] In some embodiments, the first camera is a main camera (wide-angle camera), the second camera is an ultra-wide-angle camera, and the third camera is a telephoto camera.

[0054] In an embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference, and the color of the image captured by each camera in the camera array is adjusted separately. The colors of the array cameras are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by the cameras in the camera array.

[0055] In combination with the second aspect, in one possible implementation, the processor is specifically used to: convert the second image from an M-channel image to a three-channel image by downsampling from the M channels of the second image to the three channels of the first image according to the first conversion function, wherein the first conversion function is a function that uses a downsampling matrix of size M×3 to perform matrix multiplication with the second image.

[0056] In an embodiment of the present application, the M channels of the second image are downsampled to 3 channels through a downsampling matrix, and a mapping relationship is established between the second image and the color space where the image can be obtained by the first camera. Since the second image contains a number of color channels of size M, the spectral information included is richer than the spectral information corresponding to the first image, so it is mapped to the color space where the image can be obtained by the first camera. The obtained three-channel image is closer to the true color of the first scene, and then the image after the downsampling of the second image channel can be used to adjust the color of the first image, thereby improving the color authenticity and consistency of the image taken by the first camera.

[0057] In combination with the second aspect, in a possible implementation, the processor is further specifically used to: before converting the second image from an M-channel image to a three-channel image through a first conversion function, align the perspective FOV of the first image and the second image.

[0058] In an embodiment of the present application, before converting the second image from an M-channel image to a three-channel image (i.e., establishing a mapping relationship between the second image and the color space in which the image can be obtained by the first camera), the first image and the second image are first aligned in perspective, thereby avoiding the impact on the authenticity and consistency of the color adjustment of the first image due to the perspective difference between the first image and the second image.

[0059] In combination with the second aspect, in a possible implementation manner, the processor is further configured to: determine the first conversion function based on the second image after FOV alignment.

[0060] In some embodiments, the estimation method of the downsampling matrix used by the first conversion function can be any one of the Bayesian method, the least squares method, the deep learning method, or other estimation methods.

[0061] In combination with the second aspect, in one possible implementation method, the processor is specifically used to: perform spectral band super-resolution on the second image according to the spectral sensing curve corresponding to the second image to obtain a hyperspectral image corresponding to the second image, where the hyperspectral image corresponding to the second image is an N-channel image, where N is a positive integer much larger than M; down-sample from the N channels of the hyperspectral image corresponding to the second image to the three channels of the first image according to the first conversion function, thereby converting the hyperspectral image corresponding to the second image from an N-channel image to a three-channel image, where the first conversion function is a function that uses a downsampling matrix of size N×3 to perform matrix multiplication with the second image.

[0062] In an embodiment of the present application, the N channels of the hyperspectral image of the second image are downsampled to 3 channels through a downsampling matrix, and a mapping relationship is established between the hyperspectral image of the second image and the color space where the image can be obtained by the first camera. Since the hyperspectral image of the second image contains a number of color channels of size N, the spectral information included is richer than the spectral information corresponding to the first image, so it is mapped to the color space where the image can be obtained by the first camera. The obtained three-channel image is closer to the true color of the first scene, and then the image after the downsampling of the hyperspectral image channel of the second image can be used to adjust the color of the first image, thereby improving the color authenticity and consistency of the image taken by the first camera.

[0063] In combination with the second aspect, in a possible implementation manner, the processor is further configured to: determine the first conversion function according to a hyperspectral image corresponding to the second image and a spectral sensing curve corresponding to the first image.

[0064] In a third aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0065] In a fourth aspect, a computer-readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the method in the above-mentioned first aspect or any possible implementation of the first aspect is implemented.

[0066] In a fifth aspect, a chip is provided, in which instructions are stored. When the chip is run on a device, the chip executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0067] In a sixth aspect, a computer program product is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the method in the above-mentioned first aspect or any possible implementation of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] FIG1 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0069] FIG2 is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0070] FIG3 is a schematic diagram of an array camera provided in an embodiment of the present application;

[0071] FIG4 is a schematic flow chart of a method for camera color calibration;

[0072] FIG5 is a schematic flow chart of another method for camera color calibration;

[0073] FIG6 is a schematic diagram of another array camera provided in an embodiment of the present application;

[0074] FIG7 is a schematic diagram of the structure of a single-point multispectral camera provided in an embodiment of the present application;

[0075] FIG8 is a flow chart of a method for performing camera color calibration using a single-point multispectral camera according to an embodiment of the present application;

[0076] FIG9 is a schematic diagram of an RGGB sensor mode provided in an embodiment of the present application;

[0077] FIG10 is a schematic diagram of spectral perception of a high-resolution multispectral device provided in an embodiment of the present application;

[0078] FIG11 is a schematic flowchart of a camera color calibration method provided in an embodiment of the present application;

[0079] FIG12 is a schematic flowchart of another camera color calibration method provided in an embodiment of the present application;

[0080] FIG13 is a schematic diagram of a system framework corresponding to a camera color calibration method provided in an embodiment of the present application;

[0081] FIG14 is a flow chart of another camera color calibration method according to an embodiment of the present application;

[0082] FIG15 is a flow chart of another camera color calibration method according to an embodiment of the present application. DETAILED DESCRIPTION

[0083] The technical solutions of this application will be described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, rather than all the embodiments.

[0084] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" or "multiple" refers to two or more than two.

[0085] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0086] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0087] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "one embodiment," "some embodiments," "another embodiment," and "other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically stated. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically stated.

[0088] The method provided in the embodiments of the present application can be applied to electronic devices with a time display function or a time recognition function, for example, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices, and other electronic devices. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0089] 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0090] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0092] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0093] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0094] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0095] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0096] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0097] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0098] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0099] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0100] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0101] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0102] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0103] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0104] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0105] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0106] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0107] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0108] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0109] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0110] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0111] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0112] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0113] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0114] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an App required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0115] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0116] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0117] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0118] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0119] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0120] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0121] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0122] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0123] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0124] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an embedded SIM (eSIM) card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0125] It should be understood that the phone cards in the embodiments of the present application include but are not limited to SIM cards, eSIM cards, universal subscriber identity modules (USIM), universal integrated circuit cards (UICC), and the like.

[0126] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0127] Figure 2 is a software structure diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer, from top to bottom. The application layer can include a series of application packages.

[0128] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0129] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0130] As shown in FIG2 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0131] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0132] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0133] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0134] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0135] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0136] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0137] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0138] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0139] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0140] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0141] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0142] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0143] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0144] A 2D graphics engine is a drawing engine for 2D drawings.

[0145] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0146] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Hongmeng and other systems.

[0147] The technical solutions of the embodiments of the present application can be applied to image processing scenarios involving multiple cameras. For example, they can be applied to scenarios such as color calibration of images captured by an array camera.

[0148] Among them, the electronic device can be a television, a desktop computer, a laptop computer, or a portable electronic device such as a mobile phone, a folding screen, a tablet computer, a camera, a video camera, a video recorder, or a smart home device, such as a refrigerator, a washing machine, a sweeper, or any electronic device with multiple cameras. It can also be an electronic device in a 5G network or an electronic device in a future evolved public land mobile communication network (PLMN), etc.

[0149] For example, FIG3 shows a schematic diagram of a camera setting provided in an embodiment of the present application.

[0150] As shown in FIG3 , the electronic device 300 is provided with a camera array, which includes a camera 310 , a camera 320 and a camera 330 .

[0151] In some embodiments, camera 310 is an ultra-wide-angle camera, camera 320 is a wide-angle camera which is also the main camera, and camera 330 is a telephoto camera.

[0152] Taking mobile phones as an example, smartphone cameras are typically presented in the form of a camera array, which includes multiple cameras. These cameras differ primarily in their focal lengths, primarily categorized as ultra-wide-angle, wide-angle (main), and telephoto. While different phones may have different camera configurations, they all have at least two cameras, capable of independent imaging and recording. The captured images or videos are directly presented to the user. However, because images captured by different cameras in the same scene can easily exhibit significant color differences, this can severely impact the user's photography experience. Consequently, users have high expectations for color consistency across the images captured by each camera in the camera array.

[0153] For example, FIG4 shows a schematic flow chart of a method 400 for camera color calibration. The method is a method for simultaneously starting multiple cameras. As shown in FIG4 , the method 400 includes:

[0154] S401: Using one camera (usually a main camera) among multiple cameras provided on a first device as a reference, perform real-time calibration on the other cameras among the multiple cameras.

[0155] Among them, the above-mentioned multiple cameras are all three-channel cameras, that is, the images captured by the above-mentioned multiple cameras are all three-channel images.

[0156] Specifically, one of the multiple cameras provided on the first device may be used as a reference to calibrate each of the other cameras in the multiple cameras in real time.

[0157] Among them, the color of the image captured by the camera is calibrated in real time.

[0158] In this method, due to the differences in multiple dimensions such as shooting angle of view (FOV), shooting brightness, shooting contrast, shooting dynamic range, and sensor spectral response consistency among different camera modules, one of the multiple cameras is used as a benchmark to calibrate the other cameras. Since these differences between the cameras are not well taken into account, the accuracy of the color calibration of the images taken by the cameras is relatively low. In addition, the calibration process requires at least two cameras to be turned on at the same time, which will result in high power consumption.

[0159] For example, FIG5 shows a schematic flow chart of another method 500 for camera color calibration. The method is an offline calibration method. As shown in FIG5 , the method 500 includes:

[0160] S501: Turn on a first camera and a second camera of an electronic device simultaneously, wherein the first camera serves as a reference camera for calibrating the second camera.

[0161] The first camera and the second camera are both three-channel cameras, that is, the images captured by the first camera and the second camera are both three-channel images.

[0162] S502: Determine a migration matrix / migration function between the first camera and the second camera according to the color perception distribution of the first camera and the color perception distribution of the second camera.

[0163] It can be understood that S501 and S502 are the process of calibrating the second camera based on the first camera. After the calibration is completed (that is, after the migration matrix / migration function between the first camera and the second camera is obtained), the first camera and the second camera are turned off.

[0164] S503: When photographing the first object through the first camera, turn on the first camera and collect a first image corresponding to the first object through the first camera.

[0165] S504: Determine a second image corresponding to the second camera according to the migration matrix / migration function between the first camera and the second camera and the first image.

[0166] Specifically, when photographing a first object through a first camera, the first camera is turned on, and the first camera estimates the current scene spectrum and determines the corresponding module migration parameters based on the migration matrix / migration function between the first camera and the second camera; then, the second image corresponding to the second camera is determined based on the module migration parameters and the first image.

[0167] In this method, since scene spectrum estimation is very difficult and the scene color cannot be perceived in detail, it is difficult to determine appropriate module migration parameters, resulting in relatively low accuracy of camera color calibration.

[0168] In summary, the current camera color calibration methods all have the problems of low accuracy and high calibration cost (labor cost, power consumption cost). The images captured by each camera in the camera array have inconsistent and unrealistic colors.

[0169] In view of this, the embodiments of the present application provide a method, device and electronic device for camera color calibration. In this method, a high-resolution multispectral camera is set on the electronic device, and the images captured by the high-resolution multispectral camera are used as a reference to perform color calibration on the cameras in the camera array set on the electronic device. This can avoid the problem of inaccurate calibration caused by calibration between array cameras, improve the accuracy and stability of camera color calibration, and make the colors of images output by multiple cameras in the camera array more consistent and more realistic. In addition, the high-resolution multispectral camera is small in size and can also reduce the architectural redundancy of the electronic device. The power consumption generated by the calibration process is also lower, which can save calibration costs and thus improve the user's shooting experience.

[0170] For example, FIG6 shows a schematic diagram of another camera setting provided in an embodiment of the present application.

[0171] As shown in FIG6 , the electronic device 600 is provided with a camera array and a high-resolution multispectral camera 610 . The camera array includes a camera 310 , a camera 320 , and a camera 330 .

[0172] In some embodiments, camera 310 is an ultra-wide-angle camera, camera 320 is a wide-angle camera which is also the main camera, and camera 330 is a telephoto camera.

[0173] Among them, the spectral perception capability of the high-resolution multispectral camera is much stronger than that of the ultra-wide-angle camera, the wide-angle camera and the telephoto camera. In some embodiments, the solution of the embodiment of the present application can be understood as: taking the multispectral image of the first scene captured by the high-resolution multispectral camera 610 as a reference, the red, green and blue (RGB) images of the first scene captured by the ultra-wide-angle camera 310, the wide-angle camera 320, and the telephoto camera 330 are color calibrated respectively.

[0174] It should be understood that this embodiment is only a schematic illustration of the camera setting method. The high-resolution multispectral camera 610 can be set independently of the camera array or in the camera array. This application does not limit the setting method of the high-resolution multispectral camera 610 on the electronic device.

[0175] In order to more clearly understand the advantages of using a high-resolution multispectral camera for camera color calibration, the process of using a single-point multispectral camera for camera color calibration is described in detail with reference to FIG7 and FIG8 .

[0176] Figure 7 shows the structure of a single-point multispectral camera. As shown in Figure 7, the single-point multispectral camera includes a parallel light-dispersing film and a single-point multispectral sensor. Light incident from different directions passes through the light-dispersing film. Due to the film's averaging effect, the amplitude and angle of light emitted from the film remain the same. The emitted light then passes through the single-point multispectral sensor, which perceives the average reflectance spectrum of the scene. After passing through the single-point multispectral sensor, the multiple light rays emitted from the light-dispersing film output a single-point spectral perception result of 1×M dimensions, i.e., a vector of 1×M dimensions. Because this 1×M single-point spectral perception result only has one globally averaged point (i.e., a single point) with spectral resolution, spatial mapping is impossible. The specific spatial location of each array camera corresponding to this single point cannot be determined, making it impossible to estimate the spectral band downsampling mapping function (i.e., convert a multi-channel image to a three-channel image).

[0177] Furthermore, FIG8 shows a flow chart of a method 800 for performing camera color calibration using a single-point multispectral camera. As shown in FIG8 , the method 800 includes:

[0178] S801: The main camera, wide-angle camera, or telephoto camera captures a RAW image of the current scene.

[0179] S802: The single-point camera synchronously obtains a single-point spectral perception result of the current scene, where the single-point spectral perception result is a single-point spectral perception result with a dimension of 1×M.

[0180] S803: Classify the light source based on the single-point spectral perception result, that is, determine the light source category of the current scene.

[0181] Among them, the light source classification based on single-point spectral perception results has the problems of low accuracy and limited scenes in the determined light source category.

[0182] S804: Based on the light source category of the current scene, determine the color parameters according to a pre-calibrated color parameter-light source category association table.

[0183] The pre-calibrated color parameter-light source category association table refers to a pre-calibrated color parameter-light source category association table that maps wide-angle / telephoto to the main camera.

[0184] S805: Send the determined color parameters to the ISP channels of the main camera / wide-angle / telephoto camera.

[0185] S806: Adjust the color of the wide-angle / telephoto camera according to the color parameters.

[0186] In this method, the light source classification based on the single-point spectral perception results has the problems of low accuracy and scene limitation. Therefore, the camera color calibration using a single-point multispectral camera is discrete, imprecise, and non-robust. It is easily affected by mixed light sources, so it is impossible to accurately perform color calibration and achieve multi-camera color consistency.

[0187] Therefore, in response to the technical problems existing in the above-mentioned single-point multispectral camera, the embodiment of the present application uses a high-resolution multispectral camera (a multi-point multispectral camera) to perform camera color calibration. In order to more clearly understand the camera color calibration method provided in the embodiment of the present application, the high-resolution multispectral camera (or high-resolution multispectral device) is first introduced in detail below.

[0188] Currently, the structure of image sensors used in electronic devices is basically a Bayer pattern structure. A basic color pixel consists of 2×2=4 pixels, on which three different color filters (such as red, green and blue (RGB), red, yellow and blue (RYB)) are placed, forming an RGGB sensor or RYYB sensor, or a color sensing unit similar to RGGB or RYYB. The entire image sensor is composed of a large number of rows and columns of color sensing units.

[0189] Among them, RGGB is a color mode, which is a mosaic color filter array formed by arranging RGB filters on the grid of light sensing components. In this arrangement, 50% is green, 25% is red, and the remaining 25% is blue; the RYYB sensor changes the arrangement of the underlying filters of the RGGB array, replacing two green pixels (G) with two yellow pixels (Y) to form an RYYB sensor.

[0190] The main difference between high-resolution multispectral devices and RGGB devices and RYYB devices is that their color perception units have more spectral perception functions (i.e., more spectral bands). Specifically, the number of spectral perception functions (i.e., the number of spectral bands) of RGGB devices and RYYB devices is 3 (red, green, and blue or red, yellow, and blue), while the number of spectral perception functions of high-resolution multispectral devices is greater than 3, for example, it can be 3×3=9 spectral bands. Therefore, high-resolution multispectral devices can obtain spectral perception capabilities far stronger than 3 channels without significantly reducing spatial resolution. In other words, compared with the three-channel spectral perception capabilities of RGGB devices and RYYB devices, high-resolution multispectral devices have stronger spectral perception capabilities and the colors of the perceived multispectral images are more realistic.

[0191] Exemplarily, taking the RGGB device as an example, FIG9 shows a schematic diagram of spectral perception of a three-channel spectral device provided in an embodiment of the present application.

[0192] As shown in Figure 9, black fill represents red, diagonal fills pointing leftward from high to low represent green, and diagonal fills pointing rightward from high to low represent blue. In an RGGB device, RGB filters are arranged on a grid of light sensing components, forming a mosaic color filter array. In this arrangement, 50% is green, 25% is red, and the remaining 25% is blue. This corresponds to a three-channel (red, green, and blue) spectral sensing capability.

[0193] For example, FIG10 shows a schematic diagram of spectral perception of a high-resolution multi-spectral device provided in an embodiment of the present application.

[0194] As shown in (a) in Figure 10, in one embodiment, the number of spectral bands perceived by the high-resolution multispectral device is 5, for example, may include red (marked as 1 in (a) in Figure 10), orange (marked as 2 in (a) in Figure 10), yellow (marked as 3 in (a) in Figure 10), blue (marked as 4 in (a) in Figure 10), and purple (marked as 5 in (a) in Figure 10), and the corresponding multispectral response curve may be shown, for example, as shown in (c) in Figure 10; the multispectral bands perceived by the high-resolution multispectral device shown in (a) in Figure 10 may be converted into hyperspectral bands as shown in (b) in Figure 10, and the corresponding hyperspectral response curve may be shown, for example, as shown in (d) in Figure 10.

[0195] It can be seen that compared with the three-channel spectrum sensed by the three-channel spectral device, the high-resolution multispectral device has a stronger spectral perception ability, and the color of the multispectral image collected by the high-resolution multispectral device is closer to the real color.

[0196] Specifically:

[0197] (1) The three-channel camera is limited by the material capabilities. The corresponding spectral perception curve is very different from that of the human eye. The color restoration ability of the captured image is weak, and some colors are prone to color cast. For example, when capturing images of real red-orange, cyan, and blue-purple, the corresponding colors in the output image deviate from red-orange, cyan, and blue-purple.

[0198] The multispectral perception curve and hyperspectral perception curve corresponding to the high-resolution multispectral camera are slightly different from the spectral perception curve corresponding to the human eye, and the color restoration ability of the collected images is strong, so there is basically no color cast problem.

[0199] In an embodiment of the present application, a high-resolution multispectral camera can reduce the dimensionality of the corresponding spectral perception function from a high dimension, that is, based on the perceived multispectral bands, it performs spectral band downsampling to obtain a three-channel image with more realistic colors.

[0200] The spectral perception curve can also be described as a spectral response curve.

[0201] (2) The human eye and brain have the ability to adapt to colors, which is mainly related to the color of the ambient light source. It is difficult to accurately estimate the color of the light source from the three-channel color information collected by a three-channel (for example: RGB / RYB) camera, and the output image is prone to inaccurate white balance problems, which leads to color cast of the entire image; high-resolution multispectral cameras have more channels, which greatly increases the spatial color information corresponding to the RAW image collected by the high-resolution multispectral camera, and the color of the output image is closer to the true color of the collected object. Correspondingly, the white balance capability is also significantly enhanced, which can solve the problem of image color cast to a large extent. Among them, white balance refers to the reference color determined as the color of the image. When the white balance is inaccurate (that is, the reference color is inaccurate), it will naturally lead to color cast of the entire image.

[0202] (3) The spectral perception function of the high-resolution multispectral camera can be reduced in dimension to the spectral perception function corresponding to the human eye, the spectral perception function corresponding to the RGB camera, and the spectral perception function corresponding to the RYB camera (it can also be understood as: the spectral perception function corresponding to the high-resolution multispectral camera is fitted with the spectral perception function corresponding to the human eye, the spectral perception function corresponding to the RGB camera, and the spectral perception function corresponding to the RYB camera). In other words, the multi-channel image collected by the high-resolution multispectral camera can be converted into a three-channel image. Since the authenticity of the multi-channel image collected by the high-resolution multispectral camera is very high, the converted three-channel image obtained by the conversion is relatively Compared with the three-channel image captured by the three-channel camera, the converted three-channel image is more realistic. The dimensionality reduction fitting of the spectral band from high dimension to low dimension is very accurate. Since the high-resolution multispectral camera can perform the above-mentioned spectral band dimensionality reduction, it is equivalent to the high-resolution multispectral camera being able to simultaneously capture the color map perceived by the human eye, the color map perceived by the ultra-wide-angle camera, the color map perceived by the wide-angle (main camera) camera, and the color map perceived by the telephoto camera. Moreover, there is no difference between these captured color maps in terms of FOV, brightness, contrast, dynamic range, overexposure, etc., which can achieve better color consistency and higher authenticity of the images captured by multiple cameras in the camera array.

[0203] Therefore, the colors perceived by the high-resolution multispectral camera are very accurate, and the original image (RAW image) distribution of the ultra-wide-angle camera, wide-angle (main camera) camera and telephoto camera with the same information source can be simulated through spectral band dimensionality reduction (that is, through spectral band dimensionality reduction, the perceived multi-light channel image is converted into a three-channel image). There is no FOV difference between the simulated multiple RAW image distributions, and there is no problem of inaccurate white balance. Therefore, the high-resolution multispectral camera can be used as a global color observer. Different types of cameras (including ultra-wide-angle cameras, wide-angle (main camera) cameras and telephoto cameras, including the first and second supplies of each module, etc.) are aligned with the color perceived by the high-resolution multispectral camera, so that the color performance is highly consistent and highly realistic.

[0204] For example, FIG11 shows a schematic flow chart of a method 1100 for camera color calibration provided in an embodiment of the present application. As shown in FIG11 , the method 1100 includes:

[0205] S1101: Simultaneously turning on a first camera and a high-resolution multispectral camera of an electronic device, where the first camera of the electronic device captures a first image, which is an original image of a first scene.

[0206] The first image is a three-channel image, for example, it can be an RGB image or an RYB image.

[0207] Among them, the first camera of the electronic device may include one or more of a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera.

[0208] S1102: While the first camera is capturing the first image, the high-resolution multispectral camera of the electronic device is capturing the second image, where the second image is the original multispectral image of the first scene.

[0209] The second image is an M-channel image, where M is a positive integer greater than 3.

[0210] It can also be understood as: the first image captured by the first camera corresponds to spectral information of 3 frequency bands; the second image captured by the high-resolution multispectral camera corresponds to spectral information of M frequency bands, and the spatial color information of the second image is much richer than the spatial color information of the first image. In other words, the spectral information perceived by the high-resolution multispectral camera is richer than the spectral information perceived by the first camera, and is closer to the true color of the first scene.

[0211] S1103: Convert the second image from a multi-channel image to a three-channel image through a first conversion function, wherein the first conversion function is a conversion function between the high-resolution multispectral camera and the first camera, and is used to represent a spectral band mapping relationship between the high-resolution multispectral camera and the first camera.

[0212] In a specific embodiment, spectral band down-sampling is performed on the second image according to the first conversion function, so as to convert the second image from a multi-channel image into a three-channel image.

[0213] The first conversion function is a function that performs matrix multiplication on the downsampling matrix and the second image.

[0214] In some embodiments, the estimation method of the downsampling matrix used by the first conversion function can be any one of the Bayesian method, the least squares method, the deep learning method, or other estimation methods.

[0215] Downsampling, also known as downsampling or decimation, is a fundamental concept in digital signal processing. It involves sampling a sequence of sample values ​​at intervals of a certain number of sample values. The resulting new sequence is a downsample of the original sequence. In image processing, the main purpose of downsampling is to reduce the image size to fit the display area or to generate a thumbnail of the corresponding image. Considering an image in matrix form, this means converting the image within an s×s window of the original image into a single pixel. The value of this pixel is the average or maximum value of all the pixels within the window.

[0216] In image processing, the principle of downsampling is: for an image I of size M×N, downsample it s times, that is, obtain a resolution image of size (M / s)×(N / s), where s should be a common divisor of M and N.

[0217] S1104: Using the three-channel image converted from the second image as a reference, perform color calibration on the first image to obtain first color parameters.

[0218] S1105: Send the first color parameter to the ISP channel corresponding to the first camera, so that the first camera adjusts the color of the image captured by the first camera according to the first color parameter.

[0219] In some embodiments, the electronic device further includes a second camera, which is a three-channel camera. The second camera can be color calibrated based on the high-resolution multispectral camera, and the color calibration process is the same as the color calibration process for the first camera described above.

[0220] In an embodiment of the present application, a color calibration method for an array camera based on a high-resolution multispectral camera is provided. In this method, since the images captured by the high-resolution multispectral camera are multi-channel images, their spatial color information is richer and the corresponding image colors are more authentic, the images captured by the high-resolution multispectral camera are used as the color reference to calibrate the colors of the images captured by each camera in the camera array respectively, which can improve the authenticity and consistency of the colors of the images captured by multiple cameras in the camera array; at the same time, since the high-resolution multispectral camera is small in size and the power consumption generated during the calibration process is low, it can reduce the architectural redundancy and calibration cost of electronic equipment during the color management development process. This method provides a unified baseline for each camera and improves debugging efficiency.

[0221] For example, Figure 12 shows a schematic flow chart of another camera color calibration method 1200 provided in an embodiment of the present application. As shown in Figure 12, the electronic device includes a camera array, which includes a main camera, an ultra-wide-angle camera, and a telephoto camera. The method 1200 includes:

[0222] S1201 to S1205 are processes for color calibration of the main camera of the electronic device based on the high-resolution multispectral camera.

[0223] S1201: Turn on a main camera and a high-resolution multispectral camera of an electronic device at the same time, and the main camera of the electronic device captures a first image, where the first image is an original image of a first scene.

[0224] The first image is a three-channel image, for example, it can be an RGB image or an RYB image.

[0225] S1202: While the main camera is capturing the first image, the high-resolution multispectral camera of the electronic device is capturing a second image, where the second image is a multispectral original image of the first scene.

[0226] The second image is an M-channel image, where M is a positive integer greater than 3.

[0227] It can also be understood that: the first image captured by the main camera is a three-channel image, and the second image captured by the high-resolution multispectral camera is a multi-channel image. The spatial color information of the second image is far richer than the spatial color information of the first image. That is to say, the spectral information perceived by the high-resolution multispectral camera is richer than the spectral information perceived by the main camera, and is closer to the true color of the first scene.

[0228] S1203: Convert the second image from a multi-channel image to a three-channel image through a first conversion function, wherein the first conversion function is a conversion function between the high-resolution multispectral camera and the main camera, and is used to represent a spectral band mapping relationship between the high-resolution multispectral camera and the main camera.

[0229] In a specific embodiment, spectral band down-sampling is performed on the second image according to the first conversion function, thereby converting the second image from a multi-channel image to a three-channel image.

[0230] In one implementation, downsampling is performed from the M (or k×k) channels corresponding to the second image to three channels according to the first conversion function, thereby converting the second image from a multi-channel image to a three-channel image, wherein the first conversion function may be a multi-channel image with a size of M×3 (or k×k). 2 ×3) downsampling matrix and the second image to perform matrix multiplication.

[0231] S1204: Using the three-channel image converted from the second image as a reference, perform color calibration on the first image to obtain first color parameters.

[0232] In some embodiments, the first color parameter may include any one or more of an auto white balance (AWB) parameter, a color adjustment (CA) parameter, a color calibration (CC) parameter, and a 3D look up table (3DLUT). In addition, the first color parameter may also be other color-related parameters, which is not limited in this application.

[0233] S1205: Send the first color parameter to the ISP channel corresponding to the main camera, so that the main camera adjusts the color of the image captured by the main camera according to the first color parameter.

[0234] In some embodiments, S1201 to S1204 are an offline calibration process. After the first color parameters corresponding to the main camera are obtained through the offline calibration process, when the main camera performs image capture, when the captured image is processed by the ISP corresponding to the main camera, the color of the captured image is adjusted according to the first color parameters to make the image output by the main camera more realistic.

[0235] In some other embodiments, S1201 to S1204 are an online calibration process. When the user captures an image through the main camera, the high-resolution multispectral camera is turned on synchronously when the main camera is turned on. While the main camera is capturing images, the high-resolution multispectral camera captures images of the same scene. The first color parameter is obtained through the above S1201 to S1204, and when the captured image is processed by the ISP corresponding to the main camera, the color of the captured image is adjusted according to the first color parameter to make the image output by the main camera more realistic.

[0236] In some embodiments, the color standard process of the main camera can occur each time the main camera is used to capture an image, or it can occur in response to a user's calibration operation, or it can be calibrated offline (or online) after each certain number of images are taken, or it can be calibrated every certain period of time; it can also be other calibration times. This application does not limit the timing of the color calibration of the camera, nor does it limit the calibration scene. It can be an offline calibration or an online calibration.

[0237] In the embodiment of the present application, since the second image contains a number of color channels of size M, the spectral information included is richer than the spectral information corresponding to the first image, so mapping it to the main camera can obtain the color space where the image is located, and the obtained three-channel image is closer to the true color of the first scene, and then the image after downsampling the second image channel can be used to adjust the color of the first image, thereby improving the color accuracy and consistency of the image taken by the main camera.

[0238] S1206 to S1210 are processes for color calibration of the ultra-wide-angle camera of the electronic device based on the high-resolution multispectral camera.

[0239] S1206: Simultaneously turning on the ultra-wide-angle camera and the high-resolution multispectral camera of the electronic device, so that the ultra-wide-angle camera of the electronic device captures a third image, where the third image is an original image of the second scene.

[0240] The third image is a three-channel image, for example, it can be an RGB image or an RYB image.

[0241] S1207: While the ultra-wide-angle camera is capturing the third image, the high-resolution multispectral camera of the electronic device is capturing a fourth image, where the fourth image is a multispectral original image of the second scene.

[0242] The fourth image is an M-channel image, where M is a positive integer greater than 3.

[0243] It can also be understood that the third image captured by the ultra-wide-angle camera is a three-channel image, and the fourth image captured by the high-resolution multispectral camera is a multi-channel image. The spatial color information of the fourth image is far richer than the spatial color information of the third image. In other words, the spectral information perceived by the high-resolution multispectral camera is richer than the spectral information perceived by the ultra-wide-angle camera, and is closer to the true color of the second scene.

[0244] S1208: Convert the fourth image from a multi-channel image to a three-channel image through a second conversion function, wherein the second conversion function is a conversion function between the high-resolution multispectral camera and the ultra-wide-angle camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the ultra-wide-angle camera.

[0245] In a specific embodiment, spectral band down-sampling is performed on the fourth image according to the second conversion function, thereby converting the fourth image from a multi-channel image to a three-channel image.

[0246] In some embodiments, the estimation method of the downsampling matrix used by the second conversion function can be any one of the Bayesian method, the least squares method, the deep learning method, or other estimation methods.

[0247] In one implementation, downsampling is performed from the M (or k×k) channels corresponding to the fourth image to three channels according to the second conversion function, thereby converting the fourth image from a multi-channel image to a three-channel image, wherein the second conversion function may be a multi-channel image of size M×3 (or k×k). 2 ×3) downsampling matrix and the fourth image to perform matrix multiplication.

[0248] S1209: Using the three-channel image converted from the fourth image as a reference, perform color calibration on the third image to obtain second color parameters.

[0249] The explanation of the second color parameter is the same as the above explanation of the first color parameter, and for the sake of brevity, it will not be repeated here.

[0250] S1210: Send the second color parameter to the ISP channel corresponding to the ultra-wide-angle camera, so that the ultra-wide-angle camera adjusts the color of the image captured by the ultra-wide-angle camera according to the second color parameter.

[0251] In some embodiments, the second scene is the same as the first scene, and the fourth image and the second image are the same image. That is, in some implementations, when the ultra-wide-angle camera and the main camera both capture (either synchronously or asynchronously within a certain time threshold) the original image of the first scene, S1207 may not be performed. The high-resolution multispectral camera only needs to synchronously capture a multispectral original image of the first scene once, and color calibration of the ultra-wide-angle camera and the main camera is performed using this multispectral original image of the first scene.

[0252] In some embodiments, S1206 to S1209 are an offline calibration process. After the second color parameters corresponding to the ultra-wide-angle camera are obtained through the offline calibration process, when the ultra-wide-angle camera captures an image, when the captured image is processed by the ISP corresponding to the ultra-wide-angle camera, the color of the captured image is adjusted according to the second color parameters to make the image output by the ultra-wide-angle camera more authentic.

[0253] In some other embodiments, S1206 to S1209 are an online calibration process. When the user captures images through the ultra-wide-angle camera, the high-resolution multispectral camera is turned on synchronously when the ultra-wide-angle camera is turned on. While the ultra-wide-angle camera is capturing images, the high-resolution multispectral camera captures images of the same scene. The second color parameters are obtained through the above S1206 to S1209, and when the captured image is processed by the ISP corresponding to the ultra-wide-angle camera, the color of the captured image is adjusted according to the second color parameters to make the image output by the ultra-wide-angle camera more authentic.

[0254] In some embodiments, the color standard process of the ultra-wide-angle camera can occur each time the ultra-wide-angle camera is used to capture an image, or it can occur in response to a user's calibration operation, or it can be calibrated offline (or online) after each certain number of images are taken, or it can be calibrated every certain period of time; it can also be other calibration times. The present application does not limit the timing of the color calibration of the camera, nor does it limit the calibration scene. It can be an offline calibration or an online calibration.

[0255] In the embodiment of the present application, since the fourth image contains a number of color channels of size M, the spectral information included is richer than the spectral information corresponding to the third image, so mapping it to the ultra-wide-angle camera can obtain the color space where the image is located, and the obtained three-channel image is closer to the true color of the second scene, and then the image after downsampling the fourth image channel can be used to adjust the color of the third image, thereby improving the color accuracy and consistency of the image taken by the ultra-wide-angle camera.

[0256] S1211 to S1215 are the processes of color calibration of the telephoto camera of the electronic device based on the high-resolution multispectral camera.

[0257] S1211: Turn on the telephoto camera and the high-resolution multispectral camera of the electronic device at the same time, and the telephoto camera of the electronic device collects a fifth image, which is an original image of the third scene.

[0258] The fifth image is a three-channel image, for example, it can be an RGB image or an RYB image.

[0259] S1212: While the telephoto camera is capturing the fifth image, the high-resolution multispectral camera of the electronic device is capturing a sixth image, where the sixth image is a multispectral original image of the third scene.

[0260] The sixth image is an M-channel image, where M is a positive integer greater than 3.

[0261] It can also be understood that: the fifth image captured by the telephoto camera is a three-channel image, and the sixth image captured by the high-resolution multispectral camera is a multi-channel image. The spatial color information of the sixth image is far richer than the spatial color information of the fifth image. In other words, the spectral information perceived by the high-resolution multispectral camera is richer than the spectral information perceived by the telephoto camera, and is closer to the true color of the third scene.

[0262] S1213: Convert the sixth image from a multi-channel image to a three-channel image through a third conversion function, wherein the third conversion function is a conversion function between the high-resolution multispectral camera and the telephoto camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the telephoto camera.

[0263] In a specific embodiment, spectral band down-sampling is performed on the sixth image according to the third conversion function, thereby converting the sixth image from a multi-channel image to a three-channel image.

[0264] In some embodiments, the estimation method of the downsampling matrix used by the third conversion function can be any one of the Bayesian method, the least squares method, the deep learning method, or other estimation methods.

[0265] In some embodiments, the first / second / third conversion functions mentioned above may be understood as first / second / third mapping parameters.

[0266] In one implementation, downsampling is performed from the M (or k×k) channels corresponding to the sixth image to three channels according to the third conversion function, thereby converting the sixth image from a multi-channel image to a three-channel image, wherein the third conversion function may be a multi-channel image of size M×3 (or k×k). 2 ×3) downsampling matrix and the sixth image to perform matrix multiplication.

[0267] S1214: Using the three-channel image converted from the sixth image as a reference, perform color calibration on the fifth image to obtain third color parameters.

[0268] The explanation of the third color parameter is the same as the above explanation of the first color parameter, and for the sake of brevity, it will not be repeated here.

[0269] S1215: Send the third color parameter to the ISP channel corresponding to the telephoto camera, so that the telephoto camera adjusts the color of the image captured by the telephoto camera according to the third color parameter.

[0270] In some embodiments, the third scene is the same as the first scene, and the sixth image and the second image are the same image. That is, in some implementations, when the telephoto camera and the main camera both capture (either synchronously or asynchronously within a certain time threshold) the original image of the first scene, S1212 may not be performed. The high-resolution multispectral camera only needs to synchronously capture a single multispectral original image of the first scene, and color calibration of the telephoto camera and the main camera is performed using this multispectral original image of the first scene.

[0271] In some other embodiments, the third scene and the second scene are the same, and the sixth image and the fourth image are the same image. That is, in some implementations, when the telephoto camera and the ultra-wide-angle camera both capture (either synchronously or asynchronously within a certain time threshold) the original image of the second scene, S1212 may not be performed, and the high-resolution multispectral camera only needs to synchronously capture a single multispectral original image of the second scene, and color calibration of the telephoto camera and the ultra-wide-angle camera is performed using this multispectral original image of the second scene.

[0272] In some other embodiments, the third scene, the second scene, and the first scene are all the same, and the sixth image, the fourth image, and the second image are all the same image. That is, in some implementations, when the ultra-wide-angle camera, the telephoto camera, and the main camera all capture (either synchronously or asynchronously within a certain time threshold) the original image of the first scene, S1207 and S1212 may not be executed, and the high-resolution multispectral camera only needs to synchronously capture the multispectral original image of the first scene once, and color calibration of the telephoto camera, the ultra-wide-angle camera, and the main camera is performed using the multispectral original image of the first scene.

[0273] In some embodiments, S1211 to S1214 are an offline calibration process. After the third color parameter corresponding to the telephoto camera is obtained through the offline calibration process, when the telephoto angle camera performs image capture, when the captured image is processed by the ISP corresponding to the telephoto camera, the color of the captured image will be adjusted according to the third color parameter to make the image output by the telephoto camera more realistic.

[0274] In some other embodiments, S1211 to S1214 are an online calibration process. When the user captures an image through the telephoto camera, the high-resolution multispectral camera is turned on synchronously when the telephoto camera is turned on. While the telephoto camera is capturing images, the high-resolution multispectral camera is capturing images of the same scene. The third color parameter is obtained through the above S1211 to S1214, and when the captured image is processed by the ISP corresponding to the telephoto camera, the color of the captured image is adjusted according to the third color parameter to make the image output by the telephoto camera more authentic.

[0275] In some embodiments, the color standard process of the telephoto camera may occur each time an image is captured by the telephoto camera, or it may occur in response to a user's calibration operation, or it may be calibrated offline (or online) after a certain number of images are taken, or it may be calibrated every certain period of time; it may also be calibrated at other times. This application does not limit the timing of the color calibration of the camera, nor does it limit the calibration scene. It may be an offline calibration or an online calibration.

[0276] Among them, the parameter types of the first color parameter, the second color parameter and the third color parameter should be consistent; that is, the color-related modules in the ISP path of the main camera, the color-related modules in the ISP path of the ultra-wide-angle camera and the color-related modules in the ISP path of the telephoto camera are the same.

[0277] In the embodiment of the present application, since the images captured by the high-resolution multispectral camera are multi-channel images, their spatial color information is richer and the corresponding image colors are more realistic. The images captured by the high-resolution multispectral camera are used as the color reference to calibrate the colors of the images captured by each camera in the camera array, so that the colors of the pictures captured by each camera are aligned with the colors perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the colors of the images captured by multiple cameras in the camera array.

[0278] For example, FIG13 shows a schematic diagram of a system framework corresponding to a camera color calibration method provided in an embodiment of the present application.

[0279] As shown in FIG13 , the electronic device includes a main camera, a high-resolution multispectral camera, a color parameter determination module, and an image signal processor (ISP). The system framework is used to perform the following steps:

[0280] S1301: Simultaneously turn on a main camera and a high-resolution multispectral camera of an electronic device, and the main camera captures a first RAW image, where the first RAW image is a RAW image of a first scene.

[0281] The first RAW image is a three-channel image, for example, it can be an RGB image or an RYB image.

[0282] S1302: While the main camera is capturing the first RAW image, the high-resolution multispectral camera is capturing a second RAW image, where the second RAW image is a RAW image of the first scene.

[0283] The second RAW image is an M-channel image, where M is a positive integer greater than 3.

[0284] S1303: The color parameter determination module determines a first color parameter according to the first RAW image and the second RAW image, and sends the first color parameter to the ISP channel corresponding to the main camera.

[0285] Specifically, the color parameter determination module converts the second RAW image from an M-channel image to a three-channel image through a first conversion function, wherein the first conversion function is a conversion function between the high-resolution multispectral camera and the main camera, and is used to represent the spectral band mapping relationship between the high-resolution multispectral camera and the main camera; then, based on the three-channel image converted from the second RAW image, the first RAW image is color calibrated to obtain the first color parameters.

[0286] The explanation of the first color parameter has been explained in detail in the embodiment shown in FIG11 , and will not be repeated here for the sake of brevity.

[0287] S1304: The main camera performs color adjustment on the image captured by the main camera according to the first color parameter.

[0288] In some embodiments, the color calibration of the main camera by the high-resolution multispectral camera is an online calibration. During the process of processing the first RAW image through the ISP channel corresponding to the main camera, the color of the first RAW image is adjusted according to the first color parameter, and the color-adjusted image is output, that is, the main camera image.

[0289] In some other embodiments, the color calibration of the main camera by the high-resolution multispectral camera is an offline calibration. After obtaining the first color parameter, when an image is captured by the main camera, the captured image is color-adjusted according to the first color parameter during the process of processing the captured image through the ISP channel corresponding to the main camera, and the color-adjusted image is output, that is, the main camera image.

[0290] Furthermore, the electronic device also includes an ultra-wide-angle camera, and the ultra-wide-angle camera of the electronic device can be color calibrated based on the high-resolution multispectral camera. The system architecture diagram corresponding to the color calibration process is similar to the above-mentioned system architecture diagram for color calibration of the main camera of the electronic device based on the high-resolution multispectral camera. For the sake of brevity, it will not be repeated here.

[0291] Furthermore, the electronic device also includes a telephoto camera, and the high-resolution multispectral camera can be used as a reference to perform color calibration on the telephoto camera of the electronic device. The system architecture diagram corresponding to the color calibration process is similar to the system architecture diagram for color calibration of the main camera of the electronic device using the high-resolution multispectral camera as a reference. For the sake of brevity, it will not be repeated here. In the embodiment of the present application, the image captured by the high-resolution multispectral camera is used as the color reference to calibrate the color of the image captured by each camera in the camera array, so that the color of the picture captured by each camera is aligned with the color perceived by the high-resolution multispectral camera, which can improve the authenticity and consistency of the color of the images captured by multiple cameras.

[0292] For example, FIG14 shows a flowchart corresponding to another camera color calibration method 1400 provided in an embodiment of the present application. As shown in FIG14 , the method 1400 includes:

[0293] S1401 and S1402 are the same as S1301 and S1302 in the embodiment shown in FIG13 , and are not described again for the sake of brevity.

[0294] S1403: Align the first RAW image and the second RAW image in terms of field of view (FOV).

[0295] In one implementation, the FOV cropping and alignment of the first RAW image and the second RAW image are performed by calibration.

[0296] S1404: Estimate a first conversion function based on the second RAW image after FOV alignment. The first conversion function can also be understood as a spectral band mapping parameter between the high-resolution multispectral camera and the main camera.

[0297] In some embodiments, the first conversion function is a downsampling matrix for downsampling from the FOV-aligned second RAW image to the FOV-aligned first RAW image.

[0298] The method for estimating the downsampling matrix used by the first transfer function has been described in detail in the aforementioned embodiment and will not be repeated here for the sake of brevity.

[0299] 1405: Convert the second RAW image after FOV alignment from a multi-channel image to a three-channel image using a first conversion function.

[0300] It should be understood that the three-channel image converted from the second RAW image has higher color authenticity than the three-channel image captured by the main camera.

[0301] In one implementation, M=k×k, and according to the first conversion function, spectral band downsampling is performed from the k×k channels corresponding to the second RAW image to the three-channel image, thereby converting the second RAW image from a multi-channel image to a three-channel image, wherein the first conversion function is to use a size of k 2 A function for performing matrix multiplication of a downsampling matrix of size × 3 with the second image, where the value of k is a positive integer greater than or equal to 3.

[0302] S1406: Using the three-channel image converted from the second RAW image after FOV alignment as a reference, statistically correct the first RAW image after FOV alignment to obtain first color parameters corresponding to the main camera.

[0303] The explanation of the first color parameter has been explained in detail in the embodiment shown in FIG11 , and will not be repeated here for the sake of brevity.

[0304] In some embodiments, the aligned first RAW image is statistically corrected, and the statistical correction method used may include white point correction, etc.

[0305] S1407: Send the first color parameter to the ISP channel corresponding to the main camera.

[0306] S1408: In the ISP of the first camera, color adjustment is performed on the first RAW image after the FOV alignment according to the first color parameter, and the color-adjusted image is output, that is, the main camera image.

[0307] In some embodiments, in the ISP of the first camera, after obtaining the first color parameter, the main camera performs color adjustment on the image captured by the main camera according to the first color parameter.

[0308] Furthermore, the electronic device also includes an ultra-wide-angle camera, and the ultra-wide-angle camera of the electronic device can be color calibrated based on the high-resolution multispectral camera. The color calibration process can be similar to the above-mentioned process of color calibrating the main camera of the electronic device based on the high-resolution multispectral camera (S1401 to S1408). For the sake of brevity, it will not be repeated here.

[0309] Furthermore, the electronic device also includes a telephoto camera, and the telephoto camera of the electronic device can be color calibrated based on the high-resolution multispectral camera. The color calibration process is similar to the above-mentioned process of color calibrating the main camera of the electronic device based on the high-resolution multispectral camera (S1401 to S1408). For the sake of brevity, it will not be repeated here.

[0310] In an embodiment of the present application, a mapping relationship can be constructed between a high-band resolution image acquired by a high-resolution multispectral camera and a low-band resolution (RGB three-channel) image of each array camera. Based on the mapping relationship, the high-band resolution image is downsampled to a low-band resolution image to obtain a three-channel image with higher color authenticity corresponding to the low-band resolution image. The low-band resolution image is calibrated using the three-channel image with higher color authenticity, and the color parameters corresponding to each array camera can be obtained. Then, each array camera adjusts the color of the captured image according to the corresponding color parameters, so that the colors of the pictures captured by each camera are aligned with the colors perceived by the high-resolution multispectral camera, thereby improving the color authenticity and consistency of the images captured by multiple cameras in the camera array.

[0311] For example, FIG15 shows a flowchart corresponding to another camera color calibration method 1500 provided in an embodiment of the present application. As shown in FIG15 , the method 1500 includes:

[0312] S1501: Turn on the main camera and the high-resolution multispectral camera at the same time. The main camera collects a first RAW image, where the first RAW image is a RAW image of a first scene.

[0313] The explanation of this step is the same as that of S1301 in the embodiment shown in FIG13 , and will not be repeated here for the sake of brevity.

[0314] S1502: Generate a first response curve corresponding to the first RAW image.

[0315] The first response curve may be understood as a spectral perception curve corresponding to the first RAW image. For details, please refer to the embodiment shown in FIG. 10 .

[0316] S1503: While the main camera is capturing the first RAW image, the high-resolution multispectral camera is capturing a second RAW image, where the second RAW image is a RAW image of the first scene.

[0317] The explanation of this step is the same as that of S1302 in the embodiment shown in FIG13 , and will not be repeated here for the sake of brevity.

[0318] S1504: Generate a second response curve corresponding to the second RAW image.

[0319] The second response curve may be understood as a spectral perception curve corresponding to the second RAW image. For details, please refer to the embodiment shown in FIG. 10 .

[0320] S1505: Based on the second response curve, perform spectral band super-resolution on the second RAW image to obtain a hyperspectral image corresponding to the second RAW image.

[0321] It should be understood that the number of spectral bands corresponding to the hyperspectral image corresponding to the second RAW image is much greater than the number of spectral bands corresponding to the second RAW image.

[0322] In some embodiments, the second RAW image corresponds to M spectral bands, and the hyperspectral image corresponding to the second RAW image corresponds to N spectral bands, wherein N is much larger than M. When M is k 2 When N>>k 2 .

[0323] In some embodiments, the method for performing spectral band super-resolution on the second RAW image includes a super-resolution reconstruction method based on compressed sensing and / or a spectral band super-resolution method based on a neural network. In addition, it can also be other spectral band super-resolution methods, which are not limited in this application.

[0324] S1506: Determine a first conversion function based on the hyperspectral image corresponding to the second RAW image and the first response curve. The first conversion function can also be understood as a spectral band mapping parameter between the high-resolution multispectral camera and the main camera.

[0325] In some embodiments, the first conversion function is a function of performing matrix multiplication of a downsampling matrix for downsampling the hyperspectral image corresponding to the second RAW image to the first RAW image and the second image.

[0326] S1507: Convert the second RAW image into a three-channel image using the first conversion function.

[0327] It should be understood that the three-channel image converted from the second RAW image has higher color authenticity than the three-channel image captured by the main camera.

[0328] In one implementation, based on a downsampling matrix of size N×3, spectral band downsampling is performed from N channels corresponding to the hyperspectral image corresponding to the second RAW image to three channels, and the hyperspectral image corresponding to the second RAW image is converted into a three-channel image, thereby converting the second RAW image from a multi-channel image to a three-channel image, where N is much greater than M.

[0329] S1508: Based on the converted three-channel image, statistical correction is performed on the first RAW image to obtain first color parameters corresponding to the main camera.

[0330] The explanation of the first color parameter has been explained in detail in the embodiment shown in FIG11 , and will not be repeated here for the sake of brevity.

[0331] S1509: Send the first color parameter to the ISP channel corresponding to the main camera.

[0332] S1510: In the ISP of the first camera, color adjustment is performed on the first RAW image according to the first color parameter, and the color-adjusted image is output, that is, the main camera image.

[0333] In some embodiments, in the ISP of the main camera, after obtaining the first color parameter, the main camera performs color adjustment on the image captured by the main camera according to the first color parameter.

[0334] Furthermore, the electronic device also includes an ultra-wide-angle camera, and the ultra-wide-angle camera of the electronic device can be color calibrated based on the high-resolution multispectral camera. The color calibration process can be similar to the above-mentioned process of color calibrating the main camera of the electronic device based on the high-resolution multispectral camera (S1501 to S1510). For the sake of brevity, it will not be repeated here.

[0335] Furthermore, the electronic device also includes a telephoto camera, and the telephoto camera of the electronic device can be color calibrated based on the high-resolution multispectral camera. The color calibration process is similar to the above-mentioned process of color calibrating the main camera of the electronic device based on the high-resolution multispectral camera (S1501 to S1510). For the sake of brevity, it will not be repeated here.

[0336] In one implementation, the main camera performs online color calibration during the first shooting process to obtain first color parameters, and performs color adjustment on the image obtained by the first shooting according to the first color parameters. Thereafter, when the main camera shoots an image again, the color of the captured image is directly adjusted according to the first color parameters.

[0337] In another implementation, the main camera obtains a first color parameter through online calibration or offline calibration in a first time period, and the images captured during the first time period are all color-adjusted according to the first color parameter; the main camera obtains a second color parameter through online calibration or offline calibration in a second time period, and the images captured during the second time period are all color-adjusted according to the second color parameter.

[0338] In another implementation, the main camera obtains the first color parameter in advance through offline calibration, and when the main camera captures an image, the color of the captured image is adjusted according to the first color parameter.

[0339] It should be understood that the multispectral camera described in the embodiments of the present application (or the above-mentioned high-resolution multispectral camera, or the above-mentioned second camera) can be a color filter array (CFA) that realizes differentiation through chemical dyes in terms of sensor function implementation; it can also be a CFA that realizes differentiation through coating interference; it can also be a CFA that realizes differentiation through a metasurface Wiener structure; it can also be other sensor function implementation methods, and this application does not limit this.

[0340] It should also be understood that the multispectral camera (or the above-mentioned high-resolution multispectral camera, or the above-mentioned second camera) described in the embodiments of the present application can be independent of the existing cameras (for example, wide-angle, ultra-wide-angle, telephoto, etc.) of the electronic device in terms of module function implementation. It can also form a certain structural appearance with the existing camera, such as a hybrid structural appearance; it can also be other module function implementation methods, which are not limited in this application.

[0341] The above-mentioned color parameters may include one or more parameters corresponding to the color module in the ISP path of the camera, and may also include one or more parameters corresponding to the brightness module in the ISP path of the camera.

[0342] One or more of the modules or units described herein can be implemented in software, hardware, or a combination of the two. When any of the above modules or units is implemented in software, the software exists in the form of computer program instructions and is stored in a memory, and a processor can be used to execute the program instructions and implement the above method flow. The processor may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller (MCU), or an artificial intelligence processor, etc., a computing device that runs software, each computing device may include one or more cores for executing software instructions to perform operations or processing. The processor may be built into an SoC (system on chip) or an application specific integrated circuit (ASIC), or it may be an independent semiconductor chip. In addition to the core for executing software instructions to perform operations or processing within the processor, it may further include necessary hardware accelerators, such as a field programmable gate array (FPGA), a PLD (programmable logic device), or a logic circuit that implements dedicated logic operations.

[0343] When the modules or units described in this document are implemented in hardware, the hardware can be any one or any combination of a CPU, a microprocessor, a DSP, an MCU, an artificial intelligence processor, an ASIC, a SoC, an FPGA, a PLD, a dedicated digital circuit, a hardware accelerator, or a non-integrated discrete device, which can run the necessary software or not rely on the software to execute the above method flow.

[0344] When the modules or units described herein are implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0345] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0346] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0347] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0348] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0349] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0350] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0351] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for camera color calibration, characterized in that: The method is applied to an electronic device, the electronic device including a camera array and a high-resolution multispectral camera, and the method includes: When a first camera and the high-resolution multispectral camera are simultaneously turned on, a first image is captured by the first camera, where the first image is an original image of a first scene, the first image is a three-channel image, and the camera array includes the first camera; Synchronously collecting a second image through the high-resolution multispectral camera, where the second image is a multispectral original image of the first scene, and the second image is an M-channel image, where M is a positive integer greater than 3; Converting the second image from an M-channel image to a three-channel image using a first conversion function; performing color calibration on the first image based on a three-channel image converted from the second image to obtain first color parameters; The first color parameter is sent to an image signal processor (ISP) path of the first camera, so that the first camera performs color adjustment on an image captured by the first camera according to the first color parameter.

2. The method according to claim 1, characterized in that The camera array further includes a second camera, and the method further includes: Capturing a third image through the second camera, wherein the third image is an original image of the first scene and is a three-channel image; performing color calibration on the third image based on the three-channel image converted from the second image to obtain second color parameters; The second color parameter is sent to the image signal processor (ISP) path of the second camera, so that the second camera adjusts the color of the image captured by the second camera according to the second color parameter.

3. The method according to claim 1, characterized in that The camera array further includes a second camera, and the method further includes: When the second camera and the high-resolution multispectral camera are turned on at the same time, a third image is captured by the second camera, where the third image is an original image of the second scene and is a three-channel image; Synchronously collecting a fourth image by the high-resolution multispectral camera, wherein the fourth image is a multispectral original image of the second scene and the fourth image is an M-channel image; Converting the fourth image from an M-channel image to a three-channel image using a second conversion function; performing color calibration on the third image based on a three-channel image converted from the fourth image to obtain second color parameters; The second color parameter is sent to the image signal processor (ISP) path of the second camera, so that the second camera adjusts the color of the image captured by the second camera according to the second color parameter.

4. The method according to claim 2 or 3, characterized in that The first camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera; the second camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera; and the first camera and the second camera are different.

5. The method according to any one of claims 1 to 4, characterized in that The converting the second image from an M-channel image to a three-channel image by using a first conversion function includes: The second image is converted from an M-channel image to a three-channel image by downsampling from the M channels of the second image to the three channels of the first image according to the first conversion function, wherein the first conversion function is a function of performing matrix multiplication on the second image using a downsampling matrix of size M×3.

6. The method according to claim 5, characterized in that Before converting the second image from an M-channel image to a three-channel image using the first conversion function, the method further includes: The first image and the second image are aligned with respect to a field of view (FOV).

7. The method according to claim 6, characterized in that The method further comprises: The first conversion function is determined based on the second image after FOV alignment.

8. The method according to any one of claims 1 to 4, characterized in that The converting the second image from an M-channel image to a three-channel image by using a first conversion function includes: performing spectral band super-resolution on the second image according to the spectral sensing curve corresponding to the second image to obtain a hyperspectral image corresponding to the second image, wherein the hyperspectral image corresponding to the second image is an N-channel image, where N is a positive integer much greater than M; The hyperspectral image corresponding to the second image is converted from an N-channel image to a three-channel image by downsampling from the N channels of the hyperspectral image corresponding to the second image to the three channels of the first image according to the first conversion function, wherein the first conversion function is a function that performs matrix multiplication on the second image by using a downsampling matrix of size N×3.

9. The method according to claim 8, characterized in that The method further comprises: The first conversion function is determined according to the hyperspectral image corresponding to the second image and the spectral sensing curve corresponding to the first image.

10. The method according to any one of claims 1 to 9, characterized in that The first color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

11. An electronic device, characterized in that: The electronic device includes a camera array, a high-resolution multispectral camera and a processor, wherein the camera array includes a first camera, wherein: The first camera is configured to capture a first image when the first camera and the high-resolution multispectral camera are simultaneously turned on, where the first image is an original image of a first scene and is a three-channel image; The high-resolution multispectral camera is used to synchronously capture a second image, where the second image is a multispectral original image of the first scene, and the second image is an M-channel image, where M is a positive integer greater than 3; The processor is configured to convert the second image from an M-channel image to a three-channel image through a first conversion function; The processor is further configured to perform color calibration on the first image based on a three-channel image converted from the second image to obtain first color parameters; The processor is further configured to send the first color parameter to an image signal processor (ISP) path of the first camera, so that the first camera can perform color adjustment on the image captured by the first camera according to the first color parameter.

12. The electronic device according to claim 11, wherein: The camera array further includes a second camera, wherein: The second camera is used to capture a third image, where the third image is an original image of the first scene and is a three-channel image; The processor is further configured to perform color calibration on the third image based on the three-channel image converted from the second image to obtain second color parameters; The processor is further configured to send the second color parameter to an image signal processor (ISP) path of the second camera, so that the second camera performs color adjustment on an image captured by the second camera according to the second color parameter.

13. The electronic device according to claim 11, wherein: The camera array further includes a second camera, wherein: The second camera is configured to capture a third image through the second camera when the second camera and the high-resolution multispectral camera are simultaneously turned on, where the third image is an original image of the second scene and is a three-channel image; The high-resolution multispectral camera is further used to synchronously capture a fourth image, where the fourth image is a multispectral original image of the second scene and is an M-channel image; The processor is further configured to convert the fourth image from an M-channel image into a three-channel image through a second conversion function; The processor is further configured to perform color calibration on the third image based on a three-channel image converted from the fourth image to obtain second color parameters; The processor is further configured to send the second color parameter to an image signal processor (ISP) path of the second camera, so that the second camera can perform color adjustment on the image captured by the second camera according to the second color parameter.

14. The electronic device according to claim 12 or 13, characterized in that: The first camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera; the second camera is any one of a main camera, an ultra-wide-angle camera, and a telephoto camera; and the first camera and the second camera are different.

15. The electronic device according to any one of claims 11 to 14, characterized in that: The processor is specifically configured to: The second image is converted from an M-channel image to a three-channel image by downsampling from the M channels of the second image to the three channels of the first image according to the first conversion function, wherein the first conversion function is a function of performing matrix multiplication of the second image with a downsampling matrix of size M×3.

16. The electronic device according to claim 15, characterized in that The processor is further specifically configured to: Before converting the second image from an M-channel image to a three-channel image using a first conversion function, the first image and the second image are aligned with respect to a viewing angle FOV.

17. The electronic device according to claim 16, wherein: The processor is further configured to: The first conversion function is determined based on the second image after FOV alignment.

18. The electronic device according to any one of claims 11 to 14, characterized in that: The processor is specifically configured to: performing spectral band super-resolution on the second image according to the spectral sensing curve corresponding to the second image to obtain a hyperspectral image corresponding to the second image, wherein the hyperspectral image corresponding to the second image is an N-channel image, where N is a positive integer much greater than M; The hyperspectral image corresponding to the second image is converted from an N-channel image to a three-channel image by downsampling from the N channels of the hyperspectral image corresponding to the second image to the three channels of the first image according to the first conversion function, wherein the first conversion function is a function that performs matrix multiplication on the second image by using a downsampling matrix of size N×3.

19. The electronic device according to claim 18, wherein: The processor is further configured to: The first conversion function is determined according to the hyperspectral image corresponding to the second image and the spectral sensing curve corresponding to the first image.

20. The electronic device according to any one of claims 11 to 19, characterized in that: The first color parameter includes any one or more of an automatic white balance parameter, a color adjustment parameter, a color calibration parameter, and a 3D lookup table.

21. An electronic device, characterized in that: include: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that The storage medium stores a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 10 is implemented.

23. A chip, characterized in that: Instructions are stored in the chip, and when the instructions are executed, the method according to any one of claims 1 to 10 is implemented.

24. A computer program product, characterized in that The computer program product stores a program or an instruction, and when the program or the instruction is executed, the method according to any one of claims 1 to 10 is implemented.

Images