Image color calibration method and system and computer readable storage medium

Abstract

The application provides an image color calibration method and system and a computer readable storage medium. The image color calibration method comprises: acquiring a first image, a second image and a third image; obtaining a label image according to the first image and the second image; obtaining a training model according to the second image, the third image and the label image; acquiring a fourth image and a fifth image; inputting the fourth image and the fifth image into the training model to obtain a calibrated image of the fourth image; wherein the first image is collected by a first spectrum camera; the second image and the fourth image are collected by a three-channel camera; and the third image and the fifth image are collected by a second spectrum camera. The image color calibration method and the image color calibration system can quickly and low-costly realize color calibration of images collected by the three-channel camera, the calibrated image has high-precision color and high spatial resolution, and is helpful to improve user experience.

Description

Technical Field

[0001] This invention relates generally to the field of image processing technology, and more particularly to an image color calibration method, an image color calibration system, and a computer-readable storage medium. Background Technology

[0002] Three-channel cameras are widely used in various electronic devices. With the increasing demand for photography, users have higher requirements for the quality of images captured by three-channel cameras. Existing three-channel cameras often suffer from one or more defects, such as color deviation and low spatial resolution, making it difficult to meet user needs. Therefore, how to improve the quality of images captured by three-channel cameras is the technical problem that this invention aims to solve.

[0003] The content of the background section is merely the technology known to the inventor and does not necessarily represent the prior art in this field. Summary of the Invention

[0004] To address one or more of the problems existing in the prior art, this invention provides an image color calibration method, an image color calibration system, and a computer-readable storage medium. The image color calibration method and system of this invention can quickly and cost-effectively calibrate the color of images acquired by a three-channel camera. The calibrated images possess both high color accuracy and high spatial resolution, contributing to an improved user experience.

[0005] According to a first aspect of the present invention, an image color calibration method is provided. The image color calibration method includes: acquiring a first image, a second image, and a third image; obtaining a label image based on the first image and the second image; obtaining a training model based on the second image, the third image, and the label image; acquiring a fourth image and a fifth image; inputting the fourth image and the fifth image into the training model to obtain a calibration image of the fourth image; wherein the first image is acquired using a first spectral camera; the second image and the fourth image are acquired using a three-channel camera; and the third image and the fifth image are acquired using a second spectral camera.

[0006] Optionally, the spectral response range of the second spectral camera falls within the spectral response range of the first spectral camera; the spectral resolution of the first image is higher than that of the second image; the three-channel camera includes an RGB camera or an RYB camera.

[0007] Optionally, the step of obtaining a label image based on the first image and the second image includes: obtaining the spectral curve of each pixel of the first image; obtaining an XYZ color space image of the first image based on the spectral curve of each pixel and the tristimulus curve of the human eye; and obtaining the label image based on the second image and the XYZ color space image.

[0008] Optionally, obtaining the label image based on the second image and the XYZ color space image includes: converting the second image to the LAB color space to obtain a sixth image; converting the XYZ color space image to the LAB color space to obtain a seventh image; determining the statistical characteristics of the sixth image and the seventh image; adjusting the LAB value corresponding to each pixel of the sixth image based on the statistical characteristics to obtain an eighth image; and converting the eighth image to the RGB color space to obtain the label image.

[0009] Optionally, determining the statistical characteristics of the sixth image and the seventh image includes: determining the mean and standard deviation of the LAB value corresponding to each pixel of the sixth image; and determining the mean and standard deviation of the LAB value corresponding to each pixel of the seventh image.

[0010] Optionally, adjusting the LAB value corresponding to each pixel of the sixth image based on the statistical characteristics includes: adjusting the LAB value corresponding to each pixel of the sixth image according to the mean and standard deviation of the LAB values ​​corresponding to each pixel of the sixth image and the seventh image.

[0011] Optionally, obtaining the training model based on the second image, the third image, and the label image includes: training the second image and the third image with the label image as the target to obtain the training model.

[0012] Optionally, the image color calibration method further includes: acquiring a first image, a second image, and a third image of a target in a different scene; obtaining a label image of the target in a different scene based on the first image and the second image of the target in a different scene; and obtaining a training model of the target in a different scene based on the second image, the third image, and the label image of the target in a different scene.

[0013] According to a second aspect of the present invention, an image color calibration system is provided. The image color calibration system includes a processor configured to perform the image color calibration method as described above.

[0014] Optionally, the image color calibration system further includes: a first spectral camera coupled to the processor and configured to acquire the first image; a three-channel camera coupled to the processor and configured to acquire the second image and the fourth image; a second spectral camera coupled to the processor and configured to acquire the third image and the fifth image; wherein the spectral response range of the second spectral camera falls within the spectral response range of the first spectral camera; the spectral resolution of the first image is higher than the spectral resolution of the second image; and the three-channel camera includes an RGB camera or an RYB camera.

[0015] Optionally, the first spectral camera is disposed on a first electronic device; the three-channel camera and the second spectral camera are disposed on a second electronic device.

[0016] According to a third aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes computer-executable instructions stored thereon, which, when executed by a processor, implement the image color calibration method as described above.

[0017] The image color calibration method and system of the present invention can realize the color calibration of images acquired by a three-channel camera and obtain a calibration image with high spatial resolution and high-precision color information.

[0018] During the calibration phase, the present invention acquires a first image through a first spectral camera and a second image through a three-channel camera. The first image has high-fidelity color information, and the second image has high spatial resolution. Based on the first image and the second image, a label image is obtained, which has both high-fidelity color information and high spatial resolution.

[0019] During the training phase, this invention obtains a training model based on the second image, the third image, and the label image. The second image is a color image acquired through a three-channel camera. The third image is a multispectral image acquired through a second spectral camera. Using the label image as the target, training is performed on the second and third images to obtain the training model. This model can be used to calibrate images acquired by the three-channel camera, possessing both high-fidelity color information and high spatial resolution. In obtaining the label image, a limited dataset and a simple, fast algorithm can be used, utilizing the tristimulus curve of the human eye, to obtain a label image that conforms to human color perception.

[0020] In the application phase, this invention acquires a fourth image through a three-channel camera and a fifth image through a second spectral camera. The multispectral data of the fifth image can supplement the information of the fourth image acquired by the three-channel camera. The fourth and fifth images are input into the training model. After training the model, the fourth image can be calibrated, improving the color accuracy of the images captured by the three-channel camera. This results in a calibrated image with high spatial resolution, high-precision color information, and color perception that conforms to human vision, thus enhancing the user experience.

[0021] The image color calibration method and system of the present invention combine the hardware advantages of the second spectral camera, which is small in size, light in weight and low in cost, with a three-channel camera. Based on statistical characteristics, the label image is obtained without the need for one-to-one pixel registration, which can improve processing efficiency, reduce costs and computing power, and maintain high accuracy. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the following description of the embodiments will be provided as examples. The drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation of the present invention.

[0023] Figure 1 A schematic flowchart of an exemplary image color calibration method consistent with some embodiments of the present invention is shown.

[0024] Figure 2 A schematic diagram of an exemplary image color calibration system consistent with some embodiments of the present invention is shown.

[0025] Figure 3 A schematic diagram of an exemplary image color calibration system consistent with some embodiments of the present invention is shown.

[0026] Figure 4 A schematic diagram of an exemplary image color calibration system consistent with some embodiments of the present invention is shown.

[0027] Figure 5 A schematic diagram of an exemplary second spectral camera consistent with some embodiments of the present invention is shown.

[0028] Figure 6 A flowchart illustrating an exemplary step S2 consistent with some embodiments of the present invention is shown.

[0029] Figure 7A schematic diagram of an exemplary human eye tristimulation curve consistent with some embodiments of the present invention is shown.

[0030] Figure 8 A schematic diagram of the operation flow of an exemplary sub-step S23, consistent with some embodiments of the present invention, is shown.

[0031] Figure 9 A schematic diagram of an exemplary step S5 consistent with some embodiments of the present invention is shown.

[0032] Figure 10 This diagram illustrates an exemplary comparison of the effects before and after fourth image calibration, consistent with some embodiments of the present invention. Detailed Implementation

[0033] In the following description, only certain exemplary embodiments are shown. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0034] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0035] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "coupling" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, an electrical connection, or a connection that allows for communication; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0036] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0037] The following provides many different embodiments or examples for implementing various structures of the invention. To simplify the invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the invention. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. In addition, examples of various specific processes and materials are provided in this invention, but those skilled in the art will recognize the application of other processes and / or the use of other materials.

[0038] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0039] This invention provides an image color calibration method. The image color calibration method includes: acquiring a first image, a second image, and a third image; obtaining a label image based on the first and second images; obtaining a training model based on the second, third, and label images; acquiring a fourth and a fifth image; and inputting the fourth and fifth images into the training model to obtain a calibration image of the fourth image. Specifically, the first image is acquired using a first spectral camera; the second and fourth images are acquired using a three-channel camera; and the third and fifth images are acquired using a second spectral camera.

[0040] The present invention also provides an image color calibration system. The image color calibration system includes a processor. The processor can execute an image color calibration method.

[0041] The image color calibration method and system of the present invention can quickly and cost-effectively calibrate the color of images acquired by a three-channel camera. The calibrated images have both high color accuracy and high spatial resolution, which helps to improve the user experience.

[0042] The technical solution of this invention will be described below using the example of an image color calibration system's processor executing an image color calibration method. It should be understood that the image color calibration method of this invention can also be executed by a processor outside the image color calibration system, such as a processor of a computer or other device.

[0043] Figure 1 A schematic flowchart of an exemplary image color calibration method 10 consistent with some embodiments of the present invention is shown. Figure 2 A schematic diagram of an exemplary image color calibration system 20 consistent with some embodiments of the present invention is shown. For example... Figure 1 , Figure 2 As shown, the image color calibration method 10 includes steps S1 to S5. The image color calibration system 20 includes a processor 200. The processor 200 can execute the image color calibration method 10.

[0044] In some embodiments, the processor 200 may include processing circuitry, a central processing unit (CPU), a microcontroller unit (MCU), a digital signal processor (DSP), a graphics processing unit (GPU), an accelerator, a neural processing unit (NPU), a tensor processing unit (TPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, gate devices, or transistor logic devices, or similar devices.

[0045] In some embodiments, steps S1-S5 generally include three stages: a calibration stage, a training stage, and an application stage. The calibration stage may include steps S1-S2 or step S2. The calibration stage can be understood as the stage of obtaining the labeled image. The training stage may include step S3. The training stage can be understood as the stage of training on a third image (multispectral image) to obtain a trained model. The application stage may include S4-S5. The application stage can be understood as the stage of calibrating the images acquired by the three-channel camera. It should be noted that the division of these three stages is only illustrative and the present invention is not limited thereto. The following describes each step in detail.

[0046] In some embodiments, in step S1, the processor 200 may acquire a first image IMAG1, a second image IMAG2, and a third image IMAG3. Figure 3 A schematic diagram of an exemplary image color calibration system 20 consistent with some embodiments of the present invention is shown. For example... Figure 3 As shown, the image color calibration system 20 includes a processor 200, a first spectral camera 201, a second spectral camera 202, and a three-channel camera 203. The first spectral camera 201 is coupled to the processor 200. The first spectral camera 201 can acquire a first image IMAG1. The second spectral camera 202 is coupled to the processor 200. The second spectral camera 202 can acquire a third image IMAG3. The three-channel camera 203 is coupled to the processor 200. The three-channel camera 203 can acquire a second image IMAG2.

[0047] In some embodiments, the processor 200 can acquire a first image IMAG1, a second image IMAG2, and a third image IMAG3 from the first spectral camera 201, the three-channel camera 203, and the second spectral camera 202, respectively. For example, the processor 200 can communicate with the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203. The first spectral camera 201, the second spectral camera 202, and the three-channel camera 203 can simultaneously capture raw images of the same scene target, obtaining the first image IMAG1, the third image IMAG3, and the second image IMAG2, and communicate these images to the processor 200. Thus, the processor 200 can acquire the first image IMAG1, the second image IMAG2, and the third image IMAG3.

[0048] In some embodiments, the image color calibration system 20 may include a memory (not shown). The memory may be coupled to the processor 200, the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203. A first image IMAG1, a third image IMAG3, and a second image IMAG2 may be stored in the memory. The processor 200 can retrieve the first image IMAG1, the second image IMAG2, and the third image IMAG3 from the memory. Thus, the processor 200 can also acquire the first image IMAG1, the second image IMAG2, and the third image IMAG3.

[0049] In some embodiments, the memory may include random access memory (RAM) or non-volatile memory (NVM). Further, the memory may include at least one of phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), read-only memory (ROM), and electrically erasable programmable read-only memory (EEPROM). In some embodiments, the memory may include cloud storage. In some embodiments, the memory may be implemented in hardware and / or software.

[0050] In some embodiments, the processor 200 can control the operation of at least one of the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203. For example, the processor 200 can control the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203 to synchronously or time-divisionally acquire a first image IMAG1, a third image IMAG3, and a second image IMAG2 under the same scene target. For example, the processor 200 can control the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203 to periodically acquire the first image IMAG1, the third image IMAG3, and the second image IMAG2 under the same scene target. For example, the processor 200 can control the enabling or disabling of the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203, and the enabling or disabling of acquisition operations. In some embodiments, the processor 200 can control one or more of the following shooting parameters of the images acquired by the first spectral camera 201, the second spectral camera 202, and the three-channel camera 203: frame rate, resolution, contrast, and brightness. For example, the processor 200 can control the operation of at least one of the first spectroscopic camera 201, the second spectroscopic camera 202, and the three-channel camera 203 based on user input commands. User input commands may include at least one of voice commands, gesture commands, image commands, fingerprint commands, or text commands.

[0051] In some embodiments, the first spectral camera 201 may include a hyperspectral camera or a superspectral camera. The second spectral camera 202 may include a multispectral camera. Both the first spectral camera 201 and the second spectral camera 202 can acquire spectral images. A spectral image includes multiple pixels. Each pixel includes three-dimensional data (x, y, λ). Here, x and y represent spatial dimensions, indicating the horizontal and vertical positions of the pixel in the image, and representing pixel coordinates. λ represents the spectral dimension, indicating wavelength. Spectral images can reflect the reflection, absorption, or emission characteristics of an object at different wavelengths. The spectral response range of the second spectral camera 202 falls within the spectral response range of the first spectral camera 201. The spectral response range of the second spectral camera 202 is less than or equal to the spectral response range of the first spectral camera 201. The first spectral camera 201 has more bands than the second spectral camera 202. The spectral resolution of the first spectral camera 201 is higher than that of the second spectral camera 202. The spectral continuity of the first spectral camera 201 is better than that of the second spectral camera 202. The first image IMAG1 is a hyperspectral image or a superspectral image. The third image IMAG3 is a multispectral image. The first image IMAG1 includes raw spectral image data acquired by the first spectral camera 201. In the first image IMAG1, each pixel includes hyperspectral or hyperspectral data. The third image IMAG3 includes raw spectral image data acquired by the second spectral camera 202. In the third image IMAG3, each pixel includes multispectral data. The first image IMAG1 has higher spectral resolution and richer spectral information than the third image IMAG3, and has higher fidelity color information.

[0052] In some embodiments, the three-channel camera 203 includes an RGB camera or an RYB camera. Both RGB and RYB cameras have their advantages; RGB cameras offer better color performance, while RYB cameras are more suitable for shooting in nighttime or low-light environments. In some embodiments, the three-channel camera 203 may include an ultra-wide-angle color camera, a wide-angle color camera, a telephoto color camera, or a macro color camera, etc. In practical applications, a suitable three-channel camera can be selected according to requirements. The three-channel camera 203 can capture a second image IMAG2. The second image IMAG2 is a color image. The second image IMAG2 includes the raw color image data captured by the three-channel camera 203. The second image IMAG2 has a high spatial resolution. The spatial resolution of the second image IMAG2 is higher than that of the first image IMAG1 and the third image IMAG3. The spatial resolution of the first image IMAG1 and the third image IMAG3 can be the same or different, depending on the actual needs. In some embodiments, the number of channels of the three-channel camera 203 is less than the number of channels of the second spectral camera 202. The number of channels of the second spectral camera 202 is less than the number of channels of the first spectral camera 201. In some embodiments, the spectral resolution of the first spectral camera 201 is higher than that of the second spectral camera 202. The spectral resolution of the second spectral camera 202 is higher than that of the three-channel camera 203.

[0053] In some embodiments, such as Figure 3 As shown, a first spectral camera 201 is disposed on a first electronic device E1. The first electronic device E1 may include a hyperspectral camera, a hyperspectral analyzer, a hyperspectral camera, a hyperspectral analyzer, etc. The first electronic device E1 may include components such as gratings and FP-cavity types. In some embodiments, a second spectral camera 202 and a three-channel camera 203 are disposed on a second electronic device E2. The three-channel camera 203 can serve as the main camera of the second electronic device E2. The second spectral camera 202 can serve as a secondary camera of the second electronic device E2. The second electronic device E2 may include mobile phones, tablets, laptops, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, ultramobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices, and other electronic devices. This invention does not limit the specific types of the first and second electronic devices; in practical applications, they can be configured according to requirements.

[0054] In some embodiments, the first electronic device E1 and the second electronic device E2 may each include a memory. A first image IMAG1 may be stored in the memory of the first electronic device E1. A second image IMAG2 and a third image IMAG3 may be stored in the memory of the second electronic device E2. In some embodiments, the first electronic device E1 and the second electronic device E2 may share the memory. For example, the memory may be built into the first electronic device E1. For example, the memory may be built into the second electronic device E2. For example, the memory may be partially built into the first electronic device E1 and partially built into the second electronic device E2. For example, the memory may be externally located in both the first electronic device E1 and the second electronic device E2. In some embodiments, the memory may be built into and / or externally located in the processor 200.

[0055] In some embodiments, refer to Figure 3 The processor 200 can be externally located in the first electronic device E1 and the second electronic device E2. Figure 4 A schematic diagram of an exemplary image color calibration system 20 consistent with some embodiments of the present invention is shown. For example... Figure 4 As shown, processor 200 can be built into the second electronic device E2. In some embodiments, processor 200 can be built into the second spectroscopic camera 202. In some embodiments, processor 200 can be built into the three-channel camera 203. In some embodiments, the second spectroscopic camera 202 and the three-channel camera 203 can each include processor 200. In some embodiments, the second spectroscopic camera 202 and the three-channel camera 203 can share processor 200.

[0056] Figure 5 A schematic diagram of an exemplary second spectral camera 202 consistent with some embodiments of the present invention is shown. For example... Figure 5 As shown, the second spectral camera 202 includes a microlens array 2021, a filter unit array 2022, and a photoelectric sensor array 2023. The processor 200 can be coupled to the photoelectric sensor array 2023. Although not shown in the figure, the upstream of the optical path of the microlens array 2021 may also include a lens or lens group.

[0057] like Figure 5 As shown, the microlens array 2021 includes multiple microlenses 20210. The multiple microlenses 20210 can be arranged in a one-dimensional or two-dimensional array. The microlenses 20210 can converge the incident light L. The incident surface of the microlenses 20210 can include an anti-reflection coating, which can improve light throughput, reduce reflection, and suppress noise.

[0058] like Figure 5As shown, the filter unit array 2022 is disposed downstream of the optical path of the microlens array 2021. The filter unit array 2022 includes a plurality of filter units 20220. Each filter unit 20220 includes a plurality of filter sub-units 20222. The plurality of filter sub-units 20222 can be arranged in a one-dimensional array or a two-dimensional array. In some embodiments, the plurality of filter sub-units 20222 in each filter unit 20220 can be arranged in an n*n array, where n is a positive integer. For example, see... Figure 5 The image above shows a partially enlarged view of the filter unit array 2022. The filter unit 20220 includes nine filter sub-units 20222, arranged in a 3x3 array. For example, the filter unit 20220 may include four filter sub-units 20222, arranged in a 2x2 array. For example, the filter unit 20220 may include sixteen filter sub-units 20222, arranged in a 4x4 array. In some embodiments, multiple filter sub-units 20222 of the filter unit 20220 may be arranged in an n*m array, where n and m are positive integers, and n≠m. For example, the filter unit 20220 may include six filter sub-units 20222, arranged in a 2x3 array. For example, the filter unit 20220 may include 12 filter sub-units 20222, and the 12 filter sub-units 20222 may be arranged in a 3*4 filter sub-unit array. It should be noted that the present invention does not limit the number and arrangement of filter sub-units in each filter unit; in practical applications, it can be set according to requirements.

[0059] In some embodiments, the filter subunit 20222 may have a specific transmittance profile. The filter subunit 20222 may allow incident light of a preset wavelength band to pass through. One filter subunit 20222 may form a channel. One channel may allow incident light of a preset wavelength band to pass through. In some embodiments, the filter subunit 20222 may include a filter film, filter, nanoarray, grating, or similar device to achieve wavelength selectivity.

[0060] In some embodiments, the transmittance profiles of the individual filter sub-units 20222 within the same filter unit 20220 may be different. For example, the transmittance profiles of the individual filter sub-units 20222 within the same filter unit 20220 may be different for each individual unit. For instance, the filter unit 20220 may include nine filter sub-units 20222, each with a different transmittance profile. In this configuration, one filter sub-unit 20222 can form one channel. The nine filter sub-units 20222 of the filter unit 20220 can form nine channels CH1 to CH9, allowing incident light of nine wavelength bands λ1 to λ9 to pass through, respectively. For example, the transmittance profiles of the individual filter sub-units 20222 within the same filter unit 20220 may be partially the same and partially different. For example, in the filter unit 20220, there are nine filter sub-units 20222, two of which have the same transmittance curve, while the other seven have different transmittance curves. In this configuration, the nine filter sub-units 20222 of the filter unit 20220 can form eight channels, allowing incident light of eight different wavelengths to pass through. It should be noted that this example only uses the filter unit 20220 with nine filter sub-units 20222 as an illustration, and the invention is not limited thereto. In practical applications, the number, arrangement, transmittance, number of channels, and device type of the filter unit can all be configured according to requirements.

[0061] like Figure 5 As shown, the photoelectric sensor array 2023 is disposed downstream of the optical path of the filter unit array 2022. The photoelectric sensor array 2023 includes multiple photoelectric sensors 20230. The multiple photoelectric sensors 20230 can be arranged in a one-dimensional array or a two-dimensional array. The photoelectric sensors 20230 can receive incident light of a preset wavelength band passing through the filter subunit 20222 and perform photoelectric conversion. In some embodiments, the photoelectric sensor 20230 may include one or more pixels 20233. The multiple pixels 20233 can be arranged in a one-dimensional or two-dimensional array. The pixel 20233 can serve as the smallest photosensitive unit of the photoelectric sensor array 2023, converting the light signal incident on it into an electrical signal. In some embodiments, the pixel 20233 may include photosensitive elements such as photodiodes or phototransistors, for example, charge-coupled devices (CCDs) or complementary metal-oxide-semiconductor (CMOS) devices.

[0062] like Figure 2-5As shown, the processor 200 can be coupled to the photoelectric sensor array 2023. The microlens array 2021, the filter unit array 2022, the photoelectric sensor array 2023, and the processor 200 can be disposed on a single spectral chip, which helps to improve integration and reduce the overall size. The processor 200 can generate a multispectral image based on the electrical signal output by the photoelectric sensor array 2023. In some embodiments, one microlens 20210 can correspond to one filter subunit 20222. One filter subunit 20222 can correspond to one pixel 20233. One pixel 20233 can correspond to a pixel in the spectral image. The processor 200 can determine the band corresponding to the filter subunit 20222 and determine the gray value of the pixel based on the electrical signal output by the pixel 20233. It should be noted that the present invention does not limit the correspondence between the microlens 20210, the filter subunit 20222, the pixel 20233, and the pixel. Optionally, one microlens 20210 can correspond to multiple filter sub-units 20222. Optionally, one filter sub-unit 20222 can correspond to multiple pixels 20233. Optionally, multiple pixels 20233 can correspond to one pixel. In practical applications, the configuration can be adjusted according to requirements.

[0063] In some embodiments, in step S2, the processor 200 can obtain a label image IMAGLabel based on a first image IMAG1 and a second image IMAG2. The first image IMAG1 is a hyperspectral or superspectral image. The first image IMAG1 has high-fidelity color information. The second image IMAG2 is a color image. The second image IMAG2 has high spatial resolution. The processor 200 can process the first image IMAG1 and the second image IMAG2 to obtain the label image IMAGLabel. The label image IMAGLabel possesses both high-fidelity color information and high spatial resolution.

[0064] Figure 6 A flowchart illustrating an exemplary step S2 consistent with some embodiments of the present invention is shown. For example... Figure 6 As shown, step S2 includes sub-steps S21-S23. Processor 200 executes sub-steps S21-S23 to obtain the label image IMAGLabel.

[0065] In some embodiments, in sub-step S21, the processor 200 can acquire the spectral curve of each pixel of the first image IMAG1. The first image IMAG1 is a hyperspectral image or a hyperspectral image. The first image IMAG1 includes multiple pixels. Each pixel includes hyperspectral data or hyperspectral data. The processor 200 can perform spectral inversion on the first image IMAG1, extract the wavelength information of each pixel from the first image IMAG1, and generate a spectral reflectance characteristic curve for each pixel. Each pixel corresponds to a high-precision spectral reflectance characteristic curve. The horizontal axis of the spectral reflectance characteristic curve represents wavelength, and the vertical axis represents spectral reflectance. The spectral reflectance characteristic curve can reflect the relationship between the reflectance of an object and wavelength. Spectral reflectance can describe the reflectance characteristics exhibited by the surface of an object under light of various wavelengths.

[0066] In some embodiments, in sub-step S22, the processor 200 can obtain an XYZ color space image of the first image IMAG1 based on the spectral curve of each pixel of the first image IMAG1 and the tristimulus curve of the human eye. The spectral curve includes a spectral reflectance characteristic curve. Figure 7 A schematic diagram of an exemplary human eye tristimulation curve consistent with some embodiments of the present invention is shown. For example... Figure 7 As shown, the horizontal axis represents wavelength, and the vertical axis represents tristimulus values. The wavelength range is from 380nm to 780nm. x(λ), y(λ), and z(λ) are the spectral tristimulus values ​​of a standard observer, which can describe the human eye's color perception. The three components x, y, and z represent the human eye's sensitivity to brightness, red-green, and blue-yellow colors, respectively. When calculating the tristimulus values, the processor 200 can multiply the reflectance spectrum of the spectral curve of each pixel of the first image IMAG1 with the tristimulus values ​​of the human eye's tristimulus curve according to the corresponding wavelength to obtain the tristimulus values ​​of each pixel at each wavelength. By traversing all pixels of the first image IMAG1, the processor 200 can determine the tristimulus values ​​of all pixels of the first image IMAG1 at each wavelength, thereby obtaining the XYZ color space image IMAGXYZ of the first image IMAG1. The XYZ color space image IMAGXYZ comes from the first image IMAG1 and the human eye's tristimulus curve, and not only has high-fidelity color information, but also conforms to the human eye's color perception.

[0067] In sub-step S23, the processor 200 can obtain the label image IMAGLabel based on the second image IMAG2 and the XYZ color space image IMAGXYZ. The second image IMAG2 is a color image with high spatial resolution. The XYZ color space image IMAGXYZ has high-precision color information, which conforms to the human eye's color perception. The processor 200 can perform style transfer from the second image IMAG2 to the XYZ color space image IMAGXYZ to obtain the label image IMAGLabel. The label image IMAGLabel combines high-precision color and high spatial resolution, and conforms to the human eye's color perception.

[0068] Figure 8 A schematic diagram illustrating the operation flow of an exemplary sub-step S23, consistent with some embodiments of the present invention, is shown. For example... Figure 8 As shown, sub-step S23 includes operations OP1-OP5.

[0069] In some embodiments, by operating OP1, the processor 200 can convert the second image IMAG2 to the LAB color space to obtain the sixth image IMAG6. The second image IMAG2 is a color image. For example, the second image IMAG2 is an RGB image. The processor 200 can convert the second image IMAG2 from the RGB color space to the LAB color space (luminance channel L, chrominance channels A and B) based on the conversion relationship between the RGB color space and the LAB color space. For example, the second image IMAG2 is an RYB image. The processor 200 can convert the RYB image to an RGB image, and then convert the RGB image from the RGB color space to the LAB color space. The processor 200 can use the XYZ color space as an intermediate medium, first converting the second image IMAG2 from the RGB color space to the XYZ color space based on the conversion relationship between the RGB color space and the XYZ color space, and then converting it from the XYZ color space to the LAB color space based on the conversion relationship between the XYZ color space and the LAB color space, to obtain the sixth image IMAG6. The sixth image IMAG6 is a LAB color space image. Compared to the second image IMAG2, the sixth image IMAG6 has a wider color gamut, can express richer color information, and conforms to the characteristics of human vision.

[0070] In some embodiments, during operation OP2, the processor 200 can convert the XYZ color space image IMAGXYZ to the LAB color space to obtain the seventh image IMAG7. The processor 200 can convert the XYZ color space image IMAGXYZ from the XYZ color space to the LAB color space based on the conversion relationship between the XYZ and LAB color spaces, thus obtaining the seventh image IMAG7. The seventh image IMAG7 is a LAB color space image. After conversion, the seventh image IMAG7 has a wider color gamut, richer, more delicate, and more uniform colors, higher color fidelity and accuracy, and conforms to the human eye's color perception.

[0071] In some embodiments, during operation OP3, processor 200 can determine the statistical characteristics of the sixth image IMAG6 and the seventh image IMAG7. Both the sixth image IMAG6 and the seventh image IMAG7 are images in the LAB color space. Processor 200 can determine the mean and standard deviation of the LAB values ​​corresponding to each pixel of the sixth image IMAG6, thus determining the statistical characteristics of the sixth image IMAG6. Similarly, processor 200 can determine the mean and standard deviation of the LAB values ​​corresponding to each pixel of the seventh image IMAG7, thus determining the statistical characteristics of the seventh image IMAG7.

[0072] In some embodiments, the processor 200 may determine the mean LAB value corresponding to each pixel of the sixth image IMAG6 based on (Equation 1). In other words, This represents the average values ​​of the L, A, and B channels of the sixth image IMAG6.

[0073] ... (Equation 1).

[0074] Where N represents the number of pixels in the sixth image IMAG6, and N is a positive integer. This represents the grayscale value of the i-th pixel in the p-channel of the sixth image IMAG6. The p-channel can be one of the L-channel, A-channel, or B-channel. According to Equation 1, the processor 200 can obtain the average values ​​of the L, A, and B channels of the entire sixth image IMAG6.

[0075] In some embodiments, the processor 200 may determine the standard deviation of the LAB value corresponding to each pixel of the sixth image IMAG6 based on (Equation 2). In other words, This represents the standard deviation of each of the L, A, and B channels of the sixth image IMAG6.

[0076] ... (Equation 2).

[0077] Where N represents the number of pixels in the sixth image IMAG6. According to Equation 2, the processor 200 can obtain the standard deviation of each channel (L, A, B) of the entire sixth image IMAG6.

[0078] In some embodiments, the processor 200 may determine the mean LAB value corresponding to each pixel of the seventh image IMAG7 based on (Equation 3). In other words, This represents the average values ​​of the L, A, and B channels of the seventh image IMAG7.

[0079] ... (Equation 3).

[0080] Where M represents the number of pixels in the seventh image IMAG7, and M is a positive integer. This represents the grayscale value of the i-th pixel in the p-channel of the seventh image IMAG7. The p-channel can be one of the L-channel, A-channel, or B-channel. According to Equation 3, the processor 200 can obtain the average values ​​of the L, A, and B channels of the entire seventh image IMAG7.

[0081] It should be noted that this invention does not limit the relationship between the resolutions of the sixth image IMAG6 and the seventh image IMAG7, nor does it limit the relationship between the number of pixels N and M of the sixth image IMAG6 and the seventh image IMAG7. In practical applications, these can be set according to requirements. For example, the resolution of the sixth image IMAG6 may include 1600*1384, and the resolution of the seventh image IMAG7 may include 1440*900, etc.

[0082] In some embodiments, the processor 200 may determine the standard deviation of the LAB value corresponding to each pixel of the seventh image IMAG7 based on (Equation 4). In other words, This represents the standard deviation of each of the L, A, and B channels of the seventh image IMAG7.

[0083] ... (Equation 4).

[0084] According to (Equation 4), the processor 200 can obtain the standard deviation of each channel of the L, A, and B channels of the full image IMAG7 of the seventh image.

[0085] In some embodiments, during operation OP4, the processor 200 can adjust the LAB value corresponding to each pixel of the sixth image IMAG6 based on statistical characteristics to obtain the eighth image IMAG8.

[0086] In some embodiments, the processor 200 can adjust the LAB value corresponding to each pixel of the sixth image based on the mean and standard deviation of the LAB values ​​corresponding to each pixel of the sixth and seventh images. For example, the processor 200 can adjust the LAB value based on the mean and standard deviation of the LAB values ​​of the sixth image IMAG6. Standard deviation The mean of the seventh image IMAG7 Standard deviation The LAB values ​​corresponding to each pixel of the sixth image IMAG6 are adjusted to obtain the eighth image IMAG8. The eighth image IMAG8 is a LAB color space image.

[0087] In some embodiments, the processor 200 can adjust the LAB value corresponding to each pixel of the sixth image IMAG6 based on (Equation 5), and by traversing all pixels, the eighth image IMAG8 can be obtained.

[0088] ... (Equation 5).

[0089] in, This represents the grayscale value of the i-th pixel in the p channel of the eighth image IMAG8. This represents the average values ​​of the L, A, and B channels of the seventh image IMAG7. This represents the standard deviation of each of the L, A, and B channels of the seventh image IMAG7. This represents the standard deviation of each of the L, A, and B channels of the sixth image IMAG6. This represents the grayscale value of the i-th pixel in the p channel of the sixth image IMAG6. This represents the average values ​​of the L, A, and B channels of the sixth image IMAG6.

[0090] It is understandable that the resolution of the eighth image IMAG8 is the same as that of the sixth image IMAG6. The texture features of the eighth image IMAG8 are derived from the sixth image IMAG6. The color features of the eighth image IMAG8 are derived from the seventh image IMAG7 and are the same as or similar to the seventh image IMAG7.

[0091] In some embodiments, during operation OP5, the processor 200 can convert the eighth image IMAG8 to the RGB color space to obtain the label image IMAGLabel. The eighth image IMAG8 is a LAB color space image. The processor 200 can first convert the eighth image IMAG8 from the LAB color space to the XYZ color space based on the conversion relationship between the LAB color space and the XYZ color space. Then, based on the conversion relationship between the XYZ color space and the RGB color space, it can convert it from the XYZ color space to the RGB color space to obtain the label image IMAGLabel. The label image IMAGLabel is an RGB color space image and has both high-precision color and high spatial resolution.

[0092] Based on the second image IMAG2 and the XYZ color space image IMAGXYZ, and using color space conversion and statistical characteristics, the processor 200 can perform style transfer from the second image IMAG2 to the XYZ color space image IMAGXYZ, obtaining a label image IMAGLabel with high-precision color information and high spatial resolution. This invention processes based on statistical characteristics, eliminating the need for one-to-one pixel registration, saving computational power, improving processing efficiency, and without sacrificing color accuracy and spatial resolution, achieving a balance between high color accuracy, high spatial resolution, high efficiency, and low computational power.

[0093] In the aforementioned embodiment, in step S2, the processor 200 obtains the label image IMAGLabel based on the first image IMAG1 and the second image IMAG2. Alternatively, the processor 200 can also obtain the label image IMAGLabel using a standard color chart and the second image IMAG2. Specifically, this may include the following operations: Under a characteristic light source environment, the processor 200 can acquire the reflected light spectrum curve of each color card in the standard color chart. The processor 200 can use the spectrum curve and the tristimulus curve of the human eye to obtain the XYZ values ​​of each color block of the standard color chart under this light source. Converting the XYZ values ​​of each color block of the standard color chart under this light source to the RGB color space, the RGB values ​​of each color block in the standard color chart under this light source can be obtained. Under the same light source environment, the three-channel camera 203 acquires raw image data with the standard color chart. The processor 200 performs ISP processing on the raw data to obtain an RGB image. The processor 200 adjusts the second image IMAG2 according to the RGB values ​​of the color blocks to obtain the label image IMAGLabel.

[0094] In some embodiments, in step S3, the processor 200 can obtain a training model based on the second image IMAG2, the third image IMAG3, and the label image IMAGLabel. The second image IMAG2 is a color image. The second image IMAG2 has high spatial resolution, but slightly lower color accuracy. The third image IMAG3 is a multispectral image, with better color accuracy than the second image IMAG2, but slightly lower spatial resolution. The label image IMAGLabel combines high color accuracy and high spatial resolution. Based on the second image IMAG2, the third image IMAG3, and the label image IMAGLabel, the processor 200 can obtain a training model that combines high color accuracy and high spatial resolution.

[0095] In some embodiments, the processor 200 can train a training model on a second image IMAG2 and a third image IMAG3, targeting the label image IMAGLabel. The training model possesses not only high-precision color information but also high spatial resolution information. The training model can be used to calibrate images acquired by the three-channel camera 203.

[0096] In some embodiments, the processor 200 can train a training model on the second image IMAG2 and the third image IMAG3, using the label image IMAGLabel as the target, based on a neural network algorithm. In some embodiments, the neural network algorithm includes a convolutional neural network (CNN). The processor 200 can train the second image IMAG2 and the third image IMAG3 based on a convolutional neural network (CNN). For example, the processor 200 can extract features from the label image IMAGLabel, the second image IMAG2, and the third image IMAG3 using convolutional kernels based on the convolutional neural network (CNN). Image features may include one or more features such as color features, texture features, shape features, spatial relationship features, edge features, contour features, corner features, and semantic features. For example, the processor 200 can extract high-precision color features (color features) and high spatial resolution features (texture features) from the label image IMAGLabel. For example, the processor 200 can extract texture features from the second image IMAG2. The processor 200 can extract multispectral features (color features) from the third image IMAG3. The processor 200 targets the high-precision color features and high spatial resolution features of the label image IMAGLabel, and trains it on the second image IMAG2 and the third image IMAG3 to obtain a trained model that simultaneously possesses high-precision color features and high spatial resolution features. It should be noted that the second image IMAG2 and the third image IMAG3 are images of the same target scene.

[0097] It should be noted that this invention does not limit the type of neural network. In some embodiments, the neural network algorithm may include at least one of recurrent neural networks (RNN), LSTM neural networks, Transformer neural networks, etc. In practical applications, a suitable neural network algorithm can be selected according to requirements. In some embodiments, the neural network algorithm may be built into the processor 200 or memory as program instructions. In some embodiments, the neural network model may be built into the processor 200 or memory. By executing the neural network algorithm, the processor 200 can drive the neural network model, and can train color images (e.g., the second image IMAG2, etc.) and multispectral images (e.g., the third image IMAG3, etc.) to obtain a trained model.

[0098] In some embodiments, the processor 200 can acquire first, second, and third images of different scene targets. Based on the first and second images of different scene targets, the processor 200 can obtain label images corresponding to different scene targets. Based on the second, third, and label images of different scene targets, the processor 200 can obtain training models for different scene targets. For example, based on the second, third, and label images of different scene targets, the processor 200 can obtain a training dataset. Based on the training dataset of different scene targets, the processor 200 can obtain a training model. For example, different scene targets may include outdoor scene targets such as grasslands, beaches, sunny beaches, parks, forests, deserts, animals, and plants. For example, different scene targets may include indoor scene targets such as hotels, restaurants, KTVs, offices, shopping malls, libraries, exhibition halls, museums, classrooms, factories, and hospitals. The training dataset includes training images under various scene targets and can serve as a training image database. The processor 200 can train the training dataset to obtain a training model. The training model can be used to calibrate color images acquired by the three-channel camera 203. In some embodiments, the training dataset can be stored in memory. In some embodiments, the training model may include a neural network model. This training model may include at least one neural network algorithm selected from convolutional neural networks (CNN), recurrent neural networks (RNN), LSTM neural networks, Transformer neural networks, etc.

[0099] In some embodiments, in step S4, the processor 200 can acquire a fourth image IMAG4 and a fifth image IMAG5. The fourth image IMAG4 can be acquired by a three-channel camera 203. The processor 200 acquires the fourth image IMAG4 in the same or similar manner as it acquires the second image IMAG2. The fourth image IMAG4 is a color image. The fourth image IMAG4 includes raw color image data acquired by the three-channel camera 203. The fourth image IMAG4 has high spatial resolution. The fifth image IMAG5 can be acquired by a second spectral camera 202. The processor 200 acquires the fifth image IMAG5 in the same or similar manner as it acquires the third image IMAG3. The fifth image IMAG5 is a multispectral image. It should be noted that the fourth image IMAG4 and the fifth image IMAG5 are images of the same scene target.

[0100] In some embodiments, in step S5, the processor 200 can input the fourth image IMAG4 and the fifth image IMAG5 into the training model to obtain a calibration image of the fourth image IMAG4. Figure 9 A schematic diagram illustrating an exemplary step S5 consistent with some embodiments of the present invention is shown. For example... Figure 9 As shown, processor 200 can input the fourth image IMAG4 and the fifth image IMAG5 into training model M. After training model M, it can output a calibration image IMAG4' of the fourth image IMAG4. Training model M can include at least one neural network algorithm such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), LSTM Neural Network, and Transformer Neural Network. Processor 200 executes the neural network algorithm, inputs the fourth image IMAG4 and the fifth image IMAG5 into training model M, and after training model M, it can obtain and output the calibration image IMAG4' of the fourth image IMAG4. It can be understood that the fourth image IMAG4 comes from the three-channel camera 203, is a three-channel color image with high spatial resolution, but limited color information and color deviation. The fifth image IMAG5 comes from the second spectral camera 202, is a multispectral image, has more channels than the fourth image IMAG4, and has richer color information than the fourth image IMAG4. Training model M has both high-precision color features and high spatial resolution features. The processor 200 uses the training model M to fuse the fourth image IMAG4 and the fifth image IMAG5, which can calibrate the color of the fourth image IMAG4 to obtain a calibrated image IMAG4' with high spatial resolution and high-precision color, and is suitable for human color perception. Figure 10 This diagram illustrates an exemplary comparison of the effects of fourth image IMAG4 calibration before and after calibration, consistent with some embodiments of the present invention. For example... Figure 10As shown, the left image is the fourth image IMAG4 before calibration. The right image is the calibrated image IMAG4' of the fourth image IMAG4. Clearly, after calibration, the image colors are more vibrant and detailed, closer to the true colors observed by the human eye, significantly improving image quality.

[0101] In some embodiments, the processor 200 may preprocess at least one of the first image IMAG1, the second image IMAG2, the third image IMAG3, the fourth image IMAG4, and the fifth image IMAG5. For example, preprocessing may include ISP processing. ISP processing may include at least one of black level compensation (BLC), lens shading correction (LSC), bad pixel correction, color interpolation, noise removal, white balance (AWB) correction, color correction, gamma correction, edge enhancement, and contrast enhancement. This helps reduce image distortion to achieve more accurate color reproduction.

[0102] The image color calibration method and system of the present invention can realize the color calibration of images acquired by a three-channel camera and obtain a calibration image with high spatial resolution and high-precision color information.

[0103] During the calibration phase, the present invention acquires a first image through a first spectral camera and a second image through a three-channel camera. The first image has high-fidelity color information, and the second image has high spatial resolution. Based on the first image and the second image, a label image is obtained, which has both high-fidelity color information and high spatial resolution.

[0104] During the training phase, this invention obtains a training model based on the second image, the third image, and the label image. The second image is a color image acquired through a three-channel camera. The third image is a multispectral image acquired through a second spectral camera. Using the label image as the target, training is performed on the second and third images to obtain the training model. This model can be used to calibrate images acquired by the three-channel camera, possessing both high-fidelity color information and high spatial resolution. In obtaining the label image, a limited dataset and a simple, fast algorithm can be used, utilizing the tristimulus curve of the human eye, to obtain a label image that conforms to human color perception.

[0105] In the application phase, this invention acquires a fourth image through a three-channel camera and a fifth image through a second spectral camera. The multispectral data of the fifth image can supplement the information of the fourth image acquired by the three-channel camera. The fourth and fifth images are input into the training model. After training the model, the fourth image can be calibrated, improving the color accuracy of the images captured by the three-channel camera. This results in a calibrated image with high spatial resolution, high-precision color information, and color perception that conforms to human vision, thus enhancing the user experience.

[0106] The image color calibration method and system of the present invention combine the hardware advantages of the second spectral camera, which is small in size, light in weight and low in cost, with a three-channel camera. Based on statistical characteristics, the label image is obtained without the need for one-to-one pixel registration, which can improve processing efficiency, reduce costs and computing power, and maintain high accuracy.

[0107] The present invention also provides a computer-readable storage medium. The computer-readable storage medium includes computer-executable instructions stored thereon, which, when executed by a processor, implement the image color calibration method 10 as described above.

[0108] In some embodiments, the present invention may take the form of a computer program product implemented on one or more storage media containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: PRAM, SRAM, DRAM, other types of RAM, ROM, EEPROM, flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0109] It should be noted that this specification provides method operation steps as shown in the embodiments or diagrams, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. In actual system or device products, the methods shown in the embodiments or flowcharts can be executed sequentially or in parallel.

[0110] It should be noted that although several modules of the image color calibration system have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be implemented in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0111] It should be noted that the present invention may include Figure 1-10 Any one or more features of any one or more embodiments. In other words, not all features shown in the figures need to be implemented simultaneously in the image color calibration method / image color calibration system of the present invention.

[0112] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image color calibration method, characterized by, The method comprises: acquiring a first image, a second image and a third image; obtaining a label image according to the first image and the second image; obtaining a training model according to the second image, the third image and the label image; acquiring a fourth image and a fifth image; inputting the fourth image and the fifth image into the training model to obtain a calibrated image of the fourth image; wherein the first image is acquired by a first spectral camera; the first image is a hyperspectral image or an ultra-spectral image; the second image and the fourth image are acquired by a three-channel camera; the third image and the fifth image are acquired by a second spectral camera; the spectral response range of the second spectral camera falls within the spectral response range of the first spectral camera; the spectral resolution of the first image is higher than the spectral resolution of the second image; wherein obtaining the label image comprises: acquiring a spectral curve of each pixel of the first image; obtaining an XYZ color space image of the first image according to the spectral curve of each pixel and a human eye tristimulus curve; obtaining the label image according to the second image and the XYZ color space image, comprising: converting the second image to LAB color space to obtain a sixth image; converting the XYZ color space image to LAB color space to obtain a seventh image; determining statistical characteristics of the sixth image and the seventh image; adjusting the LAB value corresponding to each pixel of the sixth image based on the statistical characteristics to obtain an eighth image; converting the eighth image to RGB color space to obtain the label image; wherein obtaining the training model comprises: training the second image and the third image with the label image as a target to obtain the training model.

2. The image color calibration method of claim 1, wherein, The three-channel camera comprises an RGB camera or an RYB camera.

3. The image color calibration method according to claim 1, wherein determining the statistical characteristics of the sixth image and the seventh image comprises: determining the mean and standard deviation of the LAB value corresponding to each pixel of the sixth image; determining the mean and standard deviation of the LAB value corresponding to each pixel of the seventh image; adjusting the LAB value corresponding to each pixel of the sixth image based on the statistical characteristics comprises: adjusting the LAB value corresponding to each pixel of the sixth image according to the mean and standard deviation of the LAB value corresponding to each pixel of the sixth image and the seventh image.

4. The image color calibration method according to any one of claims 1-3, characterized in that, The method further comprises: acquiring a first image, a second image and a third image of different scene targets; obtaining a label image of different scene targets according to the first image and the second image of the different scene targets; obtaining a training model of different scene targets according to the second image, the third image and the label image of the different scene targets.

5. An image color calibration system, characterized by, The method comprises: a processor configured to perform the image color calibration method according to any one of claims 1-4.

6. The image color calibration system of claim 5, wherein, The method further comprises: a first spectral camera coupled to the processor and configured to acquire the first image; a third channel camera coupled to the processor and configured to capture the second image and the fourth image; a second spectral camera coupled to the processor and configured to capture the third image and the fifth image; wherein a spectral response range of the second spectral camera falls within a spectral response range of the first spectral camera; a spectral resolution of the first image is higher than a spectral resolution of the second image; and the third channel camera comprises an RGB camera or an RYB camera.

7. The image color calibration system of claim 6, wherein, The first spectral camera is disposed on a first electronic device; and the third channel camera and the second spectral camera are disposed on a second electronic device.

8. A computer-readable storage medium, characterized in that, computer executable instructions stored thereon that, when executed by a processor, perform the image color calibration method of any one of claims 1-4.

Images