Color reproduction apparatus and method, image processor

Abstract

The present disclosure relates to a color restoration device and method, and an image processor. The color restoration device comprises a spectrometer module configured to acquire a color temperature distribution image; and an image signal processor configured to perform color restoration processing on each color sub-region at a corresponding position in a second image region of a to-be-processed image according to color temperature data of each color temperature sub-region in a first image region of the color temperature distribution image, wherein a similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold. After the color restoration processing, the color of an image captured by a camera module under various light conditions is almost identical to the color of a scene viewed by a human eye.

Description

Technical Field

[0001] This disclosure relates to the field of camera product technology, and in particular to color reproduction apparatus and method, and image processor. Background Technology

[0002] Currently, image sensors cannot detect and correct color changes under different lighting conditions, and their output images may exhibit color distortion. Therefore, images acquired by image sensors need to undergo white balance processing to achieve correct color balance. Summary of the Invention

[0003] This disclosure provides a color restoration apparatus and method, as well as an image processor, to address the shortcomings of related technologies.

[0004] According to a first aspect of the present disclosure, a color reproduction apparatus is provided, comprising:

[0005] Spectrometer module, used to acquire color temperature distribution images;

[0006] An image signal processor is configured to perform color restoration processing on each color sub-region at a corresponding position in a second image region of the image to be processed, based on the color temperature data of each color temperature sub-region in a first image region of the color temperature distribution image, wherein the similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold.

[0007] Optionally, the spectrometer module includes:

[0008] Spectrometer lens;

[0009] A filtering component is used to filter ambient light passing through the lens of the spectrometer so that light of different wavelengths can pass through in sequence.

[0010] An image sensor is used to convert light of different wavelengths into the color temperature distribution image.

[0011] Optionally, the filtering component includes:

[0012] Top flat glass;

[0013] The lower flat glass is arranged parallel to the upper flat glass, and the lower flat glass and the upper flat glass form a planar parallel cavity;

[0014] A driving assembly, connected to at least one of the upper and lower flat glass plates, adjusts the height of the planar parallel cavity by driving at least one of the upper and lower flat glass plates to move, wherein the wavelength of light transmitted from the planar parallel cavity corresponds to the height.

[0015] Optionally, the image sensor is a black and white image sensor.

[0016] According to a second aspect of the present disclosure, a color reproduction method is provided, comprising:

[0017] Obtain the color temperature distribution image;

[0018] Based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image, color restoration processing is performed on each color sub-region at the corresponding position in the second image region of the image to be processed, wherein the similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold.

[0019] Optionally, acquiring the color temperature distribution image includes:

[0020] During the process of the camera module acquiring the image to be processed, the color temperature distribution image acquired by the spectrometer module is obtained; wherein, the first image region of the color temperature distribution image overlaps with the second image region of the image to be processed.

[0021] Optionally, before performing color restoration processing on each color sub-region at a corresponding position in the second image region of the image to be processed, the method further includes:

[0022] Based on the coordinate calibration results of the spectrometer module and the camera module, the positional correspondence between each color temperature sub-region and each color sub-region is determined.

[0023] Optionally, the step of performing color restoration processing on corresponding color sub-regions in the second image region of the image to be processed based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image includes:

[0024] Based on the color temperature data of each pixel in the color temperature sub-region, color restoration processing is performed on each pixel at the corresponding position in the color sub-region.

[0025] According to a third aspect of the present disclosure, an image processor is provided, comprising:

[0026] The acquisition module is used to acquire color temperature distribution images;

[0027] The processing module is used to perform color restoration processing on each color sub-region at a corresponding position in the second image region of the image to be processed based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image, wherein the similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold.

[0028] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0029] processor;

[0030] Memory used to store processor-executable instructions;

[0031] The processor is configured to execute the instructions to implement the color restoration method described in any of the preceding claims.

[0032] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the color restoration method described in any of the preceding claims.

[0033] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0034] In this embodiment of the disclosure, the color temperature data of the first image region in an image with a color temperature distribution similar to or the same as that of the second image region of the image to be processed is used to perform color restoration processing on each sub-region of the second image region. After the above color restoration processing, the colors of the photos taken by the camera module under various lighting conditions are almost exactly the same as the colors of the scene seen by the human eye, accurately restoring the original color of the subject.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0037] Figure 1 This is a cross-sectional structural schematic diagram of a spectrometer module according to an embodiment of the present disclosure;

[0038] Figure 2a This is a schematic cross-sectional view of an upper and lower flat glass according to an embodiment of the present disclosure.

[0039] Figure 2b This is a schematic cross-sectional view of another upper and lower flat glass structure according to an embodiment of the present disclosure.

[0040] Figure 3a This is a schematic diagram of a color reproduction device according to an embodiment of the present disclosure;

[0041] Figure 3b This is a partial structural schematic diagram of a color reproduction device according to an embodiment of the present disclosure;

[0042] Figure 3c This is a partial structural schematic diagram of another color reproduction device according to an embodiment of the present disclosure;

[0043] Figure 3d yes Figure 3b A schematic diagram of the color reproduction device along the cross section of AA'.

[0044] Figure 3e This is a partial structural schematic diagram of a color reproduction system according to an embodiment of the present disclosure;

[0045] Figure 3f This is a partial structural schematic diagram of another color reproduction system according to an embodiment of the present disclosure;

[0046] Figure 4a This is a schematic diagram of a color temperature distribution image and an image to be processed, according to an embodiment of the present disclosure;

[0047] Figure 4b This is a schematic diagram of another color temperature distribution image and an image to be processed, according to an embodiment of the present disclosure;

[0048] Figure 5 This is a flowchart illustrating a color restoration method according to an embodiment of the present disclosure;

[0049] Figure 6 This is a schematic diagram of an image processor module according to an embodiment of the present disclosure;

[0050] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0052] In photography, light sources are always defined by their color temperature. The unit of color temperature is Kelvin. Like Fahrenheit and Celsius, Kelvin is a unit of temperature measurement. For cameras, color temperature is a matter of white balance. A camera's image sensor cannot detect and correct color changes under different lighting conditions, and the image output by the image sensor may show color distortion, which is a major headache for many photography enthusiasts. The color of a subject changes under various lighting conditions. White objects show the most significant changes: under indoor tungsten lighting, white objects appear orange-yellow, and scenes photographed under such lighting conditions will appear yellowish; however, under a clear blue sky, they will have a bluish tint, and scenes photographed under such lighting conditions will appear bluish. To minimize the influence of external light on the color of the subject and to reproduce the original color of the subject under different color temperature conditions, color correction, or white balance processing, is needed on the images captured by the camera to achieve correct color balance.

[0053] Currently, cameras generally have a spectral sensor located below the lens. The spectral sensor can sense multiple channels, such as the R, G, and B sensing channels, to obtain the overall average R, G, and B values ​​of the ambient light, which is the overall color information within the camera's field of view. However, it cannot determine the local color information of each small area within the camera's field of view and perform overall color restoration of the image captured by the camera based on this overall color information. Since the overall color information cannot truly reflect the color information of the local area, it cannot accurately restore the original color of the subject.

[0054] Based on the above problems, in this embodiment of the disclosure, the color temperature data of each color sub-region in the image to be processed is determined by using a color temperature distribution image that has the same or similar color temperature data as the image to be processed, and color restoration processing is performed on it to improve the accuracy of color restoration.

[0055] Figure 1 This is a cross-sectional structural schematic diagram of a spectrometer module according to an embodiment of the present disclosure. The spectrometer module can acquire color temperature distribution images within its imaging area. See also... Figure 1 The spectrometer module includes a spectrometer lens 11, a filter assembly 12, and an image sensor 13. The spectrometer lens 11 is an imaging lens that can transmit the entire visible light spectrum. The filter assembly 12 can filter the ambient light passing through the spectrometer lens 11 to allow light of different wavelengths to pass through sequentially. The image sensor can convert light of different wavelengths into a color temperature distribution image.

[0056] Optionally, the spectrometer lens 11 includes a first lens group 111 and a second lens group 112. The filter assembly 12 includes an upper plate glass 121, a lower plate glass 122, and a drive assembly 123. The upper plate glass 121 and the lower plate glass 122 are located between the first lens group 111 and the second lens group 112. The parallel upper plate glass 111 and the lower plate glass 112 form a planar parallel cavity. The inner surfaces of the upper plate glass 121 and the lower plate glass 122 have high reflectivity. Ambient light passing through the spectrometer lens is reflected multiple times in the planar parallel cavity, achieving a multi-beam interference effect. The transmittance of the filter assembly is related to the height of the planar parallel cavity, that is, to the vertical distance between the upper plate glass and the lower plate glass. The driving assembly 123 is connected to at least one of the upper flat glass 111 and the lower flat glass 112. By driving the movement of the flat glass connected to it, the driving assembly can adjust the height of the planar parallel cavity to transmit light of a wavelength corresponding to the height, and transmit only one wavelength at a time. That is, the wavelength of the light transmitted through the planar parallel cavity corresponds to the height. The image sensor can convert light of different wavelengths into a color temperature distribution image.

[0057] The drive component 123 may, but is not limited to, a micro-displacement motor, such as an electromagnetic motor, a shape memory metal motor, a piezoelectric motor, or an ultrasonic motor.

[0058] The correspondence between the height of the planar parallel cavity and the wavelength of light transmitted by the filter component can be determined, but is not limited to, according to the Fabry-Perot cavity multi-beam interference principle. Based on this correspondence, the control strategy of the motor (taking the drive component as an example) is determined so that the motor rotates according to the control strategy, thereby driving at least one of the upper and lower plate glass to move along the height direction of the planar parallel cavity to change the height of the planar parallel cavity.

[0059] Taking the movement of a flat glass plate driven by a motor to change the height of a planar parallel cavity as an example, see [link / reference]. Figure 2a Assume that initially, the upper flat glass is at position A and the lower flat glass is at position B1. When the camera module starts, the motor starts, see [reference needed]. Figure 2a Currently, the flat glass is at position B1, the height of the planar parallel cavity 124 is L, and the filter component transmits light of wavelength λ1; see [link to relevant documentation]. Figure 2b Driven by the motor, the flat glass moves from position B1 to position B2, at which point the filter element transmits light with wavelength λ2; driven by the motor, the flat glass moves from position B2 to position B3, at which point the filter element transmits light with wavelength λ3; the motor then drives the flat glass downwards sequentially until it reaches... Figure 2b B in n Position, at this time the filtering component transmits wavelength λ nThe light. Each wavelength of light reaches the monochrome image sensor, which records a color distribution. The combined wavelengths are λ1, λ2, λ3, ..., λ... n The color distribution is obtained, that is, the color temperature distribution image within the entire field of view of the spectrometer module.

[0060] It should be noted that the height of the planar parallel cavity, that is, the distance between B1 and B2, the distance between B1 and B3, and the distance between B1 and B... n The distance between them is related to the wavelength of the light used to obtain the color temperature distribution image; the distances or the position information of the two flat glass plates can be defined in the control strategy of the drive component; the wavelength of each light can also be defined in the control strategy to determine the height of the planar parallel cavity corresponding to the wavelength of each light when a color distribution image needs to be generated, and then control the drive component to adjust the height of the planar parallel cavity.

[0061] Regarding the various wavelengths of light required to generate the color temperature distribution image, a user interface for selecting wavelengths can be provided for users to choose the required wavelengths according to their actual needs; alternatively, the wavelengths can be defined by factory settings.

[0062] Alternatively, since black and white image sensors have better light transmittance than color image sensors and can receive all visible light rays, the image sensor in the spectrometer module can be a black and white image sensor.

[0063] Figure 3a This is a schematic diagram of a color restoration device according to an embodiment of the present disclosure. The color restoration device may include a spectrometer module and an image signal processor. The spectrometer module is electrically connected to the image signal processor. The image signal processor can acquire a color temperature distribution image collected by the spectrometer module, and perform color restoration processing on each color sub-region at the corresponding position in the second image region of the image to be processed based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image. The similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold.

[0064] Figure 3b This is a partial structural schematic diagram of a color reproduction device according to an embodiment of the present disclosure. Figure 3c This is a partial structural schematic diagram of another color reproduction device according to an embodiment of the present disclosure. See also... Figure 3b and Figure 3cThe color reproduction device may include a camera module, a spectrometer module, and an image signal processor (not shown in the figure). The camera module and the spectrometer module may be connected via, but are not limited to, a reinforcing plate. The image signal processor is electrically connected to both the camera module and the spectrometer module. The resolution of the spectrometer module may be the same as or different from that of the camera module.

[0065] It should be noted that the positional relationship between the camera module and the spectrometer module is not limited to... Figure 3b The line connecting the center of the camera module's lens and the center of the spectrometer module's lens is parallel to the horizontal plane; it could also be... Figure 3c The line connecting the center of the lens of the camera module and the center of the lens of the spectrometer module is parallel to the vertical plane; the line connecting the center of the lens of the camera module and the center of the lens of the spectrometer module can also have an angle of inclination with the horizontal or vertical plane. The end face of the lens of the camera module and the end face of the lens of the spectrometer module can be on the same plane or on different planes.

[0066] However, it's important to note that regardless of the positional relationship between the camera module and the spectrometer module, their fields of view need to partially overlap. This ensures that the color temperature distribution image acquired by the spectrometer module has a first image region that overlaps with the second image region of the image to be processed acquired by the camera module. Furthermore, if the color temperature distribution image and the image to be processed were acquired within the same time period, the similarity between the color temperature data of the first image region and the color temperature data of the second image region must be greater than a similarity threshold. Of course, acquiring the color temperature distribution image and the image to be processed within the same time period does not mean they must be acquired simultaneously. The color temperature distribution image can be acquired first, followed by the image to be processed, or vice versa, as long as the time difference between the two image acquisitions is short.

[0067] The image signal processor (ESP) can acquire color images captured by the camera module, i.e., the images to be processed. These images exhibit color distortion under illumination and require color restoration processing. The ESP can also acquire color temperature distribution images captured by the spectrometer module. The color temperature data of each color temperature sub-region in the first region of this color temperature distribution image has a similarity greater than a similarity threshold with the color temperature data of corresponding color sub-regions in the second image region of the image to be processed. This means that the color temperature data of each color temperature sub-region accurately reflects the color temperature data of its corresponding location. Therefore, the ESP can perform color restoration processing on the corresponding color sub-regions in the second image region based on the color temperature data of each color temperature sub-region in the first image region, thereby restoring the original color of the subject in the image to be processed.

[0068] Figure 3d yes Figure 3bThe schematic diagram of the color reproduction device along AA' is shown below. Figure 3d The camera module may include a camera lens 21, a filter 22, a color image sensor 23, and a motor 24, etc. The camera module may also include Figure 3b The FPC (Flexible Printed Circuit) and connector shown can convert the electrical signals representing color images collected by the color image sensor into image data, and send them to peripheral devices through the connector. The peripheral devices may be, for example, image signal processors. The specific structure of the camera module will not be described here.

[0069] Similar to camera modules, Figure 1 The spectrometer module shown may also include an FPC and a connector. The FPC can convert the electrical signals representing the color temperature distribution image acquired by the black and white image sensor of the spectrometer module into image data and send them to peripheral devices through the connector. The peripheral devices may be, for example, an image signal processor.

[0070] The first image region of the color temperature distribution image is the region that overlaps with the second image region of the image to be processed. The overlapping region corresponds to the field of view, which is the overlapping region of the field of view of the camera module and the spectrometer module. Therefore, the color temperature data of the first image region can truly reflect the color temperature data of the second image region. By using the first image region to perform color restoration processing on the second image region, the original color of the photographed target can be accurately restored.

[0071] Understandably, the larger the area occupied by the second image region in the image to be processed, the larger the area that can reproduce the color of the image to be processed. This requires increasing the overlap area of ​​the field of view of the camera module and the spectrometer module. When the field of view of the camera module and the spectrometer module is fixed, the closer the distance between them, the larger the overlap area of ​​the field of view. It is necessary to ensure that the distance between the camera module and the spectrometer module is less than the distance threshold.

[0072] See Figure 3e When the field of view θ1 of the spectrometer module covers the field of view θ2 of the camera module, see [reference needed]. Figure 4a The overlapping area between the color temperature distribution image and the image to be processed (the area filled with diagonal lines in the figure) represents the entire area of ​​the image to be processed. That is, the size of the second image area is equal to the size of the image to be processed. At this time, the color temperature data of the first image area can reflect the color temperature data of the entire image to be processed. Each small square in the figure represents a pixel.

[0073] See Figure 3f When the field of view θ1 of the spectrometer module partially coincides with the field of view θ2 of the camera module, see [reference needed]. Figure 4bThe overlapping area between the color temperature distribution image and the image to be processed (the area filled with diagonal lines in the figure) is a part of the image to be processed. That is, the size of the second image area is smaller than the size of the image to be processed. At this time, the color temperature data of the first image area can only be a part of the color temperature data of the image to be processed.

[0074] The specific implementation process of color restoration of the image to be processed is explained below.

[0075] Figure 5 This is a flowchart illustrating a color restoration method according to embodiments of the present disclosure, applied to an image signal processor. See also... Figure 5 The method may include the following steps:

[0076] Step 501: Obtain the color temperature distribution image.

[0077] The color temperature distribution image is used to restore the color of the image to be processed. It is required that the similarity between the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image and the color temperature data of each color sub-region in the corresponding position in the second image region of the image to be processed is greater than the similarity threshold.

[0078] In one embodiment, the color temperature distribution image can be obtained through... Figures 3b-3f The color reproduction device shown can trigger the spectrometer module to acquire a color temperature distribution image during the process of the camera module acquiring the image to be processed; or, after the camera module has completed acquiring the image to be processed, trigger the spectrometer module to acquire a color temperature distribution image; or, when the camera module is started but has not acquired the image to be processed, trigger the spectrometer module to acquire a color temperature distribution image, and after the color temperature distribution image acquisition is completed, the camera module acquires the image to be processed.

[0079] See the positional relationship between the camera module and the spectrometer module. Figure 3e or Figure 3f As shown, there is an overlapping area between the field of view of the camera module and the spectrometer module. Correspondingly, there is an overlapping area between the color temperature distribution image acquired by the spectrometer module and the image to be processed acquired by the camera module. That is, the first image area of ​​the color temperature distribution image overlaps with the second image area of ​​the image to be processed.

[0080] After the spectrometer module completes the acquisition of the color temperature distribution image, the signal processor can store the color temperature distribution image, as well as the location data and environmental data (including temperature, light intensity, etc.) at the time of image capture, locally or on the network.

[0081] In another embodiment, the color temperature distribution image can also be a historically acquired image. When the camera module starts up, the image signal processor can search locally or on the network for a color temperature distribution image that matches the current shooting location and environment (including temperature, light intensity, etc.) and contains the same subject. During color restoration, an image recognition algorithm can be used to identify the area in the color temperature distribution image containing the same subject as the image to be processed as the first image area, and the area in the image to be processed containing the same subject as the color temperature distribution image as the second image area.

[0082] Step 502: Based on the color temperature data of each color temperature sub-region in the first image region, perform color restoration processing on each color sub-region at the corresponding position in the second image region.

[0083] See Figure 4a If the overlapping area between the color temperature distribution image and the image to be processed is the entire area of ​​the image to be processed, then the entire image to be processed can be color restored based on the color temperature data of each color temperature sub-region in the first image region.

[0084] See Figure 4b If the color temperature distribution image and the image to be processed only have a partial overlap, that is, the size of the second image region is smaller than the size of the image to be processed, then the color restoration processing of the second image region in the image to be processed can be performed based on the color temperature data of each color temperature sub-region in the first image region.

[0085] Since the first image region and the second image region overlap, the color temperature data of each color temperature sub-region in the first image region can accurately reflect the color temperature data of each color sub-region at the corresponding position in the second image region, thereby greatly improving the accuracy of color restoration.

[0086] Of course, for other regions in the image to be processed besides the second image region, color restoration processing can also be performed on these other regions based on the color temperature distribution image. Figure 4b Taking the color temperature distribution image and the image to be processed as an example, the regions where the pixels in columns r1 and r2 of the image to be processed are located do not overlap with the color temperature distribution image. However, the regions where the pixels in columns r1 and r2 are located are close to the regions where the pixels in column R1 of the color temperature distribution image are located. Therefore, color restoration processing can be performed on the color temperature data of the regions where the pixels in columns r1 and r2 are located based on the color temperature data of the regions where the pixels in column R1 are located.

[0087] It should be noted that the division of sub-regions can be set according to actual needs. For example, the 4 pixels in the first image region can be divided into a color temperature sub-region, and the 8 pixels in the second image region can be divided into a color sub-region. Then, the color of the 8-pixel color sub-region can be restored based on the color temperature sub-region of the 4 pixels. Alternatively, each pixel in the first image region can be divided into a color temperature sub-region, and each pixel in the second image region can be divided into a color sub-region. Then, the color of the 1-pixel color sub-region can be restored based on the color temperature sub-region of the 1-pixel.

[0088] Before performing color restoration, it is necessary to determine the positional correspondence between each color temperature sub-region in the first image region and each color sub-region in the second image region. This positional correspondence can be determined based on the coordinate calibration results of the spectrometer module and the camera module.

[0089] During coordinate calibration, the camera module and the spectrometer module can simultaneously photograph the calibration board. The calibration board contains multiple calibration marks, which are arranged according to certain rules. Based on the position information of each calibration mark in the image acquired by the camera module, the position information of each calibration mark in the image acquired by the spectrometer module, and the spatial position information between the camera module and the spectrometer module, coordinate calibration is performed on the camera module and the spectrometer module to determine the positional correspondence between each color temperature sub-region in the first image region and each color sub-region in the second image region.

[0090] The following example illustrates the implementation process of color restoration, using the color temperature sub-region as an example to demonstrate how each pixel in the color sub-region performs color restoration on the corresponding pixel in the color sub-region:

[0091] S1. Color temperature estimation, which estimates the average color difference (Y, C) of each pixel in the first image region in the YCbCr color space. b C r The following formula can be used, but is not limited to, to convert the RGB color space of each pixel to the YCbCr color space;

[0092]

[0093] Where A is the color coordinate transformation matrix, used to convert the RGB color space of the color temperature distribution image to the YCbCr color space.

[0094] S2. Gain calculation, which calculates the gain (correction factor) u and v of the R and B channels.

[0095] The calculation process may, but is not limited to, using: if |C b |>|C r |, and C b If |C > 0, then u = u - λ; if |Cb |>|C r |, and C b If |C ≤ 0, then u = u + λ; if |C b |≤|C r |, and C b If > 0, then v = v - λ; if |C b |≤|C r |, and C b If ≤0, then v=v+λ; where the initial value u=v=1.

[0096] S3. Color temperature correction: Multiply the R and B channels of each pixel at the corresponding position in the second image area by u and v respectively to achieve white balance.

[0097] After step S3, if C b With C r If the value is not within its preset range, return to S2 to accurately obtain the image white balance gain through continuous iteration. b With C r Each preset range can be set according to actual needs; generally, the smaller the better.

[0098] Figure 6 This is a schematic diagram of an image processor module according to an embodiment of the present disclosure, which may include:

[0099] Acquisition module 61 is used to acquire color temperature distribution images;

[0100] The processing module 62 is used to perform color restoration processing on each color sub-region at a corresponding position in the second image region of the image to be processed based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image, wherein the similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold.

[0101] Optionally, during the process of the camera module acquiring the image to be processed, the acquisition module 61 acquires the color temperature distribution image acquired by the spectrometer module; wherein, the first image region of the color temperature distribution image overlaps with the second image region of the image to be processed.

[0102] Optionally, the processing module 62 is specifically used for:

[0103] Based on the color temperature data of each pixel in the color temperature sub-region, color restoration processing is performed on each pixel at the corresponding position in the color sub-region.

[0104] Optionally, the image processor also includes:

[0105] The determination module is used to determine the positional correspondence between each color temperature sub-region and each color sub-region based on the coordinate calibration results of the spectrometer module and the camera module.

[0106] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments of the relevant methods, and will not be elaborated upon here.

[0107] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0108] Embodiments of this disclosure also provide an electronic device, comprising:

[0109] processor;

[0110] Memory used to store processor-executable instructions;

[0111] The processor is configured to implement the color restoration method described in any of the above embodiments.

[0112] Embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the color restoration method described in any of the above embodiments.

[0113] Figure 7 This is a block diagram illustrating an apparatus for color reproduction according to embodiments of the present disclosure. The apparatus may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0114] like Figure 7As shown, device 700 may include one or more of the following components: processing component 702, memory 704, power supply component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 714, and communication component 716. The device also includes an antenna module (e.g., connectable to communication component 716), which includes: a radiator 1, a ground point 2, a feed terminal 3, and a resonant circuit 4. The radiator 1 includes an open end 11, and the ground point 2 is disposed on the radiator 1. The feed terminal 3 is electrically connected to a first connection point 5 on the radiator 1. A first terminal 41 of the resonant circuit is electrically connected to the first connection point 5, and a second terminal 42 of the resonant circuit is grounded. The resonant circuit 4 includes an adjustable unit. The distance from the first connection point 5 to the open end 11 is less than the distance from the first connection point 5 to the ground point 2.

[0115] Processing component 702 typically controls the overall operation of device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0116] Memory 704 is configured to store various types of data to support the operation of device 700. Examples of this data include instructions for any application or method operating on device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0117] Power supply assembly 706 provides power to various components of device 700. Power supply assembly 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 700.

[0118] Multimedia component 708 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities. The front-facing camera and rear-facing camera may be a camera module as shown in any of the above embodiments. Multimedia component 708 may also include a color reproduction device as shown in any of the above embodiments.

[0119] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0120] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0121] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of device 700. For example, sensor assembly 714 may detect the on / off state of device 700, the relative positioning of components such as the display and keypad of device 700, changes in the position of device 700 or a component of device 700, the presence or absence of user contact with device 700, the orientation or acceleration / deceleration of device 700, and temperature changes of device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0122] Communication component 716 is configured to facilitate wired or wireless communication between device 700 and other devices. Device 700 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0123] In an exemplary embodiment, device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described in any of the above embodiments.

[0124] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of the device 700 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0125] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0126] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A color restoration device, characterized by, include: Spectrometer module, used to acquire color temperature distribution images; An image signal processor is configured to perform color restoration processing on corresponding color sub-regions in a second image region of an image to be processed based on the color temperature data of each color temperature sub-region in a first image region of the color temperature distribution image. The first image region is a region identified from the color temperature distribution image that contains the same subject as the image to be processed. The second image region is a region identified from the image to be processed that contains the same subject as the color temperature distribution image. The first image region overlaps with the second image region, and the similarity between the color temperature data of each color temperature sub-region and the color temperature data of the corresponding color sub-regions is greater than a similarity threshold.

2. The color reproduction apparatus of claim 1, wherein The spectrometer module includes: Spectrometer lens; A filtering component is used to filter ambient light passing through the lens of the spectrometer so that light of different wavelengths can pass through in sequence. An image sensor is used to convert light of different wavelengths into the color temperature distribution image.

3. The color reproduction apparatus of claim 2, wherein The filtering component includes: Top flat glass; The lower flat glass is arranged parallel to the upper flat glass, and the lower flat glass and the upper flat glass form a planar parallel cavity; A driving assembly, connected to at least one of the upper and lower flat glass plates, adjusts the height of the planar parallel cavity by driving at least one of the upper and lower flat glass plates to move, wherein the wavelength of light transmitted from the planar parallel cavity corresponds to the height.

4. The color reproduction apparatus of claim 2, wherein The image sensor is a black and white image sensor.

5. The color reproduction apparatus of claim 1, wherein Also includes: A camera module, wherein the distance between the camera module and the spectrometer module is less than a distance threshold, so that the second image region of the image to be processed acquired by the camera module overlaps with the first image region of the color temperature distribution image.

6. A color reproduction method characterized by, include: Obtain the color temperature distribution image; Based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image, color restoration processing is performed on each color sub-region at the corresponding position in the second image region of the image to be processed. Here, the first image region is the region identified from the color temperature distribution image that contains the same subject as the image to be processed, and the second image region is the region identified from the image to be processed that contains the same subject as the color temperature distribution image. The first image region and the second image region overlap, and the similarity between the color temperature data of each color temperature sub-region and the color temperature data of the corresponding color sub-region is greater than a similarity threshold.

7. The color reproduction method of claim 6, wherein, The acquisition of the color temperature distribution image includes: During the process of the camera module acquiring the image to be processed, the color temperature distribution image acquired by the spectrometer module is obtained; wherein, the first image region of the color temperature distribution image overlaps with the second image region of the image to be processed.

8. The color reproduction method according to claim 7, wherein Before performing color restoration processing on each color sub-region at the corresponding position in the second image region of the image to be processed, the method further includes: Based on the coordinate calibration results of the spectrometer module and the camera module, the positional correspondence between each color temperature sub-region and each color sub-region is determined.

9. The color reproduction method of claim 6, wherein, The step of performing color restoration processing on corresponding color sub-regions in the second image region of the image to be processed based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image includes: Based on the color temperature data of each pixel in the color temperature sub-region, color restoration processing is performed on each pixel at the corresponding position in the color sub-region.

10. An image processor, characterized by, include: The acquisition module is used to acquire color temperature distribution images; The processing module is used to perform color restoration processing on each color sub-region at a corresponding position in the second image region of the image to be processed based on the color temperature data of each color temperature sub-region in the first image region of the color temperature distribution image. The first image region is the region identified from the color temperature distribution image that contains the same subject as the image to be processed, and the second image region is the region identified from the image to be processed that contains the same subject as the color temperature distribution image. The first image region and the second image region overlap, and the similarity between the color temperature data of each color temperature sub-region and the color temperature data of each color sub-region at the corresponding position is greater than a similarity threshold.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the color restoration method according to any one of claims 6 to 9.

12. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the color restoration method according to any one of claims 6 to 9.

Images