Image processing method, device and storage medium

Abstract

This disclosure relates to an image processing method, apparatus, and storage medium. The method includes: acquiring first spectral response data from a first camera and second spectral response data from a second camera; acquiring irradiance data of a light signal to be measured; determining a first pixel response output by the first camera and a second pixel response output by the second camera based on the irradiance data, the first spectral response data, and the second spectral response data of the light signal to be measured; and determining a pixel deviation between the images output by the first camera and the second camera after receiving the light signal to be measured in different spectral frequency bands based on the first pixel response and the second pixel response, wherein the pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

Description

Technical Field

[0001] This disclosure relates to the field of image technology, and in particular to an image processing method, apparatus and storage medium. Background Technology

[0002] Because the shooting distance and shooting range of a single camera are limited, in order to meet the needs of application scenarios with multiple perspectives, multiple scenes and multiple shooting distances, and to enhance the shooting capabilities of terminal devices, terminal devices are equipped with multiple cameras to perform zoom shooting and meet the shooting needs of multiple application scenarios.

[0003] However, because different cameras have different sensitivities to different colors of light signals, when the camera used to display the preview image on the terminal device is switched, the preview image may exhibit color jumps and flickering, making it impossible to achieve a smooth and natural transition between different cameras. Summary of the Invention

[0004] This disclosure provides an image processing method, apparatus, and storage medium.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0006] Acquire the first spectral response data of the first camera and the second spectral response data of the second camera;

[0007] Acquire irradiance data of the light signal to be measured;

[0008] Based on the irradiance data of the light signal to be measured, the first spectral response data, and the second spectral response data, the first pixel response output by the first camera and the second pixel response output by the second camera are determined. The first pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the first camera; the second pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the second camera.

[0009] Based on the first pixel response and the second pixel response, the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands is determined. The pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

[0010] Optionally, determining the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands, based on the first pixel response and the second pixel response, includes:

[0011] The first deviation is obtained based on the difference between the first pixel response and the second pixel response;

[0012] The pixel deviation between the first camera and the second camera is determined based on the first deviation.

[0013] Optionally, determining the pixel deviation between the first camera and the second camera based on the first deviation includes:

[0014] The first deviation is defined as the pixel deviation between the first camera and the second camera;

[0015] or,

[0016] The pixel deviation between the first camera and the second camera is determined by multiplying the brightness response gradient of the second camera at the current brightness, which is determined based on the second pixel response and the brightness response of the second camera, with the first deviation.

[0017] Optionally, the first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels in the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera;

[0018] The step of determining the first pixel response output by the first camera and the second pixel response output by the second camera based on the irradiance data of the light signal under test, the first spectral response data, and the second spectral response data includes:

[0019] Determine the grayscale distribution curves corresponding to the first signals output by the first camera and the second camera after receiving the radiation of the light signal to be measured; wherein, the grayscale distribution curves are used to describe the intensity of the light signal to be measured in different spectral frequency bands received by the cameras;

[0020] The product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera is integrated to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera.

[0021] The product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera is integrated to determine the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera.

[0022] Optionally, determining the grayscale distribution curve corresponding to the first signal output by the first camera and the second camera after receiving the radiation of the light signal to be measured includes:

[0023] Based on the irradiance data of the light signal to be measured, the first light energy of the incident light signals of the first camera and the second camera is determined;

[0024] Based on the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured;

[0025] The number of photons of different frequencies received by the first camera and the second camera is determined based on the ratio of the first light energy to the plurality of second light energies.

[0026] Based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

[0027] Optionally, the first pixel response includes: a plurality of first pixel responses corresponding to pixels of a plurality of different color channels in the first camera; the second pixel response includes: a plurality of second pixel responses corresponding to pixels of a plurality of different color channels in the second camera;

[0028] The step of obtaining the first deviation amount based on the difference between the first pixel response and the second pixel response includes:

[0029] The first deviation amount corresponding to the pixels of the multiple different color channels is determined based on the difference between the multiple first pixel responses corresponding to the pixels of the multiple different color channels in the first camera and the multiple second pixel responses corresponding to the pixels of the multiple different color channels in the second camera.

[0030] Optionally, acquiring the irradiance data of the light signal to be measured includes:

[0031] The light signal to be measured is used to obtain the irradiance of multiple channels with different wavelengths;

[0032] The irradiance data of the light signal under test is estimated based on the irradiance of the multiple different wavelength channels.

[0033] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0034] The acquisition module is used to acquire the first spectral response data of the first camera and the second spectral response data of the second camera; and to acquire the irradiance data of the light signal to be measured.

[0035] The first determining module is used to determine the first pixel response output by the first camera and the second pixel response output by the second camera based on the irradiance data, the first spectral response data and the second spectral response data of the light signal to be measured, wherein the first pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the first camera; and the second pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the second camera.

[0036] The second determining module is used to determine, based on the first pixel response and the second pixel response, the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands. The pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

[0037] Optionally, the second determining module is configured to:

[0038] The first deviation is obtained based on the difference between the first pixel response and the second pixel response;

[0039] The pixel deviation between the first camera and the second camera is determined based on the first deviation.

[0040] Optionally, the second determining module is configured to:

[0041] The first deviation is defined as the pixel deviation between the first camera and the second camera;

[0042] or,

[0043] The pixel deviation between the first camera and the second camera is determined by multiplying the brightness response gradient of the second camera at the current brightness, which is determined based on the second pixel response and the brightness response of the second camera, with the first deviation.

[0044] Optionally, the first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels in the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera;

[0045] The first determining module is used for:

[0046] Determine the grayscale distribution curves corresponding to the first signals output by the first camera and the second camera after receiving the radiation of the light signal to be measured; wherein, the grayscale distribution curves are used to describe the intensity of the light signal to be measured in different spectral frequency bands received by the cameras;

[0047] The product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera is integrated to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera.

[0048] The product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera is integrated to determine the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera.

[0049] Optionally, the first determining module is further configured to:

[0050] Based on the irradiance data of the light signal to be measured, the first light energy of the incident light signals of the first camera and the second camera is determined;

[0051] Based on the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured;

[0052] The number of photons of different frequencies received by the first camera and the second camera is determined based on the ratio of the first light energy to the plurality of second light energies.

[0053] Based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

[0054] Optionally, the first pixel response includes: a plurality of first pixel responses corresponding to pixels of a plurality of different color channels in the first camera; the second pixel response includes: a plurality of second pixel responses corresponding to pixels of a plurality of different color channels in the second camera;

[0055] The second determining module is used for:

[0056] The first deviation amount corresponding to the pixels of the multiple different color channels is determined based on the difference between the multiple first pixel responses corresponding to the pixels of the multiple different color channels in the first camera and the multiple second pixel responses corresponding to the pixels of the multiple different color channels in the second camera.

[0057] Optionally, the acquisition module is used to:

[0058] The light signal to be measured is used to obtain the irradiance of multiple channels with different wavelengths;

[0059] The irradiance data of the light signal under test is estimated based on the irradiance of the multiple different wavelength channels.

[0060] According to a third aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0061] processor;

[0062] Memory used to store executable instructions;

[0063] The processor is configured to, when executing executable instructions stored in the memory, implement the steps of the image processing method according to the first aspect of the present disclosure.

[0064] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an image processing apparatus, the image processing apparatus is enabled to perform steps in the image processing method as described in the first aspect of the present disclosure.

[0065] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0066] This embodiment of the disclosure determines the first pixel response and the second pixel response of the first camera and the second camera respectively by acquiring the first spectral response data of the first camera, the second spectral response data of the second camera, and the irradiance data of the light signal to be measured. Since the first pixel response and the second pixel response can reflect the intensity of the light signal to be measured received by pixels of different color channels in the first camera and the second camera, the pixel difference between the output images of the first camera and the second camera can be determined based on the intensity of the light signal to be measured received by pixels of different color channels corresponding to the first pixel response and the second pixel response.

[0067] By using pixel difference to reflect the color difference of each pixel in the output images of the first and second cameras, the output image of the second camera can be corrected using this pixel difference, reducing the color difference between the pixels in the output images of the first and second cameras, reducing color jumps and flickering in the preview screen when switching cameras, and achieving a smooth and natural switching between different cameras.

[0068] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0070] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 1 .

[0071] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 2 .

[0072] Figure 3 This is a schematic diagram illustrating the spectral response curve of a camera according to an exemplary embodiment.

[0073] Figure 4 This is a schematic diagram illustrating the irradiance curve of an optical signal according to an exemplary embodiment.

[0074] Figure 5 This is a schematic diagram illustrating the brightness response curve of a camera according to an exemplary embodiment.

[0075] Figure 6 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 3 .

[0076] Figure 7 This is a schematic diagram illustrating a first image captured by a main camera according to an exemplary embodiment.

[0077] Figure 8 This is a schematic diagram illustrating a second image captured by a secondary camera according to an exemplary embodiment.

[0078] Figure 9 This is a schematic diagram illustrating the spectral response curves of the main and secondary cameras according to an exemplary embodiment.

[0079] Figure 10 This is a schematic diagram of the spectral response curve of an ambient light signal according to an exemplary embodiment.

[0080] Figure 11 This is a schematic diagram of a corrected third image according to an exemplary embodiment.

[0081] Figure 12 This is a schematic diagram of the structure of an image processing apparatus according to an exemplary embodiment.

[0082] Figure 13 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. Detailed Implementation

[0083] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses consistent with some aspects of this disclosure as detailed in the appended claims.

[0084] To meet the shooting needs of various application scenarios, terminal devices are usually equipped with multiple cameras; however, different cameras have different spectral response curves, that is, different cameras have different sensitivities to light signals of different wavelengths; this causes different cameras to output different image pixel values ​​under the same color light signal radiation, and thus the white balance parameters calculated by different cameras are also different.

[0085] When the camera used by the terminal device to display the preview screen changes in automatic white balance mode, the white balance parameters corresponding to different cameras will change abruptly, causing the preview screen to exhibit color jumps and flickering, which reduces the user experience.

[0086] The following two methods are commonly used in related technologies to solve the above problems:

[0087] First, there is the pixel mapping model based on deep learning. However, this method requires a large amount of data collection and training for each camera. On the one hand, due to the many differences between cameras, such as the field of view, it is difficult to collect training data for the cameras. On the other hand, the resulting pixel mapping model is not universal. Every time the camera is changed, data needs to be collected again and retrained.

[0088] Second, there is white balance based on semantic segmentation patterns, but this method relies too much on the model, resulting in a large difference between the colors in the captured image and the colors seen by the naked eye.

[0089] Based on this, the present disclosure provides an image processing method. Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 1 ,like Figure 1 As shown, the method includes:

[0090] Step S101: Obtain the first spectral response data of the first camera and the second spectral response data of the second camera;

[0091] Step S102: Obtain the irradiance data of the light signal to be measured;

[0092] Step S103: Based on the irradiance data, the first spectral response data, and the second spectral response data of the light signal to be measured, determine the first pixel response output by the first camera and the second pixel response output by the second camera, wherein the first pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the first camera; the second pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the second camera.

[0093] Step S104: Based on the first pixel response and the second pixel response, determine the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands. The pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

[0094] In this embodiment, the image processing method can be executed by an image processing device, which can be configured in a terminal device, and the terminal device has at least two cameras or two or more cameras connected to it simultaneously. Here, the terminal device can be a smartphone, tablet computer, or wearable electronic device, etc.

[0095] In step S101, the first spectral response data of the first camera includes at least the first spectral response curve of the first camera, and the second spectral response data of the second camera includes at least the second spectral response curve of the second camera.

[0096] The first and second cameras can be detected using a detection device to obtain the first spectral response curve corresponding to the first camera and the second spectral response curve corresponding to the second camera.

[0097] Here, the detection device can be a monochromator; for example, in order to improve the accuracy of the measurement, the monochromator can be used to perform multiple measurements on the first camera and the second camera to obtain multiple sets of measurement data; the average of the multiple sets of measurement data is determined as the first spectral response curve of the first camera and the second spectral response curve of the second camera.

[0098] It should be noted that, in this embodiment, the spectral response curve refers to the relationship between the photocurrent generated by the image sensor and the wavelength when the intensity of the incident light remains constant; the spectral response curve can be used to describe the sensitivity of the image sensor in the camera to light signals of different wavelengths.

[0099] The first camera and / or the second camera can be a wide-angle camera, a telephoto camera, or a main camera; for example, when the first camera is the main camera, the second camera can be a telephoto camera.

[0100] It should be noted that a camera includes a lens and an image sensor; light is focused by the lens and projected onto the image sensor, where each photosensitive element undergoes a photoelectric effect, outputting an electrical signal. Because the parameters of the image sensors differ between different cameras, the spectral response curves of different cameras will also differ.

[0101] In step S102, the irradiance data includes at least an irradiance curve; the light signal to be measured can be the ambient light signal of the environment where the terminal device is located, and the irradiance curve of the ambient light signal of the environment where the terminal device is located can be determined by scene analysis of the environment where the terminal device is located.

[0102] Here, the irradiance of the light signal to be measured refers to the radiant energy received per unit area per unit time on the surface of the object on which the light signal to be measured is radiated.

[0103] For example, based on the pre-stored CIE standard light source spectral distribution, the ambient light signal of the environment where the terminal device is located can be analyzed to obtain the spectral power distribution (i.e., irradiance curve) of the ambient light signal.

[0104] It should be noted that the CIE standard light source spectral distribution is a spectral power distribution of multiple light sources defined by the International Commission on Illumination to describe the color of non-self-emitting sources and to provide colorimetric analysis parameters.

[0105] In step S103, the first pixel response can be used to describe the intensity of the light signal to be measured received by pixels of different color channels in the first camera; the second pixel response can be used to describe the intensity of the light signal to be measured received by pixels of different color channels in the second camera.

[0106] The light energy of the light signals of different wavelengths received by the first camera and the second camera can be determined based on the irradiance data of the light signal to be measured.

[0107] Since pixels in different color channels of the image sensor can receive different wavelengths of the light signals to be measured, the first pixel response of the first camera can be determined based on the first spectral response curve of the first camera and the light energy of the light signals to be measured at different wavelengths received by the first camera.

[0108] The second pixel response of the second camera is determined based on the second spectral response curve of the second camera and the light energy of the light signals of different wavelengths received by the second camera.

[0109] It should be noted that in order for an image sensor to be able to sense the intensity of light signals of different colors (wavelengths), a mosaic filter containing only red, green, and blue can be covered on the surface of the pixel of the image sensor. The filter removes light signals of other wavelengths (i.e., colors different from the filter) from the light signal to be measured incident on the pixel, so that a single pixel only receives light signals of a certain color and senses the intensity of that color light signal.

[0110] Based on the light energy of the light signals of different wavelengths received by the first and second cameras, determine the electrical signals output by the pixels of different color channels in the first and second cameras when they receive the light signals of different wavelengths and the photoelectric effect occurs; based on the electrical signals output by the pixels of different color channels in the first and second cameras, determine the intensity of the light signals of different colors received by the first and second cameras.

[0111] In step S104, after determining the first pixel response of the first camera and the second pixel response of the second camera, the first pixel value corresponding to the first pixel response and the second pixel value corresponding to the second pixel response can be determined based on the brightness response of the first camera and the second camera; the pixel deviation amount is determined based on the difference between the first pixel value and the second pixel value.

[0112] Here, the brightness response refers to the correspondence between the intensity of the light signal to be measured sensed by the image sensor inside the camera and the pixel value of the image output by the camera.

[0113] The first pixel value corresponding to the intensity of the light signal to be measured received by pixels of different color channels in the first camera can be determined based on the first pixel response and the brightness response of the first camera; the second pixel value corresponding to the intensity of the light signal to be measured received by pixels of different color channels in the second camera can be determined based on the second pixel response and the brightness response of the second camera.

[0114] To ensure a smooth and natural switching between the two cameras, the pixel deviation between the output images of the first and second cameras can be determined. The output image of the second camera can then be corrected based on this pixel deviation, ensuring that the corrected image maintains color consistency with the output image of the first camera. This reduces color jumps and flickering between the output images of the first and second cameras during camera switching, allowing users to switch between cameras seamlessly.

[0115] Optionally, the step S104 of determining the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands, based on the first pixel response and the second pixel response, includes:

[0116] The first deviation is obtained based on the difference between the first pixel response and the second pixel response;

[0117] The pixel deviation between the first camera and the second camera is determined based on the first deviation.

[0118] In this embodiment of the disclosure, the first deviation can be used to describe the intensity difference of the light signal to be measured perceived by pixels of different color channels in the first and second cameras.

[0119] For the first camera and the second camera, the light energy of the light signal to be measured incident on the first camera and the second camera is the same. However, because the first spectral response curve of the first camera and the second spectral response curve of the second camera are different, the first camera and the second camera have different sensitivities to light signals to be measured of different wavelengths. The intensity of light signals to be measured of different wavelengths that the pixels of different color channels in the first camera and the second camera can perceive is different. As a result, the first camera and the second camera receive the same light signal to be measured, but there are differences in the pixel values ​​of the output images.

[0120] The first deviation can be determined based on the difference between the first pixel response of the first camera and the second pixel response of the second camera; and the pixel deviation corresponding to the first deviation can be determined by using the first deviation between the first camera and the second camera, as well as the brightness response of the first camera and the second camera.

[0121] Optionally, determining the pixel deviation between the first camera and the second camera based on the first deviation includes:

[0122] The first deviation is defined as the pixel deviation between the first camera and the second camera;

[0123] or,

[0124] The pixel deviation between the first camera and the second camera is determined by multiplying the brightness response gradient of the second camera at the current brightness, which is determined based on the second pixel response and the brightness response of the second camera, with the first deviation.

[0125] In this embodiment of the disclosure, the first offset can be used to describe the intensity difference of the light signal to be measured perceived by pixels of different color channels in the first and second cameras.

[0126] The pixel difference between the images captured by the first camera and the second camera is due to the different sensitivities of the first camera and the second camera to light signals of different wavelengths. This results in a difference in the ability of pixels in different color channels of the first camera and the second camera to perceive the intensity of the light signal under test. For light signals of the same color (same wavelength), the pixel value converted from the electrical signal output by the first camera under test is different from the pixel value of the electrical signal output by the second camera under test.

[0127] Therefore, in order to reduce the pixel deviation between the output images of the first camera and the second camera, the intensity difference of the light signal to be measured perceived by the pixels of different color channels in the first camera and the second camera (i.e., the first offset) can be directly used as the pixel offset corresponding to the pixels of different color channels, and the pixel values ​​corresponding to the different color channels of the RGB format image acquired by the second camera can be corrected.

[0128] It's important to note that RGB is a standard for representing colors in the digital field, also known as a color space. In RGB format images, each pixel value is identified by three components: R, G, and B. A specific color is represented by different combinations of the brightness values ​​of the three primary colors R, G, and B. If each component uses 8 bits, then one pixel uses 3 * 8 = 24 bits to represent it.

[0129] For example, the pixel deviation between the first color channel of the first camera and the second camera can be determined based on the first deviation of the pixels in the first color channel (such as the R channel) of the first camera and the second camera.

[0130] Based on the first deviation amount corresponding to the pixels of the second color channel (such as the G channel) in the first camera and the second camera, determine the pixel deviation amount corresponding to the second color channel between the first camera and the second camera;

[0131] Based on the first deviation amount corresponding to the pixels of the third color channel (such as the B channel) in the first camera and the second camera, determine the pixel deviation amount corresponding to the third color channel between the first camera and the second camera;

[0132] When the terminal device switches from the first camera to the second camera, it can correct the pixel values ​​of each pixel in the image captured by the second camera based on the pixel deviation of the first color channel, the pixel deviation of the second color channel, and the pixel deviation of the third color channel, respectively, and obtain and output the corrected image.

[0133] Considering that the parameters of the image sensors in the first and second cameras may differ, the brightness response of the first and second cameras may also differ. That is, the intensity of the light signal to be measured is the same for the image sensors in the first and second cameras, but the pixel values ​​of the images output by the first and second cameras are different.

[0134] In this case, even if the intensity difference of the light signal to be measured in different color channels of the first and second cameras is compensated based on the first deviation amount, the output image of the second camera after correction may still have pixel differences from the output image of the first camera because there may be differences in brightness response between the cameras.

[0135] In order to reduce the pixel difference between the output images of different cameras caused by the difference in brightness response between cameras, this embodiment of the present disclosure determines the brightness response gradient of the second camera under the current brightness environment based on the brightness response of the second camera; and determines the pixel deviation between the output images of the first camera and the second camera as the product of the brightness response gradient and the first deviation amount.

[0136] It should be noted that, since the brightness response and brightness response gradient of the second camera are the device parameter information of the second camera, the brightness response and brightness response gradient of the second camera can be obtained by acquiring the device parameter information of the second camera.

[0137] The brightness of the current environment where the second camera is located can be determined based on the pixel values ​​of the image currently captured by the second camera; the brightness response gradient corresponding to the brightness can be determined based on the brightness of the current environment where the second camera is located; and the product of the brightness response gradient corresponding to the brightness and the first deviation between the first camera and the second camera can be determined as the pixel deviation between the first camera and the second camera.

[0138] For example, the brightness response gradient of the first color channel (such as the R channel) of the second camera under the current brightness environment can be obtained, and the product of the brightness response gradient of the first color channel and the first deviation of the first color channel can be determined as the pixel deviation of the first color channel between the first camera and the second camera.

[0139] Obtain the brightness response gradient corresponding to the second color channel (such as the G channel) of the second camera under the current brightness environment, and determine the pixel deviation between the first camera and the second camera by multiplying the brightness response gradient corresponding to the second color channel and the first deviation corresponding to the second color channel.

[0140] Obtain the brightness response gradient of the third color channel (e.g., B channel) of the second camera under the current brightness environment, and determine the pixel deviation of the third color channel between the first camera and the second camera by multiplying the brightness response gradient of the third color channel and the first deviation of the third color channel.

[0141] When the terminal device switches from the first camera to the second camera, it can correct the pixel values ​​of each pixel in the image captured by the second camera based on the pixel deviation of the first color channel, the pixel deviation of the second color channel, and the pixel deviation of the third color channel, respectively, and obtain and output the corrected image.

[0142] Optionally, the first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels in the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera;

[0143] The step S103, which determines the first pixel response output by the first camera and the second pixel response output by the second camera based on the irradiance data of the light signal under test, the first spectral response data, and the second spectral response data, includes:

[0144] Determine the grayscale distribution curves corresponding to the first signals output by the first camera and the second camera after receiving the radiation of the light signal to be measured; wherein, the grayscale distribution curves are used to describe the intensity of the light signal to be measured in different spectral frequency bands received by the cameras;

[0145] The product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera is integrated to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera.

[0146] The product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera is integrated to determine the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera.

[0147] In this embodiment of the disclosure, the first signal output by the pixels of different color channels of the first camera and the second camera under the radiation of the light signals to be measured at different wavelengths is related to the light energy of the light signals to be measured at different wavelengths received by the first camera and the second camera.

[0148] Therefore, based on the irradiance data of the light signal to be measured, the light energy of the light signals of different wavelengths received by the first camera and the second camera can be determined, and based on the light energy of the light signals of different wavelengths received by the first camera and the second camera, the signal values ​​of the first signals output by the first camera and the second camera under the irradiation of the light signals of different wavelengths can be determined respectively; and based on the signal values ​​of the first signals output by the first camera and the second camera, the gray value corresponding to the signal value of the first signal can be determined.

[0149] Here, the first signal can be an electrical signal. It should be noted that, because each pixel in the image sensor of the first and second cameras undergoes the photoelectric effect under the radiation of the light signal to be measured, it outputs an electrical signal that reflects the intensity of the light signal to be measured received by the pixel; and by performing analog-to-digital conversion on the electrical signal output by the pixel, grayscale data reflecting the intensity of the light signal to be measured from the image perspective is formed.

[0150] Understandably, the grayscale distribution curve can be used to describe the intensity of the light signal to be measured received by the camera in different spectral bands; if the grayscale value corresponding to a certain pixel is larger, the signal value of the first signal output by that pixel is larger, and the intensity of the light signal to be measured received by that pixel is also larger.

[0151] Because the pixels of multiple different color channels in the camera have different sensitivities to light signals of different wavelengths, a pixel of a single color channel can only receive light signals of the wavelength range corresponding to that color channel in the light signal to be measured, and output a first signal to describe the intensity of the light signal of the wavelength range corresponding to that color channel.

[0152] Therefore, when determining the first pixel response of the first camera, it is necessary to integrate the product of the multiple first sub-response curves corresponding to the pixels of different color channels in the first camera and the gray-scale distribution curves reflecting the intensity of the light signals to be measured at different wavelengths received by the first camera, and obtain the first pixel response used to describe the intensity of the light signals to be measured received by the pixels of each different color channel in the first camera.

[0153] When determining the second pixel response of the second camera, it is necessary to integrate the product of the multiple second sub-response curves corresponding to the pixels of different color channels in the second camera and the grayscale distribution curves reflecting the intensity of the light signals to be measured at different wavelengths received by the second camera, to obtain the second pixel response used to describe the intensity of the light signals to be measured received by the pixels of each different color channel in the second camera.

[0154] Optionally, determining the grayscale distribution curve corresponding to the first signal output by the first camera and the second camera after receiving the radiation of the light signal to be measured includes:

[0155] Based on the irradiance data of the light signal to be measured, the first light energy of the incident light signals of the first camera and the second camera is determined;

[0156] Based on the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured;

[0157] The number of photons of different frequencies received by the first camera and the second camera is determined based on the ratio of the first light energy to the plurality of second light energies.

[0158] Based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

[0159] In this embodiment of the disclosure, the grayscale distribution curves of the first camera and the second camera can be used to describe the signal intensity of the first signal output by the first camera and the second camera under the radiation of the light signal to be measured at different wavelengths; and the signal intensity of the first signal is related to the number of photoelectrons generated by the photoelectric effect when the first camera and the second camera are irradiated by the light signal to be measured at different wavelengths, and is also related to the number of photons contained in the light signal to be measured at different wavelengths.

[0160] Understandably, the more photons contained in light signals of different wavelengths, the more photoelectrons the camera will generate under the radiation of light signals of different wavelengths, and the stronger the signal strength of the first signal output by the camera will be. Therefore, by obtaining the number of photons in the light signals to be measured at different wavelengths, the grayscale distribution curves corresponding to the first camera and the second camera can be determined.

[0161] The first light energy of the light signals of different wavelengths received by the first and second cameras can be determined based on the irradiance curve of the light signal to be measured. Here, the first light energy of the light signals of different wavelengths is the sum of the energy generated by each photon in the light signal of different wavelengths.

[0162] Based on the frequency of each photon in the light signal to be measured, the second light energy generated by the first camera and the second camera when receiving photons of different frequencies is determined respectively; here, the second light energy is the energy generated by a single photon.

[0163] It should be noted that the energy produced by a single photon is related to the photon's frequency; photons of the same frequency produce the same amount of energy.

[0164] The number of photons corresponding to different wavelengths received by the first camera and the second camera can be determined based on the ratio between the first light energy corresponding to the light signal under test at different wavelengths and the second light energy corresponding to the photons at different wavelengths.

[0165] In some embodiments, the grayscale value corresponding to the first signal output by the first camera and the second camera under the radiation of light signals of different wavelengths is positively correlated with the number of photons corresponding to different wavelengths received by the first camera and the second camera.

[0166] Here, the positive correlation coefficient between the gray value of the first signal output by the first camera and the second camera under the radiation of the light signal to be measured at different wavelengths and the number of photons corresponding to different wavelengths received by the first camera and the second camera can be determined by the parameters of the image sensor in the first camera and the second camera (e.g., the size of the photosensitive element in the image sensor, the limiting frequency, etc.).

[0167] Optionally, the first pixel response includes: a plurality of first pixel responses corresponding to pixels of a plurality of different color channels in the first camera; the second pixel response includes: a plurality of second pixel responses corresponding to pixels of a plurality of different color channels in the second camera;

[0168] The step of obtaining the first deviation amount based on the difference between the first pixel response and the second pixel response includes:

[0169] The first deviation amount corresponding to the pixels of the multiple different color channels is determined based on the difference between the multiple first pixel responses corresponding to the pixels of the multiple different color channels in the first camera and the multiple second pixel responses corresponding to the pixels of the multiple different color channels in the second camera.

[0170] In this embodiment of the disclosure, based on the multiple first pixel responses corresponding to pixels of multiple different color channels in the first camera and the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera, the difference between the first pixel response and the second pixel response corresponding to the same color channel can be determined as the first deviation amount corresponding to the pixel of that color channel.

[0171] For example, the first deviation amount corresponding to the first color channel can be determined based on the difference between the first pixel response corresponding to the pixel of the first color channel (such as the R channel) in the first camera and the second pixel response corresponding to the pixel of the first color channel in the second camera.

[0172] The first deviation amount corresponding to the second color channel is determined based on the difference between the first pixel response corresponding to the pixel of the second color channel (such as the G channel) in the first camera and the second pixel response corresponding to the pixel of the second color channel in the second camera.

[0173] The first deviation amount corresponding to the third color channel is determined based on the difference between the first pixel response corresponding to the pixel of the third color channel (such as the B channel) in the first camera and the second pixel response corresponding to the pixel of the third color channel in the second camera.

[0174] In order to use the first deviation amount corresponding to the first color channel, the second color channel and the third color channel to determine the pixel deviation amount corresponding to the first color channel, the second color channel and the third color channel respectively.

[0175] Optionally, acquiring the irradiance data of the light signal to be measured includes:

[0176] The light signal to be measured is used to obtain the irradiance of multiple channels with different wavelengths;

[0177] The irradiance data of the light signal under test is estimated based on the irradiance of the multiple different wavelength channels.

[0178] In this embodiment of the disclosure, an illuminance meter or a color temperature sensor can be used to measure the light signal to be measured, and the spectral distribution of the light signal to be measured in multiple different wavelength channels can be obtained. Based on the spectral distribution of the light signal to be measured in multiple different wavelength channels, the irradiance of the light signal to be measured in multiple different wavelength channels can be determined.

[0179] It should be noted that the irradiance of an optical signal can be obtained by superimposing the spectral distribution and reflectance coefficient of the optical signal.

[0180] Based on the irradiance of the light signal under test from multiple channels with different wavelengths, the irradiance curve of the light signal under test can be estimated by interpolation.

[0181] This disclosure also provides the following embodiments:

[0182] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 2 The method includes:

[0183] Step S201: Obtain the first spectral response data of the first camera and the second spectral response data of the second camera;

[0184] In this example, a monochromator can be used to measure the spectral response curves of the first camera and the second camera in the terminal device; by averaging the results of multiple measurements, a more accurate first spectral response curve of the first camera and second spectral response curve of the second camera can be obtained.

[0185] Here, the first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels in the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera.

[0186] It should be noted that a camera includes a lens and an image sensor. The incident light signal from the camera is focused into the image sensor through the lens. Each photosensitive element (i.e., pixel) in the image sensor undergoes a photoelectric effect based on the intensity of the light signal it receives, and outputs an electrical signal. This electrical signal can reflect the intensity of the light signal received by the photosensitive element, but it cannot reflect the color of the received light signal.

[0187] To address this, a Bayer-type image sensor (i.e., a color filter is placed in front of the image sensor) can be used to separate the red, green, and blue (R, Gb, Gr, B) components of the incident light signal. This allows the photosensitive elements of different color channels within the image sensor to output electrical signals corresponding to the different colors of light received. For example... Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the spectral response curve of a camera according to an exemplary embodiment.

[0188] Step S202: Obtain the irradiance data of the light signal to be measured;

[0189] In this example, the light signal to be measured can be the ambient light signal of the environment in which the terminal device is located. By performing scene analysis on the ambient light signal, the spectral distribution of the ambient light signal can be determined from the pre-stored CIE standard light source spectrum, and the irradiance curve of the ambient light signal can be determined.

[0190] It should be noted that the irradiance curve of a light signal can be formed by superimposing the spectral distribution and reflectance coefficient of the light signal; for the same scene, they should have the same irradiance curve. For example... Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the irradiance curve of an optical signal according to an exemplary embodiment.

[0191] Step S203: Based on the irradiance data of the light signal to be measured, determine the first light energy of the incident light signals of the first camera and the second camera; and determine the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured.

[0192] In this example, the light energy corresponding to each spectral frequency band can be determined based on the irradiance curve of the ambient light signal using the photoelectric effect.

[0193] It should be noted that a camera includes a lens and a photoelectric sensor. For the photoelectric sensor, the light energy received by each pixel is proportional to the irradiance of the incident light signal. Let the proportionality coefficient be k1, then the light energy is:

[0194] E = k1P;

[0195] Wherein, E is the light energy received by each pixel of the photoelectric sensor; P is the irradiance of the ambient light signal, and the irradiance corresponding to the ambient light signal in different spectral bands may be different; the proportionality coefficient k1 is related to the size and process of the pixels in the photoelectric sensor.

[0196] The second light energy is the energy generated when the first camera or the second camera receives a photon.

[0197] It should be noted that, according to the photoelectric effect, for photons of different frequencies γ, when bombarding the surface of a PN junction with a cutoff frequency of γ0, the work function transferred to the electron is:

[0198] W = h(γ - γ0);

[0199] Wherein, W is the energy generated by a single photon; h is Planck's constant; γ is the photon frequency; and γ0 is the cutoff frequency of the photosensitive element in the image sensor.

[0200] In step S204, the number of photons of different frequencies received by the first camera and the second camera is determined according to the ratio of the first light energy to the plurality of second light energies; based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

[0201] It should be noted that light is focused by the lens and projected into the image sensor. Each photosensitive element in the image sensor undergoes a photoelectric effect based on the intensity of the light it receives, and outputs an electrical signal. The electrical signal is then converted from analog to digital to obtain a digital quantity, which is called grayscale or gray level.

[0202] For image sensors, the grayscale value converted from the stimulated electrical signal is proportional to the number of photons received within the measurement range:

[0203]

[0204] Wherein, Q(λ) is the gray level corresponding to the optical signal with wavelength λ; k2 is the scaling factor; n is the number of photons contained in the optical signal with wavelength λ; K = k1k2 / hc, and c is the speed of light.

[0205] Step S205: Integrate the product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera; Integrate the product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to the pixels of multiple different color channels in the second camera to determine the multiple second pixel responses corresponding to the pixels of multiple different color channels in the second camera.

[0206] In this example, since the pixels of different color channels in the image sensors of the first and second cameras do not receive all photons, the pixel responses of the pixels of multiple different color channels in the first and second cameras can be determined based on the spectral response curves of the pixels of multiple different color channels in the first and second cameras.

[0207] Here, the pixel response can be used to describe the intensity of the incident light signal received by pixels in different color channels within the camera.

[0208] Pixel responses corresponding to pixel x in different color channels:

[0209] X=∫Q(λ)R x (λ)dλ;

[0210] Wherein, X is the pixel response corresponding to pixel x; Q(λ) is the grayscale distribution converted from the electrical signal generated by the image sensor under stimulation; and R... x (λ) represents the spectral response corresponding to pixel x.

[0211] Since both the spectral response curve and the grayscale distribution are numerical functions, the pixel response corresponding to pixel x in different color channels can be rewritten as a numerical integral formula:

[0212] X=K△λ∑P(λ)R x (λ)λ;

[0213] Step S206: A first deviation amount can be obtained based on the difference between the first pixel response and the second pixel response; the product of the brightness response gradient of the second camera at the current brightness, determined based on the second pixel response and the brightness response of the second camera, and the first deviation amount is determined as the pixel deviation amount between the first camera and the second camera.

[0214] Here, the pixel deviation is used to correct the image captured by the second camera when switching from the first camera to the second camera.

[0215] For cameras, due to image compression mapping, there exists a non-linear brightness mapping relationship between the brightness of the incident light signal measured by the image sensor inside the camera and the pixel values ​​(i.e., RGB values) of the final output image; for example... Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the brightness response curve of a camera according to an exemplary embodiment.

[0216] Here, the brightness response of the camera can be:

[0217] I X = f(X);

[0218] Among them, I X is the pixel value; f(·) is the brightness response of the camera.

[0219] Therefore, the pixel values ​​of the image finally output by the camera can be expressed as:

[0220] I X =f(K△λ∑P(λ)R x (λ)λ);

[0221] The pixel deviation between the image captured by the first camera and the image captured by the second camera is:

[0222] △I X =I X -I X ′=f(X)-f(X′);

[0223] Wherein, the △I X The I is the pixel deviation amount; X The I is the pixel value of the image output by the first camera. X X' represents the pixel value of the image output by the second camera; X represents the pixel response of the first camera, and X' represents the pixel response of the second camera.

[0224] If the terminal device switches from the first camera to the second camera, considering the difference in the photosensor (i.e., the second camera) under the current brightness environment, the brightness response curve of the second camera can be used to determine the brightness response gradient of the second camera under the current brightness environment, and the pixel deviation between the two cameras can be determined.

[0225] That is, the pixel deviation between the image captured by the first camera and the image captured by the second camera:

[0226] △I X ≈a(I X ′)(XX′)=a(I X ′)K△λ∑P(λ)[R x (λ)-R x ′(λ)]λ;

[0227] Among them, a(I X ′) represents the brightness response gradient of the second camera under the current brightness environment. Since a(I X ′) There is a prior estimate, which can be obtained by looking up a table; P(λ) is the irradiance of the incident light signal, which can be obtained by a color temperature sensor and does not differ for different cameras; R x (λ) represents the first spectral response corresponding to pixel x of the first camera, and R x ′(λ) is the second spectral response corresponding to pixel x of the second camera, and R x (λ)-R x ′(λ) can be obtained a priori through measurement.

[0228] Therefore, by selecting a suitable color temperature sensor to measure the irradiance of the incident light signals of the first and second cameras, a relatively accurate pixel offset can be determined. When the terminal device switches from the first camera to the second camera, the pixel offset is used to correct the image captured by the second camera, and the corrected image is output.

[0229] Since the white balance parameters of the image after pixel offset correction are the same as those of the image captured by the first camera, the flickering problem caused by the difference in white balance parameters between cameras can be effectively reduced, and a smoother and more natural transition can be achieved between different cameras.

[0230] For example, such as Figure 6 As shown, Figure 6 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 3 .

[0231] Step S301: Determine if there is an illuminance meter; if there is no illuminance meter, proceed to step S302; if there is an illuminance meter, proceed to step S303.

[0232] Step S302: Using scene analysis, determine the spectral distribution of the ambient light signal based on the pre-stored CIE standard light source spectrum;

[0233] Step S303: Measure the ten-channel spectrum in real time using a lux meter, and estimate the global spectrum of the ambient light signal using interpolation.

[0234] Step S304: Acquire the spectral response curve of the main camera;

[0235] Step S305: Acquire the spectral response curve of the secondary camera;

[0236] Step S306: Based on the spectral response curves of the main and secondary cameras and the spectral distribution of the ambient light signal, determine the brightness of the ambient light signal received by the pixels of different color channels of the main and secondary cameras.

[0237] Step S307: Calculate the white balance reference points for the main and secondary cameras;

[0238] Step S308: Estimate the brightness response gradient of the secondary camera under the current brightness environment based on the brightness response curve of the secondary camera.

[0239] Step S309: Determine the pixel offset of the secondary camera under the current brightness environment.

[0240] For example, to better demonstrate the image optimization effect of the above image processing method, images of the same scene can be captured first using the main and secondary cameras of the terminal device, such as... Figure 7 , 8 As shown, Figure 7 This is a schematic diagram illustrating a first image captured by a main camera according to an exemplary embodiment; Figure 8 This is a schematic diagram illustrating a second image captured by a secondary camera according to an exemplary embodiment. The spectral response curves of pixels in different color channels of the primary and secondary cameras are obtained; as shown... Figure 9 As shown, Figure 9 This is a schematic diagram illustrating the spectral response curves of the main and secondary cameras according to an exemplary embodiment; the spectral distribution of the ambient light signal of the environment in which the terminal device is located is obtained through a color temperature sensor; such as... Figure 10 As shown, Figure 10 This is a schematic diagram of the spectral response curve of an ambient light signal according to an exemplary embodiment.

[0241] Based on the spectral response of pixels in different color channels of the main and secondary cameras and the spectral distribution of ambient light signals in the environment where the terminal device is located, the pixel offset of the secondary camera under the current brightness environment is determined. This pixel offset is then used to correct the second image captured by the secondary camera, resulting in the third image. For example... Figure 11 As shown, Figure 11 This is a schematic diagram of a corrected third image according to an exemplary embodiment.

[0242] This disclosure also provides an image processing apparatus. Figure 12 This is a schematic diagram illustrating the structure of an image processing apparatus according to an exemplary embodiment, such as... Figure 12 As shown, the image processing device 100 includes:

[0243] The acquisition module 101 is used to acquire the first spectral response data of the first camera and the second spectral response data of the second camera; and to acquire the irradiance data of the light signal to be measured.

[0244] The first determining module 102 is used to determine the first pixel response output by the first camera and the second pixel response output by the second camera based on the irradiance data, the first spectral response data and the second spectral response data of the light signal to be measured, wherein the first pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the first camera; and the second pixel response is used to describe the intensity of the light signal to be measured received by pixels of different color channels in the second camera.

[0245] The second determining module 103 is used to determine, based on the first pixel response and the second pixel response, the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands. The pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

[0246] Optionally, the second determining module 103 is configured to:

[0247] The first deviation is obtained based on the difference between the first pixel response and the second pixel response;

[0248] The pixel deviation between the first camera and the second camera is determined based on the first deviation.

[0249] Optionally, the second determining module 103 is configured to:

[0250] The first deviation is defined as the pixel deviation between the first camera and the second camera;

[0251] or,

[0252] The pixel deviation between the first camera and the second camera is determined by multiplying the brightness response gradient of the second camera at the current brightness, which is determined based on the second pixel response and the brightness response of the second camera, with the first deviation.

[0253] Optionally, the first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels in the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera;

[0254] The first determining module 101 is used for:

[0255] Determine the grayscale distribution curves corresponding to the first signals output by the first camera and the second camera after receiving the radiation of the light signal to be measured; wherein, the grayscale distribution curves are used to describe the intensity of the light signal to be measured in different spectral frequency bands received by the cameras;

[0256] The product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera is integrated to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera.

[0257] The product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera is integrated to determine the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera.

[0258] Optionally, the first determining module 101 is further configured to:

[0259] Based on the irradiance data of the light signal to be measured, the first light energy of the incident light signals of the first camera and the second camera is determined;

[0260] Based on the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured;

[0261] The number of photons of different frequencies received by the first camera and the second camera is determined based on the ratio of the first light energy to the plurality of second light energies.

[0262] Based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

[0263] Optionally, the first pixel response includes: a plurality of first pixel responses corresponding to pixels of a plurality of different color channels in the first camera; the second pixel response includes: a plurality of second pixel responses corresponding to pixels of a plurality of different color channels in the second camera;

[0264] The second determining module 103 is used for:

[0265] The first deviation amount corresponding to the pixels of the multiple different color channels is determined based on the difference between the multiple first pixel responses corresponding to the pixels of the multiple different color channels in the first camera and the multiple second pixel responses corresponding to the pixels of the multiple different color channels in the second camera.

[0266] Optionally, the acquisition module 101 is configured to:

[0267] The light signal to be measured is used to obtain the irradiance of multiple channels with different wavelengths;

[0268] The irradiance data of the light signal under test is estimated based on the irradiance of the multiple different wavelength channels.

[0269] Figure 13 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, a mobile computer, etc.

[0270] Reference Figure 13 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0271] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0272] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0273] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0274] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0275] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0276] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0277] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0278] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as Wi-Fi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0279] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0280] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0281] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0282] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized by, The method includes: Acquire the first spectral response data of the first camera and the second spectral response data of the second camera; Acquire irradiance data of the light signal to be measured; Based on the irradiance data of the light signal under test, the first spectral response data, and the second spectral response data, a first pixel response output by the first camera and a second pixel response output by the second camera are determined. The first pixel response is determined based on the grayscale distribution curve corresponding to the first camera and a first sub-response curve included in the first spectral response data. The first sub-response curve characterizes the spectral sensitivity of pixels in different color channels within the first camera. The first pixel response describes the intensity of the light signal under test received by pixels in different color channels within the first camera. The second pixel response is determined based on the grayscale distribution curve corresponding to the second camera and a second sub-response curve included in the second spectral response data. The second sub-response curve characterizes the spectral sensitivity of pixels in different color channels within the second camera. The grayscale distribution curve is determined based on the irradiance data of the light signal under test and describes the intensity of the light signal under test in different spectral frequency bands received by the camera. The second pixel response describes the intensity of the light signal under test received by pixels in different color channels within the second camera. Based on the first pixel response and the second pixel response, the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands is determined. The pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

2. The method of claim 1, wherein, The step of determining the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands, based on the first pixel response and the second pixel response, includes: The first deviation is obtained based on the difference between the first pixel response and the second pixel response; The pixel deviation between the first camera and the second camera is determined based on the first deviation.

3. The method of claim 2, wherein, Determining the pixel deviation between the first camera and the second camera based on the first deviation includes: The first deviation is defined as the pixel deviation between the first camera and the second camera; or, The pixel deviation between the first camera and the second camera is determined by multiplying the brightness response gradient of the second camera at the current brightness, which is determined based on the second pixel response and the brightness response of the second camera, with the first deviation.

4. The method of claim 1, wherein, The first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels within the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels within the second camera; The step of determining the first pixel response output by the first camera and the second pixel response output by the second camera based on the irradiance data of the light signal under test, the first spectral response data, and the second spectral response data includes: The product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera is integrated to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera. The product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera is integrated to determine the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera.

5. The method of claim 4, wherein, The method further includes: Based on the irradiance data of the light signal to be measured, the first light energy of the incident light signals of the first camera and the second camera is determined; Based on the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured; The number of photons of different frequencies received by the first camera and the second camera is determined based on the ratio of the first light energy to the plurality of second light energies. Based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

6. The method of claim 2, wherein, The first pixel response includes: multiple first pixel responses corresponding to pixels of multiple different color channels within the first camera; the second pixel response includes: multiple second pixel responses corresponding to pixels of multiple different color channels within the second camera; The step of obtaining the first deviation amount based on the difference between the first pixel response and the second pixel response includes: The first deviation amount corresponding to the pixels of the multiple different color channels is determined based on the difference between the multiple first pixel responses corresponding to the pixels of the multiple different color channels in the first camera and the multiple second pixel responses corresponding to the pixels of the multiple different color channels in the second camera.

7. The method of claim 1, wherein, The acquisition of irradiance data for the light signal to be measured includes: The light signal to be measured is used to obtain the irradiance of multiple channels with different wavelengths; The irradiance data of the light signal under test is estimated based on the irradiance of the multiple different wavelength channels.

8. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the first spectral response data of the first camera and the second spectral response data of the second camera; Acquire irradiance data of the light signal to be measured; A first determining module is configured to determine a first pixel response output by the first camera and a second pixel response output by the second camera based on the irradiance data, the first spectral response data, and the second spectral response data of the light signal to be measured. The first pixel response is determined based on the grayscale distribution curve corresponding to the first camera and a first sub-response curve included in the first spectral response data. The first sub-response curve characterizes the spectral sensitivity of pixels in different color channels within the first camera. The first pixel response describes the intensity of the light signal to be measured received by pixels in different color channels within the first camera. The second pixel response is determined based on the grayscale distribution curve corresponding to the second camera and a second sub-response curve included in the second spectral response data. The second sub-response curve characterizes the spectral sensitivity of pixels in different color channels within the second camera. The grayscale distribution curve is determined based on the irradiance data of the light signal to be measured and describes the intensity of the light signal to be measured in different spectral frequency bands received by the camera. The second pixel response describes the intensity of the light signal to be measured received by pixels in different color channels within the second camera. The second determining module is used to determine, based on the first pixel response and the second pixel response, the pixel deviation between the output images of the first camera and the second camera after receiving the light signals to be measured in different spectral frequency bands. The pixel deviation is used to correct the image acquired by the second camera when switching from the first camera to the second camera.

9. The apparatus according to claim 8, characterized in that, The second determining module is used for: The first deviation is obtained based on the difference between the first pixel response and the second pixel response; The pixel deviation between the first camera and the second camera is determined based on the first deviation.

10. The apparatus according to claim 9, characterized in that, The second determining module is used for: The first deviation is defined as the pixel deviation between the first camera and the second camera; or, The pixel deviation between the first camera and the second camera is determined by multiplying the brightness response gradient of the second camera at the current brightness, which is determined based on the second pixel response and the brightness response of the second camera, with the first deviation.

11. The apparatus according to claim 8, characterized in that, The first spectral response data includes: multiple first sub-response curves corresponding to pixels of multiple different color channels within the first camera; the second spectral response data includes: multiple second sub-response curves corresponding to pixels of multiple different color channels within the second camera; The first determining module is used for: The product of the grayscale distribution curve corresponding to the first camera and the multiple first sub-response curves corresponding to the pixels of multiple different color channels in the first camera is integrated to determine the multiple first pixel responses corresponding to the pixels of multiple different color channels in the first camera. The product of the grayscale distribution curve corresponding to the second camera and the multiple second sub-response curves corresponding to pixels of multiple different color channels in the second camera is integrated to determine the multiple second pixel responses corresponding to pixels of multiple different color channels in the second camera.

12. The apparatus according to claim 11, characterized in that, The first determining module is further configured to: Based on the irradiance data of the light signal to be measured, the first light energy of the incident light signals of the first camera and the second camera is determined; Based on the multiple second light energies generated by the first camera and the second camera respectively after receiving multiple photons of different frequencies in the light signal to be measured; The number of photons of different frequencies received by the first camera and the second camera is determined based on the ratio of the first light energy to the plurality of second light energies. Based on the number of photons of different frequencies received by the first camera and the second camera, the grayscale distribution curves corresponding to the first camera and the second camera are determined.

13. The apparatus according to claim 9, characterized in that, The first pixel response includes: multiple first pixel responses corresponding to pixels of multiple different color channels within the first camera; the second pixel response includes: multiple second pixel responses corresponding to pixels of multiple different color channels within the second camera; The second determining module is used for: The first deviation amount corresponding to the pixels of the multiple different color channels is determined based on the difference between the multiple first pixel responses corresponding to the pixels of the multiple different color channels in the first camera and the multiple second pixel responses corresponding to the pixels of the multiple different color channels in the second camera.

14. The apparatus according to claim 8, characterized in that, The acquisition module is used for: The light signal to be measured is used to obtain the irradiance of multiple channels with different wavelengths; The irradiance data of the light signal under test is estimated based on the irradiance of the multiple different wavelength channels.

15. An image processing apparatus, characterized in that, include: processor; Memory used to store executable instructions; The processor is configured to implement the image processing method according to any one of claims 1-7 when executing executable instructions stored in the memory.

16. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an image processing apparatus, enable the image processing apparatus to perform the image processing method of any one of claims 1-7.

Images