Photosensitive data correction circuit, method, device, electronic equipment and medium

Abstract

The present application discloses a photosensitive data correction circuit, method, device, electronic device, medium and sensor calibration method and device, which belongs to the field of data processing technology. The photosensitive data correction circuit includes: a first circuit, the first circuit includes N first input interfaces, a first calculation module and N first output interfaces; the N first input interfaces are suitable for being electrically connected to the N photosensitive channels of the first image sensor in a one-to-one correspondence; the input end of the first calculation module is electrically connected to the N first input interfaces, and the output end of the first calculation module is electrically connected to the N first output interfaces; the first calculation module is used to correct the photosensitive data of the first image sensor based on a spectral correction matrix, and the spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the target function.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and specifically relates to a photosensitive data correction circuit, method, device, electronic equipment, medium and sensor calibration method and device. Background Art

[0002] At present, most mobile terminals use image sensors with different focal lengths to cover as wide a range of object distances as possible. When users switch between different image sensors, the brightness or color of the captured image may change due to the different photosensitive curves of different image sensors, thus affecting the user's overall perception of the captured image. For example, when a user is shooting an image, if he wants to zoom in on the details of the preview image, he needs to switch to image sensors with different magnifications. Since different image sensors have different photosensitive curves, users will see preview images with inconsistent brightness or color before and after switching image sensors. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a photosensitive data correction circuit, method, device, electronic device and medium, which can solve the problem of brightness or color differences between captured images generated by different image sensors.

[0004] In a first aspect, an embodiment of the present application provides a photosensitive data correction circuit, the circuit comprising:

[0005] a first circuit, the first circuit comprising N first input interfaces, a first calculation module and N first output interfaces; the N first input interfaces are suitable for being electrically connected to the N photosensitive channels of the first image sensor in a one-to-one correspondence; an input end of the first calculation module is electrically connected to the N first input interfaces, and an output end of the first calculation module is electrically connected to the N first output interfaces;

[0006] The first calculation module is used to correct the photosensitive data of the first image sensor based on a spectral correction matrix, where the spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the target function.

[0007] In a second aspect, an embodiment of the present application provides a method for correcting photosensitive data, the method comprising:

[0008] Acquire an image captured by a first image sensor;

[0009] Correcting the photosensitive data corresponding to the image based on the spectral correction matrix;

[0010] The spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function.

[0011] In a third aspect, an embodiment of the present application provides a photosensitive data correction device, the device comprising:

[0012] A first acquisition module, used to acquire a first image captured by a first image sensor;

[0013] A correction module, used for correcting the photosensitive data corresponding to the first image based on a spectral correction matrix;

[0014] The spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes the photosensitive data correction circuit as described in the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the second aspect are implemented.

[0017] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the second aspect are implemented.

[0018] In a seventh aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the second aspect.

[0019] In an eighth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the second aspect.

[0020] In a ninth aspect, an embodiment of the present application provides a sensor calibration method, the method comprising:

[0021] Acquire a spectral response function of the first image sensor and a spectral response function of the second image sensor;

[0022] determining a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor;

[0023] The spectral correction matrix is ​​calibrated as a correction parameter of the first image sensor.

[0024] In a tenth aspect, an embodiment of the present application provides a sensor calibration device, the device comprising:

[0025] A second acquisition module, used to acquire a spectral response function of the first image sensor and a spectral response function of the second image sensor;

[0026] a determination module, configured to determine a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor;

[0027] A calibration module is used to calibrate the spectral correction matrix into a correction parameter of the first image sensor.

[0028] In an embodiment of the present application, the first input interface of the first circuit is electrically connected to the photosensitive channel of the first image sensor, so that the first calculation module in the first circuit can use a spectral correction matrix to correct the photosensitive data of the first image sensor to reduce the spectral response differences between different image sensors, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color collected by different image sensors; at the same time, the spectral correction matrix can be determined based on the spectral response functions of different image sensors, thereby realizing mutual correction between different image sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a circuit structure diagram of a photosensitive data correction circuit provided in an embodiment of the present application;

[0030] Figure 2 is a schematic diagram of the deployment of the photosensitive data correction circuit provided in an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of a spectral response curve of a photosensitive data correction circuit provided by an embodiment of the present application;

[0032] Figure 4 is a circuit structure diagram of a first calculation module in a photosensitive data correction circuit provided in an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of the circuit structure of a first calculation module in a photosensitive data correction circuit provided by an embodiment of the present application;

[0034] Figure 6 It is a flowchart of a method for correcting photosensitive data provided in an embodiment of the present application;

[0035] Figure 7 is a structural schematic diagram of a photosensitive data correction device provided in an embodiment of the present application;

[0036] Figure 8 is a flow chart of a sensor calibration method provided in an embodiment of the present application;

[0037] Fig. 9 is a structural schematic diagram of a sensor calibration device provided in an embodiment of the present application;

[0038] Fig.10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0039] Fig.11 It is a hardware schematic diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0041] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated before and after are in an "or" relationship. In conjunction with the accompanying drawings, the photosensitive data correction circuit, method, device, electronic device and medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0042] like Figure 1 As shown, the photosensitive data correction circuit includes: a first circuit.

[0043] The first circuit includes N first input interfaces 110 , a first calculation module 120 , and N first output interfaces 130 .

[0044] The N first input interfaces 110 are suitable for being electrically connected to the N photosensitive channels of the first image sensor in a one-to-one correspondence; the input end of the first calculation module 120 is electrically connected to the N first input interfaces 110 , and the output end of the first calculation module 120 is electrically connected to the N first output interfaces 130 .

[0045] The first calculation module 120 is used to correct the photosensitive data of the first image sensor based on a spectral correction matrix, where the spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as an objective function.

[0046] The first circuit may further include a first circuit switch 140 , and the first circuit switch 140 is used to control the connection or disconnection of the first circuit.

[0047] In actual implementation, after the first circuit is turned on by the first circuit switch 140, the first image sensor inputs the collected photosensitive data to the N first input interfaces 110 of the first circuit via the N photosensitive channels. The first image sensor may be a sensor for collecting light signals in a mobile terminal, and may include N photosensitive channels, each of which may collect light signals of different wavelengths, i.e., photosensitive data.

[0048] For example: the N photosensitive channels of the first image sensor can be photosensitive channels based on the RGB domain, for example, including: R (red) channel, G (green) channel and B (blue) channel, then the photosensitive data can be red light, green light and blue light; it can also be photosensitive channels of other different color domains, for example, including: W (white) channel or Y (yellow) channel, etc., then the photosensitive data can be white light or yellow light; or it can also be a photosensitive channel determined according to the color cardinality of human eye sensitivity, which is not specifically limited in the embodiments of the present application.

[0049] In the related art, color correction between image sensors is performed simply through white balance, which can only stretch and transform the R and B dimensions. However, the color vision system of the human eye is a three-dimensional bionic model of xyz, so at least a correction model with three degrees of freedom must be introduced to restore the spectral difference problem. White balance refers to configuring R and B gains of different proportions based on the statistical information of the picture (G / R and G / B) so that the picture reaches an ideal distribution close to the "white" perceived by the human brain.

[0050] Therefore, the embodiment of the present application can ensure the optimization of the consistency of multi-channel sensors at a higher data dimension by adopting N photosensitive channels.

[0051] After receiving the photosensitive data, the first circuit may use the spectral correction matrix stored in the first calculation module 120 to correct the photosensitive data of the first image sensor, and output the corrected data to a subsequent processing module through the N first output interfaces 130 .

[0052] The spectral correction matrix may be pre-stored data or data calculated in real time during the use of the first image sensor, and the embodiment of the present application does not specifically limit this.

[0053] The spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function, and the spectral correction matrix can reflect the spectral response difference between the first image sensor and the second image sensor.

[0054] It is understandable that the main camera sensor and the secondary camera sensor of the mobile terminal will use image sensors from different manufacturers and with different processes due to factors such as cost, which will lead to different basic qualities between image sensors. Due to different coating processes and material differences, the filters on the Bayer array (Bayer Pattern) will respond differently to light of the same frequency, resulting in differences in spectral responses between different sensors. The Bell array is a pixel arrangement format designed for CMOS sensors, mostly with 2x2 as the basic unit, and arranged periodically. There are four commonly used formats: RGGB, GRBG, BGGR, and GBRG.

[0055] It should be noted that the existing technology is to obtain the calibration knowledge prior by statistically analyzing the differences in the sensor statistical values ​​G / R and G / B at the characteristic color temperature points in advance, and then guide the allocation of white balance gain to reduce the differences in brightness or color between the captured images generated by different image sensors.

[0056] Traditional models often use the automatic white balance (AWB) solution during the calibration process. However, the AWB solution cannot fundamentally explain the cause of the difference in multi-camera consistency and lacks theoretical support. In addition, the statistical information used in the calibration is the result of the dimension reduction of the characteristic color temperature spectrum after sensor integration. The degradation of the data dimension will lead to unreliable knowledge priors. Therefore, the knowledge priors obtained by statistics cannot represent the spectral response differences between the two sensors.

[0057] The above solution also relies on the color temperature value estimated by the white balance algorithm. Once the color temperature is not accurately judged, the previous prior relationship may be called incorrectly, and may even have the opposite effect.

[0058] The embodiment of the present application starts from the principle of image sensor imaging, does not need to rely on traditional color temperature judgment, gives the fundamental reasons that affect the brightness and color consistency between multiple cameras, and determines that the spectral response difference between different image sensors is the fundamental factor affecting the consistency of multiple cameras. The analysis process is as follows:

[0059] From the image sensor photosensitivity model, we can know that:

[0060]

[0061]

[0062] Among them, A represents the digital signal response function of the first image sensor, B represents the digital signal response function of the second image sensor, Q is the spectral response function of the sensor, R is the reflectivity function of the object, and I is the spectral function of the illuminated object itself. In other words, the digital signal response function of the image sensor is actually the integration of the spectral response function of the image sensor itself, the reflectivity function of the object, and the spectral function of the illuminated object itself in different spectral bands.

[0063] Because the first image sensor and the second image sensor take the same image of the same object, R A (λ)=R B (λ) and I A (λ)=I B (λ), so the problem of inconsistent digital signal response of the final data of the image sensor, that is, the difference in brightness or color between the captured images generated by different image sensors, can be solved by calculating the difference in spectral response function between the first image sensor and the second image sensor.

[0064] The difference in spectral response function can be solved in the following way. First, the spectral response data of the first image sensor and the second image sensor at different wavelengths are sampled respectively, and an [N×M] matrix can be constructed based on the discrete spectral response data, where N is the number of photosensitive channels and M is the number of sampling points. Then, by constructing the formula [N×N] QE [N×M] B =[N×M] A or [N×N] QE [N×M] A =[N×M] B By solving, we can determine the spectral correction matrix [N×N] QE .

[0065] It can be understood that, when the spectral response function of the first image sensor A is determined as the target function, it can be calculated based on [N×N] QE [N×M] B =[N×M] A , determine the spectral correction matrix of the second image sensor B, and then correct the photosensitive data of the second image sensor B.

[0066] When the spectral response function of the second image sensor B is determined as the target function, it can be calculated according to [N×N]QE [N×M] A =[N×M] B , determine the spectral correction matrix of the first image sensor A, and then correct the photosensitive data of the first image sensor A.

[0067] The spectral correction matrix can be solved using the least squares method or the singular value decomposition (SVD) method. Through the discrete sampling method, the spectral response matrices of different image sensors can be obtained and the spectral correction matrix can be determined by constructing the linear relationship between the image sensor matrices. Without relying on traditional color temperature judgment, the consistency of the output of multiple image sensors can be ensured at a lower level and higher data dimension.

[0068] It should be noted that the least squares method is a mathematical optimization technique. It finds the best function matching the data by minimizing the sum of squares of errors. SVD refers to decomposing the matrix A into U∑V T In the form of, U is an M×M matrix, ∑ is an M×N matrix, all elements except the elements on the main diagonal are 0, each element on the main diagonal is called a singular value, and V is an n×n matrix. U and V are both unitary matrices, that is, they satisfy:

[0069] U T U=I,V T V=I.

[0070] In one embodiment, the first image sensor described in the embodiments of the present application is a calibrated image sensor.

[0071] Since the data obtained by the first image sensor and the second image sensor are the results of integration and quantization, it is impossible to directly deploy the spectral correction matrix to the spectral response function of the first image sensor and the second image sensor. Therefore, the position of deploying the photosensitivity correction data circuit can be re-determined by the following formula.

[0072] From the image sensor photosensitivity model, we can know that:

[0073]

[0074]

[0075] Q A (λ)*R A (λ) = Q B (λ)*R B (λ)=K(λ) (3)

[0076] I B (λ)=I A(λ)*QE correct (4)

[0077] Substituting (3) and (4) into (1) we obtain: And because the matrix satisfies the associative law of multiplication:

[0078]

[0079] Then we can get:

[0080] B=A*QE correct ;

[0081] Among them, QE correct is the spectral correction matrix.

[0082] From B = A * QE correct It can be seen that the photosensitive data correction circuit storing the spectral correction matrix can be deployed in the image signal processing (ISP) path of the first image sensor A, and the final correction effect will not be affected by the data integration dimensionality reduction. For example, it can be set after the demosaicing module, such as Figure 2 shown.

[0083] Because each pixel in the Bell array can only represent one color, the image data in the Bell domain looks like a grid, similar to a mosaic. Demosaic interpolates this mosaic image into an RGB image, and interpolates the data of each grid into data represented by itself and the other two colors around it.

[0084] On the basis of retaining the original ISP path, the photosensitive data correction circuit can be implemented through a programmable integrated circuit (Field Programmable Gate Array, FPGA) or taped into the system on chip (System on Chip, SoC). Since the algorithm is relatively simple, the system has no pressure on the real-time performance and power consumption of data stream processing.

[0085] Among them, ISP can analyze the photosensitive data collected by the image sensor into color signals and brightness signals; for the digital images collected by the image sensor, ISP can also perform operations such as brightening, filtering, and color gamut restoration, so that the data collected by the image sensor is closer to or even exceeds the actual picture perceived by the human eye.

[0086] Before calibration, it is necessary to determine the spectral response function of the first image sensor and the spectral response function of the second image sensor.

[0087] Since the image sensor can only collect a limited range of spectral wavelengths, the target spectrum within the target wavelength range can be selected to sample the spectral response data. The target step wavelength for sampling can be determined to a suitable value based on actual needs, and it cannot be too large or too small. If the target step wavelength is too large, there will be too much sampling data, which will increase the amount of calculation and affect the correction efficiency; if the target step wavelength is too small, there will be too little sampling data, which will affect the correction accuracy.

[0088] In one embodiment, the wavelength range of visible light is generally 380nm-780nm, but since ordinary cameras or video cameras can only capture red, green and blue color band information and only cover the band range of 400nm-700nm, the visible light band range of 400nm-700nm is taken as the target wavelength range.

[0089] According to the signal response value d of the first image sensor A A (λ), the first spectral response value r corresponding to the target spectrum on the measuring instrument A (λ) and the first exposure time t A (λ), and the signal response value d of the second image sensor B B (λ), the second spectral response value r corresponding to the target spectrum on the measuring instrument B (λ) and the second exposure time t B (λ). Based on the formula q(λ)=d(λ) / (r(λ)*t(λ)), the spectral response functions of the three photosensitive channels R, G, and B of the first image sensor A and the second image sensor B can be obtained respectively. The closest approximate solution of A compared to B is obtained by the least squares method or SVD decomposition method.

[0090] The spectral response function of the second image sensor is used as the objective function to determine the spectral correction matrix of the spectral response function of the first image sensor. The specific calculation formula is as follows: A (λ)*QE correct =q B (λ), solving this formula can obtain the spectral correction matrix QE correct .

[0091] like Figure 3 As shown, when the photosensitive data correction circuit is effective based on the RGB domain, the upper figure is the spectral response curve of the first image sensor A and the second image sensor B before correction. The lower figure is the spectral response curve of the first image sensor A and the second image sensor B after correction. The dotted line represents the spectral response curve of the R channel, G channel, and B channel of the first image sensor A at 400nm-700nm, respectively, and the solid line represents the spectral response curve of the R channel, G channel, and B channel of the second image sensor B at 400nm-700nm, respectively. Among them, the spectral response curve is determined based on the spectral response function.

[0092] Depend on Figure 3 It can be seen that in the embodiment of the present application, the spectral response function of the second image sensor B is used as the target function, and the spectral response curve of the second image sensor B can be used as a reference to correct the spectral response curve of the first image sensor.

[0093] After correction, the spectral response curve of the first image sensor A is closer to the spectral response curve of the second image sensor B, that is, the degree of overlap between the spectral response curves of the first image sensor A and the second image sensor B becomes higher, indicating that the spectral response difference between the first image sensor A and the second image sensor B becomes smaller.

[0094] It can be understood that in the embodiment of the present application, the spectral response function of the first image sensor can also be used as the target function, and the spectral response curve of the first image sensor A can be used as a reference to correct the spectral response curve of the second image sensor B.

[0095] The photosensitive device correction circuit provided in the embodiment of the present application is electrically connected to the photosensitive channel of the first image sensor through the first input interface of the first circuit, so that the first calculation module in the first circuit can use the spectral correction matrix to correct the photosensitive data of the first image sensor to reduce the spectral response difference between different image sensors, thereby reducing the brightness or color difference between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color collected by different image sensors; at the same time, the spectral correction matrix can be determined based on the spectral response functions of different image sensors, so that mutual correction between different image sensors can be achieved.

[0096] In one embodiment, the first calculation module 120 includes N×N first registers 410 , N first adders 420 , and N first multipliers 430 .

[0097] The input ends of the N first multipliers 430 are electrically connected to the N first input interfaces 110 one by one, the output end of each first multiplier 430 is electrically connected to the input end of the N first registers 410 , and any first register 410 is electrically connected to a single first multiplier 430 .

[0098] Input terminals of the N first adders 420 are electrically connected to output terminals of the N first registers 410 in a one-to-one correspondence, and any first register 410 is electrically connected to a single first adder 420 .

[0099] The N first registers 410 connected to any first adder are electrically connected to the N first multipliers 430 in a one-to-one correspondence; the output ends of the N first adders 420 are electrically connected to the N first output interfaces 130 in a one-to-one correspondence.

[0100] The spectrum correction matrix is ​​a matrix with N rows and N columns; the N×N first registers 410 are used to store the correction parameters corresponding to the spectrum correction matrix in a one-to-one correspondence.

[0101] In actual implementation, according to the spectrum correction matrix [N×N] determined in the above embodiment QE , N×N first registers 410 are used to store the correction parameters corresponding to the spectrum correction matrix in a one-to-one correspondence.

[0102] like Figure 4 As shown, the first calculation module 120 includes: N×N first registers 410: DR1, DR2, DR3...DR N …DR N×N ; N first adders 420: A1, A2, A3...A N ; N first multipliers 430: M1, M2, M3...M N The photosensitive data collected by the first image sensor is input into the N multipliers 430 in the first circuit through the N first input interfaces 110, the output end of each first multiplier 430 is electrically connected to the input end of the N first registers 410, and the correction parameters stored in the N first registers 410 connected to each multiplier 430 correspond one-to-one to the correction parameters of each row in the spectral correction matrix.

[0103] Furthermore, N first registers 410 corresponding one-to-one to the correction parameters of each column in the spectral correction matrix are connected to N first adders 420, and the photosensitive data of the first image sensor passes through the N first registers 410 connected to each multiplier 430 respectively, and the N first registers 410 electrically connected one-to-one to the N first multipliers 430 are commonly input into any one of the first adders 420, and are output through the N first output interfaces 130, thereby outputting the corrected photosensitive data.

[0104] For example: Figure 5 As shown, the 3×3 first registers 510 correspond to the correction parameters in the stored spectrum correction matrix, that is, the spectrum correction matrix is ​​[3×3] QE , specifically:

[0105]

[0106] Then we have:

[0107] Rout=Rin*Rr+Gin*Rg+Bin*Rb;

[0108] Gout=Rin*Gr+Gin*Gg+Bin*Gb;

[0109] Bout=Rin*Br+Gin*Bg+Bin*Bb.

[0110] Among them, Rin, Gin and Bin are the input data of the three photosensitive channels respectively, and Rout, Gout and Bout are the output data of the three photosensitive channels respectively.

[0111] The photosensitive data correction circuit provided in the embodiment of the present application is not only easy to deploy but also can simplify the correction process of the photosensitive data of each photosensitive channel by setting a first calculation module with a simple structure.

[0112] In one embodiment, the photosensitive data correction circuit further includes:

[0113] A second circuit, the second circuit comprising N second input interfaces, a second calculation module and N second output interfaces; the N second input interfaces are suitable for being electrically connected to the N photosensitive channels of the second image sensor in a one-to-one correspondence; an input end of the second calculation module is electrically connected to the N second input interfaces, and an output end of the second calculation module is electrically connected to the N second output interfaces;

[0114] The second calculation module is used to adjust the light-sensitive data of the second image sensor based on the reference correction matrix.

[0115] The photosensitive data correction circuit may also include a second circuit, and the circuit elements and circuit structure of the second circuit and the first circuit are consistent with those of the first circuit, which will not be repeated here. The difference is that the N×N second registers in the second calculation module store the correction parameters corresponding to the reference correction matrix in a one-to-one correspondence. The reference correction matrix can be a unit matrix or other standard matrices. The unit matrix is ​​a square matrix in which all elements on the diagonal from the upper left corner to the lower right corner are 1, and all other elements are 0. Other standard matrices may be matrices corresponding to standard spectral response curves.

[0116] It can be understood that both the first calculation module and the second calculation module can store the spectrum correction matrix and the reference correction matrix. That is, in the case of correcting the first image sensor, the first calculation module in the first circuit stores the spectrum correction matrix, and the second calculation module in the second circuit stores the reference correction matrix; in the case of correcting the second image sensor, the second calculation module in the second circuit stores the spectrum correction matrix, and the first calculation module in the first circuit stores the reference correction matrix.

[0117] In actual implementation, the target image sensor to be calibrated can be determined according to actual needs, and can be implemented by setting multiple groups of photosensitive data correction circuits and combining multiple groups of circuit switches. Alternatively, in a group of photosensitive data correction circuits, a parameter switching device can be set inside the first circuit and the second circuit to switch the correction parameters to be used based on the target image sensor to be calibrated.

[0118] The photosensitive data correction circuit provided in the embodiment of the present application can achieve mutual correction between the first image sensor and the second image sensor by setting the second circuit, so that the user can see smooth and color-free captured images during the process of switching between different image sensors, thereby achieving imperceptible switching of image sensors.

[0119] like Figure 6 As shown, the photosensitive data correction method provided in the embodiment of the present application includes: step 610 and step 620.

[0120] Step 610: Acquire a first image captured by a first image sensor;

[0121] Step 620: Correct the photosensitive data corresponding to the first image based on the spectral correction matrix;

[0122] The spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function.

[0123] The executor of the photosensitive data correction method may be an electronic device or a functional module or functional entity in the electronic device that can implement the photosensitive data correction method. The electronic devices mentioned in the embodiments of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, wearable devices, etc. The photosensitive data correction method provided in the embodiments of the present application is described below using the electronic device as an example of the executor.

[0124] It is understandable that an electronic device may be equipped with multiple camera modules, and different camera modules may be provided with different image sensors. When a user uses an electronic device to take pictures, the user may switch image sensors, and the image brightness or color may change due to the different sensitivity curves of different image sensors.

[0125] For example, when a user needs to zoom in on the details of an image, the user is generally required to use gestures to reduce the field of view (FOV) of the screen to achieve the effect of zooming in on the details. The above process requires switching different image sensors to achieve this, and during the switching process, the zooming in on the image details will lead to a decrease in the basic quality of the picture.

[0126] It should be noted that when using a mobile phone, FOV multi-finger users will control the reduction and enlargement of the mobile phone's field of view by sliding the screen with their index finger and thumb. However, because the wide-angle and main camera lens sensors of the mobile phone are fixed-focus with a fixed field of view, in order to meet the needs of customers, the field of view change of the mobile phone preview is achieved by electronic cropping and scaling the sensor with a larger field of view to achieve pixel interpolation. When the target magnification of the next sensor design is reached, the viewfinder will switch to the field of view of the next sensor to achieve an effect similar to optical zoom.

[0127] In order to solve this problem, a first image captured by a first image sensor is first acquired in step 610. The first image is an original preview image displayed on a shooting interface of an electronic device.

[0128] In step 620, when the first image sensor is switched to the second image sensor, the photosensitive data corresponding to the first image can be corrected based on the spectral correction matrix in the electronic device to obtain a second image of the second image sensor, and the second image is the corrected first image.

[0129] On the shooting preview interface of the electronic device, switching the first image sensor to the second image sensor is represented by the first image being transformed into the second image. When no correction is performed, the user can clearly feel the change in the captured image. After the correction, the color difference and brightness difference between the first image and the second image become smaller.

[0130] The spectral correction matrix is ​​determined based on the spectral response function of the first image sensor and the spectral response function of the second image sensor, and is specifically obtained in the following manner.

[0131] From the image sensor photosensitivity model, we can know that:

[0132]

[0133]

[0134] Among them, A represents the digital signal response function of the first image sensor, B represents the digital signal response function of the second image sensor, Q is the spectral response function of the sensor, R is the reflectivity function of the object, and I is the spectral function of the illuminated object itself. In other words, the digital signal response function of the image sensor is actually the integration of the spectral response function of the image sensor itself, the reflectivity function of the object, and the spectral function of the illuminated object itself in different spectral bands.

[0135] Because the first image sensor and the second image sensor take the same image of the same object, R A (λ)=R B (λ) and I A(λ)=I B (λ), so the problem of inconsistent digital signal response of the final data of the image sensor, that is, the difference in brightness or color between the captured images generated by different image sensors, can be solved by calculating the difference in spectral response function between the first image sensor and the second image sensor.

[0136] The difference in spectral response function can be solved in the following way. First, the spectral response data of the first image sensor and the second image sensor at different wavelengths are sampled respectively, and an [N×M] matrix can be constructed based on the discrete spectral response data, where N is the number of photosensitive channels and M is the number of sampling points. Then, by constructing the formula [N×N] QE [N×M] B =[N×M] A or [N×N] QE [N×M] A =[N×M] B By solving, the spectral correction matrix can be determined.

[0137] In one embodiment, the spectral correction matrix can be pre-calibrated in the image sensor to facilitate users to directly shoot based on the photosensitive data correction method provided in the embodiment of the present application.

[0138] It is understandable that the spectral correction matrix can be determined by the photosensitive data correction circuit in the above embodiment, and can also be predetermined in a laboratory with the aid of a calibration device, which is not specifically limited here.

[0139] In actual implementation, a monochromator can be used in conjunction with an integrating sphere to emit a target spectrum within a target wavelength range. A monochromator is a component in a spectral instrument that produces monochromatic light. The structure of a monochromator is similar to that of a spectrograph, which separates a series of narrow-band electromagnetic radiation from a wide-band radiation beam.

[0140] For example, the spectrum from 400nm to 700nm can be emitted 61 times in steps of 5nm. The signal response value d of the first image sensor A is recorded. A (λ), the first spectral response value r corresponding to the target spectrum on the measuring instrument A (λ) and the first exposure time t A (λ). Record the signal response value d of the second image sensor B B (λ), the second spectral response value r corresponding to the target spectrum on the measuring instrument B (λ) and the second exposure time t B (λ). At the same time, record the spectral response value r(λ) of the ambient light on the spectral radiometer (e.g. PR655). A (λ) and d B(λ) Preprocess the data, that is, subtract the black level (Black Level Substaction, BLS).

[0141] In order to retain the data of low-voltage signals, the image sensor will add a base value to the signal before converting the analog signal to a digital signal to prevent the loss of low-bit signals during the quantization process. However, this will cause the displacement of the signal data, so BLS is to subtract the previous base value from the data finally input to the ISP channel to restore the signal to the original data.

[0142] Based on the formula q(λ)=d(λ) / (r(λ)*t(λ)), the spectral response functions of the first image sensor A and the second image sensor B can be measured respectively. The closest approximate solution of A to B is obtained by the least square method or SVD decomposition method, and the approximate solution is the spectral correction matrix.

[0143] The spectral response function of the second image sensor is used as the objective function to determine the spectral correction matrix of the spectral response function of the first image sensor. The specific calculation formula is as follows: A (λ)*QE correct =q B (λ), solving this formula can obtain the spectral correction matrix QE correct .

[0144] The device deployment and parameter calibration process is as follows: the spectral correction matrix is ​​stored in the spectral correction device of the first image sensor A, and the reference correction matrix is ​​stored in the spectral correction device of the first image sensor B. The spectral correction device can be deployed in the ISP processing path of the first image sensor A and the second image sensor B.

[0145] In one embodiment, the main switches of the spectrum correction devices in the ISP processing paths of the first image sensor A and the second image sensor B are turned on respectively.

[0146] The correction parameter configured by the spectrum correction device corresponding to the first image sensor A is a spectrum correction matrix, and the correction parameter configured by the spectrum correction device corresponding to the second image sensor B is a unit matrix.

[0147] When the photosensitive data of the first image sensor A and the second image sensor B flow into the two spectral correction devices respectively, the spectral correction device of the first image sensor A will weight the photosensitive data of the three input photosensitive channels according to the spectral correction matrix and output it to the subsequent image processing module; while the spectral correction device in the second image sensor B will not make any changes to the photosensitive data of the three photosensitive channels R, G, and B of the first image sensor A, and maintain the original numerical output, thereby correcting the photosensitive data of the first image sensor A.

[0148] The photosensitive data correction method provided in the embodiment of the present application can reduce the spectral response differences between different image sensors by correcting the photosensitive data of the first image sensor based on the spectral correction matrix, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color captured by different image sensors.

[0149] In one embodiment, the spectral response function of the first image sensor is determined according to a first spectral response value of the first image sensor under a target spectrum and a first exposure time;

[0150] The spectral response function of the second image sensor is determined according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

[0151] It is understandable that, since the image sensor can collect a limited range of spectral wavelengths, a target spectrum within the target wavelength range can be selected to sample the spectral response data. The target step wavelength for sampling can be determined to a suitable value based on actual needs, and it cannot be too large or too small. If the target step wavelength is too large, there will be too much sampling data, which will increase the amount of calculation and affect the correction efficiency; if the target step wavelength is too small, there will be too little sampling data, which will affect the correction accuracy.

[0152] In one embodiment, the wavelength range of visible light is generally 380nm-780nm, but since ordinary cameras or video cameras can only capture red, green and blue color band information and only cover the band range of 400nm-700nm, the visible light band range of 400nm-700nm is taken as the target wavelength range.

[0153] According to the signal response value d of the first image sensor A A (λ), the first spectral response value r corresponding to the target spectrum on the measuring instrument A (λ) and the first exposure time t A (λ), and the signal response value d of the second image sensor B B (λ), the second spectral response value r corresponding to the target spectrum on the measuring instrument B (λ) and the second exposure time t B (λ). Based on the formula q(λ)=d(λ) / (r(λ)*t(λ)), the spectral response functions of the first image sensor A and the second image sensor B can be obtained respectively. The closest approximate solution of A to B is obtained by the least square method or SVD decomposition method, that is, the spectral correction matrix.

[0154] Assuming that both the first image sensor and the second image sensor include three photosensitive channels, based on the number of sampling points M and the spectral response functions of the two, the matrix corresponding to the spectral response data of the first image sensor is [3×M] A , the matrix corresponding to the spectral response data of the second image sensor is [3×M] B When the target step wavelength is 5 nm, M = 61. Then, according to [3×3] QE [3×M] A =[3×M] B , we can find the quantum efficiency (QE) correction matrix that makes the A matrix closest to the B matrix. The QE correction matrix is ​​the spectral correction matrix. Among them, QE mainly refers to the efficiency of the image sensor in photoelectric conversion of light signals in different bands.

[0155] The photosensitive device correction method provided in the embodiment of the present application determines the spectral response function of the first image sensor and the spectral response function of the second image sensor based on the spectral response values ​​and exposure time of the first image sensor and the second image sensor under the target spectrum, which can ensure the correction accuracy and improve the correction efficiency.

[0156] It should be noted that the photosensitive data correction method provided in the embodiment of the present application can be executed by a photosensitive data correction device, or a control module in the photosensitive data correction device for executing the photosensitive data correction method. In the embodiment of the present application, the photosensitive data correction device provided in the embodiment of the present application is described by taking the photosensitive data correction device executing the photosensitive data correction method as an example.

[0157] like Figure 7 As shown, the photosensitive data correction device includes: a first acquisition module 710 and a correction module 720.

[0158] A first acquisition module 710 is used to acquire a first image captured by a first image sensor;

[0159] A correction module 720, configured to correct the photosensitive data corresponding to the first image based on a spectral correction matrix;

[0160] The spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function.

[0161] The photosensitive data correction device provided in the embodiment of the present application can reduce the spectral response differences between different image sensors by correcting the photosensitive data of the first image sensor based on the spectral correction matrix, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color captured by different image sensors.

[0162] In some embodiments, the spectral response function of the first image sensor is determined according to a first spectral response value of the first image sensor under a target spectrum and a first exposure time;

[0163] The spectral response function of the second image sensor is determined according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

[0164] The photosensitive data correction device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0165] The photosensitive data correction device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0166] The photosensitive data correction device provided in the embodiment of the present application can achieve Figure 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0167] In one embodiment, the present application also provides an electronic device, including the photosensitive data correction circuit described in each of the above embodiments, which will not be described again here to avoid repetition.

[0168] like Figure 8As shown, an embodiment of the present application also provides a sensor calibration method, including: step 810, step 820 and step 830.

[0169] Step 810: Obtain a spectral response function of the first image sensor and a spectral response function of the second image sensor;

[0170] Step 820: Determine a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor;

[0171] Step 830: calibrate the spectral correction matrix as correction parameters of the first image sensor.

[0172] The execution subject of the sensor calibration method in the embodiment of the present application can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited by the present invention.

[0173] The sensor calibration method provided in the embodiment of the present application is described below using a server as an example of the execution subject.

[0174] In step 810, a monochromator can be used in conjunction with an integrating sphere to emit a target spectrum within a target wavelength range. For example, a spectrum from 400nm to 700nm can be emitted 61 times in 5nm steps. The signal response value d of the first image sensor A is recorded. A (λ), the first spectral response value r corresponding to the target spectrum on the measuring instrument A (λ) and the first exposure time t A (λ). Record the signal response value d of the second image sensor B B (λ), the second spectral response value r corresponding to the target spectrum on the measuring instrument B (λ) and the second exposure time t B (λ). At the same time, record the spectral response value r(λ) of the ambient light on the spectral radiometer (e.g. PR655).A (λ) and d B (λ) Preprocess the data, that is, subtract the black level.

[0175] Based on the formula q(λ)=d(λ) / (r(λ)*t(λ)), the spectral response function of the first image sensor and the spectral response function of the second image sensor B can be measured and obtained respectively.

[0176] In step 820, based on the spectral response function of the first image sensor A and the spectral response function of the second image sensor B, the closest approximate solution of A to B is obtained by the least square method or SVD decomposition method, that is, the spectral correction matrix. The specific calculation formula is: A (λ)*QE correct =q B (λ), solving this formula can obtain the spectral correction matrix QE correct .

[0177] In step 830, the spectral correction matrix is ​​stored in the spectral correction device of the first image sensor A, and the reference correction matrix is ​​stored in the spectral correction device of the first image sensor B. The spectral correction device may be disposed in the ISP processing paths of the first image sensor A and the second image sensor B.

[0178] In actual implementation, the main switches of the spectrum correction devices in the ISP processing paths of the first image sensor A and the second image sensor B are turned on respectively.

[0179] The correction parameter configured by the spectrum correction device corresponding to the first image sensor A is a spectrum correction matrix, and the correction parameter configured by the spectrum correction device corresponding to the second image sensor B is a unit matrix.

[0180] When the photosensitive data of the first image sensor A and the second image sensor B flow into the two spectral correction devices respectively, the spectral correction device of the first image sensor A will weight the photosensitive data of the input photosensitive channel according to the spectral correction matrix and output it to the subsequent image processing module; while the spectral correction device in the second image sensor B will not make any changes to the photosensitive data of the first image sensor A, and maintain the original numerical output, thereby correcting the photosensitive data of the first image sensor A.

[0181] The sensor calibration method provided in the embodiment of the present application can reduce the spectral response differences between different image sensors by calibrating the spectral correction matrix as the correction parameters of the first image sensor, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color captured by different image sensors.

[0182] In one embodiment, obtaining the spectral response function of the first image sensor and the spectral response function of the second image sensor includes:

[0183] Acquire a spectral response function of the first image sensor according to a first spectral response value of the first image sensor under a target spectrum and a first exposure time;

[0184] A spectral response function of the second image sensor is acquired according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

[0185] In the embodiment of the present application, the method for acquiring the spectral response function of the first image sensor and the spectral response function of the second image sensor is the same as the method for determining the spectral response function in the above embodiment, which will not be repeated here.

[0186] It should be noted that the sensor calibration method provided in the embodiment of the present application can be executed by a sensor calibration device or a control module in the sensor calibration device for executing the sensor calibration method. In the embodiment of the present application, the sensor calibration device provided in the embodiment of the present application is described by taking the sensor calibration method executed by the sensor calibration device as an example.

[0187] like Fig. 9 As shown, the sensor calibration device includes: a second acquisition module 910 , a determination module 920 and a calibration module 930 .

[0188] A second acquisition module 910, configured to acquire a spectral response function of the first image sensor and a spectral response function of the second image sensor;

[0189] A determination module 920, configured to determine a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor;

[0190] The calibration module 930 is used to calibrate the spectral correction matrix as a correction parameter of the first image sensor.

[0191] The sensor calibration device provided in the embodiment of the present application can reduce the spectral response differences between different image sensors by calibrating the spectral correction matrix as the correction parameters of the first image sensor, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color captured by different image sensors.

[0192] Optionally, the second acquisition module 910 is specifically configured to:

[0193] acquiring a spectral response function of the first image sensor according to a first spectral response value of the first image sensor under a target spectrum and a first exposure time;

[0194] A spectral response function of the second image sensor is acquired according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

[0195] The sensor calibration device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0196] The sensor calibration device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0197] The sensor calibration device provided in the embodiment of the present application can achieve Figure 8 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0198] Alternatively, if Fig.10 As shown, an embodiment of the present application also provides an electronic device 1000, including a processor 1001, a memory 1002, and a program or instruction stored in the memory 1002 and executable on the processor 1001. When the program or instruction is executed by the processor 1001, each process of the above-mentioned photosensitive data correction method or sensor calibration method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0199] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0200] Fig.11A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0201] The electronic device 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109 and a processor 1110 and other components.

[0202] Those skilled in the art will appreciate that the electronic device 1100 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0203] The processor 1110 is configured to obtain a first image captured by a first image sensor;

[0204] Correcting the photosensitive data corresponding to the first image based on the spectral correction matrix;

[0205] The spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function.

[0206] The electronic device provided in the embodiment of the present application can reduce the spectral response differences between different image sensors by correcting the photosensitivity data of the first image sensor based on the spectral correction matrix, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and can ensure the consistency of the brightness and color of the images captured by different image sensors.

[0207] Optionally, the spectral response function of the first image sensor is determined according to a first spectral response value and a first exposure time of the first image sensor under a target spectrum;

[0208] The spectral response function of the second image sensor is determined according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

[0209] Alternatively, the processor 1110 is configured to obtain a spectral response function of the first image sensor and a spectral response function of the second image sensor;

[0210] determining a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor;

[0211] The spectral correction matrix is ​​calibrated as a correction parameter of the first image sensor.

[0212] The electronic device provided in the embodiment of the present application can reduce the spectral response differences between different image sensors by calibrating the spectral correction matrix as the correction parameters of the first image sensor, thereby reducing the brightness or color differences between the captured images generated by different image sensors, and ensuring the consistency of image brightness and image color captured by different image sensors.

[0213] Optionally, the processor 1110 is further configured to obtain a spectral response function of the first image sensor according to a first spectral response value and a first exposure time of the first image sensor under a target spectrum;

[0214] A spectral response function of the second image sensor is acquired according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

[0215] It should be understood that in the embodiment of the present application, the input unit 1104 may include a graphics processor (Graphics Processing Unit, GPU) 11041 and a microphone 11042, and the graphics processor 11041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 1109 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 1110 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1110.

[0216] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned photosensitive data correction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0217] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0218] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned photosensitive data correction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0219] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0220] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0221] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0222] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A photosensitive data correction circuit, characterized in that: include: A first circuit, the first circuit comprising N first input interfaces, a first calculation module and N first output interfaces; The N first input interfaces are suitable for being electrically connected to the N photosensitive channels of the first image sensor in a one-to-one correspondence; An input end of the first computing module is electrically connected to the N first input interfaces, and an output end of the first computing module is electrically connected to the N first output interfaces; The first calculation module is used to correct the photosensitive data of the first image sensor based on a spectral correction matrix, wherein the spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the target function; Wherein, the spectral response function of the first image sensor is determined according to a first spectral response value and a first exposure time of the first image sensor under a target spectrum; The spectral response function of the second image sensor is determined according to a second spectral response value and a second exposure time of the second image sensor under the target spectrum.

2. The photosensitive data correction circuit according to claim 1, characterized in that: The first calculation module includes N×N first registers, N first adders and N first multipliers; The input ends of the N first multipliers are electrically connected to the N first input interfaces in a one-to-one correspondence, the output end of each of the first multipliers is electrically connected to the input ends of the N first registers, and any first register is electrically connected to a single first multiplier; The input terminals of the N first adders are electrically connected to the output terminals of the N first registers in a one-to-one correspondence, and any one of the first registers is electrically connected to a single first adder; The N first registers connected to any of the first adders are electrically connected to the N first multipliers in a one-to-one correspondence; The output ends of the N first adders are electrically connected to the N first output interfaces in a one-to-one correspondence; The spectrum correction matrix is ​​a matrix with N rows and N columns; the N×N first registers are used to store the correction parameters corresponding to the spectrum correction matrix in a one-to-one correspondence.

3. The photosensitive data correction circuit according to claim 1 or 2, characterized in that: The photosensitive data correction circuit also includes: A second circuit, the second circuit comprising N second input interfaces, a second calculation module and N second output interfaces; The N second input interfaces are suitable for being electrically connected to the N photosensitive channels of the second image sensor in a one-to-one correspondence; An input end of the second computing module is electrically connected to the N second input interfaces, and an output end of the second computing module is electrically connected to the N second output interfaces; The second calculation module is used to adjust the light-sensitive data of the second image sensor based on a reference correction matrix.

4. A method for correcting photosensitive data, characterized in that: include: Acquire a first image captured by a first image sensor; Correcting the photosensitive data corresponding to the first image based on the spectral correction matrix; Wherein, the spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function; Wherein, the spectral response function of the first image sensor is determined according to a first spectral response value and a first exposure time of the first image sensor under a target spectrum; The spectral response function of the second image sensor is determined according to a second spectral response value and a second exposure time of the second image sensor under the target spectrum.

5. A photosensitive data correction device, characterized in that: include: A first acquisition module, used to acquire a first image captured by a first image sensor; A correction module, used for correcting the photosensitive data corresponding to the first image based on a spectral correction matrix; Wherein, the spectral correction matrix is ​​a correction matrix of the spectral response function of the first image sensor determined by taking the spectral response function of the second image sensor as the objective function; Wherein, the spectral response function of the first image sensor is determined according to a first spectral response value and a first exposure time of the first image sensor under a target spectrum; The spectral response function of the second image sensor is determined according to a second spectral response value and a second exposure time of the second image sensor under the target spectrum.

6. An electronic device, characterized in that: It comprises the photosensitive data correction circuit described in any one of claims 1 to 3.

7. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the photosensitive data correction method as claimed in claim 4 is implemented.

8. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the photosensitive data correction method as described in claim 4 is implemented.

9. A sensor calibration method, characterized in that: include: Acquire a spectral response function of the first image sensor and a spectral response function of the second image sensor; determining a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor; Calibrate the spectral correction matrix as correction parameters of the first image sensor; The step of acquiring the spectral response function of the first image sensor and the spectral response function of the second image sensor includes: Acquire a spectral response function of the first image sensor according to a first spectral response value of the first image sensor under a target spectrum and a first exposure time; A spectral response function of the second image sensor is acquired according to a second spectral response value of the second image sensor under the target spectrum and a second exposure time.

10. A sensor calibration device, characterized in that: include: A second acquisition module, used to acquire a spectral response function of the first image sensor and a spectral response function of the second image sensor; a determination module, configured to determine a spectral correction matrix of the first image sensor relative to the second image sensor according to the spectral response function of the first image sensor and the spectral response function of the second image sensor; A calibration module, used for calibrating the spectral correction matrix as a correction parameter of the first image sensor; The second acquisition module is further used to acquire a spectral response function of the first image sensor according to a first spectral response value and a first exposure time of the first image sensor under a target spectrum; The second acquisition module is further configured to acquire a spectral response function of the second image sensor according to a second spectral response value and a second exposure time of the second image sensor under the target spectrum.

Images