Image processing method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the invention discloses an image processing method, and the method comprises the steps: obtaining images shot by at least two cameras and the scene information of the images shot by the at least two cameras, and carrying out the calibration of the images according to the scene information of the images shot by the at least two cameras and the calibration data corresponding to the scene information of the images shot by the at least two cameras. And determining exposure correction parameters of the at least two cameras according to the calibration data, correcting the exposure quantity of the main camera in the at least two cameras by using the exposure correction parameters to obtain corrected exposure quantity, and correcting the exposure quantity of the main camera in the at least two cameras according to the corrected exposure quantity to obtain the exposure quantity of the main camera in the at least two cameras according to the corresponding relation between the preset scene information and the exposure parameters calibrated under the preset scene information. And processing the images shot by the at least two cameras to obtain processed images and displaying the processed images. The embodiment of the invention further provides an image processing device, electronic equipment and a computer storage medium.

Description

Technical Field

[0001] The present application relates to a multi-camera image processing technology, and more particularly to an image processing method, device, electronic device and computer storage medium. Background Art

[0002] In order to achieve exposure consistency among multiple cameras, a hardware synchronization solution can be used to improve the exposure consistency of multiple cameras, and a software solution can also be used to improve the exposure consistency of multiple cameras.

[0003] However, in improving the consistency of multi-camera exposure, the use of hardware synchronization will increase cost and complexity, there are also hardware compatibility issues, and it is limited by the accuracy of clock synchronization. The use of software solutions consumes a lot of computing resources, is sensitive to image content and lighting conditions, and requires a lot of data for model training, which leads to poor accuracy in the consistency of multi-camera exposure. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, device, electronic device, and computer storage medium, which can improve the accuracy of consistency of multi-camera exposure.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0007] Acquire images captured by at least two cameras and scene information of the images captured by the at least two cameras;

[0008] Determining exposure correction parameters of the at least two cameras according to scene information of the images captured by the at least two cameras and calibration data corresponding to the scene information of the images captured by the at least two cameras; wherein the calibration data is: a correspondence between preset scene information and exposure parameters calibrated under the preset scene information;

[0009] Using the exposure correction parameter, correcting the exposure of a main camera among the at least two cameras to obtain a corrected exposure;

[0010] The images captured by the at least two cameras are processed according to the corrected exposure to obtain and display processed images.

[0011] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0012] An acquisition module, used to acquire images taken by at least two cameras and scene information of the images taken by the at least two cameras;

[0013] A determination module, configured to determine exposure correction parameters of the at least two cameras according to scene information of the images captured by the at least two cameras and calibration data corresponding to the scene information of the images captured by the at least two cameras; wherein the calibration data is: a correspondence between preset scene information and exposure parameters calibrated under the preset scene information;

[0014] A correction module, used to correct the exposure of a main camera among the at least two cameras by using the exposure correction parameter to obtain a corrected exposure;

[0015] The processing and display module is used to process the images taken by the at least two cameras according to the corrected exposure, obtain the processed images and display them.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a storage medium storing instructions executable by the processor; the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in one or more of the above embodiments is executed.

[0017] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in one or more of the above embodiments.

[0018] The embodiments of the present application provide an image processing method, device, electronic device and computer storage medium, which obtain images taken by at least two cameras and scene information of the images taken by at least two cameras, determine exposure correction parameters of the at least two cameras according to the scene information of the images taken by at least two cameras and calibration data corresponding to the scene information of the images taken by at least two cameras, the calibration data being: a correspondence between preset scene information and exposure parameters calibrated under the preset scene information, use the exposure correction parameters to correct the exposure of a main camera in the at least two cameras to obtain a corrected exposure, process the images taken by the at least two cameras according to the corrected exposure, obtain the processed images and display them; that is, in the embodiments of the present application, by obtaining the images taken by at least two cameras and scene information of the images taken by at least two cameras, and then based on at least two The exposure correction parameters of at least two cameras are determined based on the scene information of the image captured by the camera, the preset scene information corresponding thereto, and the exposure parameters calibrated under the preset scene information, so that the exposure correction parameters of the at least two cameras obtained are obtained by considering the scene information of the image captured by the at least two cameras, the preset scene information corresponding thereto, and the exposure correction parameters calibrated under the preset scene information, that is, the exposure correction parameters are determined by fusing the scene information of multiple cameras, the preset scene information corresponding thereto, and the exposure parameters calibrated under the preset scene information. Then, the determined exposure correction parameters are used to correct the exposure amount, so that the obtained corrected exposure amount is more consistent with the scene information and calibration data of the images captured by the at least two cameras, thereby improving the exposure consistency of the images captured by the at least two cameras using the corrected exposure amount, thereby improving the image consistency under multiple cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic flow chart of an optional image processing method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of an arrangement of images captured by different cameras provided in an optional embodiment of the present application;

[0021] Figure 3 A flowchart of Example 1 of an optional image processing method provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of Example 2 of an optional image processing method provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of an optional image processing device provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0026] In the related art, in order to improve the consistency of exposure between multiple cameras, a hardware synchronization-based solution and / or a software-based solution may be adopted.

[0027] In the solution based on hardware synchronization, a synchronization signal generator is used. Specifically, an external synchronization signal generator is used to provide a unified trigger signal for multiple cameras to ensure that all cameras start and end exposure at the same time, thereby achieving accurate synchronization of exposure time. In addition, hardware timestamp alignment can also be used to add accurate timestamps to the image data of each camera. In post-processing, images taken by different cameras are aligned according to the timestamps to achieve exposure time conversion.

[0028] However, the use of hardware devices such as synchronization signal generators will increase the cost and complexity of the system. For example, in some multi-lens camera systems, it is necessary to add high-quality synchronization signal generators and complex wiring to connect the cameras, which not only increases the hardware cost, but also may increase the size and weight of the system, which is not suitable for some application scenarios that are sensitive to portability and cost.

[0029] In addition, cameras of different models or brands may have compatibility issues with the synchronization signal generator. Even different product lines of the same brand may have different hardware interfaces and electrical characteristics, which makes it impossible to directly use a unified synchronization signal for control, and additional adaptation and debugging work is required.

[0030] In addition, although using the same local crystal oscillator can reduce errors and delays, it is still difficult to achieve absolute synchronization in actual applications. As time goes by, the ambient temperature changes, and the characteristics of the hardware itself vary, the clock signal may have slight deviations, thus affecting the consistency of the exposure time. For scenes with extremely high requirements for exposure synchronization accuracy, such as high-speed imaging in scientific experiments, it may not meet the requirements.

[0031] In the software-based solution, a grayscale histogram-based matching method can be used. Specifically, the grayscale histograms of images taken by different cameras are calculated, and then the exposure time is adjusted to make the grayscale histograms of each image as similar as possible. For example, the overall brightness distribution of the image can be made consistent by changing parameters such as exposure time and exposure gain, thereby achieving exposure consistency. Although this method is relatively simple, it is sensitive to changes in image content and lighting conditions, and may need to be dynamically adjusted according to actual conditions.

[0032] Machine learning and deep learning methods can also be used. Specifically, machine learning or deep learning algorithms can be used to learn the mapping relationship between images taken by different cameras, and then the exposure time can be automatically adjusted based on this relationship. For example, a deep learning model such as a convolutional neural network can be used to input multiple camera image pairs to automatically learn and predict the exposure adjustment parameters required for each camera to achieve exposure consistency.

[0033] However, some complex software algorithms, such as those based on machine learning and deep learning, require a lot of computing resources for model training and image data processing. This may lead to slow system operation and poor real-time performance, which cannot meet some application scenarios with high real-time requirements; methods based on grayscale histogram matching are sensitive to changes in image content and lighting conditions. If the light intensity, color distribution, etc. in the shooting scene change, it may cause differences in the grayscale histogram, thereby affecting the adjustment effect of the exposure time; machine learning and deep learning methods require a large amount of labeled data to train the model to learn the mapping relationship between different camera images. Obtaining these labeled data often requires a lot of time and manpower costs, and in actual applications, it may be necessary to continuously update and expand the data to adapt to different scenarios and conditions. Otherwise, the generalization ability of the model may be limited, resulting in a decrease in the accuracy of exposure time conversion.

[0034] It can be seen that in the related art, the accuracy of multi-camera exposure consistency needs to be improved.

[0035] To solve the technical problem of poor consistency of multi-camera exposure, the present application provides an image processing method. Figure 1 A flowchart of an optional image processing method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the image processing method may include:

[0036] S101: Acquire images captured by at least two cameras and scene information of images captured by at least two cameras;

[0037] The image processing method provided in the embodiment of the present application can be applied to an electronic device, which may be a smart phone, smart glasses, a tablet computer, a smart camera device, etc.

[0038] Among them, the above-mentioned at least two cameras may include: at least two of: a main camera, a telephoto camera and a wide-angle camera. Of course, other cameras may also be included. Here, the embodiment of the present application does not make specific limitations on this.

[0039] In an embodiment of the present application, when at least two cameras in an electronic device capture images, the images captured by the at least two cameras are first acquired, and at the same time, scene information of the images captured by the at least two cameras is also acquired, wherein the scene information includes: environmental information and image parameters.

[0040] That is to say, when acquiring images taken by at least two cameras, environmental information of the images taken by at least two cameras is also acquired, and image parameters of the images taken by at least two cameras are also acquired, wherein the image parameters can be obtained based on the entire picture of the image taken by each of the at least two cameras. Of course, they can also be obtained based on a partial picture of the image taken by each of the at least two cameras. Here, the embodiments of the present application do not make specific limitations on this.

[0041] The environmental information may be the light intensity of the current environment, the environmental color temperature, etc., and the image parameters may be image brightness, image depth, etc. Here, the embodiment of the present application does not specifically limit this.

[0042] In this way, images taken by at least two cameras and environmental information and image parameters of the images taken by at least two cameras can be obtained.

[0043] S102: determining exposure correction parameters of at least two cameras according to scene information of images captured by at least two cameras and calibration data corresponding to the scene information of images captured by at least two cameras;

[0044] The scene information of the images taken by at least two cameras is obtained through the above S101, and the corresponding calibration data can be obtained based on the scene information of the images taken by at least two cameras, wherein the calibration data is: the correspondence between the preset scene information and the exposure parameters calibrated under the preset scene information.

[0045] That is to say, after obtaining the scene information captured by at least two cameras, the corresponding calibration data can be found according to the specific value of each scene information in the scene information. If the scene information of the camera is one of the preset scene information, one set of calibration data in the calibration data corresponding to the scene information is the calibration data corresponding to the preset scene information, and the other set of calibration data can be empty, or can be any set of calibration data in the calibration data of each camera.

[0046] If the scene information is not one of the preset scene information, the calibration data is two sets of calibration data in the calibration data, and each scene information in the scene information falls into the interval formed by the data corresponding to the scene information in the two sets of calibration data. For example, the color temperature in the scene information is 3700, then the color temperature of one set of calibration data in the two sets of calibration data can be 3000K, and the color temperature of the other set of calibration data can be 4500K.

[0047] In this way, the calibration data corresponding to the scene information of the images taken by at least two cameras is determined, and then the exposure correction parameters of the at least two cameras are determined based on the scene information of the images taken by at least two cameras and the calibration data corresponding to the scene information of the images taken by at least two cameras; here, the correspondence between the scene information, the calibration data and the exposure correction parameters for each camera can be stored in local preset, so as to determine the exposure correction parameters of each camera.

[0048] Alternatively, after determining the scene information of the image captured by each camera and the calibration data corresponding to the scene information of the image captured by each camera, the exposure correction parameters of at least two cameras may be calculated, for example, by performing calculations for each scene information to obtain the exposure parameters of each camera, and then determining the exposure correction parameters of at least two cameras. This is not specifically limited in the embodiments of the present application.

[0049] S103: Correcting the exposure of a main camera among the at least two cameras using the exposure correction parameter to obtain a corrected exposure;

[0050] The exposure correction parameters of at least two cameras are obtained through the above S102. In S103, the exposure correction parameters of at least two cameras can be used to correct the exposure of the main camera, so as to obtain the corrected exposure.

[0051] The exposure of the main camera can be obtained by obtaining statistical information (stats) of the main camera based on the image captured by the main camera, and then obtaining the exposure of the main camera based on the statistical information of the main camera.

[0052] It should be noted that the exposure correction parameter of the at least two cameras mentioned above can be obtained based on the exposure parameter of each camera in the at least two cameras, and is a numerical value.

[0053] S104: Processing the images captured by at least two cameras according to the corrected exposure to obtain and display processed images.

[0054] After obtaining the corrected exposure through the above S103, in S104, the images captured by at least two cameras can be processed according to the corrected exposure. The processing here can be to process the image captured by each of the at least two cameras separately, or to process only the image of the camera corresponding to the image displayed on the display interface of the electronic device of the at least two cameras. Here, the embodiment of the present application does not make any specific limitation on this.

[0055] In order to obtain images captured by at least two cameras and scene information of the images captured by at least two cameras, in an optional embodiment, S101 may include:

[0056] Acquire images captured by at least two cameras and environmental information of the captured images;

[0057] Image parameters of the images taken by the at least two cameras are determined according to the images taken by the at least two cameras to obtain scene information of the images taken by the at least two cameras.

[0058] It can be understood that when at least two cameras capture images, the electronic device obtains the image captured by each of the at least two cameras. At the same time, the at least two cameras also obtain the environmental information of the captured images through their own sensors, thereby enabling the electronic device to obtain the environmental information of the image captured by each of the at least two cameras.

[0059] After acquiring the image taken by each of the at least two cameras, image parameters are extracted for the image taken by each camera to obtain image parameters of the image taken by each camera. The image parameters can be extracted here by calculation. For example, after acquiring the image taken by each camera, the image brightness of the image taken by each camera is calculated; after acquiring the depth image of the image taken by each camera, the image depth of the image taken by each camera can be directly acquired.

[0060] In this way, image parameters of images taken by at least two cameras can be obtained through images taken by at least two cameras, thereby obtaining image parameters of images taken by at least two cameras, thereby obtaining scene information of images taken by at least two cameras and providing a data basis for determining exposure correction parameters of at least two cameras.

[0061] In order to determine the exposure correction parameters of at least two cameras, in an optional embodiment, S102 may include:

[0062] Determining, from calibration data of each camera, a first set of calibration data of each camera and a second set of calibration data of each camera according to scene information of an image captured by each camera of the at least two cameras;

[0063] Determine exposure parameters of each camera according to scene information of an image captured by each camera, a first set of calibration data of each camera, and a second set of calibration data of each camera;

[0064] Exposure correction parameters of the at least two cameras are determined according to a ratio of an exposure parameter of one of the at least two cameras other than the main camera to an exposure parameter of the main camera.

[0065] It can be understood that after knowing the scene information of the image captured by each of the at least two cameras, the interpolation table lookup method can be used to determine the first set of calibration data for each camera and the second set of calibration data for each camera from the pre-stored calibration data of each camera.

[0066] It should be noted that if the scene information is one of the preset scene information, then the first set of calibration data is the calibration data corresponding to the preset scene information, and the other set of calibration data is empty, or is any set of calibration data in the calibration data of each camera. If the scene information is not one of the preset scene information, two sets of calibration data can be found, and each scene information in the scene information of the image captured by each camera falls within the interval formed by each scene information in the two sets of calibration data. In this way, the first set of calibration data for each camera and the second set of calibration data for each camera can be obtained.

[0067] Then, after obtaining the scene information of the image taken by each camera, the first set of calibration data of each camera and the second set of calibration data of each camera, the exposure parameters of each camera can be determined. Since each set of calibration data has the exposure parameters under the scene information, here, based on the scene information of the two sets of calibration data and a certain scene information in the scene information of the image taken by each camera, the exposure parameters of each camera can be obtained by interpolation with the horizontal axis as a certain scene information and the vertical axis as the exposure parameter.

[0068] In addition, based on the scene information of the two sets of calibration data and the scene information of the image captured by each camera, the weight value of the exposure parameter of each camera in each set of calibration data can be determined, and based on the weight value of the exposure parameter of each camera in each set of calibration data and the exposure parameter of each camera in each set of calibration data, the exposure parameter of each camera can be determined. Here, the embodiments of the present application do not make specific limitations on this.

[0069] Since the main camera among the at least two cameras has a certain reference significance, here, if the image of the main camera is displayed on the display interface of the electronic device, then the exposure parameters of one of the at least two cameras other than the main camera and the exposure parameters of the main camera are obtained from the exposure parameters of each camera determined, and then the exposure correction parameters of the at least two cameras are determined according to the ratio of the exposure parameters of one of the at least two cameras other than the main camera to the exposure parameters of the main camera. If the image of the main camera is not displayed on the display interface of the electronic device, then the exposure parameters of the camera whose image is displayed on the display interface of the electronic device are obtained from the exposure parameters of each camera determined as the exposure parameters of one of the at least two cameras other than the main camera, and then the exposure correction parameters of the at least two cameras are determined according to the ratio of the exposure parameters of one of the at least two cameras other than the main camera to the exposure parameters of the main camera.

[0070] Among them, the ratio of the exposure parameters of one of the at least two cameras other than the main camera to the exposure parameters of the main camera can be directly used as the exposure correction parameter of the at least two cameras, and the ratio value can be further corrected to obtain the exposure correction parameter of the at least two cameras; here, the embodiment of the present application does not make any specific limitation on this.

[0071] In this way, two sets of calibration data can be determined by the above-mentioned interpolation table lookup method. The exposure parameters of each camera can be determined based on the scene information of the image taken by each camera and the two sets of calibration data, and then the exposure correction parameters of at least two cameras can be determined. Since the obtained exposure correction parameters are based on the current scene information of multiple cameras and the pre-calibrated calibration data, the exposure correction parameters are more suitable for the current scene under multiple cameras, which improves the consistency of multi-camera exposure and thus improves the image effect of multi-camera shooting.

[0072] In order to determine the exposure parameters of each camera, in an optional embodiment, determining the exposure parameters of each camera according to the scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera may include:

[0073] Determine, according to scene information of an image captured by each camera, a first set of calibration data of each camera, and a second set of calibration data of each camera, a weight value of an exposure parameter in the first set of calibration data of each camera and a weight value of an exposure parameter in the second set of calibration data of each camera;

[0074] Using the weight value of the exposure parameter in the first set of calibration data of each camera and the weight value of the exposure parameter in the second set of calibration data of each camera, the exposure parameters in the first set of calibration data of each camera and the exposure parameters in the second set of calibration data of each camera are weighted summed to obtain the exposure parameters of each camera.

[0075] It can be understood that after obtaining the scene information of the image taken by each camera and the two sets of calibration data of each camera, the weight value of the exposure parameter in each set of calibration data of each camera can be determined, and then based on the weight value of the exposure parameter in each set of calibration data of each camera, the exposure parameters in each set of calibration data of each camera are weightedly summed to obtain the exposure parameters of each camera.

[0076] Here, it should be noted that in the above-mentioned determination of the weight value of the exposure parameter in each set of calibration data of each camera, the preset scene information, the correspondence between the calibration data and the weight value can be used to determine the weight value of the exposure parameter in each set of calibration data of each camera. The weight value of the exposure parameter in each set of calibration data of each camera can also be calculated by a preset calculation formula. Here, the embodiment of the present application does not make specific limitations on this.

[0077] In this way, the weight value of the exposure parameter in each set of calibration data of each camera is first determined through the scene information of the image taken by each camera and the two sets of calibration data of each camera, and then the exposure parameters of each camera are determined, so that the determined exposure parameters are more in line with the current scene information, which is conducive to determining exposure correction parameters that are more suitable for the exposure consistency of multiple cameras, so as to improve the accuracy of the consistency of multi-camera exposure, thereby improving the image effect of multi-camera shooting.

[0078] In order to determine the weight value of the exposure parameter in each set of calibration data of each camera, in an optional embodiment, determining the weight value of the exposure parameter in the first set of calibration data of each camera and the weight value of the exposure parameter in the second set of calibration data of each camera according to scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera may include:

[0079] Determine a first sub-weight value of each scene information in the scene information of each camera according to each scene information in the scene information of the image captured by each camera, each scene information in the scene information of the first set of calibration data of each camera, and each scene information in the scene information of the second set of calibration data of each camera;

[0080] Multiplying the first sub-weight values ​​of each scene information in the scene information of the image captured by each camera to obtain the weight value of the exposure parameter in the first set of calibration data of each camera;

[0081] Subtracting the first sub-weight value of each scene information in the scene information of the image captured by each camera from one to obtain the second sub-weight value of each scene information in the scene information of each camera;

[0082] The second sub-weight values ​​of each scene information in the scene information of the image captured by each camera are multiplied to obtain the weight value of the exposure parameter in the second set of calibration data of each camera.

[0083] It can be understood that after obtaining the scene information of the image captured by each camera and the two sets of calibration data corresponding to the scene information, for each camera, the first sub-weight value of each scene information can be calculated according to each scene information of the image captured by it and each scene information in each set of calibration data. For example, for light intensity, the first sub-weight value of light intensity can be calculated according to the light intensity of the light image captured by each camera and the light intensity in each calibration data. The following formula can be used to calculate the first sub-weight value of each scene information in the scene information of each camera and the second sub-weight value of each scene information in the scene information of each camera:

[0084] WRatio2=(Cur_Info-Info1) / (Info2-Info1) (1)

[0085] WRatio1=1–WRatio2 (2)

[0086] Among them, Cur_info represents the scene information of the image taken by each camera, info1 represents the scene information of the first set of calibration data of each camera, info2 represents the scene information of the second set of calibration data of each camera, WRatio2 represents the first sub-weight value of each scene information in the scene information of each camera, and WRatio1 represents the second sub-weight value of each scene information in the scene information of each camera.

[0087] After obtaining the first sub-weight value of each scene information in the scene information of each camera, for each camera, the first sub-weight value of each scene information can be multiplied to obtain the weight value of the exposure parameter in the first set of calibration data of each camera.

[0088] In order to obtain the weight value of the exposure parameter in the second set of calibration data of each camera, for each camera, first subtract the first sub-weight value of each scene information from one to obtain the second sub-weight value of each scene information, and then multiply the second sub-weight values ​​of each scene information to obtain the weight value of the exposure parameter in the second set of calibration data of each camera.

[0089] In this way, by first determining the first sub-weight value of each scene information in the scene information of each camera, the weight value of the exposure parameter in the first set of calibration data of each camera and the weight value of the exposure parameter in the second set of calibration data of each camera can be determined. Based on this, the exposure parameters of each camera are determined by weighted summation, so that the determined exposure parameters of each camera are closer to the actual rules of the calibration data, which is beneficial to improving the accuracy of multi-camera exposure consistency.

[0090] In order to obtain image parameters of scene information in the calibration data, in an optional embodiment, the method may further include:

[0091] Obtaining calibration images of at least two cameras under preset scene information;

[0092] Determining image parameters of the calibration image of each of the at least two cameras based on the calibration images of the at least two cameras;

[0093] Image parameters of scene information in calibration data of each camera are determined according to image parameters of the calibration image of each camera of at least two cameras.

[0094] It can be understood that after setting the preset scene information, calibrated images of at least two cameras are obtained under the preset scene. In order to obtain calibration data, it is necessary to adjust the exposure parameters so that the image parameters are the preset image parameters, so that the preset scene information and the calibrated exposure parameters under the preset scene information can be obtained.

[0095] After adjusting the exposure parameters, it is necessary to determine the image parameters of the calibration image of each camera according to the calibration image of each camera in at least two cameras, and use them as the image parameters of the scene information in the calibration data of each camera.

[0096] It can be seen that in the calibration process, the image parameters are determined based on the calibration image of each camera. Here, the image parameters can be obtained by using the feature extraction method of the image parameters, and the image parameters can also be obtained by using the model method. Here, whether it is the feature extraction method or the model method, the image parameters can be determined based on the entire picture of the calibration image of each camera, or based on the partial picture of the calibration image of each camera; here, the embodiment of the present application does not make specific limitations on this.

[0097] In this way, by processing the calibration image, the image parameters in the calibration data can be obtained, so that the obtained calibration data is more accurate, which is beneficial to improving the accuracy of multi-camera exposure consistency.

[0098] Further, in order to obtain image parameters of the calibration image of each camera during the calibration process, in an optional embodiment, when the at least two cameras include: a main camera, a telephoto camera, and a wide-angle camera, determining the image parameters of the calibration image of each camera in the at least two cameras according to the calibration images of the at least two cameras may include:

[0099] Determine the image parameters of the calibration image of the main camera and the image parameters of the partial image corresponding to the calibration image of the telephoto camera in the calibration image of the main camera as the image parameters of the calibration image of the main camera;

[0100] Determine the image parameters of the partial image corresponding to the calibration image of the main camera in the calibration image of the wide-angle camera as the image parameters of the calibration image of the wide-angle camera;

[0101] The image parameters of the calibration image of the telephoto camera are determined as the image parameters of the calibration image of the telephoto camera.

[0102] It can be understood that for electronic devices with three cameras, they generally include a main camera, a telephoto camera and a wide-angle camera, among which the range of images captured by the wide-angle camera is the largest, followed by the range of images captured by the main camera, and finally the range of images captured by the telephoto camera.

[0103] In this case, in order to obtain the image parameters of the calibrated image of the main camera, here, in the process of calibrating the main camera, the calibrated image of the main camera and the calibrated image of the telephoto camera are obtained, the image parameters of the calibrated image of the main camera are determined, and the image parameters of the partial image corresponding to the calibrated image of the telephoto camera in the calibrated image of the main camera are determined, and these two image parameters are used as the image parameters of the calibrated image of the main camera.

[0104] In the process of calibrating the wide-angle camera, the calibrated image of the main camera and the calibrated image of the wide-angle camera are obtained, and the image parameters of the partial image corresponding to the calibrated image of the main camera in the calibrated image of the wide-angle camera are used as the image parameters of the calibrated image of the wide-angle camera.

[0105] In the process of calibrating the telephoto camera, a calibration image of the main camera is obtained, and image parameters of the calibration image of the telephoto camera are used as image parameters of the calibration image of the telephoto camera.

[0106] The above-mentioned image parameter may be image brightness.

[0107] In this way, the image parameters of the calibration images of the three cameras can be determined, thereby obtaining the image parameters in the calibration data. The image parameters of the calibration images of the three cameras are determined in the above manner, so that the accuracy of the image parameters in the obtained calibration data is further improved, which is beneficial for obtaining the exposure parameters of each camera as a reference, thereby improving the accuracy of the consistency of multi-camera exposure.

[0108] In an optional embodiment, when at least two cameras are a main camera and a wide-angle camera, the image parameters in the calibration data of the main camera are image parameters of a calibration image of the main camera; when at least two cameras are a main camera and a telephoto camera, the image parameters in the calibration data of the main camera are image parameters of a partial image in the calibration image of the main camera corresponding to the calibration image of the telephoto camera.

[0109] It can be understood that, for the scene captured by the main camera and the wide-angle camera at the same time, for the main camera, the image parameters in the calibration data can be the image parameters of the calibration image of the main camera, and for the scene captured by the main camera and the telephoto camera at the same time, for the main camera, the image parameters in the calibration data can be the image parameters of the partial image corresponding to the calibration image of the telephoto camera in the calibration image of the main camera.

[0110] In this way, the image parameters in the calibration data of the main camera are selected differently in different application scenarios. Selecting different calibration data to correct the exposure of images taken by multiple cameras can improve the exposure consistency of images taken by multiple cameras and improve the image effect of images taken by multiple cameras.

[0111] In addition, in order to obtain the image parameters of the calibration image of each camera, in an optional embodiment, determining the image parameters of the calibration image of each camera of the at least two cameras according to the calibration images of the at least two cameras may include:

[0112] An image parameter of the calibration image of each of the at least two cameras is determined according to the value of one or more channels of the calibration image of each of the at least two cameras.

[0113] It can be understood that each calibrated image has values ​​of multiple channels. For example, when the calibrated image is a RAW image, it can include values ​​of four channels: R, Gr, Gb, and B. For example, when calculating the image brightness, the image parameters of the calibrated image of each of the at least two cameras can be determined based on the value of one channel of the calibrated image of each of the at least two cameras. The image parameters of the calibrated image of each camera can also be determined based on the values ​​of any two channels of the calibrated image of each camera. The image parameters of the calibrated image of each camera can also be determined based on the values ​​of any three channels of the calibrated image of each camera. The image parameters of the calibrated image of each camera can also be determined based on the values ​​of four channels of the calibrated image of each camera. Here, the embodiments of the present application do not make specific limitations on this.

[0114] In this way, the image parameters of the calibration image of each camera can be determined by the values ​​of the above one or more channels, so that the method of determining the image parameters is more flexible and diverse, and can be applicable to the determination of image parameters of various image formats.

[0115] In order to further improve the accuracy of the image parameters in the calibration data, in an optional embodiment, the above method may further include:

[0116] Performing image recognition on the calibrated image of each camera of at least two cameras to obtain an image recognized by each camera;

[0117] According to the image recognized by each camera, the calibration image of each camera is re-determined to obtain the calibration images of at least two cameras.

[0118] It can be understood that in the process of calibrating at least two cameras, after obtaining the calibrated image of each camera, image recognition can be performed on the calibrated image to obtain the image recognized by each camera. For example, the target object in the image can be identified. At this time, the image recognized by each camera is used as the calibration image of each camera; for example, the occluded part in the image can be identified, and the image of the occluded part can be deleted from the calibration image to obtain the calibrated image of each camera again; it is also possible to identify the image part whose image parameters fall within a preset range, delete the part from the calibration image, and obtain the calibrated image of each camera again.

[0119] In this way, the calibration image of each camera can be retrieved through image recognition, so that the retrieved calibration image of each camera has more reference value, thereby making the determined calibration data more reference-worthy.

[0120] With respect to the above exposure parameters, in an optional embodiment, the exposure parameters may include one or more of the following: exposure time, exposure gain.

[0121] It can be understood that the exposure parameter can be one or all of the exposure time and exposure gain. Thus, when calculating the exposure parameter of each camera, if the exposure parameter includes the exposure time and the exposure gain, then the exposure parameter in each set of calibration data of each camera is equal to the exposure time multiplied by the exposure gain.

[0122] In this way, by determining the exposure time and / or exposure gain of each camera, the exposure correction parameters determined according to the exposure parameters of each camera take into account the exposure parameters of multiple cameras, which can improve the accuracy of the consistency of multi-camera exposure, thereby improving the image effect of multi-camera shooting.

[0123] With respect to the above scene information, in an optional embodiment, the environment information in the scene information may include one or more of the following: ambient light intensity, ambient color temperature;

[0124] The image parameters in the scene information may include one or more of the following: image brightness, image depth.

[0125] It can be understood that the above-mentioned environmental information can be one or all of the ambient light intensity and the ambient color temperature, and the above-mentioned image parameters can be one or all of the image brightness and the image depth. In this way, the preset scene information in the calibration data is more diverse, which can improve the accuracy of the consistency of multi-camera exposure, thereby improving the image effect of multi-camera shooting.

[0126] The following is an example to describe the image processing method described in one or more of the above embodiments.

[0127] First, perform laboratory data calibration:

[0128] The conditions for data calibration may include scene information such as ambient light intensity (luxindex), color temperature (CCT) and average brightness (avgluma);

[0129] The ambient light intensity (luxindex) includes different lighting conditions, such as:

[0130] The first set of data: luxindex1 is 200; the second set of data: luxindex2 is 300; the third set of data: luxindex3 is 400; the fourth set of data: luxindex4 is 500.

[0131] The color temperature (CCT) includes different color temperature conditions, for example: CCT is selected between 2700K--6500K, the first set of data: CCT1 is 3000K; the second set of data: CCT2 is 4500K; the third set of data: CCT3 is 6000K.

[0132] For the average brightness (avgluma), taking 10-bit data (maximum value 1024) as an example, the first set of data: avgluma1 is 20% of the maximum value, recorded as 204.8; the second set of data: avgluma2 is 30% of the maximum value, recorded as 307.2; the third set of data: avgluma3 is 40% of the maximum value, recorded as 409.6.

[0133] The brightness mean is calculated as follows:

[0134] The brightness calculation is performed on the RAW image. Assuming that the RAW image is arranged in the format of R, Gr, Gb, B, the brightness is calculated using the following formula:

[0135] avgluma=(R+Gr+Gb+B) / 4 (3)

[0136] Figure 2 A schematic diagram of an optional arrangement of images taken by different cameras provided in an embodiment of the present application, such as Figure 2 As shown, the field of view (FOV) of the wide-angle camera is greater than the FOV of the main camera, and the FOV of the main camera is greater than the FOV of the telephoto camera.

[0137] Among them, the average brightness of the main camera is calculated in two ways. The first one is the average brightness of the full picture, recorded as MainFullSizeAvgLuma, and the second one is the average brightness of the picture cropped by the main camera according to the telephoto FOV, recorded as MainCropSizeAvgLuma. The average brightness of the wide angle is calculated as the average brightness of the picture cropped by the wide angle according to the main camera FOV, recorded as WideCropSizeAvgLuma. The average brightness of the telephoto is calculated as the average brightness of the telephoto full picture, recorded as TeleFullSizeAvgLuma.

[0138] The exposure parameters (expinfo) include exposure time (exptime) and exposure gain (expgain).

[0139] The following are the specific steps for calibration:

[0140] Environmental conditions: Under a uniform flat background, with direct light, the camera has a fixed exposure gain (expgain), for example, fixed at 1.0. The calibration for different cameras is as follows:

[0141] Main camera calibration:

[0142] Adjust the current ambient brightness to luxindex1 according to the illuminance meter, adjust the current ambient color temperature to CCT1 according to the color temperature sensor, adjust the exposure time exptime so that MainFullSizeAvgLuma reaches avgluma1, and record the exptime at this time as MFS_Lux1_CCT1_Luma1_ET. Adjust the exposure time exptime again so that MainCropSizeAvgLuma reaches avgluma1, and record the exptime at this time as MCS_Lux1_CCT1_Luma1_ET.

[0143] Then, keeping luxindex and CCT unchanged, adjust the exposure time exptime so that MainFullSizeAvgLuma reaches avgluma2, and record the exptime at this time as MFS_Lux1_CCT1_Luma2_ET. Adjust the exposure time exptime again so that MainCropSizeAvgLuma reaches avgluma2, and record the exptime at this time as MCS_Lux1_CCT1_Luma2_ET.

[0144] By analogy, we can get:

[0145] MFS_Lux1_CCT1_Luma3_ET, MCS_Lux1_CCT1_Luma3_ET,

[0146] MFS_Lux1_CCT2_Luma1_ET, MCS_Lux1_CCT2_Luma1_ET,

[0147] MFS_Lux1_CCT2_Luma2_ET, MCS_Lux1_CCT2_Luma2_ET,

[0148] Calibration data under different luxindex and CTT such as MFS_Lux1_CCT2_Luma3_ET and MCS_Lux1_CCT2_Luma3_ET.

[0149] Calibration of wide-angle camera:

[0150] Adjust the current ambient brightness to luxindex1 according to the illuminance meter, adjust the current ambient color temperature to CCT1 according to the color temperature sensor, adjust the exposure time exptime so that WideCropSizeAvgLuma reaches avgluma1, and record the exptime as WCS_Lux1_CCT1_Luma1_ET.

[0151] By analogy, we can get:

[0152] WCS_Lux1_CCT1_Luma2_ET, WCS_Lux1_CCT1_Luma3_ET,

[0153] WCS_Lux1_CCT2_Luma1_ET, WCS_Lux1_CCT2_Luma2_ET,

[0154] Calibration data under different luxindex and CTT such as WFS_Lux1_CCT2_Luma3_ET.

[0155] Telephoto camera calibration:

[0156] Adjust the current ambient brightness to luxindex1 according to the illuminance meter, adjust the current ambient color temperature to CCT1 according to the color temperature sensor, adjust the exposure time exptime so that TeleFullSizeAvgLuma reaches avgluma1, and record the exptime as TFS_Lux1_CCT1_Luma1_ET.

[0157] By analogy, we can get:

[0158] TFS_Lux1_CCT1_Luma2_ET, TFS_Lux1_CCT1_Luma3_ET,

[0159] TFS_Lux1_CCT2_Luma1_ET, TFS_Lux1_CCT2_Luma2_ET,

[0160] Calibration data under different luxindex and CTT such as TFS_Lux1_CCT2_Luma3_ET.

[0161] Figure 3 A flowchart of an example 1 of an optional image processing method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the adaptive echo cancellation algorithm (Acoustic Echo Cancellation, AEC) is used to expose multiple camera images:

[0162] When only the main sensor is running: execute S304->S305

[0163] That is, the exposure expindex is directly calculated based on the stats obtained from the RAW image and then output.

[0164] When the other cameras are running, the main camera also runs together, and the steps S301->S302->S303->S304->S305 are performed;

[0165] S301: Input scene information (Seneor Info) into the module LookupTime;

[0166] S302: Module LookupTime outputs exposure time coefficient timeFactor;

[0167] Among them, the exposure time factor timeFactor is equivalent to the exposure parameter of each camera mentioned above;

[0168] S303: Input the exposure time coefficient calculated from the two scene information into the Mapping module;

[0169] S304: Calculate the exposure of the main camera lens according to stats and input it into the Mapping module;

[0170] S305: The final exposure is obtained through module Mapping input.

[0171] The final exposure amount is used to expose the images captured by the multiple cameras.

[0172] Among them, the module LookupTime is introduced as follows:

[0173] Figure 4 A flowchart of Example 2 of an optional image processing method provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the method may include:

[0174] S401: This module performs interpolation lookup in the exposure time list (exptime_list) according to the acquired Sensor Info;

[0175] Among them, Sensor Info may include the current light intensity (Cur_luxindex), the current color temperature (Cur_CTT), and the current brightness (Cur_Luma).

[0176] The interpolation method is coefficient distance interpolation, and two sets of calibration data for each camera are obtained from the calibration data:

[0177] The first set of calibration data is: luxindex1, CCT1, avgLuma1 corresponds to exptime1

[0178] The second set of calibration data is: luxindex2, CCT2, avgLuma2 corresponding to exptime2

[0179] S402: Calculate the exposure time coefficient of each camera:

[0180] LuxRatio2=(Cur_luxindex-luxindex1) / (luxindex2-luxindex1) (4)

[0181] LuxRatio1=1–LuxRatio2 (5)

[0182] Among them, LuxRatio2 represents the first sub-weight value of the light intensity in the scene information of each camera, and LuxRatio1 represents the second sub-weight value of the light intensity in the scene information of each camera.

[0183] CCTRatio2=(Cur_CTT–CTT1) / (CTT2-CTT1) (6)

[0184] CCTRatio1=1–CCTRatio2 (7)

[0185] Among them, CCTRatio2 represents the first sub-weight value of the color temperature in the scene information of each camera, and LuxRatio1 represents the second sub-weight value of the color temperature in the scene information of each camera.

[0186] LumaRatio2=(Cur_Luma–avgLuma1) / (avgLuma2-avgLuma1) (8)

[0187] LumaRatio1=1–LumaRatio2 (9)

[0188] Among them, LumaRatio2 represents the first sub-weight value of the brightness in the scene information of each camera, and LuxRatio1 represents the second sub-weight value of the brightness in the scene information of each camera.

[0189] The current timefactor is

[0190] Timefactor=exptime1*LuxRatio1*CCTRatio1*LumaRatio1

[0191] +exptime2*LuxRatio2*CCTRatio2*LumaRatio2(10)

[0192] Among them, exptime_list usage instructions

[0193] When the wide-angle camera and main camera are running:

[0194] The exposure time list of the wide-angle camera is composed of: WCS_Lux1_CCT1_Luma1_ET and other data; the exposure time list of the main camera is composed of: MFS_Lux1_CCT1_Luma1_ET and other data

[0195] When the telephoto camera and main camera are running:

[0196] The exposure time list of the telephoto camera is composed of TFS_Lux1_CCT1_Luma1_ET, etc. The exposure time list of the main camera is composed of MCS_Lux1_CCT1_Luma1_ET, etc.

[0197] Among them, module Mapping introduction

[0198] The calculated exposure conversion coefficient expindexRatio is equal to the quotient of timeFactor2 calculated by othersensor divided by timeFactor1 calculated by the main camera;

[0199] The output exposure expindex_output is equal to the product of the exposure expindex calculated by the main camera and the exposure conversion coefficient expindexRatio.

[0200] Among them, for scene information expansion:

[0201] Scene information is not limited to ambient brightness, ambient color temperature, image brightness, scene depth, scene subject, scene geometry, scene occlusion relationship and other scene-related conditions; wherein, the scene subject may be a target object, and after identifying the target object, the target object area is used as the basis for calculating the image brightness, the scene geometry may be brightness values ​​of different areas, and a new image area may be determined based on the brightness value, and used as the basis for calculating the image brightness, and the scene occlusion relationship may obtain the occlusion relationship of the image, thereby removing the occlusion relationship to obtain a new image, and using it as the basis for calculating the image brightness.

[0202] Extension for brightness mean calculation: The brightness mean calculation is not limited to mean calculation. You can also assign weights to R, Gr, Gb, and B, or directly use the values ​​of the single channels of R, Gr, Gb, and B for subsequent calculations.

[0203] Expansion of data calibration conditions: Calibration conditions are not limited to fixing exposure gain and adjusting exposure time to achieve expected conditions, but also include fixing exposure time and adjusting exposure gain to achieve expected conditions, or adjusting both to achieve expected conditions.

[0204] In this example, the three-camera exposure consistency technology can ensure that the three images taken at the same time have similar brightness and color performance, thereby improving the brightness and color balance, and the colors captured by different cameras can be kept coordinated and unified, making the photos look more natural and realistic.

[0205] In some functions that require multi-camera fusion, such as panoramic shooting or ultra-wide-angle-main-telephoto lens fusion zoom shooting, exposure consistency can effectively reduce image artifacts at the stitching. If the exposure is inconsistent, there will be obvious brightness differences or color faults in the stitching area. By maintaining consistent exposure, the stitched image can have a natural transition, as if it was shot with one lens, greatly improving the visual quality of the image and reducing image stitching artifacts.

[0206] It can be seen that the three-camera exposure consistency technical solution can improve image quality.

[0207] In this example, ordinary users often do not manually adjust the exposure parameters of each camera when shooting with mobile phones and other devices. The three-camera exposure consistency technology can automatically ensure that no matter which camera the user switches to, or zooms between different cameras, the exposure effect can remain stable, allowing users to focus more on the shooting content, improve the shooting experience, and provide more stable shooting effects.

[0208] In this example, photography enthusiasts have higher requirements for exposure accuracy and consistency. The three-camera exposure consistency technology allows them to obtain more reliable exposure effects when using multi-camera devices for creative shooting, such as dual-main camera blur, multi-focal length continuous shooting, etc., and better utilize the photography function of the device to meet their needs for high-quality image creation and professional shooting needs.

[0209] It can be seen that the three-camera exposure consistency technical solution can enhance the user experience.

[0210] An embodiment of the present application provides an image processing method, which obtains images taken by at least two cameras and scenes of the images taken by at least two cameras, and then determines exposure correction parameters of the at least two cameras based on scene information of the images taken by the at least two cameras, as well as preset scene information corresponding thereto and exposure parameters calibrated under the preset scene information, so that the exposure correction parameters of the at least two cameras obtained are obtained by considering the scenes of the images taken by the at least two cameras, the preset scene information corresponding thereto, and the exposure correction parameters calibrated under the preset scene information, that is, the exposure correction parameters are determined by fusing scene information of multiple cameras, the preset scene information corresponding thereto, and the exposure parameters calibrated under the preset scene information. Then, the determined exposure correction parameters are used to correct the exposure amount, so that the obtained corrected exposure amount is more consistent with the scene information and calibration data of the images taken by the at least two cameras, thereby improving the exposure consistency of the images taken by the at least two cameras using the corrected exposure amount, thereby improving the image consistency under multiple cameras.

[0211] Based on the same inventive concept as the above-mentioned embodiment, the embodiment of the present application provides an image processing device, Figure 5 A schematic diagram of the structure of an optional image processing device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the image processing device includes: an acquisition module 51, a determination module 52, a correction module 53 and a processing and display module 54; wherein,

[0212] An acquisition module 51 is used to acquire images taken by at least two cameras and scene information of images taken by at least two cameras;

[0213] A determination module 52 is used to determine exposure correction parameters of at least two cameras according to scene information of images captured by at least two cameras and calibration data corresponding to the scene information of images captured by at least two cameras; wherein the calibration data is: a correspondence between preset scene information and exposure parameters calibrated under the preset scene information;

[0214] A correction module 53, configured to correct the exposure of a main camera among the at least two cameras using an exposure correction parameter to obtain a corrected exposure;

[0215] The processing and display module 54 is used to process the images taken by at least two cameras according to the corrected exposure, obtain the processed images and display them.

[0216] In an optional embodiment, the scene information includes: environmental information and image parameters.

[0217] In an optional embodiment, the determination module 52 is specifically used to: determine a first set of calibration data for each camera and a second set of calibration data for each camera from the calibration data of each camera according to scene information of the image captured by each camera of the at least two cameras; determine exposure parameters of each camera according to the scene information of the image captured by each camera, the first set of calibration data for each camera, and the second set of calibration data for each camera; determine exposure correction parameters of at least two cameras according to a ratio value of the exposure parameters of one of the at least two cameras other than the main camera to the exposure parameters of the main camera.

[0218] In an optional embodiment, the determination module 52 determines the exposure parameters of each camera based on the scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera, including: determining the weight value of the exposure parameters in the first set of calibration data of each camera and the weight value of the exposure parameters in the second set of calibration data of each camera based on the scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera; using the weight value of the exposure parameters in the first set of calibration data of each camera and the weight value of the exposure parameters in the second set of calibration data of each camera, performing weighted summation of the exposure parameters in the first set of calibration data of each camera and the exposure parameters in the second set of calibration data of each camera to obtain the exposure parameters of each camera.

[0219] In an optional embodiment, the determination module 52 determines the weight value of the exposure parameter in the first set of calibration data of each camera and the weight value of the exposure parameter in the second set of calibration data of each camera according to the scene information of the image captured by each camera, the first set of calibration data of each camera and the second set of calibration data of each camera, including: determining the first sub-weight value of each scene information in the scene information of the image captured by each camera according to each scene information in the scene information of the image captured by each camera, each scene information in the scene information of the first set of calibration data of each camera and each scene information in the scene information of the second set of calibration data of each camera; multiplying the first sub-weight value of each scene information in the scene information of the image captured by each camera to obtain the weight value of the exposure parameter in the first set of calibration data of each camera; subtracting the first sub-weight value of each scene information in the scene information of the image captured by each camera from one to obtain the second sub-weight value of each scene information in the scene information of each camera; multiplying the second sub-weight value of each scene information in the scene information of the image captured by each camera to obtain the weight value of the exposure parameter in the second set of calibration data of each camera.

[0220] In an optional embodiment, the device is also used to: obtain calibration images of at least two cameras under preset scene information; determine image parameters of the calibration image of each of the at least two cameras based on the calibration images of the at least two cameras; determine image parameters of the scene information in the calibration data of each camera based on the image parameters of the calibration image of each of the at least two cameras.

[0221] In an optional embodiment, when at least two cameras include: a main camera, a telephoto camera and a wide-angle camera, the device determines the image parameters of the calibrated image of each of the at least two cameras based on the calibrated images of the at least two cameras, including: determining the image parameters of the calibrated image of the main camera and the image parameters of the partial image corresponding to the calibrated image of the telephoto camera in the calibrated image of the main camera as the image parameters of the calibrated image of the main camera; determining the image parameters of the partial image corresponding to the calibrated image of the main camera in the calibrated image of the wide-angle camera as the image parameters of the calibrated image of the wide-angle camera; and determining the image parameters of the calibrated image of the telephoto camera as the image parameters of the calibrated image of the telephoto camera.

[0222] In an optional embodiment, when at least two cameras are a main camera and a wide-angle camera, the image parameters in the calibration data of the main camera are image parameters of a calibration image of the main camera; when at least two cameras are a main camera and a telephoto camera, the image parameters in the calibration data of the main camera are image parameters of a partial image in the calibration image of the main camera corresponding to the calibration image of the telephoto camera.

[0223] In an optional embodiment, the device determines image parameters of the calibration image of each of the at least two cameras based on the calibration images of the at least two cameras, including: determining the image parameters of the calibration image of each of the at least two cameras based on values ​​of one or more channels of the calibration image of each of the at least two cameras.

[0224] In an optional embodiment, the device is also used to: perform image recognition on the calibration image of each camera of at least two cameras to obtain the image recognized by each camera; and re-determine the calibration image of each camera based on the image recognized by each camera to obtain the calibration images of at least two cameras.

[0225] In an optional embodiment, the exposure parameters include one or more of the following: exposure time, exposure gain.

[0226] In an optional embodiment, the environmental information in the scene information includes one or more of the following: ambient light intensity, ambient color temperature; the image parameters in the scene information include one or more of the following: image brightness, image depth.

[0227] In an optional embodiment, the acquisition module 51 is specifically used to: acquire images taken by at least two cameras and environmental information of the images; determine image parameters of the images taken by at least two cameras based on the images taken by at least two cameras to obtain scene information of the images taken by at least two cameras.

[0228] In practical applications, the acquisition module 51, determination module 52, correction module 53 and processing and display module 54 can be implemented by a processor located on the image processing device, specifically a CPU, a microprocessor (Microprocessor Unit, MPU), a digital signal processor (Digital Signal Processing, DSP) or a field programmable gate array (Field Programmable Gate Array, FPGA) and the like.

[0229] Figure 6 A schematic diagram of the structure of an optional electronic device provided in an embodiment of the present application, such as Figure 6 As shown, an embodiment of the present application provides an electronic device 600, including:

[0230] A processor 61 and a storage medium 62 storing instructions executable by the processor; the storage medium 62 relies on the processor 61 to perform operations through a communication bus 63, and when the instructions are executed by the processor, the image processing method executed by the processor side in one or more of the above embodiments is executed.

[0231] It should be noted that in actual application, the various components in the computer device are coupled together through the communication bus 63. It is understandable that the communication bus 63 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 63 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as communication buses 63.

[0232] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image processing method described in one or more of the above embodiments.

[0233] Among them, the computer-readable storage medium can be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and other memories.

[0234] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0235] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0236] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0238] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A method for processing an image, characterized in that: include: Acquire images captured by at least two cameras and scene information of the images captured by the at least two cameras; Determining exposure correction parameters of the at least two cameras according to scene information of the images captured by the at least two cameras and calibration data corresponding to the scene information of the images captured by the at least two cameras; wherein the calibration data is: a correspondence between preset scene information and exposure parameters calibrated under the preset scene information; Using the exposure correction parameter, correcting the exposure of a main camera among the at least two cameras to obtain a corrected exposure; The images captured by the at least two cameras are processed according to the corrected exposure to obtain and display processed images.

2. The method according to claim 1, characterized in that The scene information includes: environmental information and image parameters.

3. The method according to claim 2, characterized in that The step of determining exposure correction parameters of the at least two cameras according to scene information of the images captured by the at least two cameras and calibration data corresponding to the scene information of the images captured by the at least two cameras comprises: Determining, from calibration data of each camera according to scene information of an image captured by each of the at least two cameras, a first set of calibration data of each camera and a second set of calibration data of each camera; Determine exposure parameters of each camera according to scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera; The exposure correction parameters of the at least two cameras are determined according to a ratio of an exposure parameter of one of the at least two cameras other than the main camera to an exposure parameter of the main camera.

4. The method according to claim 3, characterized in that: Determining the exposure parameters of each camera according to the scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera includes: Determine, according to scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera, a weight value of the exposure parameter in the first set of calibration data of each camera and a weight value of the exposure parameter in the second set of calibration data of each camera; Using the weight value of the exposure parameter in the first set of calibration data of each camera and the weight value of the exposure parameter in the second set of calibration data of each camera, a weighted sum is performed on the exposure parameters in the first set of calibration data of each camera and the exposure parameters in the second set of calibration data of each camera to obtain the exposure parameters of each camera.

5. The method according to claim 4, characterized in that Determining the weight value of the exposure parameter in the first set of calibration data of each camera and the weight value of the exposure parameter in the second set of calibration data of each camera according to the scene information of the image captured by each camera, the first set of calibration data of each camera, and the second set of calibration data of each camera, comprises: Determine a first sub-weight value of each scene information in the scene information of the image captured by each camera according to each scene information in the scene information of the image captured by each camera, each scene information in the scene information of the first set of calibration data of each camera, and each scene information in the scene information of the second set of calibration data of each camera; Multiplying the first sub-weight values ​​of each scene information in the scene information of the image captured by each camera to obtain the weight value of the exposure parameter in the first set of calibration data of each camera; Subtracting the first sub-weight value of each scene information in the scene information of the image captured by each camera from one to obtain the second sub-weight value of each scene information in the scene information of each camera; The second sub-weight values ​​of each scene information in the scene information of the image captured by each camera are multiplied to obtain the weight value of the exposure parameter in the second set of calibration data of each camera.

6. The method according to any one of claims 2 to 5, characterized in that: The method further comprises: Acquire calibration images of the at least two cameras under preset scene information; Determining image parameters of the calibration image of each of the at least two cameras according to the calibration images of the at least two cameras; According to the image parameters of the calibration image of each camera of the at least two cameras, the image parameters of the scene information in the calibration data of each camera are determined.

7. The method according to claim 6, characterized in that When the at least two cameras include: a main camera, a telephoto camera, and a wide-angle camera, determining the image parameters of the calibrated image of each camera of the at least two cameras according to the calibrated images of the at least two cameras includes: Determining image parameters of the calibration image of the main camera and image parameters of a partial image corresponding to the calibration image of the telephoto camera in the calibration image of the main camera as image parameters of the calibration image of the main camera; Determining image parameters of a portion of the image corresponding to the calibration image of the main camera in the calibration image of the wide-angle camera as image parameters of the calibration image of the wide-angle camera; The image parameters of the calibration image of the telephoto camera are determined as the image parameters of the calibration image of the telephoto camera.

8. The method according to claim 7, characterized in that When the at least two cameras are the main camera and the wide-angle camera, the image parameters in the calibration data of the main camera are image parameters of a calibrated image of the main camera; when the at least two cameras are the main camera and a telephoto camera, the image parameters in the calibration data of the main camera are image parameters of a partial image corresponding to the calibrated image of the telephoto camera in the calibrated image of the main camera.

9. The method according to claim 6, characterized in that The step of determining the image parameters of the calibration image of each of the at least two cameras according to the calibration images of the at least two cameras comprises: An image parameter of the calibration image of each of the at least two cameras is determined according to the value of one or more channels of the calibration image of each of the at least two cameras.

10. The method according to claim 6, characterized in that The method further comprises: Performing image recognition on a calibrated image of each of the at least two cameras to obtain an image recognized by each camera; According to the image recognized by each camera, a calibration image of each camera is re-determined to obtain the calibration images of the at least two cameras.

11. The method according to any one of claims 1 to 5, characterized in that: The exposure parameters include one or more of the following: exposure time, exposure gain.

12. The method according to any one of claims 1 to 5, characterized in that: The environmental information in the scene information includes one or more of the following: ambient light intensity, ambient color temperature; The image parameters in the scene information include one or more of the following: image brightness and image depth.

13. The method according to any one of claims 1 to 5, characterized in that: The acquiring of images captured by at least two cameras and scene information of the images captured by the at least two cameras includes: Acquire images captured by the at least two cameras and environmental information of the images; Image parameters of the images captured by the at least two cameras are determined according to the images captured by the at least two cameras to obtain scene information of the images captured by the at least two cameras.

14. An image processing device, characterized in that: include: An acquisition module, used to acquire images taken by at least two cameras and scene information of the images taken by the at least two cameras; A determination module, configured to determine exposure correction parameters of the at least two cameras according to scene information of the images captured by the at least two cameras and calibration data corresponding to the scene information of the images captured by the at least two cameras; wherein the calibration data is: a correspondence between preset scene information and exposure parameters calibrated under the preset scene information; A correction module, used to correct the exposure of a main camera among the at least two cameras by using the exposure correction parameter to obtain a corrected exposure; The processing and display module is used to process the images taken by the at least two cameras according to the corrected exposure, obtain the processed images and display them.

15. An electronic device, characterized in that: include: A processor and a storage medium storing instructions executable by the processor; The storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image processing method described in any one of claims 1 to 13 is executed.

16. A computer storage medium, characterized in that: Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the image processing method described in any one of claims 1 to 13.

Images