Image processing method, electronic device, and computer readable medium

Abstract

The present disclosure provides an image processing method, including: determining a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera according to the field of view angle of a first camera and the field of view angle of a second camera; determining a calibration parameter according to at least one block data pair, wherein the block data pair is composed of a first image block in the first target area and a second image block in the second target area; and calibrating the image captured by the second camera according to the calibration parameter. The present disclosure also provides an electronic device and a computer-readable medium.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing method, an electronic device, and a computer-readable medium. Background Art

[0002] In order to enrich the focal length of the terminal, improve the imaging quality, and thus improve the shooting performance, the terminal is usually equipped with multiple cameras, each camera has corresponding functions, such as an ultra-wide-angle camera, a telephoto camera, and a blur camera.

[0003] When taking photos with a terminal, the camera is often switched, and the images before and after the switch are prone to color inconsistency, which affects the user experience. Summary of the invention

[0004] The embodiments of the present disclosure provide an image processing method, an electronic device, and a computer-readable medium.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, including: determining a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera based on a field of view angle of a first camera and a field of view angle of a second camera; determining calibration parameters based on at least one block data pair, wherein the block data pair consists of a first image block in the first target area and a second image block in the second target area; and calibrating the image captured by the second camera based on the calibration parameters.

[0006] In a second aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method described in the first aspect of the embodiment of the present disclosure.

[0007] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the image processing method described in the first aspect of the embodiment of the present disclosure is implemented.

[0008] The image processing method provided by the embodiment of the present disclosure determines the first target area and the second target area of ​​the corresponding overlapping area according to the field of view angles of the first camera and the second camera before and after switching, and then determines the calibration parameters according to the block data pairs composed of the image blocks in the first target area and the image blocks in the second target area, and then calibrates the image of the second camera after switching, so that the colors of the pictures before and after the switching of multiple cameras remain consistent, which is conducive to achieving a smooth transition of multi-camera switching and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a flowchart of an image processing method in an embodiment of the present disclosure;

[0010] Figure 2 is a flowchart of some steps in another image processing method in an embodiment of the present disclosure;

[0011] Figure 3 is a flowchart of some steps in another image processing method in an embodiment of the present disclosure;

[0012] Figure 4 is a flowchart of some steps in another image processing method in an embodiment of the present disclosure;

[0013] Figure 5 is a flowchart of some steps in another image processing method in an embodiment of the present disclosure;

[0014] Figure 6 is a flowchart of some steps in another image processing method in an embodiment of the present disclosure;

[0015] Figure 7 is a schematic diagram of the composition of an electronic device in an embodiment of the present disclosure;

[0016] Figure 8 is a schematic diagram of the composition of a computer-readable medium in an embodiment of the present disclosure;

[0017] Fig. 9 is a schematic diagram of camera switching in an embodiment of the present disclosure;

[0018] Fig.10 is a schematic diagram of a field of view overlapping area in an embodiment of the present disclosure;

[0019] Fig.11 It is a schematic diagram of dividing an image block in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the image processing method, electronic device, and computer-readable medium provided by the present disclosure are described in detail below in conjunction with the accompanying drawings.

[0021] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0022] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0023] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0024] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0025] When shooting the same scene before and after multiple cameras are switched, color inconsistency is likely to occur. The reason is that the RAW data entering the image signal processing (ISP) algorithm is different, mainly manifested in the differences in sensors and lenses, and statistical information differences in the field of view (FOV). The consistency problem before and after multiple cameras are switched can be alleviated by calibrating the key nodes of the pipeline data collection, including lens shading correction (LSC), color correction matrix (CCM), etc. However, the above solution is relatively complex to implement, and the effect cannot be effectively guaranteed, and a smooth transition of multiple camera switching cannot be achieved.

[0026] In view of this, firstly, refer to Figure 1 , the present disclosure provides an image processing method, which may include:

[0027] S1. Determine, according to the field of view angle of the first camera and the field of view angle of the second camera, a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera;

[0028] S2. determining a calibration parameter according to at least one block data pair, wherein the block data pair consists of a first image block in the first target area and a second image block in the second target area;

[0029] S3. Calibrate the image captured by the second camera according to the calibration parameters.

[0030] The image processing method provided in the embodiments of the present disclosure can be applied to terminals such as mobile phones, tablets, watches, etc. The embodiments of the present disclosure do not specifically limit this.

[0031] In the disclosed embodiment, the first camera refers to the camera before switching, and the second camera refers to the camera after switching. In the disclosed embodiment, the first camera is switched to the second camera, which can correspond to the switching scene when multiple cameras are shooting simultaneously, or can correspond to the switching scene when one camera is shooting in real time and another camera is cold-started. The disclosed embodiment does not make special restrictions on this. In some embodiments, the first camera and the second camera shoot the same scene.

[0032] In some embodiments, the first target area and the second target area have the same scene content. For example, the entire scene content in an image captured by one camera may be the same as part of the scene content in an image captured by another camera, or part of the scene content in an image captured by one camera may be the same as part of the scene content in an image captured by another camera. The embodiments of the present disclosure do not specifically limit this.

[0033] In some embodiments, the first image is an image captured by the first camera when the image is stable, and the second image is an image captured after the image of the second camera is stabilized after camera switching.

[0034] In the embodiment of the present disclosure, the first target area and the second target area are respectively divided into at least one image block, and the first image block in the first target area and the second image block in the second target area form a block data pair.

[0035] In some embodiments, the second camera is calibrated with the first camera as a reference. When determining the calibration parameters, the first image block constituting the block data pair is also used as a reference. The calibration parameters are determined according to the difference between the first image block and the second image block. When the second camera is calibrated based on the calibration parameters, the second camera can have better color consistency with the first camera.

[0036] The image processing method provided by the embodiment of the present disclosure determines the first target area and the second target area of ​​the corresponding overlapping area according to the field of view angles of the first camera and the second camera before and after switching, and then determines the calibration parameters according to the block data pairs composed of the image blocks in the first target area and the image blocks in the second target area, and then calibrates the image of the second camera after switching, so that the colors of the pictures before and after the switching of multiple cameras remain consistent, which is conducive to achieving a smooth transition of multi-camera switching and improving the user experience.

[0037] In the disclosed embodiment, the overlapping area is divided into a plurality of image blocks, and the calibration parameters are determined according to the image blocks, which can also reduce the amount of calculation and the complexity of the algorithm.

[0038] The embodiments of the present disclosure do not specifically limit how to determine the overlapping area according to the field of view of the first camera and the second camera.

[0039] In some embodiments, the field of view of the first camera and the field of view of the second camera are usually different in size. In determining the overlapping area, the field of view of the first camera and the field of view of the second camera can be preliminarily matched, and a region of interest (ROI) can be captured in the image with a larger field of view. Then, further matching is performed to determine the first target area and the second target area.

[0040] Accordingly, in some embodiments, reference Figure 2 , determining a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera according to a field of view angle of the first camera and a field of view angle of the second camera may include:

[0041] S111, determining a zoom factor according to a field of view angle of the first camera and a field of view angle of the second camera;

[0042] S112, determining a region of interest in a first image to be matched according to the scaling factor, wherein the first image to be matched is the one with a larger field of view angle between the first image and the second image;

[0043] S113. Determine the first target area and the second target area based on the same scene content in the region of interest and the second image to be matched, wherein the second image to be matched is the one with a smaller field of view between the first image and the second image, and the first target area and the second target area correspond to the same scene content.

[0044] In the disclosed embodiment, by determining the region of interest through preliminary matching, the scope of image processing during further matching can be reduced, thereby improving image processing efficiency.

[0045] The disclosed embodiment does not specifically limit how to capture ROI in an image with a larger field of view. For example, the field of view FOV of the first camera cam1 is cam1 Greater than the field of view FOV of the second camera cam2 cam2 , the size of the first image is (h1, w1), the size of the second image is (h2, w2), and in the first image, a ROI of size (h1×k, w1×k) is intercepted with (h1 / 2±Δx, w1 / 2±Δy) as the center, where Δx and Δy represent the components of the first camera center deviating from the second camera center, and k represents the scaling factor.

[0046] The disclosed embodiment does not specifically limit how to determine the scaling factor.

[0047] In some embodiments, the scaling factor may be calculated according to formula (1):

[0048]

[0049] Where k represents the zoom factor, FOV cam1 Indicates the field of view of the first camera, FOV cam2 Indicates the field of view of the second camera.

[0050] In some embodiments, since the number of pixels of the image captured by the first camera and the image captured by the second camera is different, the size of the same scene content in the first image is different from that in the second image, so the same scene content in the images captured by different cameras needs to be scaled to the same size.

[0051] Accordingly, in some embodiments, determining the first target area and the second target area according to the same scene content in the region of interest and the second image may include:

[0052] Determine a first candidate region in the region of interest corresponding to the same scene content;

[0053] Determine a second candidate region in the second image to be matched corresponding to the same scene content;

[0054] The first candidate region and the second candidate region are scaled to the same size according to the scaling factor to obtain the first target region and the second target region.

[0055] In some embodiments, reference Figure 3 , determining a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera according to a field of view angle of the first camera and a field of view angle of the second camera may include:

[0056] S121, performing Fourier-Melin Transform according to the first image and the second image to obtain an affine transformation matrix;

[0057] S122. Perform affine transformation processing according to the affine transformation matrix to obtain the first target area and the second target area.

[0058] In the embodiment of the present disclosure, when determining the first target area and the second target area through steps S121 to S122, only the Y component in the YUV color space of the first camera and the second camera may be collected, thereby reducing the amount of calculation.

[0059] Applying Fourier-Melin Transform, we get the affine transformation matrix, which can be expressed as:

[0060] affineMatrix=FourierMellinTransform(img1,img2);

[0061] Among them, img1 represents the first image, and img2 represents the second image.

[0062] Affine transformation is performed according to the affine transformation matrix to obtain overlapping areas of the same size, which can be expressed as:

[0063] FOV′=WarpAffine(FOV,affineMatrix).

[0064] In some embodiments, the first target area and the second target area are respectively divided into at least one image block, and the first image block in the first target area and the second image block in the second target area form a block data pair.

[0065] Accordingly, in some embodiments, reference Figure 4 , determining a calibration parameter based on at least one block data pair may include:

[0066] S21, matching a plurality of first image blocks in the first target area with a plurality of second image blocks in the second target area to obtain the at least one block data pair, wherein the plurality of first image blocks in the first target area correspond one-to-one to the plurality of second image blocks in the second target area;

[0067] S22. Determine the calibration parameter according to the at least one block data pair.

[0068] In some embodiments, matching the plurality of first image blocks in the first target area with the plurality of second image blocks in the second target area to obtain the at least one block data pair may include:

[0069] Dividing the first target area into a plurality of the first image blocks;

[0070] dividing the second target area into a plurality of second image blocks;

[0071] The first image block and the second image block whose color difference is greater than a preset threshold are eliminated to obtain the at least one block data pair.

[0072] In the disclosed embodiment, by eliminating the first image block and the second image block whose color difference is greater than a preset threshold, noise data pairs caused by local motion or other factors can be screened out, thereby eliminating noise interference for optimization calculation during calibration.

[0073] In some embodiments, removing the first image block and the second image block whose color difference is greater than a preset threshold to obtain the at least one block data pair may include:

[0074] Determining the mean value of each color component in the first image block;

[0075] Determining the mean value of each color component in the second image block;

[0076] The color difference between the first image block and the second image block is determined according to the mean value of each color component in the first image block and the mean value of each color component in the second image block, and the first image block and the second image block whose color difference is greater than a preset threshold are eliminated to obtain the at least one block data pair.

[0077] For example, the first target area is divided into M×N first image blocks, denoted as block m,n ,m∈[1,M],n∈[1,N]; the second target area is divided into M×N second image blocks, denoted as block′ m,n ,m∈[1,M],n∈[1,N]. Calculate the mean of the R, G, and B color components in the first image block, expressed as rgb m,n =[r mean ,g mean ,b mean ]; Calculate the mean of the R, G, and B color components in the second image block, expressed as rgb′ m,n =[r′ mean ,g′ mean ,b′ mean ].

[0078] In some embodiments, the first image block and the second image block satisfying formula (2) are eliminated:

[0079]

[0080] Among them, rgb i,j Represents the first image block, rgb′ i,j represents the second image block, r mean represents the mean value of the red component in the first image block, g mean represents the mean value of the green component in the first image block, b mean represents the mean value of the blue component in the first image block, r′ mean represents the mean value of the red component in the second image block, g′ mean represents the mean value of the green component in the second image block, b′ mean represents the mean value of the blue component in the second image block, where i and j are integers.

[0081] In some embodiments, determining the calibration parameter based on the at least one block data pair comprises:

[0082] Calculating a minimum value of a loss function according to the block data pair, wherein the loss function represents a functional relationship between the calibration parameter and the block data pair;

[0083] When the loss function takes a minimum value, the calibration parameter is obtained.

[0084] In some embodiments, the minimum value of the loss function can be calculated according to formula (3):

[0085]

[0086] Among them, f(args) represents the loss function, cm and ob are calibration parameters, ob=[obr,obg,obb];rgb i Represents the first image block of the block data pair, rgb′ i represents the second image block constituting the block data pair, C is an adjustable parameter, and I is the number of the block data pairs.

[0087] In some embodiments, the minimum value of the loss function can be calculated according to a gradient descent algorithm.

[0088] In some embodiments, reference Figure 5 , calibrating the image captured by the second camera according to the calibration parameters may include:

[0089] S31, determining a linear color calibration model according to the calibration parameters;

[0090] S32: Calibrate the image captured by the second camera according to the color calibration model.

[0091] In some embodiments, the linear color calibration model is expressed as formula (4):

[0092] src correct =cm×src+ob (4)

[0093] Among them, src correct represents the calibrated image, src represents the image before calibration, cm and ob are the calibration parameters.

[0094] For example, at coordinates (x, y):

[0095] src correct (x, y) = [r correct , g correct , bcorrect ];

[0096] src(x,y)=[r,g,b];

[0097] src correct (x, y) = cm × src (x, y) + ob is expressed as:

[0098] r correct =a 11 ×r+a 12 ×g+a 13 ×b+obr;

[0099] g correct =a 21 ×r+a 22 ×g+a 23 ×b+obg;

[0100] b correct =a 31 ×r+a 32 ×g+a 33 ×b+obb.

[0101] In some embodiments, reference Figure 6 , calibrating the image captured by the second camera may also include:

[0102] S33, performing a gradient fusion process on the image before calibration and the image after calibration to obtain a target image.

[0103] In the disclosed embodiment, the gradient fusion process refers to fusing the image before calibration and the image after calibration according to a certain weight. After the camera is switched, the later the frame is, the greater the weight of the calibrated image.

[0104] In some embodiments, a gradual blending process is performed on the first T frames after the switch.

[0105] In the disclosed embodiment, the image before calibration and the image after calibration are subjected to gradual fusion processing, which can alleviate the problem of color jump before and after camera switching.

[0106] In some embodiments, performing a gradient fusion process on the pre-calibrated image and the post-calibrated image includes:

[0107] A gradual fusion process is performed on the image before calibration and the image after calibration according to a fusion weight value, wherein the fusion weight value is positively correlated with the sequence number of the current frame in the previous T frames.

[0108] In some embodiments, the image before calibration and the image after calibration may be subjected to a gradient fusion process according to formula (5):

[0109] dst=(1-α)×src+α×src correct , α=t / (T-1) (5)

[0110] Among them, dst represents the target image, src correct represents the calibrated image, src represents the image before calibration, t represents the sequence number of the current frame, and T is an integer greater than 1.

[0111] Second, refer to Figure 7 , an embodiment of the present disclosure provides an electronic device, which may include:

[0112] One or more processors 101;

[0113] A memory 102 storing one or more programs, which, when executed by one or more processors, enables the one or more processors to implement the image processing method according to the first aspect of the embodiment of the present disclosure;

[0114] One or more I / 0 interfaces 103 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0115] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU) and the like; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus) and the like.

[0116] In some embodiments, the processor 101, the memory 102, and the I / O interface 103 are connected to each other via a bus 104, and further connected to other components of the computing device.

[0117] In some embodiments, the electronic device is a camera, a terminal, etc. This disclosure does not specifically limit this.

[0118] Fourthly, refer to Figure 8 The embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the image processing method described in the first aspect of the embodiment of the present disclosure is implemented.

[0119] In order to enable those skilled in the art to more clearly understand the technical solution provided by the embodiment of the present disclosure, the technical solution provided by the embodiment of the present disclosure is described below through an embodiment:

[0120] In this embodiment, the process of solving the color consistency problem before and after camera switching through image processing may include:

[0121] (1) Preliminarily matching the FOV of the first camera before switching with the FOV of the second camera after switching to reduce the scope of image processing during further matching.

[0122] For example, combined with Fig. 9 , Fig. 9 (a) is the first image captured by the first camera, and (b) is the second image captured by the second camera; the field of view FOV of the first camera cam1 cam1 Greater than the field of view FOV of the second camera cam2 cam2 , the size of the first image is (h1, w1), the size of the second image is (h2, w2), and in the first image, a ROI of size (h1×k, w1×k) is intercepted with (h1 / 2±Δx, w1 / 2±Δy) as the center, as shown in Fig. 9 As shown in the dotted box in (c), Δx and Δy represent the components of the first camera center deviating from the second camera center, k represents the scaling factor,

[0123] (2) The FOV of the first camera is further matched with the FOV of the switched second camera to obtain a first target area and a second target area, wherein the first target area and the second target area have the same scene content.

[0124] For example, a first candidate region and a second candidate region corresponding to the same scene content are obtained through FOV matching, and the first candidate region and the second candidate region are scaled to the same size according to the scaling factor to obtain a first target region and a second target region. The first candidate region is scaled to the same size as the second candidate region, such as Fig.10 shown.

[0125] For example, applying Fourier-Melin Transform to obtain the affine transformation matrix can be expressed as: affineMatrix = FourierMellinTransform (img1, img2); wherein img1 represents the first image, and img2 represents the second image; performing affine transformation processing according to the affine transformation matrix to obtain the first target area and the second target area of ​​the same size can be expressed as: FOV′ = WarpAffine (FOV, affineMatrix).

[0126] (3) Divide the first target area into a plurality of first image blocks, and divide the second target area into a plurality of second image blocks, and respectively calculate the mean of each color component in the first image blocks, and respectively calculate the mean of each color component in the second image blocks.

[0127] For example, Fig.11 As shown, the first target area is divided into M×N first image blocks, denoted as block m，n , m∈[1,M], n∈[1,N]; the second target area is divided into M×N second image blocks, denoted as block′ m，n , m∈[1,M], n∈[1,N]. Calculate the mean of the R, G, and B color components in the first image block, expressed as rgb m，n =[r mean , g mean , b mean ]; Calculate the mean of the R, G, and B color components in the second image block, expressed as rgb′ m,n =[r′ mean , g′ mean , b′ mean ].

[0128] (4) Eliminate image blocks with large color differences through block matching algorithm.

[0129] For example, the first image block and the second image block that satisfy the following formula are eliminated:

[0130]

[0131] Among them, rgb i，j Represents the first image block, rgb′ i，j represents the second image block, r mean represents the mean value of the red component in the first image block, g mean represents the mean value of the green component in the first image block, b mean represents the mean value of the blue component in the first image block, r′ mean represents the mean value of the red component in the second image block, g′ mean represents the mean value of the green component in the second image block, b′ mean represents the mean value of the blue component in the second image block, i and j are integers, and ξ is a predefined parameter.

[0132] (5) A linear color calibration model is constructed using the color calibration matrix and the color offset, and the calibration parameters corresponding to the minimum value of the loss function are calculated using the matching block data pairs.

[0133] For example, the loss function is f(args), and the calibration parameters of the color calibration model include cm and ob. The minimum value of the loss function f(args) is calculated according to the following formula to determine the calibration parameters:

[0134]

[0135] in,

[0136]

[0137] ob=[obr,obg,obb];

[0138] RGB i Represents the first image block of the block data pair, rgb′ i represents the second image block constituting the block data pair, C is an adjustable parameter, and I is the number of the block data pairs.

[0139] In this embodiment, the minimum value of the loss function is calculated according to the Gradient Descent algorithm.

[0140] (6) Construct a color calibration model according to the calibration parameters and calibrate the second camera.

[0141] For example, the linear color calibration model is expressed as the following formula:

[0142] src correct =cm×src+ob;

[0143] Among them, src correct represents the calibrated image, src represents the image before calibration, cm and ob are the calibration parameters.

[0144] At coordinates (x, y):

[0145] src correct (x, y) = [r correct , g correct , b correct ];

[0146] src(x,y)=[r,g,b];

[0147] src correct (x, y) = cm × src (x, y) + ob is expressed as:

[0148] r correct =a 11 ×r+a 12 ×g+a 13 ×b+obr;

[0149] g correct =a 21 ×r+a 22 ×g+a 23 ×b+obg;

[0150] b correct =a 31 ×r+a 32 ×g+a 33 ×b+obb.

[0151] In the first T frames of the second camera after switching, the image color jump is prevented by gradient fusion processing, which can be expressed as the following formula:

[0152] dst=(1-α)×src+α×src correct , α=t / (T-1);

[0153] Among them, dst represents the target image, src correct represents the calibrated image, src represents the image before calibration, t represents the tth frame after switching, and T is an integer greater than 1.

[0154] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0155] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. An image processing method, comprising: Determine, according to the field of view angle of the first camera and the field of view angle of the second camera, a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera; determining a calibration parameter based on at least one block data pair, wherein the block data pair consists of a first image block in the first target area and a second image block in the second target area; The image captured by the second camera is calibrated according to the calibration parameters.

2. The image processing method according to claim 1, wherein: Determining a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera according to a field of view angle of the first camera and a field of view angle of the second camera, including: Determining a zoom factor according to a field of view angle of the first camera and a field of view angle of the second camera; determining a region of interest in a first image to be matched according to the scaling factor, wherein the first image to be matched is the one with a larger field of view angle between the first image and the second image; The first target area and the second target area are determined based on the same scene content in the region of interest and the second image to be matched, wherein the second image to be matched is the one with a smaller field of view between the first image and the second image, and the first target area and the second target area correspond to the same scene content.

3. The image processing method according to claim 2, wherein: Determining the first target area and the second target area according to the same scene content in the region of interest and the second image includes: Determine a first candidate region in the region of interest corresponding to the same scene content; Determine a second candidate region in the second image to be matched corresponding to the same scene content; The first candidate region and the second candidate region are scaled to the same size according to the scaling factor to obtain the first target region and the second target region.

4. The image processing method according to claim 1, wherein: Determining a first target area in a first image captured by the first camera and a second target area in a second image captured by the second camera according to a field of view angle of the first camera and a field of view angle of the second camera, including: Performing Fourier-Mellin transform on the first image and the second image to obtain an affine transformation matrix; Affine transformation processing is performed according to the affine transformation matrix to obtain the first target area and the second target area.

5. The image processing method according to any one of claims 1 to 4, wherein: Determining calibration parameters based on at least one block data pair includes: Matching a plurality of first image blocks in the first target area with a plurality of second image blocks in the second target area to obtain the at least one block data pair, wherein the plurality of first image blocks in the first target area correspond one-to-one to the plurality of second image blocks in the second target area; The calibration parameter is determined based on the at least one block data pair.

6. The image processing method according to claim 5, wherein: Matching a plurality of first image blocks in the first target area with a plurality of second image blocks in the second target area to obtain the at least one block data pair includes: Dividing the first target area into a plurality of the first image blocks; dividing the second target area into a plurality of second image blocks; The first image block and the second image block whose color difference is greater than a preset threshold are eliminated to obtain the at least one block data pair.

7. The image processing method according to claim 6, wherein: Eliminating the first image block and the second image block whose color difference is greater than a preset threshold to obtain the at least one block data pair includes: Determining the mean value of each color component in the first image block; Determining the mean value of each color component in the second image block; The color difference between the first image block and the second image block is determined according to the mean value of each color component in the first image block and the mean value of each color component in the second image block, and the first image block and the second image block whose color difference is greater than a preset threshold are eliminated to obtain the at least one block data pair.

8. The image processing method according to claim 7, wherein: Eliminate the first image block and the second image block that satisfy the following formula: Among them, rgb i,j Represents the first image block, rgb′ i,j represents the second image block, r mean represents the mean value of the red component in the first image block, g mean represents the mean value of the green component in the first image block, b mean represents the mean value of the blue component in the first image block, r′ mean represents the mean value of the red component in the second image block, g′ mean represents the mean value of the green component in the second image block, b′ mean represents the mean value of the blue component in the second image block, where i and j are integers.

9. The image processing method according to claim 5, wherein: Determining the calibration parameter according to the at least one block data pair comprises: Calculating a minimum value of a loss function according to the block data pair, wherein the loss function represents a functional relationship between the calibration parameter and the block data pair; When the loss function takes a minimum value, the calibration parameter is obtained.

10. The image processing method according to claim 9, wherein: The minimum value of the loss function is calculated according to the following formula: Among them, f(args) represents the loss function, cm and ob are calibration parameters, ob=[obr,obg,obb];rgb i Represents the first image block of the block data pair, rgb′ i represents the second image block constituting the block data pair, C is an adjustable parameter, and I is the number of the block data pairs.

11. The image processing method according to any one of claims 1 to 4, wherein: Calibrating the image captured by the second camera according to the calibration parameters, including: Determining a linear color calibration model according to the calibration parameters; The image captured by the second camera is calibrated according to the color calibration model.

12. An electronic device comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1 to 11.

13. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the image processing method according to any one of claims 1 to 11 is implemented.

Images