Calibration method of image, electronic device and unmanned aerial vehicle

Abstract

This application relates to the field of image calibration technology, and in particular to an image calibration method, electronic device, and drone. In the image calibration method provided by the embodiments of this application, the electronic device can acquire first transformation parameters during initial calibration, the first transformation parameters including a first displacement parameter; the electronic device can also acquire the distance from the target to the imaging device during actual shooting; since the first transformation parameter and the distance from the target to the imaging device are known, the electronic device can acquire the second displacement parameters of the first and second images acquired at that distance based on the second displacement parameter, the correlation between the first displacement parameter and the distance, thereby acquiring the second transformation parameters at the distance from the target to the imaging device based on the second displacement parameter and the first transformation parameter, improving the image calibration accuracy.

Description

Technical Field

[0001] This application relates to the field of image calibration technology, and in particular to an image calibration method, electronic equipment, and drone. Background Technology

[0002] Cameras are core equipment in the surveillance field. However, in low-light conditions at night, the images obtained by cameras are not clear enough, leading to a sharp drop in the accuracy of image recognition. One solution is to take two photos with different spectra (e.g., infrared and low light) using the camera and then fuse them to obtain a clear image.

[0003] Since two images with different spectra depict the same subject, there is a correspondence between the pixels in the two images. It is necessary to find corresponding pixels in both images so that each pair of corresponding pixels is aligned as much as possible during fusion, and then the aligned pixels are merged together to reduce the appearance of "ghosting".

[0004] However, existing image calibration methods are all based on the initial calibration parameters obtained at the initial calibration distance during the initial factory calibration to align two images. However, in the actual shooting process after leaving the factory, the distance from the shooting target to the imaging device is usually different from the initial calibration distance. Aligning two images solely with the initial calibration parameters will result in an unsatisfactory image alignment effect. Summary of the Invention

[0005] This application provides an image calibration method, electronic device, and drone that can acquire calibration parameters, i.e., second displacement parameters, corresponding to the distance between the target and the imaging device during actual shooting, thereby improving the image calibration accuracy.

[0006] To address the aforementioned technical problems, the embodiments of this application provide the following technical solutions:

[0007] In a first aspect of this application, an image calibration method is provided. This method is applied to an electronic device, which includes an imaging device comprising a first imaging device for acquiring a first image and a second imaging device for acquiring a second image. In this method: the electronic device acquires a first transformation parameter obtained during initial calibration. The first transformation parameter characterizes the spatial transformation relationship between the first image and the second image acquired at an initial calibration distance. The first transformation parameter includes a first displacement parameter. The electronic device acquires the distance from the target to the imaging device. The electronic device acquires a second displacement parameter based on the distance, the first displacement parameter, and a correlation relationship. The second displacement parameter represents the displacement transformation between the first image and the second image acquired at the stated distance, and the correlation relationship represents the correlation between the second displacement parameter, the first displacement parameter, and the distance. The electronic device acquires the second transformation parameter based on the second displacement parameter and the first transformation parameter. The second transformation parameter characterizes the spatial transformation relationship between the first image and the second image acquired at the stated distance. In this embodiment, the electronic device can acquire first transformation parameters during initial calibration, including first displacement parameters. The electronic device can also acquire the distance from the target to the imaging device during actual shooting. Since the first transformation parameters and the distance from the target to the imaging device are known, the electronic device can acquire the second displacement parameters of the first and second images acquired at that distance based on the second displacement parameters, the correlation between the first displacement parameters and the distance, and thus obtain the second transformation parameters at the distance from the target to the imaging device, improving image calibration accuracy. Furthermore, for different electronic devices, only one calibration at the initial calibration distance is needed. Through the aforementioned correlation, the second transformation parameters of each camera at any target distance from the imaging device can be obtained.

[0008] In some embodiments, in the correlation, when the first displacement parameter remains unchanged, the second displacement parameter and the distance have a first mapping relationship; the first mapping relationship is obtained by linear fitting of data composed of multiple calibration distances and the second displacement parameter of a calibration device at each of the multiple calibration distances, wherein the calibration device is of the same model as the electronic device.

[0009] In some embodiments, in the correlation, when the distance remains constant, the second displacement parameter and the first displacement parameter have a second mapping relationship; the second mapping relationship is obtained by linearly fitting the data composed of the first displacement parameters of multiple calibration devices and the second displacement parameters of each of the multiple calibration devices at the same calibration distance.

[0010] In some embodiments, the first displacement parameter includes a parameter m for representing the horizontal displacement.20 and the parameter m used to represent vertical displacement 50 The second displacement parameter includes the parameter m used to represent the horizontal displacement. 2L and the parameter m used to represent vertical displacement 5L The first mapping relationship includes: m 2L The mapping relationship between the distance and the distance, and m 5L The mapping relationship between the distance and the distance; the second mapping relationship includes: m 2L With m 50 The mapping relationship between them, and m 5L With m 50 The mapping relationship between them.

[0011] In some embodiments, obtaining the first transformation parameters includes: obtaining the first transformation parameters in the initial projection matrix H0, wherein:

[0012]

[0013] Where m0, m1, m3, and m4 represent the scaling and rotation of the image, m2 represents the horizontal displacement of the image, m5 represents the vertical displacement of the image, m6 and m7 represent the deformation of the image in the horizontal and vertical directions, respectively, and m8 is a weighting factor that is always 1 under normalization conditions.

[0014] In some embodiments, after obtaining the second transformation parameter based on the second displacement parameter and the first transformation parameter, the method further includes: obtaining the projection matrix H between the first image and the second image acquired at a distance. L ,in:

[0015]

[0016] The second displacement parameter includes m 2L and m 5L , and m 2L Used to represent the horizontal displacement of an image, m 5L Used to represent the vertical displacement of an image. H0 and H L The values ​​m0, m1, m3, m4, m6, m7, and m8 are the same.

[0017] In some embodiments, in order to more conveniently obtain the distance from the target to the imaging device, the step of obtaining the distance from the target to the imaging device includes: obtaining the phase difference generated by the imaging device during phase focusing; and determining the distance from the target to the imaging device based on the phase difference.

[0018] In some embodiments, the step of determining the distance from the target to the imaging device based on the phase difference includes: the electronic device determining a code value corresponding to the phase difference, wherein the phase difference and the code value have a mapping relationship; and the electronic device determining the distance from the target to the imaging device based on the code value.

[0019] In a second aspect of this application, an electronic device is provided, comprising: a processor and a memory storing instructions, which, when executed by the processor, cause the electronic device to perform the method described in the first aspect.

[0020] In some embodiments, the electronic device includes a drone.

[0021] In a third aspect of this application, a computer-readable storage medium is provided that stores instructions which, when executed by an electronic device, cause the electronic device to perform the method described in the first aspect.

[0022] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0024] Figure 1 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of a drone provided in one embodiment of this application;

[0026] Figure 3 This is a flowchart of an image calibration method provided in one embodiment of this application;

[0027] Figure 4 This is a schematic diagram of an image calibration scene provided in one embodiment of this application. Detailed Implementation

[0028] The principles and spirit of this disclosure will be described below with reference to several exemplary embodiments illustrated in the accompanying drawings. It should be understood that these specific embodiments are described merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0029] As used herein, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects and are used only to distinguish the objects referred to, without implying a particular spatial order, temporal order, order of importance, etc., of the objects referred to.

[0030] Figure 1 An exemplary hardware structure of an electronic device is shown, the electronic device 10 including a processor 11, a memory 12 and a camera device 13.

[0031] The electronic devices involved in the embodiments of this application may include handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem. They may also include drones, cellular phones, smartphones, personal digital assistant (PDA) computers, tablet computers, laptop computers, cameras, video recorders, cameras, smartwatches, smart wristbands, in-vehicle computers, and other electronic devices with imaging capabilities. The embodiments of this application do not impose special limitations on the specific form of the above-mentioned electronic devices.

[0032] Unmanned aerial vehicles (UAVs) are unmanned aircraft controlled by radio remote control equipment and their own program control devices. UAVs are widely used in various fields, including military applications as well as civilian applications such as agricultural plant protection, power line inspection, geological exploration, environmental monitoring, forest fire prevention, and aerial filming. UAVs include, but are not limited to, unmanned helicopters, unmanned fixed-wing aircraft, unmanned multi-rotor aircraft, unmanned airships, and unmanned paragliders.

[0033] Figure 2 An example structure of a drone is shown below; please refer to... Figure 2 The drone 100 includes a fuselage 120, an arm 130 connected to the fuselage 120, a power unit 140 located on the arm 130, and a camera unit 110 located on the fuselage. The power unit 140 may include a motor and a propeller connected to the motor. The rotation of the motor shaft drives the propeller to rotate, thereby providing lift to the drone.

[0034] Please refer to Figure 1 The drone 100 may also include a gimbal 150, and a camera device 110 is mounted on the fuselage 120 via the gimbal 150. The gimbal 150 is used to reduce or even eliminate the vibration transmitted from the power unit 140 to the camera device 110, so as to ensure that the camera device 110 can capture stable and clear images or videos.

[0035] The processor 11 connects various parts of the entire camera device 13 using various interfaces and lines. By running or executing software programs stored in the memory 12 and calling data stored in the memory 12, it performs various functions of the imaging device and processes data, such as implementing the image calibration method described in any embodiment of this application.

[0036] The memory 12, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer-executable program instructions. The memory 12 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the imaging device, etc.

[0037] Furthermore, memory 12 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 12 may optionally include memory remotely located relative to processor 11, and this remote memory may be connected to camera device 13 via a network.

[0038] Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0039] The camera device 13 can take pictures and record videos. The components constituting the camera device 13 include a lens, a first imaging device, and a second imaging device, which can specifically be image sensors (also called sensors). The lens is used to transmit light; the sensor is used to convert the light signals from the lens into electrical signals and record them as raw images.

[0040] The function of a lens is to project the light image of the observed target onto the sensor of the imaging device 13. A lens combines various optical components (mirrors, transmissive mirrors, prisms) of different shapes and media (plastic, glass, or crystal) in a specific way, allowing light to pass through these components and, after transmission or reflection, change its direction of propagation according to human needs before being received by the receiving device, thus completing the optical imaging process. Generally, a lens is composed of multiple sets of lenses with different curvatures arranged at different intervals. The selection of spacing, lens curvature, and light transmittance determines the focal length of the lens. The main parameters of a lens include: effective focal length, aperture, maximum image plane, field of view, distortion, and relative illumination; the values ​​of each parameter determine the overall performance of the lens.

[0041] A sensor is a device that converts optical images into electronic signals and is widely used in imaging devices and other electro-optical equipment. Common sensors include charge-coupled devices (CCDs) and complementary metal-oxide-semiconductor (CMOS) sensors. Both CCDs and CMOS sensors have a large number (e.g., tens of millions) of photodiodes, each called a photosensitive element, and each photosensitive element corresponds to a pixel. During exposure, after receiving light, the photodiode converts the light signal into an electrical signal containing brightness (or brightness and color), and the image is thus reconstructed. Bayer arrays are a common image sensor technology. Bayer arrays use Bayer color filters to allow different pixels to perceive one of the three primary colors of light: red, blue, and green. These pixels are interleaved, and then a demosaicing interpolation algorithm is used to obtain the original image. Bayer arrays can be applied to CCDs or CMOS sensors, and sensors using Bayer arrays are called Bayer sensors. In addition to Bayer sensors, there are other sensor technologies such as X3 (developed by Foveon). X3 technology uses a three-layer photosensitive element, with each layer recording one color channel of RGB, thus enabling an image sensor to capture all colors on a single pixel.

[0042] In some cases, the imaging device 13 may include different sensors, such as a dual-lens dual-sensor imaging device 13 and a single-lens dual-sensor imaging device 13. These sensors are used to sense multiple spectra and image the same subject. When the number of lenses is less than the number of sensors, a beam splitter may also be provided between the lenses and sensors to decompose the light entering from one lens onto multiple sensors, ensuring that each sensor receives light.

[0043] When using a dual-lens dual-sensor imaging device or a single-lens dual-sensor imaging device to capture images (or record videos), in the dual-sensor configuration, one sensor is a color sensor used to sense visible light and generate a color image; the other sensor is a monochrome sensor used to sense infrared light. Alternatively, in the dual-sensor configuration, one infrared sensor is used to sense infrared light, and the other is a low-light sensor used to sense low-light conditions. The processor aligns the first and second images generated by the sensors using calibration values, and then fuses the two aligned images together to generate a fused image. The fused image has a better effect than the single image before fusion.

[0044] In some cases, the first imaging device and the second imaging device acquire a first image and a second image simultaneously. The first image and the second image are two images with different spectra, but they depict the same target being photographed; therefore, there is a correspondence between the pixels of the first image and the second image. In this embodiment, the purpose of image calibration is to obtain the calibration values ​​of corresponding pixels in the two captured images as accurately as possible. Adjusting the images based on the calibration values ​​can reduce the deviation between the images, so that corresponding pixels are merged together as much as possible during fusion. The calibration values ​​in this embodiment include a second transformation parameter.

[0045] Since the first image and the second image depict the same photographed object (i.e., the target image), the pixels of the two images being merged have a corresponding relationship. During merging, if there is a large deviation between the two images, causing corresponding pixels to not merge together, it will affect the merging effect. Before the electronic device leaves the factory, the factory may calibrate the first image and the second image based on an initial calibration distance to obtain a first transformation parameter. The first transformation parameter is used to represent the spatial transformation relationship when the first image and the second image are aligned. The initial calibration distance is the distance from the calibration object to the imaging device during the initial calibration. However, in the actual shooting process after the electronic device leaves the factory, the distance from the target image to the imaging device is usually different from the initial calibration distance. Therefore, when using the first transformation parameter to align the actually captured first image and the second image, there will still be a large deviation between the two images. Based on this, this application provides an image calibration method that can obtain the distance from the target image to the imaging device and obtain a second transformation parameter between the first image and the second image acquired at that distance, thereby reducing the deviation between the first image and the second image when aligning them. To facilitate the reader's understanding of this application, specific embodiments are described below.

[0046] Please see Figure 3 This application provides an image calibration method for use in electronic devices, such as... Figure 3 As shown, the method includes the following steps:

[0047] Step 31: Obtain the first transformation parameter, which is used to characterize the spatial transformation relationship between the first image and the second image acquired at the initial calibration distance. The first transformation parameter includes the first displacement parameter.

[0048] Before electronic devices leave the factory, factory technicians perform initial calibration on the first and second images acquired by the first and second imaging devices to obtain first transformation parameters. These first transformation parameters characterize the spatial transformation relationship between the first and second images acquired at the initial calibration distance. Specifically, the first transformation parameters are used to align each pixel of the first and second images to the same coordinate system.

[0049] Please see Figure 4 The initial calibration process is as follows: The first imaging device 401 and the second imaging device 402 of the electronic device 400 take images at a certain distance L0 (e.g., 6 meters) from the calibration object a, respectively, to obtain a first image and a second image. The electronic device 400 extracts feature points from the first image and the second image respectively, and performs feature point matching to obtain multiple matching feature point pairs. Then, based on the horizontal and vertical coordinate values ​​of each matching feature point pair, the electronic device calculates the parameter values ​​m0-m7 of the first transformation parameters of the initial projection matrix H0. Specifically, the electronic device can extract feature points such as edges and contours from the first image and the second image, and perform feature point matching to obtain each matching feature point pair. Among them, the Canny edge detection algorithm can be used to detect feature points such as edges and contours in the image. In addition, other feature point matching methods in the prior art can also be used for feature point matching.

[0050] During the initial calibration process, the distance between the first imaging device and the calibration object and the calibration object is the initial calibration distance. The initial calibration distance can also be any other suitable distance, such as 5 meters, 7 meters, or 8 meters. In some embodiments, the initial projection matrix H0 is specifically as follows:

[0051]

[0052] Where m0, m1, m3, and m4 represent the scaling and rotation of the image; m2 and m5 are the first displacement parameters, and m2 and m5 represent the displacement of the image in the horizontal and vertical directions, respectively; m6 and m7 represent the deformation of the image in the horizontal and vertical directions, respectively; m8 is a weighting factor, which is always 1 under normalization conditions.

[0053] Step 32: Obtain the distance from the target to the imaging device;

[0054] In this embodiment, when the imaging device captures a first image and a second image of the target, the distance between the target and the imaging device can also be referred to as the object distance or the real-world distance. In practical applications, there are various methods for obtaining the object distance, such as laser ranging or radar ranging, but these methods require additional expensive equipment, such as laser equipment and radar equipment.

[0055] To reduce the cost and simplify the image calibration process, embodiments of this application can determine the object distance based on the phase difference during phase-detection autofocus. For example, when an electronic device uses phase detection autofocus (PDAF) for focusing, it can acquire the phase difference (PD) detected at the current position. Since there is a mapping relationship between the phase difference and the code value (i.e., image distance), a code value corresponding to a phase difference can be obtained. The mapping relationship can be presented as a mapping table or formula. For example, the electronic device can acquire a preset PD-code value mapping table, and then find the image distance in the PD-code value mapping table using a known PD value in practical applications. The image distance (code value) is an important parameter in a camera device, and the electronic device can adjust the lens based on the image distance to obtain a clearer image.

[0056] Specifically, the electronic device can also acquire the mapping relationship between image distance and object distance. In some embodiments, the mapping relationship between image distance and object distance can be a mathematical formula or a mapping table, as shown in Table 1. Referring to Table 1, each object distance value in the mapping table has a corresponding image distance value. After acquiring an image distance, a real-world distance can be obtained by looking up the corresponding table.

[0057] Table 1

[0058] code value code1 code2 code3 code4 code5 code6 code7 code8 distance 1 meter 2 meters 4 meters 8 meters 16 meters 32 meters 64 meters 128 meters

[0059] It should be noted that Table 1 only shows a portion of the distances and image distances as examples. In practical applications, more image distances and object distances can be included. The more object distance and image distance values ​​there are, and the smaller the difference between adjacent object distances, the higher the accuracy of the obtained image distance.

[0060] In practical applications, the mapping relationship between image distance and object distance can be pre-defined. For example, the target can be set to be located at a series of distance points, such as 1 meter, 2 meters, 3 meters, etc. At each distance point, the lens is focused, and the code value after the image is in focus is recorded to obtain a calibration table.

[0061] If the code value does not match any code value in the mapping table, but lies between two code values, for example, at the third code value... m and the fourth code valuem Between them, it can be based on the third distance d m Corresponding image distance code m and the fourth distance d n Corresponding image distance code n Calculate the first image distance. For example, the object distance can be calculated using bilinear interpolation.

[0062] Step 33: Based on the distance, the first displacement parameter, and the correlation, obtain the second displacement parameter of the first image and the second image acquired at the distance, wherein the correlation is used to represent the correlation between the second displacement parameter, the first displacement parameter, and the distance;

[0063] In this embodiment, the electronic device can acquire the correlation between the second displacement parameter, the first displacement parameter, and the distance; wherein, this correlation can be pre-calibrated using the first and second images acquired by the calibration device. Figure 1 The electronic devices in the series are electronic devices of the same model with the same structure.

[0064] The aforementioned relationships can be obtained in the following ways:

[0065] Step 310: The calculation device can determine multiple calibration distances; at each calibration distance, a second displacement parameter of the calibration device is obtained; based on the second displacement parameter obtained at each calibration distance and each calibration distance, a first mapping relationship between the second displacement parameter of the calibration device and the calibration distance is obtained;

[0066] The calibration distance is the distance between the calibration equipment and the calibration object. Multiple calibration distances are possible, such as 1 meter, 2 meters, 3 meters...100 meters. The same distance between the calibration equipment and the calibration object is adjusted for each measurement.

[0067] For example: when the distance between the calibration device and the calibration object is adjusted to 1 meter, the calibration device acquires a first image and a second image. During the image calibration process of the first and second images, the second displacement parameter (m) is measured. 21 m 51 ).

[0068] In some embodiments, the method for the computing device to calculate the second displacement parameter specifically involves: the computing device extracting feature points from the first image and the second image acquired by the calibration device, and performing feature point matching to obtain multiple matching feature point pairs. Then, based on the horizontal and vertical coordinate values ​​of each matching feature point pair, the computing device calculates the second displacement parameter (m) between the first image and the second image. 2L m 5L ).

[0069] When the distance between the calibration equipment and the calibration object is adjusted to 2 meters, the measured second displacement parameter is (m). 22 m 52 ).

[0070] Measure again at other calibration distances (e.g., 3 meters, 4 meters, 5 meters... 100 meters) to obtain a greater number of measurement results.

[0071] Based on the above multiple measurement results, the computing device processor obtained the first mapping relationship between the second displacement parameter of the calibration device and the calibration distance through linear fitting, that is, the formula (1) for the change of the second displacement parameter with the object distance.

[0072] m 2L =P+bL①

[0073] m 5L =Q+dL②

[0074] Where L is the object distance; m 2L and m 5L m is the second displacement parameter. 2L Used to represent the horizontal displacement of an image, m 5L Used to represent the displacement of the image in the vertical displacement direction; P, Q, b, and d are known constants.

[0075] Step 320: The calculation device determines the first displacement parameters of multiple calibration devices; at a calibration distance, the second displacement parameters of each of the multiple calibration devices are obtained, where the calibration distance is not equal to the initial calibration distance; based on the first displacement parameters of the multiple calibration devices and the second displacement parameters of each calibration device, a second mapping relationship between the second displacement parameters and the first displacement parameters of the calibration devices is determined.

[0076] For each calibration device, which is of the same model, for example, it could be the first calibration device, the second calibration device, and so on up to the Nth camera, where N is a positive integer. For example, N could be 1, 2, 3, ..., 100. The distance between each calibration device and the calibration object is adjusted to the same distance L. This same distance L is any calibration distance different from the initial calibration distance; that is, L is not equal to L0. Figure 4 As shown, L is greater than L0 or less than L0. For example, when the initial calibration distance is 6 meters, the same distance L can be 8 meters or 9 meters, etc.

[0077] For example, the distance between the first calibration device and the calibration object is adjusted to 8 meters. The first calibration device acquires the first image and the second image, and the calculation device calculates the second displacement parameter (m). 21 ', m 51 ').

[0078] The distance between the second calibration device and the calibration object is adjusted to 8 meters. The second calibration device acquires the first and second images, and the calculation device measures the pixel calibration value (m) using the first and second images. 21 ', m 51 ')

[0079] Adjust the distance between other calibration equipment and the calibration object to 8 meters, and use other calibration equipment to perform measurements again to obtain more measurement results.

[0080] Based on the first displacement parameters of different calibration devices and the above multiple measurement results, a second mapping relationship was obtained through linear fitting (e.g., least squares fitting), that is: the formula (2) for the change of the second displacement parameters of different calibration devices with the first displacement parameters under the same calibration distance; as follows:

[0081] m 2L =am2+E①

[0082] m 5L =cm5+F②

[0083] Where m2 and m5 are the first displacement parameters, m 2L and m 5L The second displacement parameter; m2 and m 2L Used to represent the horizontal displacement of the image, m5 and m 5L Used to represent the displacement of the image in the vertical displacement direction, E, F, a, and c are known constants.

[0084] Step 330: The computing device determines the correlation between the second displacement parameter and the object distance and the first displacement parameter based on the first mapping relationship and the second mapping relationship.

[0085] In some embodiments, the correlation includes a calibration formula. The electronic device can obtain the calibration formula as follows using formulas (1) and (2):

[0086] m 2L =f(m2,L)=am2+bL①

[0087] m 5L =f′(m5,L)=cm5+dL②

[0088] Where m2 and m5 are the first displacement parameters, m 2L and m 5L The second displacement parameter; m2 and m 2L Used to represent the horizontal displacement of an image (i.e., the first image or the second image), m5 and m 5LUsed to represent the displacement of the image (i.e., the first image or the second image) in the vertical displacement direction; L is the distance from the target to the imaging device (i.e., the object distance); a, b, c, and d are constants.

[0089] The process of obtaining the correlation (calibration formula) has been described above. It should be noted that this formula is a simple and easy-to-calculate formula. When higher precision is required for the calculation results, the calibration formula may change. The calibration formulas obtained may differ for the same model of calibration equipment from different manufacturers, or for different models from the same manufacturer. However, it can be seen that the calibration formula is related to the first displacement parameter of the electronic device and the distance from the target to the imaging device. In some embodiments, the units of the first and second displacement parameters can be pixels, and the unit of the object distance can be meters.

[0090] By substituting the first displacement parameter of the electronic device and the object distance when the electronic device acquires the image into the calibration formula, the second displacement parameter of the image captured by the electronic device at the corresponding object distance can be obtained. For different electronic devices, each electronic device only needs to be calibrated once at the initial calibration distance to obtain the first transformation parameter. Through the above correlation and the distance from any target captured by each electronic device to the imaging device, the second transformation parameter of the electronic device at that distance can be obtained.

[0091] Step 34: Obtain a second transformation parameter based on the second displacement parameter and the first transformation parameter. The second transformation parameter is used to characterize the spatial transformation relationship between the first image and the second image acquired at the distance.

[0092] In this embodiment, based on the first transformation parameter and the above calibration formula, the second transformation parameter includes m0, m1, and m... 2L m3, m4, m 5L m6, m7 and m8, where m 2L =am2+bL, m 5L =cm5+dL.

[0093] When the distance from the target to the imaging device is L, the projection matrix H between the first image and the second image acquired by the electronic device is... L Specifically as follows:

[0094]

[0095] Where m0, m1, m3, and m4 represent the scaling and rotation of the image, respectively; m 2L Indicates the horizontal displacement of the image; m 5Lm1 represents the vertical displacement of the image; m6 and m7 represent the horizontal and vertical deformation of the image, respectively. m8 is a weighting factor, which is always 1 under normalization conditions.

[0096] The main characteristics of images acquired at different object distances are changes in the horizontal and vertical displacements of the image. Therefore, H0 and H L The values ​​m0, m1, m3, m4, m6, m7, and m8 are the same, but m 2L Unlike m2, m 5L Unlike m5. Where: m 2L =am2+bL, m 5L =cm5+dL.

[0097] The projection matrix is ​​used to map the first image and the second image into the same coordinate system. For example, an electronic device can obtain a mapped image of the first image by mapping using the calculated projection matrix, and then align the mapped image of the first image with the second image into the same coordinate system; the mapping process is also an image alignment process. In some other embodiments, the mapped image of the second image can also be aligned with the first image.

[0098] In this embodiment, the electronic device can acquire a first transformation parameter during initial calibration, which includes a first displacement parameter. The electronic device can also acquire the distance from the target to the imaging device during actual shooting. Since the first transformation parameter and the distance from the target to the imaging device are known, the electronic device can acquire the second displacement parameters of the first and second images acquired at that distance based on the second displacement parameter, the correlation between the first displacement parameter and the distance, and thus obtain the second transformation parameters at the distance from the target to the imaging device, improving image calibration accuracy. Furthermore, for different electronic devices, only one calibration at the initial calibration distance is needed. Through the aforementioned correlation, the second transformation parameters of each camera at any target distance from the imaging device can be obtained.

[0099] This application also provides a computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 1 One of the processors 11 can enable the above one or more processors to execute the focusing method in any of the above method embodiments, for example, to execute the method steps 31 to 34 in FIG3 described above.

[0100] This application also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by a machine, cause the machine to perform the focusing method described above. For example, performing the above-described... Figure 3 Steps 31 to 34 of the method.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image calibration method characterized by, The method is applied to an electronic device, which includes an imaging device, comprising a first imaging device for acquiring a first image and a second imaging device for acquiring a second image. The method includes: A first transformation parameter is obtained during the initial calibration. The first transformation parameter is used to characterize the spatial transformation relationship between the first image and the second image acquired at the initial calibration distance. The first transformation parameter includes a first displacement parameter. Obtain the distance from the target to the imaging device; A second displacement parameter is obtained based on the distance, the first displacement parameter, and the correlation relationship. The second displacement parameter is used to represent the displacement transformation between the first image and the second image acquired at the distance. The correlation relationship is used to represent the correlation between the second displacement parameter, the first displacement parameter, and the distance. A second transformation parameter is obtained based on the second displacement parameter and the first transformation parameter. The second transformation parameter is used to characterize the spatial transformation relationship between the first image and the second image acquired at the distance. In the aforementioned relationship, when the first displacement parameter remains unchanged, the second displacement parameter and the distance have a first mapping relationship; The first mapping relationship is obtained by linearly fitting data consisting of multiple calibration distances and a second displacement parameter of a calibration device at each of the multiple calibration distances. The calibration device is the same model as the electronic device. The calibration distance is the distance from the calibration object to the imaging device. In the aforementioned correlation, when the distance remains constant, the second displacement parameter and the first displacement parameter have a second mapping relationship; The second mapping relationship is obtained by linearly fitting data composed of the first displacement parameters of multiple calibration devices and the second displacement parameters of each of the multiple calibration devices at the same calibration distance.

2. The method of claim 1, wherein, The first displacement parameter includes a parameter for representing horizontal displacement of the image and a parameter for representing vertical displacement of the image ; The second displacement parameter includes a parameter for representing horizontal displacement of the image and a parameter for representing vertical displacement of the image ; The first mapping relationship includes: The mapping relationship between the distance and the distance, and, The mapping relationship between the distance and the distance; The second mapping relationship includes: The mapping relationship between and The mapping relationship between and 3. The method according to claim 1 or 2, characterized in that, Obtaining the first transformation parameter comprises: obtaining the first transformation parameter in an initial projection matrix wherein: Where m0, m1, m3, and m4 represent the scaling and rotation of the image, and the first displacement parameter includes... and , Used to represent the horizontal displacement of an image. m6 and m7 are used to represent the vertical displacement of the image, respectively, and m8 is a weighting factor that is always 1 under normalization conditions.

4. The method of claim 3, wherein, After obtaining the second transformation parameter based on the second displacement parameter and the first transformation parameter, the method further includes: acquiring a projection matrix between the first image and the second image down-sampled from the distance wherein: Wherein, the second displacement parameter includes and ,and Used to represent the horizontal displacement of an image. Used to indicate the vertical displacement of the image; and The values ​​m0, m1, m3, m4, m6, m7, and m8 are the same.

5. The method of claim 1, wherein, The process of obtaining the distance from the target to the imaging device includes: The phase difference generated during phase focusing in the imaging device is obtained; The distance from the target to the imaging device is determined based on the phase difference.

6. The method of claim 5, wherein, Determining the distance from the target to the imaging device based on the phase difference includes: Determine the code value corresponding to the phase difference, and the phase difference and the code value have a mapping relationship; The distance from the target to the imaging device is determined based on the code value.

7. An electronic device, comprising: include: A processor and a memory storing instructions, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 6.

8. The electronic device of claim 7, wherein, The electronic devices include drones.

Images