Camera parameter calibration method and device, electronic equipment, storage medium and product

Abstract

The invention relates to a camera parameter calibration method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: the electronic equipment comprises a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is smaller than or equal to the preset resolution threshold; the electronic equipment firstly shoots a target calibration board set through a first camera and a second camera to obtain a first image set and a second image set. Wherein the target calibration board set comprises a plurality of calibration boards, and the positions of the feature points on the calibration boards are different. Secondly, acquiring coordinates of feature points on each first image in the first image set, and acquiring coordinates of feature points on each second image in the second image set by adopting a preset pixel method; therefore, the target extrinsic parameters between the first camera and the second camera can be accurately obtained according to the coordinates of the feature points on the first images and the coordinates of the feature points on the second images.

Description

Technical Field

[0001] The present application relates to the field of camera technology, and in particular to a camera parameter calibration method, device, electronic device, storage medium, and product. Background Art

[0002] With the rapid development of camera-related technologies, the images captured by cameras are becoming clearer and clearer. The camera parameter calibration process is one of the key links in determining whether the camera can capture clear images.

[0003] Traditional methods generally use Zhang's calibration method to calibrate the intrinsic and extrinsic parameters of the camera. However, the traditional Zhang's calibration method can accurately calibrate the parameters of high-resolution cameras, but in the process of using the traditional Zhang's calibration method to calibrate the parameters of low-resolution cameras, the accuracy is very low, and it is even impossible to calibrate the parameters of low-resolution cameras. Summary of the invention

[0004] The embodiments of the present application provide a camera parameter calibration method, device, electronic device, and computer-readable storage medium, which can accurately obtain the external parameters between a high-resolution camera and a low-resolution camera.

[0005] In one aspect, a camera parameter calibration method is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the method includes:

[0006] The target calibration plate set is photographed by the first camera and the second camera respectively to obtain a first image set and a second image set; the target calibration plate set includes a plurality of calibration plates, and the positions of the feature points on each calibration plate are different;

[0007] Acquire the coordinates of the feature points on each first image in the first image set, and acquire the coordinates of the feature points on each second image in the second image set by using a preset pixel method;

[0008] The target extrinsic parameters between the first camera and the second camera are obtained according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the extrinsic parameters between the target calibration plate set and the first camera, and the initial extrinsic parameters between the first camera and the second camera.

[0009] On the other hand, a camera parameter calibration device is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the device includes:

[0010] An image set acquisition module, used to photograph a target calibration plate set through the first camera and the second camera respectively to obtain a first image set and a second image set; the target calibration plate set includes a plurality of calibration plates, and the positions of feature points on each calibration plate are different;

[0011] a coordinate acquisition module, used to acquire the coordinates of the feature points on each first image in the first image set, and to acquire the coordinates of the feature points on each second image in the second image set by using a preset pixel method;

[0012] The external parameter acquisition module is used to obtain the target external parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera.

[0013] On the other hand, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the camera parameter calibration method as described above.

[0014] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the camera parameter calibration method described above are implemented.

[0015] On the other hand, a computer program product is provided, comprising a computer program, which implements the steps of the camera parameter calibration method as described above when executed by a processor.

[0016] The camera parameter calibration method, device, electronic device, storage medium, and product, the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the electronic device firstly shoots the target calibration plate set through the first camera and the second camera respectively to obtain a first image set and a second image set. Among them, the target calibration plate set includes multiple calibration plates, and the positions of the feature points on each calibration plate are different. Secondly, the coordinates of the feature points on each first image in the first image set are obtained, and the coordinates of the feature points on each second image in the second image set are obtained by using a preset pixel method. Although the resolution of each second image is low, the pixel information of the preset pattern can be accurately obtained by the preset pixel method, and then, the coordinates of the feature points on each second image in the second image set can be accurately obtained based on the pixel information of the preset pattern. Then, according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera, the target external parameters between the first camera and the second camera can be accurately obtained. That is, the external parameters between the high-resolution camera and the low-resolution camera are accurately obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A diagram showing an application environment of a camera parameter calibration method in one embodiment;

[0019] Figure 2 is a flowchart of a camera parameter calibration method in one embodiment;

[0020] Figure 3 is a schematic diagram of the positional relationship between a first camera, a second camera, and a display device in an electronic device in one embodiment;

[0021] Figure 4 is a schematic diagram of multiple calibration plates in one embodiment;

[0022] Figure 5 for Figure 2 A flow chart of a method for obtaining the center coordinates of a preset circular bright spot on the second image according to a target pixel area on the second image and a preset pixel method;

[0023] Figure 6A schematic diagram of a second image obtained by photographing a calibration plate in a target calibration plate set by a second camera in an embodiment;

[0024] Fig. 7A It is a flowchart of a method for obtaining the coordinates of the center of a preset circular bright spot on a second image according to the coordinates of a pixel dividing line, the sum of the pixel values ​​of pixels in a first pixel area, and the sum of the pixel values ​​of pixels in a second pixel area in one embodiment;

[0025] Figure 7B is a schematic diagram of a virtual circle in one embodiment;

[0026] Figure 8 It is a schematic diagram of obtaining the coordinates of feature points on a first image taken by a first camera in one embodiment;

[0027] Fig. 9 for Figure 2 A flowchart of a method for obtaining target external parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera;

[0028] Fig.10 is a schematic diagram of multiple calibration plates in another embodiment;

[0029] Fig.11 is a schematic diagram of a camera parameter calibration method in an exemplary embodiment;

[0030] Fig.12 is a structural block diagram of a camera parameter calibration device in one embodiment;

[0031] Fig.13 for Fig.12 The structural block diagram of the coordinate acquisition module;

[0032] Fig.14 FIG. 1 is a schematic diagram of the internal structure of an electronic device in an embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first camera may be referred to as a second camera, and similarly, a second camera may be referred to as a first camera. Both the first camera and the second camera are cameras, but they are not the same camera.

[0035] With the rapid development of camera-related technologies, the images captured by cameras are becoming clearer and clearer. The camera parameter calibration process is one of the key links in determining whether the camera can capture clear images.

[0036] Traditional methods generally use Zhang's calibration method to calibrate the intrinsic and extrinsic parameters of the camera. However, the traditional Zhang's calibration method can accurately calibrate the parameters of high-resolution cameras, but in the process of using the traditional Zhang's calibration method to calibrate the parameters of low-resolution cameras, the accuracy is very low, and it is even impossible to calibrate the parameters of low-resolution cameras. Therefore, how to accurately calibrate the parameters of low-resolution cameras is an urgent problem to be solved.

[0037] Figure 1 FIG. 1 is a schematic diagram of an application environment of a camera parameter calibration method in an embodiment. Figure 1 As shown, the application environment includes an electronic device 120, and the electronic device 120 includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold. The electronic device 120 uses the first camera and the second camera to shoot the target calibration plate set respectively to obtain a first image set and a second image set; the target calibration plate set includes multiple calibration plates, and the positions of the feature points on each calibration plate are different; the coordinates of the feature points on each first image in the first image set are obtained, and the coordinates of the feature points on each second image in the second image set are obtained by using a preset pixel method; according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera, the target external parameters between the first camera and the second camera are obtained. Among them, the electronic device 120 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, smart cars, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc.

[0038] Figure 2 FIG. 1 is a flowchart of a camera parameter calibration method in an embodiment. The camera parameter calibration method in this embodiment is executed on Figure 1The electronic device in the example is described, the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold. Figure 2 As shown, the camera parameter calibration method includes steps 220 to 260, wherein:

[0039] Step 220 , photographing the target calibration plate set by the first camera and the second camera respectively to obtain a first image set and a second image set; the target calibration plate set includes a plurality of calibration plates, and the positions of the feature points on each calibration plate are different.

[0040] The resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold. Here, the preset resolution threshold can be any threshold less than or equal to 10*10, and this application does not limit this. In other words, the first camera is a high-resolution camera, and the second camera is a low-resolution camera.

[0041] Among them, the calibration plate is a flat plate containing a pattern for camera calibration. Here, the target calibration plate set includes multiple calibration plates, and the positions of the feature points on each calibration plate are different. The feature points on the calibration plate may refer to the marking points on the calibration plate. For example, the marking points on the calibration plate may include feature points preset on the calibration plate, and these preset feature points may be feature points of a preset pattern on the calibration plate. Here, the positions of the feature points on each calibration plate are different, that is, the same feature points are preset on each calibration plate, but the positions of the feature points on different calibration plates are different.

[0042] Combination Figure 3 FIG. 1 is a schematic diagram of the positional relationship between the first camera, the second camera, and the display device in an electronic device in an embodiment. The display device may be a display or an LED array, which is not limited in the present application. Multiple calibration plate images may be sequentially displayed on the display device to obtain multiple calibration plates, that is, to obtain a target calibration plate set, and the positions of the feature points on each calibration plate image are different.

[0043] The electronic device may use two cameras to cyclically photograph each calibration plate in the target calibration plate set to obtain a first image set and a second image set. Exemplarily, the target calibration plate set is photographed by the first camera to obtain a first image set, and the target calibration plate set is photographed by the second camera to obtain a second image set. Exemplarily, the electronic device uses the first camera to photograph each calibration plate in the target calibration plate set to obtain a first image set, and uses the second camera to photograph each calibration plate in the target calibration plate set to obtain a second image set. For each first image in the first image set, there is a second image corresponding to the first image in the second image set, and the group of first images and second images are all obtained by photographing the same calibration image.

[0044] Step 240 , obtaining the coordinates of the feature points on each first image in the first image set, and obtaining the coordinates of the feature points on each second image in the second image set using a preset pixel method.

[0045] Since the first image set is an image set obtained by photographing each calibration plate in the target calibration plate set with the first camera (high-resolution camera), the Zhang calibration method can be used to accurately obtain the coordinates of the feature points on each first image in the first image set. The second image set is an image set obtained by photographing each calibration plate in the target calibration plate set with the second camera (low-resolution camera). Since the resolution of each second image in the second image set is less than or equal to the preset resolution threshold, the Zhang calibration method cannot accurately obtain the coordinates of the feature points on each second image in the second image set.

[0046] Then, the preset pixel method can be used to obtain the coordinates of the feature points on each second image in the second image set. Although the resolution of each second image in the second image set is less than or equal to the preset resolution threshold, the preset pixel method can still accurately obtain the pixel features on each second image, and then accurately obtain the coordinates of the feature points on each second image in the second image set.

[0047] Assuming that the preset feature points on the calibration plate can be the feature points of the preset pattern on the calibration plate, although the resolution of each second image is low, the pixel information of the preset pattern can be accurately obtained through the preset pixel method, and then, the coordinates of the feature points on each second image in the second image set can be accurately obtained based on the pixel information of the preset pattern.

[0048] Step 260, obtaining target extrinsic parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the extrinsic parameters between the target calibration plate set and the first camera, and the initial extrinsic parameters between the first camera and the second camera.

[0049] In the process of calculating the target external parameters between the first camera and the second camera, the external parameters between the target calibration plate set and the first camera and the initial external parameters between the first camera and the second camera are first obtained. Figure 3 As shown, the external parameters between the target calibration plate set and the first camera refer to the external parameters between the display device coordinate system where the display device is located and the first camera coordinate system. Here, the external parameters are used to determine the relative position relationship between the display device coordinate system and the first camera coordinate system, including the rotation vector R1 and the translation vector T1.

[0050] Here, the initial external parameters between the first camera and the second camera may be external parameters preset based on empirical values, which is not limited in the present application. Similarly, the initial external parameters between the first camera and the second camera include a rotation vector Rd and a translation vector Td.

[0051] Then, for each group of first images and second images in the first image set and the second image set, the predicted coordinates of the feature points on each second image are calculated according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera.

[0052] Finally, the reprojection error between the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image is calculated, and then the target extrinsic parameters between the first camera and the second camera can be obtained.

[0053] In an embodiment of the present application, the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the electronic device firstly shoots the target calibration plate set through the first camera and the second camera respectively to obtain a first image set and a second image set. Among them, the target calibration plate set includes multiple calibration plates, and the positions of the feature points on each calibration plate are different. Secondly, the coordinates of the feature points on each first image in the first image set are obtained, and the coordinates of the feature points on each second image in the second image set are obtained by using a preset pixel method. Although the resolution of each second image is low, the pixel information of the preset pattern can be accurately obtained by the preset pixel method, and then, the coordinates of the feature points on each second image in the second image set can be accurately obtained based on the pixel information of the preset pattern. Then, according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera, the target external parameters between the first camera and the second camera can be accurately obtained. That is, the external parameters between the high-resolution camera and the low-resolution camera are accurately obtained.

[0054] In the previous embodiment, a process of obtaining the coordinates of feature points on each second image in the second image set by using a preset pixel method, and then obtaining the target external parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera is described. In this embodiment, it is further described that the calibration plate includes a preset circular bright spot, and the center of the preset circular bright spot is the feature point on the calibration plate; step 240, obtaining the coordinates of the feature points on each second image in the second image set by using the preset pixel method, includes:

[0055] For each second image in the second image set, according to the target pixel area on the second image and the preset pixel method, the center coordinates of the preset circular bright spot on the second image are obtained; the target pixel area is the area on the second image corresponding to the preset circular bright spot on the calibration plate corresponding to the second image;

[0056] The coordinates of the center of the preset circular bright spot on each second image are used as the coordinates of the feature point on each second image.

[0057] like Figure 4 FIG. 1 is a schematic diagram of multiple calibration plates in one embodiment. These multiple calibration plates can be obtained by displaying multiple calibration plate images on a display device. The calibration plate includes a preset circular bright spot, and the center of the preset circular bright spot is a characteristic point on the calibration plate. Figure 4 As shown, the preset circular bright spots on different calibration plates are at different positions, that is, the centers of the preset circular bright spots on different calibration plates (characteristic points on the calibration plates) are at different positions.

[0058] For each second image in the second image set, first, a target pixel area on the second image is obtained, that is, pixels whose pixel values ​​are not (0,0,0) are obtained from the second image, and these pixel points are the areas corresponding to the preset circular bright spots on the calibration plate corresponding to the second image, and these pixel points constitute the target pixel area.

[0059] Then, according to the target pixel area on the second image and the preset pixel method, the center coordinates of the preset circular bright spot on the second image are obtained. Exemplarily, according to the pixel value size relationship of the pixels in the target pixel area on the second image, the pixel dividing line of the target pixel area is obtained. Then, the target pixel area is divided according to the pixel dividing line to obtain a first pixel area and a second pixel area. Based on the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area, as the area size of the first part of the virtual circle and the area size of the second part, a virtual circle is constructed. Then, based on the virtual circle, the center coordinates of the preset circular bright spot on the second image are obtained.

[0060] In the embodiment of the present application, when the electronic device uses the preset pixel method to obtain the coordinates of the feature points on each second image in the second image set, first, for each second image in the second image set, according to the target pixel area on the second image and the preset pixel method, the center coordinates of the preset circular bright spot on the second image are obtained. The target pixel area is the area on the second image corresponding to the preset circular bright spot on the calibration plate corresponding to the second image. The center coordinates of the preset circular bright spot on each second image are used as the coordinates of the feature points on each second image.

[0061] In the process of obtaining the center coordinates of the preset circular bright spot on the second image according to the target pixel area on the second image and the preset pixel method, exemplarily, according to the pixel value size relationship of the pixels in the target pixel area on the second image, the pixel dividing line of the target pixel area is obtained. Then the target pixel area is divided according to the pixel dividing line to obtain the first pixel area and the second pixel area. Based on the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area, the area size of the first part of the virtual circle and the area size of the second part are used to construct the virtual circle. Since the resolution of each second image is less than or equal to the preset resolution threshold, the center coordinates of the preset circular bright spot on the second image cannot be clearly determined. Therefore, according to the pixel value size relationship of the pixels in the target pixel area on the second image, the pixel dividing line of the target pixel area is obtained. Then the target pixel area is divided according to the pixel dividing line to obtain the first pixel area and the second pixel area. Based on the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area, the area size of the first part of the virtual circle and the area size of the second part are used to construct the virtual circle. In this way, the method of directly determining the center coordinates of the preset circular bright spot on the second image is converted into the method of determining the center coordinates from the virtual circle. Here, the dividing line and the area size on the virtual circle are both accurate values, so the center coordinates of the preset circular bright spot on the second image can be accurately obtained based on the virtual circle.

[0062] In the previous embodiment, it is described that when a preset circular bright spot is included on the calibration plate and the center of the preset circular bright spot is a feature point on the calibration plate, for each second image in the second image set, according to the target pixel area on the second image and the preset pixel method, the center coordinates of the preset circular bright spot on the second image are obtained, and the center coordinates are the coordinates of the feature point on each second image. In this embodiment, it is further described that Figure 5 As shown, according to the target pixel area on the second image and the preset pixel method, the center coordinates of the preset circular bright spot on the second image are obtained, including:

[0063] Step 520, obtaining a pixel segmentation line of the target pixel area according to the pixel value relationship of the pixels in the target pixel area.

[0064] like Figure 6 FIG. 1 is a schematic diagram of a second image obtained by photographing a calibration plate in a target calibration plate set by a second camera in an embodiment. It can be seen that Figure 6 The resolution of the second image shown in is 6*8, which is obviously less than or equal to 10*10, so the second image is a low-resolution image. For the low-resolution second image, the center coordinates of the preset circular bright spot on the second image cannot be accurately obtained by binarization and contour fitting methods. Since the preset pixel method can accurately obtain the pixel information of the preset pattern, the center coordinates of the preset circular bright spot on the second image can be obtained according to the target pixel area on the second image and the preset pixel method.

[0065] For example, the electronic device can obtain the pixel segmentation line of the target pixel area according to the pixel value size relationship of the pixels in the target pixel area. Figure 6 620 is the target pixel region on the second image, and the target pixel region includes pixels whose pixel values ​​are not 0. Then, based on the size of the pixel values ​​of each pixel in the target pixel region, the pixel value size relationship of the pixels in the target pixel region is determined. Next, the pixel segmentation line of the target pixel region is obtained according to the pixel value size relationship, and the pixel segmentation line is located between two pixels. Here, the position of the maximum pixel value in the target pixel region can be determined based on the pixel value size relationship, and then the pixel segmentation line of the target pixel region is determined based on the position of the maximum pixel value.

[0066] Step 540 , dividing the target pixel region according to the pixel dividing line to obtain a first pixel region and a second pixel region.

[0067] After obtaining the pixel division line of the target pixel area, the target pixel area can be divided based on the pixel division line to obtain the first pixel area and the second pixel area. Here, assuming that the pixel division line is a division line 640 in the first direction, the target pixel area can be divided according to the first direction based on the pixel division line to obtain the first pixel area and the second pixel area in the first direction; assuming that the pixel division line is a division line 660 in the second direction, the target pixel area can be divided according to the second direction based on the pixel division line to obtain the first pixel area and the second pixel area in the second direction. Among them, the first direction and the second direction are perpendicular to each other. Of course, the direction of the pixel division line is not limited here.

[0068] Step 560, obtaining the coordinates of the center of the preset circular bright spot on the second image according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area, and the target mapping relationship; the target mapping relationship includes a mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area at the target image size ratio, and the distance between the coordinates of the center of the preset circular bright spot on the second image and the pixel dividing line.

[0069] After obtaining the pixel division line of the target pixel area as described above, and dividing the target pixel area based on the pixel division line to obtain the first pixel area and the second pixel area, the coordinates of the pixel division line, the sum of the pixel values ​​of the pixels in the first pixel area, and the sum of the pixel values ​​of the pixels in the second pixel area can be further obtained. Assuming that the pixel division line is a division line in the first direction, the coordinates of the pixel division line in the second direction can be obtained; assuming that the pixel division line is a division line in the second direction, the coordinates of the pixel division line in the first direction can be obtained. The pixel values ​​of each pixel in the first pixel area and the pixel values ​​of each pixel in the second pixel area can also be obtained.

[0070] Then, according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area, and the target mapping relationship, the coordinates of the center of the preset circular bright spot on the second image are obtained. Here, the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area can be calculated based on the pixel values ​​of the pixels in the first pixel area.

[0071] The target mapping relationship includes the mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area at the target image size ratio, and the distance between the center coordinates of the preset circular bright spot on the second image and the pixel dividing line. In other words, based on the target mapping relationship, the mapping relationship between the sum of the pixel values ​​of each pixel in the first pixel area and the sum of the pixel values ​​of each pixel in the second pixel area and the distance between the center coordinates of the preset circular bright spot on the second image and the pixel dividing line can be obtained.

[0072] Therefore, according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of each pixel in the first pixel area and the sum of the pixel values ​​of each pixel in the second pixel area combined with the target mapping relationship, the distance between the center coordinates of the preset circular bright spot on the second image and the pixel dividing line can be calculated. Furthermore, the center coordinates of the preset circular bright spot on the second image can be obtained in combination with the coordinates of the pixel dividing line.

[0073] In an embodiment of the present application, the electronic device uses a preset pixel method to obtain a pixel dividing line of the target pixel area according to the pixel value size relationship of the pixels in the target pixel area. Then, the target pixel area is divided based on the pixel dividing line to obtain a first pixel area and a second pixel area. Among them, the target mapping relationship includes the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area under the target image size ratio, and the mapping relationship between the center coordinates of the preset circular bright spot on the second image and the distance between the pixel dividing line. Therefore, according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area and the target mapping relationship, the distance between the center coordinates of the preset circular bright spot on the second image and the pixel dividing line can be calculated. Furthermore, the center coordinates of the preset circular bright spot on the second image can be obtained in combination with the coordinates of the pixel dividing line. In this way, the preset pixel method combined with the target mapping relationship can accurately calculate the center coordinates of the preset circular bright spot on each second image.

[0074] In the previous embodiment, a process is described in which the preset pixel method combined with the target mapping relationship can accurately calculate the coordinates of the center of the preset circular bright spot on each second image. In this embodiment, it is further described that the pixel segmentation line includes a first pixel segmentation line in a first direction and a second pixel segmentation line in a second direction, and the first direction and the second direction are perpendicular to each other; according to the pixel value relationship of the pixels in the target pixel area, the pixel segmentation line of the target pixel area is obtained, including:

[0075] According to the pixel value size relationship of pixels in the target pixel area in the first direction, a first pixel dividing line in the target pixel area in the first direction is obtained; the pixel values ​​of pixels on one side of the first pixel dividing line in the target pixel area are greater than the pixel values ​​of pixels on the other side of the first pixel dividing line;

[0076] According to the relationship between the pixel values ​​of pixels in the target pixel area in the second direction, a second pixel dividing line in the target pixel area in the second direction is obtained; the pixel values ​​of the pixels on one side of the second pixel dividing line in the target pixel area are greater than the pixel values ​​of the pixels on the other side of the second pixel dividing line.

[0077] Combination Figure 6As shown, the first direction here can be the X-axis direction, and the second direction can be the Y-axis direction. Then, when obtaining the first pixel segmentation line of the target pixel area in the X-axis direction, the first pixel segmentation line in the X-axis direction can be determined according to the pixel value size relationship of the pixels in the target pixel area in the first direction. Among them, the pixel value of the pixel on one side of the first pixel segmentation line in the target pixel area is greater than the pixel value of the pixel on the other side of the first pixel segmentation line. Optionally, the pixel A corresponding to the maximum pixel value can be obtained from the target pixel area first, and then it is determined whether the pixel value of the pixel on the left side of the pixel A in the X-axis direction is greater than the pixel value of the pixel on the right side of the pixel A in the X-axis direction. If so, the left boundary of the pixel A is used as the pixel segmentation line in the X-axis direction. If not, the right boundary of the pixel A is used as the first pixel segmentation line in the X-axis direction.

[0078] Then, when obtaining the second pixel segmentation line of the target pixel area in the Y-axis direction, the second pixel segmentation line in the Y-axis direction can be determined according to the pixel value size relationship of the pixels in the target pixel area in the first direction. Wherein, the pixel value of the pixel on one side of the second pixel segmentation line in the target pixel area is greater than the pixel value of the pixel on the other side of the second pixel segmentation line. Optionally, the pixel A corresponding to the maximum pixel value can be obtained from the target pixel area first, and then it is determined whether the pixel value of the pixel on the upper side of the pixel A in the Y-axis direction is greater than the pixel value of the pixel on the lower side of the pixel A in the Y-axis direction. If so, the upper boundary of the pixel A is used as the pixel segmentation line in the Y-axis direction. If not, the lower boundary of the pixel A is used as the second pixel segmentation line in the Y-axis direction.

[0079] In the embodiment of the present application, the electronic device can obtain the first pixel dividing line of the target pixel area in the first direction according to the pixel value size relationship of the pixels in the target pixel area in the first direction. And the pixel value of the pixel on one side of the first pixel dividing line in the target pixel area is greater than the pixel value of the pixel on the other side of the first pixel dividing line.

[0080] And according to the pixel value size relationship of the pixels in the target pixel area in the second direction, the second pixel dividing line of the target pixel area in the second direction is obtained. And the pixel value of the pixel on one side of the second pixel dividing line in the target pixel area is greater than the pixel value of the pixel on the other side of the second pixel dividing line. In this way, the target pixel area can be accurately divided according to the pixel value size through the pixel dividing line.

[0081] In the previous embodiment, the process of obtaining the first pixel segmentation line of the target pixel region in the first direction and the first pixel segmentation line of the target pixel region in the second direction is described. In this embodiment, it is further described that the target pixel region is divided according to the pixel segmentation line to obtain the first pixel region and the second pixel region, including:

[0082] Dividing the target pixel area according to the first pixel dividing line to obtain a first pixel area and a second pixel area in a first direction;

[0083] The target pixel region is divided according to the second pixel dividing line to obtain a first pixel region and a second pixel region in a second direction.

[0084] Combination Figure 6 As shown, after obtaining the first pixel division line of the target pixel region in the X-axis direction and the second pixel division line of the target pixel region in the Y-axis direction, the target pixel region can be divided according to the first pixel division line in the X-axis direction to obtain the first pixel region and the second pixel region in the X-axis direction. The target pixel region can also be divided according to the second pixel division line in the Y-axis direction to obtain the first pixel region and the second pixel region in the Y-axis direction.

[0085] In an embodiment of the present application, after obtaining the first pixel division line of the target pixel area in the X-axis direction and the second pixel division line of the target pixel area in the Y-axis direction, the target pixel area can be divided according to the X-axis direction and the Y-axis direction respectively to obtain the first pixel area and the second pixel area in the X-axis direction and the first pixel area and the second pixel area in the Y-axis direction.

[0086] Furthermore, the first distance from the center of the preset circular bright spot on the second image to the first pixel dividing line can be obtained based on the sum of the pixel values ​​of the pixels in the first pixel area in the X-axis direction and the sum of the pixel values ​​of the pixels in the second pixel area in the X-axis direction in combination with the target mapping relationship. The first distance from the center of the preset circular bright spot on the second image to the second pixel dividing line can be obtained based on the sum of the pixel values ​​of the pixels in the first pixel area in the Y-axis direction and the sum of the pixel values ​​of the pixels in the second pixel area in the Y-axis direction in combination with the target mapping relationship. Combined with the coordinates of the first pixel dividing line and the coordinates of the second pixel dividing line, the coordinates of the center of the preset circular bright spot on the second image can be obtained. In this way, the preset pixel method combined with the target mapping relationship can accurately calculate the coordinates of the center of the preset circular bright spot on each second image.

[0087] In the previous embodiment, the process of dividing the target pixel area based on the first pixel dividing line in the X-axis direction to obtain the first pixel area and the second pixel area in the X-axis direction and dividing the target pixel area based on the second pixel dividing line in the Y-axis direction to obtain the first pixel area and the second pixel area in the Y-axis direction is described. Fig. 7A As shown, according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area, the center coordinates of the preset circular bright spot on the second image are obtained, including:

[0088] Step 720, for the first direction, obtain the first distance from the center of the preset circular bright spot on the second image to the first pixel dividing line according to the sum of the pixel values ​​of the pixels in the first pixel area in the first direction and the sum of the pixel values ​​of the pixels in the second pixel area in the first direction combined with the target mapping relationship.

[0089] The first direction here can be the X-axis direction, and the second direction can be the Y-axis direction. The target mapping relationship includes a first target mapping relationship and a second target mapping relationship. Here, the first target mapping relationship includes the mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area in the first direction and the sum of the pixel values ​​of the pixels in the second pixel area in the first direction and the first distance under the target image size ratio. In other words, based on the first target mapping relationship, the mapping relationship between the sum of the pixel values ​​of each pixel in the first pixel area in the first direction and the sum of the pixel values ​​of each pixel in the second pixel area in the first direction and the distance between the center coordinates of the preset circular bright spot on the second image and the first pixel dividing line can be obtained. Here, the target image size ratio Rpic is the ratio between the size of the preset circular bright spot on the calibration plate and the size of the target pixel area on the second image. The target image size ratio Rpic can be calculated using the following formulas (1) and (2):

[0090]

[0091] R pic =U / a Formula (2)

[0092] Among them, K s is the initial design value of the intrinsic parameter matrix of the first camera (low-resolution camera), R real is the actual radius of the preset circular bright spot on the display device, and dist is the distance from the display device to the first camera and the second camera.

[0093] The target mapping relationship can be expressed as LUT (L, S1 / S2, R pic) is represented by LUT, which is a display lookup table (Look-Up-Table). Among them, L represents the distance from the center of the preset circular bright spot on the second image to the pixel dividing line. The sum of the pixel values ​​of the pixels in the first pixel area can be represented by S1, and S1 is used as the area size of the first part of the virtual circle. The sum of the pixel values ​​of the pixels in the second pixel area can be represented by S2, and S2 is used as the area size of the second part of the virtual circle. Then, a virtual circle can be constructed based on the area size S1 of the first part and the area size S2 of the second part. Exemplarily, with respect to the X-axis direction, the sum of the pixel values ​​of the pixels in the first pixel area in the X-axis direction can be represented by SX1, and SX1 is used as the area size of the first part of the virtual circle. The sum of the pixel values ​​of the pixels in the second pixel area in the X-axis direction can be represented by SX2, and SX2 is used as the area size of the second part of the virtual circle. Then, a virtual circle can be constructed based on the area size SX1 of the first part and the area size SX2 of the second part. Here, the first target mapping relationship can be represented by LUT (L x ,SX1 / SX2,R pic ) is used to represent it. Among them, L x Represents a first distance from the center of a preset circular bright spot on the second image to a first pixel dividing line.

[0094] Then, for the X-axis direction, according to the sum SX1 of the pixel values ​​of the pixels in the first pixel area in the X-axis direction and the sum SX2 of the pixel values ​​of the pixels in the second pixel area in the X-axis direction, combined with the first target mapping relationship LUT (L x ,SX1 / SX2,R pic ), the first distance L from the center of the preset circular bright spot on the second image to the first pixel dividing line can be obtained x .

[0095] Step 740, for the second direction, obtain the second distance from the center of the preset circular bright spot on the second image to the second pixel dividing line based on the sum of the pixel values ​​of the pixels in the first pixel area in the second direction and the sum of the pixel values ​​of the pixels in the second pixel area in the second direction combined with the target mapping relationship.

[0096] For example, for the Y-axis direction, the sum of the pixel values ​​of the pixels in the first pixel area in the Y-axis direction can be represented by SY1, and SY1 is used as the area size of the first part of the virtual circle. The sum of the pixel values ​​of the pixels in the second pixel area in the Y-axis direction can be represented by SY2, and SY2 is used as the area size of the second part of the virtual circle. Then, a virtual circle can be constructed based on the area size SY1 of the first part and the area size SY2 of the second part. Here, the second target mapping relationship can be represented by LUT (L y ,SX1 / SX2,R pic ) is used to represent it. Among them, Ly Represents the second distance from the center of the preset circular bright spot on the second image to the second pixel dividing line.

[0097] Then, for the Y-axis direction, according to the sum of the pixel values ​​SY1 of the pixels in the first pixel area in the Y-axis direction and the sum of the pixel values ​​SY2 of the pixels in the second pixel area in the Y-axis direction, combined with the second target mapping relationship LUT (L y ,SY1 / SY2,R pic ), the second distance L from the center of the preset circular bright spot on the second image to the second pixel dividing line can be obtained y .

[0098] Step 760, obtain the coordinates of the center of the preset circular bright spot on the second image in the first direction according to the coordinates of the first pixel dividing line in the first direction and the first distance, and obtain the coordinates of the center of the preset circular bright spot on the second image in the second direction according to the coordinates of the second pixel dividing line in the second direction and the second distance.

[0099] First, obtain the coordinate X of the first pixel dividing line in the first direction cut , get the first distance L x Since the first distance L x is the distance from the center of the preset circular bright spot on the second image to the first pixel dividing line. Therefore, according to the coordinate X of the first pixel dividing line in the first direction, cut , get the first distance L x , the coordinate X of the center of the preset circular bright spot on the second image in the first direction can be obtained circle For example, the following formula (3) can be used to calculate the coordinate X of the center of the preset circular bright spot on the second image in the first direction: circle :

[0100] X circle =X cut -L x Formula (3)

[0101] Second, obtain the coordinate Y of the second pixel dividing line in the second direction cut , get the second distance L y Since the second distance L y is the distance from the center of the preset circular bright spot on the second image to the second pixel dividing line. Therefore, according to the coordinate Y of the second pixel dividing line in the second direction, cut , get the second distance L y , the coordinate Y of the center of the preset circular bright spot on the second image in the second direction can be obtained circle For example, the following formula (4) can be used to calculate the coordinate Y of the center of the preset circular bright spot on the second image in the second direction:circle :

[0102] Y circle =Y cut -L y Formula (4)

[0103] Step 780, based on the coordinates of the center of the preset circular bright spot in the first direction and the coordinates of the center of the preset circular bright spot in the second direction, obtain the coordinates of the center of the preset circular bright spot on the second image.

[0104] Based on the coordinate X of the center of the preset circular bright spot on the second image in the first direction circle , the coordinate Y of the center of the preset circular bright spot on the second image in the second direction circle , get the center coordinates (X circle , Y circle ).

[0105] In the embodiment of the present application, the target mapping relationship includes the mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area under the target image size ratio, and the coordinates of the center of the preset circular bright spot on the second image and the distance between the pixel dividing line. Therefore, the first distance from the center of the preset circular bright spot on the second image to the first pixel dividing line and the second distance from the center of the preset circular bright spot on the second image to the second pixel dividing line can be calculated with the help of the target mapping relationship. Then, according to the coordinates and the first distance of the first pixel dividing line in the first direction, the coordinates of the center of the preset circular bright spot on the second image in the first direction are obtained, and according to the coordinates and the second distance of the second pixel dividing line in the second direction, the coordinates of the center of the preset circular bright spot on the second image in the second direction are obtained. Finally, the coordinates of the center of the preset circular bright spot on the second image can be accurately obtained. That is, the coordinates of the center of the preset circular bright spot on the second image taken by the low-resolution camera can be accurately obtained.

[0106] In the previous embodiment, a process of obtaining the coordinates of the center of a preset circular bright spot on a second image captured by a second camera is described. In this embodiment, it is further described that the target mapping relationship includes a first target mapping relationship and a second target mapping relationship; the first target mapping relationship includes a mapping relationship between the sum of the pixel values ​​of pixels in a first pixel area in a first direction and the sum of the pixel values ​​of pixels in a second pixel area in the first direction and a first distance under the target image size ratio; the target image size ratio is the ratio between the size of the preset circular bright spot on the calibration plate and the size of the target pixel area on the second image;

[0107] The second target mapping relationship includes a mapping relationship between the sum of pixel values ​​of pixels in the first pixel area in the second direction and the sum of pixel values ​​of pixels in the second pixel area in the second direction and the second distance at the target image size ratio.

[0108] Among them, the target mapping relationship LUT (L, S1 / S2, R pic ) includes a first target mapping relationship LUT (L x ,SX1 / SX2,R pic ) and the second target mapping relationship (L y ,SY1 / SY2,R pic ). For example, the first target mapping relationship LUT (L x ,SX1 / SX2,R pic ) included in the target image size ratio R pic The sum SX1 of the pixel values ​​of the pixels in the first pixel area in the X-axis direction and the sum SX2 of the pixel values ​​of the pixels in the second pixel area in the X-axis direction are equal to the first distance L. x The second target mapping relationship (L y ,SY1 / SY2,R pic ) included in the target image size ratio R pic The sum of the pixel values ​​SY1 of the pixels in the first pixel area in the Y-axis direction and the sum of the pixel values ​​SY2 of the pixels in the second pixel area in the Y-axis direction are equal to the second distance L. y The mapping relationship between them.

[0109] Here, how to obtain the first target mapping relationship LUT (L x ,SX1 / SX2,R pic ) is used as an example for explanation. SX1 / SX2 can also be represented by a ratio Ratio. The first target mapping relationship LUT (L x ,SX1 / SX2,R pic ) can be obtained through a large amount of experimental data, assuming that the target image size ratio R pic is a fixed value, then L x Preset different values ​​and then traverse L x Different values ​​of can be combined Fig. 7A The virtual circle shown in FIG. 1 is used, and the ratio Ratio is calculated by using the following formulas (5)-(8), that is, by calculating SX1 / SX2, the first target mapping relationship LUT (L x ,SX1 / SX2,R pic ).

[0110] α=arccos(Lx / R pic ) Formula (5)

[0111] SX1=R pic 2 (PI-α) Formula (6)

[0112]

[0113] Ratio=SX1 / SX2 Formula (8)

[0114] Where PI is the ratio of circumference to circumference. The actual meaning of each variable in formula (5)-(8) can be referred to Figure 7B As shown in . The radius of the virtual circle can be controlled according to the internal parameters of the second camera (low-resolution camera) so that its radius is greater than 0.5 pixels and less than 1 pixel. Of course, the present application is not limited to this. In this way, each preset circular bright spot can correspond to 4 adjacent pixels when displayed in the low-resolution camera. SX1 represents the sum of the pixel values ​​of the pixels in the first pixel area where the virtual circle is divided along the X-axis direction, and also represents the area of ​​the first part where the virtual circle is divided along the X-axis direction. SX2 represents the sum of the pixel values ​​of the pixels in the second pixel area where the virtual circle is divided along the X-axis direction, and also represents the area of ​​the second part where the virtual circle is divided along the X-axis direction.

[0115] Similarly, referring to the above, the first target mapping relationship LUT (L x ,SX1 / SX2,R pic ) process, the second target mapping relationship (L y ,SY1 / SY2,R pic ), this application will not be repeated here.

[0116] In the embodiment of the present application, a first target mapping relationship LUT (L x ,SX1 / SX2,R pic ), the second target mapping relationship (L y ,SY1 / SY2,R pic ). Then, the accuracy of the first target mapping relationship and the second target mapping relationship are both high. Among them, the target mapping relationship includes the mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area at the target image size ratio, and the distance between the center coordinates of the preset circular bright spot on the second image and the pixel dividing line. Therefore, the center coordinates of the preset circular bright spot on the second image can be accurately obtained based on the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area and the target mapping relationship.

[0117] In the previous embodiment, a process of obtaining the coordinates of the center of a preset circular bright spot on the second image taken by the second camera is described. In this embodiment, obtaining the coordinates of the feature points on each first image in the first image set includes:

[0118] For each first image in the first image set, obtaining a binarized image of the first image;

[0119] Fitting is performed based on the contour of the binary image to obtain the circular equation of the preset circular bright spot;

[0120] Based on the circle equation, the coordinates of the feature points on the first image are obtained.

[0121] like Figure 8 FIG. 1 is a schematic diagram of obtaining coordinates of feature points on a first image captured by a first camera in one embodiment. The calibration plate includes a preset circular bright spot, and the center of the preset circular bright spot is the feature point on the calibration plate. The first image is an image obtained by capturing multiple calibration plates in a target calibration plate set by the first camera, and the resolution of the first camera is greater than a preset resolution threshold, that is, the first camera can be understood as a high-resolution camera.

[0122] Since the first camera is a high-resolution camera, the resolution of the first image is relatively high, and therefore, for each first image 820 in the first image set, the first image 820 can be binarized to obtain a binarized image 840 of the first image. Exemplarily, the Otsu algorithm can be used to binarize the first image 820, and the Otsu algorithm is an algorithm for converting a grayscale image into a binary image. Of course, other binarization processing algorithms can also be used to binarize the first image 820, and this application does not limit this.

[0123] Then, the contour of the area with a pixel value of (255, 255, 255) in the binary image is fitted to obtain the circular equation of the preset circular bright spot. Exemplarily, the canny algorithm can be used to extract the contour points of the area with a pixel value of (255, 255, 255) in the binary image. Based on the coordinates of the contour points, the Hough transform method is used to fit the equation of the circle displayed in the first image (referred to as the circular equation). Furthermore, based on the circular equation, the coordinates of the center of the circle can be obtained.

[0124] In the embodiment of the present application, for each first image in the first image set, since the resolution of the first image is relatively high, a binary image of the first image can be obtained, and fitting is performed based on the contour of the binary image to obtain a circular equation of a preset circular bright spot. Then, based on the circular equation, the coordinates of the feature points on the first image are obtained. For the first image with a relatively high resolution, the binarization processing and contour fitting method can be directly used to accurately obtain the coordinates of the feature points in the first image.

[0125] In the previous embodiment, the process of how to accurately obtain the coordinates of the feature points in the first image and the coordinates of the feature points in the second image is described. In this embodiment, it is further described as follows: Fig. 9 As shown, step 260, according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera, obtains the target external parameters between the first camera and the second camera, including:

[0126] Step 262, for the first image and the second image corresponding to each calibration plate, obtain the predicted coordinates of the feature points on the second image according to the coordinates of the feature points on the first image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera.

[0127] In the process of calculating the target external parameters between the first camera and the second camera, the external parameters between the target calibration plate set and the first camera and the initial external parameters between the first camera and the second camera are first obtained. Figure 3 As shown, the external parameters between the target calibration plate set and the first camera refer to the external parameters between the display device coordinate system where the display device is located and the first camera coordinate system. The pnp algorithm can be used to calculate the external parameters between the display device coordinate system where the display device is located and the first camera coordinate system based on the internal parameters of the first camera and the theoretical 3D space coordinates in the display device coordinate system when the preset circular bright spot is displayed on the calibration plate. Here, the external parameters are used to determine the relative position relationship between the display device coordinate system and the first camera coordinate system, including the rotation vector R1 and the translation vector T1.

[0128] Here, the initial external parameters between the first camera and the second camera may be external parameters preset based on empirical values, which is not limited in the present application. Similarly, the initial external parameters between the first camera and the second camera include a rotation vector Rd and a translation vector Td.

[0129] Then, for each group of first images and second images in the first image set and the second image set, the predicted coordinates of the feature points on each second image are calculated according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera. Exemplarily, the following formula (9) can be used to calculate the predicted coordinates of the feature points on each second image:

[0130]

[0131] in, Represents the center coordinates of the preset circular bright spot on the i-th calibration plate in the target calibration plate set in the display device coordinate system. The rotation vector R1 and the translation vector T1 represent the external parameters between the display device coordinate system and the first camera coordinate system. The rotation vector Rd and the translation vector Td represent the initial external parameters between the first camera and the second camera. K s is the initial design value of the intrinsic parameter matrix of the first camera (low-resolution camera). α represents the coefficient, It represents the center coordinates of the preset circular bright spot on the i-th calibration plate in the target calibration plate set, and the predicted center coordinates on the second image, that is, the predicted coordinates of the feature point on the second image.

[0132] Step 264 , for each second image, according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image, obtain the target extrinsic parameters between the first camera and the second camera.

[0133] After the predicted coordinates of the feature points on the second image are obtained by formula (9), the reprojection error between the coordinates of the feature points on the second image and the predicted coordinates of the feature points on the second image can be calculated, and then the target extrinsic parameters between the first camera and the second camera can be inferred based on the reprojection error. The coordinates of the feature points on the second image are the coordinates of the feature points on the second image obtained by the preset pixel method (X circle , Y circle ). Among them, the coordinates of the feature points on the second image corresponding to the i-th calibration plate are (X circle _i, Y circle _i).

[0134] The reprojection error refers to the difference between the projection of the real 3D space point on the image plane (i.e., the pixel point on the image) and the reprojection (the virtual pixel point calculated based on the camera intrinsic and extrinsic parameters). circle _i, Y circle _i) is the projection of the real three-dimensional space point on the image plane, and the predicted coordinates of the second image feature point (U i , V i) is the reprojection of the real 3D space point onto the image plane.

[0135] In the embodiment of the present application, for the first image and the second image corresponding to each calibration plate, the predicted coordinates of the feature points on the second image are obtained according to the coordinates of the feature points on the first image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera. Then, the reprojection error between the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image can be calculated, and then the target external parameters between the first camera and the second camera can be inferred based on the reprojection error. For the second image taken by the low-resolution camera, the coordinates (X circle , Y circle ), therefore, the reprojection error can be accurately calculated. Finally, the target extrinsic parameters between the first camera and the second camera can be accurately inferred based on the reprojection error.

[0136] In the previous embodiment, a process of inferring the target extrinsic parameters between the first camera and the second camera based on the reprojection error is described. In this embodiment, step 264 is further described, for each second image, according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image, obtaining the target extrinsic parameters between the first camera and the second camera, including:

[0137] For each second image, obtaining a reprojection error according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image;

[0138] A cost function is constructed based on the reprojection error, and a target extrinsic parameter between the first camera and the second camera is obtained based on the cost function.

[0139] For each second image in the second image set, according to the coordinates (X circle _i, Y circle _i), the predicted coordinates of the second image feature point (U i , V i ), calculate the reprojection error. Then construct a cost function based on the reprojection error, and obtain the target extrinsic parameters between the first camera and the second camera based on the cost function. The cost function constructed based on the reprojection error is shown in the following formula (10):

[0140]

[0141] Where n represents the number of second images. It can be seen that err represents the sum of the reprojection errors of all second images. Different rotation vectors Rd and translation vectors Td are searched by formula (10) to minimize err. The rotation vector Rd and translation vector Td corresponding to the minimum err are used as the target external parameters between the first camera and the second camera.

[0142] In the embodiment of the present application, for each second image, a reprojection error is obtained based on the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image. A cost function is then constructed based on the reprojection error, and the target extrinsic parameters between the first camera and the second camera are obtained based on the cost function. By calculating the reprojection error between the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image, the target extrinsic parameters between the first camera and the second camera corresponding to the minimum reprojection error can be searched. In this way, the target extrinsic parameters between the first camera and the second camera are accurately calibrated, so as to facilitate the subsequent accurate capture of clearer images.

[0143] In the previous embodiment, a process is described in which the target extrinsic parameters between the first camera and the second camera corresponding to the minimum reprojection error can be searched by calculating the reprojection error between the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image. In this embodiment, it is further described that the target calibration plate set also includes a full white calibration plate and a full black calibration plate; step 220, the target calibration plate set is photographed by the first camera and the second camera respectively to obtain the first image set and the second image set, including:

[0144] The target calibration plate set is photographed by the first camera and the second camera respectively to obtain a first initial image set and a second initial image set; the first initial image set includes a first initial image, a first all-white image corresponding to the all-white calibration plate, and a first all-black image corresponding to the all-black calibration plate, and the second initial image set includes a second initial image, a second all-white image corresponding to the all-white calibration plate, and a second all-black image corresponding to the all-black calibration plate;

[0145] For each first initial image in the first initial image set, the first initial image is processed according to the first all-white image and the first all-black image to obtain a first image, and a first image set is obtained based on each first image;

[0146] For each second initial image in the second initial image set, the second initial image is processed according to the second all-white image and the second all-black image to obtain a second image, and a second image set is obtained based on each second image.

[0147] like Fig.10FIG. 1 is a schematic diagram of multiple calibration plates in another embodiment. The target calibration plate set includes multiple calibration plates, which can be obtained by displaying multiple calibration plate images on a display device. The calibration plate includes a preset circular bright spot, and the center of the preset circular bright spot is a characteristic point on the calibration plate. Fig.10 As shown, on the calibration plates corresponding to 1.png-12.png, the preset circular bright spots are at different positions on different calibration plates, that is, the centers of the preset circular bright spots (feature points on the calibration plates) are at different positions on different calibration plates. The calibration plate corresponding to 0.png is a completely black calibration plate, and the calibration plate corresponding to 13.png is a completely white calibration plate.

[0148] The electronic device can use two cameras to cyclically shoot each calibration plate in the target calibration plate set to obtain a first image set and a second image set. Exemplarily, the calibration plates corresponding to 0.png-13.png in the target calibration plate set are shot by the first camera to obtain a first initial image set. The first initial image set includes a first initial image, a first all-white image corresponding to an all-white calibration plate, and a first all-black image corresponding to an all-black calibration plate. Then, for each first initial image in the first initial image set, the first initial image is processed according to the first all-white image and the first all-black image to obtain a first image, and a first image set is obtained based on each first image. For example, the first initial image is divided by the first all-white image to obtain the first image, which can be understood as dividing the pixel value of each pixel in the first initial image by the pixel value of the corresponding pixel in the first all-white image to obtain the first image. In this way, the lens shading in the first initial image can be corrected to remove the influence of the lens shading on the calibration result.

[0149] Exemplarily, the calibration plate corresponding to 0.png-13.png in the target calibration plate set is photographed by the second camera to obtain a second initial image set. The second initial image set includes a second initial image, a second all-white image corresponding to the all-white calibration plate, and a second all-black image corresponding to the all-black calibration plate. Then, for each second initial image in the second initial image set, the second initial image is processed according to the second all-white image and the second all-black image to obtain a second image, and a second image set is obtained based on each second image. For example, the second initial image is divided by the second all-white image to obtain the second image, which can be understood as dividing the pixel value of each pixel in the second initial image by the pixel value of the corresponding pixel in the second all-white image to obtain the second image. Such processing can reduce the influence of ambient light and the weak brightness that may exist when the screen displays all black on the calibration results, and reduce the influence of background noise on the calibration results.

[0150] In an embodiment of the present application, the target calibration plate set also includes an all-white calibration plate and an all-black calibration plate. Thus, the target calibration plate set is photographed by the first camera, and the obtained first initial image includes the first initial image, the first all-white image corresponding to the all-white calibration plate, and the first all-black image corresponding to the all-black calibration plate. Therefore, the first initial image can be preprocessed based on the first all-white image and the first all-black image to remove some interference to obtain the first image set. Similarly, interference can also be removed to obtain the second image set. Thus, by performing camera extrinsic calibration based on the first image set and the second image set with interference removed, it is obvious that the accuracy of camera extrinsic calibration can be improved.

[0151] In an exemplary embodiment, Fig.11 As shown, a camera parameter calibration method is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the target calibration plate set includes a plurality of calibration plates with preset circular bright spots distributed at different positions, and also includes a full white calibration plate and a full black calibration plate. The method includes:

[0152] Step 1102, photographing the target calibration plate set by the first camera and the second camera respectively to obtain a first initial image set and a second initial image set; the first initial image set includes a first initial image, a first all-white image corresponding to the all-white calibration plate, and a first all-black image corresponding to the all-black calibration plate, and the second initial image set includes a second initial image, a second all-white image corresponding to the all-white calibration plate, and a second all-black image corresponding to the all-black calibration plate;

[0153] Step 1104, for each first initial image in the first initial image set, the first initial image is processed according to the first all-white image and the first all-black image to obtain a first image, and a first image set is obtained based on each first image;

[0154] Step 1106, for each second initial image in the second initial image set, the second initial image is processed according to the second all-white image and the second all-black image to obtain a second image, and a second image set is obtained based on each second image;

[0155] Step 1108, for each first image in the first image set, obtaining a binary image of the first image;

[0156] Step 1110, performing fitting based on the contour of the binary image to obtain a circular equation of a preset circular bright spot, and obtaining the coordinates of the feature points on the first image based on the circular equation;

[0157] Step 1112, for each second image in the second image set, obtaining a pixel segmentation line of the target pixel area according to a pixel value relationship of pixels in the target pixel area;

[0158] Step 1114, dividing the target pixel region according to the pixel dividing line to obtain a first pixel region and a second pixel region;

[0159] Step 1116, obtaining the coordinates of the center of the preset circular bright spot on the second image according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area, and the target mapping relationship; the target mapping relationship includes a mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area at the target image size ratio, and the distance between the coordinates of the center of the preset circular bright spot on the second image and the pixel dividing line;

[0160] Step 1118, for the first image and the second image corresponding to each calibration plate, the predicted coordinates of the feature points on the second image are obtained according to the coordinates of the feature points on the first image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera;

[0161] Step 1120, for each second image, obtaining a reprojection error according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image;

[0162] Step 1122: construct a cost function based on the reprojection error, and obtain target extrinsic parameters between the first camera and the second camera based on the cost function.

[0163] In the embodiment of the present application, for each second image, a reprojection error is obtained based on the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image. A cost function is then constructed based on the reprojection error, and the target extrinsic parameters between the first camera and the second camera are obtained based on the cost function. By calculating the reprojection error between the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image, the target extrinsic parameters between the first camera and the second camera corresponding to the minimum reprojection error can be searched. In this way, the target extrinsic parameters between the first camera and the second camera are accurately calibrated, so as to facilitate the subsequent accurate capture of clearer images.

[0164] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0165] Based on the same inventive concept, Fig.12 As shown, a camera parameter calibration device 1200 is provided, which is applied to an electronic device, wherein the electronic device includes a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the device includes:

[0166] The image set acquisition module 1220 is used to photograph the target calibration plate set through the first camera and the second camera respectively to obtain the first image set and the second image set; the target calibration plate set includes multiple calibration plates, and the positions of the feature points on each calibration plate are different;

[0167] A coordinate acquisition module 1240 is used to acquire the coordinates of the feature points on each first image in the first image set, and to acquire the coordinates of the feature points on each second image in the second image set using a preset pixel method;

[0168] The external parameter acquisition module 1260 is used to obtain the target external parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera.

[0169] In one embodiment, the calibration plate includes a preset circular bright spot, and the center of the preset circular bright spot is a characteristic point on the calibration plate; Fig.13 As shown, the coordinate acquisition module 1240 includes:

[0170] The circle center coordinate acquisition module 1242 is used to acquire the circle center coordinates of the preset circular bright spot on the second image according to the target pixel area on the second image and the preset pixel method for each second image in the second image set; the target pixel area is the area on the second image corresponding to the preset circular bright spot on the calibration plate corresponding to the second image;

[0171] The feature point coordinate acquisition unit 1244 is used to use the coordinates of the center of the preset circular bright spot on each second image as the coordinates of the feature point on each second image.

[0172] In one embodiment, the circle center coordinate acquisition module 1242 includes:

[0173] A pixel segmentation line acquisition subunit is used to acquire a pixel segmentation line of a target pixel area according to a pixel value size relationship of pixels in the target pixel area;

[0174] A pixel region division subunit, used to divide the target pixel region according to the pixel division line to obtain a first pixel region and a second pixel region;

[0175] The center coordinate acquisition subunit is used to acquire the center coordinates of the preset circular bright spot on the second image according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area and the target mapping relationship; the target mapping relationship includes the mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area at the target image size ratio, and the distance between the center coordinates of the preset circular bright spot on the second image and the pixel dividing line.

[0176] In one embodiment, the pixel dividing line includes a first pixel dividing line in a first direction and a second pixel dividing line in a second direction, and the first direction and the second direction are perpendicular to each other;

[0177] The pixel division line acquisition subunit is also used to acquire a first pixel division line in the first direction of the target pixel area based on the relationship between the pixel values ​​of the pixels in the target pixel area in the first direction; the pixel values ​​of the pixels on one side of the first pixel division line in the target pixel area are greater than the pixel values ​​of the pixels on the other side of the first pixel division line; according to the relationship between the pixel values ​​of the pixels in the target pixel area in the second direction, acquire a second pixel division line in the second direction of the target pixel area; the pixel values ​​of the pixels on one side of the second pixel division line in the target pixel area are greater than the pixel values ​​of the pixels on the other side of the second pixel division line.

[0178] In one embodiment, the pixel area division subunit is also used to divide the target pixel area according to the first pixel division line to obtain the first pixel area and the second pixel area in the first direction; and divide the target pixel area according to the second pixel division line to obtain the first pixel area and the second pixel area in the second direction.

[0179] In one embodiment, the circle center coordinate acquisition subunit is further used to acquire, for the first direction, a first distance from the circle center of a preset circular bright spot on the second image to the first pixel dividing line according to the sum of pixel values ​​of pixels in a first pixel area in the first direction and the sum of pixel values ​​of pixels in a second pixel area in the first direction in combination with a target mapping relationship;

[0180] For the second direction, according to the sum of pixel values ​​of pixels in the first pixel area in the second direction and the sum of pixel values ​​of pixels in the second pixel area in the second direction combined with the target mapping relationship, a second distance from the center of the preset circular bright spot on the second image to the second pixel dividing line is obtained;

[0181] According to the coordinates of the first pixel dividing line in the first direction and the first distance, the coordinates of the center of the preset circular bright spot on the second image in the first direction are obtained, and according to the coordinates of the second pixel dividing line in the second direction and the second distance, the coordinates of the center of the preset circular bright spot on the second image in the second direction are obtained;

[0182] Based on the coordinates of the center of the preset circular bright spot in the first direction and the coordinates of the center of the preset circular bright spot in the second direction, the coordinates of the center of the preset circular bright spot on the second image are obtained.

[0183] In one embodiment, the target mapping relationship includes a first target mapping relationship and a second target mapping relationship; the first target mapping relationship includes a mapping relationship between the sum of pixel values ​​of pixels in a first pixel area in a first direction and the sum of pixel values ​​of pixels in a second pixel area in the first direction, and the first distance under the target image size ratio; the target image size ratio is the ratio between the size of a preset circular bright spot on the calibration plate and the size of the target pixel area on the second image;

[0184] The second target mapping relationship includes a mapping relationship between the sum of pixel values ​​of pixels in the first pixel area in the second direction and the sum of pixel values ​​of pixels in the second pixel area in the second direction and the second distance at the target image size ratio.

[0185] In one embodiment, the coordinate acquisition module 1240 is also used to obtain a binary image of the first image for each first image in the first image set; perform fitting based on the contour of the binary image to obtain a circular equation of a preset circular bright spot; and obtain the coordinates of the feature points on the first image based on the circular equation.

[0186] In one embodiment, the external reference acquisition module 1260 includes:

[0187] A predicted coordinate acquisition unit is used to acquire predicted coordinates of feature points on the second image according to the coordinates of feature points on the first image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera, for the first image and the second image corresponding to each calibration plate;

[0188] The target extrinsic parameter acquisition unit is used to acquire the target extrinsic parameter between the first camera and the second camera according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image for each second image.

[0189] In one embodiment, the target extrinsic parameter acquisition unit is further used to obtain a reprojection error for each second image based on the coordinates of the feature points on the second image and the predicted coordinates of the feature points of the second image; construct a cost function based on the reprojection error, and obtain the target extrinsic parameter between the first camera and the second camera based on the cost function.

[0190] In one embodiment, the target calibration plate set also includes a full white calibration plate and a full black calibration plate; the image set acquisition module 1220 includes:

[0191] The initial image set acquisition unit is used to photograph the target calibration plate set through the first camera and the second camera respectively to obtain a first initial image set and a second initial image set; the first initial image set includes a first initial image, a first all-white image corresponding to the all-white calibration plate, and a first all-black image corresponding to the all-black calibration plate, and the second initial image set includes a second initial image, a second all-white image corresponding to the all-white calibration plate, and a second all-black image corresponding to the all-black calibration plate;

[0192] A first processing unit is used for processing each first initial image in the first initial image set according to the first all-white image and the first all-black image to obtain a first image, and obtaining a first image set based on each first image;

[0193] The second processing unit is used to process each second initial image in the second initial image set according to the second all-white image and the second all-black image to obtain a second image, and obtain a second image set based on each second image.

[0194] The division of the various modules in the above-mentioned camera parameter calibration device is only for illustration. In other embodiments, the camera parameter calibration device may be divided into different modules as needed to complete all or part of the functions of the above-mentioned camera parameter calibration device.

[0195] For the specific definition of the camera parameter calibration device, please refer to the definition of the camera parameter calibration method above, which will not be repeated here. Each module in the above-mentioned camera parameter calibration device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0196] Fig.14 Schematic diagram of the internal structure of an electronic device in an embodiment. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, a wearable device, etc. The electronic device includes a processor and a memory connected via a system bus. Among them, the processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processing), etc. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement a camera parameter calibration method provided in each of the following embodiments. The internal memory provides a cache operating environment for the operating system computer program in the non-volatile storage medium.

[0197] The implementation of each module in the camera parameter calibration device provided in the embodiment of the present application can be in the form of a computer program. The computer program can be run on an electronic device. The program module constituted by the computer program can be stored in the memory of the electronic device. When the computer program is executed by the processor, the steps of the method described in the embodiment of the present application are implemented.

[0198] The embodiment of the present application also provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, the processors execute the steps of the camera parameter calibration method.

[0199] An embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute a camera parameter calibration method.

[0200] Any reference to memory, storage, database or other medium used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include ROM (Read-Only Memory), PROM (Programmable Read-only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-only Memory) or flash memory. Volatile memory may include RAM (Random Access Memory), which is used as an external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), Double Data Rate DDRSDRAM (Double Data Rate Synchronous Dynamic Random Access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory), SLDRAM (Sync Link Dynamic Random Access Memory), RDRAM (Rambus Dynamic Random Access Memory), and DRDRAM (Direct Rambus Dynamic Random Access Memory).

[0201] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A camera parameter calibration method, characterized in that: Applied to an electronic device, the electronic device comprises a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; the method comprises: The target calibration plate set is photographed by the first camera and the second camera respectively to obtain a first image set and a second image set; the target calibration plate set includes a plurality of calibration plates, and the positions of the feature points on each calibration plate are different; Acquire the coordinates of the feature points on each first image in the first image set, and acquire the coordinates of the feature points on each second image in the second image set by using a preset pixel method; The target extrinsic parameters between the first camera and the second camera are obtained according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the extrinsic parameters between the target calibration plate set and the first camera, and the initial extrinsic parameters between the first camera and the second camera.

2. The method according to claim 1, characterized in that The calibration plate includes a preset circular bright spot, and the center of the preset circular bright spot is a feature point on the calibration plate; the step of using a preset pixel method to obtain the coordinates of the feature point on each second image in the second image set includes: For each second image in the second image set, according to a target pixel area on the second image and the preset pixel method, the coordinates of the center of the preset circular bright spot on the second image are obtained; the target pixel area is an area on the second image corresponding to the preset circular bright spot on the calibration plate corresponding to the second image; The coordinates of the center of the preset circular bright spot on each of the second images are used as the coordinates of the feature point on each of the second images.

3. The method according to claim 2, characterized in that The step of acquiring the center coordinates of the preset circular bright spot on the second image according to the target pixel area on the second image and the preset pixel method includes: According to the pixel value relationship of the pixels in the target pixel area, a pixel segmentation line of the target pixel area is obtained; Dividing the target pixel area according to the pixel dividing line to obtain a first pixel area and a second pixel area; The coordinates of the center of the preset circular bright spot on the second image are obtained according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area and the target mapping relationship; the target mapping relationship includes a mapping relationship between the sum of the pixel values ​​of the pixels in the first pixel area and the sum of the pixel values ​​of the pixels in the second pixel area at the target image size ratio, and the distance between the coordinates of the center of the preset circular bright spot on the second image and the pixel dividing line.

4. The method according to claim 3, characterized in that The pixel segmentation lines include a first pixel segmentation line in a first direction and a second pixel segmentation line in a second direction, and the first direction and the second direction are perpendicular to each other; and obtaining the pixel segmentation lines of the target pixel area according to the pixel value relationship of the pixels in the target pixel area includes: According to the pixel value size relationship of pixels in the target pixel area in the first direction, a first pixel dividing line of the target pixel area in the first direction is obtained; the pixel values ​​of pixels on one side of the first pixel dividing line in the target pixel area are greater than the pixel values ​​of pixels on the other side of the first pixel dividing line; According to the relationship between the pixel values ​​of pixels in the target pixel area in the second direction, a second pixel dividing line of the target pixel area in the second direction is obtained; the pixel values ​​of pixels on one side of the second pixel dividing line in the target pixel area are greater than the pixel values ​​of pixels on the other side of the second pixel dividing line.

5. The method according to claim 4, characterized in that The step of dividing the target pixel region according to the pixel segmentation line to obtain a first pixel region and a second pixel region includes: Dividing the target pixel area according to the first pixel dividing line to obtain a first pixel area and a second pixel area in the first direction; The target pixel region is divided according to the second pixel dividing line to obtain a first pixel region and a second pixel region in the second direction.

6. The method according to claim 5, characterized in that The step of acquiring the coordinates of the center of the preset circular bright spot on the second image according to the coordinates of the pixel dividing line, the sum of the pixel values ​​of the pixels in the first pixel area, the sum of the pixel values ​​of the pixels in the second pixel area, and the target mapping relationship comprises: For the first direction, according to the sum of pixel values ​​of pixels in a first pixel area in the first direction and the sum of pixel values ​​of pixels in a second pixel area in the first direction combined with the target mapping relationship, a first distance from the center of the preset circular bright spot on the second image to the first pixel dividing line is obtained; For the second direction, according to the sum of pixel values ​​of pixels in the first pixel area in the second direction and the sum of pixel values ​​of pixels in the second pixel area in the second direction combined with the target mapping relationship, a second distance from the center of the preset circular bright spot on the second image to the second pixel dividing line is obtained; Acquire the coordinates of the center of the preset circular bright spot on the second image in the first direction according to the coordinates of the first pixel dividing line in the first direction and the first distance, and acquire the coordinates of the center of the preset circular bright spot on the second image in the second direction according to the coordinates of the second pixel dividing line in the second direction and the second distance; Based on the coordinates of the center of the preset circular bright spot in the first direction and the coordinates of the center of the preset circular bright spot in the second direction, the coordinates of the center of the preset circular bright spot on the second image are obtained.

7. The method according to claim 6, characterized in that The target mapping relationship includes a first target mapping relationship and a second target mapping relationship; the first target mapping relationship includes a mapping relationship between the sum of pixel values ​​of pixels in a first pixel area in the first direction and the sum of pixel values ​​of pixels in a second pixel area in the first direction at a target image size ratio, and the first distance; The target image size ratio is the ratio between the size of the preset circular bright spot on the calibration plate and the size of the target pixel area on the second image; The second target mapping relationship includes a mapping relationship between the sum of pixel values ​​of pixels in a first pixel area in the second direction and the sum of pixel values ​​of pixels in a second pixel area in the second direction at the target image size ratio, and the second distance.

8. The method according to claim 2, characterized in that: The acquiring the coordinates of the feature points on each first image in the first image set includes: For each first image in the first image set, obtaining a binary image of the first image; Fitting is performed based on the contour of the binary image to obtain a circular equation of the preset circular bright spot; Based on the circle equation, coordinates of the feature points on the first image are acquired.

9. The method according to any one of claims 1 to 8, characterized in that The step of acquiring target extrinsic parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the extrinsic parameters of the target calibration plate set and the first camera, and the initial extrinsic parameters between the first camera and the second camera comprises: For the first image and the second image corresponding to each of the calibration plates, the predicted coordinates of the feature points on the second image are obtained according to the coordinates of the feature points on the first image, the external parameters of the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera; For each of the second images, target extrinsic parameters between the first camera and the second camera are obtained according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image.

10. The method according to claim 9, characterized in that The acquiring, for each of the second images, target extrinsic parameters between the first camera and the second camera according to the coordinates of the feature points on the second images and the predicted coordinates of the feature points in the second images, comprises: For each of the second images, obtaining a reprojection error according to the coordinates of the feature points on the second image and the predicted coordinates of the feature points in the second image; A cost function is constructed based on the reprojection error, and a target extrinsic parameter between the first camera and the second camera is obtained based on the cost function.

11. The method according to any one of claims 1 to 8, characterized in that The target calibration plate set also includes a full white calibration plate and a full black calibration plate; the target calibration plate set is photographed by the first camera and the second camera respectively to obtain a first image set and a second image set, including: The target calibration plate set is photographed by the first camera and the second camera respectively to obtain a first initial image set and a second initial image set; the first initial image set includes a first initial image, a first all-white image corresponding to the all-white calibration plate, and a first all-black image corresponding to the all-black calibration plate, and the second initial image set includes a second initial image, a second all-white image corresponding to the all-white calibration plate, and a second all-black image corresponding to the all-black calibration plate; For each of the first initial images in the first initial image set, the first initial image is processed according to the first all-white image and the first all-black image to obtain a first image, and the first image set is obtained based on each of the first images; For each of the second initial images in the second initial image set, the second initial image is processed according to the second all-white image and the second all-black image to obtain a second image, and the second image set is obtained based on each of the second images.

12. A camera parameter calibration device, characterized in that: Applied to an electronic device, the electronic device comprises a first camera and a second camera, the resolution of the first camera is greater than a preset resolution threshold, and the resolution of the second camera is less than or equal to the preset resolution threshold; The device comprises: An image set acquisition module, used to photograph a target calibration plate set through the first camera and the second camera respectively to obtain a first image set and a second image set; the target calibration plate set includes a plurality of calibration plates, and the positions of feature points on each calibration plate are different; a coordinate acquisition module, used to acquire the coordinates of the feature points on each first image in the first image set, and to acquire the coordinates of the feature points on each second image in the second image set by using a preset pixel method; The external parameter acquisition module is used to obtain the target external parameters between the first camera and the second camera according to the coordinates of the feature points on each first image, the coordinates of the feature points on each second image, the external parameters between the target calibration plate set and the first camera, and the initial external parameters between the first camera and the second camera.

13. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor executes the steps of the camera parameter calibration method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the camera parameter calibration method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the camera parameter calibration method described in any one of claims 1 to 11 are implemented.

Images