Dual-camera calibration method, device, equipment and storage medium

Abstract

The application provides a double-camera calibration method, device, equipment and storage medium, the method comprises the following steps: acquiring a first detail image photographed by a first camera at each preset calibration point, performing NCC image matching processing on each first detail image and a panoramic image photographed by a second camera, and determining each first matching result; if target first detail image matching fails, dividing the target first detail image, performing NCC image matching processing on each second detail image and the panoramic image obtained, and determining the second matching result corresponding to each second detail image; determining the pixel position of the target first detail image in the panoramic image according to the pixel position of the center point of the matching successful second detail image in the panoramic image; and establishing a mapping relationship between the pixel position and the latitude and longitude according to multiple groups of target pixel positions and the latitude and longitude of each preset calibration point. The technical scheme can simplify the calibration logic and improve the calibration efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a calibration method, apparatus, device, and storage medium for dual cameras. Background Technology

[0002] Dual cameras refer to image acquisition devices with two cameras: a bullet camera and a PTZ camera. The bullet camera, also known as a stationary camera, is responsible for capturing panoramic images / scenes, while the PTZ camera, also known as a moving camera, is responsible for capturing detailed images / scenes within the panoramic image. Typically, dual cameras are calibrated before use, establishing a mapping between the pixel positions of the stationary cameras and the latitude and longitude of the moving camera's rotation. Specifically, there is a linkage between the stationary and moving cameras; clicking on a specific location on the stationary camera will cause the moving camera to rotate, centering the image at that location.

[0003] In related technologies, when calibrating dual cameras, the process typically involves first acquiring feature points in a panoramic image captured from a fixed point and feature points in a detail image captured from a moving point. Then, the feature points of these two images are matched multiple times to find the pixel position on the panoramic image of the fixed point. Finally, a mapping relationship is established between the pixel position and the latitude and longitude on the detail image of the moving point to achieve the calibration of the dual cameras.

[0004] However, the calibration logic of the above-mentioned technology is relatively complex, resulting in low calibration efficiency for dual cameras. Summary of the Invention

[0005] This invention provides a calibration method, apparatus, device, and storage medium for dual cameras, addressing the shortcomings of existing calibration methods that rely on feature matching, which are logically complex, resource-intensive, and time-consuming. The invention achieves this by segmenting a detailed image acquired from a moving point and matching it with a panoramic image acquired from a fixed point to determine the pixel position of the center point of the moving point image within the fixed point image. Based on this, a mapping relationship is established between the pixel position of the fixed point and the latitude and longitude of the moving point, thereby simplifying the calibration complexity of dual cameras, reducing resource consumption, and improving calibration efficiency.

[0006] This invention provides a calibration method for dual cameras, comprising: The first detail image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point is obtained. Normalized cross-correlation NCC image matching processing is performed on each first detail image and the panoramic image captured by the second camera to determine the first matching result corresponding to each first detail image. If the first matching result of the target first detail image is a failure, the target first detail image is divided according to the first number to obtain multiple second detail images of the same number as the first number. Then, NCC image matching processing is performed on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image. Based on the pixel position of the center point of the second detail image that is successfully matched in the second matching result in the panoramic image, determine the pixel position of the center point of the target first detail image in the panoramic image. Based on the pixel positions corresponding to the center points of each first detail image in the panoramic image, multiple sets of target pixel positions are determined. Based on each target pixel position and the latitude and longitude of the preset calibration point corresponding to each target pixel position, a mapping relationship is established between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the rotation of the first camera.

[0007] According to a dual-camera calibration method provided by the present invention, the method for determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image (which is a successfully matched second detail image) in the panoramic image includes: Based on each second matching result, determine whether the number of second detail images that are successfully matched is less than a preset number; If the number of successfully matched second detail images is less than a preset number, then the second number is obtained; the second number is greater than the first number. The second quantity is used as the new first quantity. The process of dividing the target first detail image according to the first quantity is returned to obtain multiple second detail images with the same number as the first quantity. NCC image matching processing is performed on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image. The new second matching result corresponding to each new second detail image is obtained. Based on the new second matching result, which is the pixel position of the center point of the new second detail image that is successfully matched in the panoramic image, the pixel position of the center point of the target first detail image in the panoramic image is determined.

[0008] According to the dual-camera calibration method provided by the present invention, if the second quantity is five and the number of successfully matched new second detail images is three, the method of determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the successfully matched new second detail image in the panoramic image according to the new second matching result includes: Based on the pixel positions of the center points of the two newly matched second detail images in the panoramic image, calculate the first pixel position of the center point of the target first detail image in the panoramic image. Obtain the pixel position of the center point of the third successfully matched new second detail image in the panoramic image, and use the pixel position of the third successfully matched new second detail image as the second pixel position of the center point of the target first detail image in the panoramic image. The first pixel position and the second pixel position are averaged, and the resulting average is determined as the pixel position of the center point of the target first detail image in the panoramic image.

[0009] According to a dual-camera calibration method provided by the present invention, the method for determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image (which is a successfully matched second detail image) in the panoramic image further includes: If the number of successfully matched second detail images is not less than a preset number, then the pixel position of the center point of the target first detail image in the panoramic image is calculated based on the pixel position of the center point of the successfully matched second detail image in the panoramic image.

[0010] According to a dual-camera calibration method provided by the present invention, the determination of multiple sets of target pixel positions based on the pixel positions corresponding to the center points of each first detail image in the panoramic image includes: Obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the target pixel range based on the maximum and minimum pixel positions; In the panoramic image, the pixel positions corresponding to the center points of each first detail image are eliminated from the pixel positions that are not within the target pixel range, thus obtaining multiple sets of candidate pixel positions; Multiple sets of target pixel positions are determined based on the positions of each candidate pixel.

[0011] According to a dual-camera calibration method provided by the present invention, the method further includes: Acquire the preset calibration points used when capturing each first detail image and the latitude corresponding to each preset calibration point; Obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the maximum and minimum latitudes in the latitudes corresponding to each preset calibration point based on the maximum and minimum pixel positions. The latitude ranges corresponding to the minimum and maximum latitudes are defined as the normal latitude ranges corresponding to the mapping relationship.

[0012] According to a dual-camera calibration method provided by the present invention, before acquiring the first detailed image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, the method further includes: Obtain the target longitude range and target latitude range that the first camera can move; the center point of the detailed image captured by the first camera when it rotates within the target longitude range and target latitude range is within the panoramic image captured by the second camera; Based on the target longitude range and the target latitude range, determine multiple preset calibration points within the target longitude range and within the target latitude range; each preset calibration point corresponds to one longitude and one latitude.

[0013] The present invention also provides a calibration device for dual cameras, comprising the following modules: The first matching module is used to acquire the first detail images captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, and to perform NCC image matching processing on each first detail image and the panoramic image captured by the second camera to determine the first matching result corresponding to each first detail image. The second matching module is used to divide the target first detail image into multiple second detail images according to a first number if the first matching result of the target first detail image is a matching failure, and to perform NCC image matching processing on each second detail image and the panoramic image respectively to determine the second matching result corresponding to each second detail image. The pixel position determination module is used to determine the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the successfully matched second detail image in the panoramic image. The mapping relationship establishment module is used to determine multiple sets of target pixel positions based on the pixel positions corresponding to the center points of each first detail image in the panoramic image, and to establish a mapping relationship between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the rotation of the first camera based on each target pixel position and the latitude and longitude of the preset calibration points corresponding to each target pixel position.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a dual-camera calibration method as described above.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a dual-camera calibration method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a dual-camera calibration method as described above.

[0017] The present invention provides a dual-camera calibration method, apparatus, device, and storage medium. It acquires first detail images captured by a first camera when it rotates to the latitude and longitude corresponding to each preset calibration point. Each first detail image undergoes NCC matching processing to determine a first matching result. If the matching result of the target first detail image is a failed match, the target first detail image is divided into multiple second detail images of the same number as the first number. Each second detail image and the panoramic image undergo NCC image matching processing to determine a second matching result. Then, based on the pixel position of the center point of the successfully matched second detail image in the panoramic image, the pixel position of the center point of the target first detail image in the panoramic image is determined. Finally, based on the pixel positions of the center points of each first detail image in the panoramic image, multiple sets of target pixel positions are determined. Based on the latitude and longitude of each set of target pixel positions and their corresponding preset calibration points, a mapping relationship is established between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the first camera's rotation, thus achieving dual-camera calibration. In this method, by segmenting the moving point detail image into multiple smaller images and performing NCC matching with the fixed-point panoramic image, the pixel position of the center point of the moving point detail image in the fixed-point image is determined. Based on this, a mapping relationship between the pixel position of the fixed point and the latitude and longitude of the moving point is established. This allows for accurate and automatic calibration of the dual cameras even when the moving point detail image exceeds the fixed-point panoramic image or the fixed-point panoramic image cannot cover the moving point detail image due to the vertical rotation of the fixed point, thus achieving the effect of camera-ball linkage. At the same time, since a relatively simple NCC image matching method is used in the image matching process, the calibration complexity of the dual cameras can be simplified, the occupation of equipment resources can be reduced, and the calibration efficiency can be improved. This allows for the calibration of dual cameras on any device, expanding the applicability of the calibration method. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the dual-camera calibration method provided in this embodiment of the invention.

[0020] Figure 2 This is a schematic diagram comparing panoramic images and detail images provided in an embodiment of the present invention; Figure 3 This is one of the schematic diagrams for dividing a detailed image according to an embodiment of the present invention.

[0021] Figure 4 This is the second flowchart illustrating the dual-camera calibration method provided in this embodiment of the invention.

[0022] Figure 5 This is the second schematic diagram of dividing a detailed image according to an embodiment of the present invention.

[0023] Figure 6 This is the third schematic diagram of dividing a detailed image according to an embodiment of the present invention.

[0024] Figure 7 This is the fourth schematic diagram of dividing a detailed image according to an embodiment of the present invention.

[0025] Figure 8 This is a schematic diagram of the structure of the dual-camera calibration device provided in an embodiment of the present invention.

[0026] Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Currently, when calibrating dual cameras, some techniques involve manual calibration. For example, manual calibration records the X and Y coordinates of a fixed point and the latitude and longitude of a moving point. These two coordinates are correlated through a series of calculations, allowing the corresponding latitude and longitude to be obtained by selecting X and Y coordinates later. However, this calibration method is cumbersome, costly, and inefficient. Other techniques involve automatic calibration, such as feature comparison. However, this method has complex calculation logic, and for devices with weak CPUs, a single comparison can take close to 10 seconds, making it unsuitable for these devices and resulting in low efficiency. Furthermore, some devices have a fixed point (i.e., the camera lens) with a tilt angle. This means that even if the moving point (i.e., the PTZ camera) is rotated to its maximum, the entire moving point image cannot be included in the fixed point image. This leads to inaccurate image comparison / matching results, resulting in inaccurate dual-camera calibration. Based on this, embodiments of the present invention provide a calibration method, apparatus, device, and storage medium for dual cameras, which can solve the above-mentioned technical problems.

[0029] The following is combined with Figures 1-7 This invention describes a dual-camera calibration method according to an embodiment of the present invention.

[0030] It should be noted that the execution subject of the embodiments of the present invention can be a dual-camera calibration device, an electronic device, or other devices or apparatus, without specific limitations. The following embodiments will use an electronic device as the execution subject for description. The electronic device can be a terminal or a server. In the case of a terminal, it can be the electronic device in the dual cameras, or it can be an electronic device connected to the dual cameras, such as a mobile phone, tablet computer, desktop computer, wearable device, etc.

[0031] Figure 1 This is one of the flowcharts illustrating the dual-camera calibration method provided in this embodiment of the invention, such as... Figure 1 As shown, the method includes the following steps: Step 102: Obtain the first detail image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, and perform normalized cross-correlation NCC image matching processing on each first detail image and the panoramic image captured by the second camera to determine the first matching result corresponding to each first detail image.

[0032] This embodiment mainly calibrates a dual-camera system, which includes a first camera and a second camera. The first camera is a moving-point camera, which mainly captures detailed images / scenes of the scene, while the second camera is a fixed-point camera, which mainly captures panoramic images of the scene. The calibration essentially associates the pixel positions in the panoramic image of the fixed point with the latitude and longitude of the moving point's rotation. This allows the moving point to rotate to the latitude and longitude corresponding to the selected pixel position in the fixed-point image, so as to capture detailed images at that pixel position.

[0033] In this embodiment, before calibrating the dual cameras, optionally, the target longitude range and target latitude range that the first camera can move can be obtained; the center point of the detailed image captured by the first camera when it rotates within the target longitude range and target latitude range is within the panoramic image captured by the second camera; then, based on the target longitude range and target latitude range, multiple preset calibration points are determined within the target longitude range and within the target latitude range; each preset calibration point corresponds to one longitude and one latitude.

[0034] Specifically, when the second camera (i.e., a stationary or bullet camera) can only rotate vertically, the first camera is controlled to rotate within a longitude range of 0° to 360°. The longitude range within which the center point of the image captured by the first camera (i.e., a moving or PTZ camera) remains consistently within the panoramic view of the stationary point is recorded. This longitude range can be designated as the target longitude range. Similarly, when the second camera rotates to its highest elevation and lowest depression angles, the first camera is controlled to rotate within a latitude range of 0° to 90°. The latitude range within which the center point of the image captured by the first camera remains consistently within the panoramic view of the stationary point is recorded. This latitude range can be designated as the target latitude range. In this way, when the first camera subsequently rotates within these target longitude and latitude ranges, regardless of the angle at which the second camera rotates, an overlapping image from both the first and second cameras can be found.

[0035] After obtaining the target longitude and latitude ranges, these ranges can be divided into multiple equal parts. Then, a set of longitude and latitude coordinates is determined from each longitude and latitude range, and each set is used as a preset calibration point. This results in multiple preset calibration points. The more longitude and latitude ranges divided, the more preset calibration points are available, leading to better calibration results. However, calibration speed / efficiency decreases with an excessive number of preset calibration points. Therefore, the specific number of preset calibration points can be set according to actual conditions, balancing calibration effectiveness and efficiency.

[0036] For example, assuming the target longitude range is 130°~160° and the target latitude range is 0°~60°, and assuming the target latitude is divided into 7 equal parts, resulting in 7 preset calibration points, then the latitude and longitude of the 7 preset calibration points are (130,0), (135,10), (140,20), (145,30), (150,40), (155,50), and (160,60). It should be noted that the latitude and longitude of each preset calibration point here refers to the first and last endpoints of the divided latitude and longitude range. In practice, any latitude and longitude within each range can be selected as the latitude and longitude of the corresponding preset calibration point.

[0037] After obtaining multiple preset calibration points, the first camera / moving point can be rotated to the latitude and longitude corresponding to the first preset calibration point. Then, image acquisition of the scene is performed at that latitude and longitude to obtain a detailed image. The first camera can then be rotated sequentially to the latitude and longitude corresponding to other preset calibration points to obtain a detailed image at each preset calibration point. Simultaneously, the second camera / fixed point can also synchronously acquire images of the scene to obtain a panoramic image. Then, the detailed images acquired at each preset calibration point can be converted to a scale matching the panoramic image, based on the ratio between the focal lengths of the first and second cameras and the size ratio between the detailed image and the panoramic image. Each detailed image obtained at this point is recorded as a first detailed image. Then, each first detailed image can be subjected to NCC image matching processing with the panoramic image to obtain a matching result for each first detailed image, which is recorded as a first matching result.

[0038] Here, NCC refers to Normalized Cross-Correlation. NCC image matching mainly includes the following steps: 1. Obtain a template image and a target image, where the template image is, for example, the detailed image mentioned above, and the target image is, for example, the panoramic image mentioned above; 2. For the template image, obtain the value of each pixel and calculate its average value. For the target image, move a window on the target image according to the size of the template image and calculate the average value of the pixels within each window; 3. Take the mean of the template image and the target image respectively based on the average values ​​of the pixels in the template image and the target image; 4. Calculate the cross-correlation value of the template image and the target image at each pixel position, and divide the cross-correlation value by the mean square error of the template image and the target image to obtain the normalized cross-correlation NCC value; 5. Traverse the target image and find the pixel position with the largest NCC value, which is the best matching position.

[0039] It should be noted that the NCC matching logic is relatively simple. Compared with the feature comparison method, each image matching takes only about 0.2 seconds, which is very short. Therefore, the image matching logic can be simplified, enabling dual-camera calibration on any device (including weak devices), and improving the efficiency of subsequent calibration.

[0040] After performing NCC image matching processing on each first detail image and panoramic image, a first matching result is obtained. This first matching result can include the maximum NCC value obtained for each first detail image and whether the match was successful or unsuccessful. The success or failure result can be determined by the maximum NCC threshold. For example, the maximum NCC value of each first detail image can be compared with a preset threshold. If the maximum NCC value of a certain first detail image is greater than the preset threshold, it means that the first matching result of that first detail image is a successful match; otherwise, it means that the first matching result of that first detail image is a failed match.

[0041] The specific value of the above preset threshold can be set according to the actual situation, such as 0.7, 0.8, etc.

[0042] In addition, when obtaining the maximum NCC value corresponding to each first detail image, the pixel position corresponding to the maximum NCC value in the panoramic image can also be obtained, as well as the latitude and longitude of the preset target location where the first camera was located when the first detail image was acquired.

[0043] Step 104: If the first matching result of the target first detail image is a matching failure, the target first detail image is divided according to the first number to obtain multiple second detail images of the same number as the first number, and NCC image matching processing is performed on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image.

[0044] In this step, after each of the aforementioned first detail images undergoes NCC image matching with the panoramic image, if the first matching result of a certain first detail image (denoted as the target first detail image) fails, that is, the maximum NCC value corresponding to the target first detail image is less than a preset threshold, for example, it may be close to 0 or a negative value (where an NCC value of -1 indicates that the two images are unrelated, and an NCC value of 1 indicates that the two images are identical), see, for example, see Figure 2 The diagram shown is a comparison diagram of panoramic and detail images provided by an embodiment of the present invention. The upper thin frame line represents the panoramic image, and the lower thin frame line represents the detail image. It can be seen that only half of the scene in the detail image may be the same as that in the panoramic image (i.e., the area with thick frame lines in both images). In this case, the first matching result of the two images may be a matching failure.

[0045] To facilitate accurate subsequent calibration of the dual cameras, the first detailed image of the target can be divided into a first number of second detailed images. Specifically, for cases where the detailed image of the moving point may not be fully encompassed in the panoramic image of the fixed point in the vertical direction, the first detailed image of the target can be divided into multiple second detailed images along the vertical axis; similarly, for cases where the detailed image of the moving point may not be fully encompassed in the panoramic image of the fixed point in the horizontal direction, the first detailed image of the target can be divided into multiple second detailed images along the horizontal axis. The following explanation uses the division of the first detailed image of the target into multiple second detailed images along the vertical axis as an example.

[0046] When dividing the target first detail image along the vertical axis, it can be directly divided into a first number of second detail images along the vertical axis. Alternatively, since some parts of the detail image may not be covered by the panoramic image along the horizontal axis, when dividing the second detail images, the first detail image can be divided into three parts along the horizontal axis, retaining the middle part, and then the middle part can be divided along the vertical axis to obtain multiple second detail images. For example, if the first number is 3, the first detail image can be divided into three parts along the vertical axis to obtain three second detail images. A schematic diagram of the three second detail images can be found in [reference needed]. Figure 3 The diagram shown is one of the schematic diagrams for dividing a detail image according to an embodiment of the present invention, wherein the three dashed boxes represent the divided second detail images.

[0047] After dividing the target first detail image into multiple second detail images, the obtained multiple second detail images can be re-matched with the panoramic image using NCC image matching. The matching result for each second detail image is then obtained and recorded as a second matching result. Each second matching result can include the maximum NCC value obtained for that second detail image and whether the match was successful or unsuccessful. The specific NCC image matching process can be found in the NCC image matching process for the first detail image described above, and will not be repeated here.

[0048] Step 106: Based on the pixel position of the center point of the second detail image that is successfully matched in the second matching result in the panoramic image, determine the pixel position of the center point of the target first detail image in the panoramic image.

[0049] In this step, after subdividing the first detail image that failed to match the whole image and obtaining the second matching results by performing NCC image matching on the subdivided second detail images, if there are many successfully matched second detail images among the multiple second detail images divided from the target first detail image, the pixel position of each successfully matched second detail image in the panoramic image can be obtained through the maximum NCC value of each of these successfully matched second detail images. Based on this, the pixel position of the center point of the target first detail image in the panoramic image can be calculated. Specifically, the pixel position of each successfully matched second detail image in the panoramic image includes the pixel position of the center point of each successfully matched second detail image in the panoramic image. Then, through the geometric correspondence between the center point of each second detail image and the center point of the corresponding target first detail image, the pixel position of the center point of the target first detail image in the panoramic image can be calculated.

[0050] If there are few successfully matched second detail images, the final matching result of the target first detail image can be directly determined as a failed match. Alternatively, the target first detail image can be further subdivided and NCC image matching can be performed again to determine the final matching result of the target first detail image.

[0051] Step 108: Based on the pixel positions corresponding to the center points of each first detail image in the panoramic image, determine multiple sets of target pixel positions, and establish a mapping relationship between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the first camera rotation based on each target pixel position and the latitude and longitude of the preset calibration points corresponding to each target pixel position.

[0052] In this step, the final matching result of each first detail image (including the target first detail image) can be obtained through the above steps. At the same time, the maximum NCC value in each first detail result can be obtained, and the corresponding pixel position in the panoramic image can be obtained through the maximum NCC value. In this way, the pixel position of the center point of each first detail image in the panoramic image can be obtained.

[0053] Then, the pixel position corresponding to the center point of each first detail image in the panoramic image can be used as the target pixel position. Alternatively, a subset of pixel positions can be selected (e.g., the three pixel positions with the largest NCC value) as the target pixel positions, or other methods can be used. In short, multiple sets of target pixel positions can be obtained. Then, the first detail image corresponding to each set of target pixel positions can be obtained, along with the latitude and longitude of the preset calibration used by the first camera during acquisition. This gives the latitude and longitude corresponding to each set of target pixel positions. Next, a correspondence / mapping relationship between pixel positions and their corresponding latitude and longitude can be established using multiple sets of target pixel positions and their corresponding latitude and longitude. That is, a conversion expression between pixel positions and latitude and longitude can be established. Subsequently, any pixel position input into this expression can be mapped to the latitude and longitude corresponding to that pixel position. This allows the first camera to be controlled to rotate to this latitude and longitude for detail image acquisition, achieving the goal of converting pixel positions to corresponding latitude and longitude even when the panoramic image does not completely cover the detail image.

[0054] In this embodiment, first detail images are acquired when the first camera rotates to the latitude and longitude corresponding to each preset calibration point. NCC matching processing is performed on each first detail image to determine the first matching result. If the matching result of the target first detail image is a failure, the target first detail image is divided into multiple second detail images of the same number as the first number. NCC image matching processing is then performed on each second detail image and the panoramic image to determine the second matching result. Based on the pixel position of the center point of the second detail image that is successfully matched in the panoramic image, the pixel position of the center point of the target first detail image in the panoramic image is determined. Then, based on the pixel position of the center point of each first detail image in the panoramic image, multiple sets of target pixel positions are determined. Finally, based on the target pixel positions and the latitude and longitude of the corresponding preset calibration points, a mapping relationship is established between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the first camera's rotation, thus achieving the calibration of the two cameras. In this method, by segmenting the moving point detail image into multiple smaller images and performing NCC matching with the fixed-point panoramic image, the pixel position of the center point of the moving point detail image in the fixed-point image is determined. Based on this, a mapping relationship between the pixel position of the fixed point and the latitude and longitude of the moving point is established. This allows for accurate and automatic calibration of the dual cameras even when the moving point detail image exceeds the fixed-point panoramic image or the fixed-point panoramic image cannot cover the moving point detail image due to the vertical rotation of the fixed point, thus achieving the effect of camera-ball linkage. At the same time, since a relatively simple NCC image matching method is used in the image matching process, the calibration complexity of the dual cameras can be simplified, the occupation of equipment resources can be reduced, and the calibration efficiency can be improved. This allows for the calibration of dual cameras on any device, expanding the applicability of the calibration method.

[0055] The following examples illustrate the process of further subdividing the target first detail image and re-performing NCC image matching.

[0056] Figure 4 This is a second schematic flowchart of the dual-camera calibration method provided in this embodiment of the invention, as shown below. Figure 4 As shown, step 106 above, "determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image that is successfully matched in the second matching result," includes the following steps: Step 202: Based on each second matching result, determine whether the number of second detail images that are successfully matched is less than a preset number.

[0057] Specifically, after dividing the first detail image into a first number of second detail images and performing NCC image matching on each, a second matching result can be obtained for each second detail image. The second matching result can include the corresponding maximum NCC value and whether the match was successful or unsuccessful. Then, the number of successfully matched second matching results can be counted, i.e., the number of successfully matched second detail images can be counted.

[0058] Then, the number of successfully matched second detail images is compared with a preset number. Specifically, it can be determined whether the number of successfully matched second detail images is less than the preset number. Here, the preset number can be 2, so this step can be to determine whether the number of successfully matched second detail images is less than 2.

[0059] Optionally, if the number of successfully matched second detail images is not less than a preset number, then the pixel position of the center point of the target first detail image in the panoramic image is calculated based on the pixel position of the center point of the successfully matched second detail image in the panoramic image.

[0060] In other words, if the number of successfully matched second detail images is greater than or equal to a preset number (e.g., greater than or equal to 2), it indicates that there are a large number of successfully matched second detail images. Therefore, by analyzing the pixel positions of each successfully matched second detail image in the panoramic image, we can obtain the pixel position of the center point of each successfully matched second detail image in the panoramic image. Then, by analyzing the center points of each successfully matched second detail image, we can find the center point of the target first detail image, and thus obtain the pixel position of the center point of the target first detail image in the panoramic image. For example, when the target first detail image is divided into three equal second detail images, the center point of the second second detail image in the middle is the center point of the target first detail image.

[0061] Step 204: If the number of successfully matched second detail images is less than a preset number, then obtain the second number; the second number is greater than the first number.

[0062] In this step, if the number of successfully matched second detail images is less than a preset number (e.g., less than 2), it indicates that there are too few successfully matched second detail images. This means that the target first detail image failed to match using the current segmentation method for NCC image matching. In this case, to achieve accurate calibration, the target first detail image can be re-subdivided. Specifically, a second subdivision number can be obtained, which is greater than the first number, meaning the target first detail image can be divided into more second detail images. The size of this second number can be set according to the actual situation, for example, it can be set to 5.

[0063] In addition, the first and second quantities mentioned above can generally be set to odd numbers, which makes it easier to quickly count the center point of the target detail image. That is, the center point of the target detail image is consistent with the center point of the middle second detail image among the multiple second detail images.

[0064] Step 206: Take the second quantity as the new first quantity, return to the above steps of dividing the target first detail image according to the first quantity to obtain multiple second detail images with the same number as the first quantity, and perform NCC image matching processing on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image, and obtain the new second matching result corresponding to each new second detail image.

[0065] In this step, after obtaining the second number of subdivisions of the target first detail image, this second number can be used as the new first number. Then, the process of "dividing the target first detail image according to the first number to obtain multiple second detail images of the same number as the first number, and performing NCC image matching processing on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image" can be returned to the above step 104. That is, the target first detail image can be re-divided according to the second number to obtain more second detail images. The re-divided second detail images can be recorded as new second detail images. The second matching results obtained after re-performing NCC image matching on these new second detail images can also be recorded as new second matching results.

[0066] Step 208: Based on the new second matching result, which is the pixel position of the center point of the new second detail image that has been successfully matched in the panoramic image, determine the pixel position of the center point of the target first detail image in the panoramic image.

[0067] In this step, after re-segmenting the target first detail image and performing NCC matching, a new second matching result can be obtained for each new second detail image. Then, the successfully matched new second detail images can be found, and it can be determined whether the number of successfully matched new second detail images is less than a preset number.

[0068] If the number of successfully matched new second detail images is greater than or equal to a preset number, the pixel position of the center point of each successfully matched new second detail image in the panoramic image can be obtained from the pixel positions of each new second detail image in the panoramic image. Then, the center point of the target first detail image can be obtained from the center points of each successfully matched new second detail image, and thus the pixel position of the center point of the target first detail image in the panoramic image can be obtained. For example, when the target first detail image is divided into five equal second detail images, the center point of the third second detail image in the middle is the center point of the target first detail image.

[0069] Additionally, if the number of successfully matched new second detail images is less than the preset number, that is, if there are too few successfully matched new second detail images, it means that at least 3 / 4 of the target's first detail image is no longer in the panoramic view of the fixed point. In this case, there is no substantial gun-ball linkage effect, and the preset calibration point corresponding to the target's first detail image can be discarded.

[0070] Further, as an optional embodiment, if the second quantity is five and the number of newly matched second detail images is three, then the method for specifically determining the pixel position of the center point of the target first detail image in the panoramic image in this step may include the following steps: Based on the pixel positions of the center points of the two newly matched second detail images in the panoramic image, calculate the first pixel position of the center point of the target first detail image in the panoramic image. Obtain the pixel position of the center point of the third successfully matched new second detail image in the panoramic image, and use the pixel position of the third successfully matched new second detail image as the second pixel position of the center point of the target first detail image in the panoramic image. The first pixel position and the second pixel position are averaged, and the resulting average is determined as the pixel position of the center point of the target first detail image in the panoramic image.

[0071] The schematic diagram of the five new second-detail images here can be found in [reference needed]. Figure 5 The second schematic diagram of the division of detail images provided by the embodiment of the present invention is shown, in which the five dashed boxes represent five new second detail images.

[0072] Specifically, among these five new second detail images, the three successfully matched second detail images are generally consecutive. Assuming these three detail images are the first three second detail images divided from the target first detail image, let's assume that the pixel position of the center point of the first second detail image determined during the NCC image matching process is (x1, y1) in the panoramic image, the pixel position of the center point of the second second detail image determined during the NCC image matching process is (x2, y2) in the panoramic image, and the pixel position of the center point of the third second detail image determined during the NCC image matching process is (x3, y3) in the panoramic image. Because these five second detail images are divided equally, the height of each second detail image is y2-y1. The center point of the third second detail image is the center point of the target first detail image / the entire large detail image. The distance between the center point of the third second detail image and the center point of the second second detail image is the height of one second detail image. Therefore, the pixel position corresponding to the center point of the third second detail image can be calculated from the pixel positions corresponding to the center points of the first two second detail images, that is, (x3', y3'), where x3'=x1=x2, y3'=y2+|y2-y1|.

[0073] Here, (x3', y3') can be recorded as the first pixel position, and (x3, y3) can be recorded as the second pixel position. Then, the average value of these two pixel positions can be processed, and the average value obtained is the pixel position of the center point of the target first detail image in the panoramic image. For example, it can be represented as {(x3'+x3) / 2, (y3'+y3) / 2}.

[0074] As can be seen from the above explanation, as long as there are two successfully matched second detail images, the pixel position of the center point of the target first detail image in the fixed-point panoramic image can be calculated. Moreover, the more successfully matched second detail images there are, the smaller the error of the pixel position corresponding to the center point of the first detail image will be.

[0075] In this embodiment, when there are few successfully matched sub-images in the first detail image, the first detail image can be further subdivided into more sub-images. NCC image matching and pixel position determination in the panoramic image are then performed using these more sub-images. This method of further subdividing the first detail image for matching effectively increases the probability of successful matching, ensuring that the corresponding pixel position can be determined even when the first detail image occupies a small portion of the panoramic image for subsequent calibration. Furthermore, when multiple sub-images in the first detail image are successfully matched, the pixel positions obtained through NCC matching and the calculated pixel positions can be averaged to obtain the final pixel position corresponding to the first detail image. This effectively reduces the error in the pixel position of the center point of the first detail image in the panoramic image. Moreover, when there are many successfully matched sub-images in the first detail image, the pixel position of the center point of the first detail image in the panoramic image can be directly determined, thus improving the efficiency of subsequent calibration.

[0076] The following embodiment uses the example of dividing a first detail image of a target into multiple second detail images along the vertical axis to illustrate a possible implementation of determining multiple sets of target pixel positions in the pixel positions of each first detail image.

[0077] Figure 6 This is the third flowchart illustrating the dual-camera calibration method provided in this embodiment of the invention, as shown below. Figure 6 As shown, step 108 above, "determining multiple sets of target pixel positions based on the pixel positions corresponding to the center points of each first detail image in the panoramic image," includes the following steps: Step 302: Obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the target pixel range based on the maximum and minimum pixel positions.

[0078] In this step, during the rotation of the fixed point up and down (i.e., along the vertical axis), the maximum and minimum pixel positions of the panoramic image corresponding to the fixed point along the vertical axis can also be obtained. Then, the minimum pixel position can be used as the lower limit of the target pixel range, and the maximum pixel position can be used as the upper limit of the target pixel range to obtain the target pixel range.

[0079] Step 304: Among the pixel positions corresponding to the center point of each first detail image in the panoramic image, remove pixel positions that are not within the target pixel range to obtain multiple sets of candidate pixel positions.

[0080] In this step, after obtaining the target pixel range, the ordinate of the pixel position corresponding to the center point of each first detail image can be matched with the target pixel range to determine whether the pixel position corresponding to the center point of each first detail image exceeds the target pixel range.

[0081] If the pixel position corresponding to the center point of a certain first detail image exceeds the target pixel range (for example, the ordinate of the pixel position corresponding to the first detail image is greater than the maximum pixel position, or the ordinate of the pixel position corresponding to the first detail image is less than the minimum pixel position), then the pixel position corresponding to the center point of that first detail image is discarded. In this way, all pixel positions exceeding the target pixel range corresponding to the center points of each first detail image can be discarded. The remaining pixel positions can be recorded as candidate pixel positions. Generally, there are multiple remaining candidate pixel positions, thus obtaining multiple sets of candidate pixel positions.

[0082] Step 306: Determine multiple sets of target pixel positions based on the positions of each candidate pixel.

[0083] In this step, after obtaining multiple sets of candidate pixel positions, these multiple sets of candidate pixel positions can be directly used as multiple sets of target pixel positions; or, the first detail images corresponding to these multiple sets of candidate pixel positions can be obtained first, and then the maximum NCC value obtained by each of these multiple first detail images during NCC image matching can be obtained. Then, among these maximum NCC values, the candidate pixel positions corresponding to the three largest maximum NCC values ​​can be selected as target pixel positions; or other methods can be used to determine multiple sets of target pixel positions, which are not specifically limited here.

[0084] For example, assuming there are 7 preset calibration points and 7 corresponding first detail images, after NCC image matching or image segmentation and NCC image matching in the aforementioned embodiment, the pixel position of the center point of each of these 7 first detail images in the panoramic image and the maximum NCC value can be obtained respectively. At the same time, the latitude and longitude of the preset calibration points corresponding to each of these 7 first detail images can be obtained, forming the data shown in Table 1 below: Table 1

[0085] In this diagram, X1 to X7 represent the x-coordinates of the pixel positions corresponding to the center points of the seven first detail images in the panoramic image, and Y1 to Y7 represent the y-coordinates of the pixel positions corresponding to the center points of the seven first detail images in the panoramic image. L1 to L7 represent the longitudes of the preset calibration points of the seven first detail images, and W1 to W7 represent the latitudes of the preset calibration points of the seven first detail images. M1 to M7 represent the maximum NCC values ​​of each of the seven first detail images.

[0086] The maximum pixel position on the vertical axis of the panoramic image is Ymax, and the minimum pixel position on the vertical axis is 0. It can be seen that the Y-coordinate of the pixel position corresponding to preset calibration point 1 exceeds the maximum pixel position (i.e., Y1>Ymax). Therefore, the center point of the first detail image corresponding to preset calibration point 1 is no longer in the panoramic image and needs to be removed. Similarly, the Y-coordinate of the pixel position corresponding to preset calibration point 6 exceeds the minimum pixel position (i.e., Y6<0). Therefore, the center point of the first detail image corresponding to preset calibration point 6 is also not in the panoramic image and needs to be removed. Likewise, the Y-coordinate of the pixel position corresponding to preset calibration point 7 cannot be calculated, meaning the center point of the first detail image corresponding to preset calibration point 7 may be outside the panoramic image, and this preset calibration point 7 also needs to be removed. Finally, preset calibration points 2-5 remain. These four preset calibration points can be used as candidate calibration points, and their corresponding pixel positions can be used as candidate pixel positions.

[0087] Then, using the M values ​​(M2~M5) of these four preset calibration points, the pixel positions corresponding to the three preset calibration points with the larger M values ​​are selected as the target pixel positions. At the same time, the latitude and longitude of the three selected preset calibration points are used as the latitude and longitude corresponding to the target pixel positions to establish the subsequent mapping relationship and obtain the mapping relationship between pixel positions and latitude and longitude.

[0088] In this embodiment, a mapping relationship is established by removing pixel positions that exceed the pixel range on the vertical axis of the panoramic image from the pixel positions corresponding to each first detail image. This ensures the accuracy of the established mapping relationship and improves the accuracy of the dual-camera calibration.

[0089] During the process of establishing and mapping the relationship, there may be cases where the mapped latitude and longitude exceed the panoramic view. To avoid such problems, the following embodiments propose a scheme to correct the latitude when rotating up and down from a fixed point.

[0090] Figure 7 This is the fourth flowchart illustrating the dual-camera calibration method provided in this embodiment of the invention. Figure 7 As shown, the above method also includes the following steps: Step 402: Obtain the preset calibration points used when capturing each first detail image and the latitude corresponding to each preset calibration point.

[0091] In this step, since the first camera is rotated to the latitude and longitude of each preset calibration point before shooting each first detail image, after obtaining each first detail image, the preset calibration point sampled when shooting the first detail image and its corresponding latitude and longitude can also be obtained accordingly. Then, the latitude used for each first detail image can be found from it.

[0092] Step 404: Obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the maximum and minimum latitudes in the latitudes corresponding to each preset calibration point based on the maximum and minimum pixel positions.

[0093] In this step, as mentioned above, after obtaining the pixel positions corresponding to the center points of each first detail image, the pixel positions exceeding the maximum and minimum pixel positions along the vertical axis of the panoramic image can be eliminated. Simultaneously, the preset calibration points / latitude and longitude of the first detail images corresponding to the eliminated pixel positions are also eliminated. Then, the maximum and minimum latitudes can be determined using the latitude and longitude of the preset calibration points corresponding to the remaining pixel positions. For example, according to the data in Table 1 above, the determined maximum latitude is set as W2, and the minimum latitude is set as W5.

[0094] Step 406: Determine the latitude range corresponding to the minimum and maximum latitudes as the normal latitude range corresponding to the mapping relationship.

[0095] In this step, after determining the maximum and minimum latitudes corresponding to the remaining pixel positions, the latitude range formed by the minimum and maximum latitudes is the standard / normal latitude range of the final established mapping relationship.

[0096] When performing latitude and longitude mapping later, the latitude obtained through the mapping relationship can be corrected according to the standard / normal latitude range determined above. That is, the latitude obtained through the mapping relationship is corrected to the standard / normal latitude range, thereby ensuring the best gun-ball linkage effect.

[0097] In this embodiment, by removing out-of-range pixel positions from each first detail image, the maximum and minimum latitude of the moving point rotation can be obtained, and the standard / normal latitude range of the moving point mapping can be obtained accordingly. In this way, the latitude and longitude of the subsequent mapping can be corrected by the standard / normal latitude range to ensure the best gun-ball linkage effect.

[0098] The calibration apparatus for dual cameras provided by the present invention will be described below. The calibration apparatus for dual cameras described below can be referred to in correspondence with the calibration method for dual cameras described above.

[0099] Figure 8 This is a schematic diagram of the dual-camera calibration device provided in an embodiment of the present invention. See also... Figure 8 As shown, the device may include: The first matching module 510 is used to acquire the first detail image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, and to perform NCC image matching processing on each first detail image and the panoramic image captured by the second camera to determine the first matching result corresponding to each first detail image. The second matching module 520 is used to divide the target first detail image into multiple second detail images according to a first number if the first matching result of the target first detail image is a matching failure, and perform NCC image matching processing on each second detail image and the panoramic image respectively to determine the second matching result corresponding to each second detail image. The pixel position determination module 530 is used to determine the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image that was successfully matched in the second matching result in the panoramic image. The mapping relationship establishment module 540 is used to determine multiple sets of target pixel positions based on the pixel positions corresponding to the center points of each first detail image in the panoramic image, and to establish a mapping relationship between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the first camera rotation based on each target pixel position and the latitude and longitude of the preset calibration points corresponding to each target pixel position.

[0100] In some embodiments, the pixel position determination module 530 is specifically used to determine whether the number of successfully matched second detail images is less than a preset number based on each second matching result; if the number of successfully matched second detail images is less than the preset number, then obtain a second number; the second number is greater than the first number; take the second number as the new first number, return to execute the steps of dividing the target first detail image according to the first number to obtain multiple second detail images of the same number as the first number, and performing NCC image matching processing on each second detail image and the panoramic image respectively to determine the second matching result corresponding to each second detail image, thereby obtaining a new second matching result corresponding to each new second detail image; and determine the pixel position corresponding to the center point of the target first detail image in the panoramic image based on the pixel position corresponding to the center point of the new successfully matched second detail image in the panoramic image.

[0101] Optionally, if the second quantity is five and the number of newly matched second detail images is three, the pixel position determination module 530 is specifically used to calculate the first pixel position of the center point of the target first detail image in the panoramic image based on the pixel positions of the center points of the first two newly matched second detail images in the panoramic image; obtain the pixel position of the center point of the third newly matched second detail image in the panoramic image, and use the pixel position of the third newly matched second detail image as the second pixel position of the center point of the target first detail image in the panoramic image; perform mean processing on the first pixel position and the second pixel position, and determine the obtained mean as the pixel position of the center point of the target first detail image in the panoramic image.

[0102] Optionally, the pixel position determination module 530 is further configured to, if the number of successfully matched second detail images is not less than a preset number, calculate the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the successfully matched second detail image in the panoramic image.

[0103] In some embodiments, the mapping relationship establishment module 540 is specifically used to obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the target pixel range based on the maximum and minimum pixel positions; among the pixel positions corresponding to the center points of each first detail image in the panoramic image, remove pixel positions that are not within the target pixel range to obtain multiple sets of candidate pixel positions; and determine multiple sets of target pixel positions based on each candidate pixel position.

[0104] In some embodiments, the above-described apparatus further includes: The normal latitude range determination module is used to obtain the preset calibration points used when capturing each first detail image and the latitude corresponding to each preset calibration point; obtain the maximum and minimum pixel positions of the panoramic image on the vertical axis, and determine the maximum and minimum latitudes in the latitudes corresponding to each preset calibration point based on the maximum and minimum pixel positions; and determine the latitude range corresponding to the minimum and maximum latitudes as the normal latitude range corresponding to the mapping relationship.

[0105] In some embodiments, before the first matching module 510 acquires the first detailed image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, the device further includes: The preset calibration point determination module is used to obtain the target longitude range and target latitude range that the first camera can move; the center point of the detailed image captured by the first camera when it rotates within the target longitude range and target latitude range is within the panoramic image captured by the second camera; based on the target longitude range and target latitude range, multiple preset calibration points are determined within the target longitude range and target latitude range; each preset calibration point corresponds to one longitude and one latitude.

[0106] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0107] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logic instructions in the memory 630 to execute a dual-camera calibration method. This method includes: acquiring first detail images captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point; performing NCC image matching processing on each first detail image and the panoramic image captured by the second camera to determine a first matching result corresponding to each first detail image; if the first matching result of the target first detail image is a failed match, dividing the target first detail image into multiple second detail images of the same number as the first number, and performing NCC image matching processing on each second detail image and the panoramic image to determine a second matching result corresponding to each second detail image; determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image in the panoramic image where the second matching result is a successful match; determining multiple sets of target pixel positions based on the pixel positions of the center points of each first detail image in the panoramic image, and establishing a mapping relationship between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the first camera's rotation based on each target pixel position and the latitude and longitude of the preset calibration point corresponding to each target pixel position.

[0108] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the dual-camera calibration method provided by the above methods. The method includes: acquiring first detail images captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point; performing NCC image matching processing on each first detail image and the panoramic image captured by the second camera respectively; determining a first matching result corresponding to each first detail image; if the first matching result of the target first detail image is a matching failure, then dividing the target first detail image according to a first number to obtain... Multiple second detail images, the same number as the first number, are generated, and NCC image matching processing is performed on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image. Based on the pixel position of the center point of the successfully matched second detail image in the panoramic image, the pixel position of the center point of the target first detail image in the panoramic image is determined. Based on the pixel position of the center point of each first detail image in the panoramic image, multiple sets of target pixel positions are determined, and based on each target pixel position and the latitude and longitude of the preset calibration point corresponding to each target pixel position, a mapping relationship is established between the pixel position in the panoramic image of the second camera and the latitude and longitude of the rotation of the first camera.

[0110] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program performs the dual-camera calibration method provided by the methods described above. The method includes: acquiring first detail images captured by a first camera when it rotates to the latitude and longitude corresponding to each preset calibration point; performing NCC image matching processing on each first detail image and the panoramic image captured by the second camera respectively to determine a first matching result corresponding to each first detail image; if the first matching result of the target first detail image is a matching failure, then dividing the target first detail image according to a first number to obtain a plurality of second detail images of the same number as the first number. The system firstly analyzes and processes the images, performing NCC image matching on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image. Based on the pixel position of the center point of the successfully matched second detail image in the panoramic image, the system determines the pixel position of the center point of the target first detail image in the panoramic image. Based on the pixel positions of the center points of each first detail image in the panoramic image, the system determines multiple sets of target pixel positions. Based on each target pixel position and the latitude and longitude of the preset calibration point corresponding to each target pixel position, the system establishes a mapping relationship between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the rotation of the first camera.

[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A calibration method for dual cameras, characterized in that, include: The first detail image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point is obtained. Normalized cross-correlation NCC image matching processing is performed on each of the first detail images and the panoramic image captured by the second camera to determine the first matching result corresponding to each of the first detail images. If the first matching result of the target first detail image is a failure, the target first detail image is divided according to the first number to obtain a plurality of second detail images with the same number as the first number. Then, NCC image matching processing is performed on each second detail image and the panoramic image to determine the second matching result corresponding to each second detail image. Based on the pixel position of the center point of the second detail image that is successfully matched in the second matching result in the panoramic image, the pixel position of the center point of the target first detail image in the panoramic image is determined. Based on the pixel position corresponding to the center point of each of the first detail images in the panoramic image, multiple sets of target pixel positions are determined, and based on each target pixel position and the latitude and longitude of the preset calibration point corresponding to each target pixel position, a mapping relationship is established between the pixel position in the panoramic image of the second camera and the latitude and longitude of the rotation of the first camera.

2. The dual-camera calibration method according to claim 1, characterized in that, The step of determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image (which is a successful match in the second matching result) in the panoramic image includes: Based on each of the second matching results, determine whether the number of second detail images that are successfully matched by the second matching result is less than a preset number; If the number of successfully matched second detail images is less than the preset number, then the second number is obtained; if the second number is greater than the first number. The second quantity is used as the new first quantity. The process of dividing the target first detail image according to the first quantity is returned to obtain a plurality of second detail images with the same quantity as the first quantity. NCC image matching processing is performed on each second detail image and the panoramic image respectively to determine the second matching result corresponding to each second detail image. The new second matching result corresponding to each new second detail image is obtained. Based on the new second matching result, which is the pixel position of the center point of the new second detail image that is successfully matched in the panoramic image, the pixel position of the center point of the target first detail image in the panoramic image is determined.

3. The dual-camera calibration method according to claim 2, characterized in that, If the second quantity is five, the number of successfully matched new second detail images is three. The step of determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the successfully matched new second detail image in the panoramic image according to the new second matching result includes: Based on the pixel positions in the panoramic image corresponding to the center points of the two newly matched second detail images, calculate the first pixel position in the panoramic image corresponding to the center point of the target first detail image. Obtain the pixel position in the panoramic image corresponding to the center point of the third successfully matched new second detail image, and use the pixel position corresponding to the third successfully matched new second detail image as the second pixel position in the panoramic image corresponding to the center point of the target first detail image; The first pixel position and the second pixel position are averaged, and the obtained average is determined as the pixel position of the center point of the target first detail image in the panoramic image.

4. The dual-camera calibration method according to claim 2, characterized in that, The step of determining the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image (which is a successful match in the second matching result) in the panoramic image further includes: If the number of successfully matched second detail images is not less than the preset number, then the pixel position of the center point of the target first detail image in the panoramic image is calculated based on the pixel position of the center point of the successfully matched second detail image in the panoramic image.

5. The dual-camera calibration method according to any one of claims 1 to 4, characterized in that, The step of determining multiple sets of target pixel positions based on the pixel positions corresponding to the center points of each of the first detail images in the panoramic image includes: Obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the target pixel range based on the maximum and minimum pixel positions; In the pixel position corresponding to the center point of each of the first detail images in the panoramic image, pixel positions that are not within the target pixel range are eliminated to obtain multiple sets of candidate pixel positions; The positions of the multiple sets of target pixels are determined based on the positions of each candidate pixel.

6. The dual-camera calibration method according to any one of claims 1 to 4, characterized in that, The method further includes: Acquire the preset calibration points used when capturing each of the first detail images and the latitude corresponding to each preset calibration point; Obtain the maximum and minimum pixel positions of the panoramic image along the vertical axis, and determine the maximum and minimum latitudes in the latitudes corresponding to each preset calibration point based on the maximum and minimum pixel positions. The latitude range corresponding to the minimum latitude and the maximum latitude is determined as the normal latitude range corresponding to the mapping relationship.

7. The dual-camera calibration method according to any one of claims 1 to 4, characterized in that, Before acquiring the first detailed image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, the method further includes: Obtain the target longitude range and target latitude range that the first camera can move within; the center point of the detailed image captured by the first camera when it rotates within the target longitude range and the target latitude range is within the panoramic image captured by the second camera; Based on the target longitude range and the target latitude range, a plurality of preset calibration points are determined within the target longitude range and the target latitude range; each preset calibration point corresponds to one longitude and one latitude.

8. A calibration device for dual cameras, characterized in that, include: The first matching module is used to acquire the first detail image captured by the first camera when it rotates to the latitude and longitude corresponding to each preset calibration point, and to perform normalized cross-correlation NCC image matching processing on each of the first detail images and the panoramic image captured by the second camera to determine the first matching result corresponding to each of the first detail images. The second matching module is used to divide the target first detail image into multiple second detail images according to a first number if the first matching result of the target first detail image is a matching failure, and to perform NCC image matching processing on each second detail image and the panoramic image respectively to determine the second matching result corresponding to each second detail image. A pixel position determination module is used to determine the pixel position of the center point of the target first detail image in the panoramic image based on the pixel position of the center point of the second detail image in the panoramic image, which is a successfully matched second detail image. The mapping relationship establishment module is used to determine multiple sets of target pixel positions based on the pixel positions corresponding to the center points of each of the first detail images in the panoramic image, and to establish a mapping relationship between the pixel positions in the panoramic image of the second camera and the latitude and longitude of the rotation of the first camera based on each of the target pixel positions and the latitude and longitude of the preset calibration points corresponding to each of the target pixel positions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the dual-camera calibration method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dual-camera calibration method as described in any one of claims 1 to 7.

Images