A multi-camera intrinsic parameter calibration method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a multi-camera intrinsic parameter calibration method and device, electronic equipment and storage medium, relates to the field of artificial intelligence, and particularly relates to the field of automatic driving and camera calibration. The specific implementation scheme is: obtaining images of a calibration board at different positions collected by multiple cameras at the same time, and filtering incomplete corner point images to obtain target images; the focal lengths and models of the cameras are the same, and the distance between any two cameras is less than a preset distance threshold; and the intrinsic parameters of the cameras are determined based on the target images collected by the cameras. According to the embodiment of the present disclosure, multiple cameras of the same model and focal length are fixed at similar positions, the pose of the calibration board is continuously changed, the simultaneous collection of calibration board images required by each camera can be realized, the camera image collection efficiency is improved, and then the multi-camera intrinsic parameter calibration efficiency is improved. In addition, by filtering the images, the corner point integrity of the checkerboard calibration board in the target image is improved, and then the calibration success rate and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the fields of automatic driving and camera calibration. BACKGROUND

[0002] As an important sensor in environmental perception, cameras are widely used in various high-precision scenarios. In order to obtain the position information of the target contained in the image from the image collected by the camera, the camera needs to be calibrated before use. SUMMARY

[0003] The present disclosure provides a multi-camera intrinsic parameter calibration method and device, electronic equipment and storage medium to improve the efficiency of multi-camera intrinsic parameter calibration.

[0004] According to an aspect of the present disclosure, a multi-camera intrinsic parameter calibration method is provided, comprising:

[0005] obtaining candidate images of a checkerboard calibration board collected by multiple cameras at different positions; the multiple cameras have the same focal length and model; the distance between each camera is less than a preset distance threshold;

[0006] detecting the corner points of each candidate image to obtain the number of corner points contained in each candidate image;

[0007] filtering the candidate images with incomplete corner points according to the number of corner points contained in each candidate image and the actual number of corner points in the checkerboard calibration board to obtain target images;

[0008] determining the intrinsic parameters of each camera based on the target images collected by the camera to obtain the target intrinsic parameter calibration results of the camera.

[0009] According to another aspect of the present disclosure, a multi-camera intrinsic parameter calibration device is provided, comprising:

[0010] a candidate image acquisition module configured to obtain candidate images of a checkerboard calibration board collected by multiple cameras at different positions; the multiple cameras have the same focal length and model; the distance between each camera is less than a preset distance threshold;

[0011] a corner point detection module configured to detect the corner points of each candidate image to obtain the number of corner points contained in each candidate image;

[0012] a target image acquisition module configured to filter the candidate images with incomplete corner points according to the number of corner points contained in each candidate image and the actual number of corner points in the checkerboard calibration board to obtain target images;

[0013] The intrinsic parameter calibration module is used to determine the intrinsic parameters of each camera based on the target image acquired by the camera, and obtain the target intrinsic parameter calibration result of the camera.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the multi-camera intrinsic parameter calibration methods described above.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform any of the multi-camera intrinsic parameter calibration methods described above.

[0019] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the multi-camera intrinsic parameter calibration methods described above.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a schematic diagram of a first embodiment of the multi-camera intrinsic parameter calibration method provided in this disclosure;

[0023] Figure 2 This is a schematic diagram of a chessboard calibration board;

[0024] Figure 3 This is a schematic diagram illustrating the method for obtaining target intrinsic parameter calibration results in this disclosure;

[0025] Figure 4 This is a schematic diagram of a second embodiment of the multi-camera intrinsic parameter calibration method provided in this disclosure;

[0026] Figure 5 This is a schematic diagram of a first embodiment of the multi-camera intrinsic parameter calibration device provided in this disclosure;

[0027] Figure 6 This is a block diagram of an electronic device used to implement the multi-camera intrinsic parameter calibration method of the embodiments of this disclosure. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] Cameras, as a crucial component of environmental perception, project objects from the real world onto their two-dimensional imaging plane. However, relying solely on a two-dimensional image plane camera makes it difficult to acquire three-dimensional information about targets such as vehicles in the real environment. Therefore, in practical applications, such as autonomous driving environmental perception, the three-dimensional information of vehicles and other targets in the real world is typically obtained based on the transformation relationship between the image coordinate system and the real-world coordinate system, using the pixel positions of these targets in the image. Camera intrinsic parameter calibration aims to find the correspondence between the image and the real world. To ensure the accuracy of the three-dimensional information of the determined targets, intrinsic parameter calibration is crucial before camera use, especially before cameras are installed in autonomous vehicles. Camera intrinsic parameters typically include the camera's focal length f, principal point coordinates (cx, cy), and distortion values, etc. The principal point refers to the intersection of the camera's optical axis and the imaging plane within the camera.

[0030] Currently, a commonly used intrinsic parameter calibration method is to calibrate checkerboard images using Zhang's calibration method to obtain camera intrinsic parameters. In multi-camera intrinsic parameter calibration scenarios, acquiring checkerboard images typically involves multiple image acquisitions for each individual camera, resulting in low acquisition efficiency and consequently low multi-camera intrinsic parameter calibration efficiency.

[0031] To improve the efficiency of multi-camera intrinsic parameter calibration, this disclosure provides a multi-camera intrinsic parameter calibration method, apparatus, electronic device, and storage medium. The multi-camera intrinsic parameter calibration method provided in this disclosure is first described by way of example:

[0032] The multi-camera intrinsic parameter calibration method disclosed herein can be applied to any electronic device with multi-camera intrinsic parameter calibration function. Such electronic devices can be computers, servers, mobile terminals, etc.

[0033] like Figure 1 As shown, Figure 1 This is a schematic diagram of a first embodiment of the multi-camera intrinsic parameter calibration method provided in this disclosure, which may include the following steps:

[0034] Step S101: Acquire candidate images of the checkerboard calibration board at different positions simultaneously captured by multiple cameras;

[0035] Step S102: Perform corner detection on each candidate image to obtain the number of corners contained in each candidate image;

[0036] Step S103: Based on the number of corner points contained in each candidate image and the actual number of corner points in the checkerboard calibration board, filter out candidate images with incomplete corner points to obtain each target image;

[0037] Step S104: For each camera, determine the intrinsic parameters of the camera based on the target image acquired by the camera, and obtain the target intrinsic parameter calibration result of the camera.

[0038] The sharpness of the calibration board varies depending on the focal length and imaging distance. However, when cameras of the same model and focal length acquire images of the calibration board, the sharpness distance between the calibration board and the camera is often the same. By applying this embodiment, multiple cameras of the same model and focal length are fixed in close proximity. By continuously changing the position of the calibration board, the images of the calibration board required by each camera can be acquired simultaneously, improving camera image acquisition efficiency and thus improving the efficiency of multi-camera intrinsic parameter calibration. Furthermore, Zhang's calibration method typically requires extracting corner points from the acquired checkerboard calibration board image for camera intrinsic parameter calibration. If the checkerboard corner points in the image are incomplete, it will lead to reduced calibration accuracy or even calibration failure, significantly reducing calibration efficiency. By applying this embodiment, by performing corner point detection on the checkerboard calibration board image acquired by the camera and filtering out images with incomplete corner points, the completeness of the extracted checkerboard corner points in the image can be improved, thereby increasing the calibration success rate and accuracy.

[0039] The following is an exemplary description of steps S101-S104 above:

[0040] In step S101, multiple cameras of the same model and focal length can be used to acquire images of the same calibration plate at different locations. The number of cameras can be determined according to actual needs, such as 8, 10, etc. These multiple cameras of the same model and focal length are fixed in close proximity. For example, the cameras can be placed on the same horizontal line, and the distance between any two cameras does not exceed a preset distance threshold. The preset distance threshold can be set according to actual needs, such as 80cm, 100cm, etc. Alternatively, a camera placement area can be pre-defined on the ground, and each camera can be placed within that area. The camera placement area can be set according to actual needs, such as a circular area with a radius of 1m, a square area with a side length of 1.5m, etc.

[0041] After fixing the above multiple cameras, the position of the calibration plate can be moved multiple times. After each movement of the calibration plate, the above multiple cameras can be used to simultaneously capture images of the calibration plate, so that each camera can capture images of the calibration plate at different positions in the camera viewfinder.

[0042] The above-mentioned simultaneous acquisition of images of the calibration board by multiple cameras refers to the acquisition of images of the calibration board at the same position by multiple cameras. This can be done at the same moment or within a preset time window, such as within 5 seconds or 10 seconds.

[0043] The calibration board mentioned above can be a solid circular array calibration board, a checkerboard calibration board, etc. Moving the calibration board refers to changing the distance between the calibration board and the camera, as well as the orientation of the calibration board relative to the camera, so that the calibration board appears in as many positions as possible in the image.

[0044] As one embodiment of this disclosure, a chessboard calibration board can be selected for multi-camera intrinsic parameter calibration. The chessboard calibration board is a calibration board with a chessboard pattern. Figure 2 As shown, Figure 2 A schematic diagram of a chessboard calibration board.

[0045] Each of the aforementioned cameras can acquire multiple images of the checkerboard calibration board used for calibration. The checkerboard calibration board images acquired by each camera for calibration will be referred to as target images below. The number of target images can be preset according to actual needs, such as 40, 60, etc.

[0046] In one embodiment of this disclosure, the checkerboard calibration board images captured by each camera can be filtered to obtain the target image used for calibration.

[0047] Specifically, in step S101, the aforementioned candidate images refer to the full range of calibration board images acquired by the unfiltered cameras. The number of candidate images acquired by each camera can be determined according to actual needs, such as 60, 80, etc.

[0048] Corner points are the intersections of the two sides of each square in the checkerboard calibration board. In step S102, algorithms such as Harris, FAST, and SURF can be used to detect corner points in the candidate image. It is then determined whether the number of detected corner points matches the actual number of corner points in the checkerboard calibration board. If the number of detected corner points does not match the actual number of corner points in the checkerboard calibration board, it indicates that the checkerboard image in the candidate image is incomplete; therefore, the candidate image can be filtered.

[0049] In one embodiment of this disclosure, candidate images with corner points located at image edges can also be filtered. Corner detection typically identifies pixels with large grayscale value changes as corner points. Non-corner areas with large grayscale value changes are more likely to appear at image edges, easily leading to false detections—that is, non-corner points are detected as corner points. Therefore, candidate images with detected corner points at image edges can also be filtered to reduce the possibility of false detections and further improve calibration accuracy.

[0050] During camera intrinsic parameter calibration, it is generally desirable for the checkerboard calibration plate to appear in different positions within the image in various orientations. This prevents the checkerboard calibration plate from being positioned too uniformly in the acquired images, which could lead to local optima and reduce the overall accuracy of the intrinsic parameter calibration. Therefore, in one embodiment of this disclosure, the position of the checkerboard calibration plate in each candidate image can be detected, and images with a relatively concentrated checkerboard calibration plate position can be filtered out. This ensures that the checkerboard calibration plate appears in different positions within the image as much as possible, thereby improving the accuracy of the intrinsic parameter calibration.

[0051] As a specific implementation method, the following steps can be used to filter candidate images where the positions of the checkerboard calibration board are relatively concentrated:

[0052] Step S111: For each candidate image, determine the centroid of each chessboard corner point in the candidate image;

[0053] Step S112: For each camera, determine a preset number of candidate images whose centroid positions fall within a preset area as target images.

[0054] In step S111, the centroids of the checkerboard corner points are obtained based on the coordinates of each checkerboard corner point. For example, the average coordinate of each checkerboard corner point can be used as the centroid coordinates of the checkerboard corner points in the candidate image. Of course, other methods can also be used to obtain the centroids of the checkerboard corner points.

[0055] In step S112, each candidate image can be divided into a 3x3 grid, and the number of candidate images whose centroids fall within each grid can be determined. For each 3x3 grid, a preset number of candidate images whose centroids fall within that grid are selected as candidate images. This preset number can be set separately for each 3x3 grid, or it can be set uniformly for all 3x3 grids. For example, it could be 5, 6, etc. Of course, regions can also be divided for each candidate image using other methods; this disclosure does not specifically limit this approach.

[0056] In one embodiment, the similarity of the checkerboard calibration board positions among the candidate images captured by each camera can also be determined, and candidate images with similar checkerboard calibration board positions can be filtered.

[0057] Specifically, for each camera, candidate images captured by that camera can be arranged in chronological order. For each candidate image, the pixel difference between any position on the checkerboard calibration board in that candidate image and the same position on the checkerboard calibration board in the previous candidate image is calculated. If the pixel difference is less than a preset pixel difference threshold, either the previous image or the current image can be discarded. This further ensures that the checkerboard calibration board appears in as many positions as possible within the image. The pixel difference threshold can be set according to actual needs, such as 20 pixels, 30 pixels, etc.

[0058] During image acquisition, the movement of the camera or calibration board may cause the acquired image to be blurry, resulting in a large error in the extracted checkerboard calibration board corner points and thus reducing the accuracy of the calibration results. Therefore, in one embodiment of this disclosure, images of more complex modules in the candidate images can be filtered to improve the accuracy of corner point extraction and calibration.

[0059] In one specific implementation, the sharpness of each candidate image can be obtained; and the candidate image with a sharpness greater than a preset sharpness threshold can be used as the target image.

[0060] When obtaining the sharpness of each candidate image, the grayscale value of each pixel in the image after processing by the Sobel operator can be obtained. The larger the grayscale value, the higher the sharpness of the corresponding pixel. Then, statistical values ​​of the grayscale values ​​of each pixel can be selected to measure the image sharpness, such as the average or median grayscale values. Candidate images with a sharpness greater than a preset sharpness threshold are selected as the target images.

[0061] The aforementioned preset sharpness threshold can specifically be a preset grayscale value threshold. This grayscale threshold can be selected according to actual needs, and specifically, it can be obtained based on statistical analysis of a large number of calibration images.

[0062] Alternatively, after obtaining the grayscale images of each candidate image, the sharpness of the candidate image can be obtained using a sharpness calculation formula. For example, the sharpness calculation formula could be:

[0063]

[0064] In this formula, D represents image sharpness, and f(x,y) represents the grayscale value of pixel (x,y) in the image. Images with a sharpness greater than a preset sharpness threshold can then be used as target images. This sharpness threshold can be set according to actual needs.

[0065] After filtering the candidate images, the number of target images obtained may be less than a preset threshold. Therefore, for cameras where the number of target images does not reach the preset threshold, candidate images of the checkerboard calibration board at different positions captured by the camera can be reacquired; the step of performing corner detection on each candidate image to obtain the number of corners contained in each candidate image can be returned, and the above filtering can be performed on the acquired images until the number of target images reaches the preset threshold. In this way, camera intrinsic parameter calibration based on sufficient target images can ensure the stability and accuracy of the intrinsic parameter calibration results.

[0066] In related technologies, most images used for calibration are obtained through rough manual screening. In mass production processes, this operation not only affects production efficiency but also makes it difficult to guarantee the quality of the acquired images. However, by applying the embodiments of this disclosure, using electronic devices to automatically check and manage the quality of images acquired by the camera, and only inputting images that pass the check into the subsequent calibration process, calibration failures or errors caused by image quality can be effectively avoided, thereby improving calibration accuracy and success rate, and increasing calibration efficiency.

[0067] Once the target images acquired by each camera are obtained, the intrinsic parameters of the cameras can be calibrated based on the target images.

[0068] Specifically, this disclosure uses Zhang's calibration method for camera intrinsic parameter calibration. Zhang's calibration method involves detecting feature points in the target image acquired by each camera, typically the corner points of a checkerboard calibration board. Corner points are the intersections of the two sides of each checkerboard grid. Then, based on the spatial coordinates of each feature point and its pixel coordinates in the target image, the mapping relationship from the world coordinate system to the image coordinate system is obtained. Based on this mapping relationship, the homography matrix from the world coordinate system to the image coordinate system is obtained. The homography matrix is ​​the transformation matrix from a pixel in one image to a pixel in another image. The intrinsic parameter matrix can then be obtained based on this homography matrix. By jointly solving the intrinsic parameter matrices obtained from multiple target images, the intrinsic parameter calibration results can be obtained. These intrinsic parameter calibration results include camera focal length calibration results and principal point calibration results, etc. After obtaining the above intrinsic parameter calibration results, the above intrinsic parameter calibration results can be written into the camera firmware so that the three-dimensional stereo information of targets such as vehicles in the images acquired by the camera can be obtained based on the above intrinsic parameter calibration results during subsequent camera use.

[0069] Cameras typically record a focal length reference value (f0) and principal point reference values ​​(cx0, cy0, etc.) at the factory. If the calibration result is close to the reference values, the calibration is considered successful; if it deviates significantly, the calibration is considered a failure. Due to factors such as lens error, installation error, checkerboard image quality, and the angle and position of the calibration board, calibration failures are very common. Cameras that fail to calibrate are often considered to have quality issues and are unsuitable for use in autonomous vehicles.

[0070] To accurately determine the specific reasons for internal parameter calibration failure, i.e., to determine whether there is a problem with the camera equipment itself, multiple image acquisitions and calibrations are often used for comprehensive analysis. Related technologies primarily rely on manual evaluation of single calibration results to decide whether to perform the next calibration. However, this method requires significant manual labor, is extremely inefficient, and the calibration results are greatly affected by human factors, resulting in low reliability.

[0071] In this embodiment of the disclosure, the electronic device can automatically judge the camera's intrinsic parameter calibration result determined based on the target image according to the pre-set rules, thereby determining the camera's target intrinsic parameter calibration result. Hereinafter, the intrinsic parameter calibration result determined based on the target image of the camera is referred to as the current intrinsic parameter calibration result.

[0072] Specifically, such as Figure 3 As shown, the target intrinsic parameter calibration results of each camera can be obtained based on the current intrinsic parameter calibration results of each camera through the following steps.

[0073] Step S301: Determine the number of calibrations to be performed on each camera's intrinsic parameters. If the number of calibrations is less than a preset lower threshold, proceed to step S302; if the number of calibrations is greater than a preset upper threshold, proceed to step S307; if the number of calibrations is between the preset lower threshold and the preset upper threshold, proceed to step S310.

[0074] Step S302: Determine whether the current internal parameter calibration result meets the first preset parameter range; if it does, proceed to step S303; if it does not, proceed to step S304.

[0075] Step S303: Determine the current intrinsic parameter calibration result as the target intrinsic parameter calibration result of the camera.

[0076] Step S304: Determine whether the current internal parameter calibration result meets the second preset parameter range; if it does, proceed to step S305; if it does not, proceed to step S306.

[0077] Step S305: Mark the current internal parameter calibration result as pending and output a prompt message for recalibration.

[0078] Step S306: Determine that the camera has failed calibration.

[0079] Step S307: Determine whether the difference between the current internal parameter calibration result and each historical internal parameter calibration result is less than the preset difference threshold; if it is less, proceed to step S308; if it is not less, proceed to step S309.

[0080] Step S308: Use the current intrinsic parameter calibration result and the statistical values ​​of each historical intrinsic parameter calibration result as the target intrinsic parameter calibration result of the camera.

[0081] Step S309: Determine that the camera has failed calibration.

[0082] Step S310: Output a recalibration prompt message.

[0083] The following is an exemplary description of steps S301-S310 above:

[0084] In one possible embodiment, after obtaining the current intrinsic parameter calibration result for each camera, the camera identifier, the current intrinsic parameter calibration result, and the current time information can be stored accordingly. The camera identifier and time information can uniquely identify each intrinsic parameter calibration result, facilitating subsequent determination of the camera's target intrinsic parameter calibration result based on multiple calibration results. Specifically, the camera identifier is the camera's SN code (serial number). In the following text, the intrinsic parameter calibration results recorded before this calibration are referred to as historical intrinsic parameter calibration results.

[0085] Therefore, in step S301, the historical intrinsic parameter calibration results of the camera can be obtained from the recorded calibration results based on the camera identifier. The number of calibrations for the camera is then determined based on the number of calibration results. For example, if the camera has no historical intrinsic parameter calibration results, the number of calibrations for the camera is 1. If the camera has two historical intrinsic parameter calibration results, the number of calibrations for the camera is 3.

[0086] Of course, in one possible embodiment, a calibration count record table can also be maintained to store the correspondence between camera identifiers and camera calibration counts. After each calibration is completed and the current intrinsic parameter calibration result is obtained, the calibration count for the corresponding camera is incremented by 1. Therefore, in step S301, the camera calibration count can be directly obtained from the aforementioned calibration count record table based on the camera identifier.

[0087] If the number of camera calibration attempts is less than the preset lower limit threshold, then steps S302-S306 above can be executed. The lower limit threshold can be set according to actual needs, such as 2, 3, etc.

[0088] As mentioned above, the camera's intrinsic parameter calibration results include the camera's focal length f calibration result and the camera's principal point coordinate calibration result. At the factory, the camera records the camera's focal length reference value f0 and principal point reference values ​​cx0, cy0. If the calibration result is near the reference values, the calibration is considered successful. If it differs significantly from the reference values, it is necessary to consider whether it is a problem with the camera itself, the position of the calibration plate, or other factors. Therefore, in step S302, the first preset parameter range can be determined based on the camera's focal length and principal point coordinate reference values. Specifically, the aforementioned first preset parameter range can be:

[0089]

[0090] Where ε is the focal length floating range threshold, and γ is the principal point floating range threshold. Both the focal length floating range threshold and the principal point floating range threshold can be set according to actual needs.

[0091] If the camera's current intrinsic parameter calibration result meets the first preset parameter range mentioned above, then it can be determined that the camera has passed calibration and determined the current intrinsic parameter calibration result as the camera's target intrinsic parameter calibration result. If the camera's current intrinsic parameter calibration result does not meet the first preset parameter range mentioned above, then it can be determined whether the camera needs to be recalibrated.

[0092] Inaccurate camera focal length calibration results are usually caused by factors such as lens error or installation error. Therefore, as a specific implementation method, if the focal length calibration result in the current internal parameter calibration does not meet the focal length range in the first preset parameter range mentioned above, it can be determined that the camera calibration has failed and the camera is unusable.

[0093] If the focal length calibration result meets the focal length range in the first preset parameter range, it can be determined whether the principal point calibration result meets the principal point range in the first preset parameter range. If the principal point meets the principal point range, it can be determined that the current intrinsic parameter calibration result of the camera meets the first preset parameter range. If the principal point does not meet the principal point range, it can be determined that the current intrinsic parameter calibration result does not meet the first preset parameter range.

[0094] In step S304, the second preset parameter range includes the first preset parameter range. That is, if the current internal parameter calibration result meets the first preset parameter range, then the current internal parameter calibration result must also meet the second preset parameter range.

[0095] As one specific implementation, the range of the second preset parameter mentioned above can be:

[0096]

[0097] Of course, the coefficients before ε and γ can also be other values, and can be set according to actual needs. If the camera principal point calibration result does not meet the first preset parameter range but meets the second preset parameter range, it may be due to factors such as the calibration board position and the quality of the checkerboard image. Therefore, if the current intrinsic parameter calibration result meets the second preset parameter range, the current intrinsic parameter calibration result can be recorded, and a recalibration prompt message can be output. For example, a "recalibrate" message can be displayed on an electronic device to recalibrate the camera, eliminating the influence of factors such as the calibration board position and image quality. Specifically, the process of recalibrating the camera involves re-acquiring target images of the calibration board at different positions captured by the camera, and performing intrinsic parameter calibration on the camera based on the re-acquisitioned target images.

[0098] If the camera master point does not meet the above-mentioned second preset parameter range, it can be determined that there is a problem with the camera itself and the camera is unusable.

[0099] In this way, by setting the first preset parameter range and the second preset parameter range, the current intrinsic parameter calibration result of the camera is judged, the possible causes of the problem in the current calibration result are determined, and whether the next calibration is needed is required. This eliminates the need for manual judgment based on one's own experience, thus improving the efficiency of camera calibration.

[0100] In step S301, if the number of calibrations of the camera exceeds a preset upper limit threshold, then steps S307-S309 can be executed. The preset upper limit threshold can be the same as or different from the preset lower limit threshold. For example, the upper limit threshold can be 3, 4, etc.

[0101] In step S307, the differences between the current intrinsic parameter calibration result and the focal length calibration result and principal point coordinate calibration result from each historical calibration result can be calculated to obtain multiple focal length differences and multiple principal point coordinate differences. It is then determined whether all of these multiple focal length differences are less than a focal length difference threshold, and whether all of these multiple principal point coordinate differences are less than a preset principal point coordinate difference threshold. If they are all less than this threshold, it indicates that the multiple calibration results are relatively stable; otherwise, they are unstable. Alternatively, the average value of each focal length difference and the average value of the principal point coordinate differences can be calculated, and it can be determined whether the average value of each difference is less than the corresponding preset difference threshold. If they are both less than this threshold, it indicates that the multiple calibration results are relatively stable; otherwise, they are unstable. The above difference thresholds can be set according to actual needs.

[0102] If the calibration results are stable after multiple calibrations, the statistical value of the current and historical intrinsic parameter calibration results can be used as the target intrinsic parameter calibration result. This statistical value can be the average, median, etc. For example, the average of the focal length calibration results in each calibration result can be used as the target focal length, and the average of the principal point coordinate calibration results in each calibration result can be used as the target principal point coordinates. Alternatively, the calibration result corresponding to the median of the focal length calibration results in each intrinsic parameter calibration result can be selected as the target intrinsic parameter calibration result; this disclosure does not specifically limit this choice.

[0103] If the calibration results are unstable after multiple calibrations, it can be determined that there is a problem with the camera and the camera is unusable.

[0104] In related technologies, multiple calibrations lack interdependence, requiring manual comparison of calibration parameter results for each calibration. This makes it difficult to quantify the differences and stability of each calibration result, and to determine whether repeated calibrations are caused by camera defects. Consequently, the efficiency of calibration personnel is greatly reduced, and calibration labor costs rise sharply. By recording the camera's intrinsic parameter calibration results and comprehensively analyzing the current and historical results, the cause of camera anomalies can be determined more quickly, such as whether it is due to the camera itself or the calibration plate position, thus improving camera calibration efficiency.

[0105] For cameras whose calibration counts fall between a preset lower threshold and a preset upper threshold, the calibration results are considered insufficient, resulting in low reliability. Therefore, for these cameras, the current intrinsic parameter calibration result can be recorded, and a recalibration prompt can be output to improve the reliability of the calibration results. For example, if both the preset upper and lower thresholds are 2, a recalibration prompt can be directly output for cameras with 2 calibration counts. If the preset upper threshold is 5 and the preset lower threshold is 2, a recalibration prompt can be directly output for cameras with calibration counts in the range [2,5].

[0106] As can be seen, by applying the embodiments of this disclosure, the combination of serial number (SN) and timestamp is used to manage multiple calibration results of multiple cameras, and the calibration results are automatically analyzed according to preset judgment rules to draw a final conclusion, thereby improving calibration efficiency and accuracy, and increasing the reliability of the results.

[0107] like Figure 4 As shown, Figure 4 A specific example of the multi-camera intrinsic parameter calibration method provided in this disclosure is illustrated in the following diagram, which may include the following three parts:

[0108] Image Acquisition Management: Multiple cameras of the same model and focal length simultaneously acquire images, and the acquired images are evaluated, including whether the corner points are complete, whether the image is clear, whether the checkerboard pattern is in the right position, and whether the number of target images is sufficient, until enough target images are acquired for each camera for calibration.

[0109] Camera SN Acquisition and Calibration: This includes intrinsic parameter calibration, acquiring the camera SN, acquiring the system timestamp, and saving the calibration results. Specifically, the calibration results are saved in a unique path composed of the SN and the system timestamp.

[0110] Calibration parameter management: Analyzes calibration results and provides corresponding conclusions, enabling management of intrinsic parameter results from multiple cameras and multiple calibrations. Specifically, it first checks the calibration count N for each camera. If the calibration count is 1, it determines whether the focal length f in the intrinsic parameter calibration results meets the first preset parameter range. Figure 4 If the first preset parameter range is satisfied, then it is determined whether the principal point coordinates (cx, cy) satisfy the first preset parameter range. If not, it is determined that the camera calibration has failed and there is a problem with the camera. If the principal point satisfies the first preset parameter range, then it is determined that the camera calibration is successful and the current intrinsic parameter calibration result is written to the camera firmware; if the principal point does not satisfy the first preset parameter range, then it is determined whether the principal point satisfies the second preset parameter range. Figure 4 If the second range is not met, the calibration is determined to have failed and there is a problem with the camera. If it is met, the current calibration result is set to pending, and a prompt to recalibrate is output. The above-mentioned first preset parameter range is: Where ε and γ are both preset constants. The range of the second preset parameter is:

[0111] If the total number of calibration attempts is 2, a recalibration prompt message will be output. If the total number of calibration attempts is 3, it will determine whether the three calibration results are stable. Specifically, it will determine whether the difference between the focal length and principal point in the three calibration results exceeds the preset difference threshold. If it exceeds the threshold, the calibration result is unstable, the calibration is determined to have failed, and there is a problem with the camera. If it does not exceed the threshold, the calibration result is stable, the calibration is determined to have succeeded, and the set of calibration results with the middle focal length value will be selected and written to the camera firmware.

[0112] By applying the embodiments of this disclosure, multiple cameras are used to simultaneously acquire data in multi-camera intrinsic parameter calibration scenarios, improving image acquisition efficiency and thus calibration efficiency. Secondly, by filtering the acquired images, calibration failures caused by issues such as image quality, incomplete corner points, and overly simplistic checkerboard patterns are reduced, improving calibration success rate and accuracy. Furthermore, by using a combination of serial numbers and timestamps to manage multiple cameras and multiple calibration results, automatic comprehensive analysis is performed to arrive at the final conclusion, improving calibration efficiency and accuracy while increasing the reliability of the results.

[0113] According to embodiments of this disclosure, a multi-camera intrinsic parameter calibration device is also provided, such as... Figure 5 As shown, the above-mentioned device may include:

[0114] The candidate image acquisition module 501 is used to acquire candidate images of the checkerboard calibration board at different positions simultaneously captured by multiple cameras; the multiple cameras have the same focal length and model; and the distance between each camera is less than a preset distance threshold.

[0115] The corner detection module 502 is used to perform corner detection on each of the candidate images to obtain the number of corners contained in each candidate image;

[0116] The target image acquisition module 503 is used to filter out candidate images with incomplete corners based on the number of corners contained in each candidate image and the actual number of corners in the chessboard calibration board, and obtain each target image.

[0117] The intrinsic parameter calibration module 504 is used to determine the intrinsic parameters of each camera based on the target image acquired by the camera, and obtain the target intrinsic parameter calibration result of the camera.

[0118] In one possible embodiment, the target image acquisition module described above is further configured to:

[0119] For each of the candidate images, determine the centroid of each chessboard corner point in the candidate image;

[0120] For each camera, a preset number of candidate images whose centroid positions fall within a preset area are determined as target images.

[0121] In one possible embodiment, the target image acquisition module described above is further configured to:

[0122] Obtain the sharpness of each candidate image;

[0123] Candidate images with a resolution greater than a preset resolution threshold are selected as target images.

[0124] In one possible embodiment, the target image acquisition module described above is further configured to:

[0125] For cameras where the number of target images does not reach a preset threshold, reacquire candidate images of the checkerboard calibration board at different positions captured by the camera; return to the step of performing corner detection on each candidate image to obtain the number of corners contained in each candidate image.

[0126] In one possible embodiment, determining the intrinsic parameters of each camera based on the target image acquired by the camera to obtain the target intrinsic parameter calibration result of the camera includes:

[0127] For each camera, the intrinsic parameters of the camera are determined based on the target image acquired by the camera, and the current intrinsic parameter calibration result of the camera is obtained;

[0128] Determine the number of calibrations to be performed on the intrinsic parameters of each camera;

[0129] The first camera whose calibration count is less than a preset lower limit threshold is identified;

[0130] For a first camera whose current intrinsic parameter calibration result is within a first preset parameter range, the current intrinsic parameter calibration result of the first camera is determined as the target intrinsic parameter calibration result of the first camera;

[0131] For the first camera whose current intrinsic parameter calibration result is within the second preset parameter range, record the current intrinsic parameter calibration result of the first camera, and return the step of acquiring the target image of the calibration board at different positions simultaneously acquired by multiple cameras; the second preset range includes the first preset range;

[0132] For the first camera whose current internal parameter calibration result is not within the range of the second preset parameters, it is determined that the first camera has failed calibration.

[0133] In one possible embodiment, the aforementioned intrinsic parameter calibration module is further used for:

[0134] The second camera is identified as having a calibration count greater than a preset upper limit threshold.

[0135] For the second camera, obtain the difference between the current intrinsic parameter calibration result and the historical calibration result of the second camera;

[0136] If each of the aforementioned differences is less than a preset difference threshold, then the current intrinsic parameter calibration result and the statistical value of each historical calibration result are used as the target intrinsic parameter calibration result of the second camera.

[0137] In one possible embodiment, the aforementioned intrinsic parameter calibration module is further used for:

[0138] For the third camera whose calibration count is between a preset lower limit threshold and a preset upper limit threshold, record the current intrinsic parameter calibration result of the third camera and output a recalibration prompt message.

[0139] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0140] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0141] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0142] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0143] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0144] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the multi-camera intrinsic parameter calibration method. For example, in some embodiments, the multi-camera intrinsic parameter calibration method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the multi-camera intrinsic parameter calibration method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform a multi-camera intrinsic calibration method by any other suitable means (e.g., by means of firmware).

[0145] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0150] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0151] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0152] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for calibrating the intrinsic parameters of multiple cameras, comprising: Acquire candidate images of the checkerboard calibration board at different positions, captured simultaneously by multiple cameras; The multiple cameras have the same focal length and model; the distance between each camera is less than a preset distance threshold. Corner detection is performed on each of the candidate images to obtain the number of corners contained in each candidate image; Based on the number of corner points contained in each candidate image and the actual number of corner points in the checkerboard calibration board, candidate images with incomplete corner points are filtered out to obtain each target image; For each of the candidate images, determine the centroid of each corner point in the candidate image; For each camera, a predetermined number of candidate images whose centroid positions fall within a predetermined area are determined as target images; For each camera, the similarity of the checkerboard calibration board positions among the candidate images acquired by the camera is determined, and candidate images with similar checkerboard calibration board positions are filtered out. For each camera, the intrinsic parameters of the camera are determined based on the target image acquired by the camera, and the current intrinsic parameter calibration result of the camera is obtained; Determine the number of calibrations to be performed on the intrinsic parameters of each camera; The first camera whose calibration count is less than a preset lower limit threshold is identified; For a first camera whose current intrinsic parameter calibration result is within a first preset parameter range, the current intrinsic parameter calibration result of the first camera is determined as the target intrinsic parameter calibration result of the first camera; For the first camera whose current intrinsic parameter calibration result is within the second preset parameter range, record the current intrinsic parameter calibration result of the first camera, and return the step of acquiring the target image of the calibration board at different positions simultaneously acquired by multiple cameras; the second preset parameter range includes the first preset parameter range; For the first camera whose current internal parameter calibration result is not within the range of the second preset parameters, it is determined that the first camera has failed calibration.

2. The method according to claim 1, further comprising: Obtain the sharpness of each candidate image; Candidate images with a resolution greater than a preset resolution threshold are selected as target images.

3. The method according to claim 2, further comprising: For cameras where the number of target images does not reach a preset threshold, candidate images of the checkerboard calibration board at different positions are reacquired from the camera. Return to the step of performing corner detection on each of the candidate images to obtain the number of corners contained in each candidate image.

4. The method according to claim 1, further comprising: The second camera is identified as having a calibration count greater than a preset upper limit threshold. For the second camera, obtain the difference between the current intrinsic parameter calibration result and the historical calibration result of the second camera; If each of the aforementioned differences is less than a preset difference threshold, then the current intrinsic parameter calibration result and the statistical value of each historical calibration result are used as the target intrinsic parameter calibration result of the second camera.

5. The method according to claim 4, further comprising: For the third camera whose calibration count is between a preset lower limit threshold and a preset upper limit threshold, record the current intrinsic parameter calibration result of the third camera and output a recalibration prompt message.

6. A multi-camera intrinsic parameter calibration device, comprising: The candidate image acquisition module is used to acquire candidate images of the checkerboard calibration board at different positions, captured simultaneously by multiple cameras. The multiple cameras have the same focal length and model; the distance between each camera is less than a preset distance threshold. The corner detection module is used to perform corner detection on each of the candidate images to obtain the number of corners contained in each candidate image; The target image acquisition module is used to filter candidate images with incomplete corners based on the number of corners contained in each candidate image and the actual number of corners in the checkerboard calibration board to obtain each target image; for each candidate image, determine the centroid of each corner point in the candidate image; for each camera, determine a preset number of candidate images whose centroid positions fall within a preset area as target images; for each camera, determine the similarity of the checkerboard calibration board positions among the candidate images acquired by the camera, and filter candidate images with similar checkerboard calibration board positions. The intrinsic parameter calibration module is used to determine the intrinsic parameters of each camera based on the target image acquired by the camera, obtain the current intrinsic parameter calibration result of the camera, and determine the number of calibrations to be performed on each camera. The first camera whose calibration count is less than a preset lower limit threshold is identified; For a first camera whose current intrinsic parameter calibration result is within a first preset parameter range, the current intrinsic parameter calibration result of the first camera is determined as the target intrinsic parameter calibration result of the first camera; For a first camera whose current intrinsic parameter calibration result is within the second preset parameter range, record the current intrinsic parameter calibration result of the first camera and return to the step of acquiring target images of the calibration board at different positions simultaneously acquired by multiple cameras; the second preset parameter range includes the first preset parameter range; for a first camera whose current intrinsic parameter calibration result is not within the second preset parameter range, determine that the first camera has failed calibration.

7. The apparatus according to claim 6, further comprising: Obtain the sharpness of each candidate image; Candidate images with a resolution greater than a preset resolution threshold are selected as target images.

8. The apparatus according to claim 7, further comprising: For cameras where the number of target images does not reach a preset threshold, candidate images of the checkerboard calibration board at different positions are reacquired from the camera. Return to the step of performing corner detection on each of the candidate images to obtain the number of corners contained in each candidate image.

9. The apparatus according to claim 6, further comprising: The second camera is identified as having a calibration count greater than a preset upper limit threshold. For the second camera, obtain the difference between the current intrinsic parameter calibration result and the historical calibration result of the second camera; If each of the aforementioned differences is less than a preset difference threshold, then the current intrinsic parameter calibration result and the statistical value of each historical calibration result are used as the target intrinsic parameter calibration result of the second camera.

10. The apparatus according to claim 9, further comprising: For the third camera whose calibration count is between a preset lower limit threshold and a preset upper limit threshold, record the current intrinsic parameter calibration result of the third camera and output a recalibration prompt message.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images