Calibration method for long-distance large-view-field binocular stereoscopic vision system
By using a phase-shifted circular pattern with built-in position encoding and a deep learning neural network algorithm, the calibration problem of a long-distance, large-field-of-view binocular stereo vision system was solved. High-precision feature point extraction and system calibration under defocus blur conditions were achieved, improving the versatility and flexibility of the calibration board.
Patent Information
- Application Number
- CN202511779253.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-03-13
AI Technical Summary
Existing binocular stereo vision systems cannot be effectively calibrated under long-distance, large-field-of-view conditions, especially in dangerous scenarios such as disaster sites where calibration boards cannot be placed. Furthermore, traditional calibration boards are difficult to extract feature point coordinates when in a defocused, blurred state.
A phase-shifting circular pattern with built-in position encoding is used, combined with a deep learning neural network algorithm, to generate an enable phase-shifting circular pattern. The feature point coordinates are extracted by a feature point detection neural network, and the extrinsic parameters are optimized using the Levenberg-Marquardt optimization algorithm. A calibration board is then constructed and calibrated.
It achieves high-precision feature point extraction and calibration under long-distance, large field-of-view conditions, improves the versatility and flexibility of the calibration board, and can accurately calibrate the binocular stereo vision system under defocus blur conditions.
Smart Images
Figure CN121661151A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a calibration method for a binocular stereo vision system, specifically a calibration method for a long-distance, large-field-of-view binocular stereo vision system. Background Technology
[0002] Existing binocular stereo vision measurement technology is mainly used for short-distance measurements, typically not exceeding ten meters. To accurately measure scenes at greater distances, it's necessary to increase the distance between the two eyes (baseline distance) and use lenses with longer focal lengths. In this case, because there is no shared field of view between the two eyes at close range to the binocular stereo vision system, and the area is outside the system's depth of field, traditional binocular stereo vision calibration methods require placing the calibration plate in a distant measurement area. In certain scenarios, such as disaster sites requiring on-site calibration where the measurement area is hazardous, it's impossible to place the calibration plate within the measurement area.
[0003] To address the issue of calibration plates not being able to be placed within the shared field of view of both eyes, a long, rectangular calibration plate can be fabricated, allowing each eye to observe a portion of the plate. Traditional calibration plate patterns typically include checkerboard patterns, dot arrays, or arrays of Aruco QR codes.
[0004] The corner points and circular points of the checkerboard pattern cannot be positionally encoded. Unless the two cameras in the binocular stereo vision system observe the leftmost and rightmost points of the calibration board respectively, the system cannot correlate the coordinates of the feature points extracted from the acquired calibration board image with the world coordinates of the feature points on the calibration board, thus failing to complete the calibration task. Therefore, the long strip calibration board is only suitable for binocular stereo vision systems with certain baseline lengths, lacking versatility. While Aruco QR codes can be positionally encoded, current recognition algorithms struggle to detect and recognize them when the calibration board is outside the camera's depth of field, resulting in out-of-focus blurring.
[0005] The three commonly used calibration board patterns mentioned above cannot accurately extract the coordinates of image feature points under defocused blur conditions. Phase-shift retrieval algorithms can accurately recover the absolute phase of a phase-shifted circle when it becomes defocused, thereby extracting the precise coordinates of feature points within the circle. Therefore, calibration boards constructed from phase-shifted circle patterns can be used for the calibration of defocused binocular stereo vision systems (Zhichao Meng et al., 2023 paper "Defocused calibration for large field-of-view binocular cameras"). However, because the proposed phase-shifted circle pattern cannot be position-encoded, de Bruijn sequence coding patterns must be added to the top and bottom of the calibration board to provide position encoding for the phase-shifted circle pattern, which encroaches on the effective space of the calibration pattern. Summary of the Invention
[0006] To overcome the problem that existing calibration methods for long-distance, large-field-of-view binocular stereo vision systems cannot perform position encoding using phase-shifting circle patterns, this invention discloses a calibration method for long-distance, large-field-of-view binocular stereo vision systems, specifically for systems with a large field of view and a large binocular measurement area. This invention proposes a phase-shifting circle pattern with built-in position encoding, a deep learning neural network algorithm model for high-precision extraction and position decoding of feature point coordinates from the defocused phase-shifting circle pattern, a calibration board composed of position-encoded phase-shifting circles, and its feature point extraction algorithm. This invention is achieved through the following technical solutions:
[0007] This invention discloses a calibration method for a long-range, large-field-of-view binocular stereo vision system, comprising:
[0008] S1 Generates a coded phase-shift circular pattern: The coded phase-shift circular pattern consists of two phase-shift semicircular patterns with different period numbers. Two feature points are defined, including position feature points and coding feature points. The position feature points are the center of the phase-shift circular pattern, and the coding feature points are used to calculate the code of the phase-shift circular pattern.
[0009] S2 proposes a phase-shifting circular pattern encoding method: using the center of the phase-shifting circular pattern generated by S1 as the rotation center, rotate the pattern counterclockwise in the plane of the phase-shifting circular pattern, and the number of rotation angles is the encoding of the rotated phase-shifting circular pattern;
[0010] S3 Construct a neural network for detecting feature points of a phase-shifted circular pattern: Construct a neural network for detecting feature points of a phase-shifted circular pattern, including a feature extraction module and a fully connected layer. The feature extraction module takes as input image data containing a phase-shifted circular pattern and outputs a one-dimensional feature vector of the image. The fully connected layer takes as input a one-dimensional feature vector and outputs the coordinates of the position feature points of the phase-shifted circular pattern in the image and the coordinates of the encoded feature points.
[0011] S4 Generates the dataset for training the phase-shift circular pattern feature point detection neural network: In practical applications, due to factors such as camera depth of field, shooting angle, and ambient light, the phase-shift circular pattern in the calibration image will exhibit changes in defocus blur, contrast, translation, rotation, and scaling. Apply scaling, translation, rotation, Gaussian blur, brightness adjustment, and contrast adjustment image processing to the coded phase-shift circular pattern generated in step S1. At the same time, apply the same scaling, translation, and rotation transformations to the feature points of the coded phase-shift circular pattern. By changing the image processing parameters, 100,000 training image sample-feature point coordinate ground truth data pairs are generated and divided into training set, validation set, and test set in an 8:1:1 ratio.
[0012] S5 Training the phase-shifting circular pattern feature point detection neural network: The phase-shifting circular pattern feature point detection neural network constructed in S3 is trained using the dataset generated in S4;
[0013] S6. Fabricate the phase-shifting circle calibration plate: Arrange the coded phase-shifting circle patterns generated in step S1 into an array of m rows and n columns according to the set row spacing and column spacing. Designate the middle row as the column encoding row. The phase-shifting circle patterns in this row are used for column encoding. The rotation angle of each phase-shifting circle pattern in this row is different. Each rotation angle corresponds one-to-one with the column position of the phase-shifting circle pattern in the calibration plate, thus obtaining the phase-shifting circle calibration plate.
[0014] S7 Acquire calibration images of the binocular stereo vision system; Place the phase-shifting circle calibration plate made in step S6 near the binocular stereo vision system with a large field of view at a long distance, change the posture of the phase-shifting circle calibration plate and take a set of binocular stereo vision calibration image pairs. The number of image pairs is not less than 8 pairs. The calibration plate areas captured by each pair of images do not overlap. Each image contains at least 7 complete phase-shifting circle patterns, and at least 2 of them belong to the column coding row of the calibration plate.
[0015] S8 Detect the coordinates of feature points in the calibration image; Using the phase-shifting circular pattern feature point detection neural network trained in step S5, detect the coordinates of feature points of each phase-shifting circular pattern in the calibration image acquired in step S7, including positional feature points and coded feature points, and arrange the coordinates of feature points of each phase-shifting circular pattern into a sequence in order, where the odd-numbered positions in the sequence are the coordinates of positional feature points and the even-numbered positions are the coordinates of coded feature points.
[0016] S9 Determine the row and column positions of feature points: Find the position feature points that belong to the column-coded row phase-shifted circular pattern of the calibration board among the position feature points detected in step S8, and deduce the row and column positions of all position feature points in the calibration board.
[0017] S10 Calibrate the binocular stereo vision system: Based on the coordinates of the position feature points in the calibration image of the binocular stereo vision system detected in step S8, and the row and column positions of the position feature points corresponding to the calibration board obtained in step S9, calibrate the binocular stereo vision system.
[0018] As a further improvement, in step S1 of this invention, the mathematical model for encoding the phase-shifting circular pattern is as follows:
[0019]
[0020] in These are the x and y coordinates of a coded phase-shifting circular pattern, with the pattern center located at the origin of the coordinate system. , It is the median grayscale value. It is the grayscale amplitude value. It is the radius of the coded phase-shifting circle outer contour. It is a point The distance from the origin is odd. and These are used to control the number of cycles for the two phase-shifting semicircles, respectively.
[0021] In the coordinate system described above, the coordinates of the position feature point of the coded phase-shift circle are (0,0), and the coordinates of the coded feature point are... .
[0022] As a further improvement, the specific steps in step S8 of the present invention are as follows:
[0023] 1) Adaptive binarization calibration of images;
[0024] 2) Detect the outer contours of each phase-shifting circle pattern in the binarized calibration image;
[0025] 3) Calculate the envelope rectangle of the outer contour of each phase-shifting circular pattern and crop it into a sub-image;
[0026] 4) Input the sub-image into the phase-shifting circular pattern feature point detection neural network, and output the coordinates of the positional feature points and the coordinates of the encoded feature points of the phase-shifting circular pattern in the sub-image;
[0027] 5) Transform the coordinates of feature points in the sub-image to the coordinates in the original calibration image;
[0028] 6) Arrange the coordinates of the feature points in the transformed original calibration image into a sequence. The odd-numbered positions in the sequence are the coordinates of the positional feature points, and the even-numbered positions are the coordinates of the encoded feature points.
[0029] As a further improvement, the specific steps in step S9 of the present invention are as follows:
[0030] 1) Traverse the feature point sequence obtained in step S8, and find two adjacent position feature points belonging to the phase shift circle pattern of the calibration plate column encoding row based on the positional relationship between each group of position feature points and encoding feature points.
[0031] 2) Find two other location feature points in the same column and adjacent rows of the two location feature points found in step 1);
[0032] 3) Based on the coordinates of the four positional feature points found in steps 1) and 2), calculate the perspective transformation matrix that transforms the original calibration image into an upright image (i.e., the row and column directions of the phase-shifting circular pattern are located in the horizontal and vertical directions of the image, respectively).
[0033] 4) Using the perspective transformation matrix obtained in step 3), the original calibration image is transformed into an upright image, and the feature points of each phase-shifting circle detected on the original calibration image are also transformed into the upright image through perspective transformation.
[0034] 5) Calculate the rotation angle of each phase-shifting circle pattern in the upright image after perspective transformation that belongs to the column code row of the calibration plate, calculate its corresponding code, and determine the column number of each of them in the calibration plate;
[0035] 6) Based on the relative positional relationships of the feature points in the upright image, calculate the row and column positions of all feature points in the image on the calibration plate.
[0036] As a further improvement, the specific steps in step S10 of the present invention are as follows:
[0037] 1) Based on the position of the feature points obtained in step S9, corresponding to their row and column positions on the calibration board, calculate the three-dimensional world coordinates of the feature points on the calibration board corresponding to the center point of each phase-shifting circle pattern in the binocular stereo vision system calibration image acquired in step S7.
[0038] 2) By using the coordinates of the center points of each phase-shifting circle pattern in the calibration images of each binocular camera and the three-dimensional world coordinates of the center of the phase-shifting circle pattern on the corresponding calibration plate, the intrinsic parameters and distortion coefficients of the two cameras in the binocular stereo vision system, as well as the extrinsic parameters of the two cameras when the binocular stereo vision system captures each set of calibration images, are calibrated and obtained respectively.
[0039] 3) Using the extrinsic parameters of the two cameras when each set of calibration images is captured by the binocular stereo vision system obtained in step 2), calculate the pose relationship between the two cameras in the binocular system when each set of calibration images is captured, that is, the extrinsic parameters of the binocular stereo vision system.
[0040] 4) Using the extrinsic parameters obtained in step 3) for each set of calibration images captured by the binocular stereo vision system, first calculate their average value as the initial value, then construct the following loss function, and use the Levenberg-Marquardt nonlinear least squares optimization algorithm to optimize the extrinsic parameters of the binocular stereo vision system. The loss function is:
[0041] ;
[0042] Where p represents the image coordinates of feature point P on the calibration board captured by camera A in the binocular stereo vision system, and p' represents the image coordinates of P projected onto another camera B and then mapped back to camera A by the homography matrix H of the binocular stereo vision system.
[0043] The deep learning neural network algorithm model for high-precision extraction and position decoding of feature point coordinates of the defocused phase-shift circular pattern mainly includes a feature extraction module and a fully connected layer. The feature extraction module takes as input image data containing the aforementioned phase-shift circular pattern and outputs a one-dimensional feature vector of the image. The fully connected layer module takes as input the one-dimensional feature vector of the phase-shift circular pattern and outputs the position feature points of the phase-shift circle. and encoding feature points The coordinates.
[0044] The neural network algorithm model requires a large number of data samples for training. By applying various transformations and image processing combinations such as Gaussian blur transformation, contrast transformation, translation transformation, rotation transformation, scaling transformation, and superimposed Gaussian noise to the phase-shifting circular pattern, a dataset for model training can be generated.
[0045] The calibration plate of this invention is composed of a row and column matrix of the aforementioned phase-shifting circular patterns, with known row and column spacing. One row is used for position encoding of the phase-shifting circular patterns, meaning that the rotation angle of each phase-shifting circular pattern in that row is different, and each rotation angle is associated with the position of the phase-shifting circular pattern in the calibration plate.
[0046] Binocular stereo vision calibration requires detecting the two-dimensional coordinates of feature points on the calibration board image and corresponding each specific point to its three-dimensional world coordinates on the calibration board.
[0047] The method for detecting feature point coordinates in calibration board images is as follows: 1) Detect the outer contours of each phase-shift circle pattern in the calibration board image; 2) Input the sub-image containing a single phase-shift circle pattern into the aforementioned defocused phase-shift circle pattern feature point coordinate extraction neural network, and output the position feature points of the phase-shift circle pattern. coordinates and encoded feature points 3) Convert the local coordinates of the phase-shifted circular pattern position feature points in the sub-image to the coordinates in the calibration plate image.
[0048] The method for calculating the world coordinates of the feature points on the calibration board corresponding to the feature points in the calibration board image is as follows: 1) Based on the positional relationship of the center points of each phase-shifting circle pattern in the calibration board image detected in the above steps, calculate the affine matrix that transforms the original image into an upright image (i.e., the row and column directions of the phase-shifting circle pattern are located in the horizontal and vertical directions of the image, respectively); 2) Affine transform the original image into an upright image, and affine transform the feature points of each phase-shifting circle detected on the original image onto the upright image; 3) Calculate the rotation angle of each phase-shifting circle pattern in the transformed image, and find the position code corresponding to each phase-shifting circle pattern; 4) Calculate the world coordinates of the feature points on the calibration board corresponding to the center point of each phase-shifting circle pattern in the original image based on the position code.
[0049] Once the two-dimensional coordinates of the feature points on the calibration board image and their corresponding three-dimensional world coordinates on the calibration board are obtained, the existing binocular stereo vision calibration method can be used for calibration.
[0050] The beneficial effects of this invention are as follows:
[0051] 1. This solves the problem that existing methods are difficult to calibrate long-distance, large-field-of-view binocular stereo vision systems. Such binocular stereo vision systems have a large baseline distance, a small field of view of the binocular lenses, and an imaging depth of field that is far from the system. At close range from the system, the binoculars do not share a common field of view and the imaging suffers from defocusing blur.
[0052] 2. A method for generating coded phase-shift circular patterns is proposed. While maintaining the advantage that the phase-shift circular patterns can accurately extract the coordinates of the center point of the pattern in the case of defocused and blurred imaging, the phase-shift circular patterns proposed in this invention have coded characteristics due to spatial asymmetry.
[0053] 3. A coding method for coded phase-shifted circular patterns is proposed. This method encodes through simple image rotation, and therefore can be easily decoded with simple image processing.
[0054] 4. A method for generating a calibration board using coded phase-shifting circular patterns is proposed. The coded phase-shifting circular patterns provide clues to the row and column of each phase-shifting circular pattern in the calibration board. Even if the camera only captures a local area of the calibration board, the one-to-one correspondence between the two-dimensional feature points in the calibration board image and the three-dimensional feature points on the calibration board can be determined according to the coding of the phase-shifting circular patterns. In other words, even if each of the two eyes only captures a local area of the calibration board without overlap, binocular stereo vision calibration can still be completed, which greatly improves the versatility of the calibration board and the flexibility of calibration.
[0055] 5. A neural network for detecting feature points of phase-shifting circular patterns is proposed, along with a method for generating the dataset for training the neural network. After training, the neural network can accurately extract the positional feature points and coded feature point coordinates of the phase-shifting circular pattern even when the phase-shifting circular pattern exhibits changes in focus blur, contrast, translation, rotation, and scaling.
[0056] 6. A method is proposed to first calibrate the intrinsic parameters of the two cameras in a binocular stereo vision system separately, and then use the Levenberg-Marquardt nonlinear least squares optimization algorithm to optimize the extrinsic parameters of the binocular stereo vision system, thereby greatly improving the accuracy of extrinsic parameter estimation. Attached Figure Description
[0057] Figure 1 This is a schematic diagram of the calibration method for the long-distance, large-field-of-view binocular stereo vision system of the present invention;
[0058] Figure 2 This is an example diagram of the coded phase-shifting circular pattern proposed in the calibration method for a long-distance, large-field-of-view binocular stereo vision system of the present invention;
[0059] Figure 3 This is an example diagram of the phase-shifting circular encoding proposed in the calibration method for a long-distance, large-field-of-view binocular stereo vision system of the present invention;
[0060] Figure 4 This is an example diagram of a calibration plate composed of phase-shifting circles proposed in the calibration method for a long-distance, large-field-of-view binocular stereo vision system of this invention;
[0061] Figure 5 This is a diagram of the neural network architecture for detecting the coordinates of phase-shifting circular feature points, proposed in the calibration method for a long-distance, large-field-of-view binocular stereo vision system of this invention.
[0062] Figure 6 This is an example image of a portion of the dataset used for training the phase-shifting circle feature point coordinate detection neural network proposed in this invention;
[0063] Figure 7 This is a flowchart of the calibration board image feature point coordinate detection process proposed in this invention;
[0064] Figure 8 This is a flowchart of the method for corresponding the coordinates of feature points in a calibration board image to world coordinates, as proposed in this invention.
[0065] 1 is a binocular stereo vision system, 2 is a coded phase-shifting circular calibration plate, 3 is the binocular measurement area, 4 is the area within the depth of field, 5 is the area outside the depth of field, 6 is the position feature point, and 7 is the coded feature point. Detailed Implementation
[0066] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. For the sake of brevity, some content known in the art has been omitted when describing the processes, conditions, experimental methods, etc. of the embodiments of the present invention, and the present invention does not have any particular limitations on such content.
[0067] The following will combine Figure 1-8 The calibration method of the long-distance large field-of-view dual-sided stereo vision system 1 of the present invention will be described in further detail.
[0068] like Figure 1 As shown, there is a significant distance between the two cameras in the dual-view stereo vision system 1, and the binocular measurement area 3 is located far from the system. To achieve high 3D measurement accuracy, the focal lengths of the two cameras are relatively long, resulting in a lack of a common field of view for the binoculars in areas closer to the dual-view stereo vision system 1. Furthermore, to ensure that the binocular measurement area 3 is within the camera's depth of field (area 4), areas closer to the dual-view stereo vision system 1 inevitably fall outside the camera's depth of field (area 5). Placing a coded phase-shifting circular calibration plate 2 in an area close to the dual-view stereo vision system 1 results in no overlap between the calibration plate observed by the two cameras, and the captured images of the calibration plate exhibit defocus blur.
[0069] like Figure 2 (a) shows an example of a coded phase-shift circle generated according to formula (1). To further illustrate the positions of position feature point 6 and coded feature point 7 on the phase-shift circle, in Figure 2 (b) is marked with a red cross. In this example, A=127.5, B=127.5, and n=5.
[0070] Will Figure 2 (a) The degree by which the phase-shifting circular pattern shown is rotated counterclockwise around its center is used as the code for the rotated phase-shifting circular pattern. For example... Figure 3 The image shows an example of encoding eight phase-shifting circular patterns.
[0071] like Figure 4 The figure shows an example of the coded phase-shift circle calibration plate 2 proposed in this invention. The calibration plate in the figure contains 19 rows and 36 columns of phase-shift circle patterns. The phase-shift circle pattern located in the middle row (i.e., the 10th row) is used to encode the column it is in. The codes of the phase-shift circle patterns from left to right are as follows:
[0072] (1)
[0073] Then, in the calibration plate image after the vertical transformation, the column number j of the phase-shifted circular pattern can be decoded using formula (2):
[0074] (2)
[0075] Even if the two cameras in the binoculars can only capture a portion of the coded phase-shifting circle calibration plate 2, since the phase-shifting circles in the middle row can always be captured, the row and column positions of all phase-shifting circle patterns in the image can be inferred based on their known row positions and the column positions obtained from image processing decoding. Thus, the world coordinate value corresponding to the center point of each phase-shifting circle pattern can be calculated.
[0076] like Figure 5 The diagram shows the neural network architecture for detecting the coordinates of feature points in a phase-shifted circle. An image containing a phase-shifted circle pattern is input into this network. First, it passes through a sliding window hierarchical visual transformer network for feature extraction. The extracted features are then normalized, pooled, and flattened to become a one-dimensional vector of length 4C (where C is the number of channels). Finally, it passes through a fully connected layer to output a vector. ,in These are the coordinates of the center of the phase-shifted circular pattern in the input image. These are the coordinates of the feature points in the phase-shifted circular pattern used to calculate the rotation angle in the input image.
[0077] The phase-shifting circle feature point coordinate detection neural network proposed in this invention uses a sliding window hierarchical visual transformer network, which requires a large number of sample images for training. Through... Figure 2 (a) The standard phase-shifting circular pattern shown can be combined with various transformations and image processing techniques, such as Gaussian blur transformation, contrast transformation, translation transformation, rotation transformation, scaling transformation, and superposition of Gaussian noise, to generate the required number of sample images. Figure 6 The image shown is an example of a partial sample image.
[0078] The coded phase-shift circle calibration plate 2 proposed in this invention contains rows and columns of multiple phase-shift circle patterns. The calibration plate image captured by the dual-sided stereo vision system 1 contains multiple phase-shift circle patterns, and includes at least one phase-shift circle pattern with a non-zero code. Figure 4 The calibration board is shown with phase-shifted circular patterns in the middle row. The center points of these phase-shifted circular patterns are the position feature points 6 of the calibration board. The calibration board image feature point coordinate detection process is as follows: Figure 7 As shown, the main steps include the following:
[0079] S101. The dual-sided stereo vision system 1 captures images of a calibration plate containing multiple coded phase-shifting circular patterns;
[0080] S102. Detect the outer contour of the coded phase-shifting circular pattern and determine the position and size of the sub-image rectangle containing each phase-shifting circular pattern;
[0081] S103. Input each sub-image containing a single encoded phase-shifting circle pattern into the feature point detection neural network, and output the coordinates of the phase-shifting circle position feature point 6 and the encoded feature point 7;
[0082] S104. Convert the coordinates of the feature points detected in the sub-image to the coordinates on the original calibration plate image;
[0083] S105. Store the phase-shifting circle position feature point 6 and the coded feature point 7 in two sequences, fp1 and fp2, respectively, according to the same order. fp1 is the calibration board image feature point sequence.
[0084] After detecting the sequence of feature points in the calibration board image, it is also necessary to determine the world coordinates corresponding to the coordinates of these image feature points. Figure 8 The flowchart shown is a method for mapping feature point coordinates of a calibration board image to world coordinates, which mainly includes the following specific steps:
[0085] S201. Randomly find a calibration plate image location feature point 6, an adjacent location feature point 6 in the near horizontal direction, and an adjacent location feature point 6 in the near vertical direction.
[0086] S202. Calculate the affine matrix that transforms the calibration board image into an upright image using the three positional feature points 6 mentioned above;
[0087] S203. Affine transform the coordinate sequence fp1 of position feature point 6 and the coordinate sequence fp2 of encoded feature point 7 in the original image of the calibration board into the feature point sequences fp1' and fp2' in the upright image;
[0088] S204. By decoding the corresponding phase shift circles of fp1' and fp2', find the phase shift circle of the position encoding row and determine its position in the calibration board;
[0089] S205. Determine its position in the calibration plate based on the positional relationship between the center point of other phase shift circles and the center point of the position coding row phase shift circle;
[0090] S206. Associate the coordinates of the feature points in the calibration board image in fp1 with their corresponding world coordinates.
[0091] Example
[0092] Combination Figures 1-8 The present invention is described in detail using the calibration method of the long-distance large field-of-view dual-sided stereo vision system 1, the phase-shifting circular calibration plate with position encoding, the neural network model for feature point detection of the calibration plate, and the feature point detection method of the calibration plate as examples.
[0093] 1) Construct a long-distance, wide-field-of-view, dual-sided stereoscopic vision system.
[0094] The baseline distance of the dual-view stereo vision system is about 0.5 meters, the horizontal field of view of the binocular cameras is about 10°, the depth of field range is about 30 meters in front of the system, and the binoculars have no common field of view within 1 meter in front of the system and are outside the depth of field range.
[0095] 2) Construct a calibration plate with a phase-shifting circle containing position codes.
[0096] A flat-screen display, 0.7 meters wide and 0.4 meters high, was used as the calibration board, displaying the entire screen. Figure 4 The calibration board shown consists of 19 rows and 36 columns of coded phase-shifting circle patterns, with the 10th row of phase-shifting circles used for position encoding. The world coordinates of the center of each phase-shifting circle are known.
[0097] 3) Generate a dataset for training the neural network for detecting the coordinates of phase-shifting circular feature points.
[0098] right Figure 2 (a) shows a standard encoded phase-shifting circular pattern combined with various transformations and image processing techniques such as Gaussian blur transformation, contrast transformation, translation transformation, rotation transformation, scaling transformation, and superimposed Gaussian noise to generate a dataset containing 80,000 training samples, 10,000 validation samples, and 10,000 test samples for training the phase-shifting circular feature point coordinate detection neural network.
[0099] 4) Training a neural network for detecting the coordinates of phase-shifting circular feature points
[0100] The neural network for detecting phase-shifting circle feature point coordinates was trained using the dataset generated above. After testing, the detection error of the trained neural network for feature point coordinates was less than 0.15 pixels.
[0101] 5) Calibrate a long-range, large-field-of-view dual-surface stereo vision system 1
[0102] The aforementioned dual-sided stereo vision system 1 was used to acquire images of a calibration board with a position-coded phase-shifting circle. Twenty sets of images of the calibration board in different poses were acquired. The feature point detection method proposed in this invention was used to detect the image coordinates and their corresponding world coordinates of the feature points on the calibration board. Then, a traditional binocular stereo vision calibration algorithm was used for calibration. Testing showed that the calibrated dual-sided stereo vision system 1 achieved millimeter-level accuracy in 3D measurement of a target at a distance of approximately 30 meters.
[0103] The calibration method of the long-distance large field of view dual-sided stereo vision system 1 mentioned in this invention is not limited to the embodiments described above. Various modifications and improvements can be made without departing from the principle of this invention.
[0104] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of this invention are included in this invention and are protected by the appended claims.
Claims
1. A calibration method for a long-distance, large-field-of-view binocular stereo vision system, characterized in that, include: S1 generates a coded phase-shift circular pattern: The coded phase-shift circular pattern consists of two phase-shift semicircular patterns with different period numbers. Two feature points are defined, including position feature points and coding feature points. The position feature points are the center of the phase-shift circular pattern, and the coding feature points are used to calculate the code of the phase-shift circular pattern. S2 proposes a phase-shifting circular pattern encoding method: using the center of the phase-shifting circular pattern generated by S1 as the rotation center, rotate the pattern counterclockwise in the plane of the phase-shifting circular pattern, and the number of rotation angles is the encoding of the rotated phase-shifting circular pattern; S3 constructs a neural network for detecting feature points of a phase-shifted circular pattern: The neural network for detecting feature points of a phase-shifted circular pattern includes a feature extraction module and a fully connected layer. The feature extraction module takes as input image data containing a phase-shifted circular pattern and outputs a one-dimensional feature vector of the image. The fully connected layer takes as input a one-dimensional feature vector and outputs the coordinates of the position feature points of the phase-shifted circular pattern in the image and the coordinates of the encoded feature points. S4 generates a dataset for training a phase-shift circular pattern feature point detection neural network: In practical applications, due to factors such as camera depth of field, shooting angle, and ambient light, the phase-shift circular pattern in the calibration image will exhibit changes in defocus blur, contrast, translation, rotation, and scaling. The coded phase-shift circular pattern generated in step S1 is subjected to image processing such as scaling, translation, rotation, Gaussian blur, brightness adjustment, and contrast adjustment. At the same time, the same scaling, translation, and rotation transformations are applied to the feature points of the coded phase-shift circular pattern. By changing the image processing parameters, 100,000 training image sample-feature point coordinate ground truth data pairs are generated and divided into training set, validation set, and test set in an 8:1:1 ratio. S5 Training of Phase-Shifting Circular Pattern Feature Point Detection Neural Network: The phase-shifting circular pattern feature point detection neural network constructed in S3 is trained using the dataset generated in S4; S6. Create a phase-shifting circle calibration plate: Arrange the coded phase-shifting circle patterns generated in step S1 into an array of m rows and n columns according to the set row spacing and column spacing. Designate the middle row as the column encoding row. The phase-shifting circle patterns in this row are used for column encoding. The rotation angle of each phase-shifting circle pattern in this row is different. Each rotation angle corresponds one-to-one with the column position of the phase-shifting circle pattern in the calibration plate, thus obtaining the phase-shifting circle calibration plate. S7 Acquire calibration images of the binocular stereo vision system; Place the phase-shifting circle calibration plate made in step S6 near the binocular stereo vision system with a large field of view at a long distance, change the posture of the phase-shifting circle calibration plate and take a set of binocular stereo vision calibration image pairs. The number of image pairs is not less than 8 pairs. The calibration plate areas captured by each pair of images do not overlap. Each image contains at least 7 complete phase-shifting circle patterns, and at least 2 of them belong to the column code row of the calibration plate. S8 detects the coordinates of feature points in the calibration image; using the phase-shifting circular pattern feature point detection neural network trained in step S5, the coordinates of feature points of each phase-shifting circular pattern in the calibration image acquired in step S7 are detected, including positional feature points and coded feature points. The coordinates of feature points of each phase-shifting circular pattern are arranged into a sequence in order, with the positional feature point coordinates in the odd-numbered positions and the coded feature point coordinates in the even-numbered positions. S9 Determine the row and column positions of the feature points: Find the position feature points that belong to the column-encoded row phase-shifted circular pattern of the calibration board among the position feature points detected in step S8, and deduce the row and column positions of all position feature points in the calibration board. S10 Calibrate the binocular stereo vision system: Based on the coordinates of the position feature points in the calibration image of the binocular stereo vision system detected in step S8, and the row and column positions of the position feature points corresponding to the calibration board obtained in step S9, calibrate the binocular stereo vision system.
2. The calibration method for a long-distance, large-field-of-view binocular stereo vision system according to claim 1, characterized in that, In step S1, the mathematical model for encoding the phase-shifting circular pattern is: Where u and v are the abscissa and ordinate of the coded phase-shifting circular pattern, the center of the pattern is located at the origin of the coordinate system with coordinates (0,0), A is the gray-scale median, B is the gray-scale amplitude value, r0 is the radius of the outer contour of the coded phase-shifting circle, r(u,v) is the distance between the point (u,v) and the origin, and odd numbers n and n+2 are used to control the number of cycles of the two phase-shifting semicircles, respectively. In the coordinate system described above, the coordinates of the position feature point of the coded phase-shift circle are (0,0), and the coordinates of the coded feature point are...
3. The calibration method for a long-distance, large-field-of-view binocular stereo vision system according to claim 1, characterized in that, The specific steps in step S8 are as follows: 1) Adaptive binarization calibration of images; 2) Detect the outer contours of each phase-shifting circle pattern in the binarized calibration image; 3) Calculate the envelope rectangle of the outer contour of each phase-shifting circular pattern and crop it into a sub-image; 4) Input the sub-image into the phase-shifting circular pattern feature point detection neural network, and output the coordinates of the positional feature points and the coordinates of the encoded feature points of the phase-shifting circular pattern in the sub-image; 5) Transform the coordinates of feature points in the sub-image to the coordinates in the original calibration image; 6) Arrange the coordinates of the feature points in the original calibration image into a sequence. The coordinates of the positional feature points are in the odd-numbered positions, and the coordinates of the encoded feature points are in the even-numbered positions.
4. The calibration method for a long-distance, large-field-of-view binocular stereo vision system according to claim 1, characterized in that, The specific steps in step S9 are as follows: 1) Traverse the feature point sequence obtained in step S8, and find two adjacent position feature points belonging to the phase shift circle pattern of the calibration plate column encoding row based on the positional relationship between each group of position feature points and encoding feature points. 2) Find two other location feature points in the same column and adjacent rows of the two location feature points found in step 1); 3) Based on the coordinates of the four positional feature points found in steps 1) and 2), calculate the perspective transformation matrix that transforms the original calibration image into an upright image; 4) Using the perspective transformation matrix obtained in step 3), the original calibration image is transformed into an upright image, and the feature points of each phase-shifting circle detected on the original calibration image are also transformed into the upright image through perspective transformation. 5) Calculate the rotation angle of each phase-shifting circle pattern in the upright image after perspective transformation that belongs to the column code row of the calibration plate, calculate its corresponding code, and determine the column number of each of them in the calibration plate; 6) Based on the relative positional relationships of the feature points in the upright image, calculate the row and column positions of all feature points in the image on the calibration plate.
5. The calibration method for a long-distance, large-field-of-view binocular stereo vision system according to claim 1, characterized in that, The specific steps in step S10 are as follows: 1) Based on the position of the feature points obtained in step S9, corresponding to their row and column positions on the calibration board, calculate the three-dimensional world coordinates of the feature points on the calibration board corresponding to the center point of each phase-shifting circle pattern in the binocular stereo vision system calibration image acquired in step S7. 2) By using the coordinates of the center points of each phase-shifting circle pattern in the calibration images of each binocular camera and the three-dimensional world coordinates of the center of the phase-shifting circle pattern on the corresponding calibration plate, the intrinsic parameters and distortion coefficients of the two cameras in the binocular stereo vision system, as well as the extrinsic parameters of the two cameras when the binocular stereo vision system captures each set of calibration images, are calibrated and obtained respectively. 3) Using the extrinsic parameters of the two cameras when each set of calibration images is captured by the binocular stereo vision system obtained in step 2), calculate the pose relationship between the two cameras in the binocular system when each set of calibration images is captured, that is, the extrinsic parameters of the binocular stereo vision system. 4) Using the extrinsic parameters obtained in step 3) for each set of calibration images captured by the binocular stereo vision system, first calculate their average value as the initial value, then construct the following loss function, and use the Levenberg-Marquardt nonlinear least squares optimization algorithm to optimize the extrinsic parameters of the binocular stereo vision system. The loss function is: loss = norm(||p-p'||, L2); Where p represents the image coordinates of feature point P on the calibration board captured by camera A in the binocular stereo vision system, and p' represents the image coordinates of P projected onto another camera B and then mapped back to camera A by the homography matrix H of the binocular stereo vision system.