Three-dimensional space positioning method based on hierarchical joint calibration

Abstract

The invention discloses a three-dimensional space positioning method based on hierarchical joint calibration, and relates to the field of precise positioning of an industrial long-distance large equipment hoisting task in a three-dimensional space. Comprising the following steps: calibrating a binocular camera through a long-distance binocular system calibration method; acquiring a binocular image data set of the positioned object by adopting the calibrated binocular camera; performing stereo correction on each group of binocular images in the binocular image data set, and obtaining a public ROI region of each group of binocular images; calculating a disparity map of each group of binocular images in the public ROI region through a stereo matching algorithm based on deep learning; pixel coordinates of a positioned object are obtained through an adaptive Shii-Tomasi circular diagonal mark angular point detection algorithm; and according to the pixel coordinates of the positioned object, the final coordinates of the positioned object in the three-dimensional space are obtained in combination with the disparity map. According to the method, long-distance high-precision positioning can be effectively realized, and the positioning and mounting efficiency of large and medium-sized industrial equipment can be improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned hoisting and positioning technology for large industrial equipment using binocular vision, and more specifically to a three-dimensional spatial positioning method based on hierarchical joint calibration. Background Technology

[0002] Lifting large industrial equipment has always been a major challenge in the manufacturing industry. These workpieces are large and heavy, placing stringent requirements on lifting operations. Binocular vision can quickly and accurately acquire the three-dimensional coordinate information of the workpiece, providing crucial positioning data for lifting. Compared to laser positioning technology, binocular vision is less likely to lose the target; compared to traditional contact measurement, binocular vision is more efficient and less likely to damage the workpiece.

[0003] The localization process of a binocular system includes: binocular camera calibration, target point pixel coordinate extraction, disparity map calculation, depth map calculation, and 3D reconstruction. Existing binocular system localization methods are only suitable for near-range localization. For long-range localization, they suffer from drawbacks such as reprojection errors during calibration, high computational load, layering of the 3D reconstructed point cloud, and low pixel coordinate extraction accuracy.

[0004] Therefore, how to achieve long-distance high-precision positioning to improve the automation level of positioning and hoisting tasks for large industrial equipment, reduce the personnel injury rate in hoisting accidents of large equipment, and improve the positioning and installation efficiency of large industrial equipment are problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention provides a three-dimensional spatial positioning method based on hierarchical joint calibration, so as to at least solve some of the technical problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides a three-dimensional spatial positioning method based on hierarchical joint calibration, comprising:

[0008] S1. Calibrate the binocular camera using a long-range binocular system calibration method;

[0009] S2. Use the calibrated binocular camera to acquire a binocular image dataset of the object to be located;

[0010] S3. After performing stereo correction on each group of stereo images in the stereo image dataset, obtain the common ROI region of each group of stereo images.

[0011] S4. Calculate the disparity map of each set of binocular images within the common ROI region using a deep learning-based stereo matching algorithm;

[0012] S5. Obtain the pixel coordinates of the object being located using the adaptive Shi-Tomasi circular diagonal marker corner detection algorithm;

[0013] S6. Based on the pixel coordinates of the object being located, and combined with the depth map transformed from the disparity map, obtain the final coordinates of the object being located in three-dimensional space.

[0014] Further, step S1 specifically includes:

[0015] S11. Set the initial calibration distance between the target object and the stereo camera. After optimizing the initial calibration distance, initially adjust the optical axes of the left and right cameras in the stereo camera to be nearly parallel and ensure consistent baseline distances. The optimization of the initial calibration distance is expressed as follows:

[0016]

[0017] Where Δz is the distance error; z is the initial calibration distance between the target object and the stereo camera; B is the stereo camera baseline; β is the optical axis angle between the left and right cameras in the stereo camera; and f is the focal length of the stereo camera.

[0018] S12. Using a well-adjusted binocular camera, collect calibration image datasets of the chessboard calibration board in various poses based on the quasi-isotropic overlay mapping method.

[0019] S13. Based on the calibration image dataset, the stereo camera is calibrated using a hierarchical joint calibration method to obtain the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the stereo camera.

[0020] S14. Based on the calibration results, readjust the optical axes of the left and right cameras in the binocular camera.

[0021] Further, step S12 specifically includes:

[0022] 1) Construct a non-uniform metric tensor on the SE(3) manifold. By dynamically adjusting the weights of the rotation and translation components, pose sampling exhibits local isotropy and global quasi-uniform coverage characteristics in the SE(3) manifold. The non-uniform metric tensor is defined as:

[0023]

[0024] Among them, G QIM For non-uniform metric tensors; w R Adaptive weights for rotational components; w t For the adaptive weights of the translation components, w R +w t =1; I3 is the identity matrix; ||t|| is the magnitude of the translation vector; z is the initial calibration distance;

[0025] 2) The SE(3) manifold is mapped to hyperbolic space using hyperbolic space embedding techniques;

[0026] 3) In hyperbolic space, generate a finite number of sampling points with the goal of maximizing the coverage entropy:

[0027] 4) Collect calibration image datasets of the checkerboard calibration board in various poses based on the sampling points.

[0028] Furthermore, step S13 specifically includes:

[0029] (1) Define the intrinsic parameter matrix and the extrinsic parameter matrix; the extrinsic parameter matrix includes rotation components and translation components;

[0030] (2) The corner weights of each calibration image in the calibration image dataset are optimized using dynamic weights:

[0031] (3) Establish a hierarchical joint calibration process:

[0032] 1) Calibrate the intrinsic parameter matrix of the stereo camera:

[0033] Assuming the distortion coefficients are D, and D = {k1,k2,k3,p1,p2}, an improved cost function is used as the optimization method to perform monocular high-precision intrinsic parameter matrix calibration; the cost function L is expressed as:

[0034]

[0035] Among them, X ij λ represents the actual observed pixel coordinates of the j-th corner point in the i-th calibration image; N is the total number of calibration images; M is the total number of observable corner points in the i-th calibration image; λ is the regularization coefficient.

[0036] Simultaneously, the ability to model long-range field distortion is enhanced by using a sixth-order radial distortion term calibrated at a distance, expressed as:

[0037] x dist =x(1+k1r) 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0038] Where, x dist is the sixth-order radial distortion term; x and y are the x and y coordinates of the ideal image point; k1, k2, and k3 are all radial distortion coefficients; p1 and p2 are both tangential distortion coefficients; r is the radial distance;

[0039] 2) Calibrate the extrinsic parameter matrix of the stereo camera:

[0040] In the traditional optimization of the extrinsic parameter matrix, an epipolar alignment loss is added; the epipolar alignment loss L epi Represented as:

[0041]

[0042] Where E is the essential matrix, which forces the matching points of the stereo image to satisfy the epipolar constraint; x l x is the coordinate point matched by the left image in the i-th calibration image; r Let be the coordinates of the right image matched in the i-th calibration image;

[0043] Define the joint optimization objective function and calibrate the extrinsic parameter matrix; the joint optimization objective function L joint Represented as:

[0044]

[0045] Where, π l (X i ) is the projection function of the left camera in a binocular camera setup; π r (X i x is the projection function of the right camera in a stereo camera setup; li For x l The actual observed coordinates; x ri For x r The actual observed coordinates; γ is the geometric constraint weight; t is the translation component of the binocular camera; R is the rotation component of the binocular camera;

[0046] 3) Repeat the above calibration of the intrinsic and extrinsic parameter matrices, and take the average of the intrinsic and extrinsic parameter calibration results as the final calibration result of the binocular camera.

[0047] Further, in step S3, the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients obtained from the binocular camera calibration are used to perform stereo correction on each group of binocular images; specifically, this includes:

[0048] (1) Decompose the rotation components in the extrinsic matrix into left rotation components and right rotation components;

[0049] (2) Using scene depth perception factors, construct the energy function E when decomposing the rotation component R. res :

[0050]

[0051] Wherein, S(r l ·r r -I) is the depth sensing factor function; λ is the regularization coefficient; L is the cost function; I is the identity matrix; R is the rotation component; rl For the left rotation component; r r This is the right rotation component;

[0052] (3) By dynamically optimizing the decomposition process, the left rotation component r l and right rotation component r r While maintaining orthogonal constraints, prioritize aligning the stereo correspondence points in high-confidence regions;

[0053] (4) Calculate a new rotation matrix that makes the principal optical axes of the left and right images in the binocular image parallel to the baseline of the binocular camera; rotate the left and right images using the new rotation matrix.

[0054] (5) Based on the intrinsic parameter matrix, and combined with the rotated left and right images, calculate the new left projection matrix and the new right projection matrix;

[0055] (6) Based on the new rotation matrix, the new left projection matrix and the new right projection matrix, distortion correction is performed through distortion coefficients to obtain the corrected binocular image.

[0056] Further, in step S3, based on the trained Yolov5 network model, the common ROI region of each group of stereo images is obtained; specifically including:

[0057] (1) Input the left and right images from each set of binocular images into the trained Yolov5 network model, and output the bounding box coordinates of the localized object in the left and right images respectively; expressed as:

[0058]

[0059] Where, x l and y l x represents the bounding box coordinates of the located object in the left image; r and y r x represents the bounding box coordinates of the located object in the right image; center and y center The coordinates of the center point of the bounding box; width is the width of the bounding box; height is the height of the bounding box;

[0060] (2) Obtain the bounding box coordinates of the ROI regions corresponding to the left and right images respectively, and obtain the common ROI region of the stereo images based on this; wherein, the bounding box coordinates of the common ROI region are:

[0061]

[0062] Where (x'1,y'1) and (x'2,y'2) are the bounding box coordinates of the common ROI region; (x ll ,y ll(x) represents the coordinates of the top-left bounding box of the left image; lr ,y lr (x) represents the coordinates of the bottom right bounding box of the left image; rl ,y rl (x) represents the coordinates of the top-left bounding box of the right image; rr ,y rr ) represents the coordinates of the bottom right bounding box of the right image.

[0063] Furthermore, the training process of the Yolov5 network model includes:

[0064] (1) For stereo-corrected binocular images, translation, rotation and mirroring operations are performed through optimized affine transformation to achieve data augmentation of the binocular image dataset;

[0065] The optimization process of affine transformation includes:

[0066] Establish an SE(3)-equivariant convolution kernel and design a convolution operator that satisfies the invariance of rigid body motion; process the affine transformation based on the convolution operator;

[0067] The convolution operator is represented as follows:

[0068] K(g·v)=ρ(g)·K(v)·ρ(g) -1

[0069] v = (x, y, z, θ) x ,θ y ,θ z )

[0070]

[0071] Where K(g·v) is the convolution operator; ρ(g) is the representation matrix of the rigid body transformation g; K(v) is the convolution operator applied to the input data v without rigid body transformation; x, y, and z are the position information of the input image; θ x ,θ y and θ z The input image contains attitude angle information; R p t represents the rotation component of an image during image data processing. p Represents the translation component of an image during image processing;

[0072] The parameterization method of convolution operator expansion using spherical harmonic function basis is adopted:

[0073]

[0074] in, This represents the convolution kernel defined in spherical coordinates; Q is the maximum spherical harmonic order; l represents the current spherical harmonic order; m represents the quantum number of the in-order orientation angle; r represents the radial distance; θ represents the horizontal direction angle in a spherical coordinate system. s w represents the vertical direction angle in spherical coordinates. lm The weighting coefficients represent the basis of the spherical harmonic function; It is a basis for spherical harmonic functions;

[0075] (2) Based on the enhanced stereo image dataset, the parameters of the YOLOv5 network model are optimized and trained.

[0076] Further, step S4 specifically includes:

[0077] (1) Use the CREStereo deep learning stereo matching algorithm to calculate the disparity map of the left image in each group of binocular images;

[0078] (2) Noise information in the disparity map is removed by optimized multi-scale bilateral filtering;

[0079] The optimization method for multi-scale bilateral filtering is as follows: A dynamic kernel parameter adjustment mechanism is introduced based on traditional bilateral filtering to establish a spatial color gamut joint control model based on local texture complexity; the multi-scale bilateral filtering is optimized using this spatial color gamut joint control model; the spatial color gamut joint control model is expressed as follows:

[0080]

[0081] in, Let be the gradient magnitude of pixel q; H(q) represents the local information entropy centered at q; α, β, and γ are all learnable parameters; σ s σ represents the standard deviation of the dynamic spatial domain. s0 σ is the standard deviation of the spatial domain reference. r The standard deviation of the dynamic color gamut; σ r0 E|H| is the baseline standard deviation of the color gamut; E|H| is the global average value of the information entropy of the entire image.

[0082] (3) The disparity map is smoothed by energy functional optimization algorithm;

[0083] The energy functional optimization algorithm involves introducing a curvature constraint term into cubic spline interpolation; the formula for the energy functional optimization algorithm is as follows:

[0084]

[0085] Where E(d) is the total energy; λ E μ is the weighting coefficient of the smoothing term. Erepresents the weighting coefficient of the curvature constraint term; K represents the curvature of the image edge. is the second derivative of the parallax field; It is the first derivative of the parallax field.

[0086] Further, step S5 specifically includes:

[0087] (1) Use the Shi-Tomasi corner detection algorithm based on autocorrelation matrix to identify corners in the left image: use the Shi-Tomasi corner detection algorithm to slide a fixed window in any direction on the left image and calculate the change in gray level within the fixed window; calculate the autocorrelation matrix of the region around each pixel to evaluate the degree of gray level change; if the gray level changes significantly when the window slides, it is considered that the window contains corners.

[0088] Further, step S6 specifically includes:

[0089] (1) Convert the disparity map into a depth map;

[0090] (2) Use adaptive mean filtering to remove outlier depth information from the depth map;

[0091] (3) The final coordinates in the camera coordinate system are calculated based on the transformation relationship between the pixel coordinate system and the world coordinate system.

[0092] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a three-dimensional spatial positioning method based on hierarchical joint calibration, which has the following beneficial effects:

[0093] This invention provides a high-precision long-distance calibration method for fixed-focus dual-layer combined binocular systems, which effectively reduces the reprojection error in long-distance binocular system calibration.

[0094] This invention calculates disparity maps within the common ROI region of binocular images, enabling the reduction of image resolution without altering image scale, the removal of redundant image information, and the elimination of subsequent disparity calculations.

[0095] This invention uses dynamic kernel parameter adaptive multi-scale bilateral filtering to optimize the disparity map, remove noise in the disparity map, and then combines curvature-constrained cubic spline interpolation to make the pixel changes between adjacent pixels smoother, which can effectively alleviate the layering phenomenon of 3D reconstructed point cloud map.

[0096] The inventors use a dynamic kernel parameter adaptive multi-scale bilateral filtering algorithm to remove noise during long-distance shooting, and combine it with the Shi-Tomasi corner detection algorithm based on autocorrelation matrix to improve the corner recognition effect of circular diagonal marks, thereby improving the pixel coordinate extraction accuracy.

[0097] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0098] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0099] Figure 1 This is a schematic diagram of the three-dimensional spatial positioning method based on hierarchical joint calibration provided in an embodiment of the present invention.

[0100] Figure 2 This is a schematic diagram of a checkerboard calibration plate provided in an embodiment of the present invention.

[0101] Figure 3 This is a schematic diagram of a long-range binocular system provided in an embodiment of the present invention.

[0102] Figure 4 This is a schematic diagram of the calibration process for a dual-layer joint long-range binocular system provided in an embodiment of the present invention.

[0103] Figure 5 This is a schematic diagram of the ROI region calculation process for a stereo system based on target detection, provided in an embodiment of the present invention.

[0104] Figure 6 This is a schematic diagram outlining the CREStereo deep learning network provided in an embodiment of the present invention.

[0105] Figure 7 This is a schematic diagram of a circular diagonal mark provided in an embodiment of the present invention.

[0106] Figure 8 This is a schematic diagram of the Shi-Tomasi corner detection algorithm based on the autocorrelation matrix provided in an embodiment of the present invention.

[0107] Figure 9 This is a schematic diagram illustrating the imaging principle of a binocular vision theoretical camera provided in an embodiment of the present invention.

[0108] Figure 10 A schematic diagram of the accuracy test results provided in an embodiment of the present invention.

[0109] Figure 11 This is a schematic diagram of a multiphase array visual positioning system provided in an embodiment of the present invention. Detailed Implementation

[0110] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0111] This invention discloses a three-dimensional spatial positioning method based on hierarchical joint calibration. See [link to relevant documentation]. Figure 1 As shown, it includes the following steps:

[0112] S1. Calibrate the binocular camera using a long-range binocular system calibration method;

[0113] S2. Use the calibrated binocular camera to acquire a binocular image dataset of the object to be located;

[0114] S3. After performing stereo correction on each group of stereo images in the stereo image dataset, obtain the common ROI region of each group of stereo images.

[0115] S4. Calculate the disparity map of each set of binocular images within the common ROI region using a deep learning-based stereo matching algorithm;

[0116] S5. Obtain the pixel coordinates of the object being located using the adaptive Shi-Tomasi circular diagonal marker corner detection algorithm;

[0117] S6. Based on the pixel coordinates of the object being located, and combined with the depth map transformed from the disparity map, obtain the final coordinates of the object being located in three-dimensional space.

[0118] Next, each of the above steps will be explained in detail:

[0119] In step S1 above, the binocular camera is calibrated using a long-range binocular system calibration method; specifically, this includes:

[0120] S11. Calibration distance setting and binocular camera adjustment:

[0121] (1) Set the initial calibration distance between the target object and the binocular camera, that is, the initial calibration distance between the checkerboard calibration board to be used later and the binocular camera;

[0122] In this embodiment of the invention, for long-distance high-precision positioning measurement tasks, the distance between the calibration board and the binocular system during calibration is set to 20m, and the reprojection error of the calibration result needs to be controlled below 0.1 pixels to achieve millimeter-level positioning accuracy;

[0123] (2) Optimize the initial calibration distance based on the actual scenario; expressed as:

[0124]

[0125] Where Δz represents the distance error; z is the initial calibration distance between the target object and the stereo camera; B is the stereo camera baseline; β is the optical axis angle between the left and right cameras in the stereo camera system; and f is the focal length of the stereo camera. In this embodiment of the invention, an adaptive optical axis angle adjustment technology is introduced. A high-precision servo motor adjusts the optical axis angle β of the two cameras in real time, ensuring that the distance error Δz is minimized, achieving millimeter-level positioning accuracy within a range of 15-25 meters. This technology, combined with the level and the initial calibration results of the stereo system, enables the construction of a high-precision parallel stereo system structure.

[0126] (3) Use two high-resolution industrial cameras with fixed-focus lenses to form a binocular system. Initially adjust the optical axes of the left and right cameras in the binocular system to be nearly parallel and ensure that the baseline distance is consistent.

[0127] S12. Design a checkerboard calibration board and collect a calibration image dataset:

[0128] (1) Design a chessboard calibration board:

[0129] Obtain the number of projected pixels of a single checkerboard grid in the image, and determine the optimal size of the black and white checkerboard calibration board by constraining the projected pixels (e.g., the projected pixels must be greater than 30 pixels).

[0130] Wherein, the number of projected pixels N c Represented as:

[0131]

[0132] Where Z is the calibration distance; f is the focal length of the binocular camera; s is the sensor pixel size; L c The side length of a single chessboard square;

[0133] For example, when the distance between the checkerboard calibration board and the binocular system is set to 20m during calibration, in order to achieve millimeter-level positioning accuracy at 0.1 pixels at 20m, the pixel and spatial error err is calculated as follows:

[0134]

[0135] Therefore, a side length of 200mm for each chessboard square was determined to meet the requirements. The calibration plate has 5 chessboard squares horizontally and 8 chessboard squares vertically, with a total size of 1.1 × 1.7m. For details of this chessboard calibration plate, please refer to [link / reference needed]. Figure 2 As shown. In this embodiment of the invention, the optimal size of the checkerboard is determined so that the corners are clearly distinguishable when photographed from a distance, while ensuring that the size is appropriate and can be used normally in most actual sites, avoiding the problem of inapplicability due to an excessively large calibration board.

[0136] (2) Using a well-adjusted binocular camera, a calibration image dataset of a chessboard calibration board in various poses is acquired based on the quasi-isotropic cover mapping method; wherein, the quasi-isotropic cover mapping method addresses the problems of insufficient singularity avoidance and anisotropic orientation distribution in Lie group pose sampling, specifically including:

[0137] 1) Construct a non-uniform metric tensor on the SE(3) manifold. By dynamically adjusting the weights of the rotation and translation components, pose sampling exhibits local isotropy and global quasi-uniform coverage characteristics in the SE(3) manifold. The non-uniform metric tensor is defined as:

[0138]

[0139] Among them, G QIM For non-uniform metric tensors; w R Adaptive weights for rotational components; w t For the adaptive weights of the translation components, w R +w t =1; I3 is the identity matrix; ||t|| is the magnitude of the translation vector; z is the initial calibration distance;

[0140] 2) The SE(3) manifold is mapped to hyperbolic space using hyperbolic space embedding techniques, where the SE(3) mapping parameters are... Represented as:

[0141]

[0142] Where R' is the rotation component between the binocular camera and the calibration plate; t is the translation component of the binocular camera;

[0143] 3) In hyperbolic space, generate a finite number of sampling points P with the goal of maximizing the coverage entropy:

[0144]

[0145] Where, d H p is a distance function defined in the hyperbolic space of the Poincaré disk model; x p represents the sampling points along the x-axis. y The sampling points are the y-axis. By obtaining the maximum value, the sampling points are dispersed as much as possible. By utilizing the exponential growth characteristics of hyperbolic geometry, the singular directions of SE(3) are covered within a finite number of samplings, avoiding the pose aggregation problem in traditional methods, which leads to calibration deviation.

[0146] 4) Collect calibration image datasets of the checkerboard calibration board in various poses based on sampling points; furthermore, during the collection of calibration image data, the movement of the calibration board can be controlled by a six-degree-of-freedom robotic arm; in this embodiment of the invention, when collecting the calibration dataset, the binocular system remains stationary, the checkerboard is placed 20m away from the binocular system, and the movement range cannot exceed the camera's depth of field. Based on the quasi-isotropic overlay mapping method, 13 sets of calibration image datasets of the checkerboard calibration board in 13 different poses are collected. A schematic diagram of the binocular system structure is shown below. Figure 3 As shown;

[0147] S14. Based on the calibration results, readjust the optical axes of the left and right cameras in the binocular camera.

[0148] S13. Based on the calibration image dataset, a hierarchical joint calibration method is used to calibrate the stereo camera, obtaining the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the stereo camera; specifically including:

[0149] (1) Define the intrinsic parameter matrix and the extrinsic parameter matrix; the extrinsic parameter matrix includes rotation and translation components; the intrinsic parameter matrix K is represented as:

[0150]

[0151] Among them, f x f is the focal length of the stereo camera on the x-axis; y y is the focal length of the binocular camera on the y-axis; u0 is the x-coordinate of the intersection of the optical axis and the imaging plane on the image plane; v0 is the y-coordinate of the intersection of the optical axis and the imaging plane on the image plane.

[0152] The extrinsic parameter matrix and the rotation matrix are represented as follows: Where R is the rotation component, specifically a 3×3 matrix, representing the rotation relationship between the two cameras; T is the translation matrix, specifically a 3×1 matrix, representing the relative position between the two cameras; the last row is

[0001] , used for homogeneous coordinate transformation.

[0153] (2) The corner weights of each calibration image in the calibration image dataset are optimized using dynamic weights, as follows:

[0154]

[0155] Among them, w i c represents the corner weights of the i-th calibration image; i c is the confidence score of the i-th calibration image; j Let M be the confidence score of the j-th corner point; M is the number of corner points in the i-th calibration image.

[0156] (3) Establish a hierarchical joint calibration process:

[0157] 1) Calibrate the intrinsic parameter matrix of the stereo camera:

[0158] Assume the distortion coefficients are D, and D = {k1,k2,k3,p1,p2}; use the improved cost function as the optimization method to perform monocular high-precision intrinsic parameter matrix calibration; the cost function L is expressed as:

[0159]

[0160] Among them, X ij λ represents the actual observed pixel coordinates of the j-th corner point in the i-th calibration image; N is the total number of calibration images; M is the total number of observable corner points in the i-th calibration image; λ is the regularization coefficient.

[0161] Simultaneously, the ability to model long-range field distortion is enhanced by using a sixth-order radial distortion term calibrated at a distance, expressed as:

[0162] x dist =x(1+k1r) 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0163] Where, x dist is the sixth-order radial distortion term; x and y are the x and y coordinates of the ideal image point; k1, k2, and k3 are all radial distortion coefficients; p1 and p2 are both tangential distortion coefficients; r is the radial distance;

[0164] 2) Calibrate the extrinsic parameter matrix of the stereo camera:

[0165] In the traditional optimization of the extrinsic parameter matrix, an epipolar alignment loss is added; the epipolar alignment loss L epi Represented as:

[0166]

[0167] Where E is the essential matrix, which forces the matching points of the stereo image to satisfy the epipolar constraint; x l x is the coordinate point matched by the left image in the i-th calibration image; r Let be the coordinates of the right image matched in the i-th calibration image;

[0168] Define the joint optimization objective function and calibrate the extrinsic parameter matrix; the joint optimization objective function L joint Represented as:

[0169]

[0170] Where, π l(X i ) is the projection function of the left camera in a binocular camera setup; π r (X i x is the projection function of the right camera in a stereo camera setup; li For x l The actual observed coordinates; x ri For x r The actual observed coordinates; γ is the geometric constraint weight; t is the translation component of the stereo camera; R is the rotation component in the stereo camera's extrinsic matrix;

[0171] 3) Repeat the above calibration process for the intrinsic and extrinsic parameter matrices, and take the average of the intrinsic and extrinsic parameter calibration results as the final calibration result for the stereo camera. The stereo system calibration process is as follows: Figure 4 As shown. In this embodiment of the invention, by effectively eliminating random errors, the stability and reliability of the calibration results are improved.

[0172] In step S2 above, a binocular image dataset of the object to be located is acquired using a calibrated binocular camera.

[0173] In step S3 above, after stereo correction is performed on each group of stereo images in the stereo image dataset, the common ROI region of each group of stereo images is obtained; specifically, this includes the following:

[0174] S31. Using the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients obtained from the binocular camera calibration, perform stereo correction on each group of binocular images (including the left and right images); specifically including:

[0175] (1) Decompose the rotation component R in the extrinsic parameter matrix obtained during the calibration process into a left rotation component r. l and right rotation component r r , represented as:

[0176]

[0177] (2) Using scene depth perception factors, construct the energy function E when decomposing the rotation component R. res :

[0178]

[0179] Wherein, S(r l ·r r -I) is the depth sensing factor function; λ is the regularization coefficient; L is the cost function; I is the identity matrix;

[0180] (3) By dynamically optimizing the decomposition process, the left rotation component r l and right rotation component r rWhile maintaining orthogonality constraints, the three-dimensional corresponding points in high-confidence regions are preferentially aligned using the row alignment matrix;

[0181] To adjust the epipolar lines of the images and achieve horizontal alignment between the left and right camera images, a tool called the row alignment matrix is ​​introduced, denoted as: R rect =[e1, e2, e3]

[0182]

[0183] Where T is the translation matrix, and T = [t x , t y , t z ] T ;t x t y and t z e1, e2, and e3 represent the translation distances of the stereo camera along the x, y, and z axes, respectively; the superscript T indicates transpose; e1, e2, and e3 represent the x, y, and z axes of the rotated coordinate system, respectively.

[0184] (4) Calculate a new rotation matrix that makes the principal optical axes of the left and right images in the binocular images parallel and parallel to the baseline of the binocular camera; rotate the left and right images using the new rotation matrix; wherein, the new rotation matrix is ​​expressed as:

[0185]

[0186] Among them, R l R represents the rotation component corresponding to the left image. r This represents the rotation component corresponding to the right image;

[0187] (5) Based on the intrinsic parameter matrix, and combined with the rotated left and right images, calculate the new left projection matrix and the new right projection matrix;

[0188] (6) Based on the new rotation matrix, the new left projection matrix and the new right projection matrix, distortion correction is performed through distortion coefficients to obtain the corrected binocular image.

[0189] S32. Because high-resolution industrial cameras can improve the image acquisition effect of binocular systems and reduce the system error of binocular systems, high-resolution images will reduce the system operating speed. In this embodiment, such as... Figure 5 As shown, a method for calculating the Region of Interest (ROI) in a stereo system based on object detection is proposed. Specifically, based on a trained Yolov5 network model, the common ROI region of each group of stereo images is obtained; this includes:

[0190] (1) Input the left and right images from each set of binocular images into the trained Yolov5 network model, and output the bounding box coordinates of the localized object in the left and right images respectively; expressed as:

[0191]

[0192] Where, x l and y l x represents the bounding box coordinates of the located object in the left image; r and y r x represents the bounding box coordinates of the located object in the right image; center and y center The coordinates of the center point of the bounding box; width is the width of the bounding box; height is the height of the bounding box;

[0193] (2) Obtain the bounding box coordinates of the ROI regions corresponding to the left and right images respectively, and obtain the common ROI region of the stereo images based on this; wherein, the bounding box coordinates of the common ROI region are:

[0194]

[0195] Where (x'1,y'1) and (x'2,y'2) are the bounding box coordinates of the common ROI region; (x ll ,y ll (x) represents the coordinates of the top-left bounding box of the left image; lr ,y lr (x) represents the coordinates of the bottom right bounding box of the left image; rl ,y rl (x) represents the coordinates of the top-left bounding box of the right image; rr ,y rr ) represents the coordinates of the bottom right bounding box of the right image.

[0196] In this embodiment of the invention, redundant image information in high-resolution images is removed by the ROI region calculation module of a binocular system based on object detection, thereby improving the disparity map calculation speed of the deep learning model.

[0197] The training process of the above Yolov5 network model includes:

[0198] (1) For stereo-corrected binocular images, translation, rotation and mirroring operations are performed through optimized affine transformation to achieve data augmentation of the binocular image dataset;

[0199] 1) The basic formula for affine transformation is:

[0200]

[0201] Where X' and Y' are the data after the affine transformation; X and Y are the data before the affine transformation; and A is the affine transformation matrix.

[0202] In translation operations, Where t x and t y These represent the translational distances of the binocular camera along the x and y axes, respectively.

[0203] In rotational operations, Where θ is the angle of rotation of the image around the center, in radians;

[0204] During mirroring operations, when When the image is mirrored along the y-axis, At this time, the image is mirrored along the x-axis, where h and w are used to restore the flipped image to its original position;

[0205] 2) The optimization process of affine transformation includes:

[0206] Establish an SE(3)-equivariant convolution kernel and design a convolution operator that satisfies the rigid body motion invariance; process the affine transformation based on the convolution operator; ensure that the rigid body transformation of the input data has analytical equivariance;

[0207] The convolution operator is represented as follows:

[0208] K(g·v)=ρ(g)·K(v)·ρ(g) -1

[0209] v = (x, y, z, θ) x ,θ y ,θ z )

[0210]

[0211] Where K(g·v) is the convolution operator; ρ(g) is the representation matrix of the rigid body transformation g; K(v) is the convolution operator applied to the input data v without rigid body transformation; x, y, and z are the position information of the input image; θ x ,θ y and θ z The input image contains attitude angle information; R p t represents the rotation component of an image during image data processing. p Represents the translation component of an image during image processing;

[0212] Parameterization method for convolution operators using spherical harmonic basis expansion:

[0213]

[0214] in, This represents the convolution kernel defined in spherical coordinates; Q is the maximum spherical harmonic order; l represents the current spherical harmonic order; m represents the quantum number of the in-order orientation angle; r represents the radial distance; θ represents the horizontal direction angle in a spherical coordinate system. s w represents the vertical direction angle in spherical coordinates. lm The weighting coefficients represent the basis of the spherical harmonic function; It is a basis for spherical harmonic functions;

[0215] In a binocular vision system, the target object may appear in the field of view at any angle. SE(3)-equivariant convolution can automatically align the feature representations of different viewpoints through mathematical constraints and reduce the dependence on pose enhancement data. While reducing the resolution of the original images of the left and right cameras, it does not change the original image scale, deletes redundant image information and does not affect the subsequent disparity calculation of the located object.

[0216] (2) Based on the enhanced stereo image dataset, the parameters of the YOLOv5 network model are optimized and trained, and the training speed is accelerated by GPU.

[0217] In step S4 above, the disparity map of each group of binocular images within the common ROI region is calculated using a deep learning-based stereo matching algorithm; specifically, this includes:

[0218] (1) The CREStereo deep learning stereo matching algorithm was used to calculate the disparity map of the left image in each set of binocular images; the CREStereo deep learning network used was described in the following flowchart. Figure 6 As shown. The advantages of choosing the CREStereo deep learning stereo matching model include: the deep learning model is designed with a stacked cascade architecture to adapt to high-resolution image processing; an adaptive group local correlation layer (AGCL) is designed for feature matching to mitigate the negative impact of correction errors; and a new synthetic dataset with richer variations in lighting, texture, and shape is introduced to better generalize to real-world scenes.

[0219] (2) Noise information in the disparity map is removed by optimized multi-scale bilateral filtering;

[0220] Since the CREStereo network output does not limit the disparity range, directly converting the disparity map into a depth map will result in depth map layering. Therefore, in this embodiment of the invention, a dynamic kernel parameter adjustment mechanism is introduced on the basis of traditional bilateral filtering to establish a spatial color gamut joint control model based on local texture complexity; the multi-scale bilateral filtering is optimized through the spatial color gamut joint control model; wherein, the spatial color gamut joint control model is expressed as:

[0221]

[0222] in, Let be the gradient magnitude of pixel q; H(q) represents the local information entropy centered at q; α, β, and γ are all learnable parameters; σ s σ represents the standard deviation of the dynamic spatial domain. s0 σ is the standard deviation of the spatial domain reference. r The standard deviation of the dynamic color gamut; σ r0 E|H| is the baseline standard deviation of the color gamut; E|H| is the global average value of the information entropy of the entire image.

[0223] (3) The disparity map is smoothed by energy functional optimization algorithm;

[0224] The energy functional optimization algorithm involves introducing a curvature constraint term into cubic spline interpolation; the formula for the energy functional optimization algorithm is as follows:

[0225]

[0226] Where E(d) is the total energy, and the energy functional optimization objective is to minimize this energy; λ E μ is the weighting coefficient of the smoothing term. E where is the weighting coefficient of the curvature constraint term; K is the image edge curvature, constructing an edge curvature field based on anisotropic diffusion; is the second derivative of the parallax field; This is the first derivative of the disparity field. In this embodiment of the invention, the disparity information of the disparity map is optimized by curvature-constrained cubic spline interpolation, making the pixel value changes between adjacent pixels smoother and reducing errors.

[0227] In step S5 above, such as Figure 7 As shown, this invention obtains pixel coordinates by performing corner point recognition on a circular diagonal marker placed on the object to be positioned. The implementation process is as follows: Figure 8 As shown, the image is filtered before extracting the corner points of the circular diagonal markers, and then the pixel coordinates of the object to be located are obtained through the adaptive Shi-Tomasi circular diagonal marker corner detection algorithm; specifically including:

[0228] (1) Corner detection algorithm based on autocorrelation matrix is ​​used to identify corners in the left image: a fixed window is slid along any direction on the left image using the Shi-Tomasi corner detection algorithm, and the change in gray level within the fixed window is calculated; the degree of gray level change is evaluated by calculating the autocorrelation matrix of the region around each pixel; if the gray level changes significantly when the window is slid, it is considered that the window contains corners;

[0229] The autocorrelation function is expressed as follows:

[0230]

[0231] Among them, (x p ,y p I(x) represents the pixel coordinates of window o, which are considered as the pixel coordinates of the object being positioned; p ,y p ) is (x p ,y p The corresponding pixel grayscale value; I(x) p +u,y p +v) represents the pixel grayscale value after the binocular image is moved by a displacement u in the x-axis direction and a displacement v in the y-axis direction;

[0232] For I(x) p +u,y p +v) can be approximated by a second Taylor expansion:

[0233]

[0234] Further simplification yields:

[0235]

[0236] Among them, I x Represents the pixel grayscale value on the x-axis corresponding to window o; I y This represents the pixel grayscale value on the y-axis corresponding to window o; The corner response function is R s =min(λ1,λ2)=λ min When λ min If the value exceeds the set threshold, it is considered a corner point;

[0237] The results of the Shi-Tomasi corner detection algorithm are iteratively optimized by combining the Gradient Descent algorithm. The optimization rules are as follows:

[0238]

[0239] Where, x i+1 and y i+1 The variable x represents the value of the current iteration step. i and y i This represents the value of the previous iteration step, and α represents the learning rate.

[0240] In step S6 above, the final coordinates of the object being located in three-dimensional space are obtained based on the pixel coordinates of the object and the depth map transformed from the disparity map; specifically, this includes:

[0241] (1) Convert the disparity map into a depth map, represented as:

[0242]

[0243] Where depth is the depth map, which is the Z coordinate in the binocular camera coordinate system; b is the baseline length; f is the focal length of the binocular camera in the calibration result; and d is the disparity value in the disparity map.

[0244] (2) Use adaptive mean filtering to remove outlier depth information from the depth map;

[0245] (3) The X and Y coordinates in the camera coordinate system are calculated based on the transformation relationship between the pixel coordinate system and the world coordinate system. The calculation formula is as follows:

[0246]

[0247] Where (x,y) are the pixel coordinates of the object being located; (X0,Y0) are the optical center coordinates of the stereo camera in the camera calibration results.

[0248] Specifically, in binocular vision theory, a schematic diagram of the camera imaging model is shown below. Figure 9 As shown, P is a point in the world coordinate system. l and P r P is the image point of P on the left and right camera imaging planes; X l and X r It is the distance from the imaging point to the left imaging plane, and Z is the Z coordinate of point P in the camera coordinate system xoy. The formula for converting the disparity value into depth information is as follows:

[0249] The disparity value d at point P can be expressed as:

[0250] d=|X l -X r |

[0251] Imaging point P l and P r The distance between them can be expressed as:

[0252]

[0253] It can be deduced that:

[0254]

[0255] According to the principle of similar triangles, it is easy to know that:

[0256]

[0257] Similarly, we can obtain:

[0258]

[0259] Among them, P X and P Y x and y are the coordinates of P in the camera coordinate system, x and y are the coordinates of the image point of P in the pixel coordinate system, and x0 and y0 are the coordinates of the camera optical center in the pixel coordinate system.

[0260] The final formulas for calculating the X and Y coordinates in the camera coordinate system are as follows:

[0261]

[0262] The measurement accuracy of this method was verified through experiments, and the distance measurement results are as follows: Figure 10 As shown in Table 1.

[0263] Table 1

[0264] Position 1 Position 2 Position 3 Position 4 Position 5 Position 6 Position 7 Position 8 Position 9 Position 10 Position 11 Position 12 Position 13 Position 14 Position 15 Laser ranging / mm 13952 14849 15745 16632 17543 18428 19387 20752 21474 22383 23289 24181 24651 24816 25012 Distance measurement result of this method / mm 0.65965 -1.5405 -3.2154 4.15022 1.48205 1.03186 -2.5679 -2.5859 4.99895 -3.0388 -2.9509 6.13292 -0.9617 10.1059 -11.701

[0265] Using the method proposed in this invention, when the working range of the binocular vision positioning system is 13.952m-24.65m, the ranging results can achieve millimeter-level measurement error, with an average ranging error of 2.55mm. In practical applications, this invention can be used to build a multi-phase array vision positioning system based on the aforementioned single binocular system, such as... Figure 11 As shown.

[0266] In summary, the three-dimensional spatial positioning method based on hierarchical joint calibration provided by this invention reduces the systematic error of traditional low-resolution binocular systems, improves the operating speed of high-resolution binocular systems, and enhances the long-distance three-dimensional spatial positioning accuracy based on binocular vision, thus achieving tracking and positioning of the object being located. When applied to the hoisting of large industrial equipment (i.e., when the object being located is large industrial equipment), it helps to achieve long-distance high-precision positioning, improves the automation level of large industrial equipment positioning and hoisting tasks, reduces the personnel injury rate in large equipment hoisting accidents, and improves the efficiency of large industrial equipment positioning and installation.

[0267] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0268] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional spatial positioning method based on hierarchical joint calibration, characterized in that, include: S1. Calibrate the binocular camera using a long-range binocular system calibration method; S2. Use the calibrated binocular camera to acquire a binocular image dataset of the object to be located; S3. After performing stereo correction on each group of stereo images in the stereo image dataset, obtain the common ROI region of each group of stereo images. S4. Calculate the disparity map of each set of binocular images within the common ROI region using a deep learning-based stereo matching algorithm; S5. Obtain the pixel coordinates of the object being located using the adaptive Shi-Tomasi circular diagonal marker corner detection algorithm; S6. Based on the pixel coordinates of the object being located, and combined with the depth map transformed from the disparity map, obtain the final coordinates of the object being located in three-dimensional space.

2. The three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 1, characterized in that, Step S1 specifically includes: S11. Set the initial calibration distance between the target object and the stereo camera. After optimizing the initial calibration distance, initially adjust the optical axes of the left and right cameras in the stereo camera to be nearly parallel and ensure consistent baseline distances. The optimization of the initial calibration distance is expressed as follows: Where Δz is the distance error; z is the initial calibration distance between the target object and the stereo camera; B is the stereo camera baseline; β is the optical axis angle between the left and right cameras in the stereo camera; and f is the focal length of the stereo camera. S12. Using a well-adjusted binocular camera, collect calibration image datasets of the chessboard calibration board in various poses based on the quasi-isotropic overlay mapping method. S13. Based on the calibration image dataset, the stereo camera is calibrated using a hierarchical joint calibration method to obtain the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the stereo camera. S14. Based on the calibration results, readjust the optical axes of the left and right cameras in the binocular camera.

3. The three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 2, characterized in that, Step S12 specifically includes: 1) Construct a non-uniform metric tensor on the SE(3) manifold. By dynamically adjusting the weights of the rotation and translation components, pose sampling exhibits local isotropy and global quasi-uniform coverage characteristics in the SE(3) manifold. The non-uniform metric tensor is defined as: Among them, G QIM For non-uniform metric tensors; w R Adaptive weights for rotational components; w t For the adaptive weights of the translation components, w R +w t =1; I3 is the identity matrix; ||t|| is the magnitude of the translation vector; z is the initial calibration distance; 2) The SE(3) manifold is mapped to hyperbolic space using hyperbolic space embedding techniques; 3) In hyperbolic space, generate a finite number of sampling points with the goal of maximizing the coverage entropy: 4) Collect calibration image datasets of the checkerboard calibration board in various poses based on the sampling points.

4. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 2, characterized in that, Step S13 specifically includes: (1) Define the intrinsic parameter matrix and the extrinsic parameter matrix; the extrinsic parameter matrix includes rotation components and translation components; (2) The corner weights of each calibration image in the calibration image dataset are optimized using dynamic weights: (3) Establish a hierarchical joint calibration process: 1) Calibrate the intrinsic parameter matrix of the stereo camera: Assuming the distortion coefficients are D, and D = {k1,k2,k3,p1,p2}, an improved cost function is used as the optimization method to perform monocular high-precision intrinsic parameter matrix calibration; the cost function L is expressed as: Among them, X ij λ represents the actual observed pixel coordinates of the j-th corner point in the i-th calibration image; N is the total number of calibration images; M is the total number of observable corner points in the i-th calibration image; λ is the regularization coefficient. Simultaneously, the ability to model long-range field distortion is enhanced by using a sixth-order radial distortion term calibrated at a distance, expressed as: x dist =x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 ) Where, x dist is the sixth-order radial distortion term; x and y are the x and y coordinates of the ideal image point; k1, k2, and k3 are all radial distortion coefficients; p1 and p2 are both tangential distortion coefficients; r is the radial distance; 2) Calibrate the extrinsic parameter matrix of the stereo camera: In the traditional optimization of the extrinsic parameter matrix, an epipolar alignment loss is added; the epipolar alignment loss L epi Represented as: Where E is the essential matrix, which forces the matching points of the stereo image to satisfy the epipolar constraint; x l x is the coordinate point matched by the left image in the i-th calibration image; r Let be the coordinates of the right image matched in the i-th calibration image; Define the joint optimization objective function and calibrate the extrinsic parameter matrix; the joint optimization objective function L joint Represented as: Where, π l (X i ) is the projection function of the left camera in a binocular camera setup; π r (X i x is the projection function of the right camera in a stereo camera setup; li For x l The actual observed coordinates; x ri For x r The actual observed coordinates; γ is the geometric constraint weight; t is the translation component of the binocular camera; R is the rotation component of the binocular camera; 3) Repeat the above calibration of the intrinsic and extrinsic parameter matrices, and take the average of the intrinsic and extrinsic parameter calibration results as the final calibration result of the binocular camera.

5. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 2, characterized in that, In step S3, the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients obtained from the binocular camera calibration are used to perform stereo correction on each group of binocular images; specifically, this includes: (1) Decompose the rotation components in the extrinsic matrix into left rotation components and right rotation components; (2) Using scene depth perception factors, construct the energy function E when decomposing the rotation component R. res : Wherein, S(r l ·r r -I) is the depth sensing factor function; λ is the regularization coefficient; L is the cost function; I is the identity matrix; R is the rotation component; r l For the left rotation component; r r This is the right rotation component; (3) By dynamically optimizing the decomposition process, the left rotation component r l and right rotation component r r While maintaining orthogonal constraints, prioritize aligning the stereo correspondence points in high-confidence regions; (4) Calculate a new rotation matrix that makes the principal optical axes of the left and right images in the binocular image parallel to the baseline of the binocular camera; rotate the left and right images using the new rotation matrix. (5) Based on the intrinsic parameter matrix, and combined with the rotated left and right images, calculate the new left projection matrix and the new right projection matrix; (6) Based on the new rotation matrix, the new left projection matrix and the new right projection matrix, distortion correction is performed through distortion coefficients to obtain the corrected binocular image.

6. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 1, characterized in that, In step S3, based on the trained Yolov5 network model, the common ROI region of each group of stereo images is obtained; specifically including: (1) Input the left and right images from each set of binocular images into the trained Yolov5 network model, and output the bounding box coordinates of the localized object in the left and right images respectively; expressed as: Where, x l and y l x represents the bounding box coordinates of the located object in the left image; r and y r x represents the bounding box coordinates of the located object in the right image; center and y center The coordinates of the center point of the bounding box; width is the width of the bounding box; height is the height of the bounding box; (2) Obtain the bounding box coordinates of the ROI regions corresponding to the left and right images respectively, and obtain the common ROI region of the stereo images based on this; wherein, the bounding box coordinates of the common ROI region are: Where (x'1,y'1) and (x'2,y'2) are the bounding box coordinates of the common ROI region; (x ll ,y ll (x) represents the coordinates of the top-left bounding box of the left image; lr ,y lr (x) represents the coordinates of the bottom right bounding box of the left image; rl ,y rl (x) represents the coordinates of the top-left bounding box of the right image; rr ,y rr ) represents the coordinates of the bottom right bounding box of the right image.

7. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 6, characterized in that, The training process of the Yolov5 network model includes: (1) For stereo-corrected binocular images, translation, rotation and mirroring operations are performed through optimized affine transformation to achieve data augmentation of the binocular image dataset; The optimization process of affine transformation includes: Establish an SE(3)-equivariant convolution kernel and design a convolution operator that satisfies the invariance of rigid body motion; process the affine transformation based on the convolution operator; The convolution operator is represented as follows: K(g·v)=ρ(g)·K(v)·ρ(g) -1 v=(x,y,z,θ x ,θ y ,θ z ) Where K(g·v) is the convolution operator; ρ(g) is the representation matrix of the rigid body transformation g; K(v) is the convolution operator applied to the input data v without rigid body transformation; x, y, and z are the position information of the input image; θ x ,θ y and θ z The input image contains attitude angle information; R p t represents the rotation component of an image during image data processing. p Represents the translation component of an image during image processing; The parameterization method of convolution operator expansion using spherical harmonic function basis is adopted: in, This represents the convolution kernel defined in spherical coordinates; Q is the maximum spherical harmonic order; l represents the current spherical harmonic order; m represents the quantum number of the in-order orientation angle; r represents the radial distance; θ represents the horizontal direction angle in a spherical coordinate system. s w represents the vertical direction angle in spherical coordinates. lm The weighting coefficients represent the basis of the spherical harmonic function; It is a basis for spherical harmonic functions; (2) Based on the enhanced stereo image dataset, the parameters of the YOLOv5 network model are optimized and trained.

8. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 1, characterized in that, Step S4 specifically includes: (1) Use the CREStereo deep learning stereo matching algorithm to calculate the disparity map of the left image in each group of binocular images; (2) Noise information in the disparity map is removed by optimized multi-scale bilateral filtering; The optimization method for multi-scale bilateral filtering is as follows: A dynamic kernel parameter adjustment mechanism is introduced based on traditional bilateral filtering to establish a spatial color gamut joint control model based on local texture complexity; the multi-scale bilateral filtering is optimized using this spatial color gamut joint control model; the spatial color gamut joint control model is expressed as follows: in, Let be the gradient magnitude of pixel q; H(q) represents the local information entropy centered at q; α, β, and γ are all learnable parameters; σ s σ represents the standard deviation of the dynamic spatial domain. s0 σ is the standard deviation of the spatial domain reference. r The standard deviation of the dynamic color gamut; σ r0 E|H| is the baseline standard deviation of the color gamut; E|H| is the global average value of the information entropy of the entire image. (3) The disparity map is smoothed by energy functional optimization algorithm; The energy functional optimization algorithm involves introducing a curvature constraint term into cubic spline interpolation; the formula for the energy functional optimization algorithm is as follows: Where E(d) is the total energy; λ E μ is the weighting coefficient of the smoothing term. E represents the weighting coefficient of the curvature constraint term; K represents the curvature of the image edge. is the second derivative of the parallax field; It is the first derivative of the parallax field.

9. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 1, characterized in that, Step S5 specifically includes: (1) Use the Shi-Tomasi corner detection algorithm based on autocorrelation matrix to identify corners in the left image: use the Shi-Tomasi corner detection algorithm to slide a fixed window in any direction on the left image and calculate the change in gray level within the fixed window; calculate the autocorrelation matrix of the region around each pixel to evaluate the degree of gray level change; if the gray level changes significantly when the window slides, it is considered that the window contains corners.

10. A three-dimensional spatial positioning method based on hierarchical joint calibration according to claim 1, characterized in that, Step S6 specifically includes: (1) Convert the disparity map into a depth map; (2) Use adaptive mean filtering to remove outlier depth information from the depth map; (3) The final coordinates in the camera coordinate system are calculated based on the transformation relationship between the pixel coordinate system and the world coordinate system.

Images