A high-precision camera positioning and depth ranging method and system

Abstract

The application provides a high-precision camera positioning and depth ranging method and system. The method effectively solves the problem that traditional monocular cameras cannot directly obtain depth information. By combining deep learning technology with the PnP algorithm in traditional positioning algorithms, high-precision depth estimation and camera positioning are provided. Through model training by fusing multiple data sets, the robustness and generalization ability of the model are significantly improved, enabling stable work in various different scenes and environments. Compared with traditional multi-camera systems or laser radar systems, the application has significant advantages of low cost and high precision, and can provide accurate depth estimation and positioning results without increasing expensive hardware devices.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation and computer vision technology, and to a high-precision camera positioning and depth ranging method and system based on a monocular camera and a deep learning model. Background Technology

[0002] In intelligent transportation systems, accurate spatial perception and vehicle localization are fundamental to achieving key functions such as autonomous driving, intelligent inspection, and traffic monitoring. Traditional three-dimensional spatial perception methods typically rely on devices such as binocular cameras, LiDAR, or structured light to achieve high-precision depth measurements. However, these devices suffer from high costs, complex installations, and susceptibility to environmental interference, making them difficult to deploy widely in large-scale transportation systems.

[0003] In contrast, monocular cameras, due to their simple structure, low cost, and wide applicability, have become a promising solution for achieving 3D perception in intelligent transportation scenarios. However, since monocular images lack direct depth information, it is difficult to reconstruct the true 3D spatial structure from a single frame. Therefore, how to recover accurate depth information and camera pose from monocular images remains a current technical bottleneck.

[0004] In recent years, significant progress has been made in monocular depth estimation models based on deep learning. These models can be trained on large-scale data to automatically learn the complex mapping relationship between images and depth, thereby achieving pixel-level depth estimation. However, depth maps that rely solely on the output of neural networks often suffer from problems such as inconsistent scale and large error fluctuations, making them difficult to use directly for accurate measurement or camera localization in real physical space.

[0005] On the other hand, the traditional PnP algorithm can estimate the camera pose by using camera intrinsic parameters and the correspondence between spatial points and image points. However, it relies heavily on accurate three-dimensional coordinates, and faces the problem of inaccurate positioning due to the lack of depth information, especially when only two-dimensional image input is available.

[0006] Therefore, how to integrate the strong representational capabilities of deep learning models with the geometric accuracy of the PnP algorithm to improve the ranging and localization capabilities of monocular systems has become a crucial technical problem that urgently needs to be solved in intelligent transportation systems. Therefore, this application proposes a high-precision camera localization and depth ranging method and system to at least partially address the problems that may exist in the prior art. Summary of the Invention

[0007] To overcome or at least partially solve the above problems, embodiments of the present invention provide a method and system for achieving high-precision camera positioning and depth ranging of a mobile robot in an indoor scene.

[0008] To address the problems of missing depth information, insufficient positioning accuracy, and reliance on expensive equipment in existing technologies, this invention provides a high-precision depth estimation and localization method based on a monocular camera and a deep learning model, aiming to achieve:

[0009] It can obtain depth information at the real physical scale using only a monocular camera, breaking through the technical bottleneck of "monocular depthlessness";

[0010] By integrating deep learning with the traditional PnP algorithm, the accuracy and robustness of camera pose estimation are improved.

[0011] Under low-cost hardware conditions (requiring only a monocular camera), it meets the high-precision positioning and ranging requirements of scenarios such as intelligent transportation and autonomous driving.

[0012] The embodiments of the present invention are implemented as follows:

[0013] This application provides a method for high-precision camera positioning and depth ranging of a mobile robot, the method comprising:

[0014] The traditional Zhang Zhengyou calibration method is used to calibrate the intrinsic and extrinsic parameters of the monocular camera to obtain the camera intrinsic and extrinsic parameter matrices; the PnP algorithm is combined to estimate the camera pose and obtain the initial camera pose parameters.

[0015] A relative depth map is generated by extracting features from images input from a monocular camera using a deep learning model trained on a multi-source dataset.

[0016] Based on the relative depth map, the scale factor is calculated using reference points with known real distances, and the relative depth map is converted into a real-scale depth map. Combining the real-scale depth map with the camera intrinsic and extrinsic parameter matrices, the image pixels are back-projected into three-dimensional space to obtain the three-dimensional coordinates of the scene.

[0017] By combining the depth information output by the deep learning model with the PnP algorithm, the camera intrinsic and extrinsic parameter matrices and pose are optimized; the 3D coordinates of each pixel are refined through pixel-level depth map optimization.

[0018] In some embodiments of the present invention, the traditional Zhang Zhengyou calibration method is used to calibrate the intrinsic and extrinsic parameters of the monocular camera to obtain the camera intrinsic and extrinsic parameter matrices; the PnP algorithm is combined to perform pose estimation of the camera to obtain initial camera pose parameters, including:

[0019] Capture planar checkerboard images from different angles, calculate the homography matrix between the image plane and the checkerboard world coordinate plane; calculate the camera's intrinsic and extrinsic parameter matrices:

[0020] The intrinsic parameter matrix is ​​as follows:

[0021] ,

[0022] In the formula, , Let x be the focal length of the camera in the x and y directions. , Principal point coordinates;

[0023] The extrinsic parameter matrix consists of a rotation matrix R and a translation vector T, and its formula is:

[0024] , world coordinate system ( ) mapped to the pixel points in the image coordinate system );

[0025] The PnP algorithm combines feature points in the image with their corresponding world coordinates, and uses perspective geometry principles and camera intrinsic parameters to calculate the camera's rotation matrix R and translation vector T, thereby performing camera pose estimation and obtaining the initial camera pose parameters.

[0026] In some embodiments of the present invention, the step of extracting features from the image input by a monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map includes:

[0027] The relative depth map is generated by fusing and training images and depth data from different indoor and outdoor environments, including at least the KITTI dataset, NYUv2 dataset, and MegaDepth dataset.

[0028] In some embodiments of the present invention, the feature extraction of the image input from the monocular camera using a deep learning model trained on a multi-source dataset generates a relative depth map; wherein the structure of the deep learning model includes:

[0029] The encoder uses either Vision Transformer or Swing Transformer. Vision Transformer divides the image into fixed-size image blocks, embeds them into a high-dimensional space through linear transformation, and then extracts global features through a self-attention layer and a feedforward neural network. Swing Transformer uses a windowed self-attention mechanism to fuse local and global information of the image.

[0030] The decoder contains multiple feature fusion blocks, each of which consists of two residual convolutional units. Each residual convolutional unit includes two convolutional operations, batch normalization, and a ReLU activation function.

[0031] The output layer restores the feature map to the same resolution as the input image through multi-level convolution and upsampling operations, and outputs a single-channel depth map. The operations of the output layer include: Conv2d(256,128,kernel_size=(3,3)), upsampling, Conv2d(128,32,kernel_size=(3,3)), and Conv2d(32,1,kernel_size=(1,1)).

[0032] The multi-task learning and loss function layer includes joint optimization of depth estimation and pose optimization tasks, and a joint loss function based on depth prediction error and pose optimization error.

[0033] In some embodiments of the present invention, the step of extracting features from the image input by a monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map includes:

[0034] The deep learning model is trained using a multi-task loss function, specifically, using a joint loss function of depth prediction error and pose optimization error; wherein the depth prediction error employs a scale- and offset-invariant loss function. :

[0035] ,in, and For the predicted depth value and the true depth value after scale and offset adjustment, M is the number of pixels and ρ is the type of loss function;

[0036] Scale and offset adjustments are achieved through estimation functions s and t, where,

[0037] s , ;

[0038] The predicted values ​​and the ground truth values ​​are aligned using the least squares criterion, where...

[0039] , , ;

[0040] The final loss function is:

[0041] ,in, The number of training samples. For regularization loss, .

[0042] In some embodiments of the present invention, the step of converting the relative depth map into a true-scale depth map by calculating a scale factor using a reference point with a known true distance based on the relative depth map includes:

[0043] Select pixels in the image whose true distance is known, and calculate the scale factor using the scale formula, where, The formula is:

[0044] ;

[0045] Multiply the relative depth value of each pixel in the relative depth map by the scale factor to obtain the true scale depth value;

[0046] .

[0047] In some embodiments of the present invention, the image pixels are back-projected into three-dimensional space by combining the real-scale depth map with the camera intrinsic and extrinsic parameter matrices to obtain the three-dimensional coordinates of the scene.

[0048] The formula for back-projecting the pixel into three-dimensional space is as follows:

[0049] In the formula, (u,v) are the image pixel coordinates, and d is the true scale depth value of that pixel. Let (X,Y,Z) be the inverse of the camera intrinsic parameter matrix, and let (X,Y,Z) be the three-dimensional coordinates in the camera coordinate system.

[0050] In some embodiments of the present invention, the optimization of the camera intrinsic and extrinsic parameter matrices and pose by combining the depth information output by the deep learning model with the PnP algorithm includes:

[0051] The 3D coordinates obtained by back projection and the corresponding image pixels are used as 3D-2D matching pairs, and the camera extrinsic parameters are re-estimated using the PnP algorithm.

[0052] By combining nonlinear least squares optimization methods, the camera intrinsic and extrinsic parameters, including focal length and principal point coordinates, are refined to reduce reprojection error.

[0053] Multi-scale feature fusion technology is used to locally weight and correct the depth map, ensuring the accuracy of depth estimation within different distance ranges.

[0054] In some embodiments of the present invention, a high-precision camera positioning and depth ranging system is provided, the system comprising:

[0055] The parameter acquisition module is used to calibrate the intrinsic and extrinsic parameters of the monocular camera using the traditional Zhang Zhengyou calibration method, and to obtain the camera intrinsic and extrinsic parameter matrices; combined with the PnP algorithm, the camera pose is estimated to obtain the initial camera pose parameters.

[0056] The depth prediction module is used to extract features from images input from a monocular camera using a deep learning model trained on a multi-source dataset, and generate a relative depth map.

[0057] The image calibration module is used to calculate the scale factor based on the relative depth map using reference points with known real distances, and convert the relative depth map into a real-scale depth map; combining the real-scale depth map with the camera intrinsic and extrinsic parameter matrices, the image pixels are back-projected into three-dimensional space to obtain the three-dimensional coordinates of the scene;

[0058] The projection and reconstruction module combines the depth information output by the deep learning model with the PnP algorithm to optimize the camera's intrinsic and extrinsic parameters and pose; it refines the 3D coordinates of each pixel through pixel-level depth map optimization.

[0059] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the aforementioned high-precision camera positioning and depth ranging method.

[0060] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described high-precision camera positioning and depth ranging method.

[0061] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0062] By combining a deep learning model with the PnP algorithm of traditional localization algorithms, this innovation overcomes the limitations of traditional depth estimation methods. Through optimizing the deep learning model and camera localization algorithm, this invention can control the depth estimation error to within 0.1 meters within a 20-meter range, effectively improving the accuracy of depth estimation and the reliability of localization. Furthermore, by fusing multi-source datasets for model training, this invention significantly enhances the model's robustness and generalization ability, enabling it to operate stably in various scenarios and environments. Compared to traditional multi-camera systems or LiDAR systems, this invention offers significant advantages in terms of low cost and high precision, providing accurate depth estimation and localization results without the need for expensive hardware. The technical solution of this invention can be widely applied in fields such as intelligent transportation, autonomous driving, and robot localization, possessing strong market competitiveness and practical application value, providing an economical and efficient depth estimation and localization solution for various applications. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating the steps of a high-precision camera positioning and depth ranging method according to an embodiment of the present invention.

[0065] Figure 2 A flowchart of a high-precision camera localization and depth ranging based on a monocular camera and a deep learning model is provided in an embodiment of the present invention.

[0066] Figure 3 This is a road depth pixel map obtained by a deep learning model according to an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0069] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0070] Please refer to Figure 1 This invention provides a flowchart of steps for implementing a high-precision camera positioning and depth ranging method for a mobile robot, the method comprising:

[0071] Step S1: Use the traditional Zhang Zhengyou calibration method to calibrate the intrinsic and extrinsic parameters of the monocular camera and obtain the camera intrinsic and extrinsic parameter matrices; combine the PnP algorithm to estimate the camera pose and obtain the initial camera pose parameters.

[0072] Step S2: Extract features from the image input by the monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map, thereby enabling relative depth prediction for each pixel in the image;

[0073] Step S3: Based on the relative depth map, calculate the scale factor using reference points with known real distances, and convert the relative depth map into a real-scale depth map; combine the real-scale depth map with the camera intrinsic and extrinsic parameter matrices, and back-project the image pixels into three-dimensional space to obtain the three-dimensional coordinates of the scene;

[0074] Step S4: Combine the depth information output by the deep learning model with the PnP algorithm to optimize the camera intrinsic and extrinsic matrices and pose; refine the 3D coordinates of each pixel through pixel-level depth map optimization to improve depth estimation accuracy and camera positioning accuracy.

[0075] This invention relates to monocular camera depth estimation and localization technology, and in particular to a depth ranging and camera localization method based on a combination of a monocular camera and a deep learning model. This method effectively solves the problem that traditional monocular cameras cannot directly acquire depth information. By combining deep learning technology with the PnP algorithm in traditional localization algorithms, it provides high-precision depth estimation and camera localization.

[0076] This innovation combines a deep learning model with the PnP algorithm, a traditional localization algorithm, overcoming the limitations of conventional depth estimation methods. By optimizing the deep learning model and camera localization algorithm, this invention can control the depth estimation error to within 0.1 meters within a 20-meter range, effectively improving the accuracy of depth estimation and the reliability of localization. Furthermore, by fusing multi-source datasets for model training, this invention significantly enhances the model's robustness and generalization ability, enabling it to operate stably in various scenarios and environments. Compared to traditional multi-camera systems or LiDAR systems, this invention offers significant advantages in terms of low cost and high precision, providing accurate depth estimation and localization results without the need for expensive additional hardware.

[0077] Furthermore, the traditional Zhang Zhengyou calibration method is used to calibrate the intrinsic and extrinsic parameters of the monocular camera, obtaining the camera's intrinsic and extrinsic parameter matrices; combined with the PnP algorithm, the camera pose is estimated to obtain the initial camera pose parameters, including:

[0078] Capture planar checkerboard images from different angles, calculate the homography matrix between the image plane and the checkerboard world coordinate plane; calculate the camera's intrinsic and extrinsic parameter matrices:

[0079] The intrinsic parameter matrix is ​​as follows:

[0080] ,

[0081] In the formula, , Let x be the focal length of the camera in the x and y directions. , Principal point coordinates;

[0082] The extrinsic parameter matrix consists of a rotation matrix R and a translation vector T, and its formula is:

[0083] , world coordinate system ( ) mapped to the pixel points in the image coordinate system );

[0084] The PnP algorithm combines feature points in the image with their corresponding world coordinates, and uses perspective geometry principles and camera intrinsic parameters to calculate the camera's rotation matrix R and translation vector T, thereby performing camera pose estimation and obtaining the initial camera pose parameters.

[0085] As an example, pose estimation for a monocular camera is performed using the traditional Zhang Zhengyou calibration method and the PnP algorithm to obtain the camera's intrinsic and extrinsic parameters. A deep learning model is then used to extract features from the input image, generating depth map data, which is applied to the depth estimation task to predict the depth of each pixel. Combining the deep learning model with traditional localization algorithms (such as PnP), depth estimation and position inference are performed to optimize the camera pose and the 3D structure of the scene. Based on the inferred depth results, the camera's intrinsic and extrinsic parameters and position are further optimized, and pixel-level depth map optimization further refines the coordinates of each pixel in the image, improving the accuracy of depth estimation.

[0086] In a specific example, the above-mentioned traditional Zhang Zhengyou calibration method is used to calibrate the intrinsic and extrinsic parameters of a monocular camera, obtaining the camera's intrinsic and extrinsic parameter matrices; the PnP algorithm is then used to estimate the camera's pose, yielding initial camera pose parameters; wherein, the above-mentioned Zhang Zhengyou calibration operation includes:

[0087] Prepare a checkerboard calibration board (e.g., 10×7 grids, each grid side length 30mm), place it perpendicular to the camera optical axis, and take images of the calibration board from different angles (at least 8).

[0088] The OpenCV function `cv2.findChessboardCorners()` is used to extract the corner points of the chessboard in the image, and the intrinsic parameter matrix K and distortion coefficients (such as radial distortion) are calculated using `cv2.calibrateCamera()`. , Tangential distortion , );

[0089] Example of output intrinsic parameter matrix: (Focal length 1200 pixels, principal point (640, 360)).

[0090] Initial pose estimation is performed using the PnP algorithm, including:

[0091] Select at least 5 corner points of the chessboard grid in the calibration image and determine their world coordinates (e.g., take the lower left corner of the calibration board as the origin, Z=0).

[0092] Use the cv2.solvePnP() function to take world coordinates, image corner coordinates and intrinsic parameters as input, and output rotation vector (converted to rotation matrix R) and translation vector T.

[0093] Verify using the following formula: Calculate the reprojection error and ensure that the error is ≤1 pixel.

[0094] In some embodiments of this application, the step of extracting features from an image input from a monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map includes:

[0095] The relative depth map is generated by fusing and training images and depth data from different indoor and outdoor environments, including at least the KITTI dataset, NYUv2 dataset, and MegaDepth dataset.

[0096] Multi-source dataset fusion training is employed to enhance the model's robustness and generalization ability. Specific datasets include, but are not limited to, the following: KITTI dataset, NYUv2 dataset, MegaDepth dataset, etc. These datasets encompass images and depth data from various indoor and outdoor environments, exhibiting high diversity and broad applicability. This effectively addresses performance differences in different scenarios, improving the accuracy and stability of depth estimation.

[0097] Furthermore, the method involves using a deep learning model trained on a multi-source dataset to extract features from the image input from a monocular camera, generating a relative depth map; wherein the structure of the deep learning model includes:

[0098] The encoder employs either Vision Transformer (ViT) or Swing Transformer. ViT divides the image into fixed-size image blocks, embeds them into a high-dimensional space through linear transformation, and then extracts global features through a self-attention layer and a feedforward neural network. Swing Transformer uses a windowed self-attention mechanism to fuse local and global image information.

[0099] The decoder contains multiple feature fusion blocks, each of which consists of two residual convolutional units. Each residual convolutional unit includes two convolutional operations, batch normalization, and a ReLU activation function.

[0100] The output layer restores the feature map to the same resolution as the input image through multi-level convolution and upsampling operations, and outputs a single-channel depth map. The operations of the output layer include: Conv2d(256,128,kernel_size=(3,3)), upsampling, Conv2d(128,32,kernel_size=(3,3)), and Conv2d(32,1,kernel_size=(1,1)).

[0101] The multi-task learning and loss function layer includes joint optimization of depth estimation and pose optimization tasks, and a joint loss function based on depth prediction error and pose optimization error.

[0102] The step of extracting features from images input by a monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map includes:

[0103] The deep learning model is trained using a multi-task loss function, specifically, using a joint loss function of depth prediction error and pose optimization error; wherein the depth prediction error employs a scale- and offset-invariant loss function. :

[0104] ,in, and For the predicted depth value and the true depth value after scale and offset adjustment, M is the number of pixels and ρ is the type of loss function;

[0105] Scale and offset adjustments are achieved through estimation functions s and t, where,

[0106] s , ;

[0107] The predicted values ​​and the ground truth values ​​are aligned using the least squares criterion, where...

[0108] , , ;

[0109] The final loss function is:

[0110] ,in, The number of training samples. For regularization loss, These are weight parameters.

[0111] In a specific example, relative depth map generation includes deep learning model training and depth prediction. Deep learning model training includes:

[0112] (1) Dataset preprocessing: Image enhancement (brightness adjustment, random cropping) was performed on the KITTI (outdoor road), NYUv2 (indoor scene), and MegaDepth (general scene) datasets, with a unified resolution of 640×480;

[0113] (2) Model construction: Encoder: Swin Transformer (window size 7×7, 4 layers of self-attention) is used to output feature maps at 4 scales; Decoder: 3 feature fusion blocks (each containing 2 residual convolutional units) to gradually fuse high / low scale features; Output layer: single-channel depth map is output through Conv2d(256,128)→upsampling→Conv2d(128,32)→Conv2d(32,1);

[0114] (3) Loss function: using the above , where α=0.5, and ρ is the Huber loss (balancing outlier effects);

[0115] (4) Training parameters: Adam optimizer, learning rate 1e-4, training for 50 rounds, and stopping after the validation set accuracy reaches the target.

[0116] Depth prediction includes:

[0117] Input a monocular RGB image (640×480), which is preprocessed (normalized to [0,1]) and then input into the model; the model outputs a relative depth map (pixel value range [0,1], the larger the value, the closer the distance).

[0118] In some embodiments of this application, the step of converting the relative depth map into a true-scale depth map by calculating the scale factor using reference points with known true distances based on the relative depth map includes:

[0119] Select pixels in the image whose true distance is known, and calculate the scale factor using the scale formula, where, The formula is:

[0120] ;

[0121] Multiply the relative depth value of each pixel in the relative depth map by the scale factor to obtain the true scale depth value;

[0122] .

[0123] Furthermore, by combining the real-scale depth map with the camera's intrinsic and extrinsic parameter matrices, the image pixels are back-projected into three-dimensional space to obtain the three-dimensional coordinates of the scene.

[0124] The formula for back-projecting the pixel into three-dimensional space is as follows:

[0125] In the formula, (u,v) are the image pixel coordinates, and d is the true scale depth value of that pixel. Let (X,Y,Z) be the inverse of the camera intrinsic parameter matrix, and let (X,Y,Z) be the three-dimensional coordinates in the camera coordinate system.

[0126] In a specific example, the above-mentioned true-scale depth map and 3D reconstruction include:

[0127] (1) Scale calibration: Select a reference point in the image (such as a ground point known to be 5 meters away from the camera) and read the relative depth value predicted by the model (such as 0.2).

[0128] (2) Calculate the scale factor scale=5 / 0.2=25, multiply the entire relative depth map by 25 to obtain the true scale depth map (unit: meters).

[0129] 3D backprojection: For each pixel (u,v) in the image, read its true depth d;

[0130] Through the formula: Calculate three-dimensional coordinates, for example:

[0131] If u=640, v=360, d=5m ,

[0132] but (Center point of the camera's optical axis).

[0133] In some embodiments of this application, the camera intrinsic and extrinsic matrices and pose are optimized by combining the depth information output by the deep learning model with the PnP algorithm, including: using the three-dimensional coordinates obtained by back projection and the corresponding image pixels as 3D-2D matching pairs, and re-estimating the camera extrinsic parameters (rotation matrix R and translation vector T) through the PnP algorithm.

[0134] By combining nonlinear least squares optimization methods, the camera intrinsic and extrinsic parameters, including focal length and principal point coordinates, are refined to reduce reprojection error.

[0135] Multi-scale feature fusion technology is used to locally weight and correct the depth map, ensuring the accuracy of depth estimation within different distance ranges.

[0136] In a specific example, parameter optimization and accuracy improvement include:

[0137] (1) Camera parameter optimization: Select 1000 back-projected 3D points and their corresponding image pixels as 3D-2D matching pairs;

[0138] Re-optimize R and T using cv2.solvePnPRefineLM() (Levenberg-Marquardt optimization) to reduce the reprojection error from 1 pixel to within 0.3 pixels;

[0139] (2) Intrinsic parameter optimization: by minimizing Fine-tuning the focal length and principal point improves the accuracy of intrinsic parameters. For predicting pixels.

[0140] (3) Depth map refinement: Multi-scale optimization of the depth map: 1×1 convolution is used for local correction within 10 meters, and 3×3 convolution is used for global smoothing from 10 to 20 meters; (4) Output the final depth map: The error within 20 meters is ≤0.1 meters, which meets the accuracy requirements of vehicle distance measurement in autonomous driving.

[0141] The above scheme achieves high-precision depth estimation and positioning using a monocular camera, which can be directly integrated into intelligent transportation systems or autonomous driving platforms to provide reliable three-dimensional spatial information for environmental perception.

[0142] As an example, traditional calibration methods cannot directly perceive depth information, leading to errors in depth coordinates. To address this issue, this invention combines a depth map generated by a deep learning model with the camera's pose estimation results to correct the depth coordinates obtained by traditional calibration methods. Specifically, the model compares the initial depth coordinates with the depth values ​​predicted by the deep learning model through a depth correction module. It then uses pose optimization algorithms and depth correction strategies to further correct the pixels in the depth map, ensuring that the final depth estimation result is more accurate. The corrected depth coordinates effectively reflect the 3D structure of the actual scene.

[0143] Based on the corrected depth coordinates, the camera's intrinsic and extrinsic parameters are further updated. Specifically, the traditional camera intrinsic and extrinsic parameters are optimized and updated by combining the depth map generated by the deep learning model and the pose estimation results. Using the corrected depth coordinates and the OpenCV functions `cv2.solvePnP, -np.matmul`, the camera's intrinsic parameters (such as focal length, principal point, etc.) and extrinsic parameters (such as rotation matrix and translation vector) are re-estimated, thereby improving the accuracy and stability of camera localization. This step ensures the consistency between the depth estimation and the camera localization results, ultimately improving the accuracy of the entire system.

[0144] The accuracy of the depth map is further optimized by using corrected depth coordinates and updated camera intrinsics and extrinsic parameters. Specifically, the coordinates of each pixel in the image are reprojected based on the updated camera intrinsics and extrinsic parameters. Combined with the depth prediction results from the deep learning model, global depth optimization is performed to reduce errors in the depth map. This optimization process includes weighted correction of local regions in the depth map and the use of multi-scale feature fusion technology to ensure that the final depth map maintains high accuracy across different distance ranges. Ultimately, the corrected depth map accurately reflects the 3D structure of the image, providing more precise depth estimation and camera localization results.

[0145] As an example, refer to Figure 1 and Figure 2 As shown, addressing the problem that existing technologies cannot obtain true-scale depth information and accurate pose from a single frame image using a monocular camera, this invention, "A Depth Estimation and Localization Method Based on a Monocular Camera and Deep Learning Model," provides a high-precision 3D reconstruction and localization technology solution. This method utilizes a deep learning model to estimate the scene's depth map from a monocular image and combines it with the Perspective-n-Point (PnP) algorithm to solve for the camera pose, achieving low-cost, high-precision 3D reconstruction and localization with only a monocular camera. By integrating data-driven depth estimation with traditional geometric algorithms, this invention can obtain distance information for each pixel, as well as the camera's spatial position and orientation, thereby supporting applications such as robot navigation and map building.

[0146] To achieve the above objectives, the present invention adopts the following technical solution:

[0147] Camera Calibration: The monocular camera was calibrated using Zhang Zhengyou's checkerboard calibration method. The checkerboard was placed vertically, perpendicular to the camera, and the coordinate origin, X, Y, Z axes, length, width, and depth were calibrated. The checkerboard data, including its dimensions and unit length, was determined. OpenCV functions were used to obtain the camera's intrinsic parameter matrix (focal length, principal point coordinates, etc.) and distortion coefficients. Zhang Zhengyou's calibration method involves capturing planar checkerboard images from different angles, calculating the homography matrix between the image plane and the checkerboard world coordinate plane, and thus solving for the camera's intrinsic and extrinsic parameters. Accurate camera intrinsic parameters provide the foundation for subsequent depth estimation projection into 3D space.

[0148] Depth Image Estimation: Monocular RGB images captured by a camera are input into a deep learning model. The captured images undergo preprocessing, including image enhancement, inversion, and cropping to adjust resolution. Then, they are processed by an encoder-decoder structure using a convolutional neural network and a visual Transformer to extract multi-scale features, outputting a single-channel depth map of the same size as the input image. Each pixel value in this depth map reflects the relative depth of the corresponding pixel. The model can be implemented using a ResNet residual network as the encoder to extract semantic features, and then gradually fuse high- and low-level features through upsampling or a Transformer decoder to generate the depth estimation result. The network structure can include components such as residual convolutional units (RCUs) and feature fusion modules to fuse feature maps of different scales, preserving global semantic information while refining local depth details, ultimately outputting a single-channel depth map of the scene.

[0149] True Depth Acquisition: Since the depth model acquires the relative depth of the image, it assigns a depth weight to each pixel, ranging from [0,1]. Pixels closer to each other have a weight closer to 1, while pixels farther away have a weight closer to 0. During calibration, the actual true depth of a known pixel in the calibration image is calculated using a scale factor. The depth output by the model does not have a real physical unit. To obtain a true distance (e.g., a depth map in meters), a reference point with a known true distance must be used to scale the entire depth map. The entire relative depth map is then linearly scaled to obtain the true depth map. , depth=1.0 / (inv_depth+1e-6).

[0150] Depth Coordinate Completion and Pixel Backprojection: Traditional camera calibration methods (such as Zhang Zhengyou calibration) can only obtain the camera's intrinsic and extrinsic parameters, thus mapping pixels in the image to two-dimensional image coordinates and normalized direction vectors. However, they cannot obtain the specific depth information of each pixel, and therefore cannot accurately reconstruct the three-dimensional spatial coordinates. This invention, based on traditional calibration, further incorporates the ground truth depth values ​​obtained in the previous step into a depth map predicted by a deep learning model. The depth value corresponding to each pixel is used as a scale factor to guide its backprojection from the two-dimensional image coordinates to a three-dimensional point in the camera coordinate system. This method effectively completes the depth coordinates that traditional calibration cannot obtain, thereby achieving complete three-dimensional reconstruction of all effective pixels in the entire image, providing richer and more accurate input information for subsequent pose estimation and spatial understanding.

[0151] Camera Parameter Updates and Accuracy Improvements: After introducing depth coordinate completion, this invention further updates and optimizes the camera's intrinsic and extrinsic parameters. Traditional calibration methods, lacking a sense of true depth, result in camera poses with scale ambiguity and 3D coordinate deviations. By introducing true or relatively accurate depth information provided by a deep learning model, each pixel is mapped to a 3D coordinate point with depth.

[0152] Subsequently, by combining these real 3D coordinates with their corresponding image pixel positions, the camera's extrinsic parameters are refined by re-estimating the camera's rotation matrix and translation vector (R and T). Simultaneously, intrinsic parameters such as focal length and principal point are further optimized using error minimization methods. This step significantly improves the reprojection accuracy of the 3D point cloud, thereby enhancing the overall accuracy and robustness of the entire system in depth estimation and localization tasks.

[0153] The following is a detailed implementation process of the present invention, describing each step in detail:

[0154] Traditional camera calibration and initial pose estimation:

[0155] Multiple chessboard images were captured using the Zhang Zhengyou calibration method, and the camera intrinsic parameter matrix K (focal length, principal point coordinates) and distortion coefficients were obtained using the cv2.calibrateCamera() function;

[0156] By matching feature points in the calibrated image, the camera extrinsic parameters rotation matrix R and translation vector T are estimated using the PnP algorithm (cv2.solvePnP).

[0157] The complete pixel projection model is: x=K[R|T]X, where x is the image coordinate, X is the coordinate of a point in three-dimensional space, K is the intrinsic parameter, and R and T are the extrinsic parameters.

[0158] Monocular Depth Map Estimation and Scale Calibration:

[0159] A single frame of RGB image is input into a trained depth estimation neural network, which outputs a relative depth map.

[0160] The model structure uses a ResNet-Transformer encoder-decoder architecture, which supports high-resolution detail recovery;

[0161] For a depth map, select one or more pixels in the image with known real-world distances (such as ground reference points), and calculate the scale factor to convert the relative depth output by the model to the true physical scale:

[0162] scale=known_distance / predicted_relative_depth

[0163] Multiply the entire depth map by the scale to obtain a depth map at the true scale.

[0164] Using the depth map and camera intrinsic parameter K, the pixel (u,v) and its corresponding depth d are back-projected onto the camera coordinate system to obtain the 3D coordinates:

[0165] This yields a complete spatial point cloud, which is used for subsequent positioning and updating of intrinsic and extrinsic parameters.

[0166] Coordinate correction and extrinsic parameter update:

[0167] The depth coordinates obtained in the above steps and the original image pixel pairs are used as 3D-2D matching point pairs, and the PnP algorithm is used again to back-infer the camera extrinsic parameters.

[0168] With the involvement of depth information, camera pose estimation is more accurate and can compensate for drift caused by calibration errors or changes in illumination; K, R, and T can be further optimized by combining nonlinear least squares optimization.

[0169] This invention proposes a monocular camera depth estimation and localization method that integrates deep learning and traditional geometric algorithms. Without increasing hardware costs, it can recover the true-scale depth information of a scene and the spatial position of the camera using only monocular images, greatly improving the economy and practicality of 3D reconstruction systems. This method fuses a depth map predicted by a deep neural network with a PnP pose estimation algorithm, improving the system's robustness while maintaining accuracy. It is particularly suitable for scenarios in road detection, such as those with complex lighting and sparse textures, which are difficult for traditional methods to handle. Through coordinate reconstruction after depth completion and optimization of camera parameters, it effectively corrects scale ambiguity and pose deviations in traditional calibration, ultimately significantly improving the system's localization accuracy and 3D reconstruction effect in real-world scenarios. This method is applicable to various high-precision, low-cost visual perception applications such as intelligent transportation, robot navigation, and augmented reality.

[0170] In some embodiments of this application, a high-precision camera positioning and depth ranging system is also disclosed for performing the above-described high-precision camera positioning and depth ranging method. The system includes:

[0171] The parameter acquisition module is used to calibrate the intrinsic and extrinsic parameters of the monocular camera using the traditional Zhang Zhengyou calibration method, and to obtain the camera intrinsic and extrinsic parameter matrices; combined with the PnP algorithm, the camera pose is estimated to obtain the initial camera pose parameters.

[0172] The depth prediction module is used to extract features from images input from a monocular camera using a deep learning model trained on a multi-source dataset, generate a relative depth map, and predict the relative depth of each pixel in the image.

[0173] The image calibration module is used to calculate the scale factor based on the relative depth map using reference points with known real distances, and convert the relative depth map into a real-scale depth map; combining the real-scale depth map with the camera intrinsic and extrinsic parameter matrices, the image pixels are back-projected into three-dimensional space to obtain the three-dimensional coordinates of the scene;

[0174] The projection and reconstruction module combines the depth information output by the deep learning model with the PnP algorithm to optimize the camera's intrinsic and extrinsic matrices and pose. Through pixel-level depth map optimization, the 3D coordinates of each pixel are refined, improving the accuracy of depth estimation and camera positioning.

[0175] Reference Figure 4 The present invention illustrates a computer device for a control method for automated control, which may specifically include the following:

[0176] The computer device 12 described above is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0177] Bus 18 refers to one or more of several types of bus 18 architectures, including memory bus 18 or memory controller, peripheral bus 18, graphics acceleration port, processor, or local bus 18 using any of the various bus 18 architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus 18, Micro Channel Architecture (MAC) bus 18, Enhanced ISA bus 18, Audio / Video Electronics Standards Association (VESA) local bus 18, and Peripheral Component Interconnect (PCI) bus 18.

[0178] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0179] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 4Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42 configured to perform the functions of the embodiments of the present invention.

[0180] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory. Such program modules 42 include—but are not limited to—an operating system, one or more application programs, other program modules 42, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0181] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, camera, etc.), and with one or more devices that enable medical personnel to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 4 Not shown, it can be combined with computer device 12 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing unit 16, external disk drive array, RAID system, tape drive and data backup storage system 34, etc.

[0182] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the high-precision camera positioning and depth ranging method provided in the embodiments of the present invention.

[0183] That is, when the processing unit 16 executes the above program, it performs the following: calibrating the intrinsic and extrinsic parameters of the monocular camera using the traditional Zhang Zhengyou calibration method to obtain the camera intrinsic and extrinsic parameter matrices; performing pose estimation of the camera using the PnP algorithm to obtain initial camera pose parameters; extracting features from the image input to the monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map; based on the relative depth map, calculating the scale factor using reference points with known real distances to convert the relative depth map into a true-scale depth map; combining the true-scale depth map with the camera intrinsic and extrinsic parameter matrices to back-project the image pixels into three-dimensional space to obtain the three-dimensional coordinates of the scene; optimizing the camera intrinsic and extrinsic parameter matrices and pose using the depth information output by the deep learning model and the PnP algorithm; and refining the three-dimensional coordinates of each pixel through pixel-level depth map optimization to improve depth estimation accuracy and camera positioning accuracy.

[0184] In this embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the high-precision camera positioning and depth ranging method provided in all embodiments of this application:

[0185] In other words, when the program is executed by the processor, the following steps are implemented: The monocular camera's intrinsic and extrinsic parameters are calibrated using the traditional Zhang Zhengyou calibration method to obtain the camera's intrinsic and extrinsic parameter matrices; the camera's pose is estimated using the PnP algorithm to obtain initial camera pose parameters; features are extracted from the image input to the monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map; based on the relative depth map, a scale factor is calculated using reference points with known real distances to convert the relative depth map into a true-scale depth map; combining this true-scale depth map with the camera's intrinsic and extrinsic parameter matrices, the image pixels are back-projected into 3D space to obtain the scene's 3D coordinates; the depth information output by the deep learning model is combined with the PnP algorithm to optimize the camera's intrinsic and extrinsic parameter matrices and pose; and pixel-level depth map optimization refines the 3D coordinates of each pixel, improving depth estimation accuracy and camera positioning accuracy.

[0186] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPOM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0187] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0188] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the healthcare worker's computer, partially on the healthcare worker's computer, as a standalone software package, partially on the healthcare worker's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the healthcare worker's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0189] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0190] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0191] The above provides a detailed description of a high-precision camera positioning and depth ranging method and system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A high-precision camera positioning and depth ranging method, characterized in that, The method includes: The traditional Zhang Zhengyou calibration method is used to calibrate the intrinsic and extrinsic parameters of the monocular camera to obtain the camera intrinsic and extrinsic parameter matrices; the PnP algorithm is combined to estimate the camera pose and obtain the initial camera pose parameters. A relative depth map is generated by extracting features from images input from a monocular camera using a deep learning model trained on a multi-source dataset. Based on the relative depth map, the scale factor is calculated using reference points with known real distances, and the relative depth map is converted into a real-scale depth map. Combining the real-scale depth map with the camera intrinsic and extrinsic parameter matrices, the image pixels are back-projected into three-dimensional space to obtain the three-dimensional coordinates of the scene. By combining the depth information output by the deep learning model with the PnP algorithm, the camera intrinsic and extrinsic parameter matrices and pose are optimized. This includes: using the 3D coordinates obtained from backprojection and the corresponding image pixels as 3D-2D matching pairs, and re-estimating the camera extrinsic parameters using the PnP algorithm; refining the camera intrinsic and extrinsic parameters, including focal length and principal point coordinates, using a nonlinear least squares optimization method to reduce reprojection errors; performing local weighted correction on the depth map using multi-scale feature fusion technology to ensure depth estimation accuracy within different distance ranges; and refining the 3D coordinates of each pixel through pixel-level depth map optimization.

2. The method according to claim 1, characterized in that, The traditional Zhang Zhengyou calibration method is used to calibrate the intrinsic and extrinsic parameters of the monocular camera, obtaining the camera's intrinsic and extrinsic parameter matrices. The PnP algorithm is then used to estimate the camera's pose, yielding initial camera pose parameters, including: Capture planar checkerboard images from different angles, calculate the homography matrix between the image plane and the checkerboard world coordinate plane; calculate the camera's intrinsic and extrinsic parameter matrices: The intrinsic parameter matrix is ​​as follows: ， In the formula, , Let x be the focal length of the camera in the x and y directions. , Principal point coordinates; The extrinsic parameter matrix consists of a rotation matrix R and a translation vector T, and its formula is: , world coordinate system ( ) mapped to the pixel points in the image coordinate system ); The PnP algorithm combines feature points in the image with their corresponding world coordinates, and uses perspective geometry principles and camera intrinsic parameters to calculate the camera's rotation matrix R and translation vector T, thereby performing camera pose estimation and obtaining the initial camera pose parameters.

3. The method according to claim 1, characterized in that, The step of extracting features from images input by a monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map includes: The relative depth map is generated by fusing and training images and depth data from different indoor and outdoor environments, including at least the KITTI dataset, NYUv2 dataset, and MegaDepth dataset.

4. The method according to claim 1 or 3, characterized in that, The method involves using a deep learning model trained on a multi-source dataset to extract features from images input from a monocular camera, generating a relative depth map; wherein the structure of the deep learning model includes: The encoder uses either Vision Transformer or Swing Transformer. Vision Transformer divides the image into fixed-size image blocks, embeds them into a high-dimensional space through linear transformation, and then extracts global features through a self-attention layer and a feedforward neural network. Swing Transformer uses a windowed self-attention mechanism to fuse local and global information of the image. The decoder contains multiple feature fusion blocks, each of which consists of two residual convolutional units. Each residual convolutional unit includes two convolutional operations, batch normalization, and a ReLU activation function. The output layer restores the feature map to the same resolution as the input image through multi-level convolution and upsampling operations, and outputs a single-channel depth map. The operations of the output layer include: Conv2d(256,128,kernel_size=(3,3)), upsampling, Conv2d(128,32,kernel_size=(3,3)), and Conv2d(32,1,kernel_size=(1,1)). The multi-task learning and loss function layer includes joint optimization of depth estimation and pose optimization tasks, and a joint loss function based on depth prediction error and pose optimization error.

5. The method according to claim 1 or 3, characterized in that, The step of extracting features from images input by a monocular camera using a deep learning model trained on a multi-source dataset to generate a relative depth map includes: The deep learning model is trained using a multi-task loss function, specifically, using a joint loss function of depth prediction error and pose optimization error; wherein the depth prediction error employs a scale- and offset-invariant loss function. : ,in, and For the predicted depth value and the true depth value after scale and offset adjustment, M is the number of pixels and ρ is the type of loss function; Scale and offset adjustments are achieved through estimation functions s and t, where, s ， ; The predicted values ​​and the ground truth values ​​are aligned using the least squares criterion, where... , , ; The final loss function is: ,in, The number of training samples. For regularization loss, These are weight parameters.

6. The method according to claim 1, characterized in that, The step of converting the relative depth map into a true-scale depth map by calculating the scale factor using reference points with known true distances based on the relative depth map includes: Select pixels in the image whose true distance is known, and calculate the scale factor using the scale formula, where, The formula is: ； Multiply the relative depth value of each pixel in the relative depth map by the scale factor to obtain the true scale depth value; 。 7. The method according to claim 1, characterized in that, The image pixels are back-projected into three-dimensional space by combining the real-scale depth map with the camera's intrinsic and extrinsic parameter matrices to obtain the three-dimensional coordinates of the scene. The formula for back-projecting the pixel into three-dimensional space is as follows: In the formula, (u,v) are the image pixel coordinates, and d is the true scale depth value of that pixel. Let (X,Y,Z) be the inverse of the camera intrinsic parameter matrix, and let (X,Y,Z) be the three-dimensional coordinates in the camera coordinate system.

8. A high-precision camera positioning and depth ranging system, characterized in that, The system includes: The parameter acquisition module is used to calibrate the intrinsic and extrinsic parameters of the monocular camera using the traditional Zhang Zhengyou calibration method, and to obtain the camera intrinsic and extrinsic parameter matrices; combined with the PnP algorithm, the camera pose is estimated to obtain the initial camera pose parameters. The depth prediction module is used to extract features from images input from a monocular camera using a deep learning model trained on a multi-source dataset, and generate a relative depth map. The image calibration module is used to calculate the scale factor based on the relative depth map using reference points with known real distances, and convert the relative depth map into a real-scale depth map; combining the real-scale depth map with the camera intrinsic and extrinsic parameter matrices, the image pixels are back-projected into three-dimensional space to obtain the three-dimensional coordinates of the scene; The projection and reconstruction module combines the depth information output by the deep learning model with the PnP algorithm to optimize the camera's intrinsic and extrinsic matrices and pose. This includes: using the 3D coordinates obtained from backprojection and the corresponding image pixels as 3D-2D matching pairs, and re-estimating the camera's extrinsic parameters using the PnP algorithm; refining the camera's intrinsic and extrinsic parameters, including focal length and principal point coordinates, using a nonlinear least squares optimization method to reduce reprojection errors; performing local weighted correction on the depth map using multi-scale feature fusion technology to ensure depth estimation accuracy across different distance ranges; and refining the 3D coordinates of each pixel through pixel-level depth map optimization.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images