Cross-scene monocular depth estimation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a cross-scene monocular depth estimation method and device, equipment and a storage medium. The method comprises the steps that cross-scene to-be-processed image data are acquired, then the to-be-processed image data are input into a preset estimation model, target absolute depth image data are obtained, the estimation model is determined based on multiple pieces of first image data and reference absolute depth image data corresponding to each piece of first image data, and the target absolute depth image data are obtained; and performing relative depth conversion processing on the to-be-processed image data to obtain relative depth image data, and finally determining a target depth map corresponding to the to-be-processed image data according to the absolute depth image data and the relative depth image data. According to the method provided by the invention, monocular depth estimation during cross-scene application is realized, and the accuracy of depth estimation is effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a cross-scene monocular depth estimation method, apparatus, device and storage medium. Background Technology

[0002] Monocular depth estimation technology is widely used in scenarios such as autonomous driving, robot navigation, 3D reconstruction, augmented reality, and virtual reality. It requires predicting the depth information from each pixel in a scene to the camera from a single 2D image to reconstruct the 3D geometric information of the scene. Taking autonomous driving as an example, vehicles acquire the depth information of the road scene in real time through a monocular camera, which requires processing data from multiple scenes, including indoor and outdoor views, and head-up and overhead views.

[0003] Existing monocular depth estimation methods rely on training with datasets containing real depth annotations to achieve depth prediction by learning the mapping relationship between depth values ​​and RGB images. Furthermore, their generalization ability can be improved by joint training with multiple datasets or by introducing disparity consistency constraints.

[0004] However, existing methods suffer from poor depth estimation accuracy when applied across different scenarios. Summary of the Invention

[0005] This application provides a cross-scene monocular depth estimation method, apparatus, device, and storage medium to improve the accuracy of depth estimation.

[0006] In a first aspect, embodiments of this application provide a cross-scene monocular depth estimation method, including:

[0007] Acquire image data to be processed across different scenes;

[0008] The image data to be processed is input into a preset estimation model to obtain target absolute depth image data. The estimation model is determined based on multiple first image data and reference absolute depth image data corresponding to each first image data.

[0009] The image data to be processed is subjected to relative depth transformation to obtain relative depth image data;

[0010] Based on the absolute depth image data and the relative depth image data, the target depth map corresponding to the image data to be processed is determined.

[0011] In one or more embodiments, before inputting the image data to be processed into a preset estimation model to obtain the target absolute depth image data, the method further includes:

[0012] For each first image data, clustering is performed on the first image data to obtain multiple depth regions;

[0013] The first image data is labeled according to the multiple depth regions to obtain second image data with labels, wherein the labels are used to indicate the depth region information of the second image data.

[0014] The estimation model is determined based on multiple second image data and reference absolute depth image data corresponding to each first image data.

[0015] In one or more embodiments, the clustering process of the first image data to obtain multiple depth regions includes:

[0016] Determine the first depth value data corresponding to the reference absolute depth image data corresponding to the first image data;

[0017] Perform single-image camera calibration on the first image data to determine the camera pose data;

[0018] Feature extraction is performed on the first depth value data and the camera pose data to determine the original feature vector data;

[0019] The original feature vector data is standardized to obtain standard feature vector data;

[0020] The standard feature vector data is clustered to obtain multiple depth regions.

[0021] In one or more embodiments, determining the estimation model based on a plurality of second image data and reference absolute depth image data corresponding to each first image data includes:

[0022] S1, For each second image data, the second image data is cropped and adjusted to obtain the third image data;

[0023] S2, the third image data is input into the encoder unit of the initial estimation model to obtain the first feature data. The initial estimation model includes the encoder unit, the camera unit, and the decoder unit.

[0024] S3, obtain camera image data based on the first feature data and the camera unit;

[0025] S4, input the camera image data and the first feature data into the decoder unit to obtain the predicted absolute depth image data;

[0026] S5, update the parameters of the initial estimation model based on the predicted absolute depth image data and the reference absolute depth image data, repeat steps S1-S5 until the preset loss converges to the preset threshold, and determine the estimation model.

[0027] In one or more embodiments, cropping and adjusting the second image data to obtain the third image data includes:

[0028] Based on the second image data, determine the focal length data;

[0029] Based on the focal length data, preset resolution, and preset target field of view, a target rectangular region is determined;

[0030] The second image data is cropped and adjusted according to the target rectangular region to obtain the third image data.

[0031] In one or more embodiments, determining the target depth map corresponding to the image data to be processed based on the absolute depth image data and the relative depth image data includes:

[0032] Determine a first maximum depth value for the absolute depth image data and a second maximum depth value for the relative depth image data;

[0033] The scale scaling factor is determined based on the ratio of the first maximum depth value to the second maximum depth value;

[0034] The relative depth image data is scaled according to the scale factor to determine the target depth map corresponding to the image data to be processed.

[0035] In one or more embodiments, before performing relative depth transformation processing on the image data to be processed to obtain relative depth image data, the method further includes:

[0036] Based on the image data to be processed, a geometric vector field is generated, which is used to indicate the camera geometric relationship corresponding to the image data to be processed.

[0037] The image data to be processed is jointly processed based on the geometric vector field.

[0038] Secondly, embodiments of this application provide a cross-scene monocular depth estimation device, comprising:

[0039] The acquisition module is used to acquire image data to be processed across different scenes;

[0040] The processing module is used to input the image data to be processed into a preset estimation model to obtain target absolute depth image data. The estimation model is determined based on multiple first image data and reference absolute depth image data corresponding to each first image data.

[0041] The conversion module is used to perform relative depth conversion processing on the image data to be processed to obtain relative depth image data;

[0042] The determination module is used to determine the target depth map corresponding to the image data to be processed based on the absolute depth image data and the relative depth image data.

[0043] In one or more embodiments, before inputting the image data to be processed into a preset estimation model to obtain the target absolute depth image data, the processing module is further configured to:

[0044] For each first image data, clustering is performed on the first image data to obtain multiple depth regions;

[0045] The first image data is labeled according to the multiple depth regions to obtain second image data with labels, wherein the labels are used to indicate the depth region information of the second image data.

[0046] The estimation model is determined based on multiple second image data and reference absolute depth image data corresponding to each first image data.

[0047] In one or more embodiments, the processing module performs clustering processing on the first image data to obtain multiple depth regions, specifically for:

[0048] Determine the first depth value data corresponding to the reference absolute depth image data corresponding to the first image data;

[0049] Perform single-image camera calibration on the first image data to determine the camera pose data;

[0050] Feature extraction is performed on the first depth value data and the camera pose data to determine the original feature vector data;

[0051] The original feature vector data is standardized to obtain standard feature vector data;

[0052] The standard feature vector data is clustered to obtain multiple depth regions.

[0053] In one or more embodiments, the processing module determines the estimation model based on a plurality of second image data and reference absolute depth image data corresponding to each first image data, specifically for:

[0054] S1, For each second image data, the second image data is cropped and adjusted to obtain the third image data;

[0055] S2, the third image data is input into the encoder unit of the initial estimation model to obtain the first feature data. The initial estimation model includes the encoder unit, the camera unit, and the decoder unit.

[0056] S3, obtain camera image data based on the first feature data and the camera unit;

[0057] S4, input the camera image data and the first feature data into the decoder unit to obtain the predicted absolute depth image data;

[0058] S5, update the parameters of the initial estimation model based on the predicted absolute depth image data and the reference absolute depth image data, repeat steps S1-S5 until the preset loss converges to the preset threshold, and determine the estimation model.

[0059] In one or more embodiments, the processing module crops and adjusts the second image data to obtain third image data, specifically for:

[0060] Based on the second image data, determine the focal length data;

[0061] Based on the focal length data, preset resolution, and preset target field of view, a target rectangular region is determined;

[0062] The second image data is cropped and adjusted according to the target rectangular region to obtain the third image data.

[0063] In one or more embodiments, the determining module is specifically used for:

[0064] Determine a first maximum depth value for the absolute depth image data and a second maximum depth value for the relative depth image data;

[0065] The scale scaling factor is determined based on the ratio of the first maximum depth value to the second maximum depth value;

[0066] The relative depth image data is scaled according to the scale factor to determine the target depth map corresponding to the image data to be processed.

[0067] In one or more embodiments, before performing relative depth conversion processing on the image data to be processed to obtain relative depth image data, the conversion module is specifically used for:

[0068] Based on the image data to be processed, a geometric vector field is generated, which is used to indicate the camera geometric relationship corresponding to the image data to be processed.

[0069] The image data to be processed is jointly processed based on the geometric vector field.

[0070] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0071] The memory stores computer-executed instructions;

[0072] The processor executes computer execution instructions stored in the memory, such that the processor, when executed, is used to implement the method described in the first aspect and any of the embodiments above.

[0073] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the first aspect and any of the embodiments above.

[0074] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, is used to implement the cross-scene monocular depth estimation method as described in the first aspect and various possible implementations of the first aspect.

[0075] This application provides a cross-scene monocular depth estimation method, apparatus, device, and storage medium. The method first acquires cross-scene image data to be processed, then inputs the image data to be processed into a preset estimation model to obtain target absolute depth image data. The estimation model is determined based on multiple first image data and reference absolute depth image data corresponding to each first image data. Next, relative depth transformation processing is performed on the image data to be processed to obtain relative depth image data. Finally, based on the absolute depth image data and the relative depth image data, the target depth map corresponding to the image data to be processed is determined. In the above method, by inputting the image data to be processed into a preset estimation model and combining multiple first image data and their corresponding reference absolute depth image data, a target absolute depth image can be generated. The estimation model is fully trained and can effectively capture depth features in different scenes, thereby using existing datasets to improve the accuracy of depth estimation. Relative depth prediction for cross-scene images can capture the local depth relationship between pixels. Based on the relative depth map, absolute depth is predicted, global scale information is introduced, and the absolute depth value is used to linearly map the relative depth map to recover the physically meaningful depth measurement. The phased strategy solves the bottleneck of high-resolution image processing, while preserving details and scale information, thus improving the accuracy of depth estimation. Attached Figure Description

[0076] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0077] Figure 1 A flowchart illustrating the cross-scene monocular depth estimation method provided in this application embodiment. Figure 1 ;

[0078] Figure 2 A flowchart illustrating the cross-scene monocular depth estimation method provided in this application embodiment. Figure 2 ;

[0079] Figure 3 This is a schematic diagram of the clustering depth region provided in an embodiment of this application;

[0080] Figure 4 This is a schematic diagram of the structure of the initial estimation model provided in the embodiments of this application;

[0081] Figure 5 A flowchart illustrating the cross-scene monocular depth estimation method provided in this application embodiment. Figure 3 ;

[0082] Figure 6 This is a schematic diagram of the cross-scene monocular depth estimation device provided in an embodiment of this application;

[0083] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0084] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0085] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0086] Before introducing the embodiments of this application, the terms involved in the embodiments of this application will be explained first:

[0087] Monocular depth estimation: refers to the prediction of the depth information of each pixel in a scene to the camera using only a single RGB image (monocular image) in computer vision;

[0088] Cross-scene capability: This refers to the model's ability to adapt to different scenarios. Here, "scenario" usually refers to the environment or data distribution in a visual task.

[0089] Generalization refers to the ability of a model to maintain good performance on unseen data or data different from the training distribution.

[0090] Absolute depth: refers to the actual physical distance from a pixel to the camera.

[0091] Relative depth refers to the depth relationship or order between pixels, without regard to the actual physical distance, and is usually normalized to [0, 1].

[0092] Geocalib algorithm: refers to an algorithm that recovers the intrinsic and extrinsic parameters of a camera, as well as the camera's pose (position and orientation) in some applications, so that images captured by the camera can be correctly mapped to three-dimensional world coordinates;

[0093] The DepthAnything algorithm is an advanced monocular depth estimation method designed to estimate the depth information of a scene from a single image. Its core advantage lies in its powerful generalization ability and efficient inference speed, enabling it to provide accurate depth predictions in various complex scenes.

[0094] Vector orientation field: refers to a pixel-level geometric representation that assigns a vector to each pixel in an image to describe a certain orientation or geometric relationship.

[0095] Secondly, the application background of the embodiments of this application will be explained:

[0096] Monocular depth estimation technology is widely used in autonomous driving, robot navigation, 3D reconstruction, augmented reality, and virtual reality. It requires predicting the depth information from each pixel in a scene to the camera from a single 2D image to reconstruct the scene's 3D geometry. Taking autonomous driving as an example, vehicles acquire real-time depth information of road scenes through a monocular camera, processing data from multiple scenes, including indoor and outdoor views, and head-up and overhead views. In autonomous driving, vehicles need to perceive the depth information of their surroundings in real time to achieve path planning and obstacle avoidance, such as identifying road edges, pedestrian distances, and obstacle locations. In robot navigation, robots need to perceive environmental depth through a monocular camera to adapt to complex terrain and move autonomously. In 3D reconstruction tasks, monocular depth estimation can be used to construct 3D models from multi-view images, especially suitable for scenarios where multi-view cameras cannot be deployed. In AR / VR, accurate acquisition of depth information is crucial for the natural integration of virtual objects with real-world scenes.

[0097] Existing monocular depth estimation methods rely on training with datasets containing real depth annotations to make predictions by learning the mapping relationship between depth values ​​and RGB images. Furthermore, their generalization ability can be improved by joint training with multiple datasets or by introducing disparity consistency constraints.

[0098] However, the first type of method learns a metric for depth prediction by supervising training on a specific dataset, using images with real depth annotations. These methods often achieve high accuracy on the training set or a test set with a similar distribution, but their performance degrades significantly across different scenes due to substantial differences in depth distribution, such as between indoor and outdoor environments, and between natural environments and urban street scenes.

[0099] Another type of approach attempts to improve generalization ability by using joint training across multiple datasets or self-supervised learning, such as training using disparity consistency constraints of video sequences or relative depth information across datasets. While this type of approach alleviates the problem of performance degradation across scenes to some extent, it still faces challenges such as scale uncertainty and geometric ambiguity. Furthermore, the model may still experience significant performance fluctuations when faced with changes in camera parameters or unseen scenes.

[0100] In summary, existing methods suffer from poor depth estimation accuracy when applied across different scenarios.

[0101] This application provides a cross-scene monocular depth estimation method aimed at solving the aforementioned technical problems of existing technologies. The technical concept of this application is as follows: Existing monocular depth estimation methods suffer from poor depth estimation accuracy when applied across scenes. To address the depth ambiguity problem in existing methods, this application proposes to divide the scene domain through depth map clustering, and jointly train the model with RGB images and depth maps, enabling the model to learn the depth distribution patterns of different scenes. To solve the high-resolution image processing problem, it considers first capturing local structural information through relative depth prediction, then providing global scale constraints through absolute depth recovery, and finally using absolute depth to correct relative depth, forming a closed-loop optimization. Furthermore, to address the multi-view inconsistency problem, a pixel-level geometric vector field (such as surface normal vectors and vanishing point direction fields) is introduced, combined with a multi-view joint training strategy, to explicitly model the scene geometry, and adapt to different camera parameters through a dynamic feature modulation mechanism. Finally, by jointly optimizing depth supervision, photometric consistency, and vector field consistency losses, a unified modeling of local pixel relationships and global geometric structure is achieved, significantly improving the model's generalization ability in complex scenes.

[0102] The execution subject of this application embodiment is an electronic device, which can be a terminal device, such as a laptop, desktop computer, or tablet computer, or a server. In practical applications, whether the electronic device is a terminal device or a server can be determined according to the actual situation, and no specific limitation is imposed on it.

[0103] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0104] Figure 1 A flowchart illustrating the cross-scene monocular depth estimation method provided in this application embodiment. Figure 1 .like Figure 1 As shown, this cross-scene monocular depth estimation method includes the following steps:

[0105] S110. Obtain cross-scene image data to be processed;

[0106] In this step, image data covering different environments, lighting, and scene types are collected to provide diverse inputs for subsequent depth estimation.

[0107] For example, the image to be processed must meet the basic requirements of being clear and distortion-free with uniform pixel resolution. At the same time, the camera parameters (such as focal length and shooting angle) corresponding to the image must be recorded to provide a geometric basis for depth calculation. That is, the image data to be processed includes RGB images and corresponding camera intrinsic parameter data.

[0108] In one possible implementation, images are captured by an RGB camera in indoor (living room, office) and outdoor (street, park) scenes, covering different lighting conditions such as sunny and cloudy days, as well as complex compositions with mixed foreground, middle ground, and background. The images are then stored according to "scene type-image number", the camera intrinsic parameter files are saved synchronously, and the image format is unified and the resolution is adjusted.

[0109] S120. Input the image data to be processed into the preset estimation model to obtain the target absolute depth image data;

[0110] The estimation model is determined based on multiple first image data and reference absolute depth image data corresponding to each first image data.

[0111] In this step, an estimation model is determined based on multiple first image data and the reference absolute depth image data corresponding to each first image data. This model can directly output pixel-level physical unit depth values. The image data to be processed is input into the preset estimation model, and the estimation model directly outputs the target absolute depth image data.

[0112] In one possible implementation, the image data to be processed needs to be preprocessed before being input into the preset estimation model to ensure that the format of the image data to be processed is consistent with the format of the first image data, so as to avoid scale bias affecting the prediction accuracy of the estimation model.

[0113] S130. Perform relative depth transformation on the image data to be processed to obtain relative depth image data;

[0114] In this step, relative depth transformation focuses on capturing the depth difference relationship between pixels in the image. It does not rely on the absolute physical scale and can adapt to the depth distribution characteristics of different scenes. Therefore, relative depth transformation is performed on the image data to be processed to construct the depth ranking relationship between pixels and output a normalized relative depth image.

[0115] In one possible implementation, an encoder-decoder-based depth difference network, such as the relative monocular depth estimation DepthAnything model, is used. After inputting the image to be processed, multi-scale features are extracted through multi-layer convolution, the depth similarity of adjacent pixels is calculated, and then the depth values ​​are normalized to the [0, 1] interval, where the larger the value, the farther the relative distance. Finally, a single-channel relative depth image with the same resolution as the input image is generated.

[0116] In one possible implementation, prior to step S130 above, the cross-scene monocular depth estimation method further includes the following steps:

[0117] Step 1: Generate a geometric vector field based on the image data to be processed;

[0118] Among them, the geometric vector field is used to indicate the camera geometric relationship corresponding to the image data to be processed;

[0119] For example, pixel-level geometric relationships can be modeled using camera geometric information to provide explicit geometric constraints for depth estimation and solve the problem of depth prediction bias under different viewpoints.

[0120] The geometric vector field includes the surface normal field and the direction field pointing to the vanishing point, which respectively characterize the pixel surface orientation and the global geometric trend of the scene. It is directly related to the imaging relationship defined by the camera's intrinsic and extrinsic parameters. By explicitly modeling the geometric laws of the scene, the relative monocular depth estimation model can perceive the depth change trend under different viewpoints.

[0121] In one possible implementation, the Geocalib algorithm is used to estimate the camera intrinsic parameters (focal length, principal point) and extrinsic parameters (camera pose quadruple) of the image to be processed. Based on the pinhole camera model, the surface normal vector of each pixel (describing the local plane orientation) is calculated through the mapping relationship between spatial points and pixel points. At the same time, the vanishing point of the image is detected, and a direction field pointing to the vanishing point is generated (characterizing the global perspective relationship). Finally, a dual-channel geometric vector field data with the same resolution as the input image is formed. Finally, the geometric vector field and the image data to be processed are jointly input into the relative monocular depth estimation model.

[0122] For example, from a top-down view, the relative monocular depth estimation model can compensate for prediction errors caused by rapid changes in depth in the vertical direction through the vanishing point orientation field; from a level view, the model can avoid depth distortion caused by perspective distortion through surface normal vector constraints. Geometric vector field constraints significantly improve the model's robustness to depth prediction in cross-view tasks and enhance prediction consistency across multiple viewpoints.

[0123] Step 2: Perform joint processing on the image data to be processed based on the geometric vector field.

[0124] For example, the geometric vector field is combined with the image data to be processed, and then simultaneously input into the relative monocular depth estimation model for relative depth transformation processing.

[0125] In one possible implementation, constraints of the geometric vector field are incorporated into the image feature extraction process, allowing features to simultaneously carry visual information and geometric priors, thus improving the cross-view consistency of depth prediction. Through feature modulation or constraint loss, the consistency between image features and the geometric vector field is ensured, avoiding depth errors caused by geometric distortion.

[0126] S140. Determine the target depth map corresponding to the image data to be processed based on the absolute depth image data and the relative depth image data.

[0127] In this step, based on absolute depth image data and relative depth image data, the physical scale accuracy of absolute depth and the scene adaptability of relative depth are fused to correct the bias of single depth estimation and obtain a target depth map with higher accuracy and stronger generalization.

[0128] In one possible implementation, the scale factor of the absolute depth map and the relative depth map is calculated (e.g., the statistical ratio of their depths in the same key area), and the relative depth map is scaled. Then, a weighted fusion strategy is adopted, with the near-field area dominated by absolute depth data (weight 0.7) and the far-field area given a higher relative depth weight (weight 0.5). After fusion, noise is removed by median filtering, and the final target depth map is output, which retains the meter-level physical scale and conforms to the depth hierarchy within the scene.

[0129] In one possible implementation, step S140 described above may include the following steps:

[0130] Step 1: Determine the first maximum depth value of the absolute depth image data and the second maximum depth value of the relative depth image data.

[0131] For example, key extreme values ​​of two types of depth data are extracted to provide a benchmark for subsequent scale factor calculation. The first maximum depth value is the upper limit of the actual physical distance, and the second maximum depth value is the upper limit of the relative depth ratio. The correspondence between the two is the core basis for achieving scale calibration.

[0132] In one possible implementation, the entire image to be processed... Input into the relative monocular depth estimation DepthAnything model In this process, the normalized relative depth image is obtained. , means as follows:

[0133]

[0134] An absolute depth image is represented as:

[0135]

[0136] in, A subgraph representing the entire image to be processed, i.e.

[0137] Step 2: Determine the scale scaling factor based on the ratio of the first maximum depth value to the second maximum depth value.

[0138] For example, the first maximum depth value is The second maximum depth value is Then the scaling factor is:

[0139]

[0140] Step 3: Scale the relative depth image data according to the scale factor to determine the target depth map corresponding to the image data to be processed.

[0141] For example, based on the scale factor, the absolute depth of the subgraph is utilized. relative depth of the entire map Scale recovery is performed to obtain the final target depth. for:

[0142]

[0143] By combining absolute and relative depth information, accurate conversion of depth image data is achieved, which helps to improve the accuracy and reliability of depth estimation.

[0144] The cross-scene monocular depth estimation method of this application first acquires cross-scene image data to be processed, then inputs the image data to be processed into a preset estimation model to obtain target absolute depth image data. The estimation model is determined based on multiple first image data and reference absolute depth image data corresponding to each first image data. Next, relative depth transformation processing is performed on the image data to be processed to obtain relative depth image data. Finally, the target depth map corresponding to the image data to be processed is determined based on the absolute depth image data and the relative depth image data. In this embodiment, by inputting the image data to be processed into the preset estimation model and combining multiple first image data and their corresponding reference absolute depth image data, a target absolute depth image can be generated. The estimation model, after sufficient training, can effectively capture depth features in different scenes, thereby utilizing existing datasets to improve the accuracy of depth estimation. Relative depth prediction of cross-scene images can capture local depth relationships between pixels. Predicting absolute depth based on the relative depth map introduces global scale information, uses absolute depth values ​​to linearly map the relative depth map, and recovers physically meaningful depth measurements. This phased strategy solves the bottleneck of high-resolution image processing while preserving detail and scale information, thus improving the accuracy of depth estimation.

[0145] Based on the above embodiments, Figure 2 A flowchart illustrating the cross-scene monocular depth estimation method provided in this application embodiment. Figure 2 Prior to step S120 above, this cross-scene monocular depth estimation method further includes the following steps:

[0146] S210. For each first image data, perform clustering processing on the first image data to obtain multiple depth regions.

[0147] In this step, for each first image data, the depth distribution pattern of the first image data is mined by clustering, and continuous depth values ​​are divided into semantically consistent discrete regions, thus injecting depth prior information into the model.

[0148] For example, the first image data includes an RGB image and camera intrinsic data.

[0149] In one possible implementation, for each RGB image, the Geocalib algorithm is used to obtain the camera pose quadruple, and then the minimum, maximum, and mean values ​​of its depth map are counted to form a 9-dimensional feature vector. After z-score standardization, K-means clustering (K=4) is used to calculate the sample similarity through unweighted Euclidean distance, and four cluster centers are iteratively determined. Finally, the reference absolute depth image data corresponding to the first image data is divided to form multiple depth regions of the image.

[0150] In one possible implementation, step S210 above, "clustering the first image data to obtain multiple depth regions," may include the following steps:

[0151] Step 1: Determine the first depth value data corresponding to the reference absolute depth image data corresponding to the first image data;

[0152] For example, the true depth information corresponding to the reference absolute depth image data corresponding to the first image data is obtained, providing basic data for subsequent feature extraction and depth region segmentation.

[0153] The first depth value data needs to accurately reflect the physical distance from the pixel to the camera to ensure that the subsequent clustering results fit the actual depth distribution of the scene.

[0154] In one possible implementation, the reference absolute depth image data corresponding to each RGB image is extracted, and the depth values ​​of all valid pixels are extracted from the depth map to form the first depth value data set of the image. Invalid zero values ​​or outliers are filtered out, and finally stored in the form of an array, with each element corresponding to the true physical depth of the image pixel.

[0155] Step 2: Perform single-image camera calibration on the first image data to determine the camera pose data;

[0156] For example, the purpose of single-image camera calibration is to obtain the camera's external parameters, i.e., pose information, which can characterize the camera's position and orientation when shooting, providing geometric constraints for depth feature modeling.

[0157] In one possible implementation, the Geocalib algorithm is used to calibrate each first image data. After inputting an RGB image, the algorithm analyzes the geometric structure and perspective relationships in the image to output a camera pose quadruple. This quadruple fully describes the spatial pose of the camera and is stored in correspondence with the first depth value data, preparing for the subsequent construction of multi-dimensional feature vectors.

[0158] Step 3: Extract features from the first depth value data and camera pose data to determine the original feature vector data;

[0159] For example, by extracting statistical features of depth values ​​and camera pose features, a multi-dimensional original feature vector is constructed to comprehensively characterize the depth distribution and shooting geometry of the image.

[0160] The feature vectors need to cover key statistical information of the depth data and camera pose information to provide a valid basis for subsequent clustering.

[0161] In one possible implementation, for the first depth value data set The pixel depth value is Statistical features, including minimum, maximum, mean, standard deviation, and median, are calculated. These depth statistical features are then combined with the camera pose quadruple to obtain the original feature vector data. , means as follows:

[0162]

[0163] Each dimension corresponds to a depth statistical feature or camera pose parameter. The depth statistical features include: minimum value. maximum value mean Standard deviation and median .

[0164] Step 4: Standardize the original feature vector data to obtain standard feature vector data;

[0165] For example, the core of standardization is to eliminate the dimensional differences between different feature dimensions (such as depth values ​​being in meters and pose parameters being dimensionless values), to prevent a certain feature from dominating the clustering results due to its excessively large numerical range, and to ensure that each feature has a balanced weight in the clustering.

[0166] In one possible implementation, the original feature vectors are processed using the z-score normalization method. The mean and standard deviation of each feature dimension (9 dimensions in total) are calculated over the entire dataset, and each feature dimension is then normalized.

[0167]

[0168] in, and They represent the first The dimension is the mean and standard deviation over the entire dataset.

[0169] The original feature vector data is standardized to obtain standard feature vector data with a mean of 0 and a standard deviation of 1 for each dimension, thus eliminating the influence of dimensions.

[0170] Step 5: Perform clustering on the standard feature vector data to obtain multiple depth regions.

[0171] For example, a clustering algorithm can be used to divide image pixels with similar features into the same depth region, explore the correlation between depth distribution and camera pose, and achieve discretization classification of depth range.

[0172] In one possible implementation, the K-means clustering algorithm is used, with the number of clusters K=4. The standard feature vector is used as input, and the similarity between samples is measured by calculating the unweighted Euclidean distance. The cluster centers are iteratively optimized until convergence.

[0173] Randomly select a sample as the center, and apply standardized features to each sample. To calculate the unweighted Euclidean distance:

[0174]

[0175] With probability Choose the next center and repeat until you get K (K=4) initial centers.

[0176] Based on the final cluster center, the reference absolute depth image data corresponding to the first image data is divided to form multiple depth regions of the image. Figure 3 This is a schematic diagram of the clustering depth region provided in the embodiments of this application, such as... Figure 3 As shown, clustering visualization is performed using four depth regions as an example.

[0177] S220. Label the first image data according to multiple depth regions to obtain second image data with labels;

[0178] The labels are used to indicate the depth region information of the second image data;

[0179] In this step, the deep region information obtained from clustering is bound to the first image data to obtain the second image data with labels, so that the subsequent model can learn the association between visual features and deep regions at the same time during training.

[0180] For example, the labels need to clearly indicate the depth region to which each pixel belongs, providing a supervision signal for subsequent joint training.

[0181] In one possible implementation, a pixel-level annotation method is used to assign a unique label to each depth region, generating a single-channel label map with the same resolution as the first image data. The value of each pixel in the label map corresponds to the depth region number to which it belongs, forming the second image data carrying the depth region label.

[0182] Furthermore, the second image data can be grouped according to the depth regions indicated by the labels, better constraining the scene domain of the image. Simultaneously, considering that the amount of information used for depth estimation decreases as the depth value increases, a spatially incremental strategy can be adopted to generate more depth ranges when the depth value is small. Assuming a depth range is... , then the first The depth range can be represented as:

[0183]

[0184] S230. Determine the estimation model based on multiple second image data and the reference absolute depth image data corresponding to each first image data.

[0185] For example, joint training allows the model to learn the mapping relationship between "visual features - deep regions - absolute depth," utilizing both the prior constraints provided by deep region labels and the accuracy of the measurement through absolute depth data. The training process needs to incorporate multi-dimensional loss to improve the model's cross-scene generalization ability.

[0186] In one possible implementation, an "encoder-decoder" architecture model is constructed. The encoder uses a fine-tuned DepthAnything model to extract image features, and the decoder incorporates a deep region attention mechanism. The second image data (RGB image + deep region label) is used as input, and the absolute depth image data is used as the supervision target for training.

[0187] Then, multi-loss joint optimization (including deep supervision loss, SILog loss, etc.) is adopted to allow the model to learn deep region classification and absolute depth regression at the same time. The model is iteratively trained until it converges, and finally an estimation model that can be adapted to cross-scenes is obtained.

[0188] In one possible implementation, step S230 described above may include the following steps:

[0189] S1, For each second image data, the second image data is cropped and adjusted to obtain the third image data;

[0190] For example, cropping the input scale and field of view of the unified second image data (carrying depth region labels) eliminates interference from invalid regions at the image edges, ensuring that the image input to the model meets the geometric constraints required for training.

[0191] In one possible implementation, step S1 above, "cropping and adjusting the second image data to obtain the third image data," may include the following steps:

[0192] Step 1: Determine the focal length data based on the second image data;

[0193] For example, focal length is a core parameter relating camera imaging and scene geometry, directly affecting the field of view and depth mapping. This step requires extracting or estimating the camera's focal length data from the second image data carrying depth region labels, providing a foundation for subsequent calculation of the target rectangular region.

[0194] In one possible implementation, the focal length is calculated using camera intrinsic data associated with the second image data. in, and These represent the calibrated horizontal and vertical focal lengths, respectively.

[0195] Step 2: Determine the target rectangular area based on the focal length data, preset resolution, and preset target field of view;

[0196] For example, by using the geometric relationship between focal length and field of view, an image region that fits the target field of view is calculated, so that the cropped image meets the unified field of view requirements for model training.

[0197] The preset resolution must be consistent with the input size of the subsequent model, while the preset target field of view is set according to the generalization requirements across scenes.

[0198] In one possible implementation, the preset resolution is... The preset target field of view size is Then, based on the field-of-view formula, the input second image data is calculated to match the preset target field of view. Equivalent target rectangular region .

[0199] The field of view formula is as follows:

[0200]

[0201] Step 3: Crop and adjust the second image data according to the target rectangular area to obtain the third image data.

[0202] For example, the core of cropping adjustment is to retain valid image information within the target rectangular area, eliminate redundant content outside the field of view, and ensure that the image of the input model is completely uniform in terms of field of view and resolution. During the process, it is necessary to avoid cropping to critical depth regions while ensuring that the size of the cropped image meets the preset requirements.

[0203] In one possible implementation, based on the target rectangular region The second image's RGB image is cropped. If the target rectangular area exceeds the original RGB image range, the pixels in the excess area are filled with 255. After cropping, the image is scaled to ensure that the resolution of the final output third image data strictly matches the preset resolution.

[0204] S2, input the third image data into the encoder unit of the initial estimation model to obtain the first feature data;

[0205] The initial estimation model includes an encoder unit, a camera unit, and a decoder unit.

[0206] For example, Figure 4 This is a schematic diagram of the structure of the initial estimation model provided in the embodiments of this application, as shown below. Figure 4As shown, the initial estimation model includes an encoder unit, a camera unit, and a decoder unit. The encoder unit uses a fine-tuned DepthAnything model, which can output robust depth visual features.

[0207] The third image data is input into the encoder, and feature maps of five stages are extracted through multi-layer convolution and attention mechanisms. The shallow features retain edge texture details, and the deep features are fused with the semantics of the deep region. The features of each stage are scaled according to the resolution and then stitched together to obtain the first feature data.

[0208] S3, obtain camera image data based on the first feature data and the camera unit;

[0209] For example, the camera unit models the camera's intrinsic and extrinsic parameters, combines the first feature data with imaging geometric constraints, and generates camera image data that fuses camera parameters, providing geometric priors for the decoder and mitigating depth prediction bias under multiple viewpoints.

[0210] The goal of monocular depth estimation is to recover three-dimensional structures from two-dimensional images, and the pinhole camera model provides an accurate geometric description of this process, namely, spatial points. pixel The relationship can be expressed as:

[0211]

[0212]

[0213] in, Indicates camera parameters, This represents the camera's extrinsic parameters. Since the imaging geometry varies significantly depending on the viewing angle (such as eye level and overhead view), if the network ignores the camera parameters during training, it can easily lead to depth scale bias and geometric distortion.

[0214] In one possible implementation, the camera unit includes a parameter decoupling module and a vector field generation module. First, the camera's intrinsic and extrinsic parameters are acquired. Then, the parameter decoupling module performs a nonlinear mapping between the intrinsic and extrinsic parameters and converts them into learnable feature modulation parameters. and For the first feature data Dynamic correction is performed to obtain intermediate feature data. , means as follows:

[0215]

[0216] in, This represents the first feature data. and These represent the characteristic modulation parameters, which can achieve adaptive correction for different imaging geometries.

[0217] Then, the vector field generation module calculates the pixel-level surface normal vector field and vanishing point direction field based on the corrected features, and concatenates them with the corrected features to obtain camera image data containing geometric information.

[0218] On the pixel plane, each point corresponds to a three-dimensional vector in the depth direction, forming a mapping. This vector field not only describes the spatial location of points, but also includes the geometric flow characteristics across viewpoints during training.

[0219] A multi-view joint training strategy is adopted, assuming that the same scene is trained from multiple perspectives. The depth prediction below is It can be projected to another viewpoint under the camera geometry. :

[0220]

[0221] in, and These respectively represent the perspectives and perspective The pixel coordinates below.

[0222] Furthermore, a direction field consistency constraint is introduced, requiring that depth predictions from different viewpoints be geometrically consistent, i.e.:

[0223]

[0224] in, This represents the projection and resampling operators.

[0225] S4, input the camera image data and the first feature data into the decoder unit to obtain the predicted absolute depth image data;

[0226] For example, the decoder unit gradually restores the resolution and outputs pixel-level absolute depth values ​​by fusing visual features (first feature data) with geometric features (camera image data), while introducing depth region constraints to ensure prediction accuracy.

[0227] In one possible implementation, the decoder unit adopts a DPT (Dense Prediction Transformer) structure, which multiplies the camera image data and the first feature data pixel by pixel to strengthen the correlation between geometric and visual features. Through five stages of upsampling and residual convolution blocks, the feature map resolution is gradually restored to the preset resolution. Each stage incorporates the interval center of the corresponding depth region for linear constraint.

[0228] Features of each stage of the decoder unit The resolution size is ,in, , indicating the number of each stage.

[0229] In the first phase, It is compressed into N channels, then multiplied pixel by pixel to generate a low-resolution output. In the subsequent s-th stage, the depth map of the previous stage is... Upsampling, and then using residual convolutional blocks with features The features are fused, and then the fused features are linearly combined with the center of the divided intervals to generate a predicted absolute depth map. .

[0230] S5. Update the parameters of the initial estimation model based on the predicted absolute depth image data and the reference absolute depth image data. Repeat steps S1-S5 until the preset loss converges to the preset threshold, and determine the estimation model.

[0231] For example, by calculating the loss between the predicted absolute depth image data and the reference absolute depth image data, the model parameters are iteratively optimized to ensure the absolute depth prediction accuracy of the model across different scenes. The convergence threshold needs to balance training efficiency and prediction error.

[0232] In one possible implementation, a pre-defined loss function is calculated using the training absolute depth image data corresponding to the third image data as a reference:

[0233]

[0234] in, Indicates loss due to deep surveillance. Indicates loss of photometric uniformity. This represents the field loss in the vanishing point direction. This indicates the predicted smoothing loss. This represents the scale-independent logarithmic loss. This represents the multi-scale gradient loss.

[0235] The Adam optimizer is used. After each iteration, backpropagation is used to update the parameters of the encoder unit, camera unit, and decoder unit. Training stops when the total loss of 10 consecutive epochs is less than the preset threshold of 0.01, and the final estimated model is determined.

[0236] The cross-scene monocular depth estimation method of this application first performs clustering processing on each first image data to obtain multiple depth regions. Then, it labels the first image data according to these multiple depth regions to obtain labeled second image data. The labels indicate the depth region information of the second image data. Finally, it determines the estimation model based on the multiple second image data and the corresponding reference absolute depth image data for each first image data. In this embodiment, by clustering each first image data, different depth information in the image can be effectively divided into multiple depth regions, helping the estimation model capture the spatial depth differences of different objects or scene elements, thereby improving the understanding of complex scenes. By combining multiple depth regions with the first image data to generate labeled second image data, and using multiple second image data and their corresponding reference absolute depth image data to construct the estimation model, richer depth features can be learned during training. This improves the model's generalization ability and allows it to better adapt to depth estimation tasks under different scenes and conditions. Through clustering and labeling, rich training data is provided for the depth estimation model, improving the model's accuracy and generalization ability.

[0237] Based on the above embodiments, Figure 5 A flowchart illustrating the cross-scene monocular depth estimation method provided in this application embodiment. Figure 3 Combining Figure 5 The specific process of the cross-scene monocular depth estimation method provided in the embodiments of this application is described below:

[0238] S510, acquire multiple first image data and reference absolute depth image data corresponding to each first image data;

[0239] For example, multiple first image data and the reference absolute depth image data corresponding to each first image data are used as the training dataset.

[0240] S520. Cluster each first image data to obtain multiple depth regions;

[0241] S530. Label each first image data according to multiple depth regions to obtain second image data with labels;

[0242] S540. Construct the initial estimation model;

[0243] The initial estimation model includes an encoder unit, a camera unit, and a decoder unit. The encoder unit adopts a fine-tuned DepthAnything model, and the decoder unit adopts a DPT structure.

[0244] S550. The initial estimation model is trained based on multiple first image data and the reference absolute depth image data corresponding to each first image data to obtain the preset estimation model;

[0245] S560. Input the image data to be processed into the preset estimation model to obtain the target absolute depth image data.

[0246] Based on the above embodiments, the following are embodiments of the apparatus involved in this application:

[0247] Figure 6 This is a schematic diagram of the cross-scene monocular depth estimation device provided in an embodiment of this application. Figure 6 As shown, the cross-scene monocular depth estimation device 600 includes:

[0248] The acquisition module 610 is used to acquire image data to be processed across different scenes;

[0249] The processing module 620 is used to input the image data to be processed into a preset estimation model to obtain the target absolute depth image data, wherein the estimation model is determined based on multiple first image data and reference absolute depth image data corresponding to each first image data;

[0250] The conversion module 630 is used to perform relative depth conversion processing on the image data to be processed, so as to obtain relative depth image data;

[0251] The determination module 640 is used to determine the target depth map corresponding to the image data to be processed based on the absolute depth image data and the relative depth image data.

[0252] In one or more embodiments, before inputting the image data to be processed into a preset estimation model to obtain the target absolute depth image data, the processing module 620 is further configured to:

[0253] For each first image data, clustering is performed on the first image data to obtain multiple depth regions;

[0254] The first image data is labeled according to multiple depth regions to obtain the second image data with labels, wherein the labels are used to indicate the depth region information of the second image data;

[0255] The estimation model is determined based on multiple second image data and the reference absolute depth image data corresponding to each first image data.

[0256] In one or more embodiments, the processing module 620 performs clustering processing on the first image data to obtain multiple depth regions, specifically for:

[0257] Determine the first depth value data corresponding to the reference absolute depth image data corresponding to the first image data;

[0258] Perform single-image camera calibration on the first image data to determine the camera pose data;

[0259] Feature extraction is performed on the first depth value data and camera pose data to determine the original feature vector data;

[0260] The original feature vector data is standardized to obtain standard feature vector data.

[0261] Clustering is performed on the standard feature vector data to obtain multiple depth regions.

[0262] In one or more embodiments, the processing module 620 determines an estimation model based on a plurality of second image data and reference absolute depth image data corresponding to each first image data, specifically for:

[0263] S1, For each second image data, the second image data is cropped and adjusted to obtain the third image data;

[0264] S2, input the third image data into the encoder unit of the initial estimation model to obtain the first feature data, wherein the initial estimation model includes an encoder unit, a camera unit, and a decoder unit;

[0265] S3, obtain camera image data based on the first feature data and the camera unit;

[0266] S4, input the camera image data and the first feature data into the decoder unit to obtain the predicted absolute depth image data;

[0267] S5. Update the parameters of the initial estimation model based on the predicted absolute depth image data and the reference absolute depth image data. Repeat steps S1-S5 until the preset loss converges to the preset threshold, and determine the estimation model.

[0268] In one or more embodiments, the processing module 620 crops and adjusts the second image data to obtain third image data, specifically for:

[0269] Based on the second image data, determine the focal length data;

[0270] Based on the focal length data, preset resolution, and preset target field of view, determine the target rectangular area;

[0271] The second image data is cropped and adjusted according to the target rectangular area to obtain the third image data.

[0272] In one or more embodiments, the determining module 640 is specifically used for:

[0273] Determine the first maximum depth value of the absolute depth image data and the second maximum depth value of the relative depth image data;

[0274] The scale factor is determined based on the ratio of the first maximum depth value to the second maximum depth value;

[0275] The relative depth image data is scaled based on a scale factor to determine the target depth map corresponding to the image data to be processed.

[0276] In one or more embodiments, before performing relative depth conversion on the image data to be processed to obtain relative depth image data, the conversion module 630 is specifically used for:

[0277] Based on the image data to be processed, a geometric vector field is generated, which is used to indicate the camera geometric relationship corresponding to the image data to be processed.

[0278] The image data to be processed is jointly processed based on the geometric vector field.

[0279] The cross-scene monocular depth estimation device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0280] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 includes: a processor 710, a memory 720, and a bus 730;

[0281] The memory 720 is used to store the computer-executed instructions of the processor 710;

[0282] The processor 710 is configured to execute the technical solutions of any of the foregoing method embodiments by executing computer execution instructions.

[0283] The specific implementation process of processor 710 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0284] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0285] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0286] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0287] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0288] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0289] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0290] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0291] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0292] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0293] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0294] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory (RAM), magnetic disks, or optical disks.

[0295] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0296] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for cross-scene monocular depth estimation, the method comprising: The method comprises: acquiring cross-scene image data to be processed; inputting the image data to be processed into a preset estimation model to obtain target absolute depth image data, the estimation model being determined based on a plurality of first image data and reference absolute depth image data corresponding to each first image data; performing relative depth conversion processing on the image data to be processed to obtain relative depth image data; determining a target depth map corresponding to the image data to be processed according to the absolute depth image data and the relative depth image data.

2. The method of claim 1, wherein, Before inputting the image data to be processed into the preset estimation model to obtain the target absolute depth image data, the method further comprises: for each first image data, performing clustering processing on the first image data to obtain a plurality of depth regions; annotating the first image data according to the plurality of depth regions to obtain second image data carrying labels, the labels being used to indicate depth region information of the second image data; determining the estimation model according to a plurality of second image data and reference absolute depth image data corresponding to each first image data.

3. The method of claim 2, wherein, The clustering processing on the first image data to obtain a plurality of depth regions comprises: determining first depth value data corresponding to reference absolute depth image data corresponding to the first image data; performing single-image camera calibration on the first image data to determine camera pose data; performing feature extraction on the first depth value data and the camera pose data to determine original feature vector data; performing standardization processing on the original feature vector data to obtain standard feature vector data; performing clustering division processing on the standard feature vector data to obtain a plurality of depth regions.

4. The method according to claim 2 or 3, characterized in that, The determination of the estimation model according to a plurality of second image data and reference absolute depth image data corresponding to each first image data comprises: S1, for each second image data, performing cropping adjustment on the second image data to obtain third image data; S2, inputting the third image data into an encoder unit of an initial estimation model to obtain first feature data, the initial estimation model comprising the encoder unit, a camera unit, and a decoder unit; S3, obtaining camera image data according to the first feature data and the camera unit; S4, inputting the camera image data and the first feature data into the decoder unit to obtain predicted absolute depth image data; S5, performing parameter update on the initial estimation model according to the predicted absolute depth image data and the reference absolute depth image data, repeating steps S1-S5 until a preset loss converges to a preset threshold value, and determining the estimation model.

5. The method of claim 4, wherein, The cropping adjustment on the second image data to obtain third image data comprises: determining focal length data based on the second image data; determining a target rectangular region according to the focal length data, a preset resolution, and a preset target field of view; cropping and adjusting the second image data according to the target rectangular region to obtain the third image data.

6. The method according to any one of claims 1 to 3, characterized in that, The method comprises: determining a first maximum depth value of the absolute depth image data and a second maximum depth value of the relative depth image data; determining a scale factor according to a ratio of the first maximum depth value and the second maximum depth value; performing scale conversion on the relative depth image data according to the scale factor to determine a target depth map corresponding to the image data to be processed.

7. The method of claim 6, wherein, Before the relative depth conversion processing on the image data to be processed is performed to obtain the relative depth image data, the method further comprises: generating a geometric vector field based on the image data to be processed, the geometric vector field being used to indicate a camera geometric relationship corresponding to the image data to be processed; performing joint processing on the image data to be processed according to the geometric vector field.

8. A cross-scene monocular depth estimation apparatus, comprising: The method comprises: an acquisition module configured to acquire image data to be processed across scenes; a processing module configured to input the image data to be processed into a preset estimation model to obtain target absolute depth image data, the estimation model being determined based on a plurality of first image data and a reference absolute depth image data corresponding to each first image data; a conversion module configured to perform relative depth conversion processing on the image data to be processed to obtain relative depth image data; a determination module configured to determine a target depth map corresponding to the image data to be processed according to the absolute depth image data and the relative depth image data.

9. An electronic device, comprising: The method comprises: a memory and a processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor performs the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method according to any one of claims 1-7. The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method according to any one of claims 1-7.

Images