Monocular depth estimation method and device for man-made main scene

Abstract

The invention discloses a monocular depth estimation method and a monocular depth estimation device using a human-made main scene, and relates to the technical field of image processing. The monocular depth estimation method comprises the following steps: firstly, respectively encoding an RGB image and a high-frequency component into a global image code and a high-frequency structure code; and cross attention calculation is performed on key values mapped by global image coding and high-frequency structure coding, and global features and high-frequency structure features in the image are quantized in the process to obtain richer image coding information and high-frequency structure information so as to clearly distinguish fine structures and complex backgrounds of a human body. Global semantic features and high-frequency structural features are obtained; and then an adaptive model AM is utilized to obtain dynamic weights of the global semantic features and the high-frequency structural features, the two features are fused based on the dynamic weights, and the fused features are used for guiding generation of a depth map, so that the precision of monocular depth estimation of the image is improved under the condition that all-dimensional image feature information is complete.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a monocular depth estimation method, apparatus, device, and medium for human-centric scenes. Background Technology

[0002] Monocular Depth Estimation (MDE) is a technique that infers the depth value of each pixel in a scene using only a single RGB image. Currently, commonly used image dataset generation methods typically employ diffusion models, a generative framework based on Markov chains. By defining a progressively noisy forward process and a learnable backward denoising process, it achieves a precise mapping from random noise to high-quality images. Utilizing cross-modal guiding conditions such as text, edge maps, and depth maps, the model can perform fine-grained control over the generated content, producing semantically consistent and detail-rich images. Currently, diffusion models have transcended the realm of pure image generation, demonstrating outstanding performance in intensive prediction tasks such as semantic segmentation, depth estimation, and video prediction.

[0003] After the dataset is generated and monocular depth estimation is performed, current monocular depth estimation methods mainly rely on deep learning, especially convolutional neural networks (CNN) and Transformer architectures, to learn the mapping relationship from RGB to depth from a large amount of labeled data.

[0004] Currently, when estimating monocular depth in human-centric scenes, existing deep learning models struggle to distinguish between subtle human structures and complex backgrounds. This causes the models to lose image feature information during recognition, resulting in low accuracy in monocular depth estimation. Summary of the Invention

[0005] This invention provides a monocular depth estimation method and apparatus for human-centric scenes, which can solve the problems existing in the prior art.

[0006] This invention provides a monocular depth estimation method for human-centric scenes, comprising the following steps: Acquire an RGB image of a scene with people as the main subject, and a preset depth map corresponding to the RGB image; An RGB image and a preset depth map are input into a pre-trained depth estimation model for processing to generate a predicted depth map. The depth estimation model includes a variational autoencoder, a feature enhancement and fusion module, and a variational autodecoder. The feature enhancement and fusion module is built based on the U-Net network architecture. The variational autoencoder encodes the RGB image to form an encoded image, and encodes the preset depth map and adds noise to obtain a noisy depth map latent variable; the encoded image and the noisy depth map latent variable are stacked to obtain a stacked image; The feature enhancement and fusion module performs feature enhancement and fusion on the stacked image and RGB image through the denoising network U-Net, predicts the noise information at the current time step, and based on the noise information at the current time step, uses the denoising network U-Net to continuously denoise the latent variables of the noisy depth map to obtain the denoised latent space vector. The latent space vector is restored to the predicted depth map through a variational autodecoder, and the depth data in the depth map is the monocular depth corresponding to the RGB image.

[0007] Preferably, the specific process by which the feature enhancement and fusion module obtains the denoised latent space vector includes: The feature enhancement and fusion module is built on the U-Net network architecture, and the text cross-attention mechanism between the convolutional layers of the U-Net network is replaced with the DCA-Adapter module; The DCA-Adapter module in the encoder of the U-Net denoising network in the feature enhancement and fusion module can use the Canny edge detection algorithm to extract the high-frequency components of the RGB image, and encode the RGB image and the high-frequency components respectively to obtain global image encoding and high-frequency structure encoding; it can extract features from the stacked image to obtain image features, and map the image features to query vectors; it can map the global image encoding and high-frequency structure encoding to key values ​​respectively, and perform cross-attention calculation on the query vector with the key values ​​mapped by the global image encoding and high-frequency structure encoding respectively to obtain global semantic features and high-frequency structure features respectively; Based on global image coding and high-frequency structure coding, and the current diffusion time step, the adaptive model AM is used to dynamically calculate the weights of global semantic features and high-frequency structure features to obtain dynamic global weights and dynamic high-frequency weights. Based on the dynamic global weights and dynamic high-frequency weights, the global semantic features and high-frequency structure features are weighted and fused to obtain fused features. The denoising network U-Net uses fusion features to guide the denoising process of noisy depth map latent variables to obtain the noise predicted at the current time step; and obtains the noisy depth map latent variables for the next time step based on the predicted noise and the noisy depth map latent variables. By continuously using the denoising network U-Net to denoise the latent variables of the noisy depth map, a completely denoised latent space vector is obtained.

[0008] Preferably, acquiring the stacked image includes: Using the variational autoencoder (VAE) within the pre-trained model Stable Diffusion v1.5, RGB images are processed. Encode the image to obtain the encoded image information. ; to generate a preset single-channel true depth map The image is copied into a three-channel image and then encoded using a VAE encoder to obtain the encoded depth information. ; For encoding depth The noise addition process is represented as follows: ; Among them: Gaussian noise ; Indicates cumulative noise scheduling; This represents the variance scheduling used in the diffusion process; Encoding depth Noise-added depth map latent variables are obtained after noise-addition processing. Encoded image information Latent variables of noisy depth maps The images are stacked to form a stacked image, which is then used as input to the feature enhancement fusion module.

[0009] Preferably, the acquisition of the global semantic features and high-frequency structural features includes: Extracting RGB images using the Canny edge detection algorithm high frequency components Using the CLIP image encoder, RGB images are combined with high-frequency components. Encode them separately as image codes and high-frequency coding ,in Contains global image information. Includes local texture and structural information; Feature extraction is performed on the stacked encoded image to obtain image features. Image features The data is mapped to a query vector, and cross-attention is calculated to obtain global semantic features and high-frequency structural features, represented as follows: ; ; in: For image features The query vector obtained through mapping; , , , They are encoded from images respectively. and high-frequency coding The resulting key-value matrix, output and These correspond to global semantic features and high-frequency structural features, respectively.

[0010] Preferably, the acquisition of the fusion feature includes: Using the adaptive model AM, and based on image encoding High-frequency coding The current time step of the pre-trained model StableDiffusion v1.5 The weights of global semantic features and high-frequency structural features are dynamically calculated to obtain dynamic global weights. and dynamic high-frequency weights ; The global semantic features and high-frequency structural features are weighted and fused, and represented as follows: ; in: This indicates the fusion feature.

[0011] Preferably, the training of the Stable Diffusion v1.5 model includes: A depth estimation dataset for training was generated using the open-source THuman 3.0 dataset; The RGB-D paired depth dataset was obtained by using Depth Anything V2 to predict a large number of real photos on the Internet. The difference between the depth estimation dataset and the RGB-D paired depth dataset is eliminated using affine invariant transformation, as follows: ; in: This represents the actual depth value. , These are the minimum and maximum values ​​of the depth map, respectively. The StableDiffusion v1.5 model was trained using the depth estimation dataset after eliminating the differences and the RGB-D paired depth dataset, respectively, to obtain the pre-trained model StableDiffusion v1.5.

[0012] Preferably, the RGB image is one image from an RGB image dataset, and the process of constructing the RGB image dataset includes: Using the large language model GPT-4o, based on preset diverse attributes covering gender, age, expression, action, clothing and background environment, text prompts describing people with broad semantic distribution and cross-combination of attributes are generated. Input the text prompts into the diffusion model SDXL to generate high-resolution RGB images of people in batches; The generated RGB images are input into a parallel hybrid image filter for quality screening. The filter consists of three binary classification models based on ViT, DINOv2, and CLIP. Only when all three models unanimously determine that the image is a high-quality image with normal human structure, the image is retained, and images with limb deformities or structural errors are removed, thus obtaining the RGB image dataset.

[0013] This invention also provides a monocular depth estimation device for human-centric scenes, comprising: The image generation module is used to acquire RGB images of a scene with people as the main subject, as well as a preset depth map corresponding to the RGB images; The processing module is used to input RGB images and preset depth maps into a pre-trained depth estimation model for processing to generate predicted depth maps; wherein, the depth estimation model includes a variational autoencoder, a feature enhancement and fusion module and a variational autodecoder, and the feature enhancement and fusion module is built based on the U-Net network architecture; The variational autoencoder encodes the RGB image to form an encoded image, and encodes the preset depth map and adds noise to obtain a noisy depth map latent variable; the encoded image and the noisy depth map latent variable are stacked to obtain a stacked image; The feature extraction and fusion module uses the U-Net denoising network to perform feature enhancement and fusion on the stacked image and the RGB image, predicts the noise information at the current time step, and uses the U-Net denoising network to continuously denoise the latent variables of the noisy depth map based on the noise information at the current time step, so as to obtain the denoised latent space vector. The depth estimation module reconstructs the predicted depth map from the latent space vector using a variational autodecoder. The depth data in the depth map is the monocular depth corresponding to the RGB image.

[0014] This invention also provides an electronic device, including a memory and a processor; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the monocular depth estimation method for a human-centric scene as described above.

[0015] This invention also provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the steps of a monocular depth estimation method for a human-centric scene as described above.

[0016] This invention provides a monocular depth estimation method and apparatus for human-centric scenes, which offers the following advantages compared to existing technologies: This invention first stacks the encoded RGB image and the noisy depth map latent variable at the current time step into a stacked image. The RGB image and its extracted high-frequency components are then encoded separately as global image encoding and high-frequency structure encoding. Features extracted from the stacked image are cross-attention calculated with the key values ​​mapped to the global image encoding and high-frequency structure encoding. This process quantizes the global and high-frequency structure features in the image to obtain richer image encoding and high-frequency structure information, clearly distinguishing fine human structures from complex backgrounds and obtaining global semantic features and high-frequency structure features. Then, an adaptive model AM is used to obtain the dynamic weights of the global semantic features and high-frequency structure features. Based on these dynamic weights, the global semantic features and high-frequency structure features are fused. Under a U-Net network architecture with a DCA-Adapter, the fused features and the RGB encoded image guide the denoising process of the noisy depth map latent variable in the stacked image, thereby improving the accuracy of monocular depth estimation while maintaining complete omnidirectional image feature information.

[0017] Furthermore, this invention utilizes GPT-4o to generate a large number of diverse character description prompts, allowing the prompts to vary randomly in terms of character gender, age, clothing, expression, and actions. Then, these generated prompts are used as input to SDXL to generate high-resolution character images in batches. Image filters are then used to remove deformed limb samples, retaining only structurally normal images. Finally, DCA-DepthNet is used to perform monocular depth inference on the filtered RGB images, generating one-to-one corresponding relative depth maps, thereby constructing a large-scale, highly diverse, and precisely paired RGB-D dataset. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall process of a method for generating a monocular depth estimation dataset in a human-centric scene, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the principle architecture of DCA-DepthNet, a monocular depth estimation method for human-centric scenes, provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0020] Currently, monocular depth estimation models based on neural networks have achieved excellent results in both indoor and outdoor scenes. Trained on a large number of labeled depth estimation datasets, monocular depth estimation models such as Depth Anything V2, Marigold, and MiDaS demonstrate outstanding generalization ability in most zero-shot scenarios. However, compared to the abundance of open-source datasets for indoor and outdoor non-human subjects, depth estimation datasets for human-centric scenes are extremely scarce, resulting in existing models not performing particularly well in depth estimation within human-centric scenarios.

[0021] Building and open-sourcing human-centric depth datasets presents a challenge far exceeding that of conventional scenarios, primarily in two aspects: First, the human body contains intricate structures such as hair strands and knuckles, making it difficult for consumer-grade depth sensors to capture complete and accurate depth information; second, publicly releasing high-resolution images containing faces and bodies easily infringes on portrait rights and privacy boundaries, thereby limiting the circulation and downstream applications of the dataset. Therefore, designing a depth-accurate, human-centric monocular depth estimation dataset generation method while ensuring privacy and security is currently a key focus.

[0022] Currently, the following problems remain to be solved in the generation of monocular depth estimation datasets for human-centric scenes: 1. Difficulty in obtaining high-quality RGB human images: Although some current image generation models can generate visually realistic human images, problems such as human body structure errors and limb deformities often occur during the generation process, making it difficult to guarantee the accuracy and consistency of the image in anatomical structure, thus limiting its application in high-precision vision tasks.

[0023] 2. High-quality depth maps are difficult to obtain: Existing monocular depth estimation methods perform poorly in complex scenes with humans as the main subject, and it is difficult to accurately estimate the relative depth relationships of different parts of the human body. Therefore, it is impossible to rely on existing models to build accurate and reliable RGB-D paired datasets, which affects the effectiveness of subsequent model training and evaluation.

[0024] Based on this, this invention mainly utilizes GPT-4o, SDXL, and the DCA-DepthNet developed in this invention as three core modules, which are cascaded sequentially to form a method for generating a monocular depth estimation dataset for a single-person scene. First, GPT-4o is used to generate a large number of diverse descriptive cues for the person, allowing these cues to vary randomly in terms of gender, age, clothing, expression, and actions. Next, these generated cues are used as input to SDXL to generate high-resolution person images in batches. Image filters are then used to remove deformed limb samples, retaining only images with normal structures. Finally, DCA-DepthNet is used to perform monocular depth inference on the filtered RGB images, generating one-to-one corresponding relative depth maps, thereby constructing a large-scale, highly diverse, and precisely paired RGB-D dataset. The flowchart is shown below. Figure 1 As shown. Specifically includes: I. Generation of character text descriptions.

[0025] To obtain a rich variety of human images in the final dataset, this invention uses GPT-4o as a text generation tool. By providing a human prompt word generation template to GPT-4o and requiring random modification of specific attributes such as human gender, age, expression, action, clothing, and background environment, text prompt words with wide semantic distribution and uniform attribute combination are finally obtained, which provides a foundation for the subsequent generation of high-quality and diverse RGB-D datasets.

[0026] II. RGB portrait image generation.

[0027] This invention uses SDXL as an RGB image generation tool to batch output high-resolution human images based on the aforementioned output text descriptions. Given that the randomness of SDXL can easily lead to defects such as limb deformities and structural omissions, this invention constructs a hybrid image filter to filter out these erroneous images. This hybrid image filter deploys a three-way binary classification quality discrimination model based on ViT, DINOv2, and CLIP in parallel. Only when all three classifiers unanimously classify the image as "high-quality" is the input image included in subsequent processes. This mechanism can scalably filter out low-quality samples, ensuring that the retained images all have correct skeletal proportions and complete limb structures, providing high-fidelity RGB input for subsequent depth annotation.

[0028] III. Generation of human body depth map.

[0029] This invention designs a depth estimation model, DCA-DepthNet, for human-centric scenes, as a depth labeler; by performing depth annotation on the generated RGB images, accurate paired relative depth value labels are obtained.

[0030] This invention is based on Stable Diffusion v1.5. By introducing image information and high-frequency image information, it modifies the functions of the original VAE decoder and encoder in SD, adjusts the U-Net structure, and adds the DCA-Adapter module designed in this invention. This transforms the original SD image generator into a monocular depth estimator. Its main structure is as follows: Figure 2 As shown.

[0031] To utilize SD for depth estimation tasks, this invention modifies the roles of the VAE decoder and encoder; for input RGB images... This invention directly utilizes the encoder of VAE. Encode it to obtain an RGB encoded image. For depth images First, it is copied and spliced ​​to convert it into a three-channel format to meet the input channel requirements of VAE. Then, the present invention uses the same VAE encoder to encode it to obtain the encoding depth. During the training phase, this invention performs noise addition during forward propagation according to the following formula: .

[0032] Among them: Gaussian noise ; For cumulative noise scheduling; Variance scheduling for the diffusion process; during the inference phase, the denoised latent space vectors... Through VAE decoder Restored to the predicted image .

[0033] To incorporate image information, this invention modifies the input structure of U-Net; to incorporate image conditional information, this invention modifies the number of input channels in the first convolutional layer of U-Net, changing the original 4 channels to 8 channels; during the forward propagation of U-Net, the encoded image information is... and noisy images Perform a stacking operation and use the result as input to the U-Net.

[0034] To incorporate more image and high-frequency information, this invention designs a DCA-Adapter module. Because the model inevitably loses image encoding and high-frequency image information during forward propagation, leading to a decrease in the accuracy of the final prediction result, this invention proposes a DCA-Adapter module to replace the original text cross-attention mechanism module in U-Net within SD. Specifically, for a given RGB image... This invention first utilizes the Canny edge detection algorithm to extract the high-frequency components of the image. Then, using the CLIP image encoder, they are encoded into image codes respectively. and high-frequency coding ,in Contains global image information. It contains local texture and structural information; then, if Representing the intermediate layer output features of U-Net, this invention applies the following two cross-attention mechanisms in parallel: .

[0035] .

[0036] in: For the reason The query obtained from the mapping; , , , They are respectively encoded by global encoding and high-frequency coding The resulting key-value matrix, output and These correspond to global semantic features and high-frequency structural detail features, respectively. To address the differences in high-frequency components across different images and to synchronously match the dynamic changes in the model's attention to each component during the reverse denoising process, this invention proposes an Adaptive Module (AM), the structure of which is as follows: Figure 2 As shown. This adaptive module can be based on image encoding. High-frequency coding and the current time step We obtain dynamically adjusted weights α and β, and the final cross-attention output can be expressed as: .

[0037] This invention replaces the denoising network U-Net with a DCA-Adapter module. The text cross-attention layer in the network. Therefore, the denoising network The final optimization objective can be expressed as: .

[0038] To enable the model to generalize to real-world scenarios, this invention employs a hybrid training strategy of "rendering-pseudo-labeling," which uses two training datasets: a rendering dataset and a pseudo-labeling dataset.

[0039] Training based on a rendering dataset: This invention first uses the open-source THuman3.0 dataset to generate a depth estimation dataset for training; this dataset has the ability to obtain accurate depth information, but due to its differences from real scenes, such as no background and unrealistic lighting systems, models trained only on this dataset cannot achieve excellent performance in real scenes.

[0040] Training on a pseudo-labeled dataset: To enable the model to generalize to real-world scenarios, this invention also trains the model on a pseudo-labeled dataset. This dataset is an RGB-D paired depth dataset obtained by predicting a large number of real photos on the Internet using Depth Anything V2. Although the depth values ​​in this dataset are relatively inaccurate, they do not have domain differences from real-world scenarios, allowing the model to generalize well to real-world scenes.

[0041] This invention utilizes the aforementioned rendering dataset and pseudo-label dataset for mixed training, employing a dataset size ratio of 2:1. To eliminate scale and displacement differences between different datasets, this invention employs the following affine invariant transformation: .

[0042] in: This represents the actual depth value. , These are the minimum and maximum values ​​of the depth map, respectively. Using this transformation method, the depth map can also be transformed to... The range of values ​​is determined to meet the input range requirements of the VAE encoder.

[0043] This invention addresses the current lack of depth estimation datasets for human-centric scenes. For the first time, it constructs a method for generating such datasets based on GPT-4o, Stable Diffusion XL, and its self-developed DCA-DepthNet. Building upon the existing Stable Diffusion architecture, DCA-DepthNet modifies the U-Net input structure and introduces a DCA-Adapter module, enabling the model to acquire richer image encoding and high-frequency structural information. The proposed DCA-Adapter module effectively and dynamically adjusts the weight ratios used by different image encodings during the reverse denoising process, achieving better depth estimation results. The DCA-DepthNet designed in this invention is more suitable for monocular depth estimation of human subjects, and the designed dataset generation method can effectively generate a large number of high-quality depth datasets.

[0044] The proposed method for constructing deep datasets for human-centric scenarios can generate large-scale synthetic data with high fidelity and pixel-level depth annotations in one batch. This directly fills the data gap in this field and significantly reduces the data acquisition and annotation costs for subsequent researchers in human pose estimation, behavior recognition, virtual reality, and metaverse-related fields. It also provides scalable and reproducible benchmark data support for these fields.

[0045] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A monocular depth estimation method for human-centric scenes, characterized in that, Includes the following steps: Acquire an RGB image of a scene with people as the main subject, and a preset depth map corresponding to the RGB image; An RGB image and a preset depth map are input into a pre-trained depth estimation model for processing to generate a predicted depth map. The depth estimation model includes a variational autoencoder, a feature enhancement and fusion module, and a variational autodecoder. The feature enhancement and fusion module is built based on the U-Net network architecture. The variational autoencoder encodes the RGB image to form an encoded image, and encodes the preset depth map and adds noise to obtain a noisy depth map latent variable; the encoded image and the noisy depth map latent variable are stacked to obtain a stacked image; The feature enhancement and fusion module performs feature enhancement and fusion on the stacked image and RGB image through the denoising network U-Net, predicts the noise information at the current time step, and based on the noise information at the current time step, uses the denoising network U-Net to continuously denoise the latent variables of the noisy depth map to obtain the denoised latent space vector. The latent space vector is restored to the predicted depth map through a variational autodecoder, and the depth data in the depth map is the monocular depth corresponding to the RGB image.

2. The monocular depth estimation method for a human-centric scene according to claim 1, characterized in that, The specific process by which the feature enhancement and fusion module obtains the denoised latent space vector includes: The feature enhancement and fusion module is built on the U-Net network architecture, and the text cross-attention mechanism between the convolutional layers of the U-Net network is replaced with the DCA-Adapter module; The DCA-Adapter module in the encoder of the U-Net denoising network in the feature enhancement and fusion module can use the Canny edge detection algorithm to extract the high-frequency components of the RGB image, and encode the RGB image and the high-frequency components respectively to obtain global image encoding and high-frequency structure encoding; it can extract features from the stacked image to obtain image features, and map the image features to query vectors; it can map the global image encoding and high-frequency structure encoding to key values ​​respectively, and perform cross-attention calculation on the query vector with the key values ​​mapped by the global image encoding and high-frequency structure encoding respectively to obtain global semantic features and high-frequency structure features respectively; Based on global image coding and high-frequency structure coding, and the current diffusion time step, the adaptive model AM is used to dynamically calculate the weights of global semantic features and high-frequency structure features to obtain dynamic global weights and dynamic high-frequency weights. Based on the dynamic global weights and dynamic high-frequency weights, the global semantic features and high-frequency structure features are weighted and fused to obtain fused features. The denoising network U-Net uses fusion features to guide the denoising process of noisy depth map latent variables to obtain the noise predicted at the current time step; and obtains the noisy depth map latent variables for the next time step based on the predicted noise and the noisy depth map latent variables. By continuously using the denoising network U-Net to denoise the latent variables of the noisy depth map, a completely denoised latent space vector is obtained.

3. The monocular depth estimation method for a human-centric scene according to claim 2, characterized in that, The acquisition of the stacked image includes: Using the variational autoencoder (VAE) within the pre-trained model Stable Diffusion v1.5, RGB images are processed. Encode the image to obtain the encoded image information. ; to generate a preset single-channel true depth map The image is copied into a three-channel image and then encoded using a VAE encoder to obtain the encoded depth information. ; For encoding depth The noise addition process is represented as follows: ； Among them: Gaussian noise ; Indicates cumulative noise scheduling; This represents the variance scheduling used in the diffusion process; Encoding depth Noise-added depth map latent variables are obtained after noise-addition processing. Encoded image information Latent variables of noisy depth maps The images are stacked to form a stacked image, which is then used as input to the feature enhancement fusion module.

4. The monocular depth estimation method for a human-centric scene according to claim 3, characterized in that, The acquisition of the global semantic features and high-frequency structural features includes: Extracting RGB images using the Canny edge detection algorithm high frequency components Using the CLIP image encoder, RGB images are combined with high-frequency components. Encode them separately as image codes and high-frequency coding ,in Contains global image information. Includes local texture and structural information; Feature extraction is performed on the stacked encoded image to obtain image features. Image features The data is mapped to a query vector, and cross-attention is calculated to obtain global semantic features and high-frequency structural features, represented as follows: ； ； in: For image features The query vector obtained through mapping; , , , They are encoded from images respectively. and high-frequency coding The resulting key-value matrix, output and These correspond to global semantic features and high-frequency structural features, respectively.

5. The monocular depth estimation method for a human-centric scene according to claim 4, characterized in that, The acquisition of the fusion features includes: Using the adaptive model AM, and based on image encoding High-frequency coding The current time step of the pre-trained model Stable Diffusionv1.5 The weights of global semantic features and high-frequency structural features are dynamically calculated to obtain dynamic global weights. and dynamic high-frequency weights ; The global semantic features and high-frequency structural features are weighted and fused, and represented as follows: ； in: This indicates the fusion feature.

6. The monocular depth estimation method for a human-centric scene according to claim 3, characterized in that, The training of the Stable Diffusion v1.5 model includes: A depth estimation dataset for training was generated using the open-source THuman 3.0 dataset; The RGB-D paired depth dataset was obtained by using Depth Anything V2 to predict a large number of real photos on the Internet. The difference between the depth estimation dataset and the RGB-D paired depth dataset is eliminated using affine invariant transformation, as follows: ； in: This represents the actual depth value. , These are the minimum and maximum values ​​of the depth map, respectively. The StableDiffusion v1.5 model was trained using the depth estimation dataset after eliminating the differences and the RGB-D paired depth dataset, respectively, to obtain the pre-trained model StableDiffusion v1.

5.

7. The monocular depth estimation method for a human-centric scene according to claim 1, characterized in that, The RGB image is one of the images in the RGB image dataset, and the process of constructing the RGB image dataset includes: Using the large language model GPT-4o, based on preset diverse attributes covering gender, age, expression, action, clothing and background environment, text prompts describing people with broad semantic distribution and cross-combination of attributes are generated. Input the text prompts into the diffusion model SDXL to generate high-resolution RGB images of people in batches; The generated RGB images are input into a parallel hybrid image filter for quality screening. The filter consists of three binary classification models based on ViT, DINOv2, and CLIP. Only when all three models unanimously determine that the image is a high-quality image with normal human structure, the image is retained, and images with limb deformities or structural errors are removed, thus obtaining the RGB image dataset.

8. A monocular depth estimation device for human-centric scenes, characterized in that, include: The image generation module is used to acquire RGB images of a scene with people as the main subject, as well as a preset depth map corresponding to the RGB images; The processing module is used to input RGB images and preset depth maps into a pre-trained depth estimation model for processing to generate predicted depth maps; wherein, the depth estimation model includes a variational autoencoder, a feature enhancement and fusion module and a variational autodecoder, and the feature enhancement and fusion module is built based on the U-Net network architecture; The variational autoencoder encodes the RGB image to form an encoded image, and encodes the preset depth map and adds noise to obtain a noisy depth map latent variable; the encoded image and the noisy depth map latent variable are stacked to obtain a stacked image; The feature extraction and fusion module uses the U-Net denoising network to perform feature enhancement and fusion on the stacked image and the RGB image, predicts the noise information at the current time step, and uses the U-Net denoising network to continuously denoise the latent variables of the noisy depth map based on the noise information at the current time step, so as to obtain the denoised latent space vector. The depth estimation module reconstructs the predicted depth map from the latent space vector using a variational autodecoder. The depth data in the depth map is the monocular depth corresponding to the RGB image.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the monocular depth estimation method for a human-centric scene as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the steps of a monocular depth estimation method for a human-centric scene as described in any one of claims 1 to 7.

Images