A robust monocular depth estimation method based on physical perception decoupled distillation and related devices

Abstract

The application discloses a robust monocular depth estimation method based on physical perception decoupling distillation and related equipment, and the robust monocular depth estimation method based on physical perception decoupling distillation learns geometric preserving representation in a night, a rainy day and other bad imaging scenes by constructing a training relationship of a clear image in a day, a bad observation image and a physical component, and trains a student network by using physical perception distillation and illumination perception sequential constraint, so that the student network has the performance of robust monocular depth estimation; since the degradation interference factor is explicitly separated before distillation, the robust monocular depth estimation method based on physical perception decoupling distillation can reduce the pollution of non-geometric pseudo-cues to depth supervision, and the online generation module enables the synthesized sample to adapt to the downstream geometric learning target, and has better robustness and interpretability. The application is applied to the field of computer vision and three-dimensional perception technology.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and 3D perception technology, and in particular to a robust monocular depth estimation method and related equipment based on physical perception decoupled distillation. Background Technology

[0002] Monocular depth estimation (MDE) is a technique for predicting scene distance information from a single image, offering advantages such as low cost, easy deployment, and applicability to autonomous driving and robot navigation. This technology has been widely applied in 3D scene understanding, intelligent driving perception, augmented reality, and mobile robotics. However, under adverse conditions such as nighttime or rain, illumination attenuation, atmospheric scattering, rain streaks, sensor noise, and motion blur can disrupt geometric cues in the image, leading to unstable predictions from the depth estimation model. Therefore, designing monocular depth estimation algorithms with high reliability even under harsh imaging environments has become an important research focus in this field. Summary of the Invention

[0003] To address at least one of the aforementioned technical problems, the present invention aims to provide a robust monocular depth estimation method and related equipment based on physical perception decoupled distillation.

[0004] On one hand, embodiments of the present invention include a robust monocular depth estimation method based on physically-aware decoupled distillation, wherein the robust monocular depth estimation method based on physically-aware decoupled distillation includes: Online generation step: Acquire clear daytime images and adverse target imaging conditions, and generate adverse observation images based on the clear daytime images and adverse target imaging conditions; Physical decomposition step: Perform physical decomposition on the poorly viewed image and the clear daytime image to obtain multiple physical component parameters, and obtain a geometry-preserving representation based on the physical component parameters; Distillation step: Perform at least one iterative process to obtain a robust monocular depth estimation model from the student network after all the iterative processes have been performed; in any of the iterative processes, call the online generation step and the physical decomposition step to generate the geometry-preserving representation, and use the geometry-preserving representation to train the student network.

[0005] Further, generating a harsh observation image based on the clear daytime image and the harsh imaging conditions of the target includes: Obtain the online conditional MeanFlow generation module; The clear daytime image and the adverse imaging conditions of the target are input into the online conditional MeanFlow generation module for processing; Obtain the adverse observation image output by the online conditional MeanFlow generation module.

[0006] Furthermore, the physical decomposition of the poorly viewed image and the clear daytime image to obtain multiple physical component parameters includes: According to the formula

[0007]

[0008] Decompose; among them, For the aforementioned poor observation image, The reflection component of the poorly observed image. The illumination component of the poorly observed image, The transmission component of the poorly observed image. The rain streak component of the poorly observed image. For the daytime clear image, The reflection component of the clear daytime image. The illumination component of the daytime clear image.

[0009] Further, obtaining the geometry-preserving representation based on the physical component parameters includes: The reflection component of the poorly observed image and the illumination component of the poorly observed image This serves as the geometry-preserving representation.

[0010] Further, training the student network using the geometry-preserving representation includes: Establish a teacher network; The geometry-preserving representation is input into the student network to obtain the predicted depth output by the student network; The clear daytime image is input into the teacher network, and the standard depth ground value output by the teacher network is obtained. Determine the depth distillation loss value based on the predicted depth and the true value of the standard depth; The total loss value is determined based on the depth distillation loss value. The student network is trained based on the total loss value.

[0011] Further, determining the total loss value based on the depth distillation loss value includes: By analyzing the reflection component of the poorly observed image Apply reflection consistency constraints to the illumination components of the poorly viewed image. Apply heavy illumination constraint to the transmitted component Apply spatial smoothing and range constraints to the rain streak components. Applying high-frequency sparse constraints to obtain the reconstructed image According to the reconstructed image and the aforementioned poor observation images Determine the physical decomposition and reconstruction loss value; The depth difference between the teacher network and the student network is obtained. Based on the depth difference, uncertain regions and relatively stable regions in the adverse observation image are determined. Pixel pairs are sampled from the uncertain regions and the relatively stable regions. A relative depth order label is generated based on the pixel pairs using a frozen base depth model. A physical confidence map is constructed based on the illumination component gradient of the pixel pairs. A weighted order loss value is determined based on the order label and the physical confidence map. Based on the generation velocity field and target condition consistency of the online conditional MeanFlow generation module, the generation consistency loss is determined; The total loss value is determined based on the deep distillation loss value, the physical decomposition and reconstruction loss value, the weighted order loss value, and the generation consistency loss value.

[0012] Further, determining the total loss value based on the deep distillation loss value, the physical decomposition and reconstruction loss value, the weighted order loss value, and the generation consistency loss includes: Set the first weight, second weight, and third weight; According to the formula

[0013] Perform calculations; among which, The total loss value, The depth distillation loss value, The physical decomposition and reconstruction loss value, The weighted order loss value, For the generation of consistency loss, For the first weight, This is the second weight. This is the third weight.

[0014] Furthermore, the robust monocular depth estimation method based on physical perception decoupled distillation also includes: Obtain the image to be estimated; The image to be estimated is input into the robust monocular depth estimation model for processing; Obtain the depth estimate output by the robust monocular depth estimation model.

[0015] On the other hand, embodiments of the present invention also include a computer device, including a memory and a processor, the memory for storing at least one program, and the processor for loading at least one program to execute the robust monocular depth estimation method based on physical perception decoupled distillation in the embodiments.

[0016] On the other hand, embodiments of the present invention also include a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the robust monocular depth estimation method based on physical perception decoupled distillation in the embodiments.

[0017] The beneficial effects of this invention are as follows: The robust monocular depth estimation method based on physical perception decoupled distillation in the embodiments learns geometrically preserved representations for harsh imaging scenarios such as nighttime and rainy days by constructing training relationships between clear daytime images, poorly observed images, and physical components. It also trains a student network using physical perception distillation and lighting perception order constraints, thereby enabling the student network to have robust monocular depth estimation performance. Since degradation interference factors are explicitly separated before distillation, the robust monocular depth estimation method based on physical perception decoupled distillation in the embodiments can reduce the contamination of depth supervision by non-geometric pseudo-cues. Simultaneously, the online generation module enables synthetic samples to adapt to downstream geometric learning objectives, resulting in better robustness and interpretability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall process of physical perception decoupling distillation in the embodiment; Figure 2 This is a flowchart illustrating the physical guidance decomposition and reconstruction module in the embodiment. Figure 3 This is a schematic diagram of the lighting-sensing sequential distillation process in the embodiment; Figure 4 This is a schematic diagram illustrating the steps of the robust monocular depth estimation method based on physical perception decoupled distillation in the embodiment. Detailed Implementation

[0019] Currently, monocular depth estimation algorithms can be developed in two directions: 1) supervised learning-based depth estimation, which trains the model using a large amount of data with dense depth annotations; and 2) self-supervised learning-based depth estimation, which trains the model using photometric and geometric consistency between adjacent frames or binocular images. However, both methods have limitations in real-world nighttime or rainy scenes. Supervised learning methods require expensive and difficult-to-obtain depth annotations for harsh scenes, while self-supervised methods rely on photometric consistency and rigid scene assumptions that are easily disrupted by low light, reflections, and rain streaks. Especially in autonomous driving scenarios, depth errors further affect drivable area judgment, obstacle distance estimation, and path planning safety. In mobile robot scenarios, the lack of local structure caused by low light and rain also affects obstacle avoidance, localization, and map building. Therefore, robust monocular depth estimation not only requires the model to improve on the mean error index but also to remain stable in target boundaries, road structures, distance relationships, and low-reliability texture regions.

[0020] Meanwhile, under harsh conditions, changes in image appearance are not simply style transfer, but rather image formation variations caused by a combination of physical factors such as illumination, transmission attenuation, and rain streaks. Traditional methods typically train or distill directly on degraded images, failing to distinguish between real geometric cues and degradation interference factors, potentially leading student models to learn pseudo-related features unrelated to depth. Therefore, for the needs of monocular images, decoupling degradation factors from physical perception is essential.

[0021] Acquiring and labeling data for adverse scenes is costly, and image distribution varies significantly under different weather, lighting, and road conditions. In recent years, generative distillation methods have reduced the need for adverse scene labeling by generating adverse scene samples and transferring deep knowledge from clear domain teacher models. However, diffusion generation is typically performed offline or as a frozen preprocessing step, resulting in high computational costs, and the generated results cannot be dynamically adjusted for downstream depth estimation tasks. On the other hand, while basic depth models provide strong relative geometric priors, their outputs are not trained for specific adverse conditions. If directly used as pixel-level absolute depth supervision, they may still mislead users in areas of local overexposure, shadows, rain streaks, etc. Therefore, it is necessary to combine the relative order information provided by basic depth models with physical reliability estimation, rather than applying supervision indiscriminately.

[0022] Current technologies utilize domain adaptation, image enhancement, diffusion generation, or basic deep models to improve robustness, but they still have two shortcomings: First, the generation of poorly generated samples is independent of the deep learning objective, and the generated images may not be the most favorable training samples for geometric learning; second, non-geometric factors such as illumination, transmission, and rain streaks in degraded images are not explicitly separated before distillation, which makes the clear domain teacher supervision easily contaminated in severely degraded scenes.

[0023] As can be seen from the above analysis, the robust monocular depth estimation methods used in current monocular depth estimation algorithms mainly have the following drawbacks: 1. Offline or frozen generative poor samples cannot be dynamically optimized with downstream depth estimation targets, resulting in high computational costs; Second, the coupling of degradation interference factors with geometric structure cues makes direct distillation easy for students to learn non-geometric pseudo-cues such as rain patterns, reflections, and lighting abrupt changes in the network.

[0024] Example 1 Based on the aforementioned technical problems and their underlying principles, specifically to address the coupling of degradation factors and geometric cues in monocular depth estimation under adverse conditions, this embodiment combines online adverse scene generation, a physical image formation model, and knowledge distillation for modeling. It proposes a robust monocular depth estimation method based on physically-perceived decoupling distillation. This method employs adaptive sample generation and geometry-preserving representation learning for imaging environments such as nighttime and rainy days, reducing noise from factors like illumination, transmission, and rain streaks in depth estimation. Compared to traditional robust depth estimation methods, this embodiment improves depth prediction stability without dense depth annotations in adverse scenes due to the introduction of a physical decomposition module and illumination-perceived sequential distillation, while also offering better physical interpretability. This embodiment does not simply treat nighttime or rainy images as color style changes, but rather as an image formation process jointly determined by reflection, illumination, transmission, and rain streaks. By separating these factors during training, the model can concentrate the structural information used for depth inference into a geometry-preserving representation composed of reflection and illumination, while retaining transmission attenuation and rain streak interference as constraints in the reconstruction branch, thus reducing the possibility of spurious learning from the training mechanism.

[0025] To achieve the above objectives, such as Figure 1 As shown, the specific implementation method of the robust monocular depth estimation method based on physical perception decoupled distillation in this embodiment is as follows: I. Data Preparation and Analysis This embodiment involves a deep learning algorithm for joint training and learning of (clear daytime images, poorly observed images, and physical components). It requires collecting and constructing relevant data pairs. Specifically: First, the vision scenarios for autonomous driving or robots are organized and image data are collected in the following categories: 1) clear daytime scenes, 2) low-light nighttime scenes, and 3) adverse scenes such as rainy days or rainy nights. Then, corresponding training samples are generated based on the target adverse conditions for reference.

[0026] Secondly, considering the impact of different adverse conditions on image formation, we analyze illumination attenuation, transmission variation, and high-frequency interference from rain patterns, and provide corresponding constraints on geometry-preserving components and degradation interference components.

[0027] II. Design and Training of Online Conditional MeanFlow Generation Module In this embodiment, the online generation module processes both clear daytime images and images under various adverse conditions. Traditional diffusion-based generation methods often rely on offline pre-generated samples, requiring the module to have higher training synergy to handle diverse adverse scenarios. Therefore, this embodiment uses a conditional MeanFlow network model to quickly generate images of adverse conditions such as nighttime and rainy days based on target conditions. To enhance the consistency between this module and downstream deep learning objectives, this embodiment jointly updates the generation module during the overall model training process, allowing it to learn a more suitable distribution of adverse samples for geometric learning from prior data and deep supervision. Specifically, the conditional MeanFlow generation module receives target condition encoding, which characterizes different degradation types such as nighttime, rainy days, and nighttime rainy days. The adverse observation images output by the generation module maintain a consistent scene layout with the clear daytime images, enabling subsequent teacher deep supervision to transfer within the same geometric structure. Because the generation module participates in the backpropagation of the total loss, its output does not remain at the static preprocessing result but can be gradually adjusted according to the current error region of the student network.

[0028] III. Design and Training of the Physical Sensing Decoupling Module In this embodiment, the physical perception decoupling module needs to estimate four components—reflection, illumination, transmission, and rain streaks—for the generated adverse observation images. On one hand, based on the physical properties of the adverse observation images, it first constructs the intrinsic relationship between the same scene structure and different degradation factors based on the relationship between (clear daytime images, adverse observation images, and physical components). To this end, this embodiment utilizes, for example... Figure 2 The decomposition network shown receives a poorly observed image as input and compares the reconstructed image with a clear image to learn the correlation between the two from the data. On the other hand, this physical component learning lacks a gold standard. This embodiment utilizes self-supervised learning techniques to construct reconstruction loss and consistency loss for supervising the use of signals in the decomposition network, thereby pre-training on data pairs to obtain a decoupled system with separable geometry-preserving components. Specifically, the reflection illumination decomposition network is used to extract relatively stable structural textures and brightness distributions, the transmission estimation network is used to describe spatial variations caused by rain, fog, or low visibility, and the rain ripple estimation network is used to characterize elongated, high-frequency, locally sparse occlusion interference. The components are not optimized independently but are jointly learned under constraints of image reconstruction, reflection consistency, illumination smoothness, and depth distillation, thereby avoiding degradation caused by a single component absorbing all image information.

[0029] IV. Establishing an iterative training process for decoupled distillation and sequence constraints By designing and training an online conditional MeanFlow generation module and a physics-aware decoupling module, sub-modules are obtained to perform harsh generation, physical decomposition, and depth distillation in each iteration. In the overall iterative process design, this embodiment first introduces prior knowledge of the physical decomposition operator, including transmission range constraints, transmission smoothness constraints, and rain ripple high-frequency sparsity constraints. Then, an updated joint training model is obtained, and a clear domain teacher network is used to obtain pixel-level depth distillation signals. Furthermore, the basic depth model and illumination confidence map are used as sequence constraints, resulting in the following... Figure 3 The complete physical perception decoupled distillation training process is shown. In illumination-aware sequential distillation, this embodiment first determines structurally uncertain regions based on the depth differences between the teacher and student networks. Then, pixel pairs are sampled from the uncertain and relatively stable regions, and relative depth order labels are generated through a frozen base depth model. Subsequently, a physical confidence map is constructed based on the illumination component gradient. The order loss weights are reduced at abrupt illumination changes, overexposure, or shadow edges, while the order constraints are strengthened in regions with stable illumination and visible structure. This design allows sequential distillation to focus on difficult regions while avoiding the introduction of additional noise in areas where physical observations are unreliable.

[0030] Reference Figure 4 The robust monocular depth estimation method based on physical perception decoupled distillation includes the following steps: S1. Online generation step: Acquire clear daytime images and poor target imaging conditions, and generate poor observation images based on the clear daytime images and poor target imaging conditions; S2. Physical decomposition step: Physical decomposition is performed on the poorly observed image and the clear daytime image to obtain multiple physical component parameters, and a geometrically preserved representation is obtained based on the physical component parameters; S3. Distillation step: Execute at least one iteration process to use the student network after all iterations as a robust monocular depth estimation model; in any iteration process, call the online generation step and the physical decomposition step to generate a geometry-preserving representation, and use the geometry-preserving representation to train the student network.

[0031] Specifically, in step S3, the distillation step, each iteration calls steps S1-S2 (online generation and physical decomposition steps) to generate the geometry-preserving representation corresponding to that iteration. This geometry-preserving representation is then used to train the student network. When executing step S3, if the training termination condition is met after completing an iteration (e.g., the total loss function value is less than the loss threshold or convergence), the iteration process is stopped, and the student network is trained, becoming a robust monocular depth estimation model. If the training termination condition is not met (e.g., the total loss function value is greater than the loss threshold or convergence), the next iteration is executed, i.e., steps S1-S2 are re-executed to acquire new clear daytime images and handle poor target imaging conditions.

[0032] In this embodiment, one of the iterative processes during step S3 is used as an example for illustration.

[0033] In one iteration, step S1, the online generation step, is invoked to acquire clear daytime images. and adverse imaging conditions of the target Among them, clear daytime images Images can be taken under clear daytime conditions (without interference from adverse imaging conditions such as rain or fog), or under adverse imaging conditions on the target. Specifically, it can represent adverse imaging conditions such as rain, fog, and nighttime.

[0034] In step S1, the clear daytime image is... and adverse imaging conditions of the target The input is fed into the online conditional MeanFlow generation module for processing. The online conditional MeanFlow generation module performs...

[0035] The processing generates poorly viewed images. ,in This indicates the processing of the online condition MeanFlow generation module.

[0036] By performing step S1, a clear daytime image of the content was generated. The same poor observation images Moreover, the observation images are poor. It was taken under adverse imaging conditions such as rain, fog, and night.

[0037] In step S2, a clear daytime image is constructed. Poor observation images Based on the relationship between the training data and the physical components, a reflection illumination decomposition network, a transmission estimation network, and a rain ripple estimation network are designed to estimate the geometry-preserving component and the degradation interference component, respectively. To address the problem of coupling between adverse imaging conditions and scene geometry, adverse observation images are processed... It is represented as a combination of reflection, illumination, transmission, and rain streaks, and a clear daytime image is represented as a combination of reflection and illumination, used to constrain the consistency of the same scene structure.

[0038] Specifically, in step S2, according to the formula

[0039] For poorly viewed images Decomposition is performed to obtain the reflection component of the poorly observed image. Illumination component of poorly viewed images Transmission component of poorly observed images Rain streak component in poorly observed images .

[0040] Specifically, in step S2, according to the formula

[0041] Clear images during the day The image is decomposed to obtain the reflectance component of the clear daytime image. Illumination component of clear daytime images .

[0042] In step S2, refer to Figure 2 For the reflection component of the poorly observed image obtained by decomposition Illumination component of poorly viewed images Transmission component of poorly observed images Rain streak component in poorly observed images Reflection component of clear daytime images Illumination component of clear daytime images Multiple physical component parameters, such as the reflection component of the poorly observed image. Illumination component of poorly viewed images As a geometry-preserving representation { , }

[0043] In step S3, the geometry-preserving representation is obtained by calling and executing steps S1-S2. , After that, when performing the step of training the student network using geometry-preserving representations, the following steps can be performed: S301. Establish a teacher network; S302. Input the geometry-preserving representation into the student network and obtain the predicted depth of the student network output; S303. Input the clear daytime image into the teacher network and obtain the standard depth ground value output by the teacher network; S304. Determine the depth distillation loss value based on the predicted depth and the true value of the standard depth; S305. Determine the total loss value based on the deep distillation loss value; S306. Train the student network based on the total loss value.

[0044] In step S301, a teacher network that has been pre-trained and whose parameters have been frozen is established.

[0045] In step S302, the geometry-preserving representation { , The input is fed into the student network for processing, and the predicted depth output by the student network is obtained. .

[0046] In step S303, the clear daytime image is... The input is fed into the teacher network for processing, and the standard depth truth value output by the teacher network is obtained. .

[0047] In step S304, the predicted depth is calculated. and standard depth truth The pixel depth error is used to determine the depth distillation loss value. This completes the knowledge distillation process.

[0048] In step S305, to address the lack of a pixel-level gold standard for degradation factors, a self-supervised reconstruction loss is constructed in conjunction with an image formation model. Spatial smoothing and range constraints are applied to the transmission component, and high-frequency sparsity constraints are applied to the rain streak component. Decomposition degradation is prevented through reflection consistency and reillumination constraints. Specifically, this is achieved by adjusting the reflection component of the poorly observed image. Applying reflection uniformity constraints and illumination components to poorly viewed images. Apply heavy illumination constraints and control the transmitted components Apply spatial smoothing and range constraints to rain streak components Applying high-frequency sparse constraints to obtain the reconstructed image According to the reconstructed image and poor observation images Determine the physical decomposition and reconstruction loss value .

[0049] In step S305, refer to Figure 3 To obtain the depth differences between teacher and student networks, and based on these depth differences, to identify adverse observation images. The algorithm identifies uncertain and relatively stable regions within the image. Pixel pairs are sampled from these regions. A frozen baseline depth model is used to generate relative depth order labels based on the pixel pairs. A physical confidence map is constructed based on the illumination component gradients of the pixel pairs (high weight for stable bright areas and low weight for shadow / overexposure abrupt changes). The weighted order loss value is determined based on the order labels and the physical confidence map. .

[0050] In step S305, to address the issue that offline diffusion generation is difficult to optimize with downstream tasks, training samples are generated using the one-step generation capability of the online conditional MeanFlow generation module. Based on the generation speed field and target condition consistency of the online conditional MeanFlow generation module, a consistency loss is generated. Utilizing consistency loss Constraining the generation velocity field to be consistent with the target conditions allows for dynamic updates of adverse samples as the depth estimation target is adjusted.

[0051] In step S305, for the already determined depth distillation loss value Physical decomposition and reconstruction loss value Weighted order loss value and generating consistency loss Set the first weight Second weight and third weight Specifically, the first weight Second weight and third weight All are hyperparameters, with the first weight. The second weight represents the weight of the physical decomposition self-supervised loss in the total loss. This indicates adjusting the relative depth order constraint loss weights, the third weight. This represents the constraint loss weights used to adjust the online degraded image generation network. According to the formula...

[0052] The calculated value is the total loss. .

[0053] In step S3, during one iteration, if the total loss value obtained from the iteration process is... If the loss value is less than the loss threshold or convergence occurs, then the training termination condition is met; otherwise, the training termination condition is not met, and the training termination condition is determined based on the total loss value. Update parameters for student networks, etc. Specifically, this can be done based on the total loss value. The gradient is calculated to update the trainable parameters of the online conditional MeanFlow generation module, student network, reflection estimation network, illumination estimation network, transmission estimation network, and rain ripple estimation network, while the teacher network's parameters are frozen and not updated.

[0054] In this embodiment, a trained student network can be used as a robust monocular depth estimation model. When an image that needs to be estimated for monocular depth is obtained, the image to be estimated can be input into the robust monocular depth estimation model for processing, and the depth estimate output by the robust monocular depth estimation model can be obtained, thereby realizing robust monocular depth estimation based on physical perception decoupling distillation.

[0055] Example 2 According to another embodiment of this example, a robust monocular depth estimation system based on physical perception decoupled distillation is provided, comprising: The online scene generation module is used to generate adverse observation images based on clear daytime images and adverse target conditions; The Physical Decomposition and Reconstruction module is used to recommend the reflection and illumination components that best preserve the scene geometry and suppress transmission and rain streaks. The distillation and depth prediction unit is used to complete the training process of physical perception depth distillation and illumination perception sequential distillation to obtain depth estimation results that can maximize the geometric quality of the current poor image.

[0056] To address the coupling issue between degradation factors and geometric cues in monocular depth estimation under adverse conditions, this embodiment combines online adverse scene generation, a physical image formation model, and knowledge distillation for modeling. A novel robust monocular depth estimation method based on physically-perceived decoupled distillation is proposed. This method employs adaptive sample generation and geometry-preserving representation learning for imaging environments such as nighttime and rainy days, reducing noise introduced by factors like illumination, transmission, and rain streaks in depth estimation. Compared to traditional robust depth estimation methods, this invention improves depth prediction stability even without dense depth annotations in adverse scenes due to the introduction of a physical decomposition module and illumination-perceived sequential distillation, while also offering better physical interpretability. This system can be deployed as a training phase system and an inference phase system. During the training phase, the system includes online scene generation, physical decomposition, teacher distillation, and sequential constraint modules. During the inference phase, only the student depth estimation network and necessary preprocessing modules are retained, directly outputting depth maps from the input nighttime or rainy monocular images, thus meeting the computational efficiency requirements of real-time perception applications.

[0057] To achieve the above objectives, the specific implementation method of this embodiment is as follows: I. Data Preparation and Analysis This embodiment involves a deep learning algorithm for joint training and learning of (clear daytime images, poorly observed images, and physical components). It requires collecting and constructing relevant data pairs. Specifically: First, the vision scenarios for autonomous driving or robots are organized, and image data are collected in the following categories: 1) clear daytime scenes, 2) low-light nighttime scenes, and 3) adverse scenes such as rainy days or rainy nights. Then, corresponding training samples are generated based on the target adverse conditions for reference.

[0058] Secondly, considering the impact of different adverse conditions on image formation, we analyze illumination attenuation, transmission variation, and high-frequency interference from rain patterns, and provide corresponding constraints on geometry-preserving components and degradation interference components.

[0059] Design and training of online conditional MeanFlow generation module The online generation module in this invention processes both clear daytime images and images under various adverse conditions. Traditional diffusion-based generation methods often rely on offline pre-generated samples, requiring the module to have higher training synergy to handle diverse adverse scenarios. Therefore, this embodiment uses a conditional MeanFlow network model to quickly generate images of adverse observation conditions such as nighttime and rainy days based on the target conditions. To enhance the consistency between this module and the downstream deep learning objective, this embodiment jointly updates the generation module during the overall model training process, allowing the generation module to learn a more suitable distribution of adverse samples for geometric learning from prior data and deep supervision.

[0060] II. Design and Training of the Physical Sensing Decoupling Module In this embodiment, the physical perception decoupling module needs to estimate four components—reflection, illumination, transmission, and rain streaks—for the generated degraded image. On one hand, based on the physical properties of degraded image formation, this embodiment first constructs the intrinsic relationship between the same scene structure and different degradation factors based on the relationship between (clear daytime image, degraded observation image, and physical components). To this end, this embodiment utilizes, for example... Figure 2 The decomposition network shown receives the current poorly rendered image as input and compares the reconstructed image with the clear image to learn the correlation between the two from the data. On the other hand, this physical component learning lacks a gold standard. This embodiment utilizes self-supervised learning techniques to construct reconstruction loss and consistency loss for supervising the use of the decomposition network's supervisory signals, thereby pre-training on data pairs to obtain a decoupled system with separable geometry-preserving components.

[0061] III. Establishing an iterative training process for decoupled distillation and sequence constraints By designing and training an online conditional MeanFlow generation module and a physics-aware decoupling module, sub-modules are obtained to perform harsh generation, physical decomposition, and depth distillation in each iteration. In the overall iterative process design, this embodiment first introduces prior knowledge of the physical decomposition operator, including transmission range constraints, transmission smoothness constraints, and rain ripple high-frequency sparsity constraints. Then, an updated joint training model is obtained, and a clear domain teacher network is used to obtain pixel-level depth distillation signals. Furthermore, the basic depth model and illumination confidence map are used as sequence constraints, resulting in the following... Figure 3 The complete physical perception decoupled distillation training process is shown.

[0062] IV. The key points and areas to be protected in this embodiment of the method are at least as follows: Based on the online conditional MeanFlow generation module, joint optimization of adverse sample generation and downstream depth estimation is achieved. After training, the robustness of the student network stems from two aspects: first, the online generation module provides training samples that are relevant to the target adverse conditions and optimized with the task; second, the physics decomposition module removes degradation interference from the distillation path during training, allowing the student network to primarily rely on geometry-preserving representation learning. Even with unseen lighting variations or rain streak patterns in real-world data, the model can maintain relatively stable depth inference through reflection structures and relative order constraints.

[0063] The key points and areas to be protected in the robust monocular depth estimation system based on physical perception decoupling distillation in this embodiment are at least as follows: 1. A harsh scenario generation module based on online conditional MeanFlow enables dynamic generation of task-related harsh samples; II. Based on the physical decomposition technique, the decoupling process of reflection, illumination, transmission and rain ripples is used to achieve high performance and interpretability of deep learning algorithms; Third, by combining physical perception distillation with lighting perception sequence constraints, monocular depth estimation is achieved to realize personalized and targeted robust modeling for different adverse conditions.

[0064] Compared with traditional monocular depth estimation systems, the robust monocular depth estimation system based on physical perception decoupling distillation in this embodiment has the following advantages: First, this embodiment builds a harsh scene generation module based on online conditional MeanFlow. Compared with traditional offline generation systems, it can dynamically optimize training samples for different weather and lighting conditions, and has better scene adaptability.

[0065] Secondly, this embodiment obtains the decoupled distillation process based on a physical image formation model and uses a deep network for end-to-end training. This model structure combines the high performance of a deep model with the interpretability of a physical model.

[0066] Finally, for different autonomous driving and robot navigation tasks, the robust monocular depth estimation algorithm in this embodiment can perform corresponding high-quality depth prediction for scenarios such as nighttime and rainy days, and can better connect depth estimation with downstream 3D perception tasks.

[0067] The robust monocular depth estimation method and system based on physical perception decoupled distillation in this embodiment learns geometrically preserved representations for nighttime, rainy, and other scenarios by constructing an online conditional MeanFlow adverse scene generation module and performing physical-guided decomposition to build training relationships between clear daytime images, adverse observation images, and physical components. Robust monocular depth estimation is achieved using physical perception distillation and lighting perception sequence constraints. Since degradation interference factors are explicitly separated before distillation, this invention reduces the contamination of depth supervision by non-geometric pseudo-cues. Simultaneously, the online generation module enables synthetic samples to adapt to downstream geometric learning objectives, resulting in better robustness and interpretability.

[0068] This embodiment presents a robust monocular depth estimation method and system based on physically-aware decoupled distillation, comprising an online conditional MeanFlow adverse scene generation module, a physically-guided decomposition and reconstruction module, a physically-aware depth distillation module, and an illumination-aware sequential distillation module. The online conditional MeanFlow adverse scene generation module receives clear daytime images and adverse target imaging conditions, and generates adverse observation images such as those from nighttime and rainy days during training. The physically-guided decomposition and reconstruction module decomposes the adverse observation images into reflection, illumination, transmission, and rain streak components, where reflection and illumination components serve as geometry-preserving components, and transmission and rain streak components serve as degradation interference components. For the geometry-preserving representation, the depth distillation module transfers depth supervision from the clear domain teacher network to the student network. The illumination-aware sequential distillation module weights the relative depth constraints based on illumination reliability. Based on this online generation, physically decoupled, and multi-task distillation process, the monocular depth estimation performance and robustness in complex environments such as nighttime and rainy days can be improved without requiring dense depth annotation of adverse scenes.

[0069] A robust monocular depth estimation method based on physical perception decoupled distillation as described in this embodiment can be executed by writing a computer program and writing the computer program into a computer device or storage medium. When the computer program is read out and run, the robust monocular depth estimation method based on physical perception decoupled distillation as described in this embodiment is executed, thereby achieving the same technical effect as the robust monocular depth estimation method based on physical perception decoupled distillation as described in this embodiment.

[0070] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. Furthermore, the descriptions of "upper," "lower," "left," and "right" used in this disclosure are only relative to the relative positional relationships of the components of this disclosure in the accompanying drawings. The singular forms "a" and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. Moreover, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this embodiment specification is only for describing particular embodiments and is not intended to limit the invention. The term "and / or" as used in this embodiment includes any combination of one or more of the associated listed items.

[0071] It should be understood that although the terms first, second, third, etc., may be used to describe various elements in this disclosure, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. The use of any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided in this embodiment is intended only to better illustrate embodiments of the invention and, unless otherwise required, does not impose a limitation on the scope of the invention.

[0072] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can be implemented using standard programming techniques—including a non-transitory computer-readable storage medium configured with a computer program, wherein such a storage medium causes the computer to operate in a specific and predefined manner—according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0073] Furthermore, the procedures described in this embodiment can be performed in any suitable order unless otherwise indicated by this embodiment or clearly contradicted by the context. The procedures (or variations and / or combinations thereof) described in this embodiment can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. A computer program includes multiple instructions executable by one or more processors.

[0074] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention of this embodiment includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques of the invention, the invention also includes the computer itself.

[0075] A computer program can be applied to input data to perform the functions of this embodiment, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including a specific visual depiction of physical and tangible objects generated on the display.

[0076] The above are merely preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.

Claims

1. A robust monocular depth estimation method based on physical perception decoupled distillation, characterized in that, The robust monocular depth estimation method based on physical perception decoupled distillation includes: Online generation step: Acquire clear daytime images and adverse target imaging conditions, and generate adverse observation images based on the clear daytime images and adverse target imaging conditions; Physical decomposition step: Perform physical decomposition on the poorly viewed image and the clear daytime image to obtain multiple physical component parameters, and obtain a geometry-preserving representation based on the physical component parameters; Distillation step: Perform at least one iterative process to obtain a robust monocular depth estimation model from the student network after all the iterative processes have been performed; in any of the iterative processes, call the online generation step and the physical decomposition step to generate the geometry-preserving representation, and use the geometry-preserving representation to train the student network.

2. The robust monocular depth estimation method based on physical perception decoupled distillation according to claim 1, characterized in that, The step of generating a harsh observation image based on the clear daytime image and the harsh imaging conditions of the target includes: Obtain the online conditional MeanFlow generation module; The clear daytime image and the adverse imaging conditions of the target are input into the online conditional MeanFlow generation module for processing; Obtain the adverse observation image output by the online conditional MeanFlow generation module.

3. The robust monocular depth estimation method based on physical perception decoupled distillation according to claim 1, characterized in that, The physical decomposition of the poorly viewed image and the clear daytime image yields multiple physical component parameters, including: According to the formula Decompose; among them, For the aforementioned poor observation image, The reflection component of the poorly observed image. The illumination component of the poorly observed image, The transmission component of the poorly observed image. The rain streak component of the poorly observed image. For the daytime clear image, The reflection component of the clear daytime image. The illumination component of the daytime clear image.

4. The robust monocular depth estimation method based on physical perception decoupled distillation according to claim 3, characterized in that, The step of obtaining the geometry-preserving representation based on the physical component parameters includes: The reflection component of the poorly observed image and the illumination component of the poorly observed image This serves as the geometry-preserving representation.

5. The robust monocular depth estimation method based on physical perception decoupled distillation according to claim 4, characterized in that, Training the student network using the geometry-preserving representation includes: Establish a teacher network; The geometry-preserving representation is input into the student network to obtain the predicted depth output by the student network; The clear daytime image is input into the teacher network, and the standard depth ground value output by the teacher network is obtained. Determine the depth distillation loss value based on the predicted depth and the true value of the standard depth; The total loss value is determined based on the depth distillation loss value. The student network is trained based on the total loss value.

6. The robust monocular depth estimation method based on physical perception decoupled distillation according to claim 5, characterized in that, Determining the total loss value based on the depth distillation loss value includes: By analyzing the reflection component of the poorly observed image Apply reflection consistency constraints to the illumination components of the poorly viewed image. Apply heavy illumination constraint to the transmitted component Apply spatial smoothing and range constraints to the rain streak components. Applying high-frequency sparse constraints to obtain the reconstructed image According to the reconstructed image and the aforementioned poor observation images Determine the physical decomposition and reconstruction loss value; The depth difference between the teacher network and the student network is obtained. Based on the depth difference, uncertain regions and relatively stable regions in the adverse observation image are determined. Pixel pairs are sampled from the uncertain regions and the relatively stable regions. A relative depth order label is generated based on the pixel pairs using a frozen base depth model. A physical confidence map is constructed based on the illumination component gradient of the pixel pairs. A weighted order loss value is determined based on the order label and the physical confidence map. Based on the generation velocity field and target condition consistency of the online conditional MeanFlow generation module, the generation consistency loss is determined; The total loss value is determined based on the deep distillation loss value, the physical decomposition and reconstruction loss value, the weighted order loss value, and the generation consistency loss value.

7. The robust monocular depth estimation method based on physical perception decoupled distillation according to claim 6, characterized in that, The step of determining the total loss value based on the deep distillation loss value, the physical decomposition and reconstruction loss value, the weighted order loss value, and the generation consistency loss includes: Set the first weight, second weight, and third weight; According to the formula Perform calculations; among which, The total loss value, The depth distillation loss value, The physical decomposition and reconstruction loss value, The weighted order loss value, For the generation of consistency loss, For the first weight, This is the second weight. This is the third weight.

8. The robust monocular depth estimation method based on physical perception decoupled distillation according to any one of claims 1-7, characterized in that, The robust monocular depth estimation method based on physical perception decoupled distillation also includes: Obtain the image to be estimated; The image to be estimated is input into the robust monocular depth estimation model for processing; Obtain the depth estimate output by the robust monocular depth estimation model.

9. A computer device, characterized in that, It includes a memory and a processor, the memory being used to store at least one program, and the processor being used to load at least one program to execute the robust monocular depth estimation method based on physical perception decoupled distillation as described in any one of claims 1-8.

10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the robust monocular depth estimation method based on physical perception decoupled distillation as described in any one of claims 1-8.

Images