Self-supervision monocular depth estimation system and method in unmanned driving of mine electric locomotive

Abstract

The invention provides a self-supervision monocular depth estimation system and method in unmanned driving of a mine electric locomotive, and relates to the technical field of three-dimensional scene perception. The system comprises a cross-domain feature decoupling coding module, a depth estimation decoding module and a CLIP semantic consistency guiding module. According to the method, illumination irrelevant geometric features are extracted through a'shallow separation and deep sharing 'framework, and illumination interference is stripped through orthogonal separation loss; constructing loss based on physical depth invariance to realize cross-domain knowledge distillation; extracting robust semantic features by means of a pre-trained CLIP model, and constraining depth estimation semantic consistency; and the network is trained in an end-to-end manner through multi-source collaborative loss. In the testing process, a high-quality depth map can be output only by inputting a single low-light image, expensive depth truth value marking and a high-cost sensor are not needed, the problem of underground low-light visual degradation is effectively solved, the depth estimation precision and robustness are improved, and the method is suitable for scenes such as mine robot navigation and underground autonomous inspection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional scene perception, in particular to a self-supervised monocular depth estimation system and method for unmanned driving of a mine electric locomotive. BACKGROUND

[0002] With the rapid development of mine informatization construction, unmanned mining mode is steadily advancing, and more and more intelligent vision technologies such as robot navigation and autonomous inspection are being integrated into various work links of mine exploitation. As one of the core technologies of computer vision, depth estimation aims to recover the three-dimensional scene structure from two-dimensional images, and is the key foundation for realizing functions such as autonomous navigation, obstacle detection, and scene understanding of mine electric locomotives. The current mainstream self-supervised depth estimation method is usually based on the assumption of photometric consistency (Photometric Consistency) to construct a supervision signal, that is, it is assumed that the pixel brightness of the same scene point at different viewing angles remains consistent. This method uses the camera motion between video sequences to project the reference frame to the target frame through the estimated depth and pose, and trains the depth estimation network by minimizing the reprojection error. This technical route has been relatively mature in outdoor autonomous driving scenes with sufficient light, however, the mine environment is complex, and the light condition is poor, the collected images have large noise, low illumination and insufficient contrast, and the low-light vision degradation seriously restricts the promotion and application of depth estimation technology in mine exploitation. The importance of mine low-light depth estimation technology is increasingly prominent, which aims to accurately recover the three-dimensional geometric structure of the scene in the extremely low-light underground environment through scientific algorithms and models, thereby providing reliable depth perception capability for subsequent path planning, target recognition, and safety warning tasks. In-depth research and development of depth estimation technology suitable for mine low-light environment have important significance for promoting the intelligentization and safety development of mine exploitation.

[0003] The current mainstream processing idea mainly includes two categories: one is a cascade processing strategy based on image enhancement and depth estimation, that is, an image is preprocessed by using image defogging, low-light enhancement and other algorithms to try to restore the visibility and contrast of the image, and then the enhanced image is input into a depth estimation network. However, this method has the problem of error propagation and optimization conflict aggravation. Image enhancement and depth estimation are essentially two independent tasks, and the enhancement module is more likely to introduce artifacts or damage the geometric structure of the original image when dealing with dust and reflections. The error of this pre-sequencing link will be directly propagated to the subsequent depth estimation network as a cascade error, resulting in depth prediction distortion; the other is a multi-sensor fusion or hardware-assisted strategy, that is, by introducing an active light source or relying on a laser radar to obtain sparse depth ground truth to assist network training to make up for the lack of visual information. Although hardware assistance can provide more accurate depth prior, it has the following limitations: first, it significantly increases the hardware cost, power consumption and maintenance difficulty of the system, which is not conducive to large-scale deployment in mines; second, in the humid, dusty and space-limited environment of the mine, the illumination range and penetration ability of the active light source are limited, and the measurement accuracy of the laser radar is also affected by dust scattering; third, this method has poor generalization ability and is difficult to meet the wide application requirements of low-cost and passive visual perception in mines.

[0004] The above work still focuses highly on outdoor driving scenes, and the research on extremely low-light complex environments such as underground and tunnels is still in its infancy. The existing methods have not deeply explored how to exploit and utilize other self-supervised conditions that are insensitive to changes in illumination, resulting in the network being unable to establish an effective supervised closed loop in vision degradation scenes. Therefore, it is urgent to design an endogenous solution from the geometric and semantic characteristics of the depth estimation task itself to address the core problems of difficult geometric feature decoupling and self-supervised signal failure in the extremely low-light underground scene. SUMMARY

[0005] In view of the above technical deficiencies, the purpose of the present application is to provide a self-supervised monocular depth estimation system and method for unmanned driving of a mine electric locomotive, which realizes high-precision low-light depth estimation using only monocular video sequences without expensive depth ground truth labeling, and improves the accuracy and robustness of depth estimation in low-light underground scenes.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] A self-supervised monocular depth estimation system for unmanned driving of a mine electric locomotive, comprising:

[0008] A cross-domain feature decoupling encoding module is used to extract illumination-independent geometric features from paired normal light and low-light images using a shallow separation and deep sharing architecture.

[0009] a depth estimation decoding module configured to restore the geometric features to a pixel-level depth map through multi-level upsampling;

[0010] a CLIP semantic consistency guiding module configured to extract high-level semantic features robust to illumination changes and constrain semantic consistency of depth estimation.

[0011] A self-supervised monocular depth estimation method for unmanned driving of a mine motor car, comprising the following steps:

[0012] Step 1, constructing a cross-domain feature decoupling encoding module; a dual-flow encoder architecture with shallow separation and deep sharing is adopted, the shallow separation includes independent first-layer convolution design of normal light path and low light path, and the deep sharing includes shared encoder and private encoder; the shared encoder extracts illumination-independent geometric structure features from the shallow features, and the private encoder extracts color and noise-specific features in parallel;

[0013] Step 2, calculating an orthogonal separation loss to force the shared features and the private features to be orthogonal in the feature space, ensuring that the geometric features do not contain illumination interference;

[0014] Step 3, constructing a depth consistency decoding module, restoring spatial details of the geometric features output by the shared encoder through multi-level upsampling and convolution operation, generating normal light depth prediction and low light depth prediction respectively, and constructing a depth consistency loss based on the principle that the physical depth of a scene does not change with illumination;

[0015] Step 4, constructing a CLIP semantic consistency guiding module, using a pre-trained CLIP model with frozen parameters to process the input normal light image and low light image respectively, extracting normal light and low light image semantic features respectively, and constructing a semantic consistency loss based on the principle that the semantic features of scene objects do not change with illumination;

[0016] Step 5, in the training stage, using paired normal light and low light images to train the network end-to-end, and updating the network parameters through back propagation; in the testing stage, only the low light path is tested, and in the inference stage, only a single low light image is input to output a high-quality depth map.

[0017] Preferably, Step 1 specifically comprises the following steps:

[0018] Step 1-1, the normal light path uses a standard 7x7 convolution kernel and normalization parameters pre-trained on ImageNet to process the normal light image , capturing regular texture and edge information; the low light path uses an independently defined and initialized 7x7 convolution kernel to process the low light image The convolution kernel parameter is optimized to have greater gain, and can amplify the weak low light signal in the pixel value range of 0-80 to the activation space sensitive to the network;

[0019] Step1-2, the normal light feature flow and the low light feature flow after shallow separation processing are merged into the shared ResNet backbone network, and high-level abstract features are extracted layer by layer through Layer1 to Layer4, and the shared encoder focuses on extracting the geometric structure features in the scene irrelevant to the light , which is expressed as: ; ;

[0020] Among them, represents the shared Layer1-Layer4 structure, and respectively represent the geometric shared features of the low light image and the normal light image;

[0021] Step1-3, parallel private encoders are designed in the low light path and the normal light path respectively, the private encoders respectively receive the original images and , extract the specific features of the light distribution, color deviation and sensor noise, denoted as and .

[0022] Preferably, Step2 specifically includes the following steps:

[0023] Step2-1, through the orthogonal separation loss , the shared features and the private features are forced to be orthogonal in the feature space, ensuring that the geometric features do not contain light interference, and the orthogonal separation loss is expressed as follows:

[0024] ;

[0025] ;

[0026] ;

[0027] Among them, ⟨·, ·> represents the inner product operation, ||·|| represents the vector norm, is the orthogonal threshold, is the rectified linear unit function.

[0028] Preferably, Step3 specifically includes the following steps:

[0029] Step3-1, the extracted shared geometric features and are input into the depth decoder respectively; the decoder adopts a multi-level upsampling structure to gradually restore the feature resolution to the original image size, and outputs the final depth prediction map and ;

[0030] Step3-2, based on the principle that the physical depth of the scene does not change with the light, a depth consistency loss is constructed Through this loss, the more accurate depth prediction result under normal light is used to guide the learning of the low-light path, realizing cross-domain knowledge distillation, which is expressed as follows:

[0031] .

[0032] Preferably, Step4 specifically includes the following steps:

[0033] Step4-1, load the pre-trained CLIP model and freeze all its parameters; input and into the image encoder of CLIP respectively;

[0034] Step4-2, use the powerful generalization ability of CLIP to extract high-level semantic features robust to light changes, and get and respectively; CLIP divides the image into a Patch sequence and extracts a feature vector with global semantic context through a Transformer layer;

[0035] Step4-3, calculate the semantic consistency loss between and , and the semantic consistency loss between the depth network features and the CLIP semantic features ; this loss constrains the network to maintain the consistency of semantic blocks in dark areas, thereby correcting structural artifacts in depth estimation using semantic boundaries; it is expressed as follows:

[0036] ;

[0037] wherein: , is the feature vector output by the CLIP encoder.

[0038] Preferably, Step5 specifically includes the following steps:

[0039] Step5-1, integrate the above modules, and the total optimization objective function of the system consists of four parts, driving the network to train end-to-end:

[0040] ;

[0041] Step5-2, calculate photometric reconstruction loss for normal light path and low light path respectively, based on the camera pose estimated by PoseNet, the reference frame is projected to the target frame , and the photometric reconstruction error is calculated as:

[0042] ;

[0043] wherein: is the target image synthesized by inverse projection; is used to balance SSIM and L1 loss; SSIM uses reflection padding, and the window size is 7x7.

[0044] Compared with the prior art, the present application has the following beneficial effects:

[0045] The self-supervised monocular depth estimation system and method in the mine electric locomotive unmanned driving can improve the depth estimation accuracy by constructing a cross-domain feature decoupling mechanism of "shallow separation and deep sharing" and introducing semantic priori. Firstly, independent large gain convolution kernels are designed for low light input to amplify weak signals, and the network is forced to strip interference factors such as light and noise to private encoders through orthogonal separation loss, so as to ensure that the shared encoder extracts pure geometric features; then, by using the characteristics that the scene physical depth does not change with light, the knowledge distillation from normal light path to low light path is realized through depth consistency loss; at the same time, the pre-trained CLIP model is introduced to extract high-level semantic features robust to light changes, and the semantic consistency loss is used to constrain the depth estimation network to keep the structure clear on the semantic boundary, so as to accurately restore the scene geometric structure and improve the algorithm accuracy in the case of lack of photometric information. The self-supervised monocular depth estimation method and system in the mine electric locomotive unmanned driving cooperatively act under the synergistic effect of geometric feature decoupling, depth invariance knowledge distillation and semantic boundary guidance, and the depth estimation network is trained in an end-to-end manner by using a multi-source collaborative loss optimization method, which can not only effectively deal with the visual degradation problem caused by extremely low light in the mine, but also restore accurate depth according to geometric and semantic clues in the area lacking effective photometric matching, and is especially suitable for scenes such as mine robot navigation and underground autonomous inspection which need depth perception in low light environment. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is the structural block diagram of the present application;

[0048] Figure 2 is the structural diagram of the cross-domain feature decoupling coding module;

[0049] Figure 3 is the structural diagram of the CLIP semantic consistency guiding module;

[0050] Figure 4A flowchart of the present application;

[0051] Figure 5 A test effect picture of the present application in a roadway scene. DETAILED DESCRIPTION

[0053] The present application will be further described below with reference to the accompanying drawings.

[0054] As Figures 1 to 3 shown, a self-supervised monocular depth estimation system in mine electric locomotive unmanned driving includes: 1. Cross-domain feature decoupling encoding module (CDFD-E, Cross-Domain Feature Decoupling Encoder), which is used to extract illumination-independent geometric features from paired normal light and low light images; 2. Depth estimation decoding module, which is used to restore the geometric features to the pixel-level depth map; 3. CLIP semantic consistency guiding module, which is used to extract high-level semantic features robust to illumination changes; 4. PoseNet pose estimation network, which is used to estimate camera motion and calculate photometric reconstruction loss.

[0055] Cross-domain feature decoupling encoding module, CLIP semantic consistency guiding module and multi-source collaborative loss optimization design. The cross-domain feature decoupling encoding module adopts a double-flow architecture of "shallow separation and deep sharing" to obtain image features, which includes a shared encoder and a private encoder: the shared encoder processes normal light and low light inputs through the first layer convolution designed independently, respectively. The normal light path uses standard weights to capture regular features, and the low light path uses independent large gain convolution kernels to amplify weak signals, and then merges into the shared deep network to extract illumination-independent geometric structure features; the private encoder independently extracts domain-specific features such as color and noise to assist feature decoupling. The CLIP semantic consistency guiding module is used to introduce robust semantic priors, and a pre-trained CLIP model with frozen parameters is used to directly process the input image to extract high-level semantic feature maps that are not sensitive to illumination changes. The depth estimation decoding module is used to restore the spatial resolution of the pure geometric features output by the shared encoder, and reconstructs the pixel-level depth information through multi-level upsampling. The multi-source collaborative loss optimization design includes orthogonal separation loss, depth consistency loss and semantic guiding loss, which establishes a multi-dimensional supervision signal to drive network parameter updating.

[0056] As Figure 4 shown, a self-supervised monocular depth estimation method in mine electric locomotive unmanned driving, the method specifically includes the following steps:

[0057] Step1, a cross-domain feature decoupling encoding module is constructed, a double-stream encoder architecture of shallow separation and deep sharing is adopted, the shallow separation includes independent first layer convolution design of normal light path and low light path, the deep sharing includes shared encoder and private encoder, the shared encoder extracts illumination-independent geometric structure features from shallow features, the private encoder extracts domain-specific features such as color and noise in parallel, and the specific process is as follows:

[0058] Step1-1, in view of the problems of low pixel value and poor signal-to-noise ratio in the downhole low light scene, the application does not use a unified first layer convolution, but designs two independent shallow convolution paths:

[0059] Normal light path (Layer-norm): using a standard 7x7 convolution kernel and normalization parameters pre-trained on ImageNet to process normal light images , capture regular texture and edge information. This path inherits the general visual features learned by the pre-trained model on large-scale datasets, and can effectively extract image features under normal lighting conditions.

[0060] Low light path (Layer-low): using an independently defined and initialized convolution kernel to process low light images . The convolution kernel parameters are specially optimized and have a larger gain, which can amplify the weak low light signal with pixel values in the range of 0-80 to the network-sensitive activation space. This design solves the problem of bottom layer information loss caused by the poor response of general pre-trained weights to low light input.

[0061] Step1-2, after the shallow separation processing, the normal light feature stream and the low light feature stream are merged into the shared ResNet backbone network, and high-level abstract features are extracted layer by layer through Layer1 to Layer4. Since the shallow signals have been processed and aligned, the shared network can focus on extracting the geometric structure features in the scene that are independent of illumination. This process can be represented as:

[0062] ;

[0063] ;

[0064] Wherein, represents the shared Layer1-Layer4 structure, and represent the geometric shared features of the low light image and the normal light image respectively. The shared encoder learns the geometric representation that is robust to illumination changes through joint training on normal light and low light data.

[0065] Step1-3, in order to purify the shared features and ensure that they only contain geometric information without illumination, noise and other interference factors, such asFigure 3 Two private encoders (P-Encoders) are designed for low-light and normal-light paths, respectively.

[0066] The private encoders receive the original image and as input, and extract domain-specific features such as illumination distribution, color bias, and sensor noise through independent convolutional networks, denoted as and These domain-specific features capture information that is irrelevant to the depth estimation task but closely related to the lighting conditions, providing the necessary conditions for subsequent feature decoupling.

[0067] Step 2, the orthogonal separation loss forces the shared features and the private features to be orthogonal in the feature space, ensuring that the geometric features do not contain lighting interference. The orthogonal separation loss is represented as follows:

[0068] ;

[0069] ;

[0070] ;

[0071] where ⟨·,·> represents the inner product operation, ||·|| represents the vector norm, is the orthogonal threshold, is the rectified linear unit function.

[0072] The design idea of this loss function is: if the cosine similarity of shared features and private features is too high, it means that the shared features still contain domain-specific information such as lighting, and a penalty needs to be applied. By minimizing this loss, the network is forced to strip lighting, noise, and other factors to the private branch, allowing the shared branch to focus on extracting pure geometric features.

[0073] In actual training, the orthogonal separation loss is jointly optimized with other loss terms, and the weight coefficient is set to 0.2 to balance the relationship between feature decoupling and depth estimation accuracy.

[0074] Step 3, as shown at the bottom, the depth consistency decoding module includes two sub-steps: Figure 3

[0075] Step 3-1, the extracted shared geometric features and are input into the depth decoder. The decoder uses a multi-level upsampling structure to gradually restore the feature resolution to the original image size, outputting the final depth prediction map and .​

[0076] Step3-2, based on the principle that the scene physical depth does not change with illumination, a depth consistency loss is constructed . Through this loss, the more accurate depth prediction results under normal light are used to guide the learning of the low-light path, realizing cross-domain knowledge distillation, which is expressed as follows:

[0077] ;

[0078] Through this loss, the network can still recover the true geometric structure according to the depth-invariant clues learned from the normal light path in the case of extremely scarce light signals.

[0079] Step4, using the powerful generalization ability of CLIP, extract high-level semantic features robust to illumination changes, as follows:

[0080] Step4-1, load the pre-trained CLIP model. CLIP (Contrastive Language-Image Pre-training) is a visual-linguistic model pre-trained on a large number of image-text pairs, with strong cross-modal representation learning ability and robustness to illumination changes. The present application loads the pre-trained CLIP model and freezes all its parameters to ensure that the general feature representation ability learned on large-scale data is not destroyed. The normal light image and the low light image are input into the image encoder of CLIP respectively. The image encoder of CLIP is based on the Vision Transformer architecture and can capture global semantic information.

[0081] Step4-2, semantic feature extraction, using the powerful generalization ability of CLIP, extract high-level semantic features robust to illumination changes, respectively and . CLIP divides the image into a Patch sequence and extracts feature vectors with global semantic context through Transformer layers. The dimensions of the two semantic feature vectors are both 512, containing high-level semantic information of the image, such as scene category, object distribution, spatial layout, etc., and having strong invariance to illumination changes.

[0082] Step4-3, semantic consistency loss calculation, calculate and , and the semantic consistency loss between the depth network features and the CLIP semantic features . This loss constrains the network to maintain the consistency of semantic blocks in dark areas, so as to correct the structural artifacts in depth estimation using semantic boundaries. It is expressed as follows:

[0083] ;

[0084] wherein: , is the feature vector output by the CLIP encoder. By minimizing this loss, the network is encouraged to extract semantically consistent deep features under normal and low light conditions. Since the semantic features of CLIP are robust to illumination changes, this constraint can effectively compensate for the lack of underlying photometric information, recovering clear and sharp depth boundaries in texture-deficient areas based on semantic context.

[0085] Step 5, multi-source collaborative loss optimization and training inference, as follows:

[0086] Step 5-1, integrating the above modules, the total optimization objective function of the system consists of four parts, driving end-to-end network training:

[0087] ;

[0088] wherein: is the photometric reconstruction loss, which constrains the geometric consistency of depth and pose estimation; is the orthogonal separation loss, which ensures the decoupling of geometric features and illumination features; is the depth consistency loss, which realizes cross-domain knowledge distillation; is the semantic consistency loss, which uses semantic boundaries to correct depth estimation; , , are weight coefficients used to balance the importance of each loss term

[0089] In actual training, the weight coefficients are usually set as: = 0.2, = 1.0, = 0.5. These parameters can be adjusted according to specific data sets and task requirements.

[0090] Step 5-2, photometric reconstruction loss is calculated for normal light path and low light path respectively, based on the camera pose estimated by PoseNet, the reference frame is projected to the target frame , the photometric reconstruction error is calculated:

[0091] ;

[0092] wherein: is the target image synthesized by inverse projection; is used to balance the SSIM and L1 loss; SSIM uses reflection padding with a window size of 7x7.

[0093] Step 5-3, the training and inference process is as follows:

[0094] The training phase uses paired normal light and low light images to train the network end-to-end. Specifically, adjacent frames are extracted from the underground video sequence as training samples, each sample contains: normal light sequence and the corresponding low light sequence .

[0095] The network processes both normal light and low light branches simultaneously:

[0096] 1. Normal light branch: input , through Layer-norm shallow convolution → shared encoder → deep decoder, output ;

[0097] 2. Low light branch: input , through Layer-low shallow convolution → shared encoder → deep decoder, output ;

[0098] 3. Private encoder processes and respectively, extracting domain-specific features;

[0099] 4. The CLIP model processes and respectively, extracting semantic features;

[0100] 5. PoseNet estimates camera pose, combined with depth prediction to calculate photometric reconstruction loss

[0101] Through this joint training method, the network learns cross-domain depth-invariant features: the shared encoder extracts illumination-independent geometric representation, which can handle normal light and low light inputs simultaneously; depth consistency loss ensures that the low light path is guided by the normal light path; semantic consistency loss ensures the accuracy of depth estimation on semantic boundaries.

[0102] Inference phase: only the low light path is used during testing. Input a single low light image , after Layer-low shallow convolution → shared encoder → depth decoder, the high-quality depth map can be output. The entire inference process does not require normal light images, does not require CLIP models, and does not require adjacent frame information, achieving true monocular single-frame depth estimation.

[0103] Figure 5A depth estimation visual contrast chart for testing the method in a low-light scene underground. Among them, from left to right, the input low-light image, the depth map predicted by the baseline method (Baseline), and the depth map predicted by the method (Ours) are listed in order. As shown in the figure, in the environment with extremely poor lighting conditions, the depth map generated by the baseline method has serious noise interference, the object edge is blurred, and there is a large area of false depth in the dark area. While the technical solution significantly improves the depth estimation accuracy, such as the box structure on the wall in the first row and the track edge in the second row, maintaining a relatively clear object contour, and the depth level transition is also smoother and more natural. Further illustrates the robustness and superiority of the method in self-supervised depth estimation in the low-light scene underground.

[0104] The self-supervised monocular depth estimation method and system in the unmanned mine motor car can improve the depth estimation accuracy by means of the feature decoupling mechanism of "shallow separation and deep sharing" and the CLIP semantic guidance enhancement algorithm for understanding the scene geometry structure. By enhancing the model's deep understanding of the geometry and semantics of the underground scene, the accuracy and robustness of the low-light self-supervised depth estimation algorithm can be significantly improved, reducing the dependence on expensive depth ground truth labeling for building new scene depth estimation algorithms, and getting rid of the dependence on high-cost sensors such as laser radars. High-precision depth estimation can be achieved only by relying on monocular video sequences, which is conducive to the efficient and low-cost construction and deployment of depth estimation algorithms in mine environments. Under the synergistic effect of cross-domain feature decoupling and depth invariance knowledge distillation, the depth estimation network is trained end-to-end in a multi-source collaborative loss optimization manner to adapt to the low-light scene underground. It can not only enhance the model's ability to extract pure geometric features independent of lighting and handle extreme lighting changes, but also significantly improve the model's depth estimation accuracy based on a deep understanding of scene geometry and semantic boundaries, especially suitable for practical application scenarios such as mine robot autonomous navigation, underground inspection vehicle obstacle detection, and coal mining face three-dimensional reconstruction that require accurate depth perception in extremely low-light and unevenly illuminated environments.

Claims

1. A self-supervised monocular depth estimation system for unmanned driving of mine locomotives, characterized in that, include: A cross-domain feature decoupling coding module is used to extract illumination-independent geometric features from paired normal light and low light images using a shallow-separation-deep-sharing architecture. The depth estimation decoding module is used to recover pixel-level depth maps from geometric features through multi-level upsampling; The CLIP semantic consistency guidance module is used to extract high-level semantic features that are robust to changes in illumination and to constrain the semantic consistency of depth estimation.

2. A self-supervised monocular depth estimation method for unmanned operation of mine locomotives, characterized in that, Includes the following steps: Step 1: Construct a cross-domain feature decoupling coding module; adopt a dual-stream encoder architecture with shallow separation and deep sharing. Shallow separation includes independent first-layer convolutional designs for normal light paths and low light paths, while deep sharing includes a shared encoder and a private encoder. The shared encoder extracts illumination-independent geometric features from shallow features, while the private encoder extracts color and noise-specific features in parallel. Step 2: Calculate the orthogonal separation loss to force shared features and private features to be orthogonal in the feature space, ensuring that geometric features do not contain illumination interference; Step 3: Construct a depth consistency decoding module. The geometric features output by the shared encoder are used to recover spatial details through multi-level upsampling and convolution operations. Normal light depth prediction and low light depth prediction are generated respectively. Based on the principle that the physical depth of the scene does not change with the illumination, a depth consistency loss is constructed. Step 4: Construct the CLIP semantic consistency guidance module. Use the pre-trained CLIP model with frozen parameters to process the input normal light image and low light image respectively, and extract the semantic features of the normal light and low light images respectively. Based on the principle that the semantic features of scene objects do not change with illumination, construct the semantic consistency loss. Step 5: During the training phase, the network is trained end-to-end using paired normal light and low light images, and the network parameters are updated through backpropagation. During testing, only low light paths are tested, and during the inference phase, a high-quality depth map can be output by inputting only a single low light image.

3. The self-supervised monocular depth estimation method for unmanned operation of mine locomotives as described in claim 2, characterized in that, Step 1 specifically includes the following steps: Step 1-1: Normal light path processing of normal light images using standard 7×7 convolutional kernels and normalized parameters pre-trained on ImageNet. It captures regular texture and edge information; the low-light path uses independently defined and initialized 7×7 convolutional kernels to process low-light images. The optimized convolution kernel parameters have greater gain, enabling the amplification and mapping of weak low-light signals with pixel values ​​in the range of 0-80 into the network's sensitive activation space. Steps 1-2: After shallow separation, the normal light feature stream and the low light feature stream are merged into a shared ResNet backbone network. High-level abstract features are extracted layer by layer from Layer 1 to Layer 4. The shared encoder focuses on extracting geometric structural features in the scene that are independent of illumination. , represented as: ； ； in, This represents a shared Layer1-Layer4 structure. and These represent the geometrically shared features of low-light and normal-light images, respectively. Steps 1-3 involve designing parallel proprietary encoders for both the low-light and normal-light paths, with each proprietary encoder receiving the original image. and Extract specific features of illumination distribution, color deviation, and sensor noise, denoted as... and .

4. The self-supervised monocular depth estimation method for unmanned operation of mine locomotives as described in claim 3, characterized in that, Step 2 specifically includes the following steps: Step 2-1, using orthogonal separation loss Forced sharing of features With private characteristics In the feature space, the features are orthogonal, ensuring that the geometric features do not contain illumination interference. The orthogonal separation loss is expressed as follows: ； ； ； Where ⟨· and ·> represent inner product operations, and ||·|| represents the vector norm. Orthogonal threshold, To correct the linear unit function.

5. The self-supervised monocular depth estimation method for unmanned operation of mine locomotives as described in claim 4, characterized in that, Step 3 specifically includes the following steps: Step 3-1, Extracted shared geometric features and The data are input into the depth decoder; the decoder uses a multi-stage upsampling structure to gradually restore the feature resolution to the original image size, and outputs the final depth prediction map. and ; Step 3-2: Based on the principle that the physical depth of the scene does not change with illumination, construct a depth consistency loss function. By using this loss function, the more accurate depth prediction results under normal lighting are used to guide the learning of low-light paths, achieving cross-domain knowledge distillation, as shown below: 。 6. The self-supervised monocular depth estimation method for unmanned operation of mine locomotives as described in claim 5, characterized in that, Step 4 specifically includes the following steps: Step 4-1: Load the pre-trained CLIP model and freeze all its parameters; and Input the image encoder into CLIP respectively; Step 4-2: Leveraging CLIP's powerful generalization ability, extract high-level semantic features robust to illumination changes, and obtain... and CLIP segments the image into a sequence of patches and extracts feature vectors with global semantic context through a Transformer layer. Step 4-3, Calculation and Semantic consistency loss between features and CLIP semantic features, and between deep network features and CLIP semantic features. This loss-constrained network maintains semantic block consistency in dark regions, thereby using semantic boundaries to correct structural artifacts in depth estimation; it is represented as follows: ； in: , This is the feature vector output by the CLIP encoder.

7. The self-supervised monocular depth estimation method for unmanned driving of mine locomotives as described in claim 6, characterized in that, Step 5 specifically includes the following steps: Step 5-1: Combining the above modules, the system's overall optimization objective function consists of four parts, driving end-to-end network training: ； Step 5-2: Calculate the photometric reconstruction loss for both the normal light path and the low-light path. Based on the camera pose estimated by PoseNet, the reference frame... Projected onto target frame Calculate the photometric reconstruction error: ； in: The target image synthesized through inverse projection; This is used to balance the SSIM and L1 losses; SSIM uses reflection fill with a window size of 7×7.

Images