An uncertainty-aware multi-modal mine road extraction method

Abstract

This invention relates to the field of road extraction technology in mining areas, and discloses a multimodal road extraction method for mining areas with uncertainty perception. Addressing the problems of inconsistent image modal and trajectory modal quality, information role asymmetry leading to uncontrolled utilization of trajectory structure cues, susceptibility to interference with visual representations, and insufficient road network continuity, this invention acquires remote sensing images and vehicle positioning trajectory data to construct trajectory heatmaps and strip-shaped trajectory structure prior maps. It extracts image features and trajectory prior features through a dual-branch network, and injects the trajectory priors into the image branches in a unidirectional, asymmetric manner. Furthermore, it combines long-range connectivity enhancement and uncertainty-perceived modal reliability assessment with adaptive fusion to output the mining area road extraction results. This method can improve the accuracy, continuity, and robustness of road extraction in complex mining area scenarios.

Description

Technical Field

[0001] This invention relates to the field of mining area road extraction technology, and in particular to a multimodal mining area road extraction method with uncertainty perception. Background Technology

[0002] Road extraction technology belongs to the interdisciplinary field of remote sensing image processing, geographic information processing, and intelligent transportation perception. Its main task is to automatically identify road areas and reconstruct road networks based on aerial imagery, satellite imagery, and other spatial information data. The development of existing technologies has roughly evolved from traditional image processing to deep learning, from single image input to multi-source heterogeneous data fusion, and from pixel-level segmentation to continuous road network representation. Current research indicates that the core development direction of road extraction methods focuses on multimodal remote sensing data fusion and optimizing road network-level representations to meet practical application requirements, moving beyond pixel-level identification.

[0003] In the extraction of roads in single-modal mining areas, existing studies typically use high-resolution remote sensing imagery or UAV imagery as input, employing convolutional neural networks and their improved models to segment road regions. Road identification primarily relies on image appearance features, such as texture, edges, and spectrum. However, in open-pit mine scenarios, roads are often obscured by dust, shadowed by deep pits, have blurred boundaries, and experience frequent changes in road morphology. Relying solely on image information can easily lead to road breaks, missed detections, and false detections.

[0004] To improve the robustness of road extraction, existing multimodal road extraction methods have begun to incorporate auxiliary information such as GPS trajectories and LiDAR point clouds, and achieve the fusion of image and trajectory information through feature stitching, bi-branch coding, attention mechanisms, or gated propagation. These methods can, to some extent, utilize trajectory information to supplement road structure clues and improve the robustness of road extraction. However, in open-pit mine road extraction scenarios, image and trajectory modalities are not entirely equivalent information sources: remote sensing images mainly provide road boundaries, textures, and spatial geometry information, serving as the primary basis for road area identification; vehicle positioning trajectories mainly reflect vehicle travel paths and road framework orientation, making them more suitable as structural cues. Due to the complexities of open-pit mine areas, such as dust occlusion, deep pit shadows, weak boundaries, homogeneous backgrounds, trajectory drift, sparse sampling, and local pseudo-structures, the quality and reliability of the two modalities often differ at different spatial locations.

[0005] If symmetrical interaction or unconstrained fusion methods are directly adopted, the distorted response in the trajectory may still interfere with the image branches' representation of the true road texture, boundaries, and semantics. On the other hand, if over-reliance is placed on local image features, it is difficult to maintain the topological continuity of slender roads and complex intersections in occluded, shadowed, or sparse trajectory areas. Therefore, how to effectively utilize the road structure cues of the trajectory while avoiding the pollution of visual representation by distorted trajectories, and ultimately maintaining the continuous representation of the mining area road network, under conditions of inconsistent modal quality and unequal information roles, remains a technical problem that needs to be solved in open-pit mine road extraction. Summary of the Invention

[0006] To address the challenge of existing methods failing to maintain stable visual representations and road network continuity while utilizing trajectory structure cues, this invention proposes a multimodal road extraction method for uncertainty perception in mining areas.

[0007] To achieve the above objectives, the present invention adopts the following technical solution, including: A multimodal method for road extraction in mining areas with uncertainty perception includes the following steps: S1: Acquire remote sensing images of the target open-pit mine area and vehicle positioning trajectory data corresponding to the space of the remote sensing images, as input data for the multimodal mine area road extraction network; S2: Constructing a trajectory heatmap generation unit: Performing spatial registration between the trajectory and the image and constructing a trajectory heatmap; S3: Constructing a prior map of a strip trajectory: Using the trajectory heatmap as input, a stable and clear prior map of a strip trajectory is constructed through image processing; S4: Establish a dual-branch feature extraction network: Construct an image branch and a trajectory branch based on remote sensing images and a strip trajectory prior map, and extract image features and trajectory prior features respectively; S5: Constructing an asymmetric prior injection unit: Based on a dual-branch feature extraction network, guide weights are constructed to control the channel dimension and spatial position when trajectory prior features are injected into image features, and the image branches are guided to learn the road structure in a unidirectional and asymmetric manner. S6: Constructing a long-range connectivity enhancement unit: Aggregate the deep features of the image branch and the deep features of the trajectory branch after asymmetric structure prior injection to generate preliminary shared bottleneck features; apply a long-range connectivity enhancement module to the preliminary shared bottleneck features to enhance long-range continuous expression capabilities and obtain enhanced shared bottleneck features; S7: Constructing an uncertainty-aware modal adaptive fusion unit: Decode the enhanced shared bottleneck features, construct a modal adaptive fusion unit based on the decoded features, adaptively adjust the contributions of trajectory modality and image modality in the final prediction, and obtain the final road prediction result; S8: Use a joint loss function to optimize the training of the entire network.

[0008] Preferably, in step S4, both the image branch and the trajectory branch are feature extraction networks, and each has a built-in image encoder and trajectory encoder with the same structure, both employing a hierarchical convolutional encoder; in the i-th encoding stage, the image encoder and trajectory encoder respectively output image features. Trajectory Prior Features .

[0009] Preferably, in step S5, the guiding weights include channel guiding weights and spatial guiding weights; the channel guiding weights are based on image features. Trajectory Prior Features Global average pooling is used to extract the global context descriptors of each component. The two descriptors are then fused and input into a fully connected network, where they are activated by an activation function to obtain channel gating. Spatial guided weights are based on image features. Trajectory Prior Features The concatenation and convolution along the channel dimension, followed by activation by an activation function, yields a spatially gated array. ; Gating the channel and spatial gating By combining these elements, a combined channel-space gating system is obtained. : .

[0010] Preferably, a unidirectional, asymmetric approach is used to guide image branches to learn road structure, including: joint gating of channels and space. Trajectory Prior Features Weighted injection features Injection features Image features are added using a one-way residual method. middle.

[0011] Preferably, in step S6, the preliminary shared bottleneck feature is obtained by splicing the deepest enhanced feature of the image branch after asymmetric structure prior injection with the deepest feature of the trajectory branch.

[0012] Preferably, the long-range connectivity enhancement module includes: performing a first bottleneck operation on the shared bottleneck features and performing long-range context aggregation operations along multiple different directions to generate a connectivity modulation map for characterizing the connectivity relationships between features; fusing the long-range responses in different directions to generate a connectivity modulation map; using the connectivity modulation map to perform element-wise modulation on the initial shared bottleneck features to obtain connectivity enhancement features; and performing a second bottleneck refinement operation on the connectivity enhancement features to output the enhanced shared bottleneck features.

[0013] Preferably, both the first bottleneck operation and the second bottleneck operation include one or more combinations of directional convolution, multi-scale pooling, and dilated convolution.

[0014] Preferably, in step S7, the modality adaptive fusion unit includes mapping the decoded features to a logits graph and performing non-negative activation to generate evidence values ​​for each modality; constructing distribution parameters based on the evidence values; calculating the total evidence strength; and calculating the category prediction probability vector and uncertainty for each modality based on the total evidence strength; and adaptively generating fusion weights to control the contribution ratio of each modality in the final prediction based on the uncertainty of each modality.

[0015] Preferably, the fusion unit includes: weighting the fusion coefficients with the category prediction probability vectors of each modality to obtain the final fusion prediction result.

[0016] Preferably, the joint loss function At least include segmentation loss loss of evidence Topology preservation loss and boundary consistency loss :

[0017] in, The weight coefficients represent the weights that change dynamically with the number of training iterations (t), and are used to control the evidence regularization loss. In the joint loss function The percentage of contributions; and These represent fixed weighting coefficients, used to control the topology preservation loss. and boundary consistency loss In the joint loss function The percentage of contribution.

[0018] The advantages of this invention are: (1) This invention transforms the vehicle positioning trajectory from ordinary auxiliary input into a strip-shaped trajectory structure prior, so that the trajectory information mainly serves as a road skeleton orientation and structural cues. Through a unidirectional, asymmetric structure prior injection method, the trajectory prior guides the image branches to learn the road structure in a controlled manner during the feature extraction stage, rather than being unconstrainedly symmetrically mixed with image features. This reduces the interference of trajectory drift, sparse sampling, and local pseudo-structures on the visual backbone representation and improves the stability of image feature representation.

[0019] (2) In view of the problems of long and thin roads, weak boundaries, easy occlusion and complex intersection areas in open mines, this invention introduces a long-range connectivity enhancement mechanism to perform context compensation and connectivity restoration in areas with degraded local image evidence or insufficient trajectory evidence, enhance the long-distance structural association of long and thin roads and complex intersection areas, reduce road breakage and fragmentation, and improve the integrity and continuity of road extraction results.

[0020] (3) This invention uses an uncertainty-aware modal reliability assessment and adaptive fusion mechanism to dynamically allocate the fusion weights of the two modalities based on the evidence strength and uncertainty of the image modal and trajectory modal during the prediction stage, so that the final road prediction result can adapt to the modal quality changes at different spatial locations, thereby improving the robustness and accuracy of the road extraction result.

[0021] (4) The present invention takes into account both pixel-level segmentation accuracy and road network-level topology integrity, and can obtain road results that are more suitable for mine route planning, transportation organization, unmanned driving and digital mine modeling. Attached Figure Description

[0022] Figure 1 A flowchart of a multimodal road extraction method for uncertainty perception in mining areas; Figure 2 This is an overall architecture diagram of a multimodal road extraction method for uncertainty perception in mining areas. Figure 3 A schematic diagram of the asymmetric structure prior injection module for a multimodal mining area road extraction method based on uncertainty perception; Figure 4 This is a schematic diagram of an uncertainty-aware modal adaptive fusion unit for a multimodal road extraction method in a mining area. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0024] A multimodal method for road extraction in mining areas with uncertainty perception, such as Figure 1 and Figure 2 As shown, it includes the following steps: S1: Obtain multimodal input data for the multimodal mining area road extraction network. Acquire remote sensing images of the target open-pit mine area and vehicle positioning trajectory data corresponding to the spatial location of the remote sensing images.

[0025] The remote sensing images are obtained from UAV images, aerial images, or satellite remote sensing images.

[0026] Vehicle positioning trajectory data is collected by vehicle-mounted positioning terminals, mining truck scheduling systems or other positioning systems; the vehicle positioning trajectory data includes the vehicle's latitude and longitude, timestamp and vehicle attribute information in the mining area road network, and the vehicle attribute information includes selectable speed and heading.

[0027] S2: Constructing the trajectory heatmap generation unit: Performing spatial registration between the trajectory and the image and constructing the trajectory heatmap.

[0028] S21: Preprocess vehicle positioning trajectory data

[0029] Preprocessing operations include outlier removal, time synchronization, coordinate transformation, and spatial registration; the preprocessed vehicle positioning trajectory data and remote sensing images are unified into the same coordinate system; Belonging to the same coordinate system includes the same geographic coordinate system or the same image coordinate system; S22: Project the vehicle positioning trajectory data onto the corresponding pixel grid of the remote sensing image, and count the number of trajectory points in each pixel grid to obtain the initial trajectory density map; S23: Normalize and smooth the initial trajectory density map to obtain the trajectory heatmap; Preferably, a logarithmic transformation is used to normalize the density values ​​of the initial trajectory density map to obtain a normalized density map; then, a Gaussian filter or a mean filter is used to smooth the normalized density map.

[0030] S3: Constructing a structurally stable and clear strip trajectory prior map based on trajectory heatmap.

[0031] Trajectory heatmaps are grayscale density distribution maps that reflect the density of trajectory distribution. They are numerically continuous, have blurred edges, uneven widths, and no clear center line, making them difficult to use directly as guiding information. Therefore, it is necessary to construct a relatively stable banded trajectory prior map with a clear center line, uniform width, and smooth edges through image processing.

[0032] Using the trajectory heatmap as input, image processing methods include one or more of the following: skeleton extraction, thresholding, morphological thinning, dilation, and max pooling expansion.

[0033] Preferably, a heatmap is first used as input, and low-density noise is removed by thresholding to output a binary trajectory mask. The binary trajectory mask is then used as input to extract the central skeleton through skeleton extraction or morphological thinning to output a skeleton map. The skeleton map is then input into dilation or max pooling to expand pixels, fill breaks, smooth edges, and output a band-shaped trajectory prior map with uniform and continuous width, reducing the impact of single trajectory point drift on local structure judgment.

[0034] S4: Establish a dual-branch feature extraction network

[0035] Construct image branch feature extraction network and trajectory branch feature extraction network.

[0036] The remote sensing image is input into the image branch feature extraction network, which extracts multi-scale road texture features, boundary features, and contextual semantic features of the remote sensing image through the built-in image encoder.

[0037] The prior image of the strip trajectory is input into the trajectory branch feature extraction network, and the traffic structure features and linear continuity features of the vehicle trajectory are extracted through the built-in trajectory encoder.

[0038] Preferably, the image encoder and the trajectory encoder have the same structure, both employing a hierarchical convolutional encoder.

[0039] In the i-th encoding stage, the image encoder and the trajectory encoder output image features respectively. Trajectory Prior Features .

[0040] like Figure 3 As shown: S5: Asymmetric structure prior injection of image features Current layer image features based on image branching Prior features of the current layer trajectory and trajectory branch A guiding weight is generated, which is used to control the channel dimension and spatial position when trajectory prior features are injected into image features.

[0041] The guidance weights include channel guidance weights and spatial guidance weights; the channel guidance weights are the control weights of the channel dimension, and the spatial guidance weights are the control weights of the spatial dimension.

[0042] Specifically, channel-guided weights are based on image features. Trajectory Prior Features Global average pooling is used to extract the respective global context descriptors. The two descriptors are then fused and input into the fully connected network, and then... Activation function activation results in channel gating Access control The final output parameters for channel-guided weights. The global context descriptor reflects the overall distribution of the image and prior trajectories. Based on this, the channel-guided weights determine which semantic channels should be injected more into the prior. For example, if the global image is dark, the prior of the motion channels, which are independent of illumination, can be enhanced.

[0043] Spatial guided weights based on image features Trajectory Prior Features The data is concatenated along the channel dimension and fed into the convolutional network, then... Activation function activation results in spatial gating By utilizing the local receptive field of convolution, the network can learn the degree of local correlation or conflict between image features and prior features at each spatial location, thereby determining the strength of injecting prior information at that location. For example, if the two are inconsistent at motion boundaries, the spatial weights are reduced; if image features are unreliable in occluded regions, the spatial weights are increased.

[0044] Gating the channel and spatial gating By combining these elements, a channel-space joint gating system is obtained. , Channel-space joint gating Used to control the intensity and position of injecting prior trajectory information into image features.

[0045] Prior features of the trajectory Perform element-wise weighting, where weighting refers to the joint gating of channel and space. The weight value of each element and the trajectory prior features Multiplying the feature values ​​at the same position in the matrix yields the injected feature. : .

[0046] Inject features Image features are added using a one-way residual method. In this process, enhanced deep image features are obtained. : This completes the asymmetric prior injection of the image branches.

[0047] The unidirectional addition refers to the following: the trajectory mode participates in the image branch representation update as structural cue information, while the image branch features are not written into the deep representation of the trajectory branch in reverse, thereby reducing the pollution of the image backbone representation by trajectory drift and pseudo-structure.

[0048] The trajectory mode is preferably a priori feature of the trajectory. The image branch representation is preferably an image feature. The image backbone representation is preferably a hierarchical visual feature representation formed by the image branches at each encoding stage.

[0049] By constructing prior features of trajectory structure Furthermore, by employing a unidirectional and asymmetric approach to guide image branch learning, the system can reduce the reverse interference of distorted trajectories on image features from a mechanistic perspective. This allows the prior trajectory structure to serve only as guiding information to assist image branches in learning the road structure, rather than being unconstrainedly and symmetrically mixed with image features. This avoids the reverse interference of the original correct texture, boundary, and semantics of image branches due to trajectory distortion anomalies such as vehicle bumps and positioning drift, thus alleviating the problem of cross-modal mutual contamination and ensuring the stability of image features.

[0050] S6: Establish a long-range connectivity enhancement unit: Aggregate the deep enhancement features of the image branch after asymmetric structure prior injection with the deep features of the trajectory branch to obtain preliminary shared bottleneck features; apply a long-range connectivity enhancement module to the preliminary shared bottleneck features to enhance the long-range continuous representation of narrow roads, occluded roads and intersections to obtain enhanced shared bottleneck features.

[0051] Preferably, the initial shared bottleneck features are obtained by concatenating the deepest enhanced features of the image branch with the deepest features of the trajectory branch.

[0052] Preferably, the long-range connectivity enhancement module includes: performing a first bottleneck operation enhancement on the initial shared bottleneck features to expand the receptive field and enhance the deep context representation capability; performing long-range context aggregation on the enhanced initial shared bottleneck features along multiple different directions to obtain long-range responses in different directions; fusing the long-range responses in different directions to form a connectivity modulation map for representing the connectivity relationship between features; using the connectivity modulation map to perform element-wise modulation on the initial shared bottleneck features to obtain connectivity enhancement features; and performing a second bottleneck refinement operation on the connectivity enhancement features to further refine the deep context and enhance the structure of the modulated features, outputting the enhanced shared bottleneck features.

[0053] The first bottleneck operation may include one or more combinations of directional convolution, multi-scale pooling, and dilated convolution to aggregate mid-range and long-range contextual information and perform context compensation for narrow roads, occluded roads, and intersections. Through the first bottleneck operation, the receptive field is expanded and the continuous representation of narrow roads and intersections is enhanced in complex mining scenarios such as dust occlusion, deep pit shadows, weak boundaries, and sparse trajectories, thereby reducing the phenomenon of breakage and fragmentation.

[0054] Preferably, the enhanced features of the first bottleneck are aggregated along the long-range context in both the horizontal and vertical directions to explicitly model the extension relationship of the road in different directions, thereby enhancing the long-range structural association of the road skeleton. The second bottleneck operation is the same as the first bottleneck operation and may include one or more combinations of directional convolution, multi-scale pooling and dilated convolution, used to further refine the context and restructure the connectivity enhancement features after connectivity modulation, so as to improve the expressive power of shared bottleneck features in terms of road skeleton continuity, intersection area integrity and complex scene stability.

[0055] The aforementioned long-range connectivity enhancement module enables contextual compensation and connectivity restoration of road structures in situations where local image degradation, sparse trajectories, or incomplete evidence exists, thereby improving the completeness and continuity of road extraction results in complex open-pit mine scenarios.

[0056] like Figure 4 As shown: S7: Constructing an uncertainty-aware modal adaptive fusion unit: S71: The enhanced shared bottleneck features are fed into the image branch decoder and trajectory branch decoder respectively to obtain the image branch decoding features. Trajectory branch decoding features ; Image branch decoding features using an image classification head. The mapping is performed to obtain the image branch vector, denoted as... ; Decode features of trajectory branches using a trajectory classification head. By performing mapping, we obtain the trajectory branch vector, denoted as... .

[0057] S72: For the image branch vector and trajectory branch vector Non-negative activation is performed using a non-negative activation function to obtain evidence values ​​for each image modality. Evidence values ​​for trajectory modes ; Preferably, the non-negative activation function is selected function:

[0058] S73, Based on the evidence value of the image modality Evidence values ​​for trajectory modes Construct the corresponding modalities The distribution parameters are calculated, and the total strength of evidence and the uncertainty of the image modality are calculated sequentially. Uncertainty of trajectory modes .

[0059] Specifically, the corresponding evidence value for each modality is obtained by incrementing it by one. Distribution parameters:

[0060] in, Representing image modalities Distribution parameters; Representing trajectory modes Distribution parameters.

[0061] For any mode m, its The distribution parameter is denoted as .

[0062] For any modality m, its total strength of evidence is denoted as . Its expression is

[0063] Where k represents the number of categories, K represents the maximum number of categories, and preferably, in the road extraction task, K=2, corresponding to road category and non-road category; This indicates that mode m is in the kth category. parameter.

[0064] Based on overall strength of evidence Calculate the class prediction probability vector for mode m. and uncertainty Its expression is:

[0065] Correspondingly, when m is an image modality, i.e. At that time, the image modality prediction probability is obtained. and image modal uncertainty ; When m is a trajectory mode, i.e. When the trajectory mode prediction probability is obtained and Trajectory mode uncertainty .

[0066] Overall strength of evidence With uncertainty Inversely proportional. Total strength of evidence The larger the value, the greater the uncertainty. The smaller the value; the stronger the overall evidence. The smaller the value, the greater the uncertainty. The larger.

[0067] S74. Based on the image modal uncertainty and trajectory mode uncertainty Adaptively generated fusion coefficients, i.e., image modality fusion weights, are used to control the contribution ratio of each modality in the final prediction. Weights for trajectory mode fusion .

[0068] Specifically, the initial weighted response of each mode is first calculated based on the uncertainty:

[0069] in, This represents the preset minimum basic weight, used to retain the minimum contribution of each mode even when both modes have high uncertainty. Indicates the initial weights of the image modalities; This represents the initial weights of the trajectory modes.

[0070] The initial weight response is then normalized to obtain the image modality fusion weights. Weights for trajectory mode fusion Its expression is:

[0071] The adaptive rule is expressed as: when At that time, image modality weights The corresponding increase will improve the contribution of image modalities to the final prediction; when Trajectory mode weights The corresponding increase improves the contribution of the trajectory mode to the final prediction; when both modes have high uncertainty, the minimum basic weight is used. By retaining the minimum contribution of both modes and combining the shared structural information after the aforementioned long-range connectivity enhancement, the dominant influence of extremely unreliable modes on the final result is suppressed.

[0072] S75: Utilize the image modality fusion weights Weights for trajectory mode fusion Image mode prediction probability Trajectory mode prediction probability We perform weighted fusion to obtain the final fusion prediction result:

[0073] in, This represents the fused road prediction probability map, used to characterize the probability of each pixel belonging to a road category.

[0074] The fused road prediction probability map of the final output Thresholding is performed to obtain a binary road mask. This binary road mask is used to represent the final segmentation result between road areas and non-road areas.

[0075] Through the aforementioned uncertainty-aware modal adaptive fusion unit, the contribution of each modality in the final prediction can be adaptively adjusted to address the dynamic changes in the reliability of image and trajectory modalities at different spatial locations. For example, in areas where the apparent information is unreliable due to dust occlusion, shadow coverage, or weak boundaries, the trajectory modality weight is increased if it has high evidentiary strength to compensate for insufficient image information. Conversely, in areas where the trajectory exhibits positioning drift, local pseudostructure, or sparse sampling, the image modality weight is increased if it has high evidentiary strength to suppress the interference of distorted trajectories on the final result. This improves the robustness, continuity, and accuracy of road extraction results in complex open-pit mine scenarios.

[0076] S8: Optimize the entire network using a joint loss function.

[0077] The mining area road extraction network is optimized and trained using a joint loss function.

[0078] The joint loss function At least include segmentation loss loss of evidence Topology preservation loss and boundary consistency loss .

[0079]

[0080] Among them, segmentation loss A weighted combination of binary cross-entropy loss and Dice loss is used to supervise road region prediction and improve the accuracy of road region identification. The weight coefficients represent the weights that change dynamically with the number of training iterations (t), and are used to control the evidence regularization loss. In the joint loss function The percentage of contributions; Evidence regularity loss Including Kullback-Leibler divergence The regularization term is used to constrain the evidence values ​​obtained from the image branch and trajectory branch in S73 and their corresponding Dirichlet distribution parameters, thereby suppressing unreasonable high-confidence predictions in areas with insufficient information.

[0081] The regions with insufficient information include blurred image regions, shadow-occluded regions, sparse trajectory regions, and regions with significant trajectory drift. This loss method can improve the reliability of uncertainty estimation in the modality adaptive fusion unit. This represents a fixed weighting coefficient used to control the topology preservation loss. In the joint loss function The percentage of contributions; Topology Preservation Loss Based on soft skeleton consistency, clDice loss or other topological connectivity loss is used to constrain the continuity of the road skeleton and network connectivity of the road prediction probability graph P, maintaining the continuity of the road skeleton; enhancing the ability of the long-range connectivity enhancement unit in S6 to model the long-distance structure of the road; and promoting the trajectory prior injection in S5 to more effectively serve the road skeleton representation, thereby reducing road breakage and fragmentation.

[0082] This represents a fixed weighting coefficient used to control boundary consistency loss. In the joint loss function The percentage of contributions; Boundary consistency loss This includes boundary constraint terms based on spatial gradient response consistency, used to constrain the consistency between the predicted road probability map P and the actual road boundaries in terms of spatial location and geometry. Boundary consistency loss. It mainly functions in the decoding stage of S7 and the asymmetric structure prior injection process in S5, which can reduce boundary ambiguity, outward expansion and adhesion problems, and improve the geometric quality of the road edge area.

[0083] After training, the remote sensing images of the mining area to be tested and the vehicle positioning trajectory data are input into the mining area road extraction network, and the road extraction results can be output.

[0084] The output can be either a pixel-level binary road mask, or the binary road mask can be transformed into a road network centerline or a road vector map by skeleton extraction, vectorization, and topological connection processing.

[0085] The following describes the multimodal mining area road extraction method based on specific embodiments.

[0086] For a well-trained mining area road extraction network, the input is a remote sensing image of the target open-pit mine area and vehicle positioning trajectory data corresponding to the remote sensing image space. The image features and trajectory features are extracted sequentially by the dual-branch feature extraction network of the mining area road extraction network. The asymmetric structure prior injection module guides the image branch learning, and with the trajectory branch as the guide, clear and stable image features are constructed. The long-range connectivity enhancement module strengthens the continuous representation of narrow roads and intersection areas. The uncertainty-aware modality adaptive fusion module adaptively allocates weights and outputs a stable and clear predicted binary road mask, road network centerline or road vector map.

[0087] Of course, those skilled in the art will recognize that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0088] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0089] The technologies, shapes, and structures not described in detail in this invention are all known technologies.

Claims

1. An uncertainty-aware multi-modal mine road extraction method, characterized in that, Includes the following steps: S1: Acquire remote sensing images of the target open-pit mine area and vehicle positioning trajectory data corresponding to the space of the remote sensing images, as input data for the multimodal mine area road extraction network; S2: Constructing a trajectory heatmap generation unit: Performing spatial registration between the trajectory and the image and constructing a trajectory heatmap; S3: Constructing a prior map of a strip trajectory: Using the trajectory heatmap as input, a stable and clear prior map of a strip trajectory is constructed through image processing; S4: Establish a dual-branch feature extraction network: Construct an image branch and a trajectory branch based on remote sensing images and a strip trajectory prior map, and extract image features and trajectory prior features respectively; S5: Constructing an asymmetric prior injection unit: Based on a dual-branch feature extraction network, guide weights are constructed to control the channel dimension and spatial position when trajectory prior features are injected into image features, and the image branches are guided to learn the road structure in a unidirectional and asymmetric manner. S6: Constructing a long-range connectivity enhancement unit: Aggregate the deep features of the image branch and the deep features of the trajectory branch after asymmetric structure prior injection to generate preliminary shared bottleneck features; apply a long-range connectivity enhancement module to the preliminary shared bottleneck features to enhance long-range continuous expression capabilities and obtain enhanced shared bottleneck features; S7: Constructing an uncertainty-aware modal adaptive fusion unit: Decode the enhanced shared bottleneck features, construct a modal adaptive fusion unit based on the decoded features, adaptively adjust the contributions of trajectory modality and image modality in the final prediction, and obtain the final road prediction result; S8: Use a joint loss function to optimize the training of the entire network.

2. The multimodal mining area road extraction method with uncertainty perception as described in claim 1, characterized in that, In step S4, the image branch and the trajectory branch are both feature extraction networks, and respectively have an image encoder and a trajectory encoder built-in, and have the same structure, both using a hierarchical convolutional encoder; the image encoder and the trajectory encoder respectively output image features and trajectory prior features at the i-th encoding stage.

3. The multimodal road extraction method for uncertainty perception in mining areas as described in claim 2, characterized in that, In step S5, the guiding weights include channel guiding weights and spatial guiding weights; the channel guiding weights are based on image features. Trajectory Prior Features Global average pooling is used to extract the global context descriptors of each component. The two descriptors are then fused and input into a fully connected network, where they are activated by an activation function to obtain channel gating. Spatial guided weights are based on image features. Trajectory Prior Features The concatenation and convolution along the channel dimension, followed by activation by an activation function, yields a spatially gated array. ; Gating the channel and spatial gating By combining these elements, a combined channel-space gating system is obtained. : .

4. The multimodal mining area road extraction method with uncertainty perception as described in claim 3, characterized in that, The image branch learns road structure using a unidirectional, asymmetric approach, including: joint gating of channels and space. Trajectory Prior Features Weighted injection features Injection features Image features are added using a one-way residual method. middle.

5. The multimodal mining area road extraction method with uncertainty perception as described in claim 4, characterized in that, In step S6, the preliminary shared bottleneck feature is obtained by concatenating the deepest enhanced feature of the image branch after asymmetric structure prior injection with the deepest feature of the trajectory branch.

6. The multimodal mining area road extraction method with uncertainty perception as described in claim 5, characterized in that, The long-range connectivity enhancement module includes: performing a first bottleneck operation on the shared bottleneck features and performing long-range context aggregation operations along multiple different directions to generate a connectivity modulation map for characterizing the connectivity relationships between features; fusing the long-range responses in different directions to generate a connectivity modulation map; using the connectivity modulation map to perform element-wise modulation on the initial shared bottleneck features to obtain connectivity enhancement features; and performing a second bottleneck refinement operation on the connectivity enhancement features to output the enhanced shared bottleneck features.

7. The multimodal mining area road extraction method with uncertainty perception as described in claim 6, characterized in that, Both the first and second bottleneck operations include one or more combinations of directional convolution, multi-scale pooling, and dilated convolution.

8. The multimodal mining area road extraction method with uncertainty perception as described in claim 1, characterized in that, In step S7, the modality adaptive fusion unit includes mapping the decoded features to a logits graph and performing non-negative activation to generate evidence values ​​for each modality; constructing distribution parameters based on the evidence values; calculating the total evidence strength; and calculating the category prediction probability vector and uncertainty for each modality based on the total evidence strength; and adaptively generating fusion weights to control the contribution ratio of each modality in the final prediction based on the uncertainty of each modality.

9. The multimodal mining area road extraction method with uncertainty perception as described in claim 8, characterized in that: The fusion unit includes: weighting the fusion coefficients with the category prediction probability vectors of each modality to obtain the final fusion prediction result.

10. The multimodal mining area road extraction method with uncertainty perception as described in claim 1, characterized in that: The joint loss function At least include segmentation loss loss of evidence Topology preservation loss and boundary consistency loss : in, The weight coefficients represent the weights that change dynamically with the number of training iterations (t), and are used to control the evidence regularization loss. In the joint loss function The percentage of contributions; and These represent fixed weighting coefficients, used to control the topology preservation loss. and boundary consistency loss In the joint loss function The percentage of contribution.

Images