A deep learning blind lane detection algorithm based on multi-scale feature fusion

Abstract

The application discloses a kind of deep learning blind path detection algorithm based on multi-scale feature fusion, comprising the following steps: collecting blind path data and marking, collecting multi-scale features, fusion model architecture design, creating blind path identification model, extracting blind path model key area features, enhancing the blind path edge features of feature extraction network, training the blind path identification model of key feature area, reconstructing blind path identification model, optimizing and post-processing the reconstructed model, verifying and evaluating experimental results;The application collects and marks multiple sources of data, strengthens the generalization ability of the model, covers diverse scenarios in data collection, fully simulates the actual use environment, collects different dimensions in layers, ensures data richness and diversity, uses the way of combining pixel-by-pixel segmentation and bounding box detection in the marking link, ensures balanced data distribution, so that the model can still maintain stable detection performance in complex environment, significantly enhances the generalization ability.

Description

Technical Field

[0001] This invention relates to the field of blind path detection algorithm technology, specifically a deep learning blind path detection algorithm based on multi-scale feature fusion. Background Technology

[0002] With the continuous development of civilized society, society is paying more and more attention to the travel issues of people with disabilities, especially the barrier-free travel of visually impaired people. At present, guide canes and guide dogs are still the two main tools to help them travel. Although guide canes are popular because of their affordable price and simple operation, their detection range is limited and cannot fully guarantee the safety of visually impaired people. In comparison, although guide dogs can provide more comprehensive safety protection, the training process is long and costly, and the success rate is not satisfactory. In recent years, deep learning algorithms have shown good performance in computer tasks such as target recognition and target classification. Because of their powerful feature extraction and generalization capabilities, as well as the advantages of not requiring complex feature engineering, they have been applied to various fields.

[0003] A search revealed a design method and deployment system for a deep learning-based tactile paving recognition model on the Chinese patent website (Announcement No.: CN118823718A). This method includes constructing a tactile paving dataset, preprocessing and data augmenting the tactile paving images in the dataset, creating a tactile paving recognition model based on the classic deep learning object detection algorithm YOLOv5-Seg, improving the model by replacing the backbone network ResNet101 with ShuffleNet, replacing the coupling head with a decoupling head (used for edge detection in the tactile paving images), and adding an SEA attention mechanism between the last layer of the feature extraction network and the FPN. The improved tactile paving recognition model is trained on the preprocessed and data-augmented dataset to obtain the tactile paving images to be recognized. The trained model then identifies the tactile paving in the images, making the model more lightweight and reducing the deployment difficulty to some extent, while improving recognition speed and accuracy. However, it still has several shortcomings.

[0004] The system suffers from insufficient multi-scale feature processing capabilities, limiting its performance in small target detection. It only replaces the backbone network with ShuffleNet and decouples the head edge detection, without explicitly designing a multi-scale feature extraction module. Although the SEA attention mechanism can optimize feature weights, the lack of hierarchical fusion of multi-scale semantic and detailed information may lead to insufficient accuracy in detecting blind paths for small targets.

[0005] The system has a single edge feature enhancement mechanism, insufficient segmentation boundary refinement, and only uses a decoupling head for edge detection. It does not perform cross-layer fusion with deep semantic features, resulting in insufficient interaction between edge information and semantic information. It lacks multi-scale edge fusion strategies and attention-guided edge enhancement mechanisms. In complex backgrounds such as texture interference or partial occlusion, the continuity and accuracy of the blind path boundary are low.

[0006] The system's strategy for balancing lightweighting and accuracy is not comprehensive enough. It only improves speed by replacing the backbone network with ShuffleNet and decoupling the head design, but does not optimize the feature fusion module for lightweighting, nor does it introduce group convolution or dilated convolution to reduce computation. In addition, it does not mention quantization or model compression techniques. When deployed on embedded devices, it may face problems such as high computational resource consumption or insufficient real-time performance.

[0007] The system lacks data diversity and targeted training strategies. It only mentions preprocessing and data augmentation, but does not specify the diversity of data collection scenarios. The training strategy does not emphasize multi-scale input or small target augmentation, which may lead to weak robustness of the model in extreme environments such as low light or severe occlusion. Summary of the Invention

[0008] The purpose of this invention is to provide a deep learning blind path detection algorithm based on multi-scale feature fusion, in order to solve the problems mentioned in the background art, such as insufficient multi-scale feature processing capability, limited small target detection performance, single edge feature enhancement mechanism, insufficient segmentation boundary refinement, insufficient balance between model lightweighting and accuracy strategy, and insufficient data diversity and training strategy targeting.

[0009] Therefore, this invention provides a deep learning-based blind path detection algorithm based on multi-scale feature fusion, comprising the following steps:

[0010] S1. Collect and label data of tactile paving;

[0011] S2. Collect multi-scale features and integrate them into the model architecture design;

[0012] S3. Initially create a blind path recognition model;

[0013] S4. Extract key area features from the tactile paving model;

[0014] S5. Enhance the edge features of the blind path in the feature extraction network;

[0015] S6. Train the blind path recognition model for key feature areas;

[0016] S7. Reconstruct the tactile paving identification model;

[0017] S8. Optimize and post-process the reconstructed model;

[0018] S9. Verify and evaluate the experimental results.

[0019] Preferably, in step S1, the specific steps for collecting and labeling tactile paving data are as follows:

[0020] (1) Clearly define the diverse scenarios for data collection: covering indoor and outdoor areas such as main urban roads, sidewalks, shopping malls, subway stations, and campuses, to simulate different environments of tactile paving in actual use, accurately collect tactile paving images, obtain high-resolution, multi-angle tactile paving image data, and record metadata such as shooting equipment parameters, geographical location, time, and weather during the collection process;

[0021] (2) Data collection implementation: Data collection is carried out at different times and under different weather conditions to obtain images of the tactile paving under various lighting and climate conditions. Based on the different types, materials, colors, and integrity of the tactile paving, targeted data collection is carried out to ensure the richness and diversity of the data.

[0022] (3) Label the collected data: Label the collected images pixel by pixel or bounding box by bounding box. For semantic segmentation tasks, the boundaries of the blind path area need to be accurately delineated. For object detection tasks, the blind path objects need to be accurately selected and their categories and attributes need to be marked. The labeled data should be organized to ensure the manageability and traceability of the data. The labeled data should be divided into training set, validation set and test set. When dividing, ensure that the data distribution of different scenes, lighting conditions and blind path types in each set is balanced to meet the needs of subsequent model training and evaluation.

[0023] Preferably, in step S2, the specific steps for collecting multi-scale features and designing the fusion model architecture are as follows:

[0024] (1) Constructing a multi-scale feature extraction module: In the backbone of the basic network, set up parallel branches of dilated convolution with different dilation rates. The formula for the output feature map of each branch is as follows: In this model, Conv2D is a 2D convolution operation, BN is batch normalization, and ReLU is the activation function. By using different dilation rates, each branch can capture features from different receptive fields, achieving multi-scale feature extraction. Then, a feature pyramid network is built. Utilizing feature maps from different levels in the base network, deep semantic information is fused with shallow detail information through upsampling and lateral connections to construct the feature pyramid, providing a foundation for subsequent multi-scale detection. The fusion formula for feature maps from different levels is as follows: ,in For deep, low-resolution feature maps, Upsample is the upsampling operation. To generate shallow high-resolution feature maps, a parallel multi-branch network structure is designed. Each branch uses a different size convolution kernel or a different downsampling rate. Different convolution kernel sizes and downsampling rates enable each branch to process the image at different scales and output feature maps at different scales, thus enriching the feature representation.

[0025] (2) The feature maps of different scales are spliced ​​in the channel dimension to expand the feature dimension and obtain more comprehensive feature information. After splicing, the number of channels is compressed by 1×1 convolution to reduce the amount of computation. At the same time, the spliced ​​features are fused and the feature maps of different scales are added element by element. However, different weights are assigned to each feature map before addition. Channel attention mechanism and spatial attention mechanism are introduced. The channel attention mechanism weights the feature map according to the importance of each channel to enhance the key channel features. The spatial attention mechanism focuses on the spatial area where the blind path is located in the feature map and suppresses irrelevant background information, thereby achieving more effective multi-scale feature fusion.

[0026] Preferably, in step S3, the specific steps for initially creating the tactile paving recognition model are as follows:

[0027] (1) Design the decoder: adopt a semantic segmentation architecture and design the decoder structure, such as the symmetric decoder of U-Net. By transposing convolution or upsampling combined with convolution operation, the feature map resolution is gradually restored, and the fused multi-scale features are mapped to the segmentation result of the same size as the input image.

[0028] (2) Determine the output layer and loss function: The output layer uses the Softmax activation function to convert the decoder output into a pixel-level class probability distribution. The output dimension is H×W×C, where H and W are the image height and width, and C is the number of classes.

[0029] (3) Define the loss function: use cross-entropy loss combined with Dice loss, the formula is as follows ,in Measure the difference between the predicted probability and the true label. Focus on the overlap of the segmented regions, where α and β are weighting coefficients;

[0030] (4) Finally, use a deep learning framework to instantiate the modules designed above and build a complete blind path recognition model. After the model is created, perform forward propagation by inputting random data to check whether the output dimension of the model meets expectations and ensure that the data flows smoothly between modules. If the output dimension is wrong or the model reports an error, check the network layer parameter settings, connection methods, etc., and make targeted adjustments and optimizations to ensure the correctness of the model structure.

[0031] Preferably, in step S4, the specific steps for extracting the key area features of the tactile paving model are as follows:

[0032] (1) Based on the morphological features of the tactile paving in the image and the annotation information of the dataset, the average size, aspect ratio, color distribution range and other prior knowledge of the tactile paving area are statistically analyzed to provide a basis for the subsequent key area selection. Then, during the forward propagation of the model, the feature map set output by the multi-scale feature extraction module or feature fusion module is obtained. Using the channel attention mechanism, the channel dimension of each feature map is weighted and the channel weight is calculated. ,in and This is a fully connected layer, where H and W are the height and width of the feature map, respectively, and are weighted... Weighting the feature map F highlights the channel features related to the tactile paving. ;

[0033] (2) Using the anchor box concept in object detection, based on the prior size information of the blind path, a series of candidate region boxes are generated on the feature map. For each candidate region box Calculate its response value on the feature map. ,in The pixel coordinates within the bounding box are determined by setting a response threshold. Filter out response values The bounding boxes are used as key region candidates;

[0034] (3) For the selected key region candidates, feature extraction and aggregation are performed on feature maps at different scales. Assuming that the feature map at scale s... Above, the area frame Perform bilinear interpolation to unify the feature size to a fixed size M×N to obtain the feature vector. Concat represents concatenation along the channel dimension to generate a key region feature vector containing multi-scale information. .

[0035] Preferably, in step S5, the specific steps for enhancing the tactile paving edge features of the feature extraction network are as follows:

[0036] (1) Extracting edge features: In the multi-scale feature map, select the shallow feature map with rich spatial details, use the Sobel operator or Canny operator to perform edge detection to obtain edge features, then construct the edge enhancement module, fuse the edge features with the original feature map, and extract edge features on feature maps of different scales respectively, and design a multi-scale edge fusion strategy.

[0037] (2) Based on accurate edge features, a binary classification loss of edge pixels and non-edge pixels is adopted, combined with bounding box regression loss: ,in For edge classification loss, For edge location regression loss, This is the balance coefficient;

[0038] (3) Introduce a spatial attention mechanism to focus on feature enhancement in the edge region. Suppress interference from non-edge regions through the attention mechanism to enhance edge feature expression. Design an edge-semantic feature fusion module to combine the enhanced edge features with deep semantic features. Finally, in the model training stage, combine the main task loss and edge constraint loss and optimize the model parameters through gradient backpropagation so that the model can enhance its ability to capture the edge features of the blind path while learning semantic information.

[0039] Preferably, in step S6, the specific steps for training the tactile paving recognition model for key feature regions are as follows:

[0040] (1) Determine the hyperparameters for training, including learning rate, batch size, and number of training epochs. At the same time, select the Adam optimizer and set its hyperparameters. , , ;

[0041] (2) Use a deep learning framework to save the training set data and perform real-time data augmentation. Then normalize the image data and keep the label data in its original labeling format. In order to prevent overfitting, the shuffle parameter of the data loader can be enabled to shuffle the data order in each epoch.

[0042] (3) The preprocessed image data is input into the blind path recognition model, and passes through the multi-scale feature extraction module, feature fusion module, and decoder in sequence to obtain the prediction result of the model. Then, the loss value is calculated according to the model task type, and combined with the cross-entropy loss. and loss The total loss is Where α and β are weighting coefficients;

[0043] (4) The gradient of the loss function with respect to all trainable parameters of the model is calculated by backpropagation algorithm. Then, the parameters are updated according to the Adam optimizer. During the parameter update process, the optimizer adjusts the model parameters to minimize the loss function based on the calculated gradient and the set learning rate. Finally, the model is evaluated using validation set data after each epoch of training.

[0044] Preferably, in step S7, the specific steps for reconstructing the tactile paving recognition model are as follows:

[0045] (1) Use model analysis tools to count the parameters and computational cost of each layer, identify layers that contribute little to performance but consume a lot of resources. For convolutional layers, if the weights of most output channels of a certain layer are close to 0, consider pruning that channel. For fully connected layers, if the number of neurons is too large and the improvement in final accuracy is limited, the number of neurons can be reduced. At the same time, if the multi-scale feature fusion module in the original model is computationally complex and has poor performance, it can be replaced with a more efficient structure to enhance the feature fusion effect. Then, add auxiliary modules to address the difficulties in blind path detection.

[0046] (2) Initialize the backbone network with model weights pre-trained on a large-scale image dataset, and then fine-tune it on the blind path detection dataset. When fine-tuning, the parameters of the backbone network can be fixed first, and only the newly added layers can be trained. After the model is stable, some layers of the backbone network can be gradually released for joint training. Hyperparameters such as learning rate, batch size, and loss function weight coefficients can be readjusted by grid search, random search or Bayesian optimization. Then, structured pruning or unstructured pruning methods can be used to remove unimportant connections or parameters.

[0047] (3) Adjust the feature fusion order: Change the connection order or fusion method of the multi-scale feature extraction module and the fusion module, and redesign the branch structure: For multi-branch feature extraction networks, re-plan the number of branches, convolution kernel size and dilation rate;

[0048] (4) Train multiple blind path recognition models with different structures or different initializations. In the inference stage, use the averaging method, voting method or weighted averaging method to fuse the prediction results of multiple models, average the probability maps predicted by multiple models, and then obtain the final result through threshold segmentation to improve the stability and accuracy of the model. Finally, after completing the model structure and parameter adjustment, use the adjusted model to retrain on the training set.

[0049] Preferably, in step S8, the specific steps for optimizing and post-processing the reconstructed model are as follows:

[0050] (1) Lightweight compression of the model: Structured pruning is used to delete unimportant channels or connections in the model to reduce model complexity. INT8 quantization technology is used to convert model parameters from floating point to integer to reduce memory usage and computation. The model size is reduced while ensuring accuracy, and the inference speed is improved.

[0051] (2) Then, the inference framework is accelerated: the reconstructed model is converted into the ONNX format. With the help of inference frameworks such as TensorRT and ONNXRuntime, the model computation graph is optimized through optimization techniques such as layer fusion and dynamic memory management to adapt to different hardware platforms and achieve efficient inference of the model on GPUs, embedded devices, etc.

[0052] (3) Secondly, optimize the segmentation results: use the Otsu algorithm to automatically calculate the optimal segmentation threshold, binarize the probability map output by the model, and then connect the adjacent blind path areas through morphological operations such as dilation and erosion, remove isolated noise points, and refine the blind path boundary.

[0053] (4) Finally, the detection results are calibrated: Soft-NMS or DIoU-NMS algorithm is used to replace the traditional non-maximum suppression, optimize the bounding box screening, dynamically adjust the confidence threshold according to the size of the detection box, improve the detection rate of small-sized blind paths, and reduce the false detection of large-sized blind paths.

[0054] Preferably, the specific steps for verifying and evaluating the experimental results in the nine steps are as follows:

[0055] (1) Calculate and evaluate the indicators: calculate the proportion of pixels that are correctly predicted to the total number of pixels. ,in The number of pixels in category i that were correctly predicted. Let i be the number of pixels that are predicted to be class j from the true class i. Then calculate the IoU for each class and take the average: , Where TP represents true examples, FP represents false negative examples, k represents the number of classes, and finally, precision P and recall R are considered: , , ;

[0056] (2) By gradually removing model components, compare the performance changes under different configurations to verify the effectiveness of each component. For example, compare the basic model, basic model + multi-scale feature fusion and basic model + multi-scale feature fusion + edge enhancement, and then compare with mainstream algorithms on the same test set. Record the metrics of each algorithm and draw a performance comparison table or bar chart.

[0057] (3) Visualize the prediction results: Overlay the blind path area predicted by the model onto the original image, compare the actual annotation with the prediction results, intuitively display the detection effect, and verify the effectiveness of the algorithm.

[0058] The present invention proposes a deep learning-based blind path detection algorithm based on multi-scale feature fusion, the advantages of which are:

[0059] This invention features multi-source data acquisition and annotation, enhancing the model's generalization ability. The data acquisition covers diverse scenarios, including images of tactile paving under different time periods and weather conditions, comprehensively simulating the actual usage environment. Layered acquisition is conducted for different dimensions to ensure data richness and diversity. The annotation process combines pixel-by-pixel segmentation with bounding box detection. At the same time, the training set, validation set, and test set are scientifically divided to ensure balanced data distribution, effectively avoiding model overfitting. This allows the model to maintain stable detection performance even in complex environments, significantly enhancing its generalization ability.

[0060] This invention employs a multi-scale feature fusion architecture to optimize feature representation capabilities and construct a multi-scale feature extraction module. By leveraging dilated convolution parallel branches, feature pyramid networks, and multi-branch network structures, it effectively extracts features from different sensory fields, fully integrating deep semantic information with shallow detail information. This addresses the detection challenges caused by scale variations in tactile paving in images. In the feature fusion stage, a channel and spatial attention mechanism is used to automatically focus on key channels and spatial regions of the tactile paving, suppressing background noise interference. Through feature map stitching and 1×1 convolution compression, the feature dimension and computational load are balanced, significantly improving the model's ability to represent tactile paving features. Whether it's the details of small-sized tactile paving or the overall structure of large-sized tactile paving, it can accurately capture the features, effectively improving capture accuracy.

[0061] This invention enhances key regions and edges to improve detection accuracy. Based on prior knowledge of tactile paving, it uses the anchor frame concept to locate key regions, reducing unnecessary computation and improving detection efficiency. Through channel attention weighting and multi-scale feature aggregation, it enhances the feature response of tactile paving regions, especially improving the detection capability of small or blurred tactile paving. It extracts edge details and optimizes the tactile paving boundary segmentation by combining edge-semantic feature fusion module with deep semantics. It also optimizes edge classification and regression loss together to reduce the problem of blurred segmentation edges, making the detection results more accurate and effectively reducing the false detection and false detection rates.

[0062] This invention optimizes and post-processes the model to achieve efficient deployment and application. During training, it employs joint optimization of cross-entropy loss and Dice loss to balance classification accuracy and region overlap. Combined with the Adam optimizer and data augmentation strategies, it accelerates model convergence and reduces overfitting. Through structural pruning, quantization compression, and other techniques, it lightweights the model, reducing its complexity and computational load, making it suitable for embedded devices and meeting real-time detection requirements. Post-processing utilizes Otsu threshold segmentation, morphological operations, and an improved NMS algorithm to eliminate noise in the detection results and optimize bounding box selection, further improving the quality and reliability of the detection results. This provides strong support for practical applications such as intelligent guide devices for the blind and urban road inspection.

[0063] This invention undergoes scientific experimental verification to ensure the reliability of the algorithm. Ablation experiments are used to quantify the contribution of each module to performance, verifying the rationality and effectiveness of the algorithm design. A horizontal comparison with mainstream algorithms is conducted to demonstrate the superiority of this algorithm from multiple indicators. Combined with visualization results, the detection effect is presented intuitively, providing sufficient evidence for the reliability of the algorithm and enhancing its credibility in academic and applied fields. Attached Figure Description

[0064] Figure 1 This is a flowchart of the deep learning-based blind path detection algorithm based on multi-scale feature fusion of the present invention. Detailed Implementation

[0065] The technical solution of the present invention will now be described in detail through specific embodiments.

[0066] Please see Figure 1 This invention provides a deep learning-based blind path detection algorithm based on multi-scale feature fusion, comprising the following steps:

[0067] S1. Collect and label tactile paving data; the specific steps for collecting and labeling tactile paving data are as follows:

[0068] (1) Clearly define the diverse scenarios for data collection: covering indoor and outdoor areas such as main urban roads, sidewalks, shopping malls, subway stations, and campuses, to simulate different environments of tactile paving in actual use, accurately collect tactile paving images, obtain high-resolution, multi-angle tactile paving image data, and record metadata such as shooting equipment parameters, geographical location, time, and weather during the collection process;

[0069] (2) Data collection implementation: Data collection is carried out at different times and under different weather conditions to obtain images of the tactile paving under various lighting and climate conditions. Based on the different types, materials, colors, and integrity of the tactile paving, targeted data collection is carried out to ensure the richness and diversity of the data.

[0070] (3) Label the collected data: Label the collected images pixel by pixel or bounding box by bounding box. For semantic segmentation tasks, the boundaries of the blind path area need to be accurately delineated. For object detection tasks, the blind path objects need to be accurately selected and their categories and attributes need to be marked. The labeled data should be organized to ensure the manageability and traceability of the data. The labeled data should be divided into training set, validation set and test set. When dividing, ensure that the data distribution of different scenes, lighting conditions and blind path types in each set is balanced to meet the needs of subsequent model training and evaluation.

[0071] In this embodiment, step S1 constructs a high-quality tactile paving detection dataset through multi-dimensional data collection, standardized annotation, and scientific dataset partitioning. The dataset covers diverse scenarios such as urban main roads, shopping malls, and campuses, encompassing different times, weather conditions, and tactile paving types, materials, and completeness. Metadata is recorded to ensure the authenticity of the scenarios. Pixel-by-pixel segmentation or bounding box annotation is used to mark categories and attributes, standardizing data management. The training set, validation set, and test set are divided according to the principle of balance to ensure consistent data distribution across sets. This provides comprehensive and reliable data support for model training and evaluation, enhancing the model's generalization ability to complex real-world scenarios.

[0072] S2. Collect multi-scale features and fuse them for model architecture design; the specific steps for collecting multi-scale features and fusing them for model architecture design are as follows:

[0073] (1) Constructing a multi-scale feature extraction module: In the backbone of the basic network, set up parallel branches of dilated convolution with different dilation rates. The formula for the output feature map of each branch is as follows: In this model, Conv2D is a 2D convolution operation, BN is batch normalization, and ReLU is the activation function. By using different dilation rates, each branch can capture features from different receptive fields, achieving multi-scale feature extraction. Then, a feature pyramid network is built. Utilizing feature maps from different levels in the base network, deep semantic information is fused with shallow detail information through upsampling and lateral connections to construct the feature pyramid, providing a foundation for subsequent multi-scale detection. The fusion formula for feature maps from different levels is as follows: ,in For deep, low-resolution feature maps, Upsample is the upsampling operation. To generate shallow high-resolution feature maps, a parallel multi-branch network structure is designed. Each branch uses a different size convolution kernel or a different downsampling rate. Different convolution kernel sizes and downsampling rates enable each branch to process the image at different scales and output feature maps at different scales, thus enriching the feature representation.

[0074] (2) Feature maps of different scales are spliced ​​in the channel dimension to expand the feature dimension and obtain more comprehensive feature information. After splicing, the number of channels is compressed by 1×1 convolution to reduce the amount of computation. At the same time, the spliced ​​features are fused and the feature maps of different scales are added element by element. However, before the addition, different weights are assigned to each feature map. Channel attention mechanism and spatial attention mechanism are introduced. Channel attention mechanism weights according to the importance of each channel of the feature map to enhance the key channel features. Spatial attention mechanism focuses on the spatial area where the blind path is located in the feature map and suppresses irrelevant background information, thereby achieving more effective multi-scale feature fusion.

[0075] In this embodiment, step S2 achieves efficient extraction and fusion of multi-scale features through multi-level design. It constructs dilated convolution parallel branches, feature pyramid networks, and multi-branch structures in the backbone of the basic network. It uses different dilation rates, convolution kernel sizes, and downsampling rates to capture cross-scale features from local details to global semantics, solving the detection problem of blind paths caused by scale changes. In the feature fusion stage, it adopts a combination of channel splicing and 1×1 convolution compression to balance feature dimensions and computational load, and introduces channel and spatial attention mechanisms to dynamically focus on key feature channels and spatial regions of blind paths, suppress background interference, realize adaptive weighted fusion of multi-scale features, and improve the model's ability to express and detect blind paths of different scales.

[0076] S3. Initial creation of a tactile paving recognition model; the specific steps for initially creating a tactile paving recognition model are as follows:

[0077] (1) Design the decoder: adopt a semantic segmentation architecture and design the decoder structure, such as the symmetric decoder of U-Net. By transposing convolution or upsampling combined with convolution operation, the feature map resolution is gradually restored, and the fused multi-scale features are mapped to the segmentation result of the same size as the input image.

[0078] (2) Determine the output layer and loss function: The output layer uses the Softmax activation function to convert the decoder output into a pixel-level class probability distribution. The output dimension is H×W×C, where H and W are the image height and width, and C is the number of classes.

[0079] (3) Define the loss function: use cross-entropy loss combined with Dice loss, the formula is as follows ,in Measure the difference between the predicted probability and the true label. Focus on the overlap of the segmented regions, where α and β are weighting coefficients;

[0080] (4) Finally, use a deep learning framework to instantiate the modules designed above and build a complete blind path recognition model. After the model is created, perform forward propagation by inputting random data to check whether the output dimension of the model meets expectations and ensure that the data flows smoothly between modules. If the output dimension is wrong or the model reports an error, check the network layer parameter settings, connection methods, etc., and make targeted adjustments and optimizations to ensure the correctness of the model structure.

[0081] In this embodiment, step S3 constructs a complete blind path recognition model framework based on a semantic segmentation architecture. By designing a symmetric decoder, transposed convolution and upsampling operations are used to restore the feature map resolution, realizing the mapping from multi-scale features to pixel-level segmentation results. The output layer uses the Softmax function to generate pixel category probability distributions to adapt to the requirements of semantic segmentation tasks. The loss function combines cross-entropy and Dice loss to balance classification accuracy and region overlap, thereby improving segmentation accuracy. Finally, each module is instantiated through a deep learning framework, and random data is input to verify the correctness of the model structure, ensuring smooth data flow and laying the foundation for subsequent model training.

[0082] S4. Extract key area features from the tactile paving model; the specific steps for extracting key area features from the tactile paving model are as follows:

[0083] (1) Based on the morphological features of the tactile paving in the image and the annotation information of the dataset, the average size, aspect ratio, color distribution range and other prior knowledge of the tactile paving area are statistically analyzed to provide a basis for the subsequent key area selection. Then, during the forward propagation of the model, the feature map set output by the multi-scale feature extraction module or feature fusion module is obtained. Using the channel attention mechanism, the channel dimension of each feature map is weighted and the channel weight is calculated. ,in and This is a fully connected layer, where H and W are the height and width of the feature map, respectively, and are weighted... Weighting the feature map F highlights the channel features related to the tactile paving. ;

[0084] (2) Using the anchor box concept in object detection, based on the prior size information of the blind path, a series of candidate region boxes are generated on the feature map. For each candidate region box Calculate its response value on the feature map. ,in The pixel coordinates within the bounding box are determined by setting a response threshold. Filter out response values The bounding boxes are used as key region candidates;

[0085] (3) For the selected key region candidates, feature extraction and aggregation are performed on feature maps at different scales. Assuming that the feature map at scale s... Above, the area frame Perform bilinear interpolation to unify the feature size to a fixed size M×N to obtain the feature vector. Concat represents concatenation along the channel dimension to generate a key region feature vector containing multi-scale information. ;

[0086] In this embodiment, step S4 accurately extracts key features of the tactile paving by combining prior knowledge and attention mechanisms. First, based on statistical priors of the tactile paving morphology features, a benchmark is provided for key area selection. Then, the channel attention mechanism is used to weight the multi-scale feature maps to enhance the response of the tactile paving-related channels. Next, candidate regions are generated on the feature maps using the anchor box concept. High-probability tactile paving regions are located by calculating response values ​​and filtering thresholds. Finally, the selected key regions are subjected to size unification and feature aggregation on feature maps of different scales. By splicing cross-scale features, a feature vector containing multi-dimensional information is generated, achieving efficient representation of key areas of the tactile paving, reducing invalid background interference, and improving detection efficiency and accuracy.

[0087] S5. Enhance the tactile paving edge features of the feature extraction network; the specific steps for enhancing the tactile paving edge features of the feature extraction network are as follows:

[0088] (1) Extracting edge features: In the multi-scale feature map, select the shallow feature map with rich spatial details, use the Sobel operator or Canny operator to perform edge detection to obtain edge features, then construct the edge enhancement module, fuse the edge features with the original feature map, and extract edge features on feature maps of different scales respectively, and design a multi-scale edge fusion strategy.

[0089] (2) Based on accurate edge features, a binary classification loss of edge pixels and non-edge pixels is adopted, combined with bounding box regression loss: ,in For edge classification loss, For edge location regression loss, This is the balance coefficient;

[0090] (3) Introduce a spatial attention mechanism to focus on feature enhancement in the edge region. Suppress interference from non-edge regions through the attention mechanism to enhance edge feature expression. Design an edge-semantic feature fusion module to combine the enhanced edge features with deep semantic features. Finally, in the model training stage, combine the main task loss and edge constraint loss and optimize the model parameters through gradient backpropagation so that the model can enhance its ability to capture the edge features of the blind path while learning semantic information.

[0091] In this embodiment, step S5 enhances the edge features of the tactile paving through a multi-strategy collaborative approach. First, edge information is extracted from the shallow details of the multi-scale feature map using the Sobel or Canny operator, and an enhancement module is constructed to fuse the edge features with the original features. At the same time, a multi-scale fusion strategy is designed to improve the edge integrity. Then, a loss function combining binary classification and bounding box regression is used to accurately constrain the edge position and category prediction. Finally, a spatial attention mechanism is introduced to focus on the edge region, suppress background interference, and fuse deep semantic features. By jointly optimizing the parameters through the main task loss and the edge constraint loss, the model can significantly enhance its ability to capture edge details while learning the semantics of the tactile paving, thereby improving the accuracy and continuity of the segmentation boundary.

[0092] S6. Train the tactile paving recognition model for key feature areas; the specific steps for training the tactile paving recognition model for key feature areas are as follows:

[0093] (1) Determine the hyperparameters for training, including learning rate, batch size, and number of training epochs. At the same time, select the Adam optimizer and set its hyperparameters. , , ;

[0094] (2) Use a deep learning framework to save the training set data and perform real-time data augmentation. Then normalize the image data and keep the label data in its original labeling format. In order to prevent overfitting, the shuffle parameter of the data loader can be enabled to shuffle the data order in each epoch.

[0095] (3) The preprocessed image data is input into the blind path recognition model, and passes through the multi-scale feature extraction module, feature fusion module, and decoder in sequence to obtain the prediction result of the model. Then, the loss value is calculated according to the model task type, and combined with the cross-entropy loss. and loss The total loss is Where α and β are weighting coefficients;

[0096] (4) Calculate the gradient of the loss function with respect to all trainable parameters of the model through the backpropagation algorithm, and then perform parameter updates according to the Adam optimizer. During the parameter update process, the optimizer adjusts the model parameters to minimize the loss function based on the calculated gradient and the set learning rate. Finally, after each epoch of training, the model is evaluated using the validation set data.

[0097] In this embodiment, step S6 optimizes the performance of the tactile paving recognition model by scientifically configuring the training strategy. First, it determines hyperparameters such as learning rate and batch size, selects the Adam optimizer and sets the parameters appropriately to lay the foundation for training. Then, it performs real-time augmentation and normalization on the training data, and shuffles the data order in each epoch to enhance data diversity and prevent overfitting. The preprocessed data is then input into the model, and the prediction results are obtained through multi-module processing. The total loss is calculated by combining cross-entropy loss and Dice loss. Finally, the gradient is calculated using the backpropagation algorithm, and the parameters are updated with the help of the Adam optimizer. The model is iteratively optimized while minimizing the loss function, and the training effect is evaluated through the validation set to ensure continuous improvement in model performance.

[0098] S7. Reconstruct the tactile paving recognition model; the specific steps for reconstructing the tactile paving recognition model are as follows:

[0099] (1) Use model analysis tools to count the parameters and computational cost of each layer, identify layers that contribute little to performance but consume a lot of resources. For convolutional layers, if the weights of most output channels of a certain layer are close to 0, consider pruning that channel. For fully connected layers, if the number of neurons is too large and the improvement in final accuracy is limited, the number of neurons can be reduced. At the same time, if the multi-scale feature fusion module in the original model is computationally complex and has poor performance, it can be replaced with a more efficient structure to enhance the feature fusion effect. Then, add auxiliary modules to address the difficulties in blind path detection.

[0100] (2) Initialize the backbone network with model weights pre-trained on a large-scale image dataset, and then fine-tune it on the blind path detection dataset. When fine-tuning, the parameters of the backbone network can be fixed first, and only the newly added layers can be trained. After the model is stable, some layers of the backbone network can be gradually released for joint training. Hyperparameters such as learning rate, batch size, and loss function weight coefficients can be readjusted by grid search, random search or Bayesian optimization. Then, structured pruning or unstructured pruning methods can be used to remove unimportant connections or parameters.

[0101] (3) Adjust the feature fusion order: Change the connection order or fusion method of the multi-scale feature extraction module and the fusion module, and redesign the branch structure: For multi-branch feature extraction networks, re-plan the number of branches, convolution kernel size and dilation rate;

[0102] (4) Train multiple blind path recognition models with different structures or different initializations. In the inference stage, use the averaging method, voting method or weighted averaging method to fuse the prediction results of multiple models, average the probability maps predicted by multiple models, and then obtain the final result through threshold segmentation to improve the stability and accuracy of the model. Finally, after completing the model structure and parameter adjustment, use the adjusted model to retrain on the training set.

[0103] In this embodiment, step S7 improves model efficiency and accuracy through a multi-dimensional optimization strategy. First, redundant components are identified based on model analysis tools, inefficient channels or neurons are pruned, complex modules are replaced, and targeted auxiliary modules are added. Then, the backbone network is initialized using pre-trained weights, and a phased fine-tuning and hyperparameter optimization strategy is adopted to improve training efficiency. Next, the feature fusion order and branch structure are adjusted, and the multi-scale feature interaction method is optimized. Finally, the prediction results of multiple models are fused through ensemble learning to enhance stability and accuracy. Finally, the adjusted model is retrained to achieve efficient utilization of computing resources and a dual improvement in detection performance.

[0104] S8. Optimize and post-process the reconstructed model; the specific steps for optimizing and post-processing the reconstructed model are as follows:

[0105] (1) Lightweight compression of the model: Structured pruning is used to delete unimportant channels or connections in the model to reduce model complexity. INT8 quantization technology is used to convert model parameters from floating point to integer to reduce memory usage and computation. The model size is reduced while ensuring accuracy, and the inference speed is improved.

[0106] (2) Then, the inference framework is accelerated: the reconstructed model is converted into the ONNX format. With the help of inference frameworks such as TensorRT and ONNXRuntime, the model computation graph is optimized through optimization techniques such as layer fusion and dynamic memory management to adapt to different hardware platforms and achieve efficient inference of the model on GPUs, embedded devices, etc.

[0107] (3) Secondly, optimize the segmentation results: use the Otsu algorithm to automatically calculate the optimal segmentation threshold, binarize the probability map output by the model, and then connect the adjacent blind path areas through morphological operations such as dilation and erosion, remove isolated noise points, and refine the blind path boundary.

[0108] (4) Finally, the detection results are calibrated: Soft-NMS or DIoU-NMS algorithms are used to replace traditional non-maximum suppression, optimize the bounding box screening, and dynamically adjust the confidence threshold according to the size of the detection box to improve the detection rate of small-sized blind paths and reduce false detection of large-sized blind paths.

[0109] In this embodiment, step S8 comprehensively improves the model's practicality and detection performance from four dimensions: model performance optimization, inference acceleration, segmentation result optimization, and detection result calibration. It achieves model lightweighting through structured pruning and INT8 quantization, reducing computational load while maintaining accuracy. It optimizes the model's computational graph using inference frameworks such as TensorRT and ONNXRuntime, adapting to multiple hardware platforms and improving inference speed. It optimizes segmentation results using the Otsu algorithm and morphological operations, refining tactile paving boundaries and removing noise. It calibrates detection results using improved Soft-NMS or DIoU-NMS algorithms and dynamic threshold adjustment strategies, improving the detection rate of small-sized tactile paving and reducing false detections of large-sized tactile paving, ensuring accurate and reliable final output.

[0110] S9. Verify and evaluate the experimental results; the specific steps for verifying and evaluating the experimental results are as follows:

[0111] (1) Calculate and evaluate the indicators: calculate the proportion of pixels that are correctly predicted to the total number of pixels. ,in The number of pixels in category i that were correctly predicted. Let i be the number of pixels that are predicted to be class j from the true class i. Then calculate the IoU for each class and take the average: , Where TP represents true examples, FP represents false negative examples, k represents the number of classes, and finally, precision P and recall R are considered: , , ;

[0112] (2) By gradually removing model components, compare the performance changes under different configurations to verify the effectiveness of each component. For example, compare the basic model, basic model + multi-scale feature fusion and basic model + multi-scale feature fusion + edge enhancement, and then compare with mainstream algorithms on the same test set. Record the metrics of each algorithm and draw a performance comparison table or bar chart.

[0113] (3) Visualize the prediction results: Overlay the blind path area predicted by the model onto the original image, compare the actual annotation with the prediction results, intuitively display the detection effect, and verify the effectiveness of the algorithm;

[0114] In this embodiment, step S9 comprehensively and scientifically verifies the effectiveness and reliability of the algorithm through quantitative index calculation, multi-dimensional comparative experiments, and visualization analysis. First, key indicators such as pixel accuracy, average intersection-union ratio, precision, and recall are used to quantitatively evaluate the model prediction results and measure the detection accuracy from a numerical perspective. Second, ablation experiments are conducted to remove key components of the model and compare performance changes to verify the effectiveness of each module. The model is then compared horizontally with mainstream algorithms on a unified test set to intuitively demonstrate its advantages. Finally, the model prediction results are overlaid with the original image for visualization. By intuitively comparing the real labeled and predicted regions, the feasibility and accuracy of the algorithm in real-world scenarios are verified from a visual perspective.

[0115] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A deep learning-based blind path detection algorithm based on multi-scale feature fusion, characterized in that: Includes the following steps: S1. Collect and label data of tactile paving; S2. Collect multi-scale features and integrate them into the model architecture design; S3. Initially create a blind path recognition model; S4. Extract key area features from the tactile paving model; S5. Enhance the edge features of the blind path in the feature extraction network; S6. Train the blind path recognition model for key feature areas; S7. Reconstruct the tactile paving identification model; S8. Optimize and post-process the reconstructed model; S9. Verify and evaluate the experimental results.

2. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S1, the specific steps for collecting and labeling tactile paving data are as follows: (1) Clearly define the diverse scenarios for data collection: cover different indoor and outdoor areas to simulate different environments of tactile paving in actual use, accurately collect tactile paving images, obtain high-resolution, multi-angle tactile paving image data, and record all metadata during the collection process; (2) Data collection implementation: Data collection is carried out at different times and under different weather conditions to obtain images of the tactile paving under various lighting and climate conditions. According to the different tactile paving, targeted collection is carried out to ensure the richness and diversity of the data. (3) Label the collected data: Label the collected images one bounding box at a time. For semantic segmentation tasks, the boundaries of the blind path area need to be accurately delineated. For object detection tasks, the blind path objects need to be accurately selected and their categories and attributes need to be marked. The labeled data should be organized to ensure the manageability and traceability of the data. The labeled data should be divided into training set, validation set and test set. When dividing, ensure that the data distribution of different scenes, lighting conditions and blind path types in each set is balanced to meet the needs of subsequent model training and evaluation.

3. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S2, the specific steps for collecting multi-scale features and designing the fusion model architecture are as follows: (1) Constructing a multi-scale feature extraction module: In the backbone of the basic network, set up parallel branches of dilated convolution with different dilation rates. The formula for the output feature map of each branch is as follows: In this model, Conv2D is a 2D convolution operation, BN is batch normalization, and ReLU is the activation function. By using different dilation rates, each branch can capture features from different receptive fields, achieving multi-scale feature extraction. Then, a feature pyramid network is built. Utilizing feature maps from different levels in the base network, deep semantic information is fused with shallow detail information through upsampling and lateral connections to construct the feature pyramid, providing a foundation for subsequent multi-scale detection. The fusion formula for feature maps from different levels is as follows: ,in For deep, low-resolution feature maps, Upsample is the upsampling operation. To generate shallow high-resolution feature maps, a parallel multi-branch network structure is designed. Each branch uses a different size convolution kernel and a different downsampling rate. The different convolution kernel size and downsampling rate enable each branch to process the image at different scales and output feature maps at different scales, thus enriching the feature representation. (2) The feature maps of different scales are spliced ​​in the channel dimension to expand the feature dimension and obtain more comprehensive feature information. After splicing, the number of channels is compressed by 1×1 convolution to reduce the amount of computation. At the same time, the spliced ​​features are fused and the feature maps of different scales are added element by element. However, different weights are assigned to each feature map before addition. Channel attention mechanism and spatial attention mechanism are introduced. The channel attention mechanism weights the feature map according to the importance of each channel to enhance the key channel features. The spatial attention mechanism focuses on the spatial area where the blind path is located in the feature map and suppresses irrelevant background information, thereby achieving more effective multi-scale feature fusion.

4. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S3, the specific steps for initially creating the tactile paving recognition model are as follows: (1) Design the decoder: adopt the semantic segmentation architecture, design the decoder structure, adopt the symmetric decoder of U-Net, and gradually restore the feature map resolution by combining upsampling and convolution operations, and map the fused multi-scale features to the segmentation result of the same size as the input image; (2) Determine the output layer and loss function: The output layer uses the Softmax activation function to convert the decoder output into a pixel-level class probability distribution. The output dimension is H×W×C, where H and W are the image height and width, and C is the number of classes. (3) Define the loss function: use cross-entropy loss combined with Dice loss, the formula is as follows ,in Measure the difference between the predicted probability and the true label. Focus on the overlap of the segmented regions, where α and β are weighting coefficients; (4) Finally, use a deep learning framework to instantiate the modules designed above and build a complete blind path recognition model. After the model is created, perform forward propagation by inputting random data to check whether the output dimension of the model meets expectations and ensure that the data flows smoothly between modules. If the output dimension is wrong or the model reports an error, check the network layer parameter settings, connection methods, etc., and make targeted adjustments and optimizations to ensure the correctness of the model structure.

5. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S4, the specific steps for extracting the key area features of the tactile paving model are as follows: (1) Based on the morphological features of the tactile paving in the image and the annotation information of the dataset, the prior knowledge of the tactile paving area is statistically analyzed to provide a basis for the subsequent key area selection. Then, during the forward propagation of the model, the feature map set output by the multi-scale feature extraction module and the feature fusion module is obtained. Using the channel attention mechanism, the channel dimension of each feature map is weighted and the channel weight is calculated. ,in and This is a fully connected layer, where H and W are the height and width of the feature map, respectively, and are weighted... Weighting the feature map F highlights the channel features related to the tactile paving. ; (2) Using the anchor box concept in object detection, based on the prior size information of the blind path, a series of candidate region boxes are generated on the feature map. For each candidate region box Calculate its response value on the feature map. ,in The pixel coordinates within the bounding box are determined by setting a response threshold. Filter out response values The region bounding boxes are used as key region candidates; (3) For the selected key region candidates, feature extraction and aggregation are performed on feature maps at different scales. Above, the area frame Perform bilinear interpolation to unify the feature size to a fixed size M×N to obtain the feature vector. Concat represents concatenation along the channel dimension to generate a key region feature vector containing multi-scale information. .

6. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S5, the specific steps for enhancing the blind path edge features of the feature extraction network are as follows: (1) Extracting edge features: In the multi-scale feature map, a shallow feature map with rich spatial details is selected, and edge features are obtained by using the Sobel operator for edge detection. Then, an edge enhancement module is constructed to fuse the edge features with the original feature map. Edge features are extracted from feature maps at different scales, and a multi-scale edge fusion strategy is designed. (2) Based on accurate edge features, a binary classification loss of edge pixels and non-edge pixels is adopted, combined with bounding box regression loss: ,in For edge classification loss, For edge location regression loss, This is the balance coefficient; (3) Introduce a spatial attention mechanism to focus on feature enhancement in the edge region. Suppress interference from non-edge regions through the attention mechanism to enhance edge feature expression. Design an edge-semantic feature fusion module to combine the enhanced edge features with deep semantic features. Finally, in the model training stage, combine the main task loss and edge constraint loss and optimize the model parameters through gradient backpropagation so that the model can enhance its ability to capture the edge features of the blind path while learning semantic information.

7. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S6, the specific steps for training the tactile paving recognition model for key feature regions are as follows: (1) Determine the hyperparameters for training, including learning rate, batch size, and number of training epochs. At the same time, select the Adam optimizer and set its hyperparameters. , , ; (2) Use a deep learning framework to save the training set data and perform real-time data augmentation. Then normalize the image data and keep the label data in its original labeling format. In order to prevent overfitting, the shuffle parameter of the data loader can be enabled to shuffle the data order in each epoch. (3) The preprocessed image data is input into the blind path recognition model, and passes through the multi-scale feature extraction module, feature fusion module, and decoder in sequence to obtain the prediction result of the model. Then, the loss value is calculated according to the model task type, and combined with the cross-entropy loss. and loss The total loss is Where α and β are weighting coefficients; (4) The gradient of the loss function with respect to all trainable parameters of the model is calculated by backpropagation algorithm. Then, the parameters are updated according to the Adam optimizer. During the parameter update process, the optimizer adjusts the model parameters to minimize the loss function based on the calculated gradient and the set learning rate. Finally, the model is evaluated using validation set data after each epoch of training.

8. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S7, the specific steps for reconstructing the tactile paving recognition model are as follows: (1) Use model analysis tools to count the parameters and computational cost of each layer, identify layers that contribute little to performance but consume a lot of resources. For convolutional layers, if the weights of most output channels of a certain layer are close to 0, the channel can be pruned. For fully connected layers, if the number of neurons is too large and the improvement in final accuracy is limited, the number of neurons can be reduced. At the same time, if the multi-scale feature fusion module in the original model is computationally complex and has poor performance, it can be replaced with a more efficient structure to enhance the feature fusion effect. Then, add auxiliary modules to address the difficulties in blind path detection. (2) Initialize the backbone network with model weights pre-trained on a large-scale image dataset, and then fine-tune it on the blind path detection dataset. When fine-tuning, the parameters of the backbone network can be fixed first, and only the newly added layers can be trained. After the model is stable, some layers of the backbone network can be gradually released for joint training. Hyperparameters such as learning rate, batch size, and loss function weight coefficients can be readjusted by grid search, random search or Bayesian optimization. Then, structured pruning method is used to remove unimportant parameters. (3) Adjust the feature fusion order: change the fusion method of the multi-scale feature extraction module and the fusion module, and redesign the branch structure: for the multi-branch feature extraction network, re-plan the number of branches, convolution kernel size and dilation rate; (4) Train multiple blind path recognition models with different structures or different initializations. In the inference stage, use different methods to fuse the prediction results of multiple models, average the probability maps predicted by multiple models, and then obtain the final result through threshold segmentation to improve the stability and accuracy of the model. Finally, after completing the model structure and parameter adjustment, use the adjusted model to retrain on the training set.

9. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: In step S8, the specific steps for optimizing and post-processing the reconstructed model are as follows: (1) Lightweight compression of the model: Structured pruning is used to delete unimportant channels or connections in the model to reduce model complexity. INT8 quantization technology is used to convert model parameters from floating point to integer to reduce memory usage and computation. The model size is reduced while ensuring accuracy, and the inference speed is improved. (2) Then, the inference framework is accelerated: the reconstructed model is converted into ONNX format, and the model computation graph is optimized by using inference frameworks such as TensorRT and ONNXRuntime to adapt to different hardware platforms and achieve efficient inference of the model. (3) Secondly, optimize the segmentation results: use the Otsu algorithm to automatically calculate the optimal segmentation threshold, binarize the probability map output by the model, and then connect the adjacent blind path areas through morphological operations to remove isolated noise points and refine the blind path boundaries. (4) Finally, the detection results are calibrated: the Soft-NMS algorithm is used to replace the traditional non-maximum suppression, the bounding box screening is optimized, the confidence threshold is dynamically adjusted according to the size of the detection box, the detection rate of small-sized blind paths is improved, and the false detection of large-sized blind paths is reduced.

10. The deep learning-based blind path detection algorithm based on multi-scale feature fusion according to claim 1, characterized in that: The specific steps for verifying and evaluating the experimental results in the nine steps are as follows: (1) Calculate and evaluate the indicators: calculate the proportion of pixels that are correctly predicted to the total number of pixels. ,in The number of pixels in category i that were correctly predicted. Let i be the number of pixels that are predicted to be class j from the true class i. Then calculate the IoU for each class and take the average: , Where TP represents true examples, FP represents false negative examples, k represents the number of classes, and finally, precision P and recall R are considered: , , ; (2) By gradually removing model components, compare the performance changes under different configurations to verify the effectiveness of each component. For example, compare the basic model, basic model + multi-scale feature fusion and basic model + multi-scale feature fusion + edge enhancement, and then compare with mainstream algorithms on the same test set. Record the metrics of each algorithm and draw a performance comparison table or bar chart. (3) Visualize the prediction results: Overlay the blind path area predicted by the model onto the original image, compare the actual annotation with the prediction results, intuitively display the detection effect, and verify the effectiveness of the algorithm.

Images