Method for semantic segmentation and tracking of blind path of guide device based on lightweight transformer
Patent Information
- Application Number
- CN202610973376.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-10-30
AI Technical Summary
CNN受限于感受野，难以建模盲道长距离上下文；标准Transformer虽全局建模能力强，但自注意力计算复杂度高，难以在便携导盲设备上实时运行
轻量化效果显著：模型参数量≤0.8M，相比传统Transformer分割模型，参数量压缩90%以上，端侧推理延迟≤24ms，满足15帧/秒的实时处理需求，适配导盲设备端侧低功耗、实时性需求；
Smart Images
Figure REF-OBJ-1782886851709-000001 
Figure REF-OBJ-1782886851709-000002 
Figure REF-OBJ-1782886851709-000003
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent guide computer vision technology, and more specifically to a method for semantic segmentation and tracking of tactile paving in guide devices based on a lightweight Transformer. Background Technology
[0002] There are over 285 million visually impaired people worldwide, with more than 17 million in my country. Tactile paving is a crucial guarantee for their independent travel. However, urban tactile paving is commonly obstructed, damaged, or missing. Traditional white canes have limited sensing range, and guide dogs are expensive and few in number, failing to meet widespread needs. Intelligent guide devices based on computer vision have become key to solving the travel difficulties faced by visually impaired people.
[0003] Existing semantic segmentation techniques for tactile paving have evolved from hand-crafted features to convolutional neural networks (CNNs), and then to Transformers. CNNs are limited by their receptive field, making it difficult to model the long-range context of tactile paving. While standard Transformers have strong global modeling capabilities, their high self-attention computational complexity makes them difficult to run in real time on portable guide devices for the visually impaired. Existing lightweight Transformers are mostly designed for general scenarios and do not fully utilize the unique features of tactile paving, making it difficult to balance segmentation accuracy and real-time performance.
[0004] In terms of tactile paving tracking, existing methods are divided into two categories: boundary tracking and detection tracking. They generally suffer from the problem of the segmentation and tracking modules being disconnected and information not being able to be exchanged. The tracking stability is poor when the tactile paving is obstructed or when the user is moving.
[0005] In summary, existing technologies suffer from four major contradictions: accuracy versus real-time performance, global modeling versus computational efficiency, segmentation versus tracking, and mismatch between general models and specific scenarios. These technologies cannot simultaneously meet the requirements of portable guide devices for high accuracy, real-time performance at the edge, low power consumption, and high stability. Summary of the Invention
[0006] To address the shortcomings of existing technologies, the present invention aims to provide a method for semantic segmentation and tracking of tactile paving in guide devices based on a lightweight Transformer. By designing a lightweight Transformer segmentation architecture, a tactile paving-specific feature enhancement module, and a segmentation-tracking closed-loop mechanism, the method achieves high precision and real-time performance in tactile paving semantic segmentation, as well as high stability in tactile paving tracking. This method is adapted to the low power consumption requirements of portable guide devices and provides continuous and reliable walking guidance for visually impaired users.
[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for semantic segmentation and tracking of tactile paving in guide devices based on lightweight Transformer, comprising the following steps: Step 1: Use the high-definition camera built into the guide device to capture images of the road surface at a rate of 15 frames per second. During the capture process, the camera exposure parameters are automatically adjusted to adapt to different lighting scenarios, such as strong light and low light. Step 2: Perform basic preprocessing on the acquired images to remove noise, equalize illumination, and preserve the features of the tactile paving. Step 3: Construct the Mobile-Former architecture, which integrates depthwise separable convolution and lightweight self-attention mechanism to achieve semantic segmentation of blind paths; Step four: Enhance the features of the semantic mask of the blind path obtained in step three to highlight the outline of the blind path and improve the stability of subsequent tracking. Step 5: Based on the enhanced tactile paving mask from Step 4, a Kalman filter algorithm is used to achieve continuous tracking of the tactile paving, in order to solve the tracking loss problem caused by occlusion, jitter, and tactile paving interruption. Step 6: Construct a segmentation-tracking closed-loop mechanism to calibrate segmentation parameters in real time based on the tracking status, and simultaneously feed the tracking results back to the user.
[0008] As a further improvement of the present invention, the specific steps of the preprocessing in step two are as follows: Step 21, Size Normalization: Normalize the image to a size of 320×240 to reduce the computational cost of subsequent algorithms while ensuring the feature integrity of the tactile paving area. Step 22, Lightweight Gaussian Noise Reduction: A Gaussian filtering algorithm is used to remove Gaussian noise from the image, avoiding interference from noise in the extraction of blind path features, as detailed below:
[0009] in, Let be the value of the Gaussian filter kernel at coordinates (x, y). The standard deviation is Gaussian, with a fixed value of 0.8, which ensures noise reduction while preserving the edge features of the tactile paving to the maximum extent. Steps 2 and 3, Illumination equalization: Adaptive histogram equalization algorithm is used to adjust image brightness and contrast, solve the problem of blurred tactile paving features caused by uneven illumination, and ensure the consistency of tactile paving segmentation under different lighting scenarios.
[0010] As a further improvement of the present invention, step three involves constructing a Mobile-Former architecture, integrating depthwise separable convolution with a lightweight self-attention mechanism, and the specific steps for achieving semantic segmentation of the blind path are as follows: Step 31, Local Feature Extraction: Depthwise separable convolution is used to extract local features of the image, reducing computational complexity, as detailed below:
[0011] in, This is the preprocessed image feature map. For depthwise separable convolution operations, For pointwise convolution operation, Indicates feature fusion; Step 32, Lightweight Self-Attention Calculation: To address the sparsity of tactile paving features, self-attention calculation is performed only on key feature regions, avoiding the high computational overhead of full-image attention. Specifically:
[0012] Where Q, K, and V are the query matrix, key matrix, and value matrix, respectively. M is the feature dimension, and M is the tactile paving feature mask; Steps three and four: Feature fusion and segmentation output: The local convolutional features are fused with the lightweight self-attention features and input into the lightweight segmentation head, outputting a binary semantic mask for the blind path and the background. The mask value of 1 represents the tactile paving area, and 0 represents the background area.
[0013] As a further improvement of the present invention, the specific steps for feature enhancement of the tactile semantic mask obtained in step three in step four are as follows: Step 41, Edge Enhancement: The Laplacian operator is used to sharpen the edges of the tactile paving to enhance the visibility of the tactile paving outline, as detailed below:
[0014] in, These are the pixel values of the segmented tactile paving mask. This is the sharpening factor (fixed value 0.2). For the Laplace operator; Step 42, Feature Denoising: Morphological opening operation is used to remove small noise points in the segmentation mask, retaining the main area of the tactile paving, and ensuring the accuracy of target tracking.
[0015] As a further improvement to the present invention, the specific steps of step five, based on the enhanced blind path mask in step four, to achieve continuous tracking of the blind path using the Kalman filter algorithm are as follows: Step 51, Tracking State Initialization: Based on the segmented tactile paving mask, extract the bounding rectangle of the tactile paving region to determine the initial state of the tracking target. ,in The coordinates of the center of the rectangle are The width and height of the rectangle. To track the target's speed; Step 52, State Prediction: Predict the position state of the tactile paving in the next frame using the state equation of Kalman filtering, as detailed below:
[0016] in, Let A be the predicted state of the k-th frame, and let A be the state transition matrix. This is process noise; Step 53, State Update: Update the tracking state using the observation equation of the Kalman filter and the segmentation result of the current frame, as follows:
[0017]
[0018]
[0019]
[0020] in, Let H be the observation state of the k-th frame, and H be the observation matrix. To observe the noise, For Kalman gain, Let I be the state covariance matrix, and let I be the identity matrix; Step 54, Occlusion Handling: When the tactile paving is obstructed (i.e., the tracking confidence is <0.6), prediction tracking is performed only through the state equation, and the tracking state is retained; when the obstruction is removed (i.e., the tracking confidence is ≥0.6), the tracking state is quickly calibrated in conjunction with the segmentation results to avoid tracking loss.
[0021] As a further improvement to the present invention, a segmentation-tracking closed-loop mechanism is constructed, and the segmentation parameters are calibrated in real time according to the tracking status, while the tracking results are fed back to the user. The specific steps are as follows: Step 61, Tracking Confidence Detection: Calculate the overlap rate between the tracking result and the current frame segmentation result as the tracking confidence. When the confidence is <0.5, it is determined that the tracking is lost, and the re-segmentation process is automatically triggered to re-execute steps 3 to 5 to quickly find the blind path target. Step 62, Segmentation Parameter Calibration: The tracked tactile paving movement trend is fed back to the segmentation module, and the threshold of the segmentation head weight is dynamically adjusted to improve the segmentation accuracy in the tactile paving movement direction; Step 63, Result Feedback: The location and tracking status of the tactile paving are transmitted to the user through bone conduction voice + tactile vibration dual feedback to ensure that visually impaired users can clearly perceive it.
[0022] The present invention has the following beneficial effects Significantly lightweight: The number of model parameters is ≤0.8M, which is more than 90% less than the traditional Transformer segmentation model. The edge inference latency is ≤24ms, which meets the real-time processing requirement of 15 frames / second and is suitable for the low power consumption and real-time requirements of guide devices. High segmentation accuracy and robustness: The accuracy of tactile paving segmentation is ≥94.5%, of which the accuracy of worn tactile paving segmentation is ≥92% and the accuracy of occluded tactile paving segmentation is ≥89%. Compared with traditional CNN segmentation algorithms, the accuracy is improved by more than 10%, and it can be adapted to different complex scenarios. Good tracking stability: It adopts a segmentation-tracking closed-loop mechanism, with a blind track tracking loss rate of ≤1.8%, and can continuously track under occlusion conditions. It can also quickly calibrate after recovery, solving the pain point of high tracking loss rate of existing algorithms. Excellent low power consumption and adaptability: When the algorithm is running, the power consumption of the guide device is ≤100mW, and it can run continuously for more than 12 hours, meeting the battery life needs of visually impaired users for long-term outdoor travel. Operating on all devices, ensuring privacy and security: All image data, segmentation and tracking results are collected, processed and stored on the device itself, without being uploaded to the cloud, thus avoiding the leakage of sensitive data such as user travel trajectories and scene information; Wide adaptability: It can be adapted to indoor and outdoor tactile paving, worn / faded / broken tactile paving, and different lighting scenarios. It can also be adapted to different types of portable guide devices for the visually impaired, making it highly versatile. Detailed Implementation
[0023] The present invention will be further described in detail below with reference to the given embodiments.
[0024] This embodiment of a method for semantic segmentation and tracking of tactile paving in a guideway device based on a lightweight Transformer includes the following steps: Step 1: Use the high-definition camera built into the guide device to capture images of the road surface at a rate of 15 frames per second. During the capture process, the camera exposure parameters are automatically adjusted to adapt to different lighting scenarios, such as strong light and low light. Step 2: Perform basic preprocessing on the acquired images to remove noise, equalize illumination, and preserve the features of the tactile paving. Step 3: Construct the Mobile-Former architecture, which integrates depthwise separable convolution and lightweight self-attention mechanism to achieve semantic segmentation of blind paths; Step four: Enhance the features of the semantic mask of the blind path obtained in step three to highlight the outline of the blind path and improve the stability of subsequent tracking. Step 5: Based on the enhanced tactile paving mask from Step 4, a Kalman filter algorithm is used to achieve continuous tracking of the tactile paving, in order to solve the tracking loss problem caused by occlusion, jitter, and tactile paving interruption. Step 6: Construct a segmentation-tracking closed-loop mechanism to calibrate segmentation parameters in real time based on the tracking status, and simultaneously feed the tracking results back to the user.
[0025] In this embodiment, the device uses high-frequency image acquisition combined with adaptive exposure to adapt to complex lighting conditions. After image preprocessing and purification, it achieves high-precision tactile paving segmentation based on a lightweight fusion architecture. Feature enhancement is then used to optimize the segmentation effect, and Kalman filtering is employed to achieve stable tracking. Finally, a closed-loop mechanism enables module linkage and user feedback. This process solves the problems of traditional CNNs' inability to model long distances and the excessive computational power of standard Transformers. It also overcomes the technical pain points of fragmented segmentation and tracking modules and poor scene adaptability, achieving high-precision, low-power, and highly stable tactile paving detection and tracking on the edge device, resulting in accurate and continuous intelligent guidance for the blind.
[0026] Furthermore, the specific steps of the preprocessing in step two are as follows: Step 21, Size Normalization: Normalize the image to a size of 320×240 to reduce the computational cost of subsequent algorithms while ensuring the feature integrity of the tactile paving area. Step 22, Lightweight Gaussian Noise Reduction: A Gaussian filtering algorithm is used to remove Gaussian noise from the image, avoiding interference from noise in the extraction of blind path features, as detailed below:
[0027] in, Let be the value of the Gaussian filter kernel at coordinates (x, y). The standard deviation is Gaussian, with a fixed value of 0.8, which ensures noise reduction while preserving the edge features of the tactile paving to the maximum extent. Steps 2 and 3, Illumination equalization: Adaptive histogram equalization algorithm is used to adjust image brightness and contrast, solve the problem of blurred tactile paving features caused by uneven illumination, and ensure the consistency of tactile paving segmentation under different lighting scenarios.
[0028] This step involves multi-layer preprocessing, including size optimization, precise noise reduction, and illumination equalization, to comprehensively purify the input image. This solves the problem of feature extraction failure caused by complex lighting, image noise, and size redundancy, providing a high-quality image foundation for subsequent segmentation algorithms. It also assists the core algorithm in improving segmentation accuracy, while effectively reducing the computational load on the device and further ensuring real-time performance on the edge.
[0029] Furthermore, step three involves constructing the Mobile-Former architecture, integrating depthwise separable convolutions with a lightweight self-attention mechanism, to achieve semantic segmentation of the tactile paving for the visually impaired. The specific steps are as follows: Step 31, Local Feature Extraction: Depthwise separable convolution is used to extract local features of the image, reducing computational complexity, as detailed below:
[0030] in, This is the preprocessed image feature map. For depthwise separable convolution operations, For pointwise convolution operation, Indicates feature fusion; Step 32, Lightweight Self-Attention Calculation: To address the sparsity of tactile paving features, self-attention calculation is performed only on key feature regions, avoiding the high computational overhead of full-image attention. Specifically:
[0031] in, These are the query matrix, key matrix, and value matrix, respectively. For feature dimension, For tactile paving features; Steps three and four: Feature fusion and segmentation output: The local convolutional features are fused with the lightweight self-attention features and input into the lightweight segmentation head, outputting a binary semantic mask for the blind path and the background. The mask value of 1 represents the tactile paving area, and 0 represents the background area.
[0032] This step combines the advantages of convolutional local feature extraction with the global modeling capabilities of the attention mechanism, while specifically simplifying the computational logic. It solves the problem that traditional models cannot balance accuracy and computational power, adapts to the sparse feature distribution characteristics of tactile paving, and achieves high-precision tactile paving semantic segmentation with extremely low computational overhead, greatly improving the model's deployment adaptability on portable guide devices for the visually impaired.
[0033] Furthermore, the specific steps for feature enhancement of the tactile paving semantic mask obtained from step three in step four are as follows: Step 41, Edge Enhancement: The Laplacian operator is used to sharpen the edges of the tactile paving to enhance the visibility of the tactile paving outline, as detailed below:
[0034] in, These are the pixel values of the segmented tactile paving mask. The sharpening factor is a fixed value of 0.2. For the Laplace operator; Step 42, Feature Denoising: Morphological opening operation is used to remove small noise points in the segmentation mask, retaining the main area of the tactile paving, and ensuring the accuracy of target tracking.
[0035] This step performs fine-grained optimization on the segmented mask image, solving the problems of blurred edges and minor noise interference in the segmentation results. It further enhances the target features of the blind path, providing a precise and clean target area for subsequent tracking algorithms, improving tracking stability from the source, and reducing false tracking and missed tracking.
[0036] Furthermore, step five, based on the enhanced tactile paving mask from step four, employs the Kalman filter algorithm to achieve continuous tracking of the tactile paving. The specific steps are as follows: Step 51, Tracking State Initialization: Based on the segmented tactile paving mask, extract the bounding rectangle of the tactile paving region to determine the initial state of the tracking target. ,in The coordinates of the center of the rectangle are The width and height of the rectangle. To track the target's speed; Step 52, State Prediction: Predict the position state of the tactile paving in the next frame using the state equation of Kalman filtering, as detailed below:
[0037] in, Let k be the predicted state of the k-th frame. Here is the state transition matrix. This is process noise; Step 53, State Update: Update the tracking state using the observation equation of the Kalman filter and the segmentation result of the current frame, as follows:
[0038]
[0039]
[0040]
[0041] in, For the observation state of the k-th frame, For the observation matrix, To observe the noise, For Kalman gain, Let be the state covariance matrix. It is the identity matrix; Step 54, Occlusion Handling: When the tactile paving is obstructed (i.e., the tracking confidence is <0.6), prediction tracking is performed only through the state equation, and the tracking state is retained; when the obstruction is removed (i.e., the tracking confidence is ≥0.6), the tracking state is quickly calibrated in conjunction with the segmentation results to avoid tracking loss.
[0042] This step relies on the prediction and update mechanism of Kalman filtering, combined with adaptive occlusion processing logic, to solve the problem that traditional tracking algorithms are prone to losing targets in scenarios such as road surface vibration, tactile paving occlusion, and tactile paving interruption. It achieves continuous and stable tracking of tactile paving throughout the entire process, greatly improving the reliability of blind guidance in complex road conditions.
[0043] Furthermore, a segmentation-tracking closed-loop mechanism is constructed to calibrate segmentation parameters in real time based on the tracking status, while simultaneously feeding the tracking results back to the user. The specific steps are as follows: Step 61, Tracking Confidence Detection: Calculate the overlap rate between the tracking result and the current frame segmentation result as the tracking confidence. When the confidence is <0.5, it is determined that the tracking is lost, and the re-segmentation process is automatically triggered to re-execute steps 3 to 5 to quickly find the blind path target. Step 62, Segmentation Parameter Calibration: The tracked tactile paving movement trend is fed back to the segmentation module, and the threshold of the segmentation head weight is dynamically adjusted to improve the segmentation accuracy in the tactile paving movement direction; Step 63, Result Feedback: The location and tracking status of the tactile paving are transmitted to the user through bone conduction voice + tactile vibration dual feedback to ensure that visually impaired users can clearly perceive it.
[0044] This step establishes a closed-loop interactive system for the segmentation and tracking modules, completely solving the core problems of fragmented modules and lack of information exchange in traditional technologies. It improves detection accuracy in dynamic walking scenarios through dynamic parameter calibration, while the dual-modal feedback method ensures that visually impaired users can efficiently and accurately perceive road conditions, thereby improving travel safety and convenience.
[0045] In summary, this invention adopts a lightweight Transformer fusion architecture, combining image preprocessing, tactile paving feature enhancement, Kalman filter-based stable tracking, and a segmentation-tracking closed-loop interaction mechanism. It systematically solves four core problems of existing tactile paving detection technologies: the imbalance between accuracy and real-time performance, the contradiction between global modeling and computational efficiency, module fragmentation, and poor adaptability of general models to various scenarios. It successfully achieves the technical effects of low-power operation, high-precision tactile paving segmentation, and continuous and stable tracking in complex scenarios on the portable guide device, providing continuous and reliable intelligent guide services for visually impaired users to travel independently.
[0046] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. A method for semantic segmentation and tracking of tactile paving in guide devices based on lightweight Transformer, characterized in that: Includes the following steps: Step 1: Use the high-definition camera built into the guide device to capture images of the road surface at a rate of 15 frames per second. During the capture process, the camera exposure parameters are automatically adjusted to adapt to different lighting scenarios, such as strong light and low light. Step 2: Perform basic preprocessing on the acquired images to remove noise, equalize illumination, and preserve the features of the tactile paving. Step 3: Construct the Mobile-Former architecture, which integrates depthwise separable convolution and lightweight self-attention mechanism to achieve semantic segmentation of blind paths; Step four: Enhance the features of the semantic mask of the blind path obtained in step three to highlight the outline of the blind path and improve the stability of subsequent tracking. Step 5: Based on the enhanced tactile paving mask from Step 4, a Kalman filter algorithm is used to achieve continuous tracking of the tactile paving, in order to solve the tracking loss problem caused by occlusion, jitter, and tactile paving interruption. Step 6: Construct a segmentation-tracking closed-loop mechanism to calibrate segmentation parameters in real time based on the tracking status, and simultaneously feed the tracking results back to the user.
2. The method for semantic segmentation and tracking of tactile paving for visually impaired devices based on lightweight Transformer according to claim 1, characterized in that: The specific steps for preprocessing in step two are as follows: Step 21, Size Normalization: Normalize the image to a size of 320×240 to reduce the computational cost of subsequent algorithms while ensuring the feature integrity of the tactile paving area. Step 22, Lightweight Gaussian Noise Reduction: A Gaussian filtering algorithm is used to remove Gaussian noise from the image, avoiding interference from noise in the extraction of blind path features, as detailed below: in, Let be the value of the Gaussian filter kernel at coordinates (x, y). The standard deviation is Gaussian, with a fixed value of 0.8, which ensures noise reduction while preserving the edge features of the tactile paving to the maximum extent. Steps 2 and 3, Illumination equalization: Adaptive histogram equalization algorithm is used to adjust image brightness and contrast, solve the problem of blurred tactile paving features caused by uneven illumination, and ensure the consistency of tactile paving segmentation under different lighting scenarios.
3. The method for semantic segmentation and tracking of tactile paving in guide devices based on lightweight Transformer according to claim 2, characterized in that: The specific steps for constructing the Mobile-Former architecture in step three, which integrates depthwise separable convolution and lightweight self-attention mechanism, to achieve semantic segmentation of the tactile paving are as follows: Step 31, Local Feature Extraction: Depthwise separable convolution is used to extract local features of the image, reducing computational complexity, as detailed below: in, This is the preprocessed image feature map. For depthwise separable convolution operations, For pointwise convolution operation, Indicates feature fusion; Step 32, Lightweight Self-Attention Calculation: To address the sparsity of tactile paving features, self-attention calculation is performed only on key feature regions, avoiding the high computational overhead of full-image attention. Specifically: in, These are the query matrix, key matrix, and value matrix, respectively. For feature dimension, For tactile paving features; Steps three and four: Feature fusion and segmentation output: The local convolutional features are fused with the lightweight self-attention features and input into the lightweight segmentation head, outputting a binary semantic mask for the blind path and the background. The mask value of 1 represents the tactile paving area, and 0 represents the background area.
4. The method for semantic segmentation and tracking of tactile paving for visually impaired devices based on lightweight Transformer according to claim 3, characterized in that: The specific steps for feature enhancement of the tactile semantic mask obtained from step three in step four are as follows: Step 41, Edge Enhancement: The Laplacian operator is used to sharpen the edges of the tactile paving to enhance the visibility of the tactile paving outline, as detailed below: in, These are the pixel values of the segmented tactile paving mask. The sharpening factor is a fixed value of 0.
2. For the Laplace operator; Step 42, Feature Denoising: Morphological opening operation is used to remove small noise points in the segmentation mask, retaining the main area of the tactile paving, and ensuring the accuracy of target tracking.
5. The method for semantic segmentation and tracking of tactile paving in guide devices based on lightweight Transformer according to claim 4, characterized in that: The specific steps of step five, which uses the enhanced blind path mask from step four and employs the Kalman filter algorithm to achieve continuous tracking of the blind path, are as follows: Step 51, Tracking State Initialization: Based on the segmented tactile paving mask, extract the bounding rectangle of the tactile paving region to determine the initial state of the tracking target. ,in The coordinates of the center of the rectangle are The width and height of the rectangle. To track the target's speed; Step 52, State Prediction: Predict the position state of the tactile paving in the next frame using the state equation of Kalman filtering, as detailed below: in, Let k be the predicted state of the k-th frame. Here is the state transition matrix. This is process noise; Step 53, State Update: Update the tracking state using the observation equation of the Kalman filter and the segmentation result of the current frame, as follows: in, For the observation state of the k-th frame, For the observation matrix, To observe the noise, For Kalman gain, Let be the state covariance matrix. It is the identity matrix; Step 54, Occlusion Handling: When the tactile paving is obstructed (i.e., the tracking confidence is <0.6), prediction tracking is performed only through the state equation, and the tracking state is retained; when the obstruction is removed (i.e., the tracking confidence is ≥0.6), the tracking state is quickly calibrated in conjunction with the segmentation results to avoid tracking loss.
6. The method for semantic segmentation and tracking of tactile paving in guide devices based on lightweight Transformer according to claim 5, characterized in that: The specific steps for constructing the segmentation-tracking closed-loop mechanism, which calibrates the segmentation parameters in real time based on the tracking status, and simultaneously feeds back the tracking results to the user, are as follows: Step 61, Tracking Confidence Detection: Calculate the overlap rate between the tracking result and the current frame segmentation result as the tracking confidence. When the confidence is <0.5, it is determined that the tracking is lost, and the re-segmentation process is automatically triggered to re-execute steps 3 to 5 to quickly find the blind path target. Step 62, Segmentation Parameter Calibration: The tracked tactile paving movement trend is fed back to the segmentation module, and the threshold of the segmentation head weight is dynamically adjusted to improve the segmentation accuracy in the tactile paving movement direction; Step 63, Result Feedback: The location and tracking status of the tactile paving are transmitted to the user through bone conduction voice + tactile vibration dual feedback to ensure that visually impaired users can clearly perceive it.