Intelligent wheelchair environment perception method based on multi-dimensional attention mechanism and feature reorganization

Abstract

The application discloses an intelligent wheelchair environment perception method based on a multi-dimensional attention mechanism and feature reorganization, and belongs to the technical field of intelligent wheelchair environment perception and autonomous navigation; the method uses a calibrated binocular camera to collect left and right images and complete polar correction; an improved YOLOv8 target detection network is constructed, a coordinate attention mechanism is embedded, and a weighted feature fusion module is introduced, a dynamic detection head is used, and the detection capability under the condition of a complex background, strong occlusion and a small target is improved; an improved SGBM based on Census transformation generates an initial disparity, and a high-quality depth map is obtained by combining weighted least square refinement and filtering of a semantic boundary constraint; finally, the detection frame ROI and the segmentation mask are fused to statistically remove outliers, and the absolute distance of the obstacle relative to the wheelchair center is output; the method has higher ranging stability under the scene of weak texture pavement and light change, and the cost is controllable, and is suitable for real-time obstacle avoidance assistance of a health care center and indoor and outdoor roads.

Description

Technical Field

[0001] This invention belongs to the field of intelligent wheelchair environmental perception and autonomous navigation technology, and particularly relates to an intelligent wheelchair environmental perception method based on multidimensional attention mechanism and feature recombination. Background Technology

[0002] As my country's population ages, the number of elderly people continues to expand, with a significant increase in the number of disabled and semi-disabled elderly. In recent years, the government has successively introduced a number of policies related to elderly care services and smart elderly care, explicitly proposing to promote the intelligent development of the elderly care service system and utilize artificial intelligence and Internet of Things technologies to improve the quality of life and travel safety of the elderly. As an important mobility tool for the elderly and people with mobility impairments, the environmental perception capabilities of smart wheelchairs are directly related to the safety of users. In complex wheelchair driving scenarios, the perception system needs to detect, classify, and perceive the distance of obstacles in front of the user in real time and accurately to assist in obstacle avoidance decisions or path planning.

[0003] Currently, environmental perception for intelligent wheelchairs and mobile robots mainly relies on the following existing technological solutions: LiDAR-based perception solutions: While LiDAR offers high ranging accuracy and strong resistance to light interference, its core sensors are expensive, significantly increasing the overall manufacturing cost of intelligent wheelchairs and hindering large-scale deployment in affordable healthcare equipment. Furthermore, LiDAR acquires sparse point cloud data, lacking color and texture information, making it difficult to effectively identify the specific semantic category of obstacles (e.g., distinguishing between steps and carpet). Mechanical LiDAR is also often bulky, resulting in high system integration complexity. Ultrasonic or millimeter-wave radar-based perception solutions: These sensors are relatively inexpensive, but have poor spatial resolution, making it difficult to detect small obstacles in the wheelchair's path (e.g., protruding stones, low steps), and are susceptible to multipath interference, leading to false positives or false negatives. Perception solutions based on traditional visual algorithms: With the development of deep learning, visual solutions based on monocular or binocular cameras have gained attention due to their low cost and rich information content.

[0004] However, in real-world wheelchair driving scenarios, existing vision technologies still face technical bottlenecks. Regarding object detection: wheelchairs typically have a low field of view, and obstacles (especially small targets) are easily occluded when facing crowded areas, complex backgrounds, or unstructured roads. Existing general-purpose object detection networks (such as the standard YOLO series) lack sufficient feature extraction capabilities when handling multi-scale variations, severe occlusion, and background interference, easily leading to missed or false detections of key obstacles. Regarding distance measurement: traditional binocular stereo matching algorithms (such as the standard SGBM) are prone to errors in matching cost calculation when processing weakly textured regions (such as white walls or concrete pavements) or object edges, resulting in numerous holes, noise, or blurred edges in the disparity map, failing to obtain smooth, high-precision depth information, and thus affecting the reliability of distance measurement.

[0005] Existing technologies struggle to simultaneously meet the demands of low cost, high-precision identification, and stable ranging. Therefore, this invention proposes a visual obstacle detection and ranging device that is adaptable to the unique driving environment of wheelchairs, cost-effective, and highly robust. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent wheelchair environmental perception method based on a multi-dimensional attention mechanism and feature reorganization, addressing problems in existing technologies such as easy missed and false detections of obstacles, high noise and blurred edges in binocular parallax matching, easy missegmentation of the road surface, and poor ranging stability. This invention acquires environmental information by setting up a binocular stereo camera, and adaptively calculates the category and spatial depth of obstacles using an improved target detection network combined with a stereo matching algorithm. Then, it achieves accurate distance calculation of obstacles through the fusion feedback of two-dimensional detection boxes and depth information, improving environmental perception and anti-interference capabilities under multi-scale, strong occlusion, and weak texture conditions.

[0007] To achieve the above objectives, this invention provides an intelligent wheelchair environment perception method based on multidimensional attention mechanisms and feature reorganization, comprising the following steps: S1. Acquire left and right images from the calibrated binocular camera and perform distortion correction and epipolar correction; S2. Construct an improved YOLOv8 object detection network and obtain obstacle categories and 2D bounding boxes; S3. The initial disparity map is calculated using Census transform and semi-global matching SGBM based on the epipolar-corrected left and right images; S4. The initial disparity map is refined by weighted least squares (WLS) based on the semantic boundary probability map predicted by the left view to obtain the refined disparity map. S5. Based on the two-dimensional bounding box obtained in S2, determine the region of interest (ROI), and perform statistical filtering on the refined disparity map obtained in S4 within the ROI to remove outliers. S6. Introduce an anisotropic spatial prior term consistent with the wheelchair view geometry into the GrabCut energy function to suppress the missegmentation of the bottom road surface area, and perform the disparity statistics based on the segmentation mask. S7. Based on the calibration parameters, convert the parallax within the ROI into depth and output the absolute distance of the obstacle relative to the center of the wheelchair.

[0008] Preferably, the specific content of S1 is as follows: S11. Use a calibrated binocular camera to acquire left and right images, and use the Zhang Zhengyou calibration method to obtain intrinsic parameters, distortion parameters and extrinsic parameters; Suppose any point in space The projected pixel coordinates on the left and right camera image planes are as follows: ; S12. Perform distortion correction and epipolar correction on the image to make corresponding points in the left and right views collinear, simplifying the matching search; S13. Establish a basic model for binocular ranging, and calculate points based on triangulation according to the pinhole imaging model. depth The expression is as follows: ; In the formula, The focal length of the camera; The distance between the binocular baselines; For parallax; The imaging mapping relationship is established as follows: ; In the formula, This is the camera intrinsic parameter matrix; and These are the rotation matrix and the translation matrix, respectively. For depth scale factor; These are pixel coordinates; Three-dimensional coordinates; Preferably, the specific content of S2 is as follows: S21. Improve the YOLOv8 object detection network by embedding a coordinate attention mechanism (CA) in the backbone network, as shown in the following expression: ; In the formula, and These are the attention weights generated along the horizontal and vertical directions, respectively; For the input feature map, the first Each channel, coordinates are Location feature values; To enhance eigenvalues; S22. A weighted feature fusion module (WFF) is introduced into the neck network. WFF uses learnable weights to normalize and weight features at different scales, and adaptively adjusts the fusion ratio of deep semantic features and shallow detail features. Fusion output The expression is as follows: ; In the formula, To prevent small quantities with a denominator of zero; For deep feature maps; This is a shallow feature map; and These are used to control the adaptive weights of deep and shallow features, respectively. S23. A dynamic head, DynamicHead, is used in the detection head. Scale attention, spatial attention, and task attention are introduced to dynamically weight multi-scale features, spatially salient regions, and classification and regression tasks, respectively. Constructing nested attention functions The expression is as follows: ; In the formula, To adaptively adjust the feature scale; To focus on spatial discrimination areas; For dynamic response classification and regression tasks; Multi-scale features output by the neck network; S24, Output Obstacle Category Two-dimensional detection frame or bounding box And confidence level.

[0009] Preferably, the specific content of S3 is as follows: Matching costs are calculated for the epipolar-corrected left and right images; Census transform is used to enhance robustness to illumination changes, and semi-global cost aggregation SGBM is performed to obtain a preliminary disparity map. The penalty coefficient is dynamically adjusted based on the image gradient. The expression is as follows: ; In the formula, This is the edge threshold; To reduce cross-boundary smoothing at the edges; To enhance smoothness consistency at non-edge locations; This represents the image gradient.

[0010] Preferably, SGBM dynamically adjusts the smoothing penalty parameter based on the image gradient during cost aggregation, reducing cross-boundary smoothing at semantic boundaries and enhancing consistent smoothing in non-boundary regions.

[0011] Preferably, the specific content of S4 is as follows: Introducing the lightweight semantic boundary network PIDNet-S, which predicts semantic boundary probability maps using only the left view. ; A weighted least squares optimization model is constructed to smooth the disparity edges, as shown in the following expression: ; In the formula, It is a set of 4-neighborhoods or 8-neighborhoods; For smoothness intensity; For the refined parallax to be optimized; Initial parallax; For pixels in an image; Smoothing weights Combining color similarity with semantic boundary constraints, the expression is as follows: ; In the formula, and Left view In pixels and The color value at that location; This refers to the color smoothness parameter; For boundary smoothness parameters; when When crossing semantic boundaries, weights Attenuation, blocking errors across objects, smoothing while preserving sharp edges, using conjugate gradients, iterating several times to obtain refined parallax. .

[0012] Preferably, the specific content of S6 is as follows: S61. Construct the GrabCut energy function that integrates spatial location constraints; Let the set of image pixels be , pixels The tag is In the GrabCut framework, the split is expressed as minimizing an energy function, as shown in the following expression: ; In the formula, For the set of all pixel labels, Foreground / background color model parameters; in, For color data items, let the foreground / background be modeled by GMMs respectively. The unary cost expression is as follows: ; In the formula, RGB values ​​for pixel color; For the smoothing term, for the neighborhood system (4-neighborhood or 8-neighborhood), using the Potts form and introducing edge-preserving weights, the expression is as follows: ; In the formula, the weights Edge preservation based on color gradient can be used; S62. Perform anisotropic space priors; Since the viewpoint of the wheelchair camera is fixed, and the bottom central axis region of the image corresponds to the road surface directly in front of the wheelchair in most cases, this region is defined as the center coordinate of the "absolute ground region", expressed as follows: ; In the formula, The dimensions are the image width and height, with the coordinate system having the top left corner as the origin; Introducing anisotropic Gaussian spatial prior strengths, we calculate spatial prior weights. The expression is as follows: ; In the formula, The standard deviation of Gaussians in the horizontal direction; The standard deviation of Gaussians in the vertical direction; The foreground is penalized, expressed as follows: ; In the formula, For spatial prior weights; For indicator functions; set ; The final energy is in the standard binary Markov RandomField form. After each GMM update, GraphCut is used to solve for the pixel labels, resulting in a binary mask of obstacles in the foreground region. .

[0013] Preferably, the specific content of S7 is as follows: After obtaining the obstacle mask, depth calculation is performed within the obstacle area; Assume the binocular system has been calibrated and a baseline has been obtained. With focal length The disparity map is To estimate obstacle distances, the median value is used for parallax within the mask, as expressed below: ; The final obstacle depth is calculated using the following expression: ; In the formula, Pick .

[0014] Preferably, an intelligent wheelchair environmental perception system includes: a binocular camera, a processor, and a memory; the memory stores a computer program, which, when executed on the processor, causes the processor to perform the method as described in any one of claims 1-8.

[0015] Preferably, a computer-readable storage medium stores a computer program that, when run on a processor, performs the method as described in any one of claims 1-8.

[0016] Therefore, the intelligent wheelchair environmental perception method based on multidimensional attention mechanism and feature reorganization adopted in this invention has the following advantages: This method overcomes the drawbacks of high cost of traditional LiDAR solutions and low detection and ranging accuracy of traditional vision algorithms in complex backgrounds. Specifically, the improved YOLO-PA network, compared to the original YOLOv8 model, enhances feature extraction capabilities using the CA mechanism and DynamicHead. In particular, the WFF module effectively solves the information redundancy and conflict problems caused by direct stitching of features at different scales, resulting in better feature representation capabilities. Simultaneously, the improved SGBM algorithm combined with WLS filtering effectively solves the parallax discontinuity problem in common wheelchair-friendly road environments (such as white walls and concrete floors). Considering the actual rehabilitation environment, the detection and ranging fusion technology adopted in this invention can effectively suppress the interference of ground background noise on the ranging results, providing a highly reliable environmental perception solution for intelligent wheelchairs.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a diagram illustrating the overall framework of the intelligent wheelchair environment perception method based on multidimensional attention mechanism and feature reorganization of the present invention. Figure 2 A schematic diagram illustrating the improved YOLOv8 (YOLO-PA) object detection network architecture and weighted feature fusion; Figure 3 A parallax refinement framework diagram for the collaborative work of stereo matching branches and semantic boundary guiding branches; Figure 4 The graph shows the convergence and accuracy evaluation metrics of the YOLO-PA network on the training and validation sets. Figure 5 This is a diagram illustrating obstacle ROI region recognition and depth information mapping based on semantic guidance optimization. Figure 6 This is an experimental result of an intelligent wheelchair perceiving the position of obstacles in front of it in a continuous dynamic scene. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0020] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0021] The following is combined Figures 1-6 The embodiments of the present invention will be described in detail below.

[0022] Example 1 like Figure 1 As shown, the wheelchair obstacle detection and ranging method based on binocular stereo vision and improved YOLOv8 addresses problems such as frequent occlusion, significant lighting changes, and disparity holes and edge adhesion caused by weak ground texture in wheelchair driving environments. It employs a processing chain of "target detection ROI noise reduction → semantic boundary-guided stereo matching and disparity refinement → robust distance estimation (and optional spatial prior segmentation closed loop)" to achieve real-time and stable obstacle detection and ranging. The adopted technical solution includes the following steps: (1) In a wheelchair driving environment, a calibrated binocular camera is used to collect the front video stream, and the left and right views are preprocessed with epipolar correction to construct a unified imaging geometric space; (2) Construct an improved YOLOv8 object detection network (YOLO-PA). Embed coordinate attention (CA) mechanism in the backbone network to enhance the position awareness of feature maps in the horizontal and vertical directions; (3) Optimize the feature fusion strategy of the neck network. The original upsampling structure is retained, but a weighted feature fusion module (WFF) is introduced in the feature fusion stage to replace the traditional concat concatenation operation. The fusion ratio of deep semantic features and shallow detail features is adaptively adjusted by learnable weight parameters to solve the problem of imbalance of multi-scale obstacle feature information flow; (4) A dynamic head is used in the head part, which integrates scale, space and task attention mechanisms to achieve high-precision recognition of occluded and small obstacles; (5) The disparity map is calculated using the improved SGBM stereo matching algorithm based on Census transform. The Census transform is used to enhance the robustness to illumination changes. The disparity map is post-processed and optimized by combining the weighted least squares (WLS) filter. The left image is used as a guide image to smooth the disparity map and fill the holes in the weak texture area. (6) Based on the high-precision depth map generated by the two-dimensional bounding box output by the detection network and the stereo matching, an adaptive statistical filtering strategy is adopted to remove outlier noise points and calculate the absolute distance of the obstacle relative to the center of the wheelchair, so as to realize real-time ranging against interference from multiple noise sources.

[0023] The specific steps are as follows: Step 1: Construct a binocular stereo vision perception and target detection model for wheelchair driving scenarios; (1) Establish a geometric and mathematical model for binocular imaging and ranging; like Figure 3 As shown, a binocular camera system consists of two cameras arranged in parallel, left and right. Let any point in space... The projected pixel coordinates on the left and right camera image planes are respectively ; According to the pinhole imaging model, point depth It can be obtained by triangulation: ; in, The focal length of the camera; The distance between the binocular baselines; For disparity; to obtain high-precision disparity. In this embodiment, an imaging mapping relationship is established: ; in, This is the camera intrinsic parameter matrix; and These are the rotation matrix and the translation matrix, respectively. In some embodiments, the Zhang Zhengyou calibration method is used to calibrate the binocular camera, obtain the intrinsic parameter matrix, distortion parameters and extrinsic parameters, and perform epipolar correction on the acquired images so that corresponding points in the left and right views are located on the same horizontal scan line, thereby simplifying the two-dimensional search into a one-dimensional search.

[0024] (2) Construct an improved YOLOv8 (YOLO-PA) object detection network and output the ROI; To address the issues of missed detection and occlusion of small targets in low-angle images from wheelchairs, a structural improvement was made to YOLOv8, constructing a YOLO-PA detection network. Only the obstacle ROI regions output by the detection network are retained in the binocular images, thus suppressing the interference of complex backgrounds on distance estimation. Furthermore, depth information is fused with the detected spatial coordinates to achieve high-precision distance calculation. Improvements include: Feature Enhancement (Backbone) — Coordinate Attention (CA): CoordinateAttention is embedded deep within the backbone network to incorporate location information into channel attention, thereby enhancing the feature discrimination capability under occlusion and illumination changes. Its output features can be represented as: ; in and These are the attention weights generated along the horizontal and vertical directions, respectively.

[0025] Feature Fusion (Neck) - Weighted Feature Fusion (WFF): In the Neck stage, WFF replaces the traditional Concat. It improves the stability of multi-scale information interaction by adaptively adjusting the fusion ratio of deep semantic features and shallow detail features through learnable weights.

[0026] Its fusion output is: ; in To prevent small quantities with a denominator of zero.

[0027] Dynamic perception (Head) — DynamicHead nested attention: Introducing DynamicHead to enhance the ability to adapt to differences in scale, space, and task (classification / regression).

[0028] Constructing nested attention functions: ; in To adaptively adjust the feature scale; To focus on spatial discrimination areas; For dynamic response classification and regression tasks; Multi-scale features output by the neck network; YOLO-PA outputs obstacle categories. Two-dimensional detection frame (or bounding box) The confidence level and confidence level are used for subsequent stereo matching and distance measurement calculations.

[0029] Step 2: Semantic boundary-guided stereo matching and parallax refinement; To obtain high-quality depth information, this invention constructs a disparity calculation framework that works collaboratively between a stereo matching branch and a semantic boundary-guided branch: first, a preliminary disparity map is quickly generated on the CPU, and then edge-preserving refinement is performed on the GPU using semantic boundary priors, outputting a depth map with clear edges and suppressed noise. (3) Preliminary disparity is obtained by improving SGBM matching based on Census transform; Matching costs are calculated for the epipolar-corrected left and right images. This embodiment employs Census transform to enhance robustness to illumination changes, followed by semi-global cost aggregation (SGBM) to obtain a preliminary disparity map. In some embodiments, to suppress edge adhesion, the penalty coefficient is dynamically adjusted based on the image gradient during the aggregation process. : ; in For edge threshold, Used to reduce cross-boundary smoothing at edges This is used to enhance smoothness and consistency in non-edge areas. However, due to the weak ground texture and edge occlusion in wheelchair scenes, the initial parallax may still exhibit scattered noise, holes, and depth leakage, thus requiring further refinement.

[0030] (4) WLS disparity refinement with semantic boundary constraints (PIDNet-S guidance); To suppress foreground-background adhesion and depth leakage at object edges, a lightweight semantic boundary network, PIDNet-S, is introduced, which predicts semantic boundary probability maps using only the left view. (SemanticBoundaryMap), serving as a key guiding prior for subsequent refinement, is used to obtain the initial parallax. With boundary probability Then, a weighted least squares (WLS) optimization model is constructed to smooth the disparity edges: ; in It is a set of 4-neighborhoods or 8-neighborhoods. Smoothing intensity. Smoothing weight. Simultaneously, it combines color similarity with semantic boundary constraints: ; when Crossing semantic boundaries (i.e.) When the weight is relatively large, Attenuation, thereby blocking cross-object error smoothing and preserving sharp edges, can be achieved using conjugate gradients, iterating several times to obtain the refined parallax. .

[0031] Step 3: Integrating adaptive segmentation and robust ranging based on spatial priors; Traditional GrabCut primarily relies on foreground / background color statistics and boundary smoothing terms for segmentation. However, in wheelchair driving scenarios, road surfaces (such as concrete and asphalt) and low obstacles (such as stones and steps) often exhibit weak textures or metamerism, causing energy terms based solely on color to easily fall into local optima, resulting in undersegmentation (misclassifying obstacles as road surfaces) or oversegmentation (misclassifying road surface noise as obstacles). Therefore, this invention introduces a spatial position prior consistent with the wheelchair's perspective geometry to suppress false detections in the bottom road surface region and stabilize the foreground region.

[0032] (5) GrabCut energy function with spatial location constraints; Let the set of image pixels be , pixels The tag is (0 represents the background (BG), and 1 represents the foreground (FG). In the GrabCut framework, segmentation is expressed as minimizing an energy function: ; in For the set of all pixel labels, These are the parameters for the foreground / background color model.

[0033] Color data item (GMM likelihood): Let the foreground and background be modeled by GMMs respectively, and their univariate costs can be written in the form of negative log-likelihood: ; in RGB represents pixel color.

[0034] Smoothing term (boundary continuity): For neighborhood systems (4-neighborhood or 8-neighborhood), using the Potts form and introducing edge-preserving weights: ; in ersonbracket; weight An edge-preserving approach based on color gradients can be adopted (the greater the color difference, the smaller the cross-boundary penalty) to encourage the segmentation boundaries to be distributed along the edges of real objects.

[0035] (6) Prior terms in anisotropic space: Inverted Gaussian "road surface confidence" penalty; Considering the fixed viewpoint of the wheelchair camera and that the bottom central axis area of ​​the image corresponds to the road surface directly in front of the wheelchair in most cases, this area is defined as the center of the "absolute ground area": ; in The dimensions are the image width and height, with the coordinate system having the top left corner as the origin.

[0036] To explicitly suppress the misclassification of the bottom road surface as the foreground (obstacle), an anisotropic Gaussian form of spatial prior strength is introduced (which can be interpreted as road surface / background confidence): ; This formula satisfies: the closer to the bottom center... The larger the value, the more it resembles the road surface background.

[0037] Since the penalty is maximized when a pixel is labeled as foreground, the spatial term must be label-dependent (otherwise it's a constant and ineffective for optimization), therefore the penalty is only applied to the foreground: ; ; in For spatial prior weights, Let be the indicator function. Thus, the unary cost of FG / BG can be equivalently written as "If an attempt is made to classify a region as FG near the bottom center, the unary term is significantly raised, thereby forcing the region to be more inclined towards BG; when the region is far from the bottom center, the spatial penalty is weakened, and the segmentation is mainly determined by the color term and the smooth term, without suppressing real obstacles above / far away."

[0038] Furthermore, considering wheelchair kinematic constraints: a longer longitudinal warning distance is required (the range of attention extends along the direction of travel), while the lateral range of attention is limited by the width of the vehicle body; therefore, a longer warning distance is needed. This results in slower longitudinal attenuation and faster lateral attenuation, thus forming a "narrow ellipse" constraint extending along the driving direction: strong lateral suppression of noise from the roadside / grass on both sides, and longitudinal allow for the segmentation and transition of areas at greater distances.

[0039] Combining the above energy terms, the final energy is still in the standard binary Markov RandomField form. GraphCut can be used to solve for the pixel labels after each GMM update to obtain the foreground region, i.e., the binary mask of the obstacles. .

[0040] (3) Obstacle ranging based on segmented masks; After obtaining the obstacle mask, the system does not need to perform full-image disparity statistics, but only performs depth calculations within the obstacle area, thereby reducing noise interference and computational load. Assume the binocular system has been calibrated and a baseline has been obtained. With focal length The disparity map is To robustly estimate obstacle distances, a median value can be used for the in-mask parallax: ; The final obstacle measurement depth (distance) is: ; in Pick This demonstrates the closed loop of "segmentation-range measurement": spatial prior stable mask → mask suppresses background noise → more reliable disparity statistics → more stable distance estimation.

[0041] The invention is further described below using numerical simulations and experimental data. Table 1 shows the ablation experiment results, and Table 2 compares the performance of different detection methods, as detailed below: Table 1 Ablation Experiment Results

[0042] Table 2 Performance Comparison of Different Detection Methods

[0043] Figure 4 This paper demonstrates the convergence and performance metrics of the proposed YOLO-PA network during training. Training stability analysis: (The provided text appears to be incomplete and contains several errors. A more accurate translation would require the full context.) Figure 4 The training curves show that as the number of iterations increases, the localization loss (box_loss) and classification loss (obj_loss) steadily decrease and tend to plateau. Meanwhile, precision, recall, and mean average precision (mAP) rise rapidly and saturate around 300 iterations, demonstrating that the improved network has good learning ability and convergence stability.

[0044] Improved detection accuracy (see Tables 1 and 2): On a test set containing complex wheelchair scenarios, the proposed method (YOLO-PA) achieved an mAP@0.5 of 80.1%, a 6.3 percentage point improvement over the baseline model YOLOv8s (73.8%). In the high-precision evaluation metric mAP@0.5:0.95, the proposed method achieved 62.0%, a 7.1 percentage point improvement over the original model. This verifies the superior performance of the CA mechanism, WFF module, and DynamicHead in extracting fine-grained obstacle features.

[0045] Ablation experiment analysis (see Table 1): By comparing Group 1 to Group 5, it can be seen that introducing coordinate attention (CA) or weighted feature fusion (WFF) alone can improve accuracy. However, YOLO-PA with the combination of the three improvements achieves the best results in all indicators, which proves the synergistic enhancement effect between the modules.

[0046] Figure 5 The system demonstrates its perception performance in a typical indoor staircase scenario. Visual analysis shows that the system successfully identified the staircase area ahead, achieving a detection confidence level of 0.77. The depth map on the right shows that after PIDNet-S semantic boundary guidance and WLS filtering, the edge contours of the stair treads are clear, and the depth information exhibits a continuous and smooth gradient transition, effectively overcoming the parallax noise and void problems that traditional algorithms easily produce on weakly textured stone surfaces.

[0047] Figure 6 The robustness of the system in continuous dynamic scenarios was further verified. Multi-distance ranging stability: During continuous detection from 1.71m to 4.34m, the system's confidence level in identifying stairs consistently remained above 0.93, with smooth changes in ranging values. Even at greater distances and with diminishing visual features, the system still provided stable depth feedback.

[0048] Real-time performance and engineering feasibility evaluation (see Table 2): The YOLO-PA of this invention has 12.20M parameters and a model size of only 24.4MB. Although this is slightly larger than the original version, the inference speed on a high-end GPU can still reach 162FPS. When deployed on an embedded platform (such as NVIDIA Jetson), its performance of 35FPS fully meets the real-time obstacle avoidance requirements of wheelchairs in complex environments.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent wheelchair environmental perception method based on multidimensional attention mechanism and feature reorganization, characterized in that, Includes the following steps: S1. Acquire left and right images from the calibrated binocular camera and perform distortion correction and epipolar correction; S2. Construct an improved YOLOv8 object detection network and obtain obstacle categories and 2D bounding boxes; S3. The initial disparity map is calculated using Census transform and semi-global matching SGBM based on the epipolar-corrected left and right images; S4. The initial disparity map is refined by weighted least squares (WLS) based on the semantic boundary probability map predicted by the left view to obtain the refined disparity map. S5. Based on the two-dimensional bounding box obtained in S2, determine the region of interest (ROI), and perform statistical filtering on the refined disparity map obtained in S4 within the ROI to remove outliers. S6. Introduce an anisotropic spatial prior term consistent with the wheelchair view geometry into the GrabCut energy function to suppress the missegmentation of the bottom road surface area, and perform the disparity statistics based on the segmentation mask. S7. Based on the calibration parameters, convert the parallax within the ROI into depth and output the absolute distance of the obstacle relative to the center of the wheelchair.

2. The intelligent wheelchair environment perception method based on multidimensional attention mechanism and feature reorganization according to claim 1, characterized in that, The specific content of S1 is as follows: S11. Use a calibrated binocular camera to acquire left and right images, and use the Zhang Zhengyou calibration method to obtain intrinsic parameters, distortion parameters and extrinsic parameters; Suppose any point in space The projected pixel coordinates on the left and right camera image planes are as follows: ; S12. Perform distortion correction and epipolar correction on the image to make corresponding points in the left and right views collinear, simplifying the matching search; S13. Establish a basic model for binocular ranging, and calculate points based on triangulation according to the pinhole imaging model. depth The expression is as follows: ； In the formula, The focal length of the camera; The distance between the binocular baselines; For parallax; The imaging mapping relationship is established as follows: ； In the formula, This is the camera intrinsic parameter matrix; and These are the rotation matrix and the translation matrix, respectively; For depth scale factor; These are pixel coordinates; It represents three-dimensional coordinates.

3. The intelligent wheelchair environmental perception method based on multidimensional attention mechanism and feature reorganization according to claim 2, characterized in that, The specific details of S2 are as follows: S21. Improve the YOLOv8 object detection network by embedding a coordinate attention mechanism (CA) in the backbone network, as shown in the following expression: ； In the formula, and These are the attention weights generated along the horizontal and vertical directions, respectively; For the input feature map, the first Each channel, coordinates are Location feature values; To enhance eigenvalues; S22. A weighted feature fusion module (WFF) is introduced into the neck network. WFF uses learnable weights to normalize and weight features at different scales, and adaptively adjusts the fusion ratio of deep semantic features and shallow detail features. Fusion output The expression is as follows: ； In the formula, To prevent small quantities with a denominator of zero; For deep feature maps; This is a shallow feature map; and These are used to control the adaptive weights of deep and shallow features, respectively. S23. A dynamic head, DynamicHead, is used in the detection head. Scale attention, spatial attention, and task attention are introduced to dynamically weight multi-scale features, spatially salient regions, and classification and regression tasks, respectively. Constructing nested attention functions The expression is as follows: ； In the formula, To adaptively adjust the feature scale; To focus on spatial discrimination areas; For dynamic response classification and regression tasks; Multi-scale features output by the neck network; S24, Output Obstacle Category Two-dimensional detection frame or bounding box And confidence level.

4. The intelligent wheelchair environment perception method based on multidimensional attention mechanism and feature reorganization according to claim 3, characterized in that, The specific details of S3 are as follows: Matching costs are calculated for the epipolar-corrected left and right images; Census transform is used to enhance robustness to illumination changes, and semi-global cost aggregation SGBM is performed to obtain a preliminary disparity map. The penalty coefficient is dynamically adjusted based on the image gradient. The expression is as follows: ； In the formula, This is the edge threshold; To reduce cross-boundary smoothing at the edges; To enhance smoothness consistency at non-edge locations; This represents the image gradient.

5. The intelligent wheelchair environment perception method based on multidimensional attention mechanism and feature reorganization according to claim 4, characterized in that: During cost aggregation, SGBM dynamically adjusts the smoothing penalty parameter based on the image gradient, reducing cross-boundary smoothing at semantic boundaries and enhancing consistent smoothing in non-boundary regions.

6. The intelligent wheelchair environmental perception method based on multidimensional attention mechanism and feature reorganization according to claim 5, characterized in that, The specific details of S4 are as follows: Introducing the lightweight semantic boundary network PIDNet-S, which predicts semantic boundary probability maps using only the left view. ; A weighted least squares optimization model is constructed to smooth the disparity edges, as shown in the following expression: ； In the formula, It is a set of 4-neighborhoods or 8-neighborhoods; For smoothness intensity; For the refined parallax to be optimized; Initial disparity; For pixels in an image; Smoothing weights Combining color similarity with semantic boundary constraints, the expression is as follows: ； In the formula, and Left view In pixels and The color value at that location; This refers to the color smoothness parameter; For boundary smoothness parameters; when When crossing semantic boundaries, weights Attenuation, blocking errors across objects, smoothing while preserving sharp edges, using conjugate gradients, iterating several times to obtain refined parallax. .

7. The intelligent wheelchair environmental perception method based on multidimensional attention mechanism and feature reorganization according to claim 6, characterized in that, The specific details of S6 are as follows: S61. Construct the GrabCut energy function that integrates spatial location constraints; Let the set of image pixels be , pixels The tag is In the GrabCut framework, the split is expressed as minimizing an energy function, as shown in the following expression: ； In the formula, For the set of all pixel labels, Foreground / background color model parameters; in, For color data items, let the foreground / background be modeled by GMMs respectively. The unary cost expression is as follows: ； In the formula, RGB values ​​for pixel color; For the smoothing term, for the neighborhood system (4-neighborhood or 8-neighborhood), using the Potts form and introducing edge-preserving weights, the expression is as follows: ； In the formula, the weights Edge preservation based on color gradient can be used; S62. Perform anisotropic space priors; Since the viewpoint of the wheelchair camera is fixed, and the bottom central axis region of the image corresponds to the road surface directly in front of the wheelchair in most cases, this region is defined as the center coordinate of the "absolute ground region", expressed as follows: ； In the formula, The dimensions are the image width and height, with the coordinate system having the top left corner as the origin; Introducing anisotropic Gaussian spatial prior strengths, we calculate spatial prior weights. The expression is as follows: ； In the formula, The standard deviation of Gaussians in the horizontal direction; The standard deviation of Gaussians in the vertical direction; The foreground is penalized, expressed as follows: ； In the formula, For spatial prior weights; For indicator functions; set ; The final energy is in the standard binary Markov RandomField form. After each GMM update, GraphCut is used to solve for the pixel labels, resulting in a binary mask of obstacles in the foreground region. .

8. The intelligent wheelchair environment perception method based on multidimensional attention mechanism and feature reorganization according to claim 7, characterized in that, The specific details of S7 are as follows: After obtaining the obstacle mask, depth calculation is performed within the obstacle area; Assume the binocular system has been calibrated and a baseline has been obtained. With focal length The disparity map is To estimate obstacle distances, the median value is used for parallax within the mask, as expressed below: ； The final obstacle depth is calculated using the following expression: ； In the formula, Pick .

9. An intelligent wheelchair environmental perception system, characterized in that, include: A binocular camera, a processor, and a memory; the memory stores a computer program that, when executed on the processor, causes the processor to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The device contains a computer program that, when run on a processor, performs the method as described in any one of claims 1-8.

Images