Visual identification guided robot obstacle avoidance method

Abstract

The invention discloses a robot obstacle avoidance method guided by visual identification, and relates to the technical field of robot control and computer vision. The method comprises the following steps: acquiring an environment image through a multi-source visual sensor, and preprocessing through an improved Zhang Zhengyou calibration method and the like; an improved multi-scale gradient fusion algorithm is adopted to extract an obstacle edge and a candidate region; through multi-feature joint extraction, double-layer adaptive dimension reduction and an integrated classifier, precise obstacle recognition is realized; constructing a multi-dimensional risk factor system and a nonlinear coupling model to evaluate the collision risk degree; planning an obstacle avoidance path based on an improved dynamic window method; and an anti-interference second-order backstepping controller is utilized to realize accurate track tracking. According to the method, the complex environment adaptability, the recognition precision and the trajectory control stability are effectively improved, the misrecognition and missed recognition risks are reduced, the high-precision and high-reliability obstacle avoidance requirements of multi-scene robot autonomous navigation are met, and engineering popularization is easy.

Description

Technical Field

[0001] This invention relates to the fields of robot control and computer vision technology. Specifically, it relates to a visual recognition-guided robot obstacle avoidance method, which is applicable to various intelligent devices with dynamic obstacle avoidance requirements, such as mobile robots, service robots, and autonomous navigation robots. It can be widely used in scenarios such as industrial production, warehousing and logistics, smart homes, and outdoor inspection. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, mobile robots are being used more and more widely in various industries. Obstacle avoidance, as a core technology for autonomous robot navigation, directly determines the safety, stability, and reliability of robot operation. Currently, robot obstacle avoidance methods are mainly divided into three categories: visual recognition-guided, lidar-guided, and multi-sensor fusion methods. Among them, visual recognition-guided methods have become one of the mainstream application solutions due to their advantages such as lower cost and richer information acquisition dimensions. However, existing visual recognition-guided obstacle avoidance technologies still have many shortcomings: In complex dynamic environments, traditional visual sensor calibration methods are easily affected by image distortion, resulting in insufficient accuracy of environmental image preprocessing, which in turn reduces the accuracy of obstacle edge detection and feature extraction; during obstacle recognition, single feature extraction and simple dimensionality reduction algorithms are difficult to adapt to the differentiated characteristics of multiple types of obstacles, resulting in poor classification accuracy and robustness, and easy misidentification and missed identification; collision risk assessment often relies on a single spatial or temporal factor, lacking multi-dimensional risk coupling analysis, making it difficult to accurately reflect collision risks under complex situations, resulting in insufficient adaptability of obstacle avoidance strategies; in path planning and trajectory tracking control, traditional algorithms have weak collaborative optimization capabilities for collision risk, path smoothness, and target approach, and insufficient anti-interference performance, easily leading to trajectory deviation and untimely obstacle avoidance in dynamic obstacle sudden scenarios, failing to meet the requirements of high-precision and high-reliability autonomous navigation. Therefore, developing a robot obstacle avoidance method that can improve adaptability to complex environments, recognition accuracy, risk assessment accuracy, and trajectory control stability has become a technical challenge that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0003] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0004] In view of the problems existing in the above and / or prior art, the present invention is proposed.

[0005] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a visual recognition-guided robot obstacle avoidance method, comprising the following steps: Step S1: Environmental image acquisition and preprocessing. The original images of the 360° environment around the robot are acquired through a multi-source vision sensor array. The improved Zhang Zhengyou calibration method is used to calibrate the sensors to eliminate image distortion. Then, the images are processed by adaptive multi-scale noise suppression and enhancement to output high-quality environmental images. Step S2: Obstacle edge detection and candidate region extraction. An improved multi-scale gradient fusion algorithm is used to enhance obstacle edge features. A continuous and complete edge image is obtained through adaptive dual-threshold edge extraction and connection. Then, obstacle candidate regions are extracted based on morphological closing operation and an improved region growing algorithm. Step S3: Obstacle feature extraction and classification recognition. The HOG-LBP-geometric feature-deep feature joint extraction algorithm is used to obtain high-dimensional fusion features. After processing by a two-layer adaptive dimensionality reduction algorithm based on local Fisher discriminant analysis and kernel principal component analysis, the features are input into the improved decision tree-support vector machine ensemble classifier, and the obstacle category, morphological attributes and motion state are output. Step S4: Obstacle collision risk assessment. Construct a multi-dimensional risk factor system covering static attributes, dynamic behavior, feature reliability and environmental adaptability. Combine the encounter situation judgment to establish a nonlinear coupled assessment model, output the comprehensive collision risk CRI, and realize risk level classification and adaptive adjustment of early warning threshold based on fuzzy clustering and environmental feedback. Step S5: Based on the improved dynamic window method, an obstacle avoidance path planning is carried out. A dynamic window is constructed with the comprehensive collision risk CRI as the core constraint. A multi-objective optimization evaluation function that integrates collision risk, path smoothness and target approach is designed. An obstacle avoidance path is generated through rolling optimization and B-spline curve smoothing. Step S6: Based on the backstepping method, an obstacle avoidance trajectory tracking control is implemented. A tracking error model that integrates multi-dimensional errors is constructed, and an anti-interference second-order backstepping controller with a dynamic weighted virtual control law is designed. Combined with a multi-modal switching mechanism and iterative learning compensation, accurate tracking of the obstacle avoidance trajectory is achieved.

[0006] The improved Zhang Zhengyou calibration method in step S1 introduces a dynamic weighting factor. ,in By iteratively optimizing the distortion parameters, the calibration error is reduced to the pixel level. The adaptive multi-scale noise suppression and enhancement employs an improved median filtering algorithm that integrates spatial and frequency domains, with a dynamic adjustment function for the filtering window. Furthermore, it combines wavelet threshold denoising and adaptive contrast enhancement algorithms to improve image quality.

[0007] The improved multi-scale gradient fusion algorithm in step S2 constructs a Gaussian scale spatial image pyramid. ,in ( , For scale intervals, (As the initial scale), the weighting coefficients are determined using the gradient entropy and the variance of the gradient magnitude. The gradient magnitude is obtained by fusion. The adaptive dual-threshold edge extraction uses an improved sliding window Otsu algorithm to dynamically determine the high threshold. With low threshold , And based on gradient direction connectivity analysis, the connection edges are connected.

[0008] In step S3, the high-dimensional fused feature vector output by the HOG-LBP-geometric feature-deep feature joint extraction algorithm is 456-dimensional, including 256-dimensional HOG features, 64-dimensional MB-LBP features, 8-dimensional geometric feature vector, and 128-dimensional deep feature vector. The first layer of the two-layer adaptive dimensionality reduction algorithm reduces the features to 128 dimensions through local Fisher discriminant analysis, and the second layer introduces an attention mechanism and kernel principal component analysis, ultimately compressing the feature dimension to 64 dimensions. The improved decision tree-support vector machine ensemble classifier constructs a decision tree based on the Euclidean distance between class centers, the probability of class occurrence, and the contribution of feature variance. Each node uses a weighted SVM as a base classifier, and the classification results are fused through weighted voting.

[0009] The multi-dimensional risk factor system in step S4 includes spatial risk factors. The time risk factor T of dynamic obstacles r and characteristic reliability factor The nonlinear coupling evaluation model is as follows: ,in This represents the situational weighting coefficient.

[0010] The dynamic window in step S5 A CRI weight correction mechanism is introduced, with the weight function being: , The multi-objective optimization evaluation function And dynamically adjust the weighting coefficients λ1, λ2, and λ3 based on the CRI value. 3； The rolling optimization uses a circular arc trajectory prediction model to generate a predicted trajectory within the next T=3s. After multi-step collision verification, the path is smoothed using a cubic B-spline curve.

[0011] In step S6, the tracking error model defines the position error. Heading error and speed error Virtual control law is , and control coefficient , The disturbance-resistant second-order backstepping controller integrates dynamic surface control and an extended state observer, avoids the "differential explosion" problem through a low-pass filter, and estimates and compensates for unknown disturbances; the multi-modal switching mechanism adjusts the controller parameters according to the trajectory curvature and risk mutation detection results, and introduces an iterative learning compensation term to optimize tracking performance.

[0012] In step S4, the risk levels are divided into low risk (0~0.3), medium risk (0.3~0.7), and high risk (0.7~1.0), corresponding to three obstacle avoidance strategies: deceleration, steering, and emergency braking, respectively. The high-risk threshold... When the risk mutation detection operator When the value is greater than 0.25 / s, an emergency warning and emergency obstacle avoidance mode are triggered, shortening the prediction time step and control response delay.

[0013] This invention discloses a visual recognition-guided obstacle avoidance method for robots, which has the following beneficial effects: By improving the Zhang Zhengyou calibration method and adaptive multi-scale image enhancement technology, this invention effectively eliminates image distortion and noise interference. Combined with multi-feature joint extraction and a two-layer adaptive dimensionality reduction algorithm, along with an integrated classifier, it achieves accurate obstacle recognition, adapting to various types of dynamic and complex environments and reducing misidentification and missed identification. A multi-dimensional risk factor system and a nonlinear coupled evaluation model are constructed, integrating multiple factors such as space, time, feature reliability, and encounter situation. Simultaneously, adaptive risk level classification and rapid detection of sudden risks are achieved, accurately quantifying collision hazard and triggering adaptive obstacle avoidance strategies in advance, thus improving robot operational safety. With collision hazard as the core constraint, a multi-objective optimized path planning algorithm and an anti-interference second-order backstepping controller are designed, balancing path smoothness, target approach, and trajectory tracking accuracy. Through dynamic weight adjustment and iterative learning compensation, path jitter and trajectory deviation are effectively avoided. The entire method, through a multi-stage adaptive mechanism, can adapt to the environmental differences of different scenarios such as industry, warehousing, and outdoors. It does not require extensive parameter debugging for specific scenarios, and the algorithm has high computational efficiency, which can meet the real-time obstacle avoidance requirements of robots and is easy to integrate and promote in engineering. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0015] Figure 1 A flowchart of a visual recognition-guided obstacle avoidance method for robots; Figure 2 This is a flowchart of environmental image acquisition and preprocessing in this invention; Figure 3 This is a flowchart of obstacle edge detection and candidate region extraction in this invention; Figure 4 This is a flowchart of obstacle feature extraction and classification in this invention; Figure 5 This is a flowchart of the obstacle collision risk assessment in this invention; Figure 6 This is a flowchart of the obstacle avoidance path planning based on the improved dynamic window method in this invention; Figure 7 This is a flowchart of the obstacle avoidance trajectory tracking control based on the backstepping method in this invention; The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the examples in the specification.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0020] Figure 1 This is a flowchart illustrating a vision-guided robot obstacle avoidance method according to an embodiment of the present invention. It should be noted that, if substantially the same result is achieved, the vision-guided robot obstacle avoidance method of the present invention does not necessarily follow the same approach. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this visual recognition-guided robot obstacle avoidance method mainly includes the following steps: Step S1: Environmental image acquisition and preprocessing; Step S2: Obstacle edge detection and candidate region extraction; Step S3: Obstacle feature extraction and classification; Step S4: Obstacle collision risk assessment; Step S5: Obstacle avoidance path planning based on the improved dynamic window method; Step S6: Obstacle avoidance trajectory tracking control based on inverse stepping method; In a preferred embodiment of the present invention, step S1: environmental image acquisition and preprocessing includes the following steps: Multi-source vision sensor image acquisition and calibration were performed using an array of multiple vision sensors with different viewpoints to acquire raw images of the robot's surrounding environment with 360-degree coverage, ensuring comprehensive capture of static and dynamic obstacle information. Considering the unavoidable errors in sensor manufacturing and installation that can cause radial and tangential distortion in the acquired images, thus affecting the accuracy of subsequent obstacle recognition, an improved Zhang Zhengyou calibration method was used to calibrate the sensors and construct a unified image coordinate system to eliminate the effects of distortion. The distortion correction model is as follows:

[0021]

[0022]

[0023] in, The original coordinates of the distorted point. For the corrected coordinates, For radial distortion parameters, This refers to the tangential distortion parameter. To further improve calibration accuracy, a dynamic weighting factor is introduced. This factor can assign higher weights to pixels in the edge region of the image because the edge region often contains more key contour information of obstacles. By solving the distortion parameters through iterative optimization, the calibration error is finally reduced to the pixel level.

[0024] Next, adaptive multi-scale noise suppression and enhancement are performed, and an improved median filtering algorithm that integrates spatial and frequency domains is used to smooth the corrected image. First, a dynamic adjustment function for the filtering window is defined. , where k For the number of iterations, For the initial window size, Maximum window size This is the attenuation coefficient. During the filtering process, the average grayscale value of the pixels within the current filtering window is calculated. Standard deviation and entropy Construct noise judgment criteria: if and If it is determined to be a low-noise region, a small window filter is used to preserve image details; if and If the region is identified as a high-noise area, the window size is increased, and frequency domain wavelet thresholding is used to further improve the denoising effect. The wavelet thresholding function is designed as follows:

[0025] in , This represents the total number of pixels in the image. After the above filtering process, the image noise is effectively suppressed, but there may be insufficient contrast, affecting the subsequent obstacle edge detection. Therefore, an adaptive contrast enhancement algorithm is used. This enhances image detail contrast, making obstacles more clearly distinguishable from the background and providing high-quality input images for subsequent obstacle detection.

[0026] In a preferred embodiment of the present invention, step S2: obstacle edge detection and candidate region extraction includes the following steps: Multi-scale adaptive gradient edge enhancement is performed. Building upon the high-quality image preprocessed in step S1, an improved multi-scale gradient fusion algorithm is used to enhance obstacle edge features. First, a Gaussian scale space is constructed, and multi-scale Gaussian convolutions are performed on the preprocessed image to generate image pyramids of different scales. ,in ( , For scale intervals, (Initial scale). Gradient magnitude is calculated based on images at each scale. and gradient direction ,in They are respectively The Sobel gradient operator response in the direction. To address the issue of missing small components or false detection noise in single-scale edge detection, an adaptive scale weight fusion strategy is introduced, with weight coefficients... From gradient entropy ( (normalized probability of gradient direction histogram) and gradient magnitude variance Joint decision:

[0027] Where λ 1， λ² is the balance coefficient. The final fused gradient magnitude is... This achieves a balance between strong edge enhancement and weak edge preservation.

[0028] Next, adaptive dual-threshold edge extraction and concatenation are performed. Based on the enhanced gradient image from step S21, an improved sliding window Otsu algorithm is used to dynamically determine the dual thresholds. The image is then divided into... A histogram of gradient magnitudes is generated for each overlapping sliding window. By maximizing inter-class variance Solving for the optimal threshold ,in These represent the probabilities of the background and foreground, respectively. These are the mean values ​​for the background and foreground, respectively. The optimal threshold for the window is... The high threshold of this window low threshold Set as ( (Adaptive scaling factor). Apply double threshold filtering to the gradient image, retaining values ​​with amplitudes greater than [value missing]. The strong edges and amplitude are between and Furthermore, weak edges are connected to strong edges. To address the edge breakage problem, a connectivity analysis based on gradient direction is employed for weak edge points. If a strong edge point exists within its 8-neighborhood and the angle between the gradient directions of the two points is less than 30°, then the weak edge point is included in the effective edge set, ultimately forming a continuous and complete obstacle edge image.

[0029] Finally, candidate region extraction based on morphology and region growing is performed. For the edge image obtained in step S22, morphological closing operation is first performed to fill the edge gaps. The closing operation structuring element adopts an adaptive elliptic kernel. Among them, the long axis and short axis Adaptive adjustment based on edge density: , ( For local edge point density, (This is a scaling factor). Subsequently, an improved region growing algorithm is used to extract candidate regions, using non-zero pixels in the edge image as seed points. The growth criterion combines gray-level similarity and gradient consistency: let the gray level of the seed point be... The gray level of the growth point is The gradient direction is If satisfied ( (where the grayscale threshold is) and ( If a point is selected as the gradient direction threshold, it is included in the growth region. To avoid region merging due to overgrowth, a region constraint condition is introduced to calculate the compactness of the growth region. ( For the area, (where the perimeter is the area), when ( Growth stops when the compactness threshold is reached. Finally, the grown area is screened and pores are filled, discarding areas smaller than a certain threshold. or greater than The noise region is identified, resulting in the final set of obstacle candidate regions.

[0030] In a preferred embodiment of the present invention, step S3: obstacle feature extraction and classification includes the following steps: First, multi-dimensional fusion feature extraction is performed. Following the obstacle candidate region extracted in step S2, an improved HOG-LBP-geometric feature-depth feature joint extraction algorithm is used to comprehensively capture the contour, texture, shape, and spatial depth information of the obstacles. First, grayscale normalization and Gamma correction are performed on the candidate region to reduce the impact of illumination changes, dividing the region into blocks composed of 16x16 cells and 2x2 cells. The gradient direction histogram of each cell is calculated (gradient directions are divided into 18 bins), and Gaussian weighted smoothing is used to enhance feature robustness, yielding the HOG basic features. The MB-LBP algorithm with circular neighborhoods (neighborhood radius R=2, sampling points P=16) is used, with bilinear interpolation processing non-integer coordinates, followed by equivalent pattern encoding to reduce feature dimensionality and enhance local texture feature representation. A geometric feature extraction module is introduced to calculate the aspect ratio and compactness of the bounding rectangle of the candidate region. ( For the area, (perimeter of the region) and eccentricity ( Multi-dimensional geometric feature vectors are constructed using the major and minor axes of the fitted ellipse and the Hu moment invariants, respectively. Deep feature extraction is incorporated, and a modified lightweight CNN network (simplified MobileNet architecture) is used to encode candidate regions using three convolutional layers and two pooling layers, outputting a 128-dimensional deep feature vector. This feature vector adaptively captures the complex semantic information of obstacles. Finally, the normalized HOG features (256 dimensions), the encoded MB-LBP features (64 dimensions), the geometric feature vector (8 dimensions), and the deep feature vector (128 dimensions) are concatenated to form a high-dimensional fused feature vector (456 dimensions in total). This retains the strong interpretability of traditional handcrafted features while enhancing the representation ability of complex obstacles through deep features.

[0031] Following this, adaptive dimensionality reduction and enhancement are performed. To address the redundancy and curse of dimensionality issues in the high-dimensional fused features obtained in step S31, a two-layer adaptive dimensionality reduction algorithm based on Local Fisher Discriminant Analysis (LFDA) and Kernel Principal Component Analysis (KPCA) is proposed, and an attention mechanism is introduced to enhance the weights of key features. The first layer preserves the local structure of the data through LFDA, defining the weights of each sample... Nearest neighbor set ( ), calculate the within-class scatter matrix and between-class scatter matrix:

[0032]

[0033] in, The number of obstacle categories. For the first Class sample set, For the sample of nearest neighbor set For non Class sample set. By solving the generalized eigenvalue problem. Before selection The eigenvectors corresponding to the largest eigenvalues ​​form a projection matrix, mapping the high-dimensional features to a low-dimensional subspace (initially reduced to 128 dimensions). The second layer introduces an attention mechanism to calculate the information gain value for each feature dimension. ( (Information entropy), assigning weight coefficients to the top 80% of features based on information gain. This strengthens the contribution of key features. Subsequently, KPCA is used to further compress the dimensionality and enhance the nonlinear expression, employing a radial basis function (RBF):

[0034] in By optimizing the values ​​using 5-fold cross-validation (range 0.01~10), the feature dimension was reduced to 64 dimensions, resulting in enhanced features with low redundancy and high discriminative power. Finally, multi-class recognition based on improved decision tree-support vector machine (DT-SVM) and ensemble learning was performed. Addressing the high computational complexity and limited generalization ability of traditional multi-class SVMs, a hierarchical ensemble classifier was designed to achieve accurate obstacle recognition. First, the center vector of each class of samples was calculated. ( For the first (Number of samples in each class), by calculating the Euclidean distance between class centers. Define class Minimum distance to other classes Priority separation The largest category is used to construct the first-level node of the decision tree; when there are multiple categories, the first-level node of the decision tree is constructed; when there are multiple categories... When they are equal, combine the probability of occurrence of the category. and contribution of characteristic variance Prioritize separating categories with high probability and high variance contribution to reduce the accumulation of classification errors. Each node of the decision tree uses a weighted SVM as the base classifier, and the penalty parameter is adaptively adjusted according to the node sample distribution. (Assign higher weights to minority class samples when the sample is imbalanced) , The total number of samples, Given the number of samples in this class, a radial basis function kernel is selected, and its parameters are optimized through grid search. An ensemble learning strategy is introduced, employing weighted voting fusion of the SVM classification results for each leaf node of the decision tree. The weights are dynamically allocated based on the accuracy of each base classifier on the validation set. During the classification process, the enhanced features obtained in step S32 are input into the DT-SVM ensemble classifier. First, static obstacles are distinguished from dynamic obstacles. Then, the specific categories of static obstacles and the motion states of dynamic obstacles are refined. Finally, the obstacle category, morphological attributes, and motion state are output, providing comprehensive and accurate core input for subsequent path adjustment and obstacle avoidance decisions.

[0035] In a preferred embodiment of the present invention, step S4: obstacle collision risk assessment includes the following steps: Step S41: Refined modeling of multi-dimensional risk factors, closely linked to the feature extraction and classification results of Step S3, constructing a comprehensive risk factor system covering static attributes, dynamic behavior, feature reliability, and environmental adaptability. For static obstacles, extract the core geometric feature parameters from Step S31, including the aspect ratio, compactness, and Hu moment invariant of the circumscribed rectangle, and combine them with the actual physical dimensions after perspective transformation to construct a static risk basis. For dynamic obstacles, integrate the motion velocity, acceleration, and rate of change of heading angle captured by the depth features in Step S31, as well as the motion state (uniform speed, acceleration, turning) identified in Step S33, to form a dynamic behavior parameter set. Introduce an improved spatial risk factor. The formula integrating path overlap and distance attenuation effect is as follows:

[0036] in, The area of ​​overlap between the obstacle and the preset driving path. The effective area of ​​the path, The minimum distance between the obstacle and the centerline of the path. For safe distance threshold, This is the overlap weighting coefficient. This refers to the distance attenuation coefficient. A new adaptive time risk factor has been added for dynamic obstacles. Based on TCPA theory and introducing motion state correction terms:

[0037] in, ( The initial relative distance, Relative velocity, For relative acceleration, (For predicting the time step) For the maximum warning time, The time decay coefficient, This is an acceleration correction coefficient. Simultaneously, the feature information gain from step S32 and the classifier confidence are fused to construct a feature reliability factor. :

[0038] in, For the first Information gain of each feature For feature weights, The confidence score for the contribution of this feature to the classifier. For the number of effective features, This represents the total feature dimension.

[0039] Step S42: Integrate the risk coupling model with COLREGS rules and multi-source information. Based on the multi-dimensional risk factors in Step S41, and combined with encounter situation judgment and environmental adaptation correction, establish a nonlinear coupling evaluation model. First, using the obstacle categories and motion parameters identified in Step S33, calculate the encounter angle adapted to the ground scene. : ,in, For the vehicle's heading angle, The heading angle of the dynamic obstacle is used to classify three encounter situations: face-to-face (0°±5°), crossing (5°~112.5° and 247.5°~355°), and overtaking (112.5°~247.5°), and each situation is assigned a differentiated weighting coefficient. : Encountering Situation Cross-situation catching up and surpassing Static obstacles Construct a comprehensive collision hazard assessment. Coupled model, integrating spatial, temporal, reliability, and environmental correction factors:

[0040] in These are the weighting coefficients. The absolute value of the velocity difference between the carrier and the dynamic obstacle. For the speed of the carrier itself, This is the difference in heading angle. This is the heading adaptation coefficient. The model transforms the encounter responsibility allocation of the COLREGS rules into quantified weights, while also using characteristic reliability factors... It compensates for the uncertainty of feature extraction in complex environments and improves the robustness of evaluation.

[0041] Step S43: Dynamic risk level classification and adaptive adjustment of early warning threshold, based on the calculation in step S42. The system combines fuzzy clustering with real-time environmental feedback to achieve intelligent risk level classification and dynamic threshold correction. First, an improved K-means clustering algorithm is used to adaptively determine cluster centers based on sample distribution. The obstacle avoidance system is divided into three levels: low risk (0~0.3), medium risk (0.3~0.7), and high risk (0.7~1.0), corresponding to three obstacle avoidance strategies: deceleration, steering, and emergency braking, respectively. An environment-adaptive threshold correction mechanism is introduced, taking into account the current scene's illumination intensity (achieved through image grayscale variance). (Quantization) and obstacle occlusion rate (Calculated from the edge integrity features in step S31), dynamically adjust the high-risk threshold. :

[0042] in, The threshold value is set to the grayscale variance under ideal lighting conditions. When lighting is insufficient or there is severe occlusion, the threshold adaptively increases to reduce the false alarm rate. A risk mutation detection operator is designed to address abrupt changes in the motion state of dynamic obstacles. :

[0043] in, For the detection time interval, For the acceleration of the obstacle, For the maximum expected acceleration. When When a sudden risk event is detected, an emergency warning mechanism is triggered, and obstacle avoidance is prioritized. This step addresses the poor adaptability of traditional fixed thresholds through the dual protection of dynamic thresholds and sudden change detection. It forms a closed-loop optimization with the feature extraction results of step S3, ensuring the accuracy, real-time performance, and environmental adaptability of collision risk assessment.

[0044] In a preferred embodiment of the present invention, step S5, obstacle avoidance path planning based on the improved dynamic window method, includes the following steps: Step S51: Perform dynamic window and collision hazard coupling modeling, closely linking with the collision hazard assessment results of step S4, and construct a dynamic window optimization framework that integrates multiple constraints. The comprehensive collision hazard calculated in step S42 is used as the basis for this model. As the core constraint, combined with the dynamic characteristics of the carrier (maximum speed) Maximum acceleration Maximum angular velocity ), Build the initial dynamic window Its expression is:

[0045] in, These are the current velocity and angular velocity of the carrier, respectively. For maximum deceleration and maximum acceleration, For maximum angular deceleration and maximum angular acceleration, To predict the time step, we introduce... The weighting adjustment mechanism assigns differentiated weights to different velocity-angular velocity combinations within the window:

[0046] in This is the hazard attenuation coefficient. The larger the value, the lower the weight, thus prioritizing the elimination of high-risk area speed combinations. Simultaneously, this is combined with the spatial risk factor from step S41. We construct a dynamic adjustment model for the velocity obstacle (VO) set, correcting the fixed threat area assumption of traditional VO:

[0047] in , Based on the safe distance, This is a spatial risk correction coefficient that dynamically adapts the safe distance to the path overlap, improving obstacle avoidance robustness in complex scenarios.

[0048] Step S52: Design a multi-objective optimization evaluation function, integrating hazard feedback and path performance indicators to construct a hierarchical weighted evaluation system. Evaluation function Integrating three key indicators—collision risk, path smoothness, and target approach—its expression is as follows:

[0049] in, These are weighting coefficients. Collision risk indicators. Based on step S4 Design of speed obstacle sets:

[0050] For speed arrive The shortest distance between sets Distance attenuation coefficient. Path smoothness index. Curvature constraint design is adopted:

[0051] Maximum permissible curvature. Target approach index. Combining heading angle deviation and distance deviation:

[0052] The deviation between the current course and the target course. This represents the distance from the current location to the target point. To determine the maximum planning distance, a dynamic weight adjustment mechanism is introduced, based on... Real-time adjustment of the weights of each indicator: when hour, Increased to 0.6, Reduced to 0.15, prioritizing collision safety; when hour, Increased to 0.4, prioritizing path efficiency.

[0053] Step S53: Rolling optimization and path smoothing generation, based on multi-step verification of the predicted trajectory, to achieve dynamic updating and smoothing optimization of the obstacle avoidance path. First, the optimal speed combination selected in step S52 is... Using a circular trajectory prediction model to generate future Predicted trajectory within:

[0054] in , Given the current pose of the carrier. Perform multi-step collision verification on the predicted trajectory; if there exists a collision at a certain moment... satisfy Then, the evaluation function in step S52 is called again to update the optimal speed combination. Secondly, B-spline curves are introduced to smooth the path points obtained from the rolling optimization in order to predict the key nodes of the trajectory. Construct a cubic B-spline curve using the control points:

[0055] in for spline basis functions Using the coordinates of four adjacent control points, a smoothing process is applied to eliminate sharp abrupt changes in the path, improving the stability of the carrier's motion. Finally, combined with the risk mutation detection results from step S43, when... When the emergency obstacle avoidance mode is triggered, the predicted time step will be adjusted. The time has been shortened to 0.1 seconds, improving the response speed in the event of sudden dynamic obstacles and forming a closed-loop obstacle avoidance mechanism of "hazard assessment - window optimization - path smoothing".

[0056] In a preferred embodiment of the present invention, step S6: obstacle avoidance trajectory tracking control based on the backstepping method includes the following steps: Step S61: Error modeling and dynamic weighted virtual control law design, closely linked to the obstacle avoidance path planning results of step S5, using the output of step S53. The spline-smoothed trajectory is the desired trajectory. A tracking error model integrating multi-dimensional errors and environmental perception is constructed. Let the position of the desired trajectory be... Speed ​​is The actual state of the mobile carrier is Define position error Heading error ( (Desired trajectory heading angle), velocity error Considering the underactuated characteristics of the carrier, a virtual control variable is introduced. and Constructing a first-order Lyapunov function Differentiating it, we get: Design virtual control laws ( (where is the error adjustment coefficient), after substituting and simplifying, we get Further design of virtual control laws ,make This achieves initial error stabilization. The collision hazard level is then introduced in step S42. With the trajectory curvature in step S53 The coupled adaptive adjustment mechanism dynamically corrects the control coefficients. ,in Based on the coefficient, Risk level fit factor is the curvature adaptation coefficient. When... hour, Reduce to lower the speed tracking weights. Increase to prioritize ensuring course alignment; when trajectory curvature When the value is large, increase synchronously. and It enhances trajectory fit and adapts to obstacle avoidance scenarios during sharp turns.

[0057] Step S62: Construction of the dynamic surface-backstep composite controller. Based on the virtual control law in step S61, and combining the vehicle dynamic coupling characteristics and unknown disturbances, an anti-interference second-order backstep controller is designed. Define the angular velocity error. Construct synthetic Lyapunov functions Combining the three-degree-of-freedom dynamic equations of the carrier:

[0058] right Differentiate and substitute into the dynamic equations to design a speed controller. and heading controller The system integrates dynamic surface control and an extended state observer (ESO), and introduces a low-pass filter to smooth the derivative of the virtual control law. and To avoid the "differential explosion" problem of traditional backstepping methods, the filter takes the following form:

[0059] in These are the filter coefficients. Simultaneously, unknown disturbances are estimated using ESO. The observer is designed as follows:

[0060] The final controller expression with disturbance compensation is obtained:

[0061]

[0062] In the formula The error convergence coefficient is... The observed values ​​of the heading channel disturbance can be verified through Lyapunov stability analysis. This ensures that all errors are globally consistent and eventually bounded.

[0063] Step S63: Multimodal switching and tracking performance optimization. Combining the trajectory prediction information from step S5 and the risk mutation detection from step S43, an adaptive multimodal control mechanism is designed. Based on the predicted trajectory points from step S53... Real-time updates of trajectory curvature Furthermore, the controller weight coefficients are adjusted to enhance the tracking accuracy of complex trajectories. A risk mutation response mechanism is introduced, which, when the risk mutation detection operator in step S43... When this occurs, the emergency tracking mode is triggered: the filter coefficients are adjusted. Double the values ​​to 20 and 30 to shorten response delay; simultaneously increase the error adjustment coefficient. The speed is increased to 1.5 times the original, prioritizing heading tracking stability. To address the need for trajectory smoothness, an iterative learning compensation term is introduced, using the tracking error from the previous cycle to correct the current control output.

[0064] in For iterative learning gain, This represents the error from the previous cycle. A closed-loop mechanism of "error modeling - disturbance rejection control - mode switching" is formed to ensure the accuracy, robustness, and real-time performance of trajectory tracking in dynamic obstacle environments.

[0065] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A visual recognition-guided obstacle avoidance method for robots, characterized in that, Includes the following steps: Step S1: Environmental image acquisition and preprocessing. The original images of the 360° environment around the robot are acquired through a multi-source vision sensor array. The improved Zhang Zhengyou calibration method is used to calibrate the sensors to eliminate image distortion. Then, the images are processed by adaptive multi-scale noise suppression and enhancement to output high-quality environmental images. Step S2: Obstacle edge detection and candidate region extraction. An improved multi-scale gradient fusion algorithm is used to enhance obstacle edge features. A continuous and complete edge image is obtained through adaptive dual-threshold edge extraction and connection. Then, obstacle candidate regions are extracted based on morphological closing operation and an improved region growing algorithm. Step S3: Obstacle feature extraction and classification recognition. The HOG-LBP-geometric feature-deep feature joint extraction algorithm is used to obtain high-dimensional fusion features. After processing by a two-layer adaptive dimensionality reduction algorithm based on local Fisher discriminant analysis and kernel principal component analysis, the features are input into the improved decision tree-support vector machine ensemble classifier, and the obstacle category, morphological attributes and motion state are output. Step S4: Obstacle collision risk assessment. Construct a multi-dimensional risk factor system covering static attributes, dynamic behavior, feature reliability and environmental adaptability. Combine the encounter situation judgment to establish a nonlinear coupled assessment model, output the comprehensive collision risk CRI, and realize risk level classification and adaptive adjustment of early warning threshold based on fuzzy clustering and environmental feedback. Step S5: Based on the improved dynamic window method, an obstacle avoidance path planning is carried out. A dynamic window is constructed with the comprehensive collision risk CRI as the core constraint. A multi-objective optimization evaluation function that integrates collision risk, path smoothness and target approach is designed. An obstacle avoidance path is generated through rolling optimization and B-spline curve smoothing. Step S6: Based on the backstepping method, an obstacle avoidance trajectory tracking control is implemented. A tracking error model that integrates multi-dimensional errors is constructed, and an anti-interference second-order backstepping controller with a dynamic weighted virtual control law is designed. Combined with a multi-modal switching mechanism and iterative learning compensation, accurate tracking of the obstacle avoidance trajectory is achieved.

2. The visual recognition-guided robot obstacle avoidance method according to claim 1, characterized in that, The improved Zhang Zhengyou calibration method in step S1 introduces a dynamic weighting factor. ,in By iteratively optimizing the distortion parameters, the calibration error is reduced to the pixel level. The adaptive multi-scale noise suppression and enhancement employs an improved median filtering algorithm that integrates spatial and frequency domains, with a dynamic adjustment function for the filtering window. Furthermore, it combines wavelet threshold denoising and adaptive contrast enhancement algorithms to improve image quality.

3. The visual recognition-guided robot obstacle avoidance method according to claim 1, characterized in that, The improved multi-scale gradient fusion algorithm in step S2 constructs a Gaussian scale spatial image pyramid. ,in ( , For scale intervals, (As the initial scale), the weighting coefficients are determined using the gradient entropy and the variance of the gradient magnitude. The gradient magnitude is obtained by fusion. The adaptive dual-threshold edge extraction uses an improved sliding window Otsu algorithm to dynamically determine the high threshold. With low threshold , And based on gradient direction connectivity analysis, the connection edges are connected.

4. The visual recognition-guided robot obstacle avoidance method according to claim 1, characterized in that, In step S3, the high-dimensional fused feature vector output by the HOG-LBP-geometric feature-deep feature joint extraction algorithm is 456-dimensional, including 256-dimensional HOG features, 64-dimensional MB-LBP features, 8-dimensional geometric feature vector, and 128-dimensional deep feature vector. The first layer of the two-layer adaptive dimensionality reduction algorithm reduces the features to 128 dimensions through local Fisher discriminant analysis, and the second layer introduces an attention mechanism and kernel principal component analysis, ultimately compressing the feature dimension to 64 dimensions. The improved decision tree-support vector machine ensemble classifier constructs a decision tree based on the Euclidean distance between class centers, the probability of class occurrence, and the contribution of feature variance. Each node uses a weighted SVM as a base classifier, and the classification results are fused through weighted voting.

5. The visual recognition-guided robot obstacle avoidance method according to claim 1, characterized in that, The multi-dimensional risk factor system in step S4 includes spatial risk factors. The time risk factor T of dynamic obstacles r and characteristic reliability factor The nonlinear coupling evaluation model is as follows: ,in This represents the situational weighting coefficient.

6. The visual recognition-guided robot obstacle avoidance method according to claim 1, characterized in that, The dynamic window in step S5 A CRI weight correction mechanism is introduced, with the weight function being: , The multi-objective optimization evaluation function And dynamically adjust the weighting coefficients λ1, λ2, and λ3 based on the CRI value. 3； The rolling optimization uses a circular arc trajectory prediction model to generate a predicted trajectory within the next T=3s. After multi-step collision verification, the path is smoothed using a cubic B-spline curve.

7. The visual recognition-guided robot obstacle avoidance method according to claim 1, characterized in that, In step S6, the tracking error model defines the position error. Heading error and speed error Virtual control law is , and control coefficient , The disturbance-resistant second-order backstepping controller integrates dynamic surface control and an extended state observer, avoids the "differential explosion" problem through a low-pass filter, and estimates and compensates for unknown disturbances; the multi-modal switching mechanism adjusts the controller parameters according to the trajectory curvature and risk mutation detection results, and introduces an iterative learning compensation term to optimize tracking performance.

8. The visual recognition-guided robot obstacle avoidance method according to any one of claims 1-7, characterized in that, In step S4, the risk levels are divided into low risk (0~0.3), medium risk (0.3~0.7), and high risk (0.7~1.0), corresponding to three obstacle avoidance strategies: deceleration, steering, and emergency braking, respectively. The high-risk threshold... When the risk mutation detection operator When the value is greater than 0.25 / s, an emergency warning and emergency obstacle avoidance mode are triggered, shortening the prediction time step and control response delay.

Images