Edge end real-time visual auxiliary obstacle avoidance method based on improved YOLO algorithm

Abstract

An edge end real-time visual auxiliary obstacle avoidance method based on an improved YOLO algorithm belongs to the field of computer vision and deep learning, and comprises the following steps: S1, image acquisition and preprocessing; s2, target detection; s3, outputting a detection result; s4, path planning: generating a global path and a local obstacle avoidance strategy based on a detection result, and combining to form a motion track; and S5, motion control: generating a speed and steering angle instruction according to the motion track, and controlling the intelligent vehicle to move. According to the invention, the lightweight target detection model based on the improved YOLO algorithm and the application of the lightweight target detection model on the edge computing device JestanNano are combined with the vision-assisted obstacle avoidance system realized by the ROS, so that the real-time problem of edge end deployment is effectively solved while the detection precision is ensured, and a reliable technical scheme is provided for autonomous navigation of intelligent vehicles.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning, specifically a real-time visual-assisted obstacle avoidance method at the edge based on an improved YOLO algorithm. Background Technology

[0002] In the field of object detection, traditional methods have long relied on manually designed shallow feature descriptors such as Haar and HOG features, combined with classifiers like Adaboost and SVM, to achieve object recognition. This multi-stage processing mode involves complex processes such as sliding window scanning, feature extraction, and classifier selection, leading to an exponential increase in computational complexity, making it difficult to meet real-time requirements. Furthermore, manually designed features suffer significant degradation in representation capabilities when faced with dynamic scenes involving changes in target scale, sudden illumination changes, or partial occlusion, resulting in a high likelihood of missed or false detections. While the YOLO series is excellent, its large parameter count and high computational complexity require more powerful image processors for deployment. In recent years, with the development of edge computing and smart terminals, the deployment of artificial intelligence models on embedded devices has received widespread attention. However, the standard convolutional operations and coupled detection head designs still exhibit computational redundancy. In particular, Jetson Nano, due to its portability and low power consumption, is widely used in security and transportation scenarios. However, due to limitations in memory and computing power of embedded devices, a lighter-weight object detection framework is often needed to meet real-time processing requirements.

[0003] In the field of obstacle avoidance system technology, traditional solutions mostly rely on single sensors such as ultrasonic, infrared, or lidar. Ultrasonic sensors are easily affected by ambient temperature and multipath effects, resulting in significant ranging errors in complex structural scenes. Infrared sensors are severely affected by ambient light interference, and their detection range is greatly reduced in strong light. Although lidar has high accuracy, it is expensive and bulky, making it difficult to integrate into small mobile devices. While existing obstacle avoidance solutions that integrate vision introduce target detection technologies such as CNN, they mostly adopt a simple architecture of "single model + rule-driven". They lack global optimization design for multi-scale targets (such as pedestrians in the distance and obstacles nearby), dynamic environments (such as vehicle intersections and pedestrian movement), and sensor noise in complex scenes. Especially under extreme lighting conditions such as tunnels and nighttime, the model's insufficient generalization ability leads to a sharp drop in detection accuracy. Furthermore, the lack of a closed-loop optimization mechanism for perception-planning-control makes it difficult for the system to meet the requirements for long-term stable outdoor operation. Summary of the Invention

[0004] This invention provides a real-time visual-assisted obstacle avoidance method at the edge based on an improved YOLO algorithm, in order to overcome the shortcomings of the prior art.

[0005] This invention is achieved through the following technical solution: The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm includes the following steps: S1. Image Acquisition and Preprocessing: RGB images are acquired through a camera and preprocessed to adapt to the input of the lightweight YOLO model; S2, Object Detection: An improved YOLO algorithm is adopted, based on the MobileNetV3 backbone network, and a lightweight optimization module is integrated to output object category, confidence score and bounding box information; S3. Detection result output: Combine depth image data to extract target depth information and generate detection results containing depth information; S4. Path planning: Based on the detection results, generate a global path and a local obstacle avoidance strategy, and merge them to form a motion trajectory; S5. Motion Control: Generates speed and steering angle commands based on the motion trajectory to control the movement of the intelligent vehicle.

[0006] The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm described above includes the following steps in S1: S1-1: Select an industrial-grade RGB camera with high frame rate, low latency and wide field of view. Securely install it on the top of the smart vehicle or in the center above the windshield of the vehicle, ensuring that the optical axis of the lens is parallel to the ground and that the field of view can completely cover the 120° fan-shaped area in front of the smart vehicle and the core obstacle avoidance distance range of 3-10 meters. S1-2: The original high-resolution photo is cropped and resized to 640×640 pixels, and then rotated, its brightness adjusted, mirrored, and Gaussian noise added. The processed image is then incorporated into the dataset.

[0007] In the edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm described above, in step S1, each image is standardized to ensure statistical consistency of the input data. in, These are the original pixel values. and These represent the mean and standard deviation of the image, respectively. The standardized pixel values ​​are used to effectively mitigate the impact of changes in lighting conditions on feature distribution. Brightness and contrast adjustments are defined as follows: in, This is the brightness offset. This is the contrast adjustment coefficient. To simulate sensor noise, Gaussian noise is further superimposed on the image. in, This indicates that the mean is 0 and the variance is 0. Noise distribution; The enhancement process is modeled as a probability distribution. For each original image, its augmented sample set is represented as: in, For the enhancement operator of random sampling, during the online inference phase, each frame of image via enhancement operator The dynamic transformation is as follows: This allows the model to continuously adapt to different input distributions during operation.

[0008] As described above, in the edge real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm, the spatial-channel decomposition mapping operator is introduced in S2 to reduce the amount of convolution operations. We replace standard convolution with depthwise separable convolution, which consists of two parts: depthwise convolution and pointwise convolution. The difference with depthwise convolution is that each input channel corresponds to an independent convolution kernel, which is represented as: in, This represents the pixel value of the c-th channel in the output feature map; This represents the pixel value of the c-th channel in the output feature map; This represents the convolution kernel corresponding to the c-th channel; Pointwise convolution uses a 1x1 convolution and is typically used to linearly combine the outputs of each channel to achieve feature fusion. Its formula is: in, This represents the pixel values ​​of the output feature map; For the input feature of the c-th channel; This represents the convolution kernel, used to weight and fuse features from different channels; When the spatial dimensions of the input and output features are the same, a gated residual mechanism is introduced to adaptively adjust the ratio of input features to residual information, thereby improving feature fidelity and convergence stability under lightweight conditions. Introducing a bottleneck structure enhances feature extraction efficiency; The decoupled detection head consists of a localization branch and a classification branch, where: Localization branch prediction bounding box parameters Its loss function adopts the CIoU extended form with variance penalty: in, The square of the distance from the center point, The length of the diagonal of the smallest bounding rectangle. For aspect ratio consistency, The variance of the bounding box predicted by the model. , As a balancing factor, this term improves the stability and robustness of bounding box regression by explicitly penalizing high uncertainty predictions; The classification branch outputs the class probability matrix. The improved FocalLoss expression is used: in, To predict probabilities, For category weights, To focus parameters, This represents the co-occurrence correlation matrix between categories. To adjust the coefficients, a class correlation constraint term is introduced into the loss function, which can effectively suppress false activations between highly co-occurring classes in multi-class mixed scenarios and further improve classification accuracy. The nonmaximum suppression stage employs an exponential decay mechanism based on Soft-NMS: in, As the decay coefficient, Soft-NMS, as a differentiable operator, can be jointly optimized during the training phase, which helps improve the target ranking performance and reduce the computational load during the inference phase. The total loss function of the entire detection module is defined as: in, For the target existence loss term, For the set of model parameters, This represents the branch weight coefficient.

[0009] The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm described above includes the following steps in the distance threshold adaptive learning process in S3: S3-1: Target depth information extraction: Based on the target bounding box output by S2, the corresponding region of the depth image is divided into a central region and an edge region. The depth of the central region is calculated using a weighted average, and the anomaly of the edge region is suppressed using a median filter. S3-2: Multimodal Information Fusion: Introducing a cross-modal attention module, using RGB target features to guide the weight allocation of depth information, with high weight for clear regions and low weight for blurry regions.

[0010] The real-time visual-assisted obstacle avoidance method for edges based on the improved YOLO algorithm described above, specifically the S3-1 operation, is as follows: based on the target bounding box output in step S2... ,in, For the first The center coordinates and width / height of each bounding box are first determined by the depth image. According to the central area of ​​the boundary box and edge areas Divide the area into partitions and calculate the depth value of the corresponding region; For the central area The depth value was calculated using the weighted average method. : in, As a weight, it is proportional to the texture sharpness of the RGB image and can be expressed as: in, To adjust the texture weight coefficients, For RGB images at pixel positions The gradient at a given location represents the sharpness of the texture, especially for edge regions. Due to the high level of noise, median filtering is used to robustly eliminate the noise. The specific operation of S3-2 is as follows: by introducing feature alignment between the depth image and the RGB feature map, and by adjusting the feature contribution of the depth image through an attention mechanism, the depth features... Represented as: Among them, attention weight for: in, It is a feature mapping function that maps the pixel features of an RGB image to a high-dimensional space.

[0011] The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm described above includes the following steps in the path planning process in S4: S4-1: Use the A* algorithm to generate a global path based on the environment map and the target location; S4-2: Based on the dynamic window method, combined with real-time obstacle information and intelligent vehicle kinematic constraints, a local obstacle avoidance strategy is generated; S4-3: In complex scenarios, a predictive control model is adopted to minimize trajectory tracking error, control energy, and obstacle potential field penalty in the prediction time domain.

[0012] The specific operation of S4-1 in the edge real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm described above is as follows: The node synthesis cost of the A* algorithm is defined as: in, From the starting point to the node The cumulative cost, For heuristic functions, For curvature smoothness, As the obstacle potential field penalty term, by normalizing and weighting the costs of each term, the algorithm can adaptively adjust the search priority under different environmental constraints; Cumulative Cost Using the actual path length weighted form: in, For path number Node coordinates These are the weighting coefficients for the path step size. This is the path length; The heuristic function uses a smoothed Euclidean distance: in, The coordinates of the target point, To avoid regularization constants with zero gradients and to ensure the consistency of the heuristic function, its gradient constraint is as follows: In terms of path smoothness modeling, to suppress jagged paths and sharp turns, this invention introduces a curvature penalty term: in, This represents the change in heading angle of consecutive nodes. The path arc length between nodes is denoted by , which ensures the second-order continuity of the planned path by penalizing large turning angle changes. To enhance the algorithm's environmental adaptability, this invention introduces the obstacle potential field into the node cost function, assuming the obstacle set is... Each obstacle For nodes The potential energy generated is defined as: in, , If the scale is Gaussian decay, then the total potential field penalty term is: in, This represents the obstacle weighting coefficient. Given the number of obstacles, the gradient of the potential field is as follows: This feature is used during the search process to guide nodes away from obstacles and improve path safety margins; The comprehensive cost function is updated during the search process using a priority queue, and the update rule is as follows: in, The cost between adjacent nodes is considered, and node expansion adopts a minimum cost-first strategy. For the set of the node's neighborhood, that is: To further improve path smoothness and physical feasibility, this invention performs a search on the resulting path after the search is completed. Perform secondary optimization and define a smoothing energy function: in, This is a second-order difference term used to penalize high-frequency oscillations along the path. The potential field penalty weight is minimized. This enables second-order continuity and smooth transition of the path; Path planning algorithms use weight reorganization To control the balance between different objectives, the weight parameters must satisfy: Through dynamic adjustment It can achieve adaptive optimization between shortest path, smooth cornering, and obstacle safety; The specific operation of S4-2 is as follows: based on obtaining the global path, for each candidate speed pair Forward simulation prediction is performed, and the results are evaluated and selected using a cost function. The velocity sampling interval is constrained as follows: in, , These are the maximum accelerations of linear velocity and angular velocity, respectively. , To control the boundary conditions, K forward integrations are performed on each candidate velocity pair to obtain the predicted state sequence. Define a comprehensive cost function based on the predicted trajectory: in, In the above cost function, the first term encourages the trajectory to converge toward the target direction, the second term ensures that the maximum distance from the obstacle is maintained, the third term constrains and controls the smoothness of the output to prevent sudden changes in velocity, and the fourth term further suppresses the behavior of approaching the obstacle through the potential field function. The specific operation of S4-3 is as follows: The predictive control model optimizes the trajectory tracking error and control input over a finite prediction time domain to ensure that the vehicle achieves dynamic obstacle avoidance while satisfying constraints. Its predictive control model is a single-vehicle kinematics model. The optimization objective function is defined as follows: in, As a reference trajectory point, The matrix represents the obstacle potential field penalty term. These are the state error weight, control energy weight, and control rate of change weight, respectively, with ρ>0 being the potential field adjustment coefficient.

[0013] The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm described above includes the following steps in S5: S5-1: Select PID or MPC control strategy according to the scenario requirements; S5-2: Calculate speed and steering angle commands based on the motion trajectory; S5-3: Converts instructions into drive signals and sends them to the actuator to control the movement of the intelligent vehicle.

[0014] The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm, as described above, specifically operates as follows: In simple scenarios, improved PID control is employed, with the proportional coefficient dynamically adapted to the threat level. in The basic proportional coefficient is used to improve response sensitivity by adjusting parameters in real time. For complex scenarios, a hierarchical MPC control is adopted to construct the prediction time domain. With control time domain The optimization model has the following objective function: in For state vectors, For reference trajectory, Q and R are the control variables, and Q and R are the weighting matrices. The optimal control sequence is solved through rolling optimization. The specific operation of S5-2 is as follows: A real-time obstacle avoidance strategy is generated based on the detection results; speed and steering angle commands are calculated based on the motion trajectory; the speed command is combined with the control strategy output and the safe distance constraint, as shown in the formula: in As the reference speed, Distance to the obstacle As a safety threshold, The attenuation coefficient is used; the steering angle command in PID mode is: in For lateral deviation, the state equations are solved in MPC mode. The optimal sequence is output, and then the instructions are converted into drive signals, which are then transformed using the actuator dynamics model: motor speed. , The transmission ratio is... The radius is the wheel radius.

[0015] The advantages of this invention are: the lightweight target detection model based on the improved YOLO algorithm and its application on the edge computing device JestonNano, as well as the visual-assisted obstacle avoidance system implemented with ROS, effectively solve the real-time problem of edge deployment while ensuring detection accuracy, and provide a reliable technical solution for the autonomous navigation of intelligent vehicles. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the present invention; Figure 2 This is a schematic diagram of the depth-separable convolution of the present invention; Figure 3 This is a schematic diagram illustrating the deployment of the present invention on an edge computing device; Figure 4 This is a schematic diagram showing the resource usage results of the edge computing device in accordance with the present invention; Figure 5 This is a flowchart of the target detection process of the present invention; Figure 6 This is a flowchart of the detection result output of the present invention; Figure 7 This is a flowchart of the obstacle avoidance strategy of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm includes the following steps: S1. Image Acquisition and Preprocessing: RGB images are acquired through a camera and preprocessed to adapt to the input of the lightweight YOLO model; S2, Object Detection: An improved YOLO algorithm is adopted, based on the MobileNetV3 backbone network, and a lightweight optimization module is integrated to output object category, confidence score and bounding box information; S3. Detection result output: Combine depth image data to extract target depth information and generate detection results containing depth information; S4. Path planning: Based on the detection results, generate a global path and a local obstacle avoidance strategy, and merge them to form a motion trajectory; S5. Motion Control: Generates speed and steering angle commands based on the motion trajectory to control the movement of the intelligent vehicle.

[0020] Specifically, the vehicle image data acquisition and annotation process in S1 described in this embodiment includes the following steps: S1-1: Select an industrial-grade RGB camera with high frame rate, low latency and wide field of view. Securely install it on the top of the smart vehicle or in the center above the windshield of the vehicle, ensuring that the optical axis of the lens is parallel to the ground and that the field of view can completely cover the 120° fan-shaped area in front of the smart vehicle and the core obstacle avoidance distance range of 3-10 meters. S1-2: The original high-resolution photo is cropped and resized to 640×640 pixels, and then rotated, its brightness adjusted, mirrored, and Gaussian noise added. The processed image is then incorporated into the dataset.

[0021] Specifically, in S1 of this embodiment, in order to ensure the statistical consistency of the input data, each image is standardized: in, These are the original pixel values. and These represent the mean and standard deviation of the image, respectively. The standardized pixel values ​​are used to effectively mitigate the impact of changes in lighting conditions on feature distribution. In terms of data augmentation, this invention employs multiple operations to improve the model's adaptability to complex environments. Brightness and contrast adjustments are defined as follows: in, This is the brightness offset. This is the contrast adjustment coefficient. To simulate sensor noise, Gaussian noise is further superimposed on the image; in, This indicates that the mean is 0 and the variance is 0. Noise distribution; The enhancement process is modeled as a probability distribution. For each original image, its augmented sample set is represented as: in, This is an enhancement operator for random sampling. During the online inference phase, each frame of image... via enhancement operator The dynamic transformation is as follows: This allows the model to continuously adapt to different input distributions during operation.

[0022] like Figure 5 As shown, more specifically, in S2 of this embodiment, in order to improve the real-time detection performance on the embedded platform, the present invention adopts an improved YOLO series convolutional neural network structure, replaces the traditional convolutional module with a lightweight operator, and optimizes the loss function by decoupling the detection head and introducing uncertainty modeling, thus forming a detection framework that balances accuracy and computational efficiency.

[0023] like Figure 1As shown, the visual perception module of this invention is located at the front end of the system flow, entering the target detection stage after image acquisition and preprocessing. This stage uses an improved YOLO algorithm structure to decouple the branches of the detection head, outputs and encapsulates the target's localization information and classification probability to the / detection / objects topic, providing data for subsequent path planning and motion control modules.

[0024] The algorithm uses MobileNetV3 as the backbone feature extraction network and reduces the amount of convolution operations by introducing a spatial-channel decomposition mapping operator.

[0025] Let the input feature tensor be The output tensor is Traditional two-dimensional local mapping, i.e., standard convolution, uses a kernel on each output channel. Fully coupled with the input channel, its computational complexity is: in, The spatial dimensions of the output feature map are defined. To reduce computational complexity, this invention employs a spatial-channel decomposition mapping, breaking down standard convolution into two independent stages.

[0026] Introducing a spatial-channel decomposition mapping operator reduces the computational cost of convolution. This decomposition mapping first performs spatial mapping independently on each channel of the input feature, and then achieves cross-channel feature fusion through point-by-point mapping. The computational complexity is significantly reduced after decomposition. Employing depthwise separable convolutions (such as...) Figure 2 As shown, instead of standard convolution, depthwise separable convolution is divided into two parts: depthwise convolution and pointwise convolution. Depthwise convolution uses a different convolution kernel for each input channel, while pointwise convolution uses a 1x1 convolution for feature fusion, effectively reducing computational complexity.

[0027] The difference with depthwise convolution is that each input channel corresponds to an independent convolution kernel, which is represented as: in, This represents the pixel value of the c-th channel in the output feature map; This represents the pixel value of the c-th channel in the output feature map; This represents the convolution kernel corresponding to the c-th channel. For the first The spatial weight matrix of the channel.

[0028] Pointwise convolution uses a 1x1 convolution and is typically used to linearly combine the outputs of each channel to achieve feature fusion. Its formula is: in, This represents the pixel values ​​of the output feature map; For the input feature of the c-th channel; This represents the convolution kernel, used to weight and fuse features from different channels; the total complexity after decomposition is approximately: This invention introduces a layer before and after the decomposition mapping. Convolutional mapping forms the bottleneck structure: in, and These are channel dimensionality upgrade and dimensionality reduction operations, respectively. , This enhances the flexibility and compression capability of feature extraction.

[0029] When the spatial dimensions of the input and output features are consistent, this invention further introduces a gated residual mechanism to suppress gradient vanishing. The output of this structure is defined as: in, For the Sigmoid function, Represents the Hadamard product. This is a learnable gated weight matrix. Through this gated residual path, the network can adaptively adjust the input features. With residual information The proportion of this increases, thereby improving the feature fidelity and convergence stability of the model under lightweight conditions.

[0030] Based on this, in order to balance multi-scale detection performance, the feature layer The pyramid structure is formed through cross-scale fusion, and its fusion relationship can be represented as follows: in, For spatial adaptive fusion coefficients, For channel normalization mapping, This represents the upsampling operator.

[0031] When the spatial dimensions of the input and output features are the same, a gated residual mechanism is introduced to adaptively adjust the ratio of input features to residual information, thereby improving feature fidelity and convergence stability under lightweight conditions. Introducing a bottleneck structure enhances feature extraction efficiency; The decoupled detection head consists of a localization branch and a classification branch, where: Localization branch prediction bounding box parameters Its loss function adopts the CIoU extended form with variance penalty: in, The square of the distance from the center point, The length of the diagonal of the smallest bounding rectangle. For aspect ratio consistency, The variance of the bounding box predicted by the model. , This is the balancing coefficient. This term improves the stability and robustness of bounding box regression by explicitly penalizing high uncertainty predictions.

[0032] The classification branch outputs the class probability matrix. The improved FocalLoss expression is used: in, To predict probabilities, For category weights, To focus parameters, This represents the co-occurrence correlation matrix between categories. This is the adjustment coefficient. By introducing a class correlation constraint term into the loss function, false activations between highly co-occurring classes can be effectively suppressed in multi-class mixed scenarios, further improving classification accuracy.

[0033] To improve the stability and differentiability of the end-to-end inference stage, the nonmaximum suppression stage adopts an exponential decay form of Soft-NMS: in, is the decay coefficient. Soft-NMS, as a differentiable operator, can be jointly optimized during the training phase, helping to improve target ranking performance and reduce the computational load during the inference phase.

[0034] The total loss function of the entire detection module is defined as: in, For the target existence loss term, For the set of model parameters, These are the branch weight coefficients; the output of the detection module is encapsulated through the ROS communication mechanism and published to the ` / detection / objects` topic, such as... Figure 1 As shown, it provides real-time sensing input for the path planning and motion control module.

[0035] The computational overhead of this embodiment is reduced by approximately eight times compared to standard convolution, and its FLOPs compression ratio is: For common convolutional kernel sizes The theoretical FLOPs compression ratio is: The computational overhead corresponding to this compression ratio can be significantly reduced in practical applications, as shown in Table 1. Table 1 shows the execution time and resource consumption of standard convolution and depthwise separable convolution on edge devices. The experimental results show that, compared with traditional standard convolution, the computational overhead of this embodiment in the object detection task is reduced by about 8 times.

[0036] Table 1 Comparison of computational efficiency of different convolutional structures on edge devices This embodiment demonstrates the target detection effect as follows: Figure 3 As shown, detected targets are marked with rectangular boxes, and their categories and confidence levels are statistically analyzed. The resource utilization results of the edge computing device implemented in this invention are as follows: Figure 4 As shown, using this invention can reduce the computational resource consumption of target detection.

[0037] The detection and path planning results of this embodiment are shown in Table 2, and the impact of the loss function on the positioning accuracy is shown in Table 3. The dataset used in the experiment comes from a dataset collected and captured by our company. As can be seen from Tables 2 and 3, the lightweight target detection model based on the improved YOLO algorithm proposed in this invention, its application on the edge computing device JestonNano, and the visual-assisted obstacle avoidance system implemented with ROS, effectively solve the real-time problem of edge deployment while ensuring detection accuracy. This provides a reliable technical solution for the autonomous navigation of intelligent vehicles, further demonstrating the superiority and universality of the algorithm proposed in this invention.

[0038] Table 2 Comparison of test accuracy errors in different scenarios Table 3. Impact of different loss functions on positioning accuracy like Figure 6 As shown, more specifically, the distance threshold adaptive learning process in S3 described in this embodiment includes the following steps: S3-1: Target Depth Information Extraction: Based on the target bounding box output by S2, the corresponding region of the depth image is divided into a central region and an edge region. The depth of the central region is calculated using a weighted average (weights are related to RGB texture clarity), and the depth of the edge region is suppressed by median filtering. Combined with target category differential extraction, the bottom depth is taken for rigid targets, the center depth is taken for dynamic targets, and the bounding box of small targets is expanded by 10%-20%. If the standard deviation of the depth value exceeds the average depth by 15%, it is judged as an unreliable region and completion is triggered.

[0039] S3-2: Multimodal information fusion: A cross-modal attention module is introduced to guide the weight allocation of depth information with RGB target features, with high weight for clear areas and low weight for blurry areas; the depth of the current frame is corrected by a 3 to 5 frame sliding window temporal filter combined with the depth change trend of the previous frame; a six-dimensional detection vector with fusion confidence is generated by fusion, and the confidence is calculated by weighting the RGB detection confidence and depth stability.

[0040] More specifically, the specific operation of S3-1 in this embodiment is as follows: based on the target bounding box output in step S2... ,in, For the first The center coordinates and width / height of each bounding box are first determined by the depth image. According to the central area of ​​the boundary box and edge areas Divide the area into partitions and calculate the depth value of the corresponding region; For the central area The depth value was calculated using the weighted average method. : in, As a weight, it is proportional to the texture sharpness of the RGB image and can be expressed as: in, To adjust the texture weight coefficients, For RGB images at pixel positions The gradient at a given location represents the sharpness of the texture. For edge regions... Due to the high level of noise, median filtering is used to robustly eliminate the noise. The specific operation of S3-2 is as follows: To enhance the robustness of multimodal data fusion, this invention introduces a cross-modal attention mechanism. This is achieved by aligning features between the depth image and the RGB feature map, and by adjusting the feature contribution of the depth image through the attention mechanism. (Depth features) Represented as: Among them, attention weight for: in, It is a feature mapping function that maps the pixel features of an RGB image to a high-dimensional space. This attention mechanism can dynamically adjust the contribution of depth features in different regions, enhance the weighting of the visual perception module on clear contour regions, and reduce the influence of blurred regions.

[0041] To further ensure timing consistency, this invention introduces sliding window filtering and motion trend correction. Specifically, a sliding window filtering method is set up, which includes... A sliding window of frames, representing a depth sequence. The depth sequence is smoothed. The current time step after smoothing. depth value It can be calculated using a weighted average: in, As a smoothing factor, For the target speed, This represents the inter-frame time interval. To correct the depth information of the current frame, adjustments are also made based on the target's motion trend. This operation eliminates temporal fluctuations in the depth data, enhancing the stability of depth information across consecutive frames.

[0042] Finally, the depth image processed as described above is combined with the target detection information to form a six-dimensional detection vector. Defined as: in, For category ID, The target confidence level output by YOLO. For bounding box coordinates, For depth value, To integrate confidence levels, For movement trends. Fusion confidence. The calculation formula is: in, The average depth value. The standard deviation of depth, These are the weighting coefficients. This weighting function combines the stability of detection confidence and depth values, maintaining high detection accuracy in variable environments.

[0043] As a distributed communication and control platform, the ROS framework enables efficient real-time interaction between modules through a publish-subscribe message mechanism. In integrating visual inspection with ROS nodes, the visual inspection node is configured to acquire camera image data (sensor_msgs / Image) and depth data (sensor_msgs / PointCloud2) in real time via the ROS subscription interface (subscribe()). A lightweight YOLO model is then executed to perform object detection, generating detection results containing object category, confidence score, bounding box coordinates, and depth information. These results are then encapsulated into a custom ROS message and sent to a specified topic via the publish() interface for downstream nodes to subscribe to and process.

[0044] Furthermore, the path planning process in S4 described in this embodiment includes the following steps: S4-1: Use the A* algorithm to generate a global path based on the environment map and the target location.

[0045] S4-2: Based on the dynamic window method, combined with real-time obstacle information and intelligent vehicle kinematic constraints, a local obstacle avoidance strategy is generated; S4-3: In complex scenarios, a predictive control model (MPC) is adopted to minimize trajectory tracking error, control energy, and obstacle potential field penalty in the prediction time domain.

[0046] Furthermore, the specific operation of S4-1 described in this embodiment is as follows: The node synthesis cost of the A* algorithm is defined as: in, From the starting point to the node The cumulative cost, For heuristic functions, For curvature smoothness, This represents the penalty term for the obstacle potential field. By normalizing and weighting the costs of each term, the algorithm can adaptively adjust the search priority under different environmental constraints.

[0047] Cumulative Cost Using the actual path length weighted form: in, For path number Node coordinates These are the weighting coefficients for the path step size. This represents the path length.

[0048] The heuristic function uses a smoothed Euclidean distance: in, The coordinates of the target point, To avoid regularization constants with zero gradients, and to ensure the consistency of the heuristic function, its gradient constraint is as follows: In terms of path smoothness modeling, to suppress jagged paths and sharp turns, this invention introduces a curvature penalty term: in, This represents the change in heading angle of consecutive nodes. Let be the arc length of the path between nodes. This term ensures the second-order continuity of the planned path by penalizing large turns.

[0049] To enhance the algorithm's environmental adaptability, this invention incorporates the obstacle potential field into the node cost function. Let the obstacle set be... Each obstacle For nodes The potential energy generated is defined as: in, , Let the scale be Gaussian decay. Then the total potential field penalty term is: in, This represents the obstacle weighting coefficient. Let be the number of obstacles. The gradient of the potential field is in the form of: This feature is used during the search process to guide nodes away from obstacles and improve path safety margins.

[0050] The comprehensive cost function is updated during the search process using a priority queue, and the update rule is as follows: in, This represents the cost between adjacent nodes. Node expansion employs a minimum cost-first strategy. For the set of the node's neighborhood, that is: To further improve path smoothness and physical feasibility, this invention performs a search on the resulting path after the search is completed. Perform secondary optimization. Define the smoothing energy function: in, This is a second-order difference term used to penalize high-frequency oscillations along the path. The potential field penalty weight is minimized. This enables second-order continuity and smooth transition of the path.

[0051] Ultimately, the path planning algorithm achieves its goal through weight reorganization. Controlling the balance between different objectives. The weighting parameters satisfy: Through dynamic adjustment It can adaptively optimize the path shortest distance, smooth cornering, and obstacle safety. Compared with the traditional A* algorithm, the multi-objective heuristic path search algorithm proposed in this invention exhibits better smoothness, stability, and real-time performance in narrow passages and complex environments; like Figure 7 As shown, the specific operation of S4-2 is as follows: Based on the obtained global path, this invention further combines the Dynamic Window Method (DWA) to achieve dynamic optimization of the local path, thereby enhancing the response capability to sudden obstacles. The core idea of ​​DWA is to construct a dynamically feasible velocity set by combining kinematic and acceleration constraints under the current velocity state, and then perform dynamic optimization on each candidate velocity pair. Forward simulation prediction is performed, and the results are evaluated and selected using a cost function. The velocity sampling interval is constrained as follows: in, , These are the maximum accelerations of linear velocity and angular velocity, respectively. , To control the boundary conditions, K forward integrations are performed on each candidate velocity pair to obtain the predicted state sequence. Based on the predicted trajectory, a comprehensive cost function is defined as follows: in, In the aforementioned cost function, the first term encourages the trajectory to converge toward the target direction, the second term ensures maximum distance from obstacles, the third term constrains the smoothness of the output to prevent sudden velocity changes, and the fourth term further suppresses behavior that approaches obstacles through a potential field function. This is achieved through a comprehensive weighting process. With adjustments, DWA can dynamically balance safety, smoothness, and goal orientation, and ultimately select the speed pair with the minimum cost function value as the local obstacle avoidance control command. The evaluation operation of the local obstacle avoidance strategy is as follows: (1) Obstacle threat assessment: Based on the output of the visual detection node, extract the x-coordinate of the center point of the obstacle bounding box. With the center line of the image Calculate the horizontal offset ; Based on the distance between the y-coordinate of the lower edge of the obstacle bounding box and the bottom edge of the image. The actual distance to the obstacle is estimated by combining the pre-calibration coefficient k. ; Calculate threat level Defined as ,in , These are adjustable weighting coefficients used to balance the contributions of distance and offset; (2) Tiered obstacle avoidance strategy: Set a safety threshold and danger threshold According to the threat level Implement a tiered control strategy: when < When the condition is deemed safe, the current motion parameters, including linear velocity, are maintained. and angular velocity constant; when ≤ < At that time, the deceleration strategy is activated, and the linear velocity is dynamically adjusted to... ,in The initial linear velocity is constant, and the angular velocity ω remains constant. when ≥ When this happens, a steering obstacle avoidance strategy is executed: i. If >0 (obstacle located on the left side of the image), control the intelligent vehicle to turn right, and set the angular velocity ω to a positive value; ii. If <0 (obstacle located on the right side of the image), control the intelligent vehicle to turn left, setting the angular velocity ω to a negative value; iii. Simultaneously reduce the linear velocity to a safe value, with the deceleration ratio based on the threat level. Dynamic adjustment; The specific operation of S4-3 is as follows: In complex scenarios with dense obstacles or frequently changing dynamic targets, relying solely on DWA is insufficient to achieve globally optimal control. Therefore, this invention introduces Model Predictive Control (MPC) as a high-level optimization module. The predictive control model ensures that the vehicle achieves dynamic obstacle avoidance while satisfying constraints by continuously optimizing the trajectory tracking error and control input within a finite prediction time domain. Its predictive control model is a single-vehicle kinematics model: The optimization objective function is defined as follows: in, As a reference trajectory point, The matrix represents the obstacle potential field penalty term. These represent the state error weight, control energy weight, and control rate of change weight, respectively, with ρ>0 being the potential field adjustment coefficient. By solving the above constrained optimization problem through rolling optimization, the optimal control sequence can be generated in real time, ensuring the vehicle's trajectory stability and obstacle avoidance safety in complex dynamic environments. Unlike traditional single-step control, MPC possesses both prediction and feedback mechanisms, enabling it to proactively avoid potential hazards and achieve smooth control under constrained boundary conditions.

[0052] In summary, this invention ensures the feasibility and optimality of the macroscopic path through an improved global A* algorithm, provides real-time dynamic obstacle avoidance capabilities through a local DWA algorithm, and further enhances the robustness and safety of the system in complex environments through MPC optimization. The combination of these three elements forms a complete hierarchical system from global planning and local optimization to predictive control, providing more accurate and efficient technical support for the autonomous navigation and obstacle avoidance of intelligent vehicles.

[0053] Furthermore, the motion control process in S5 described in this embodiment includes the following steps: S5-1: Select PID or MPC control strategy according to the scenario requirements.

[0054] S5-2: Calculate speed and steering angle commands based on the motion trajectory.

[0055] S5-3: Converts instructions into drive signals and sends them to the actuator to control the movement of the intelligent vehicle.

[0056] Furthermore, the specific operation of S5-1 described in this embodiment is as follows: Simple scenario (number of obstacles ≤ 2 and static, threat level) < An improved PID control is adopted, with its proportional coefficient dynamically adapted to the threat level. in The basic proportional coefficient is used to improve response sensitivity by adjusting parameters in real time. Complex scenarios (number of obstacles ≥ 3 or dynamic, threat level) ≥ A hierarchical MPC control system is used to construct the prediction time domain. With control time domain The optimization model has the following objective function: in For state vectors, For reference trajectory, Q and R are the control variables, and Q and R are the weighting matrices. The optimal control sequence is solved through rolling optimization. The specific operation of S5-2 is as follows: The motion control node obtains the spatial position, size, and distance information of obstacles by subscribing to the / detection / objects topic in ROS. Based on the detection results, it generates a real-time obstacle avoidance strategy, calculates speed and steering angle commands according to the motion trajectory, and combines the speed command with the control strategy output and the safe distance constraint, as shown in the formula. in As the reference speed, Distance to the obstacle As a safety threshold, The attenuation coefficient is used; the steering angle command in PID mode is: in For lateral deviation, the state equations are solved in MPC mode. The optimal sequence is output, and then the instructions are converted into drive signals, which are then transformed using the actuator dynamics model: motor speed. , The transmission ratio is... The radius is the wheel radius.

[0057] In summary, this integrated approach achieves closed-loop optimization from perception to control: front-end visual perception and path planning generate a reference trajectory; the MPC controller outputs the optimal control quantity considering dynamic constraints and obstacle potential fields; the low-level drive module maps this to the actual execution signals of the motors and servos; and finally, the ROS publish-subscribe mechanism completes efficient coordination of the entire system. This adaptive closed-loop system ensures that edge devices maintain millisecond-level response and high robustness in complex dynamic environments, providing a reliable guarantee for autonomous obstacle avoidance in intelligent vehicles.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time visual-assisted obstacle avoidance method at the edge based on an improved YOLO algorithm, characterized in that: Includes the following steps: S1. Image Acquisition and Preprocessing: RGB images are acquired through a camera and preprocessed to adapt to the input of the lightweight YOLO model; S2, Object Detection: An improved YOLO algorithm is adopted, based on the MobileNetV3 backbone network, and a lightweight optimization module is integrated to output object category, confidence score and bounding box information; S3. Detection result output: Combine depth image data to extract target depth information and generate detection results containing depth information; S4. Path planning: Based on the detection results, generate a global path and a local obstacle avoidance strategy, and merge them to form a motion trajectory; S5. Motion Control: Generates speed and steering angle commands based on the motion trajectory to control the movement of the intelligent vehicle.

2. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: The vehicle-mounted image data acquisition and annotation process in S1 includes the following steps: S1-1: Select an industrial-grade RGB camera with high frame rate, low latency and wide field of view. Securely install it on the top of the smart vehicle or in the center above the windshield of the vehicle, ensuring that the optical axis of the lens is parallel to the ground and that the field of view can completely cover the 120° fan-shaped area in front of the smart vehicle and the core obstacle avoidance distance range of 3-10 meters. S1-2: The original high-resolution photo is cropped and resized to 640×640 pixels, and then rotated, its brightness adjusted, mirrored, and Gaussian noise added. The processed image is then incorporated into the dataset.

3. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: In step S1, to ensure statistical consistency of the input data, each image is standardized: in, These are the original pixel values. and These represent the mean and standard deviation of the image, respectively. The standardized pixel values ​​are used to effectively mitigate the impact of changes in lighting conditions on feature distribution. Brightness and contrast adjustments are defined as follows: in, This is the brightness offset. This is the contrast adjustment coefficient. To simulate sensor noise, Gaussian noise is further superimposed on the image. in, This indicates that the mean is 0 and the variance is 0. Noise distribution; The enhancement process is modeled as a probability distribution. For each original image, its augmented sample set is represented as: in, For the enhancement operator of random sampling, during the online inference phase, each frame of image via enhancement operator The dynamic transformation is as follows: This allows the model to continuously adapt to different input distributions during operation.

4. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: The S2 section introduces a spatial-channel decomposition mapping operator to reduce the amount of convolution computation. We replace standard convolution with depthwise separable convolution, which consists of two parts: depthwise convolution and pointwise convolution. The difference with depthwise convolution is that each input channel corresponds to an independent convolution kernel, which is represented as: in, This represents the pixel value of the c-th channel in the output feature map; This represents the pixel value of the c-th channel in the output feature map; This represents the convolution kernel corresponding to the c-th channel; Pointwise convolution uses a 1x1 convolution and is typically used to linearly combine the outputs of each channel to achieve feature fusion. Its formula is: in, This represents the pixel values ​​of the output feature map; For the input feature of the c-th channel; This represents the convolution kernel, used to weight and fuse features from different channels; When the spatial dimensions of the input and output features are the same, a gated residual mechanism is introduced to adaptively adjust the ratio of input features to residual information, thereby improving feature fidelity and convergence stability under lightweight conditions. Introduce a bottleneck structure to enhance feature extraction efficiency; The decoupled detection head consists of a localization branch and a classification branch, where: Localization branch prediction bounding box parameters Its loss function adopts the CIoU extended form with variance penalty: in, The square of the distance from the center point, The length of the diagonal of the smallest bounding rectangle. For aspect ratio consistency, The variance of the bounding box predicted by the model. , As a balancing factor, this term improves the stability and robustness of bounding box regression by explicitly penalizing high uncertainty predictions; The classification branch outputs the class probability matrix. The improved FocalLoss expression is used: in, To predict probabilities, For category weights, To focus parameters, This represents the co-occurrence correlation matrix between categories. To adjust the coefficients, a class correlation constraint term is introduced into the loss function, which can effectively suppress false activations between highly co-occurring classes in multi-class mixed scenarios and further improve classification accuracy. The nonmaximum suppression stage employs an exponential decay mechanism based on Soft-NMS: in, As the decay coefficient, Soft-NMS, as a differentiable operator, can be jointly optimized during the training phase, which helps improve the target ranking performance and reduce the computational load during the inference phase. The total loss function of the entire detection module is defined as: in, For the target existence loss term, For the set of model parameters, This represents the branch weight coefficient.

5. The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: The distance threshold adaptive learning process in S3 includes the following steps: S3-1: Target depth information extraction: Based on the target bounding box output by S2, the corresponding region of the depth image is divided into a central region and an edge region. The depth of the central region is calculated using a weighted average, and the anomaly of the edge region is suppressed using a median filter. S3-2: Multimodal Information Fusion: Introducing a cross-modal attention module, using RGB target features to guide the weight allocation of depth information, with high weight for clear regions and low weight for blurry regions.

6. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: The specific operation of S3-1 is as follows: based on the target bounding box output in step S2... ,in, For the first The center coordinates and width / height of each bounding box are first determined by the depth image. According to the central area of ​​the boundary box and edge areas Divide the area into partitions and calculate the depth value of the corresponding region; For the central area The depth value was calculated using the weighted average method. : in, As a weight, it is proportional to the texture sharpness of the RGB image and can be expressed as: in, To adjust the coefficients of texture weights, For RGB images at pixel positions The gradient at a given location represents the sharpness of the texture, especially for edge regions. Due to the high level of noise, median filtering is used to robustly eliminate the noise. The specific operation of S3-2 is as follows: by introducing feature alignment between the depth image and the RGB feature map, and by adjusting the feature contribution of the depth image through an attention mechanism, the depth features... Represented as: Among them, attention weight for: in, It is a feature mapping function that maps the pixel features of an RGB image to a high-dimensional space.

7. The edge-based real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: The path planning process in S4 includes the following steps: S4-1: Use the A* algorithm to generate a global path based on the environment map and the target location; S4-2: Based on the dynamic window method, combined with real-time obstacle information and intelligent vehicle kinematic constraints, a local obstacle avoidance strategy is generated; S4-3: In complex scenarios, a predictive control model is adopted to minimize trajectory tracking error, control energy, and obstacle potential field penalty in the prediction time domain.

8. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 7, characterized in that: The specific operation of S4-1 is as follows: The node synthesis cost of the A* algorithm is defined as: in, From the starting point to the node The cumulative cost, For heuristic functions, For curvature smoothness, As the obstacle potential field penalty term, by normalizing and weighting the costs of each term, the algorithm can adaptively adjust the search priority under different environmental constraints; Cumulative Cost Using the actual path length weighted form: in, For path number Node coordinates These are the path step size weighting coefficients. This represents the path length. The heuristic function uses a smoothed Euclidean distance: in, The coordinates of the target point, To avoid regularization constants with zero gradients and to ensure the consistency of the heuristic function, its gradient constraint is as follows: In terms of path smoothness modeling, to suppress jagged paths and sharp turns, this invention introduces a curvature penalty term: in, This represents the change in heading angle of consecutive nodes. The path arc length between nodes is denoted by , which ensures the second-order continuity of the planned path by penalizing large turning angle changes. To enhance the algorithm's environmental adaptability, this invention introduces the obstacle potential field into the node cost function, assuming the obstacle set is... Each obstacle For nodes The potential energy generated is defined as: in, , If the scale is Gaussian decay, then the total potential field penalty term is: in, This represents the obstacle weighting coefficient. Given the number of obstacles, the gradient of the potential field is as follows: This feature is used during the search process to guide nodes away from obstacles and improve path safety margins; The comprehensive cost function is updated during the search process using a priority queue, and the update rule is as follows: in, The cost between adjacent nodes is considered, and node expansion adopts a minimum cost-first strategy. For the set of the node's neighborhood, that is: To further improve path smoothness and physical feasibility, this invention performs a search on the resulting path after the search is completed. Perform secondary optimization and define a smoothing energy function: in, This is a second-order difference term used to penalize high-frequency oscillations along the path. The potential field penalty weight is minimized. This enables second-order continuity and smooth transition of the path; Path planning algorithms use weight reorganization To control the balance between different objectives, the weight parameters must satisfy: Through dynamic adjustment It can achieve adaptive optimization between shortest path, smooth cornering, and obstacle safety; The specific operation of S4-2 is as follows: based on obtaining the global path, for each candidate speed pair Forward simulation prediction is performed, and the results are evaluated and selected using a cost function. The velocity sampling interval is constrained as follows: in, , These are the maximum accelerations of linear velocity and angular velocity, respectively. , To control the boundary conditions, K forward integrations are performed on each candidate velocity pair to obtain the predicted state sequence. Based on the predicted trajectory, a comprehensive cost function is defined: in, In the above cost function, the first term encourages the trajectory to converge toward the target direction, the second term ensures that the maximum distance from the obstacle is maintained, the third term constrains and controls the smoothness of the output to prevent sudden changes in velocity, and the fourth term further suppresses the behavior of approaching the obstacle through the potential field function. The specific operation of S4-3 is as follows: The predictive control model optimizes the trajectory tracking error and control input over a finite prediction time domain to ensure that the vehicle achieves dynamic obstacle avoidance while satisfying constraints. Its predictive control model is a single-vehicle kinematics model. The objective function is defined as follows: in, As a reference trajectory point, The matrix represents the obstacle potential field penalty term. These are the state error weight, control energy weight, and control rate of change weight, respectively, with ρ>0 being the potential field adjustment coefficient.

9. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 1, characterized in that: The motion control process in S5 includes the following steps: S5-1: Select PID or MPC control strategy according to the scenario requirements; S5-2: Calculate speed and steering angle commands based on the motion trajectory; S5-3: Converts instructions into drive signals and sends them to the actuator to control the movement of the intelligent vehicle.

10. The edge-end real-time visual-assisted obstacle avoidance method based on the improved YOLO algorithm according to claim 9, characterized in that: The specific operation of S5-1 is as follows: In simple scenarios, an improved PID control is used, with its proportional coefficient dynamically adapted to the threat level. in The basic proportional coefficient is used to improve response sensitivity by adjusting parameters in real time. For complex scenarios, a hierarchical MPC control is adopted to construct the prediction time domain. With control time domain The optimization model has the following objective function: in For state vectors, For reference trajectory, Q and R are the control variables, and Q and R are the weighting matrices. The optimal control sequence is solved through rolling optimization. The specific operation of S5-2 is as follows: A real-time obstacle avoidance strategy is generated based on the detection results; speed and steering angle commands are calculated based on the motion trajectory; the speed command is combined with the control strategy output and the safe distance constraint, as shown in the formula: in As the reference speed, Distance to the obstacle As a safety threshold, The attenuation coefficient is used; the steering angle command in PID mode is: in For lateral deviation, the state equations are solved in MPC mode. The optimal sequence is output, and then the instructions are converted into drive signals, which are then transformed using the actuator dynamics model: motor speed. , The transmission ratio is... The radius is the wheel radius.

Images