Vehicle automatic driving method

Abstract

The invention particularly relates to a vehicle automatic driving method, which comprises the following steps of S1, calculating an optimal driving route according to a preset starting point, a preset terminal point and environmental map information, S2, adopting a capability YOLOv7 target detection model integrated with an RRT-based ROS navigation stack to help a system to quickly and accurately identify the position and the size of an obstacle, S3, identifying the obstacle, and S4, carrying out vehicle automatic driving. S4, the automatic driving monocycle rapidly avoids collision through a decision-making mechanism, and after the obstacle is avoided, the automatic driving monocycle uses FRENETIX to enhance local path planning, and a local path is re-planned.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent vehicles, and in particular relates to a vehicle automatic driving method. Background Art

[0002] The unicycle is widely loved by people all over the world because of its unique single wheel structure and light and flexible body. The unicycle is small, light and easy to carry, so people can easily bring it into the subway, bus and other means of transportation to achieve freedom of travel;

[0003] Modern wheelbarrows are basically electric, and their technical principles mainly include balance control, mechanical design, motor drive and other aspects. So far, the flexibility and maneuverability of wheelbarrows have made wheelbarrows used in public security patrols in squares and other places, as well as logistics and take-out delivery in relatively congested small areas; however, traditional wheelbarrows need to be driven manually, and manual driving of wheelbarrows during patrol and security tasks may cause security personnel to be distracted and not focus enough on inspection and patrol work. Security personnel driving traditional wheelbarrows for patrol can only rely on their own eyes to identify dangerous persons and dangerous items, and it is difficult to find highly concealed dangerous items. The self-driving unicycle invented in this article can identify highly concealed dangerous items through on-board monitoring; and in terms of logistics and take-out delivery, with the increase in labor costs, delivery work can be handed over to self-driving unicycles. In relatively congested small areas, self-driving unicycles can give full play to their small size and flexibility, and can easily pass through complex environments to achieve high efficiency in delivery. Summary of the invention

[0004] The purpose of the present invention is to provide a vehicle automatic driving method for solving the problems existing in the background technology.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for automatic driving of a vehicle comprises the following steps:

[0007] S1: Calculate an optimal driving route based on the preset starting point and end point, as well as environmental map information;

[0008] S2: The YOLOv7 target detection model, which integrates the RRT-based ROS navigation stack, helps the system quickly and accurately identify the location and size of obstacles;

[0009] S3: After identifying an obstacle, the self-driving unicycle quickly avoids collision through a decision-making mechanism;

[0010] S4: After avoiding obstacles, the autonomous unicycle uses FRENETIX to enhance local path planning and replan the local path.

[0011] Preferably, in path planning, map information and real-time traffic monitoring are used to obtain detailed information on dynamic road conditions, and shadow-covered areas are identified and extracted from high-definition satellite photos by applying a segmentation algorithm. These areas are then integrated into a multi-layered road map, allowing the self-driving unicycle to customize its route based on a balance between distance and shadow exposure.

[0012] Preferably, the base model SegmentAnything is used to segment the original satellite image, identify and depict each object in the image, evaluate the color, hue and brightness of the segmented image elements to identify the shadow parts, and by analyzing the chromaticity, those less bright areas are judged as shadow areas, and the shadow areas are aligned with the road data in OpenStreetMap to determine the percentage of each road covered by the shadow.

[0013] Preferably, the A* algorithm is used, which significantly improves the search efficiency by introducing a heuristic function to evaluate the priority of the node, and can find a nearly optimal planned path in a short time. At the same time, it calculates the ratio of the length covered by the shadow on the path to the length of the entire path, and constructs a multi-layer road map based on the shadow ratio data, including an upper shadow ratio map and a lower geographic map. The shadow route planning is integrated with the planned path to form a convenient path with maximized shadows, which is finally imported into the self-driving unicycle system.

[0014] Preferably, the self-driving unicycle has a visual algorithm that can accurately and quickly identify various obstacles during driving. It integrates the ROS navigation stack and the pre-trained YOLOv7 object detection model by combining the rapid exploration random tree RRT to enhance the working ability of the NAV-YOLO system development, and then improves a hybrid framework based on the NAV-YOLO system to integrate the capabilities of the RRT-based ROS navigation stack and the YOLOv7 target detection model.

[0015] Preferably, a custom dataset is generated using MRO hangar obstacles in real-world and simulated environments to train the YOLOv7 model, combined with public datasets such as LLVIP to enhance the model's obstacle recognition capabilities in low-light conditions.

[0016] Preferably, the YOLO model architecture is fine-tuned and a convolutional block attention mechanism module CBAM is integrated, which prioritizes important object details related to real-time detection by focusing on relevant features and suppressing background noise to improve the focus of attention on relevant features in real-time detection.

[0017] Preferably, when training the model, use the default hyperparameter settings, determine the batch size, and use an appropriate resolution for iterative training. The training phase is performed in a non-real-time environment to ensure that the model design can operate effectively under resource constraints. The reasoning speed and accuracy of the model are verified in the ROS environment to ensure that the model can run in real time on the robot's hardware. The trained YOLOv7 model and weight files are saved and transferred to a dedicated ROS package, and a custom node is created to load the model and perform object detection. The detection results of the YOLOv7 model are fused with LiDAR data, and the model performance is tested in an actual or simulated environment. Feedback is collected and necessary adjustments are made. Finally, the trained YOLOv7 model and weight files are saved and transferred to a dedicated ROS package, and a custom node is created to load the model and weights.

[0018] Preferably, the planning of rapid obstacle avoidance first determines the direction and speed of movement of the self-driving unicycle, establishes a control layer to handle local obstacle avoidance and post-collision control, and implements local obstacle avoidance through a controller based on velocity obstacle VelocityObstacle, VO; in collision control, a compliance and contact force controller is used, which uses known sensing surfaces on the shell of the self-driving unicycle to handle collisions. The controller ensures that the self-driving unicycle responds to unexpected contact and moves forward with a sliding maneuver under potential obstacle avoidance guidance away from the contact surface without colliding with other obstacles.

[0019] Preferably, FRENETIX is used to enhance local path planning, and the local path after obstacle avoidance is planned under the condition that the global path remains basically unchanged.

[0020] Advantages of the present invention: The self-driving unicycle invented in this article can identify highly concealed dangerous goods through on-board monitoring; in terms of logistics and take-out delivery, with the increase in labor costs, delivery work can be handed over to self-driving unicycles. In a small and congested area, the self-driving unicycle can give full play to its small size and flexibility, and can easily pass through a complex environment to achieve high delivery efficiency;

[0021] Its usage scenarios are in squares with heavy flow of people or areas with complex traffic. The self-driving unicycle can respond quickly to obstacles, make correct decisions and drive flexibly. The self-driving unicycle can flexibly find suitable roads in complex areas and can stop operating directly when obstacles are detected around it. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention is further illustrated by means of the non-limiting examples given in the accompanying drawings.

[0023] Figure 1 A process of a vehicle automatic driving method of the present invention Figure 1 ;

[0024] Figure 2 A process of a vehicle automatic driving method of the present invention Figure 2 ;

[0025] Figure 3 A process of a vehicle automatic driving method of the present invention Figure 3 ; DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0027] A vehicle automatic driving method of the present invention comprises the following steps:

[0028] S1: Calculate an optimal driving route based on the preset starting point and end point, as well as environmental map information;

[0029] Global path planning: Global path planning is the process of calculating the optimal driving route based on the preset starting and ending points and environmental map information before the autonomous unicycle performs a task. This process involves a comprehensive understanding of the surrounding environment, including roads, obstacles, traffic signals, etc. Global path planning usually uses a graph search algorithm. This article uses the Dijkstra algorithm to find the shortest or optimal path from the starting point to the end point.

[0030] Through this step, the global path of the self-driving unicycle that will be used for patrol or delivery can be reasonably planned, such as arranging the patrol route and the order and route of delivery. This process usually considers multiple factors such as road structure, traffic flow, and traffic light distribution to ensure the rationality, safety, and efficiency of the path. Self-driving unicycles often need to perform manned patrol tasks. In hot weather, if the global path planning can achieve driving on the road in the maximum shade, it can provide a relatively comfortable environment for patrol personnel riding the self-driving unicycle, so as to facilitate better work. We develop a route planning method based on global path planning by considering the impact of shadow ratio on the perceived temperature effect. By applying a segmentation algorithm, shadow-covered areas are identified and extracted from high-definition satellite photos; these areas are then integrated into a multi-layer route map, allowing the self-driving unicycle to customize the route based on the balance between distance and shadow exposure, thereby improving comfort and health during driving;

[0031] Key concepts include: Attitude fusion technology: also known as AHRS (Attitude and heading reference system), which integrates data from multiple sensors (such as gyroscopes, accelerometers, magnetometers, GNSS, etc.) and uses advanced algorithms (such as Kalman filtering, complementary filtering, etc.) for fusion processing to provide stable and reliable attitude information; Map matching technology: a method of matching real-time location data collected by GPS or other positioning systems with map data to improve positioning accuracy and navigation reliability; RGB mask: an important concept in image processing, especially when it comes to image synthesis, editing or analysis. RGB mask is usually not a fixed format or tool directly applied to images, but refers to a technology or method that uses color information in the RGB color space to create a mask; SegmentAnything: the model can recognize and segment any object based on text instructions or image recognition. The model consists of three main parts: image encoder, prompt encoder and mask decoder;

[0032] Steps and process: 1. Vehicle positioning: Both the vehicle model solution and the back-end fusion solution have achieved complete integration of sensors, GPS and map data, so the positioning effect is better. The vehicle model solution uses the sensor data of the vehicle CAN bus, while the back-end fusion uses the inertial navigation sensor installed in the vehicle. After fine-tuning the sensor selection and installation and use methods, the best positioning effect and motion information can be obtained;

[0033] 2. Obtain map information and derive shadow ratio: Use map information and real-time traffic monitoring to obtain detailed information about dynamic road conditions such as traffic congestion, accident reports, and road construction. When deriving the shadow ratio, note that the optimal threshold of the RGB mask is 75;

[0034] Shadow ratio derivation: (1) Use the basic model SegmentAnything to segment the original satellite image to identify and describe each object in the image; (2) Evaluate the color, hue, and brightness of the segmented image elements to identify the shadow parts. By analyzing the chromaticity, those less bright areas are determined to be shadow areas; (3) Align the shadow areas with the road data in OpenStreetMap to determine the percentage of each road covered by shadows;

[0035] 3. Calculate the path: The A* algorithm is mainly used. It evaluates the priority of nodes by introducing heuristic functions (such as Manhattan distance or Euclidean distance), which significantly improves the search efficiency and can find a near-optimal solution in a shorter time. At the same time, it calculates the proportion of the length covered by the shadow on the path to the entire path length.

[0036] 4. Multi-layer map construction: A multi-layer road map is constructed based on the shadow ratio data, including the upper shadow ratio map and the lower geographic map. The shadow ratio map is a set of all possible paths, while the geographic map is obtained from OpenStreetMap (OSM) data;

[0037] (1) Based on the shadow ratio data, a multi-layer graph containing a shadow ratio layer is constructed. The layer is a universal set containing all possible links.

[0038] (2) mapping the shadow ratio map onto a geographic map, and automatically removing it from consideration if connectivity does not exist;

[0039] 5. Route planning and user selection:

[0040] (1) Applying a modified version of Dijkstra's algorithm to provide personalized route planning suggestions, we can input the coordinates of the starting point and destination and the mode of transportation (here it is applied to the self-driving unicycle);

[0041] (2) Update the edge value and calculate the shadow ratio based on the user's preference for shadow and distance;

[0042] Calculation method: The query can be entered in the format of Query(O, D, Type), where O and D represent the coordinates of the origin and destination, denoted as (x, y) and (x', y'), respectively. The Type parameter specifies the mode of transportation, such as a self-driving unicycle. Depending on the type selected, different graphs are used. This system is able to better provide customized path planning services based on the unique preferences and needs of users. Let α be the preference for the more shaded path α∈[0,1], and 1-α is the preference weight for the distance. Therefore, for any edge with connectivity, the value of the edge will be determined by the equation:

[0043] w joint (u, v) = αV shade +(1-α)V distance

[0044] Where Vshade can be expressed as r(u, v), given two interest points u and v, R(·) is used as the shadow ratio calculation function;

[0045] 6. Dataset construction:

[0046] (1) Use satellite images as basic data and query the Google Maps API to obtain satellite images of specific cities;

[0047] (2) Process the image data after 20 times zoom and calculate the resolution and coverage;

[0048] Using 20 times magnified pile image data for processing, the resolution at the equator is 0.1246 meters / pixel (400 pixels in total, so the range at the equator is 400×0.1246=49.84m); at general latitudes, the resolution RES is:

[0049] Res=49.84×Cosin(lat)

[0050] 7. Integrate route information and import: Integrate the shadow route planning with the planned path in step 3 to form a convenient path with maximum shadows, and finally import it into the self-driving unicycle system;

[0051] S2: The YOLOv7 target detection model, which integrates the RRT-based ROS navigation stack, helps the system quickly and accurately identify the location and size of obstacles;

[0052] Accurately and quickly identify obstacles during driving: During driving, the self-driving unicycle needs to monitor the surrounding environment in real time to identify possible obstacles. This paper uses the YOLOv7 target detection model integrated with the RRT-based ROS navigation stack to help the system quickly and accurately identify the location and size of obstacles;

[0053] The use scenarios of the self-driving unicycle may be squares with complex traffic conditions and congested streets with narrow roads. This requires the self-driving unicycle to have more advanced visual algorithms that can accurately and quickly identify various obstacles during driving. We integrate the ROS navigation stack and the pre-trained YOLOv7 object detection model by combining the rapid exploration random tree (RRT) to enhance the working capabilities of the NAV-YOLO system development. Based on the NAV-YOLO system, we improved a hybrid framework that integrates the capabilities of the RRT-based ROS navigation stack and the YOLOv7 target detection model. The solution is enhanced by integrating lidar and camera sensor fusion and the YOLO target detection model, and the sensor information lidar and camera system are used to improve the adaptability of the mobile robot in changing and complex environments. At the same time, we hope that the computational complexity of the obstacle recognition algorithm can be as low as possible to compensate for the uncertainty of the environment; existing experiments have proved that the ellipsoid can better approximate the approximate obstacles. The clustering method is widely used to identify obstacles in dynamic environments, but it is not possible to quickly identify obstacles because the number of clusters needs to be known. Here, we propose an ellipsoid-based obstacle recognition method to represent obstacles and estimate their movement without knowing the number of clusters in advance. This algorithm can produce accurate output in an environment with complex obstacles, quickly and accurately identify obstacles and predict the direction of movement of obstacles, so as to ensure the safety of the self-driving unicycle during driving;

[0054] The key concepts include:

[0055] 1. CNN-Based YOLOv7 Model: YOLO is one of the many CNN-based computer vision models. It is based on a single-stage, end-to-end training and detection process. Due to its ability to perform multiple detections simultaneously, its improved variants, such as YOLOv7, have superior accuracy and inference speed, outperforming other related object detection models.

[0056] 2. Mean Average Precision (mAP): Mean Average Precision (mAP) is a commonly used evaluation metric in the field of computer vision and machine learning, especially in object detection tasks. mAP is the average of the mean accuracies of all categories. In object detection tasks, it measures the accuracy of the model for detecting different categories of objects.

[0057] 3. IoU (Intersection over Union): It is an indicator used to measure the similarity between the predicted bounding box and the true bounding box in the object detection task. It is calculated as the ratio of the area of ​​the intersection of the two bounding boxes to the area of ​​their union. The value of IoU ranges from 0 to 1. The larger the value, the higher the overlap between the predicted box and the true box. Usually, the detection is considered successful when IoU reaches 0.5 or above.

[0058] 4. FPS: Frames per second, an indicator of video playback, gaming or real-time graphics rendering performance; in video playback, high FPS can provide a smoother viewing experience, meaning faster response and less latency;

[0059] 5. LLVIP: LLVIP is a dataset that stands for “Low-Light Visible-Infrared Imaging and Processing”; the LLVIP dataset is used to expand the model’s ability to recognize objects in low-light conditions;

[0060] 6. DenseFuse network: is a neural network architecture for data fusion that combines data from different sensor sources to improve the accuracy of object detection and recognition; the DenseFuse network is used to fuse RGB (red, green, and blue) images and thermal (infrared) images to enhance the model's obstacle detection capabilities in low-light conditions;

[0061] 7.CBAM: Convolutional Block Attention Module (CBAM), also known as convolutional block attention mechanism, is an advanced attention module designed to enhance the performance of convolutional neural networks (CNNs); through a carefully designed mechanism, CBAM can identify and highlight key features in the input data, thereby optimizing the network's learning ability and prediction accuracy. CBAM enhances the model's feature learning ability by focusing on important features in the image and suppressing unimportant or redundant information;

[0062] 8. Roboflow: is a data annotation tool for machine vision and computer vision that allows users to create, train, and deploy custom models, especially for image recognition and classification tasks;

[0063] 9. Clustering method: Clustering technology is a means of unsupervised learning. Its core task is to automatically classify the elements in the data set into several groups or "clusters". In this process, the clustering algorithm strives to achieve high similarity between elements in the same cluster, while ensuring low similarity between elements in different clusters. Clustering processes the data collected by sensors (such as lidar, radar, camera, etc.) to group similar or related data points, thereby identifying and locating obstacles.

[0064] 10. Gaussian mixture model (GMM): A probability model used to represent a data set with multiple sub-populations, where the data distribution of each sub-population can be approximated by a Gaussian distribution (normal distribution); GMM is a special case of a finite mixture model, in which the distribution of the mixture components is assumed to be a Gaussian distribution. Given a set of points ν, we can express the probability density function of a GMM as:

[0065]

[0066] Where J is the number of clusters, N(ν; μj, Σj) is a multivariate Gaussian distribution, μj is the mean vector, Σj is the covariance matrix, and πj is the weight coefficient of the jth component, satisfying ∑πj=1;

[0067] 11. Khachiyan algorithm: Khachiyan algorithm is an iterative method. Each iteration takes a certain point x' as the center and constructs an ellipsoid E (n-dimensional ellipsoid) according to certain rules. The first ellipsoid E is the circle x 2 <2', its center x' is the origin; the iterative process is from x', Q to x2, Q2, and then to x3, Q3..., with a maximum of 6n iterations 2 l times, where Q, l is the input length and n is the number of unknowns; finally, the best fitting result is obtained by repeatedly adjusting the shape and position of a series of ellipses;

[0068] 12. OBB (Oriented Bounding Box): Enhances the accuracy of collision detection by fitting more closely around the object outline; the size and orientation of the OBB changes as the object rotates, always staying aligned with the object's primary orientation;

[0069] 13. Kalman Filter: Kalman Filter is an effective recursive filter used to estimate the state of a system in a noisy environment. It is based on the assumption of linear system and Gaussian noise, and combines a series of observations in a mathematically optimal way to predict the state of the system.

[0070] 14.Robot Operating System (ROS): The ROS package is a flexible framework for robotics software development; it is an open source, extensible, and widely used middleware that provides a set of tools, libraries, and conventions for building robotic applications; the following are the key ROS packages proposed in this article and their functions:

[0071] (1) ROS Navigation Stack: is a set of algorithms for robot navigation, including a global path planner and a local path planner, which work together to generate an obstacle-free path from the robot's current position to the target position;

[0072] (2) Dijkstra: As one of the global planners in the ROS navigation stack, it is used to explore the entire space to find the shortest path, but it may be computationally expensive, time-consuming, and impractical in dynamic environments;

[0073] (3) A*: Also a global planner in the ROS navigation stack, used to find the shortest path;

[0074] (4) sensor_msgs / LaserScan: ROS package for publishing LiDAR data, which provides geometric information about the environment and a 360-degree field of view;

[0075] (5) costmap_common_params.yaml: In ROS (Robot Operating System), costmap_common_params.yaml is a configuration file that defines common parameters of the costmap. A costmap is a two-dimensional grid representation used for robot navigation, where each cell has a value associated with the cost of traveling through it; this cost can be calculated based on the distance of obstacles, terrain type, or other factors; the costmap_common_params.yaml file allows developers to adjust the behavior of the costmap based on the application scenario; it can be used as a configuration file to set common parameters of the costmap, such as obstacle range, update frequency, and resolution;

[0076] (6) local_costmap_params.yaml and move_base_params.yaml: These configuration files are used to set the dynamic layer parameters of the costmap and the path planner parameters of the ROS navigation stack;

[0077] (7)ROS tf package: ROS package for coordinate transformation, ensuring that camera images and LiDAR scans are represented in the same coordinate system;

[0078] Steps process:

[0079] 1. Data set preparation and model training:

[0080] (1) Generate a custom dataset using MRO hangar obstacles (such as tools, vehicles, and structural elements) in real-world and simulated environments to train the YOLOv7 model;

[0081] (2) Combined with public datasets such as LLVIP to enhance the model's obstacle recognition ability under low-light conditions. In terms of data fusion, the DenseFuse network was used for the first time to fuse real RGB and thermal images;

[0082] 2. Model architecture adjustment:

[0083] Fine-tune the YOLO model architecture and integrate the Convolutional Block Attention Mechanism Module (CBAM), which prioritizes important object details related to real-time detection by focusing on relevant features and suppressing background noise to improve the focus on relevant features in real-time detection;

[0084] 3. Data preprocessing:

[0085] Data annotation, data augmentation (flipping and cropping techniques), and image resizing (640×640 pixels) were performed using the Roboflow tool;

[0086] 4.YOLOv7 model training and evaluation:

[0087] (1) Train the YOLOv7 model offline and evaluate it to ensure that the model can accurately identify obstacles even in low-light conditions.

[0088] (2) Ensure that the model's reasoning speed is fast and reliable to meet the real-time processing requirements in ROS;

[0089] The general process of YOLOv7 model training is as follows: prepare the data set using the process in step 1, adjust the model architecture using the process in step 2, and preprocess the data using the process in step 3; label the data set, which involves determining the boundaries of obstacles and assigning corresponding category identifiers to them; apply data enhancement methods, such as image flipping and cropping, to enhance the model's adaptability to new situations and reduce overfitting;

[0090] Resize the image to the input requirements of the YOLOv7 model, such as 640×640 pixels;

[0091] (4) Model training: Use the default hyperparameter settings, determine the batch size, and perform iterative training at an appropriate resolution;

[0092] The training phase is performed in a non-real-time environment (offline), ensuring that the model design can work effectively under resource constraints;

[0093] (5) Model evaluation: During the model training process, metrics such as mean Average Precision (mAP) and intersection over Union (IoU) are used to measure the detection accuracy of the model; ensure that the model's inference rate (Frames Per Second, FPS) can meet the standard of real-time processing;

[0094] (6) Model fine-tuning: Fine-tune the model based on the evaluation results and optimize the model parameters to improve detection accuracy and speed;

[0095] (7) Model verification: Verify the model’s reasoning speed and accuracy in the ROS environment to ensure that the model can run in real time on the robot’s hardware;

[0096] (8) Model deployment: Save the trained YOLOv7 model and weight files and transfer them to a dedicated ROS package, create a custom node to load the model and perform object detection;

[0097] (9) Model fusion: The detection results of the YOLOv7 model are fused with LiDAR data to enhance the robot's perception of the environment;

[0098] (10) Model testing and feedback: Test model performance in real or simulated environments, collect feedback and make necessary adjustments;

[0099] 5.ROS integration:

[0100] Save and transfer the trained YOLOv7 model and weight files to a dedicated ROS package, and create a custom node to load the model and weights;

[0101] 6. Sensor Fusion:

[0102] Integrate LiDAR and camera sensor data, combining LiDAR’s geometric information and 360-degree field of view with the rich environmental variables detected by YOLO;

[0103] 7. Coordinate conversion: Convert the 2D image coordinates detected by YOLO into 3D camera coordinates, and then into the robot coordinate system to understand the spatial positioning of the object in the real world;

[0104] 8. Costmap configuration: configure the laser scan data in the costmap, update the costmap unit, and provide a dynamic and accurate environment representation for the ROS navigation stack path planner to facilitate the subsequent generation of obstacle avoidance algorithms;

[0105] 9. Using the variational Bayesian estimation of the Gaussian mixture model and the Khachiyan algorithm and improved algorithm to solve the minimum volume ellipsoid problem, the obstacle range can be identified and its route can be analyzed:

[0106] (1) Plot using the GMM model, variational Bayesian estimation of Gaussian mixture model clustering based on the maximum number of components. Each color shows a different cluster, and n is the maximum number of clusters;

[0107] (2) Using Bayes’ rule, calculate its maximization:

[0108]

[0109] Where L(q) is the evidence lower bound (ELBO), and p(ν) is a fixed value, so the minimum KL divergence is equal to the maximum L(q);

[0110] (3) Simplify the problem of finding the minimum volume ellipsoid of each of the J clusters into the problem of a single minimum volume ellipsoid to simplify the calculation process; calculate the OBB of the sphere. In order to merge two ellipsoids, we use the minimum volume ellipsoid that covers the union of the ellipsoids. Note that the threshold relli,j determines which ellipsoids should be combined;

[0111] (4) Three different ellipsoid definitions are used; the first is called the general form, which is based on the norm definition:

[0112]

[0113] in and The second definition is called the quadratic form, which is an open version of the general form

[0114]

[0115] in The last definition is called the standard form, which is a modified version of the quadratic form:

[0116]

[0117] in is the center of the ellipsoid, is the rotation matrix, θ∈[0,π] is the rotation angle, is the ellipsoidal zone, and are the major and minor axes of the ellipsoid;

[0118] Let the total number of ellipsoids be Therefore, the space covered by all ellipsoids is defined as The shape of the obstacle Point representation, let K = {1, 2, ..., n o bs}; the position of the kth point is recorded as The set of all points of a given shape is A given point is covered if and only if all points in the shape are elements of at least one ellipsoid; in other words, the set O obs must be a subset of Eall;

[0119] We define the eigenvectors as the ellipsoid parameters in standard form:

[0120] fte(E)=[x c , r1, r2, θ]

[0121] The idea is that if the change in the future vectors of two frames close in time is small, they are considered to be the same object; then the change in the future vectors is determined using the Euclidean distance. Assuming that the i-th ellipsoid is generated at time t1 and the j-th ellipsoid is generated at time t2, the distance between them is defined as:

[0122] d i，j =||fre(θ i )-fre(θ i )||2

[0123] (5) The jth ellipsoid is matched with the ellipsoid with the smallest distance in the previous frame. We perform motion estimation based on the assumption that the obstacle maintains a constant linear velocity and a constant angular velocity; when two ellipsoids match, the linear and angular velocities of the jth ellipsoid are calculated using simple numerical derivatives:

[0124]

[0125] (6) After that, a Kalman filter needs to be integrated to reduce the error with more samples (because the ellipsoidal sphere is generated by an approximate algorithm);

[0126] S3: After identifying an obstacle, the self-driving unicycle quickly avoids collision through a decision-making mechanism;

[0127] Quick response to avoid obstacles: Once an obstacle is identified, the self-driving unicycle must be able to react quickly to avoid collision, which requires the system to have an efficient decision-making mechanism and fast execution capabilities;

[0128] After quickly and accurately identifying various obstacles, the self-driving unicycle needs to respond to various obstacles. Because the main application scenarios of the self-driving unicycle designed in this article may be patrolling in squares with heavy traffic, or performing logistics distribution in streets with complex environments, this requires the self-driving unicycle to have excellent obstacle avoidance capabilities. Even if a collision occurs without any other options, it can ensure that the impact force of the self-driving unicycle is significantly reduced after a quick response, and will not cause serious damage. We conducted a comparative evaluation of two obstacle avoidance modes (modulated dynamic system MDS and reactive driving support RDS). According to the safety design considerations of robot collision, contact sliding limits the force to a certain safety threshold, and a control architecture for responsive control in dense crowd navigation is produced. A crowd navigation control framework is proposed, which provides continuous obstacle avoidance and post-contact control, and is evaluated on an autonomous personal mobile vehicle. We proposed evaluation indicators for computational efficiency, controller response, and crowd interaction in natural crowds.

[0129] The key concepts include:

[0130] 1. Modulated Dynamics System (MDS): It is an advanced control framework for obstacle avoidance and navigation of robots and autonomous vehicles. The core idea of ​​MDS is to achieve obstacle avoidance by adjusting the dynamic system while maintaining continuous control of the robot or vehicle motion.

[0131] 2. Reactive Driving Support (RDS): is an obstacle avoidance method designed for autonomous vehicles that focuses on providing real-time obstacle avoidance support in complex and dynamic environments, such as crowded areas;

[0132] 3. Model Predictive Control (MPC) with Quadratic Programming Application: It is an advanced control technique that improves current control decisions by leveraging predictions of future states. In this way, MPC can effectively calculate the optimal control inputs for braking and steering while avoiding collisions, which is suitable for autonomous vehicles and other robotic systems that require real-time obstacle avoidance.

[0133] 4. Partially Observable Markov Decision Process (POMDP):) is a decision-making framework used to deal with decision-making problems in an environment where some information is visible, and the agent needs to make decisions with incomplete information; this requires the agent to be able to maintain and update its probabilistic estimates of the state of the environment and choose the best action based on these estimates; POMDP provides a powerful mathematical tool to deal with this uncertainty and has been proven to be effective in many practical applications;

[0134] 5. DR-SPAAM: is an algorithm for detecting and tracking people from two-dimensional range data; it combines spatial attention mechanism with autoregressive modeling to enhance detection accuracy and efficiency; it uses spatial attention mechanism to focus processing resources on foreground areas, i.e. those areas that are more likely to contain people; this helps to reduce the amount of computation and increase processing speed; it can also dynamically adjust its parameters and behaviors according to changes in the environment to adapt to different detection conditions and scenarios; further, it can be fused with other sensor data, such as combining LIDAR and RGBD data, to provide more comprehensive and reliable detection;

[0135] Steps process:

[0136] 1. High-level planning: First, determine the direction and speed of the self-driving unicycle;

[0137] 2. Reactive control layer: A control layer is set up to handle local obstacle avoidance and post-collision control. This control layer achieves control by utilizing the closed-loop dynamic characteristics of the robot; it assumes that there is a continuous DS given by a high-level algorithm to guide the robot to move to the desired goal (user command or DS given by the planner); the algorithm performs optimal planning for its desired goal (user-provided command, or DS given by the planner);

[0138] To explain this high-level algorithm in detail:

[0139] (1) Definition of dynamic system: A dynamic system is a set of mathematical equations used to describe the state of a system that changes over time. In this paper, dynamic systems are used to simulate and guide the motion of a robot.

[0140] (2) System goal setting: The main goal of the dynamic system is to guide the robot's movement so that it can move towards a predetermined goal (attractor); this goal can be a command given by the user through the interface, or a dynamic goal generated by the planner;

[0141] (3) Time invariance: The dynamic system used is time invariant, which means that the dynamic characteristics of the system do not change with time and can provide stable navigation behavior;

[0142] (4) Attractor: In a dynamic system, an attractor is a virtual target point to which the robot will be guided. The attractor is usually set at a certain distance in front of the robot (e.g., 20 to 50 meters).

[0143] (5) Motion planning: The dynamic system provides the robot with an optimal path to move from its current position to the attractor; this path takes into account the robot's kinematic and dynamic constraints;

[0144] (6) Representation in control space: In the robot's control space, the dynamic system represents the robot as a point that moves in Cartesian space to achieve simplified control of the robot's motion;

[0145] (7) Relationship with obstacles: The dynamic system is able to represent obstacles analytically, usually through star-shaped level sets of distance functions, which absorb the robot's footprint and allow the robot to bypass obstacles;

[0146] (8) Algorithm implementation: In implementation, the dynamic system controls the speed and direction of the robot through a set of differential equations (Mξ¨+Cξ˙=τc+τe); for example, a linear dynamic system is used to simulate the straight-line motion of the robot toward the attractor;

[0147] M: represents the virtual mass matrix of the robot, which is a 2×2 matrix corresponding to the mass distribution of the robot; ξ¨: represents the acceleration vector of the robot in Cartesian space, which is a two-dimensional vector representing the acceleration of the robot on the plane;

[0148] C: represents the centrifugal term and Coriolis term matrix, which is also a 2×2 matrix used to deal with nonlinear dynamic effects; ξ˙: represents the velocity vector of the robot in Cartesian space, which is a two-dimensional vector;

[0149] τc: represents the control force vector, which is the force calculated according to the control requirements of the robot and is used to drive the robot to move toward the target; τe: represents the external interference force vector, which may include external factors such as friction and wind force, which need to be compensated by the controller;

[0150] (9) Integration with the reaction control layer: The dynamic system is closely integrated with the reaction control layer to ensure that the robot can respond appropriately when encountering local obstacles or unexpected contacts;

[0151] (10) System feedback: Dynamic systems can adjust to changing external conditions based on the robot’s real-time status and environmental feedback;

[0152] 3. Using Modulated Dynamical Systems (MDS): Using MDS to represent obstacles, parsing them as star-shaped level sets of distance functions, allowing the robot to move as a point in Cartesian space; similarly, local obstacle avoidance can be achieved with Reactive Driving Support (RDS), which locally simulates the behavior of MDS (Minimum Distance Scheduler) by applying a velocity obstacle (VO)-based controller;

[0153] 4. Obstacle representation: Obstacles are represented by a distance function that absorbs the footprint of the self-driving unicycle so that the self-driving unicycle can be represented as a point moving in Cartesian space;

[0154] 5. Local obstacle avoidance: implemented through a velocity obstacle (VO) based controller, which behaves similarly to MDS but allows for a more accurate representation of the robot’s shape and kinematic constraints;

[0155] 6. Collision control: Use a compliance and contact force controller that uses known sensing surfaces on the self-driving unicycle shell to handle collisions; the controller ensures that the self-driving unicycle reacts to unexpected contact and moves forward with a sliding maneuver with potential obstacle avoidance guidance away from the contact surface without colliding with other obstacles;

[0156] 7. Kinetic modeling: The kinetic modeling is: where ξ˙∈R 2 A position-dependent dynamical system representing the robot's Cartesian velocity as time-invariant; Corresponding to the virtual mass of the self-driving unicycle, represents the centrifugal term and the Coriolis term, τc represents the control force, and τe represents the external disturbance that needs to be rejected;

[0157] Use a controller like this:

[0158]

[0159] in represents the driving force generated by the nominal DS from the obstacle avoidance input, acting tangentially on the collision surface (λ at contact t );F n represents the force limit selected within the range of safety and acceptability, while F crepresents the estimated contact force on the n° normal surface;

[0160] represents the negatively defined damping effect used to control sliding during contact. We transform the velocity to the domain of the self-driving unicycle by a first-order Taylor expansion, thus giving the desired velocity The control is:

[0161]

[0162] The discrete time constant (T s ) is set to the sampling time of the control loop, M is the virtual expected quality of the impulse response;

[0163] 8. Controller Design: Design a controller to generate the control force τc, taking into account the driving force generated from the obstacle avoidance input, the contact force, and the negatively defined damping effect for controlling sliding;

[0164] 9. Speed ​​transformation and desired speed control: transform the speed to the domain of the self-driving unicycle through the first-order Taylor expansion, and control the desired speed; (the formula is given in step 7)

[0165] 10. Low-level control: implement closed-loop speed regulation and provide status information feedback to higher-level controllers;

[0166] 11. Environment and crowd positioning and tracking: We set up a real-time lidar-based sensor fusion pipeline for people tracking using DR-SPAAM and YOLO’s RGB-D detection. We also used a stereo camera to estimate the state of the self-driving unicycle.

[0167] S4: After avoiding obstacles, the autonomous unicycle uses FRENETIX to enhance local path planning and replan the local path.

[0168] Optimized local path planning after obstacle avoidance (can achieve driving along the shadows) After avoiding obstacles, the self-driving unicycle needs to replan the local path to continue moving towards the destination; this local path planning is based on real-time information of the current position and environment, and FRENETIX is used to enhance the local path planning. The planner uses a multi-objective optimization strategy to improve trajectory comfort, safety and path accuracy in complex environments;

[0169] Through this step, the self-driving unicycle can make local route adjustments to cope with dynamic obstacles in the surrounding area while the global path remains basically unchanged, so as to ensure that the vehicle can continue to drive along the locally planned path while safely and accurately avoiding obstacles, avoiding congestion caused by stopping for too long, affecting time and road safety. Here we can use FRENETIX to enhance local path planning. FRENETIX is a public sampling-based trajectory planner for autonomous vehicles (AVs). The planner uses multi-objective optimization strategies to improve trajectory comfort, safety and path accuracy in complex environments, and can achieve path planning that can be completed within the specified time and with high safety. FRENETIX uses Python development for flexibility and prototyping capabilities, and also integrates C++ components to improve computational efficiency. At its core is an iterative motion planning cycle, including sampling, cost functions, risk assessment, validity checks, and safety features to continuously improve the vehicle's trajectory to cope with environmental changes; ultimately, the local path planner uses sensors to capture changes in the surrounding environment in real time, and passes these environmental data back to the global path planner so that the original path can be adjusted accordingly when necessary;

[0170] The key concepts include:

[0171] 1. Frenet coordinate system: The Frenet coordinate system is a coordinate system used for road vehicle trajectory generation and motion planning; its key features are:

[0172] (1) Path-based representation: The Frenet coordinate system is based on the centerline of the road or a reference path, and the position and motion of the vehicle are represented as a relationship to the path;

[0173] (2) Longitudinal and lateral displacement: In the Frenet coordinate system, the position of the vehicle is determined by two main parameters: longitudinal displacement (s) and lateral displacement (d). The longitudinal displacement represents the distance of the vehicle along the reference path, and the lateral displacement represents the lateral distance of the vehicle from the centerline of the reference path.

[0174] (3) Dynamic environmental adaptability: The Frenet coordinate system allows the reference path to be dynamically adjusted to adapt to changes in traffic conditions and obstacles;

[0175] (4) Simplify the planning problem: By decomposing the problem into longitudinal and lateral motion, the Frenet coordinate system simplifies the trajectory planning problem, making it easier to handle;

[0176] (5) Mathematical expression: In the Frenet coordinate system, the trajectory of the vehicle can be expressed as a parameterized curve with respect to the longitudinal and lateral displacements, for example, using a polynomial to describe the expected path of the vehicle;

[0177] (6) Trajectory generation: Using the Frenet coordinate system, the trajectory generation algorithm can create a trajectory that connects the vehicle's current state and the target state while taking into account the vehicle's dynamic constraints and environmental restrictions;

[0178] (7) Real-time feedback and adjustment: The Frenet coordinate system supports real-time feedback and trajectory adjustment in response to environmental changes and new sensor data;

[0179] (8) Integration with control algorithms: The generated trajectory can be converted back to a Cartesian coordinate system and sent to the vehicle’s control system for execution;

[0180] 2. Dijkstra algorithm: This algorithm is used to find the shortest path from a single source point to all other vertices in a weighted graph. Dijkstra algorithm is particularly suitable for processing graphs with non-negative weights;

[0181] 3. A* algorithm: A* (A star) algorithm is an algorithm for finding the shortest path from the starting point to the end point in a graph with multiple nodes on a graph plane; A algorithm is an extension of Dijkstra algorithm, which uses a heuristic method to estimate the distance from the current node to the target node, thereby improving the search efficiency;

[0182] FRENETIX algorithm steps:

[0183] 1. Trajectory planning algorithm:

[0184] First, environmental data is obtained from the map configuration. An initial optimal global path through the lantern network is required to form a reference path Γ in order to subsequently generate trajectories in the Frenet coordinate system to localize the ego vehicle. To select the optimal sequence of lantern strips, a graph optimization algorithm (such as Dijkstra or A*) is used. The selected patches are connected to each other through their centroids to form a navigable route.

[0185] 2. Trajectory planning cycle:

[0186] We distinguish between the trajectory planning cycle and the motion planning cycle. Trajectory planning is limited to the core functions of trajectory generation and evaluation, while motion planning represents the entire framework module including all modules and extensions. The most important thing in this step is trajectory sampling and trajectory evaluation, in which kinematic checks and cost calculations are also required.

[0187] (1) Trajectory sampling: first sample various data;

[0188] For the lateral trajectories, the sampling process involves different lateral distances dτ and different time horizons τ from the reference path Γ; similarly, for the longitudinal trajectories, the algorithm samples endpoints at different speeds vτ and also different time ranges; for the longitudinal trajectory generation, a quartic polynomial is used, which provides sufficient longitudinal motion while ensuring minimal jolts; by sampling m longitudinal trajectories and n lateral trajectories, a set consisting of m×n trajectories is constructed through the systematic cross-superposition of these two sets;

[0189] (2) Trajectory evaluation: Taking into account acceleration, curvature, curvature rate and yaw rate; the allowed acceleration allowed (t) is expressed as:

[0190]

[0191] Let v(t) be the velocity of the vehicle at any point on the trajectory, and a(t) be the corresponding acceleration; given a predefined maximum acceleration a max Threshold speed v switch , which depicts the transition from the constant acceleration limit to the variable acceleration limit; therefore, the kinematic feasibility is calculated by evaluating whether the actual acceleration a(t) at each trajectory point remains at -a max and the allowed range (t) is determined by:

[0192]

[0193] Here, t0 and t f denote the beginning and end of the time interval under consideration, respectively; further, the curvature κ(t) of the trajectory at any point in time must not exceed the maximum curvature κ max , the curvature is determined by the maximum permissible steering angle δ of the vehicle max And wheelbase L derived:

[0194]

[0195] Therefore, the curvature constraint can be expressed as to express;

[0196] The rate of change κ˙(t) should also be bounded to ensure a smooth transition during steering; therefore, we assume a maximum acceptable curvature rate To determine the feasibility of the trajectory, where:

[0197]

[0198] Finally, the yaw rate, which represents the rate of change of the vehicle's direction, ψ Should not exceed the max and the maximum yaw rate ψ determined by the aircraft speed v(t) max :

[0199] ψ max (t) = κ max v(t)

[0200] The constraints on the yaw rate are:

[0201]

[0202] Cost calculation:

[0203]

[0204] Ji(ξ) represents the i-th cost function in the realized cost function set, and the weighting factor ωi represents the relative importance of the corresponding cost component in the overall cost calculation;

[0205] 3. Modular architecture (divided into 8 modules)

[0206] M1: The global path planner computes the most efficient path to the target area in Cartesian coordinates; this path determines the order of patches to be traversed; the sequence of Cartesian coordinates [[x1,y1],...,[xn,yn]] is used as an interface from which a smooth continuous path is computed;

[0207] M2: The behavior and speed planner can generate new potential routes, select the best route, plan the target speed, and facilitate high-level decision-making;

[0208] M3: Occlusion-aware trajectory planning solves the risks caused by blind spots;

[0209] M4: Vehicle trajectory prediction using uncertainty integration, using the standardized ROS2 message format to establish the interface between trajectory planning and prediction algorithms;

[0210] M5: Risk assessment and danger prediction for traffic participants;

[0211] M6: We address the challenge of setting appropriate weights [w1,w2,...,wn] in sampling-based methods, especially in dynamic locations, by optimizing the trajectory selection process through a reinforcement learning-based framework.

[0212] M7: Verify and validate the most efficient trajectory through static and dynamic collision checking;

[0213] M8: A multi-agent simulation framework to study and test practical applications of motion planning frameworks.

[0214] S5: Identification of dangerous goods in public places:

[0215] Identification of dangerous objects in public places: When performing tasks in public places, the self-driving unicycle also needs to be able to identify potential dangerous objects. It uses improved pseudo-annotation in sub-terahertz security images to perform low-shot hidden target detection technology, and combines the existing Faster R-CNN and ResNet152 backbone technologies to strengthen the detection of dangerous objects that are highly hidden and difficult to detect with the naked eye, thereby improving public safety.

[0216] The content of the invention in this step is mainly to assist security personnel who use self-driving unicycles for patrol; in this regard, we can install surveillance cameras on self-driving unicycles and introduce related technologies to better detect dangerous objects and personnel; the present invention will use improved pseudo-annotations in sub-terahertz security images to perform few-shot hidden target detection technology, and combine the existing Faster R-CNN and ResNet152 backbone technologies to strengthen the detection of dangerous objects that are highly hidden and difficult to detect with the naked eye, thereby improving public safety;

[0217] The key concepts include:

[0218] 1. FLIR E60 camera: It is a high-end infrared thermal imager launched by FLIR. It is a model in the E series, designed for electrical and industrial inspection, with a variety of advanced functions and features; this thermal imager has an infrared image resolution of 320×240 pixels and can capture thermal images in real time at a frame rate of 60Hz to ensure that no details are missed; FLIR E60 is equipped with a 3-megapixel high-resolution visible light digital camera, which can capture clear visible light images, helping users to more accurately identify and understand infrared images;

[0219] 2. Two-stage fine-tuning method (TFA): is a method for adjusting and optimizing model parameters, especially in the field of machine learning and deep learning. This method usually involves two main tuning stages:

[0220] (1) Coarse Tuning: In this stage, the purpose of adjustment is to quickly approach the optimal solution. This usually involves updating parameters with a larger step size to allow for more extensive exploration in the parameter space. The coarse tuning process can help the model quickly escape from the local optimal trap and thus find a more favorable starting point.

[0221] (2) Fine Tuning: After rough tuning, the model is close to a good solution. In the fine tuning stage, the adjustment step size is reduced, with the aim of making more precise adjustments near the solution that has been found. The goal of this stage is to improve the performance of the model by optimizing the model parameters through small step size updates to obtain a more accurate solution.

[0222] 3. Fine-grained Classifier: A machine learning model designed to distinguish between very similar or closely related categories. In fine-grained classification tasks, the differences between categories may be very subtle, requiring the model to have a high degree of discrimination to accurately identify and classify.

[0223] 4. Visual Transformer (ViT): is a deep learning model based on the Transformer architecture, which was originally designed to handle natural language processing (NLP) tasks; however, ViT extends this architecture to the field of computer vision for processing image data; its core idea is to divide the image into multiple small blocks (called patches), and then linearly embed these small blocks into a series of vectors, which are then fed into the Transformer model for processing; this approach allows the model to capture the global and local features of the image, thereby achieving effective understanding of the image content;

[0224] 5. Average Precision (AP): It is a measure of the accuracy of the target detector. The calculation method is:

[0225]

[0226] where n represents each individual prediction made by the model; R(n) is a set of reference recall values, sorted by confidence score (high to low); Printerp is the precision value interpolated at each recall level; the above approximation is usually expressed as a percentage of the normalized area under the precision-recall curve;

[0227] 6. Precision Pr and recall R:

[0228] (1)Precision:

[0229] Precision measures the proportion of positive predictions that are actually positive (true positive) among all the positive predictions made by the model; in other words, it indicates how many of the positive predictions made by the model (e.g., detected objects) are correct.

[0230] (2) Recall:

[0231] Recall is an indicator of the proportion of samples correctly identified as positive by the evaluation model to all true positive samples; it reflects the ability of the model in identifying all actual positive samples, expressed as a percentage;

[0232] The values ​​of precision Pr and recall R can be calculated using the following two formulas:

[0233]

[0234] A true positive (TP) occurs when the label of a positive detection matches the actual category; a false positive (FP) occurs when the label of a positive detection is incorrect or mistakes the background for an object; a false negative (FN) occurs when the actual object is not detected; for object detection tasks, positive detections are defined by a specific IoU threshold; IoU is the ratio of the intersection area of ​​the detected ground truth bounding box with the ground to the union area of ​​the two bounding boxes;

[0235] 7. Stochastic Gradient Descent (SGD) optimizer: is an optimization algorithm used to train machine learning models, especially in deep learning and neural networks. Its basic principle is to minimize the loss function of the model by iteratively adjusting the parameters of the model (such as weights and biases).

[0236] Steps process:

[0237] Dataset preparation: We used the open-source low-contrast active submillimeter band image dataset from the China Academy of Engineering Physics (working in 140 GHz scanning array mode with a 5 mm × 5 mm scan resolution and 335,880 pixel resolution), and the thermal infrared image dataset from Imperial College London (using a commercial thermal infrared FLIR E60 camera) for fine-tuning;

[0238] Basic training: Two-stage Fine-tuning Approach (TFA) is used to train basic categories, which have abundant sample data.

[0239] Improved Pseudo-Annotation Method (IPA): (1) Annotation stage: Use multiple one-class detectors instead of fine-tuned detectors. These detectors are trained for a single hidden object category in submillimeter waveband images, which helps to alleviate the class imbalance problem in the training dataset.

[0240] (2) Classification stage: Replace the Visual Transformer (ViT) with a fine-grained classifier that is pre-trained on a thermal infrared image dataset and fine-tuned on an open-source submillimeter band image dataset; this adjustment provides more reliable and flexible label verification for the initial candidate objects, especially for hidden objects such as ceramic knives and metal daggers with similar appearance;

[0241] 4. Single-class detector training: For each hidden object category in the submillimeter waveband image, a single-class detector is trained to identify the initial candidate objects in the image;

[0242] 5. Fine-grained classifier training: Use the ResNet-18 network to extract features and pre-train on thermal infrared images through an early stopping strategy, and then fine-tune on the submillimeter band image dataset;

[0243] 6. Pseudo-annotation generation: Generate the coordinates and confidence scores of the initial candidate objects through a single-class detector, and then use a fine-grained classifier to re-verify these candidate objects and assign new category labels and confidence scores;

[0244] 7. Select pseudo-annotation: After collecting all the initial candidates, select the final pseudo-annotation candidates based on the confidence scores;

[0245] 8. Fine-tuning the object detector: The object detector trained using the TFA framework is further fine-tuned in combination with the generated high-quality pseudo-annotations.

[0246] 9. Result analysis: When detecting baseball bats, knives, and machetes, the models based on ResNet101 and Resnet152 achieved the best performance; in contrast, for guns and rifles, the model based on ResNet152 was the best; so the most effective model we obtained was ResNet152, and it is recommended to use ResNet152 as the backbone network of the Faster R-CNN model for autonomous unicycles and for detecting dangerous objects in public place monitoring systems;

[0247] S6: Fire source detection in public places:

[0248] The invention of this step is mainly used to detect fire sources in public places more efficiently and accurately and contact nearby security personnel through the alarm system after the fire source is discovered, thereby improving the safety of the area; in this step, we introduced a new framework that combines traditional methods with deep learning technology - the flame segmentation detection framework (FSDF). FSDF extracts color and texture information from images and enhances flame feature detection using hue, saturation and value (HSV) and completely local binary patterns (CLBP); in addition, YOLOv8 and vector quantized variational autoencoder (VQ-VAE) are woven into our framework structure for image segmentation and unsupervised fire detection, respectively; finally, we deploy the flame segmentation detection framework (FSDF) on a wheelbarrow to enable rapid and accurate detection of fire sources during patrols;

[0249] The key concepts include:

[0250] VQ-VAE model: includes encoder, decoder, vector quantization and embedding space; the purpose of the model is to encode the input image into a low-dimensional latent space and then reconstruct the image from this latent space;

[0251] Specific steps:

[0252] 1.HSV color space feature enhancement:

[0253] (1) Convert the input image from RGB color space to HSV color space;

[0254] (2) Setting the range of red hue, saturation, and value in the HSV color space to extract flame features in the image;

[0255] (3) Generate a binary mask using threshold processing to highlight the flame area;

[0256] (4) converting the enhanced image into a thermal map and superimposing it with the original image in proportion to obtain an enhanced flame feature image;

[0257] 2.YOLOv8 object detection:

[0258] (1) Use the YOLOv8 model to process the enhanced image and identify the flame area;

[0259] (2) Setting a lower confidence threshold to ensure that as many suspected flame areas as possible are detected;

[0260] (3) Based on the detection results of YOLOv8, the bounding box coordinates of the flame area are extracted;

[0261] 3.CLBP texture feature extraction:

[0262] (1) Crop the detected flame area and extract texture features using Complete Local Binary Pattern (CLBP);

[0263] CLBP:

[0264] Assume the center pixel is t c (x c ,y c ) The radius is R (R>0), we select the neighboring pixels around the center, there are m neighboring points. Neighboring pixel point t i The coordinates will be x c -rsin(2πi / m),yc +rcos(2πi / m); LBP selects the central pixel t by setting the threshold i The adjacent points t c , m binary numbers are generated through this threshold, and the calculation results are as follows:

[0265]

[0266] We change the binary value of "0" to "-1"; CLBP_M can be expressed as:

[0267]

[0268] Where c is the number of pixels in the entire image. i -t c |The average value of;

[0269] (2) Combining the cropped flame region and its CLBP features to form a richer feature representation;

[0270] 4. VQ-VAE Unsupervised Learning:

[0271] (1) Input the extracted flame features into the unsupervised learning model VQ-VAE (Vector Quantized Variational Autoencoders);

[0272] (2) VQ-VAE distinguishes between flame and non-flame areas by learning the distribution of flame features;

[0273] (3) Use the reconstruction error of VQ-VAE to assess whether a fire has occurred in the image;

[0274] 5.SSIM evaluation:

[0275] (1) Calculate the structural similarity index (SSIM) between the VQ-VAE reconstructed image and the original image;

[0276] (2) Determine whether there is a fire in the image by comparing the SSIM value with a preset threshold;

[0277] This indicator meets: SSIM sum =μ1SSIM1+μ2SSIM2+μ3SSIM3

[0278] Among them, SSIM1, SSIM2 and SSIM3 represent the SSIM values ​​calculated for RGB images, CLBP_S coded images and CLBP_M coded images respectively; μ1, μ2 and μ3 are weight parameters respectively; given training samples X = x1, X2, …, XD;

[0279] We can calculate SSIM based on these samples to determine the decision threshold (THR) that satisfies:

[0280]

[0281] is the SSIM between a sample X and its reconstructed output X' sum The sum of μ is their average value; the threshold THR is lower than SSIM sum The value of three standard deviations from the mean; during inference, the decision rule is: If SSIM sum (x i , x i ′)>THR, it can be determined that a fire event has occurred, otherwise it will be classified as non-fire;

[0282] S7: Facial recognition prevents potential dangers:

[0283] In this step, a face recognition system is introduced on the self-driving unicycle; the developed model uses the Haarcascade algorithm to scale the original image and convert it into a suitable format for subsequent analysis; in addition, the designed model uses principal component analysis (PCA) and ant colony optimization (ACO) for feature extraction and selection, respectively; finally, the DenseNet169 classifier is used to perform the face recognition task;

[0284] The key concepts include:

[0285] 1. Haar Cascade algorithm: It is a computer vision technique for object detection proposed by Paul Viola and Michael Jones in 2001. It is particularly suitable for face detection in real-time vision systems and is implemented in the OpenCV library. The core idea of ​​the Haar Cascade algorithm is to transform the object detection problem into a classification problem, identifying the object of interest through a series of Haar-like features and corresponding learning algorithms.

[0286] 2. PCA: (Principal Component Analysis) is a widely used statistical technique mainly used for data dimensionality reduction, feature extraction and data visualization; it transforms a set of possibly correlated variables into a set of linearly uncorrelated variables through orthogonal transformation, which are called principal components; the goal of PCA is to reduce the dimensionality of a data set while retaining the most important statistical information in the data set;

[0287] 3. FLDA: Fisher Linear Discriminant Analysis, a classic pattern recognition technology used for image recognition, image classification and other tasks; FLDA is an extension of LDA (Linear Discriminant Analysis), proposed by RA Fisher in 1936; FLDA aims to find a linear combination so that data of different categories have the largest inter-class distance and the smallest intra-class distance under this combination;

[0288] 4. Ant Colony Optimization (ACO) algorithm: It is a heuristic search algorithm that simulates the foraging behavior of ants in nature. It was first proposed by Marco Dorigo in 1992 and is mainly used to solve combinatorial optimization problems, such as the traveling salesman problem (TSP), graph coloring problems, and scheduling problems. The core idea of ​​the ACO algorithm is to simulate the process of ants finding food paths by releasing a chemical called pheromone to mark the path. Other ants tend to walk along the path with high pheromone concentration, thereby finding the optimal or near-optimal solution.

[0289] 5.SVM: Support Vector Machine (SVM) is a powerful machine learning algorithm mainly used for classification tasks. It distinguishes different categories by finding the best decision boundary (hyperplane) between data points. The criterion for choosing this decision boundary is to maximize the interval between data points on both sides of the boundary. SVM is particularly suitable for handling high-dimensional data and nonlinear problems. By using the so-called "kernel technique", it is able to find the optimal decision boundary in high-dimensional space.

[0290] 6. One-cycle strategy: One-cycle strategy is a learning method for training deep learning models. It is particularly suitable for optimizing the training process of neural networks. The core idea of ​​this strategy is to dynamically adjust the learning rate and other hyperparameters within a training cycle to achieve fast convergence and improve the performance of the model.

[0291] Specific steps:

[0292] 1. Data Collection:

[0293] (1) Collect facial images of relevant persons on the Internet;

[0294] (2) Use the Haar Cascade algorithm to filter out frontal facial images and delete other images to ensure that the database contains a unified grayscale representation of frontal facial features;

[0295] 2. Preprocessing: including removing artifacts and other processing steps in the image to improve image quality;

[0296] Other processing steps:

[0297] i. Load the input image using the built-in method cv2.imread(img path), where the image path is passed as an input parameter;

[0298] ii. After grayscale mode conversion, display;

[0299] iii. The Haar cascade classifier is loading;

[0300] (2) Use the Haar Cascade algorithm to perform grayscale conversion for subsequent analysis;

[0301] Haar value calculation: pixel value = (sum of dark pixels / number of dark pixels) - (sum of light pixels / number of light pixels)

[0302] (3) Apply data augmentation techniques to improve the model’s resistance to geographic variation;

[0303] 3. Feature extraction:

[0304] (1) Extract facial features using the principal component analysis (PCA) and Fisher linear discriminant analysis (FLDA) ensemble method;

[0305] (2) PCA projects the original image data into a new coordinate space, converting the two-dimensional facial image into a one-dimensional image vector;

[0306] (3) FLDA optimizes the projection image by calculating the intra-class scatter matrix and the inter-class scatter matrix;

[0307] 4. Feature Selection:

[0308] (1) Using the ant colony optimization (ACO) algorithm as the packaging model to select the optimal feature subset;

[0309] (2) Train the DenseNet model by initializing parameters, such as the number of ants and the number of iterations, and then selecting features based on probability;

[0310] (3) Evaluate the accuracy of the support vector machine (SVM) based on the test image set and update the pheromone intensity of the feature;

[0311] 5. Training DenseNet169:

[0312] (1) Using a staged training approach, first freeze the early layers of the network and only train the classification layer;

[0313] (2) Then load the fine-tuned model in the first stage and unfreeze all layers to train the entire model;

[0314] (3) Use a periodic learning rate instead of a fixed learning rate one-cycle strategy for training;

[0315] Loss function for class prediction:

[0316]

[0317] Where m is the number of images in the training dataset, p is the index of the input image (i.e., the pth image), q is the class index, epq is the one-hot encoded label, and dpq is the probability distribution over C classes;

[0318] 6. Use the improved DenseNet169 for facial recognition with the help of other hardware;

[0319] S8: Respond to potential dangers through surrounding sounds:

[0320] In this step, we developed a complete sound-based system for preventing potential dangers based on NLP and deep learning and deployed it on the self-driving unicycle. It can record the captured sound and transmit it to nearby security personnel for further analysis and judgment. The positioning system on the self-driving unicycle can also upload the location to the security personnel, so that a quick warning and precise positioning can be provided when danger occurs.

[0321] The key concepts include:

[0322] 1. FFT: FFT (Fast Fourier Transform) is an efficient algorithm for calculating the Fourier transform (FT) of a signal, which converts the signal from the time domain to the frequency domain. The Fourier transform is a mathematical technique used to analyze the frequency content of a signal, revealing which frequencies of sine waves and / or cosine waves the signal consists of:

[0323]

[0324] Where f(t) is the signal in the time domain, F(ω) is the signal in the frequency domain, ω is the angular frequency, and i is the imaginary unit;

[0325] 2. STFT: Short-time Fourier transform (STFT): STFT is an extension of FFT. It can capture the change of signal frequency over time by dividing the signal into short time frames and performing FFT analysis on each frame.

[0326] 3. Mel filter bank: Mel filter bank is a filter bank that simulates the sensitivity of human ears to sounds of different frequencies; it is very important in speech processing and audio analysis, especially in the generation of Mel-Spectrogram; Mel filter bank converts audio signals from frequency domain to Mel frequency domain to better reflect human perception of sound;

[0327] 4. LSTM: Long Short-Term Memory (LSTM) is a special Recurrent Neural Network (RNN) architecture that can learn and remember long-term dependencies. LSTM is designed to solve the gradient vanishing or gradient exploding problems encountered by standard RNNs when processing long sequences of data. This makes LSTM very effective in processing time series data, natural language processing, and other tasks that require memorizing long-distance dependencies.

[0328] 5.1D-CNN: One-dimensional convolutional neural network (1D-CNN) is a neural network architecture specifically designed for processing sequence data. It applies convolutional layers on one-dimensional signals to extract features. This network structure is particularly suitable for processing time series data, audio signals, or any form of linear sequence data. 1D-CNN learns local features of data by sliding convolution kernels across one dimension of the data. These convolution kernels can capture patterns and trends in the sequence.

[0329] 6.2D-CNN (Two-Dimensional Convolutional Neural Network): A two-dimensional convolutional neural network (2D-CNN) is a variant of a convolutional neural network (CNN) that is particularly effective when processing data with a two-dimensional structure, such as images. 2D-CNN extracts features by applying a convolution operation on two dimensions of the data (usually width and height).

[0330] Specific steps:

[0331] 1. Dataset preparation:

[0332] (1) Collect audio datasets, which usually contain multiple categories of sounds, such as animal calls, vehicle sounds, human activity sounds, etc.

[0333] (2) Clean the dataset and remove irrelevant or low-quality audio samples;

[0334] 2. Data preprocessing:

[0335] (1) Convert the audio files to ensure that all files have the same format and sampling rate;

[0336] (2) Perform FFT to convert the audio signal from the time domain to the frequency domain in order to analyze its frequency components;

[0337] (3) Generate a Mel-spectrogram, which is a nonlinear frequency scale that is more suitable for simulating human perception of sound;

[0338] 3. Feature extraction:

[0339] (1) Short-time Fourier transform (STFT):

[0340] The continuous audio signal is divided into short time frames (usually 20-40 milliseconds); FFT is performed on each frame to convert it from the time domain to the frequency domain; by windowing the signal at different time points and calculating the FFT, the STFT is able to capture the characteristics of the signal frequency changing over time;

[0341] (2) Generate Mel spectrum:

[0342] Apply a Mel filter bank to process the results of the STFT; the Mel filter bank consists of multiple overlapping triangular filters, each of which corresponds to the human ear's sensitivity to sound; pass the STFT spectrum through the Mel filter bank, and each filter outputs an energy value within a specific frequency range; calculate the logarithm of the energy value of each filter, and use these logarithmic values ​​as features;

[0343] (3) Conversion of Mel-spectrogram:

[0344] Convert the output of the Mel filter bank into a Mel spectrogram, which is a two-dimensional array where one dimension represents time (i.e., the frame of the STFT) and the other dimension represents the Mel frequency; the Mel spectrogram shows the frequency distribution over time, which helps capture the time-frequency characteristics of the audio signal;

[0345] (4) Data normalization:

[0346] Normalize the Mel-spectrogram to make its value range uniform for model training; normalization can be achieved by subtracting the mean and dividing by the standard deviation, which helps improve the training efficiency and performance of the model;

[0347] (5) Generate training samples:

[0348] Split the mel-spectrogram into fixed-size samples, each corresponding to a short time segment in the audio; these samples will be used as input to the deep learning model for training and predicting sound events;

[0349] (6) Label allocation:

[0350] Assign a label to each audio sample. The label corresponds to the sound category in the audio (e.g., dog barking, car horn, human voice, etc.). The label is used for supervised learning to guide the model to learn to distinguish different sound categories.

[0351] 4. Model training:

[0352] (1) Using the mel-spectrogram as input data, we train three deep learning models: LSTM, 1D-CNN, and 2D-CNN.

[0353] (2) Use cross-validation and hyperparameter tuning to optimize model performance;

[0354] (3) Implement data augmentation techniques, such as time stretching and frequency modulation, to increase the robustness of the model;

[0355] 5. Real-time audio recording module:

[0356] (1) Develop an audio recording module that can continuously monitor the sounds in the surrounding environment and record them;

[0357] (2) Implement key sound detection to determine when to start and end recording;

[0358] 6. Real-time audio prediction module:

[0359] (1) Perform the same preprocessing steps on the real-time recorded audio as the training data, including FFT and Mel-spectrogram generation;

[0360] (2) Use the trained deep learning model to classify the processed audio data;

[0361] 7. Threat Detection and Alert System:

[0362] (1) Determine whether the audio belongs to a dangerous category based on the output of the model;

[0363] (2) If a dangerous sound is detected, the audio and approximate coordinates are sent to security personnel through a preset communication protocol for further judgment.

[0364] The steps in S2 are as follows Figure 1 As shown:

[0365] 1. Camera image input: The system captures the environment image through the camera carried by the unicycle;

[0366] 2. YOLOv7 object detection model: Use the pre-trained YOLOv7 model to process camera images to detect and identify obstacles in the image;

[0367] 3. Obstacle localization: The YOLOv7 model outputs the bounding boxes of obstacles, including location and category information;

[0368] 4. Depth estimation: Use depth camera or LiDAR data to estimate the distance of obstacles;

[0369] 5. Conversion from pixel coordinates to camera coordinates: Convert 2D image coordinates to 3D camera coordinates to understand the spatial positioning of obstacles in the real world;

[0370] 6. Sensor fusion: Combining 360-degree environmental scan data from LiDAR with obstacle data detected by the camera to provide more comprehensive environmental information;

[0371] Costmap configuration: Fusion and update of data from LiDAR and YOLO detections in the ROS costmap to provide a dynamic and accurate representation of the environment for the path planner.

[0372] 8.DWA Local Planner: The DWA (Dynamic Windows Approach) algorithm is used for local trajectory planning and obstacle avoidance, taking into account the current speed of the unicycle, acceleration limits, and the immediate environment to calculate a safe and feasible path;

[0373] 9. Unicycle motion control: According to the linear and angular velocity instructions calculated by the DWA algorithm, the movement of the unicycle is controlled to avoid obstacles and move toward the target position;

[0374] 10. Data visualization and analysis: Use the RVIZ (Robot Visualizer) tool to visualize the path planning and avoidance decisions and make necessary adjustments;

[0375] The steps in S3 are as follows Figure 2 As shown:

[0376] 1. User intention identification: Identify the user's intention to use the unicycle this time through the system's user interface;

[0377] 2. High-level planning: The high-level planner determines the direction and speed of the robot's movement, or in fully autonomous mode, a linear dynamic system (DS) plans the movement to the desired goal (attractor);

[0378] 3. Reaction control layer: handles local obstacle avoidance and post-collision control, which is achieved through the closed-loop dynamics of the unicycle in its control space;

[0379] 4. Obstacle avoidance: Obstacle avoidance using velocity obstacle (VO) based optimization or modulated dynamic system (MDS);

[0380] 5. Collision control: Compliance and contact force control by using a finite contact reference force on a known sensing surface on the robot shell;

[0381] 6. Robot dynamics modeling: The unicycle dynamics is modeled as a second-order differential equation, taking into account virtual mass and control forces;

[0382] 7. Control force calculation: Calculate the required control force based on the robot’s current speed and collision conditions;

[0383] 8. Speed ​​domain conversion: convert the desired speed from the world coordinate system to the unicycle's coordinate system;

[0384] 9. Low-level control: ensure closed-loop speed control and status feedback to high-level controllers;

[0385] 10. Environment and crowd positioning and tracking: Crowd positioning and tracking through a real-time sensor fusion pipeline of LIDAR and RGBD data;

[0386] 11. Sensor integration: obstacle perception and force estimation using LIDAR, RGBD cameras, force / torque sensors, etc.

[0387] 12. SLAM (Simultaneous Localization and Mapping): Use stereo cameras and IMU (Inertial Measurement Unit) to estimate the robot state;

[0388] 13. Control command generation: Generate the final control command for the unicycle, including speed and posture adjustment;

[0389] 14. Emergency stop: When necessary, the user can intervene in the movement of the unicycle through the emergency stop switch;

[0390] 15. CAN bus communication: Communication between various control modules is carried out through the CAN bus;

[0391] 16. Analog-to-digital and digital-to-analog conversion: Performs necessary signal conversion to control the unicycle's motor and receive sensor signals;

[0392] 17. User interface: Provide a user control interface to allow users to perform manual control or intervention as needed;

[0393] The steps in S4 are as follows Figure 3 As shown:

[0394] 1. Scenario Preprocessing: Preprocess the scenario, such as obtaining environmental data from the CommonRoad scenario database;

[0395] 2. Initialize Trajectory Planner: Set the initial state and target of the trajectory planner;

[0396] 3. Calculate Initial Reference Path: Calculate the best lane sequence as the initial reference path through the graph optimization algorithm;

[0397] 4. Configure Additional Planning Modules: Set up additional planning modules such as global path planner, behavior and speed planner;

[0398] 5. Vehicle State Update: Update the latest location and trajectory data of the vehicle;

[0399] 6. Trajectory Sampling: Perform trajectory sampling in the Frenet coordinate system to generate trajectory samples;

[0400] 7. Trajectory Evaluation: Evaluate trajectory samples to check their feasibility and computational cost;

[0401] 8. Optimal or Emergency Trajectory: Select the optimal or emergency trajectory with the lowest cost and safety;

[0402] 9. Final Evaluation and Visualization: Perform a final evaluation and visualization of the selected trajectory;

[0403] 10. Scenario Update: Update scenario information according to environmental changes to prepare for the next cycle;

[0404] 11. Next Iteration: Start a new planning cycle iteration;

[0405] 12. Testing: Test the planning algorithm to verify its performance;

[0406] 13. Control: Convert the calculated trajectory into the vehicle’s control signal.

[0407] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for automatic vehicle driving, characterized in that: The following steps are involved: S1: Calculate an optimal driving route based on the preset starting point and end point, as well as environmental map information; S2: The YOLOv7 target detection model, which integrates the RRT-based ROS navigation stack, helps the system quickly and accurately identify the location and size of obstacles; S3: After identifying the obstacle, the self-driving unicycle quickly avoids collision through the decision-making mechanism; S4: After avoiding obstacles, the autonomous unicycle uses FRENETIX to enhance local path planning and replan the local path.

2. The method for automatic vehicle driving according to claim 1, characterized in that: In path planning, map information and real-time traffic monitoring are used to obtain detailed information on dynamic road conditions. By applying a segmentation algorithm, shadow-covered areas are identified and extracted from high-definition satellite photos. These areas are then integrated into a multi-layered road map, enabling the self-driving unicycle to customize its route based on the balance between distance and shadow exposure.

3. The method for automatic vehicle driving according to claim 2, characterized in that: The base model SegmentAnything is used to segment the original satellite imagery, identify and depict each object in the image, and evaluate the color, tone, and brightness of the segmented image elements to identify the shadow parts. By analyzing the chromaticity, those less bright areas are judged as shadow areas. The shadow areas are aligned with the road data in OpenStreetMap to determine the percentage of each road covered by the shadow.

4. The method for automatic vehicle driving according to claim 3, characterized in that: The A* algorithm is used to evaluate the priority of nodes by introducing a heuristic function, thereby significantly improving the search efficiency. It can find a near-optimal planned path in a shorter time and calculate the ratio of the length of the path covered by shadows to the total length of the path. A multi-layer road map is constructed based on the shadow ratio data, including an upper-layer shadow ratio map and a lower-layer geographic map. The shadow route planning is integrated with the planned path to form a convenient path with maximized shadows, which is finally imported into the self-driving unicycle system.

5. The method for automatic vehicle driving according to claim 1, characterized in that: The self-driving unicycle has a visual algorithm that can accurately and quickly identify various obstacles during driving. It combines the rapid exploration random tree RRT integrated ROS navigation stack and the pre-trained YOLOv7 object detection model to enhance the working capabilities of the NAV-YOLO system development, and then improves a hybrid framework based on the NAV-YOLO system to integrate the capabilities of the RRT-based ROS navigation stack and the YOLOv7 target detection model.

6. The method for automatic vehicle driving according to claim 5, characterized in that: A custom dataset was generated using MRO hangar obstacles in real-world and simulated environments to train the YOLOv7 model, combined with public datasets such as LLVIP to enhance the model's obstacle recognition capabilities in low-light conditions.

7. The method for automatic vehicle driving according to claim 6, characterized in that: The YOLO model architecture is fine-tuned and a convolutional block attention mechanism module (CBAM) is integrated, which prioritizes important object details related to real-time detection by focusing on relevant features and suppressing background noise to improve the focus of relevant features in real-time detection.

8. The method for automatic vehicle driving according to claim 7, characterized in that: When training the model, use the default hyperparameter settings, determine the batch size, and use the appropriate resolution for iterative training. The training phase is performed in a non-real-time environment to ensure that the model design can operate effectively under resource constraints. The inference speed and accuracy of the model are verified in the ROS environment to ensure that the model can run in real time on the robot's hardware. The trained YOLOv7 model and weight files are saved and transferred to a dedicated ROS package. A custom node is created to load the model and perform object detection. The detection results of the YOLOv7 model are fused with LiDAR data. The model performance is tested in a real or simulated environment, feedback is collected and necessary adjustments are made. Finally, the trained YOLOv7 model and weight files are saved and transferred to a dedicated ROS package, and a custom node is created to load the model and weights.

9. The method for automatic vehicle driving according to claim 1, characterized in that: The planning for rapid obstacle avoidance first determines the direction and speed of the autonomous unicycle, and sets up a control layer to handle local obstacle avoidance and post-collision control. Local obstacle avoidance is achieved through a controller based on the velocity obstacle, VO. In collision control, a compliance and contact force controller is used, which uses known sensing surfaces on the shell of the autonomous unicycle to handle collisions. The controller ensures that the autonomous unicycle responds to unexpected contact and moves forward with a sliding maneuver under potential obstacle avoidance guidance away from the contact surface without colliding with other obstacles.

10. The method for automatic vehicle driving according to claim 1, characterized in that: Use FRENETIX to enhance local path planning and plan the local path after obstacle avoidance while keeping the global path basically unchanged.

Images