Small carrier for manned and logistics and unmanned intelligent driving system thereof

Abstract

The invention provides a small carrier for manned and logistics and an unmanned intelligent driving system of the small carrier. The system is applied to the technical field of automatic driving and comprises a sensor module used for collecting the information of a carrier and the environment of the carrier, a path planning module used for performing target detection, road recognition, obstacle detection and positioning and map construction according to the information of the carrier and the environment of the carrier by using a multi-modal sensor fusion method, and obstacle avoidance and automatic path navigation are realized based on an improved path planning algorithm. The cloud management platform is used for monitoring the state of the carrier in real time, storing navigation data and vehicle health information through the cloud and supporting multi-vehicle collaborative scheduling; and the voice interaction module supports voice instruction input and is used for analyzing the voice instruction of the user, dynamically responding and re-planning the path by using the path planning module. According to the invention, the transportation efficiency and the intelligent level of the small-sized carrier for manned and logistics are improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a small vehicle for carrying people and logistics and an unmanned intelligent driving system thereof. Background Art

[0002] While existing autonomous driving technology has made some progress on open roads, traditional vehicle transportation methods face numerous challenges in closed or semi-closed environments, such as hospitals, parks, amusement parks, and shopping malls. These environments often feature complex environments, dynamic obstacles, and specialized functional requirements, placing higher demands on vehicle transportation efficiency and intelligence.

[0003] For example, in hospitals, manual transportation of medical supplies is not only labor-intensive but also susceptible to human interference, resulting in low transport efficiency and potential safety hazards. In parks and amusement parks, visitors must walk long distances to reach attractions or entertainment facilities, which not only reduces the visitor experience but also increases management complexity. In shopping malls, logistics and delivery must be completed efficiently and accurately, but traditional delivery methods often suffer from low efficiency and high error rates.

[0004] With the continuous development of autonomous driving technology, its application prospects in the field of vehicle transportation are becoming increasingly broad. However, the application of existing autonomous driving technology in these specific scenarios still has many shortcomings. On the one hand, traditional solutions rely on a single sensor for environmental perception, which makes it difficult to fully and accurately obtain information in complex environments, resulting in insufficient perception capabilities. On the other hand, existing algorithms perform poorly in dynamic obstacle avoidance, path smoothness, and computational efficiency, and their path planning is not rational, making them unable to effectively adapt to the complex environments and special requirements of these scenarios.

[0005] Therefore, there is an urgent need for an unmanned intelligent driving system for small vehicles that can fully perceive the environment, accurately locate, reasonably plan routes, and has strong adaptability to meet the intelligent transportation needs of these specific scenarios. Summary of the Invention

[0006] In order to overcome the shortcomings of the existing technology, the present invention provides a small transport vehicle for manning people and logistics and its unmanned intelligent driving system. Through the integration of technologies such as multi-sensor collaboration, improved Minkowski collision space modeling, and improved path planning algorithm, the present invention solves the technical problems of existing path planning and control systems such as unreasonable path planning and difficulty in adapting to complex constrained environments, thereby improving the transportation efficiency and intelligence level of small manned and logistics vehicles.

[0007] In a first aspect, the present invention provides an unmanned intelligent driving system for small vehicles used for passenger and logistics transportation, comprising: a sensor module, a path planning module, a cloud management platform, and a voice interaction module;

[0008] The sensor module is used to collect information about the vehicle itself and its environment;

[0009] The path planning module performs target detection, road identification, obstacle detection, positioning, and map construction based on the vehicle and its environment information and using a multimodal sensor fusion method, and implements obstacle avoidance and automatic path navigation based on an improved path planning algorithm;

[0010] The cloud management platform is used to monitor vehicle status in real time, store navigation data and vehicle health information in the cloud, and support multi-vehicle coordinated scheduling;

[0011] The voice interaction module supports voice command input, including navigation, control, query and emergency commands, and is used to parse user voice commands and respond dynamically, and replan the route using the route planning module.

[0012] Furthermore, the sensor module includes:

[0013] Binocular stereo cameras and surround-view cameras are used to obtain depth information of the environment and provide field of view. The two work together to provide rich visual perception information for the vehicle.

[0014] LiDAR / millimeter-wave radar, used to collect distance data of the vehicle;

[0015] The ultra-wideband positioning module and inertial measurement unit use extended Kalman filtering to achieve centimeter-level positioning and measure the vehicle's attitude and velocity information in real time;

[0016] Ultrasonic sensors and infrared sensors are used for low-speed, close-range obstacle avoidance and redundant detection in low-light environments, respectively;

[0017] Wheel speed meter provides accurate wheel speed information for calculating vehicle speed, direction, mileage and status monitoring.

[0018] Furthermore, the path planning module performs target detection, road recognition, and obstacle detection based on the vehicle itself and its environment information and using a multimodal sensor fusion method, including:

[0019] The target detection submodule fuses the data features of the binocular stereo camera, surround-view camera, and lidar / millimeter-wave radar, and then inputs them into the improved target perception algorithm YOLOv5 model for target detection, so as to quickly and accurately identify target objects including pedestrians, vehicles, and obstacles.

[0020] Road detection submodule: Combines visual segmentation technology to extract the drivable area from the camera image and processes the LiDAR point cloud data to extract the road edge and height information for data fusion. The fused features are then input into the road recognition model for road recognition.

[0021] Obstacle Detection Submodule: A joint calibration algorithm is used to establish a coordinate mapping relationship between the binocular camera and the lidar. The RGB-D point cloud is then fused with visual semantic information to construct a 3D obstacle model, which more accurately represents the position, shape, and size of obstacles. This model then serves as an obstacle layer in the three-dimensional map. The fused features are then input into a deep learning-based obstacle detection model for obstacle detection. Ultrasonic-based obstacle avoidance is enabled at low speeds of less than 1 m / s, and infrared sensors are used at night / in low light to ensure safety at all times.

[0022] Furthermore, the path planning module further includes:

[0023] Positioning and Mapping Submodule: This module uses point cloud data collected by the LiDAR, combined with information from binocular stereo cameras, surround-view cameras, ultrasonic sensors, and infrared sensors, to construct a three-dimensional map containing a variety of information such as terrain, buildings, roads, and obstacles. The precise position and posture information of the vehicle obtained by the ultra-wideband positioning module and the inertial measurement unit, as well as the wheel speed information obtained by the wheel speed meter, are fused using an extended Kalman filter (EKF) to correct the posture in real time.

[0024] Furthermore, the path planning module implements obstacle avoidance and automatic path navigation based on an improved path planning algorithm, including:

[0025] Collision detection and path planning submodule: uses Minkowski approximation to calculate the collision space, and uses hierarchical path planning to find a collision-free path in the collision space and use non-uniform rational B-splines for path smoothing to achieve efficient and accurate automatic navigation.

[0026] Furthermore, the collision detection and path planning submodule uses Minkowski approximation to calculate the collision space, including:

[0027] Collision Space Modeling Unit: Using Minkowski and The collision space is constructed and the Minkowski difference between the current vehicle position and the obstacle is calculated in real time to determine the shape of the obstacle in the collision space. This modeling method can more accurately reflect the spatial relationship between the vehicle and the obstacle, providing a reliable foundation for path planning.

[0028] Furthermore, the collision detection and path planning submodule searches for a collision-free path in the collision space based on hierarchical path planning and uses non-uniform rational B-splines for path smoothing, including:

[0029] Global planning unit: used to search for a collision-free rough path from the starting point to the target point in the global range of the rasterized collision space using the improved A* algorithm;

[0030] Local optimization unit: Based on the dynamic programming (DP) minimization cost function, the collision-free rough path obtained by the global planning unit is locally optimized to make it conform to the kinematic constraints and obtain the optimized shortest path;

[0031] Path smoothing unit: Non-uniform rational B-splines (NURBS) are used to smooth the shortest path obtained by the local optimization unit to generate a path curve C(t). The control point weights are dynamically adjusted according to the kinematic constraints, and ultimately a short and smooth executable path is obtained.

[0032] Furthermore, the global planning unit uses an improved A* algorithm to search for a collision-free rough path in the global range of the gridded collision space, including:

[0033] Discretize the constructed collision space into a grid map, where each grid represents an area and the state of the grid is free, obstacle or boundary;

[0034] The improved A* algorithm is used, which introduces an optimization strategy based on the traditional A* algorithm. A collision-free coarse path is searched in the gridded collision space. The actual cost of the node and the estimated value of the heuristic function are comprehensively considered to obtain the shortest path from the start node to the target node as the collision-free coarse path.

[0035] Furthermore, the local optimization unit minimizes the cost function J(x k ,u k ), the collision-free rough path obtained by the global planning unit is locally optimized to make it conform to the kinematic constraints. The optimized shortest path includes:

[0036] Assume that the gridded collision space is G(x, y), the starting point S(x s ,y s ), target point T(x t ,y t ), cost function J(x k ,u k ):

[0037]

[0038] Among them, J(x k ,uk ) indicates that from the current state x k Start, take control action u k After that, the minimum cumulative path cost to reach the target state; g(x k ,u k ) indicates that from the current state x k Start, take control action u k the immediate cost; Indicates that from the next state x k+1 Start and take all possible control actions u k+1 The minimum cumulative path cost in; the path constraint or state transfer equation is:

[0039] x k+1 =f(x k ,u k )

[0040] Among them, f(x k ,u k ) is the state transfer function, which determines the vehicle's current state x k Take control action u k After reaching the next state x k+1 The final solution:

[0041]

[0042] Among them, π* is the optimal path strategy, N is the path length;

[0043] Through the above x k+1 =f(x k ,u k ) recursively calculates the minimum cumulative path cost in each state, and finally finds the optimized shortest path from the starting point to the target point.

[0044] In a second aspect, the present invention further provides a small vehicle for carrying people and logistics, comprising the unmanned intelligent driving system as described in the first aspect.

[0045] Compared with the prior art, the present invention achieves the following beneficial effects:

[0046] By integrating multiple sensors (such as binocular stereo cameras, surround-view cameras, lidar, etc.) and multimodal sensor fusion, the present invention can fully perceive environmental information, accurately identify obstacles, and perform dynamic path planning based on an improved algorithm. The collision space is constructed through Minkowski approximation to solve the collision-free path, and the improved smoothing function is used to smooth the path, thereby achieving high efficiency, smoothness, adaptability to complex environments and high efficiency in path planning, thereby improving the overall performance and safety of the autonomous driving system, and greatly improving the transportation efficiency and intelligence level of small vehicles used for manned and logistics in closed or semi-closed scenarios. The design of this system not only meets the needs of small-scale manned or logistics transportation, but also has efficient, accurate and safe automatic navigation capabilities, remote control and unified scheduling management of multiple vehicles, improving overall transportation efficiency and resource utilization. At the same time, it supports multiple types of voice command input, and users can interact with the vehicle through voice, which improves the convenience of use.

[0047] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0049] Figure 1 This is a system flow chart of an unmanned intelligent driving system for a small vehicle used for passenger and logistics according to an embodiment of the present invention;

[0050] Figure 2 This is a module diagram of an unmanned intelligent driving system for a small vehicle used for passenger and logistics according to an embodiment of the present invention;

[0051] Figure 3 1 is a schematic diagram of submodules of the sensor module 101 according to an embodiment of the present invention;

[0052] Figure 4 1 is a schematic diagram of submodules of the path planning module 102 according to an embodiment of the present invention;

[0053] Figure 5 10 is a schematic diagram of subunits of the collision detection and path planning submodule 1025 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0056] An embodiment of the present invention provides a small vehicle for carrying people and logistics, which is used for carrying people (1-2 people) or logistics transportation and is suitable for closed or semi-closed scenarios (hospitals, parks, amusement parks, shopping malls, etc.): in hospitals, it can be used to pick up patients and transport medical supplies; in parks, it can be used as an unmanned sightseeing vehicle to provide convenience for tourists; in amusement parks, it can be used as an intelligent scooter to enhance the tourists' experience; in shopping malls, it can realize unmanned logistics distribution and improve logistics efficiency.

[0057] This small vehicle integrates multiple sensors to enhance its environmental perception capabilities. The system performs dynamic path planning based on an improved algorithm, constructs a collision space through Minkowski approximation to solve a collision-free path, and utilizes improved smoothing functions for path smoothing. These technologies enable it to achieve positioning, autonomously identify and avoid obstacles, and navigate within closed or semi-closed areas. This system design not only meets the needs of small-scale manned or logistics transportation, but also possesses efficient, accurate, and safe automatic navigation capabilities. It also enables remote control and unified scheduling and management of multiple vehicles, and supports various types of voice command input, facilitating interaction between users and unmanned vehicles. This meets the needs of different users in different scenarios, providing users with more comfortable and efficient transportation services and enhancing the user experience.

[0058] This small vehicle used for transporting people and logistics includes an unmanned intelligent driving system.

[0059] Figure 1 A flow chart of an unmanned intelligent driving system for small vehicles used for transporting people and logistics is shown; Figure 2 The figure shows a module diagram of an unmanned intelligent driving system for small vehicles used for transporting people and logistics. Figure 1 and Figure 2As shown, a small-sized vehicle unmanned intelligent driving system 100 for carrying people and logistics includes: a sensor module 101, a path planning module 102, a cloud management platform 103 and a voice interaction module 104;

[0060] The sensor module 101 is used to collect information about the vehicle itself and its environment. Figure 3 As shown, the sensor module 101 includes:

[0061] The binocular stereo camera and surround-view camera 1011 are used to obtain depth information of the environment and provide a field of view. Working together, they provide rich visual perception information for the vehicle. This embodiment of the present invention utilizes a combination of binocular stereo cameras and surround-view cameras. The binocular stereo camera obtains depth information of the environment through triangulation, while the surround-view camera provides a wider field of view, offering a 360-degree field of view. Working together, they are primarily used for target detection, road recognition, obstacle recognition, and pedestrian detection, providing rich visual perception information for unmanned vehicles.

[0062] The LiDAR / millimeter-wave radar 1012 is used to collect vehicle distance data. LiDAR uses 2D / 3D LiDAR, with 16- or 32-line options available based on cost and accuracy requirements. LiDAR measures the distance to target objects by transmitting and receiving laser beams, enabling high-precision obstacle detection and environmental mapping, providing reliable distance data for unmanned vehicle operation. Millimeter-wave radar measures distance and speed by transmitting millimeter waves and receiving reflected waves. This allows for highly accurate ranging and speed measurements, providing information about dynamic objects for positioning and mapping. The collected data includes distance, speed, and angle: each data point contains the target object's distance, speed, and angle relative to the radar.

[0063] The ultra-wideband (UWB) positioning module and the inertial measurement unit (IMU) 1013 employ extended Kalman filtering to achieve centimeter-level positioning and measure the vehicle's attitude and velocity in real time. Combining the IMU with ultra-wideband (UWB) positioning, the IMU measures the vehicle's attitude (such as pitch, roll, and yaw) and velocity in real time. During the positioning and mapping process, IMU information assists in correcting positioning errors, enhancing the unmanned vehicle's high-precision positioning capabilities.

[0064] Ultrasonic sensors and infrared sensors 1014 are used for low-speed, close-range obstacle avoidance and redundant detection in low-light environments, respectively. The ultrasonic sensor is used for close-range collision avoidance at low speeds, detecting nearby obstacles by emitting and receiving ultrasonic waves. The infrared sensor uses the infrared radiation characteristics of objects to assist in perception at night or in low-light environments, improving the safety of unmanned vehicles in complex lighting conditions.

[0065] The wheel speed meter 1015 provides accurate wheel speed information for calculating the vehicle's speed, direction, mileage, and status monitoring.

[0066] According to the structural characteristics of the unmanned vehicle, the hardware equipment is reasonably installed, and calibration and debugging are carried out to ensure that each sensor can work normally and accurately collect data.

[0067] The present invention integrates multiple sensors, which work together to provide the vehicle with comprehensive environmental perception capabilities.

[0068] A path planning module 102 performs target detection, road identification, obstacle detection, positioning, and map construction based on the vehicle and its environment information and using a multimodal sensor fusion method, and implements obstacle avoidance and automatic path navigation based on an improved path planning algorithm;

[0069] In some embodiments, as Figure 4 As shown, the path planning module 102 includes:

[0070] The target detection submodule 1021 fuses the data features from the binocular stereo camera, the surround-view camera 1011, and the lidar / millimeter-wave radar 1012, and then inputs them into a model based on the improved object perception algorithm YOLOv5 for target detection, enabling rapid and accurate identification of target objects such as pedestrians, vehicles, and obstacles. Optionally, in some embodiments, the binocular stereo camera and the surround-view camera 1011 capture rich image information, including color and texture, suitable for identifying target objects such as pedestrians and vehicles. The lidar provides high-precision three-dimensional spatial information, accurately capturing the shape, position, and velocity of target objects through point cloud data. Millimeter-wave radar has strong penetrating power and is suitable for detecting target objects in adverse weather conditions. The data collected by these different sensors is then converted to a unified coordinate system to facilitate subsequent data fusion and processing.

[0071] Multimodal sensor fusion methods, such as data-level fusion, feature-level fusion, and deep fusion, can fully leverage the strengths of different sensors and fuse data features from them. This can improve the accuracy and robustness of target detection, road recognition, and obstacle detection, providing a reliable foundation for decision-making in autonomous driving systems. The fused features are then input into a model based on the improved object perception algorithm YOLOv5 for target detection. During model training, images or video frames containing target objects such as pedestrians, vehicles, and obstacles are first collected. The images in the training dataset are annotated to indicate the location and category of the target objects. A training dataset is then constructed and the annotated training data is input into the YOLOv5 network. The network parameters are optimized using the backpropagation algorithm, enabling the model to accurately identify target objects.

[0072] Road detection submodule 1022: Combined with visual segmentation technology, it extracts the drivable area from the images of the binocular stereo camera and the surround view camera, providing a basis for the unmanned vehicle to plan the driving path, and processes the lidar point cloud data to extract the edge and height information of the road for data fusion. The fused features are input into the road recognition model for road recognition. Optionally, in some embodiments, the binocular stereo camera and the surround view camera obtain road image information, and the lidar provides three-dimensional structural information of the road, which helps to identify the boundaries and obstacles of the road. Visual segmentation technology, such as a fully convolutional network (FCN), is used to extract the drivable area from the camera image, and the lidar point cloud data is processed to extract the edge and height information of the road. The features from the camera and lidar are then fused to improve the accuracy of road recognition. The fused features are input into a road recognition model, such as a semantic segmentation model based on deep learning, for road recognition.

[0073] Obstacle Detection Submodule 1023: A coordinate mapping relationship between the binocular camera and the lidar is established through a joint calibration algorithm. The RGB-D point cloud is then integrated with visual semantic information to construct a 3D obstacle model, which more accurately represents the location, shape, and size of obstacles. The 3D obstacle model is then fed into a deep learning-based obstacle detection model as an obstacle layer within the 3D map for obstacle detection. Ultrasonic-based obstacle avoidance is enabled at low speeds of less than 1 m / s, and infrared sensors are switched to ensure safety at all times at night or in low light. Optionally, in some embodiments, features from the binocular camera and lidar are fused to construct a 3D obstacle model. This fusion can be performed at the data level, feature level, or decision level, depending on the application scenario and algorithm design. The fused features are then fed into an obstacle detection model, such as a deep learning-based object detection model, for obstacle detection. The 3D obstacle model allows for more accurate determination of the location, shape, and size of obstacles. This is achieved through the following steps:

[0074] Step 1: Establish a coordinate mapping relationship through a joint calibration algorithm, specifically including: (1) Install the binocular camera and lidar in appropriate positions on the small vehicle to ensure that they can cover the vehicle's front field of view; preliminarily calibrate the sensors to ensure that they are roughly aligned and minimize installation errors. (2) Use one or more calibration objects of known size (such as a checkerboard calibration plate) to place them in front of the vehicle to ensure that the calibration objects appear in the field of view of the binocular camera and lidar at the same time; and collect the image data of the binocular camera and the point cloud data of the lidar at the same time. (3) Extract feature points from the binocular camera image data. These feature points can be significant features such as corners and edges; extract point cloud data corresponding to the calibration objects from the lidar point cloud data. (4) Calculate the coordinate transformation matrix: Use a feature point matching algorithm (such as the ICP algorithm) to match the feature points in the binocular camera image with the feature points in the lidar point cloud. Calculate the coordinate transformation matrix (including the rotation matrix and translation vector) between the binocular camera and the lidar through the matched feature points, thereby establishing a coordinate mapping relationship between them.

[0075] Step 2: Fusion of RGB-D point cloud and visual semantic information, specifically including: (1) Obtaining RGB-D point cloud: The binocular camera calculates the disparity map through the stereo matching algorithm and then generates a depth map; combining the color image and depth map of the binocular camera to generate RGB-D point cloud data. (2) Extracting visual semantic information: Using a deep learning model (such as a convolutional neural network) to perform semantic segmentation on the color image of the binocular camera, extracting semantic information in the image (such as pedestrians, vehicles, trees, etc.). (3) Fusion of RGB-D point cloud and visual semantic information: Associate each point in the RGB-D point cloud with the corresponding pixel in the semantic segmentation result, and assign a semantic label to each point. In this way, each point in the point cloud has not only spatial coordinate information but also semantic information, forming an RGB-D point cloud with semantic information.

[0076] Step 3: Construct a 3D obstacle model, specifically including: (1) Clustering the RGB-D point cloud with semantic information using a point cloud clustering algorithm (such as the DBSCAN algorithm) to aggregate the point cloud points belonging to the same obstacle into a cluster. (2) Perform shape fitting on each cluster (such as using the least squares method to fit a plane, sphere, or cylinder, etc.) to construct a 3D obstacle model. The obstacle model contains the location, shape, size, and semantic information of the obstacle (such as pedestrians, vehicles, etc.). The 3D obstacle model provides geometric information of the obstacle, providing a basis for path planning and obstacle avoidance decisions.

[0077] Step 4: Input the constructed 3D obstacle model into the deep learning-based obstacle detection model for further detection to improve detection accuracy and robustness. During the vehicle's driving process, the point cloud data is updated in real time, and clustering, shape fitting, and obstacle detection are re-performed to achieve dynamic tracking of obstacles.

[0078] Through the above steps, the present invention can fully utilize the advantages of binocular cameras and lidar. During the vehicle's driving process, it can construct an accurate 3D obstacle model with semantic information in real time, which can reflect the dynamic changes of the surrounding environment. The 3D obstacle model is then used as the input of the obstacle detection model. Based on the information provided by the 3D obstacle model, the vehicle can determine whether obstacle avoidance measures need to be taken and determine the direction and distance of obstacle avoidance.

[0079] Positioning and map construction submodule 1024: Utilizes the point cloud data collected by the lidar, combined with the information from the binocular stereo camera, surround view camera, ultrasonic sensor, and infrared sensor, to construct a three-dimensional map containing a variety of information such as terrain, buildings, roads, and obstacles; the precise position and posture information of the vehicle obtained by the ultra-wideband positioning module and the inertial measurement unit 1013, as well as the wheel speed information obtained by the wheel speed meter 1015, are fused using the extended Kalman filter (EKF) and the posture is corrected in real time.

[0080] During the autonomous driving process, the system needs to collect environmental data in real time and continuously update the three-dimensional map. This includes using binocular cameras, lidar and other sensors to obtain images and point cloud data of the surrounding environment, and then processing and fusing them to build or update the three-dimensional map. The three-dimensional map provides important environmental information for path planning and obstacle avoidance decisions.

[0081] Next, obstacle avoidance and automatic path navigation are implemented based on an improved path planning algorithm. The improved path planning algorithm includes: (1) a hierarchical path planning method: it includes two stages: global planning and local optimization. The global planning stage uses the improved A* algorithm to search for a collision-free coarse path in the gridded collision space; the local optimization stage is based on the dynamic programming (DP) to minimize the cost function and locally optimize the path obtained by the global planning to make it conform to the kinematic constraints. (2) Path smoothing: non-uniform rational B-splines (NURBS) are used to smooth the optimized path to generate a short and smooth executable path.

[0082] During the path planning phase, the system uses road and obstacle information from the 3D map, combined with an improved path planning algorithm, to generate a collision-free driving path. During the obstacle avoidance phase, the system monitors the obstacle layer in the 3D map in real time, determines whether there is a potential collision risk, and makes appropriate obstacle avoidance decisions.

[0083] Collision detection and path planning submodule 1025: uses Minkowski approximation to calculate the collision space, and searches for a collision-free path in the collision space based on hierarchical path planning and uses non-uniform rational B-splines for path smoothing to achieve efficient and accurate automatic navigation.

[0084] Furthermore, the method of calculating the collision space using Minkowski approximation includes: a collision space modeling unit: using Minkowski and Construct a collision space and calculate the Minkowski difference between the current vehicle position and the obstacle in real time to obtain the shape of the obstacle in the collision space.

[0085] Among them, constructing the collision space specifically includes:

[0086] (1) Define the basic shape:

[0087] Let R be the vehicle shape, i.e., the area occupied by the vehicle, representing the shape of the vehicle itself, typically including the outline of the unmanned vehicle and any possible clearance areas. Let O be the obstacle shape, i.e., the obstacle boundary, representing the shape of the obstacle in the environment, where O is the position and outline of the obstacle in real space. The vehicle shape R and the obstacle shape O can be represented by polygons or more complex geometric shapes. Preferably, in this embodiment, the unmanned vehicle R is a circle with a radius of r, and the obstacle O is a polygon.

[0088] (2) Calculate the Minkowski sum

[0089] The Minkowski sum is used to expand the obstacle boundary to construct the collision space:

[0090] set up

[0091] Where p is a point belonging to O, that is, p is any point on the boundary of the obstacle. q is a point belonging to R, that is, q is any point in the area occupied by the vehicle; p+q represents vector addition, which is used to determine the boundary of the collision space. By calculating We get a region containing all possible collision locations, the collision space. When , each vertex of the obstacle is extended outward by a distance of r to form a new polygon, which is the boundary of the collision space.

[0092] Based on Minkowski difference Constructing a dynamic collision space using Minkowski difference Collision detection specifically includes:

[0093] Minkowski difference is used to determine whether two shapes intersect. Collision space obstacle C obs Minkowski difference calculate:

[0094]

[0095] C obs Represents the collision space obstacle, which represents the area in the collision space that the unmanned vehicle cannot reach. That is, the vehicle cannot enter this area when performing path planning. The calculation result of this area comes from the Minkowski difference between the obstacle shape and the vehicle shape. pq is the vector difference between point p in O and point q in R. By calculating the combination of all such differences, we can get the Minkowski difference, that is, calculate the shape relationship between O and R and get the shape of the obstacle in the collision space.

[0096] During the path planning process, the Minkowski difference between the current vehicle position and the obstacle is calculated in real time. obs If the origin (i.e., zero vector) is included, it means that the vehicle has collided with an obstacle, triggering a collision warning or emergency stop mechanism.

[0097] In specific implementation, the shortest distance from the center point of the vehicle to the obstacle boundary can be calculated and compared with the radius of the vehicle to determine whether a collision occurs.

[0098] When the vehicle shape is circular (radius r), the collision space obstacle can be calculated by expansion, that is, the expansion relationship between the vehicle geometry (circular radius r) and the obstacle O is modeled as:

[0099]

[0100] Where ||pq|| is the Euclidean distance between p and q, representing the distance from p to q. R is the vehicle radius, which refers to the size of the vehicle, specifically the radius of the space it occupies. It represents the impact of the vehicle on the expansion of obstacles in the collision space. It means that for each point p, find the nearest point q in the obstacle O and calculate the distance from p to q. O It is defined as the set of all points close to the obstacle O, and the area with a distance less than or equal to the vehicle radius r, which represents the expansion area of the obstacle in the collision space. The vehicle cannot enter these areas.

[0101] In this way, the vehicle only needs to plan along the non-collision path without worrying about its own shape. And it updates C in real time based on the environment point cloud. obs , supports dynamic obstacle tracking.

[0102] Next, after constructing the collision space, a collision-free path is searched in the collision space based on hierarchical path planning. During the path planning process, the kinematic and dynamic constraints of the vehicle are taken into account to ensure the feasibility and smoothness of the path.

[0103] Specifically, the collision detection and path planning submodule 1025 searches for a collision-free path in the collision space based on hierarchical path planning and uses non-uniform rational B-splines to perform path smoothing, including:

[0104] In the embodiment of the present invention, hierarchical path planning includes two stages: global planning and local optimization, and path smoothing after local optimization. Specifically, it is implemented by the following three units of the collision detection and path planning submodule 1025, such as Figure 5 Shown, including:

[0105] Global planning unit 10251: used to search for a collision-free rough path from the starting point to the target point in the global range of the rasterized collision space using the improved A* algorithm, including:

[0106] The constructed collision space is discretized into a grid map, where each grid represents an area, and the state of the grid cell can be idle (for example, represented by 0), obstacle (for example, represented by 1), or boundary (other values are defined according to specific needs).

[0107] The improved A* algorithm is used to introduce an optimization strategy based on the traditional A* algorithm to search for a collision-free rough path in the gridded collision space. The actual cost of the node and the estimated value of the heuristic function are comprehensively considered to obtain the shortest path from the start node to the target node as the collision-free rough path. Specifically, it includes:

[0108] Initialization: Create an open list and a closed list to track cells that have been visited and those to be visited. Initialize the starting grid, add it to the open list, and set its cost.

[0109] Search process: (1) Select the grid cell with the minimum cost function (f value) from the open list. (2) Remove the grid cell from the open list and add it to the closed list. (3) Check all neighbor grids of the grid cell: If the neighbor grid is in the closed list, ignore it; if the neighbor grid is not in the open list, calculate its cost function and add it to the open list; if the neighbor grid is already in the open list, check whether the path to it is better by changing the current grid. (4) Update the predecessor node and cost value of the neighbor grid.

[0110] Cost function: f(n) = g(n) + h(n)

[0111] Where f(n) is the total cost of node n, g(n) is the actual cost from the starting point to node n, and h(n) is the heuristic estimate of the distance from node n to the target node. Choose an appropriate heuristic function based on your needs, such as Euclidean distance or Manhattan distance. To improve search efficiency, you can optimize the heuristic function, such as by introducing a weight coefficient.

[0112] Preferably, the optimization strategies include: (1) Jump Point Search (JPS): Jump Point Search detects obstruction points on a straight path and directly skips these points, thereby reducing the search space. In a grid map, if it is found that the neighboring nodes of a node are all obstacles or boundaries, these neighboring nodes can be skipped and the next possible path can be directly searched. (2) Subnode Optimization Selection: According to the complexity of the grid map and the motion characteristics of the robot, a suitable subnode selection method is selected. For example, in a complex off-road environment, a 16-adjacency method can be selected to allow the robot more freedom of movement to avoid danger. The subnode selection rules are optimized to avoid collisions between the robot and obstacles. (3) Direction Change Penalty: A direction change penalty is introduced to reduce useless turning points in the path and make the path smoother. According to the robot's turning angle and driving speed, the impact of direction change on vehicle driving is calculated and converted into an increase in driving distance. (4) Local Area Complexity Penalty: According to the complexity of the local area, the node search space is adaptively adjusted. In areas with more obstacles and threatening objects, the search range is expanded to find a better path; in relatively open areas, the search range is narrowed to improve efficiency.

[0113] Local optimization unit 10252: Minimize the cost function J(x based on dynamic programming (DP) k ,u k ), locally optimize the collision-free rough path obtained by the global planning unit to make it conform to the kinematic constraints and obtain the optimized shortest path, including:

[0114] Assume that the gridded collision space is G(x, y), the starting point S(x s ,y s ), target point T(x t ,y t ), cost function J(x k ,u k ):

[0115]

[0116] Among them, J(x k ,u k ) indicates that from the current state (i.e. current position) x k Start, take control action uk After that, the minimum cumulative path cost to reach the target state; g(x k ,u k ) indicates that from the current state x k Start, take control action u k the immediate cost; Indicates that from the next state (ie the next position) x k+1 Start and take all possible control actions u k+1 The minimum cumulative path cost in u k 、u k+1 To control actions (such as acceleration, deceleration, etc.);

[0117] The path constraint or state transfer equation is:

[0118] x k+1 =f(x k ,u k )

[0119] Among them, f(x k ,u k ) is the state transfer function, which determines the vehicle's current state x k Take control action u k After reaching the next state x k+1 The final solution:

[0120]

[0121] Among them, π* is the optimal path strategy and N is the path length.

[0122] Through the above x k+1 =f(x k ,u k ) recursively calculates the minimum cumulative path cost in each state, and finally finds the optimized shortest path from the starting point to the target point.

[0123] Path smoothing unit 10253: uses non-uniform rational B-splines (NURBS) to smooth the shortest path obtained by the local optimization unit to generate a path curve C(t). The control point weights are dynamically adjusted according to the kinematic constraints, and ultimately a short and smooth executable path is obtained.

[0124] After the path planning is completed, the path may be not smooth. Therefore, the present invention sets a smoothing function C(t) to smooth the path so that the path is smoother and meets the kinematic and dynamic constraints. The smoothing function C(t) is composed of the control point P i , basis function N i,p (t) and weight w i Composition, defined as follows:

[0125]

[0126] Among them, w i is the weight of the control point (can be used to adjust the path smoothness); t is the path parameter, t∈[0,1]; p is the order of the basis function, which determines the depth of the recursion; n is the maximum index number of the control point. i，p (t) is the p-order basis function, defined as follows (recursively):

[0127]

[0128] Among them, t i , t i+1 , t i+p , t i+p+1 : The nodes in the node sequence determine the domain of each basis function.

[0129] N i，0 (t): Basis function of order 0, which is a piecewise constant function.

[0130] N i，p-1 (t), N i+1，p-1 (t): The lower order (p-1 order) basis functions in the recursive definition, which are used to calculate the current order basis function N i，p The values of (t) are recursively relied upon to construct higher-order basis functions.

[0131] Weighting coefficient, which represents the current parameter t relative to node t i to t i+p It controls the influence weight of the recursive basis function between the current node and the next node.

[0132] Weighting coefficient, which represents the current parameter t relative to node t i+1 to t i+p+1 The position between them is used to weight the basis function N of the second part i+1，p-1 (t).

[0133] Through the above-mentioned path planning technology, the efficiency, smoothness, adaptability to complex environments and efficiency of path planning are achieved, thereby improving the overall performance and safety of the autonomous driving system. As a result, the vehicle can achieve positioning, autonomous identification and obstacle avoidance and navigation functions in closed or semi-closed areas.

[0134] The cloud management platform 103 is used to monitor the status of the vehicle in real time. By storing navigation data and vehicle health information in the cloud, the vehicle can be remotely controlled and the status of the vehicle can be monitored to the remote terminal in real time, and it supports multi-vehicle collaborative scheduling; by storing navigation data, vehicle health information, etc. in the cloud, it provides data support and management services for the operation of the vehicle, facilitating the unified scheduling and management of multiple unmanned vehicles.

[0135] The voice interaction module 104 supports voice command input, including navigation, control, query, and emergency commands. It is used to interpret user voice commands and dynamically respond to them, and replan the route using the route planning module. For example, in some embodiments, multiple types of voice commands, including but not limited to, go forward, turn left, turn right, stop, accelerate, decelerate, query current status, battery level, system health, and emergency stop, etc., are received, and the system executes the corresponding operation or provides voice feedback based on the command.

[0136] When a vehicle is used to carry people, human control is added to facilitate user experience. Taking a small passenger vehicle as an example, a table showing the correspondence between user voice commands and vehicle responses is set up, as shown in Table 1. Users can control the vehicle's state through voice commands, facilitating interaction between the user and the vehicle.

[0137] Table 1 Comparison table of user voice commands and vehicle responses

[0138]

[0139]

[0140] According to the above-described embodiments of the present invention, positioning, obstacle avoidance, and navigation are achieved through the fusion of multiple sensors, including vision and lidar. This system design not only meets the needs of small-scale passenger or logistics transportation, but also provides efficient, accurate, and safe autonomous navigation capabilities. Furthermore, it enables remote control, unified scheduling and management of multiple vehicles, and supports various types of voice command input, providing users with more comfortable and efficient transportation services and an enhanced user experience.

[0141] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved. This is not limited herein.

[0142] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An unmanned intelligent driving system for small vehicles used for manned and logistics transportation, characterized in that: include: Sensor module, path planning module, cloud management platform and voice interaction module; The sensor module is used to collect information about the vehicle itself and its environment; The path planning module performs target detection, road identification, obstacle detection, positioning, and map construction based on the vehicle and its environment information and using a multimodal sensor fusion method, and implements obstacle avoidance and automatic path navigation based on an improved path planning algorithm; The cloud management platform is used to monitor vehicle status in real time, store navigation data and vehicle health information in the cloud, and support multi-vehicle coordinated scheduling; The voice interaction module supports voice command input, including navigation, control, query and emergency commands, and is used to parse user voice commands and respond dynamically, and replan the route using the route planning module.

2. The unmanned intelligent driving system according to claim 1, characterized in that: in, The sensor module includes: Binocular stereo cameras and surround-view cameras are used to obtain depth information of the environment and provide field of view. The two work together to provide rich visual perception information for the vehicle. LiDAR / millimeter-wave radar, used to collect distance data of the vehicle; The ultra-wideband positioning module and inertial measurement unit use extended Kalman filtering to achieve centimeter-level positioning and measure the vehicle's attitude and velocity information in real time; Ultrasonic sensors and infrared sensors are used for low-speed, close-range obstacle avoidance and redundant detection in low-light environments, respectively; Wheel speed meter provides accurate wheel speed information for calculating vehicle speed, direction, mileage and status monitoring.

3. The unmanned intelligent driving system according to claim 2, characterized in that: in, The path planning module uses multimodal sensor fusion methods to perform target detection, road recognition, and obstacle detection based on the vehicle itself and its environment information, including: The target detection submodule fuses the data features of the binocular stereo camera, surround-view camera, and lidar / millimeter-wave radar, and then inputs them into the improved target perception algorithm YOLOv5 model for target detection, so as to quickly and accurately identify target objects including pedestrians, vehicles, and obstacles. Road detection submodule: Combines visual segmentation technology to extract the drivable area from the camera image and processes the LiDAR point cloud data to extract the road edge and height information for data fusion. The fused features are then input into the road recognition model for road recognition. Obstacle Detection Submodule: A joint calibration algorithm is used to establish a coordinate mapping relationship between the binocular camera and the lidar. The RGB-D point cloud is then fused with visual semantic information to construct a 3D obstacle model, which more accurately represents the position, shape, and size of obstacles. This model then serves as an obstacle layer in the three-dimensional map. The fused features are then input into a deep learning-based obstacle detection model for obstacle detection. Ultrasonic-based obstacle avoidance is enabled at low speeds of less than 1 m / s, and infrared sensors are used at night / in low light to ensure safety at all times.

4. The unmanned intelligent driving system according to claim 3, characterized in that: in, The path planning module also includes: Positioning and Mapping Submodule: This module uses point cloud data collected by the LiDAR, combined with information from binocular stereo cameras, surround-view cameras, ultrasonic sensors, and infrared sensors, to construct a three-dimensional map containing a variety of information such as terrain, buildings, roads, and obstacles. The precise position and posture information of the vehicle obtained by the ultra-wideband positioning module and the inertial measurement unit, as well as the wheel speed information obtained by the wheel speed meter, are fused using an extended Kalman filter (EKF) to correct the posture in real time.

5. The unmanned intelligent driving system according to claim 1, characterized in that: in, The path planning module implements obstacle avoidance and automatic path navigation based on an improved path planning algorithm, including: Collision detection and path planning submodule: uses Minkowski approximation to calculate the collision space, and uses hierarchical path planning to find a collision-free path in the collision space and use non-uniform rational B-splines for path smoothing to achieve efficient and accurate automatic navigation.

6. The unmanned intelligent driving system according to claim 5, characterized in that: in, The collision detection and path planning submodule uses Minkowski approximation to calculate the collision space, including: Collision Space Modeling Unit: Using Minkowski and Construct a collision space and calculate the Minkowski difference between the current vehicle position and the obstacle in real time to obtain the shape of the obstacle in the collision space.

7. The unmanned intelligent driving system according to claim 5 or 6, characterized in that: in, The collision detection and path planning submodule searches for collision-free paths in the collision space based on hierarchical path planning and uses non-uniform rational B-splines for path smoothing, including: Global planning unit: used to search for a collision-free rough path from the starting point to the target point in the global range of the rasterized collision space using the improved A* algorithm; Local optimization unit: Based on the dynamic programming (DP) minimization cost function, the collision-free rough path obtained by the global planning unit is locally optimized to make it conform to the kinematic constraints and obtain the optimized shortest path; Path smoothing unit: Non-uniform rational B-splines (NURBS) are used to smooth the shortest path obtained by the local optimization unit to generate a path curve C(t). The control point weights are dynamically adjusted according to the kinematic constraints, and ultimately a short and smooth executable path is obtained.

8. The unmanned intelligent driving system according to claim 7, characterized in that: in, The global planning unit uses the improved A* algorithm to search for collision-free rough paths in the global range of the gridded collision space, including: Discretize the constructed collision space into a grid map, where each grid represents an area and the state of the grid is free, obstacle or boundary; The improved A* algorithm is used, which introduces an optimization strategy based on the traditional A* algorithm. A collision-free coarse path is searched in the gridded collision space. The actual cost of the node and the estimated value of the heuristic function are comprehensively considered to obtain the shortest path from the start node to the target node as the collision-free coarse path.

9. The unmanned intelligent driving system according to claim 7 or 8, characterized in that: in, The local optimization unit performs local optimization on the collision-free rough path obtained by the global planning unit based on the dynamic programming (DP) minimization cost function to make it conform to the kinematic constraints. The optimized shortest path includes: Assume that the gridded collision space is G(x, y), the starting point S(x s ,y s ), target point T(x t ,y t ), cost function J(x k ,u k ): Among them, J(x k ,u k ) indicates that from the current state x k Start, take control action u k After that, the minimum cumulative path cost to reach the target state; g(x k ,u k ) indicates that from the current state x k Start, take control action u k the immediate cost; Indicates that from the next state x k+1 Start and take all possible control actions u k+1 The minimum cumulative path cost in; the path constraint or state transfer equation is: x k+1 =f(x k ,u k ) Among them, f(x k ,u k ) is the state transfer function, which determines the vehicle's current state x k Take control action u k After reaching the next state x k+1 The final solution: Among them, π * is the optimal path strategy, N is the path length; Through the above x k+1 =f(x k ,u k ) recursively calculates the minimum cumulative path cost in each state, and finally finds the optimized shortest path from the starting point to the target point.

10. A small vehicle for carrying people and logistics, comprising the unmanned intelligent driving system according to any one of claims 1 to 9.

Images