Safety control method and device for unmanned vehicle

Abstract

The invention relates to the technical field of unmanned vehicles, and discloses a safety control method for an unmanned vehicle, and the method comprises the steps: carrying out the time and space fusion of multi-source real-time data through employing Kalman filtering, and generating environment perception result data; recognizing and marking obstacles in the surrounding environment of the vehicle through an edge detection algorithm based on a road structure; using the optimized long-short-term memory network to predict the moving path of the dynamic obstacle in the future preset time; and generating an optimal driving route according to the moving path information in combination with an improved Dijkstra algorithm. The problems that in the prior art, the obstacle recognition and response speed is low in the complex traffic environment, and especially effective decision making cannot be conducted under the condition that the environment changes frequently are solved. According to the method, the multi-source data are fused and analyzed through the improved deep learning model, and rapid response to dynamic obstacles is realized, so that potential collision is avoided, and the driving safety and the decision-making efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned vehicle, and in particular to a safety control method, device, equipment and computer program product for an unmanned vehicle. Background Art

[0002] At present, unmanned driving technology has been initially applied in urban transportation, logistics and other fields, but there are still deficiencies in safety control in complex dynamic traffic environments. The existing technology has limited processing capabilities for multi-source sensor data and cannot quickly and accurately fuse various sensor data, resulting in insufficient response in dynamic scenes, such as pedestrians crossing the road and vehicles suddenly changing lanes. Especially in traffic-intensive areas, the accuracy of obstacle prediction and path planning is low, which cannot fully meet the needs of unmanned vehicles for real-time decision-making and safe driving in complex scenarios. Therefore, there is an urgent need for a method that can achieve efficient and stable driving path planning under the conditions of multi-source data fusion and diversified dynamic obstacle prediction, so as to improve the safety and reliability of unmanned vehicles in complex traffic environments. Summary of the invention

[0003] In view of the above-mentioned technical deficiencies, the main purpose of the present invention is to provide a safety control method, device, equipment and computer program product for unmanned vehicles, aiming to solve the technical problems that the prior art has limited processing capabilities for multi-source sensor data and is unable to quickly and accurately fuse various types of sensor data, resulting in insufficient response in dynamic scenarios, such as pedestrians crossing the road and vehicles suddenly changing lanes. Especially in traffic-dense areas, the accuracy of obstacle prediction and path planning is low, and the needs of real-time decision-making and safe driving of unmanned vehicles in complex scenarios cannot be fully met.

[0004] To achieve the above object, the present invention provides a safety control method for an unmanned vehicle, wherein the safety control method for an unmanned vehicle comprises:

[0005] Preferably, the method comprises:

[0006] Step S10: Based on the multi-source real-time data collected by the multi-source sensors, Kalman filtering is used to perform temporal and spatial fusion on the multi-source real-time data, thereby forming a three-dimensional model of the vehicle's surrounding environment and generating environmental perception result data;

[0007] Step S20: using a convolutional layer in a convolutional neural network to perform a local scan on the environmental perception result data, extracting road edge feature data, road texture feature data, and object shape feature data in the environmental perception result data, and classifying the extracted features through a classifier to obtain information about objects around the vehicle, including pedestrian information, other vehicle information, and traffic sign information;

[0008] Step S30: using an edge detection algorithm based on road structure, combining vehicle driving trajectory information and road geometry information to detect and mark key road information, including boundary information, lane line information, and lane width information;

[0009] Step S40: comprehensively judging and confirming obstacle information based on the vehicle surrounding object information and road key information, combining the vehicle's driving speed data, driving direction data, relative speed data and relative acceleration data with the obstacle, and using an optimized long short-term memory network to predict the movement path of the obstacle within a preset time in the future;

[0010] Step S50: Generate an optimal driving route based on the moving path information and the improved Dijkstra algorithm.

[0011] Preferably, in step S30, the step of using the optimized long short-term memory network to predict the movement path of the obstacle within a preset future time includes:

[0012] Step S301: The vehicle speed data v vehicle (t), driving direction data θ vehicle (t), relative speed data v between the obstacle and the vehicle relative (t) and the relative acceleration data a between the obstacle and the vehicle relative (t) Input into the optimized long short-term memory network;

[0013] Step S302: In the optimized long short-term memory network, v in step S301 is converted to vehicle (t),θ vehicle (t), v relative (t), a relative (t) as the input data x at each time step t , processed by the following formula:

[0014] The input gate layer controls the impact of input data on the current state: i t =σ(W i ·[h t-1 ,x t +b i ]),

[0015] The forget gate layer determines the degree to which previous information is forgotten: f t =σ(W f ·[h t-1 ,x t +b f ]),

[0016] The output gate layer controls the output of the current time step: o t =σ(Wo ·[h t-1 ,x t +b o ]),

[0017] The cell state update layer generates the current state by memorizing the cell input information:

[0018] C t =f t ·C t-1 +i t tanh(W c ·[h t-1 ,x t ]+b C ),

[0019] Among them, W i , W f , W o , W c represents the weight matrix of each layer, b i 、b f 、b o 、b C represents the bias term of each layer, σ is the sigmoid activation function, h t-1 is the hidden state output of the previous time step t-1, C t-1 is the cell state at the previous time step, and tanh is the hyperbolic tangent activation function;

[0020] Step S303: The output of the model is the prediction of the movement path of the obstacle within the preset time in the future, including the position (x out ,y out ) and speed v out , these outputs are calculated by the last layer of LSTM.

[0021] Preferably, in step S10, comprehensively judging and confirming obstacle information based on the vehicle surrounding object information and road key information includes:

[0022] Use road geometry information and vehicle trajectory to make association judgments;

[0023] When the objects around the vehicle are static and in the vehicle's driving track, they are judged to be obstacles;

[0024] When the objects around the vehicle are dynamic objects, they are judged as obstacles regardless of whether they are in the vehicle's driving trajectory.

[0025] Preferably, in step S30, the obstacle information includes the geometric shape, edge contour, material characteristics and spatial movement trend information of the obstacle.

[0026] Preferably, in step S40, the step of generating the optimal driving route according to the moving path information in combination with the improved Dijkstra algorithm includes dynamically adjusting node weights, updating multiple moving paths of obstacles in real time, and generating the optimal driving route.

[0027] Preferably, the step of dynamically adjusting the node weights is preferentially performed when the obstacle is less than a preset safety distance from the vehicle, and by increasing the weights of nodes near the obstacle, a driving route away from the obstacle is preferentially selected.

[0028] Preferably, the method further comprises automatically adjusting the speed and direction of the vehicle according to the optimal driving route.

[0029] Preferably, the safety control device comprises:

[0030] A data fusion module is used to fuse the multi-source real-time data collected by multi-source sensors in time and space using Kalman filtering to form a three-dimensional model of the vehicle's surrounding environment and generate environmental perception result data;

[0031] A feature extraction module, used to use a convolutional layer in a convolutional neural network to perform a local scan on the environmental perception result data, extract road edge feature data, road texture feature data, and object shape feature data in the environmental perception result data, and classify the extracted features through a classifier to obtain information about objects around the vehicle, including pedestrian information, other vehicle information, and traffic sign information;

[0032] The road information detection module is used to detect and mark key road information, including boundary information, lane line information, and lane width information, using an edge detection algorithm based on the road structure, combined with vehicle driving trajectory information and road geometry information;

[0033] The obstacle prediction module is used to comprehensively determine and confirm obstacle information based on the information of objects around the vehicle and key road information, and to use an optimized long short-term memory network to predict the movement path of obstacles within a preset time in the future by combining the vehicle's driving speed data, driving direction data, relative speed data and relative acceleration data with the obstacle;

[0034] The path planning module is used to generate an optimal driving route according to the moving path information combined with the improved Dijkstra algorithm.

[0035] Preferably, the safety control device for an unmanned vehicle comprises:

[0036] A memory, a processor, and a safety control program for an unmanned vehicle stored in the memory and executable on the processor, wherein the safety control program for an unmanned vehicle, when executed by the processor, implements a safety control method for an unmanned vehicle as described in any one of claims 1 to 7.

[0037] Preferably, the computer program product includes a safety control program for an unmanned vehicle, and when the safety control program for an unmanned vehicle is executed by a processor, the safety control method for an unmanned vehicle as described in any one of claims 1 to 7 is implemented.

[0038] The beneficial effects of the present invention are as follows: compared with the prior art, the processing capability of multi-source sensor data is limited, and various sensor data cannot be quickly and accurately integrated, resulting in untimely response in dynamic scenes, such as pedestrians crossing the road and vehicles suddenly changing lanes. Especially in traffic-dense areas, the accuracy of obstacle prediction and path planning is low, and the technical problem of being unable to fully meet the needs of unmanned vehicles for real-time decision-making and safe driving in complex scenes cannot be solved. Since the present invention integrates and analyzes multi-source data through an improved deep learning model, it achieves a rapid response to dynamic obstacles, thereby avoiding potential collisions and improving driving safety and decision-making efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 FIG. 1 is a flow chart of a first embodiment of a safety control method for an unmanned vehicle according to the present invention.

[0041] Figure 2 This is a diagram of the safety control device used for unmanned vehicles according to the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Embodiment 1: Figure 1FIG. 1 is a flow chart of a first embodiment of a safety control method for an unmanned vehicle according to the present invention.

[0044] Step S10: Based on the multi-source real-time data collected by the multi-source sensors, Kalman filtering is used to fuse the multi-source real-time data in time and space, thereby forming a three-dimensional model of the vehicle's surrounding environment and generating environmental perception result data.

[0045] It should be noted that multi-source sensors refer to various types of sensor devices installed on unmanned vehicles for real-time perception of the vehicle's surrounding environment, including but not limited to laser radar, camera, millimeter wave radar, ultrasonic sensor and GPS module. These sensors can provide different types of data, such as distance, image, speed, position, etc. Multi-source real-time data refers to multi-dimensional data collected synchronously by the above sensors during the vehicle's driving process, including laser radar data, camera data, millimeter wave radar data, ultrasonic sensor data and GPS data. Laser radar data provides distance and shape information of surrounding objects; camera data provides visual information such as roads, pedestrians, and vehicles; millimeter wave radar data is used to detect the relative speed and distance of obstacles in medium and long distances; ultrasonic sensor data is used to detect obstacles at close range; GPS data is used for precise positioning information of vehicles. Environmental perception result data refers to a complete three-dimensional information model of the vehicle's surrounding environment obtained by fusing multi-source real-time data through Kalman filtering. The model includes the location information of obstacles, the speed information of obstacles, the relative distance between the vehicle and obstacles, and road and traffic conditions, such as lane lines, traffic signals, road curves and other information.

[0046] It is understandable that Kalman filtering is a recursive algorithm based on dynamic systems, which can provide optimal state estimation in noisy multi-source data. By fusing data from different sensors in time and space dimensions, the system can generate a high-precision three-dimensional environmental perception result. This perception result can provide reliable basic data for subsequent obstacle recognition, path planning and other modules, thereby ensuring the safe driving of unmanned vehicles in complex dynamic environments.

[0047] It should be understood that in this embodiment, the Kalman filter algorithm used is optimized in the process of multi-source sensor data fusion. Compared with the traditional Kalman filter algorithm, the improvement of the present invention is that it combines multi-dimensional information of space and time, and enhances the processing ability of rapidly changing scenes. This improvement is particularly reflected in the response speed to dynamic obstacles and real-time traffic conditions. Traditional Kalman filtering is mainly used for linear systems, and its core formula is state estimation and covariance update:

[0048]

[0049] in, is the predicted state, F k is the state transition matrix, is the last state estimate, B k is the control input matrix, u k is the control input.

[0050]

[0051] Among them, P k|k-1 is the forecast error covariance, Q k is the process noise covariance matrix.

[0052] Improvements in the present invention: In order to cope with the complex dynamic environment of unmanned vehicles, an extended Kalman filter (EKF) algorithm in a nonlinear scenario is added, which can process nonlinear sensor data and dynamically adjust the noise covariance matrix Q according to the changes in the real-time environment k , in order to improve the estimation accuracy. The algorithm enhances real-time performance through the following improvements:

[0053] (1) To address the problem of data frequency asynchrony when the vehicle is driving at high speed, an adaptive noise model is added to dynamically adjust the covariance matrix during the prediction process to reduce the impact of sensor data noise on the results.

[0054] (2) Based on the traditional Kalman filter, spatial geometric information and time series information are integrated, and an improved state transition model is used:

[0055]

[0056] Among them, G k It is a time-dependent noise model, and t is the time factor, which enables the system to better handle complex and rapidly changing dynamic scenes.

[0057] For example, suppose an autonomous vehicle is driving on a city road. Sensors collect data from multiple sources, including lidar, cameras, and millimeter-wave radar. At this time, a bicycle suddenly accelerates across the road in front, while another car approaches slowly. The autonomous vehicle needs to predict these dynamic obstacles and adjust its driving path to avoid collisions.

[0058]

[0059] The improved Kalman filter processing process can be:

[0060] (1) State vector initialization: Set the initial state vector x0 to contain the position and velocity information of the obstacle: x0 = [position x, position y, velocity x, velocity y] = [25, 5, 2, 0], and the initial covariance matrix P0:

[0061]

[0062] (2) Kalman filter prediction steps

[0063] Based on the data from the previous moment, predict the position and speed of the bicycle at the next moment.

[0064] State transfer equation:

[0065] Assuming the time step Δt = 1 second, the state transfer matrix F k for:

[0066]

[0067] The control input matrix Bk is:

[0068] Preliminary forecast:

[0069] (3) Covariance matrix update

[0070]

[0071] Assume that the process noise Q k for:

[0072] Then the new covariance matrix is:

[0073] (4) Adaptive noise adjustment

[0074] When a bicycle is detected accelerating, the process noise matrix is ​​adjusted in real time to make it more sensitive to fast-changing dynamic obstacles. For example:

[0075]

[0076] This allows the acceleration changes of obstacles to be captured more accurately.

[0077] (5) Measurement update

[0078] Measurement updates are performed using mmWave radar and lidar data. The measurement vector is:

[0079]

[0080] Update the equation by measuring:

[0081]

[0082] Among them, H k is the measurement matrix, Kk is the Kalman gain. Finally, we get the accurate position and speed prediction of the bicycle.

[0083] Step S20: using a convolutional layer in a convolutional neural network to perform a local scan on the environmental perception result data, extracting road edge feature data, road texture feature data, and object shape feature data in the environmental perception result data, and classifying the extracted features through a classifier to obtain information about objects around the vehicle, including pedestrian information, other vehicle information, and traffic sign information;

[0084] Step S30: using an edge detection algorithm based on road structure, combining vehicle driving trajectory information and road geometry information to detect and mark key road information, including boundary information, lane line information, and lane width information;

[0085] It should be noted that convolutional neural network refers to a deep learning model that is particularly suitable for image processing tasks. It automatically extracts features from input data through the stacking of multiple convolutional layers, pooling layers and fully connected layers. Convolutional neural networks mainly include convolutional layers (for local feature extraction), pooling layers (for dimensionality reduction and feature selection) and fully connected layers (for classification or regression output). In unmanned driving environments, convolutional neural networks are used to extract key features such as roads, vehicles, pedestrians, etc. from images and point cloud data from sensors such as cameras and lidar. The key feature data in the environmental perception result data refers to the important features that affect vehicle driving decisions extracted after being processed by the convolutional neural network. These features include but are not limited to the geometry of road edges, the spatial position of obstacles, the size and speed of obstacles, lane lines, traffic lights, pedestrian positions and vehicle driving trajectories. These data can accurately reflect the environmental status around the vehicle and support path planning and obstacle avoidance decisions. The edge detection algorithm based on road structure refers to an image processing algorithm that uses road geometry and structural information to help vehicles identify obstacles in the surrounding environment by identifying the characteristic edges of the road. Including but not limited to Sobel operator, Canny edge detection algorithm, etc., these algorithms can effectively detect road edges, lane lines, roadblocks, etc. by combining the geometric characteristics of the road and the direction of vehicle travel. Obstacles in the vehicle's surrounding environment refer to any objects or structures that may affect the normal driving of the unmanned vehicle. Including but not limited to other vehicles, pedestrians, bicycles, stop signs, traffic cones, fixed obstacles (such as roadblocks, guardrails), and dynamically moving objects (such as balls, animals, etc.).

[0086] It is understandable that convolutional neural networks can automatically extract key feature data in complex environments and provide efficient environmental perception capabilities for unmanned vehicles. At the same time, the edge detection algorithm based on road structure combined with the features extracted by convolutional neural networks can accurately identify the edges of obstacles and provide accurate obstacle annotation results in complex traffic environments.

[0087] It should be understood that the convolution kernel of traditional convolutional neural networks usually remains static throughout the training process, making it difficult to cope with complex and changing road environments. To this end, a method of adaptive convolution kernels is proposed, which enables the convolution kernel to dynamically adjust its size and shape according to the characteristics of the input data. For example, in a highway environment, the convolution kernel will expand to capture lane information in a large range, while in urban roads, the convolution kernel will shrink to accurately detect pedestrians and small obstacles. This improvement increases the flexibility and adaptability of convolutional neural networks and effectively improves the feature extraction accuracy for diverse scenarios. For example, in urban street data containing pedestrians and vehicles, traditional CNN can only recognize the outline of large vehicles under a fixed convolution kernel, missing some small pedestrians, while after using an adaptive convolution kernel, it can simultaneously recognize vehicles and pedestrians, reducing the missed detection rate by 30%. In addition, in unmanned driving scenarios, obstacles and road information have features of different scales, and traditional convolutional neural networks are usually unable to capture near-view and far-view features at the same time. In response to this, a multi-scale feature fusion mechanism is designed to extract features at different scales and fuse them through subsequent layers of the network. This technology can simultaneously detect small objects in the distance, such as traffic signs, and large objects in the near distance, such as vehicles, improving the model's ability to understand the global scene. For example, when processing road scenes with distant traffic lights and close pedestrians, traditional CNN can only accurately identify pedestrians in the near distance, while the detection accuracy of distant traffic lights is low. After multi-scale feature fusion improvement, the detection accuracy of traffic lights and pedestrians has increased by 20% and 15% respectively. Furthermore, traditional edge detection algorithms such as Sobel operator and Canny edge detection algorithm mainly rely on image grayscale changes and are easily affected by complex road environments. To address this problem, a dynamic edge detection algorithm combining road geometry and vehicle driving trajectory is introduced. This method can adapt to the geometric changes of the road when detecting road edges and lane lines, and can automatically adjust the sensitivity of edge detection when the vehicle makes a sharp turn or enters a complex road section to ensure the accuracy of edge detection. For example, in a sharp turn scene, the traditional Canny edge detection method has difficulty in accurately identifying the guardrails and traffic signs on the inside of the curve, and the edge detection accuracy is 65%. By introducing dynamic edge detection optimization of road structure, the detection accuracy is improved to 90%, effectively reducing the phenomenon of missed detection.

[0088] Step S40: comprehensively judging and confirming obstacle information based on the vehicle surrounding object information and road key information, combining the vehicle's driving speed data, driving direction data, relative speed data and relative acceleration data with the obstacle, and using an optimized long short-term memory network to predict the movement path of the obstacle within a preset time in the future;

[0089] It should be noted that the vehicle's current speed can reflect its state of motion and provide important time series information for the model; the driving direction and speed together determine the vehicle's motion trajectory, which has an important impact on predicting the relative path of obstacles; the relative speed data to the obstacle reflects the speed difference between the vehicle and the obstacle, which helps to capture the relative motion characteristics between the two; the relative acceleration data represents the acceleration change of the relative motion, which helps predict whether the obstacle accelerates, decelerates or maintains a constant speed.

[0090] It is understandable that in this implementation step, the vehicle speed, driving direction, relative speed and relative acceleration data with obstacles are combined, and these parameters are crucial for the path prediction of dynamic obstacles. Traditional dynamic obstacle prediction models usually only consider simple speed data, while ignoring the combined effects of the relative acceleration of the vehicle and the obstacle and the direction of movement. By introducing these additional features, the present invention optimizes the long short-term memory network (LSTM) model, so that the movement path of obstacles in the future can be more accurately predicted, especially in complex traffic environments.

[0091] It should be understood that compared to the traditional model that only predicts based on vehicle speed, the present invention introduces multi-dimensional data of vehicle speed, driving direction, relative speed and relative acceleration for time series prediction. These data interact with each other to better capture the relative motion changes between the vehicle and the obstacle. For example, when a vehicle approaches an obstacle quickly, combined with the analysis of relative speed and acceleration, it can accurately determine whether the obstacle will suddenly turn or accelerate, so as to plan the avoidance route in advance.

[0092] Assume that the current driverless vehicle is traveling at a speed of 80 km / h, and there is a bicycle 50 meters ahead moving at a speed of 15 km / h. The camera and radar data capture that the bicycle is accelerating, and the vehicle is approaching a curve, vehicle speed: 80 km / h, vehicle direction: curve to the right (30 degrees), relative speed with the bicycle: 65 km / h, relative acceleration: 2 m / s2 (the bicycle is accelerating), in this case, the LSTM model will comprehensively analyze the vehicle speed and direction, the relative speed and acceleration of the bicycle, and combine the time series data of the past few seconds to predict the movement trajectory of the bicycle in the next 5 seconds. The prediction result output by the model is that the bicycle will continue to accelerate into the main lane in the next 3 seconds, and then the speed will remain stable.

[0093] Step S50: Generate an optimal driving route based on the moving path information and the improved Dijkstra algorithm.

[0094] It should be noted that the traditional Dijkstra algorithm is used to find the shortest path from a starting point to all other points in a graph, and is often used in path planning for unmanned vehicles. However, when dealing with dynamic traffic environments, the traditional algorithm does not have the ability to update the position of obstacles and the status of vehicles in real time, which may lead to inflexible path planning and inability to adapt to rapid changes in traffic. In order to cope with the complex road conditions of unmanned vehicles, the present invention optimizes the Dijkstra algorithm so that it can perform real-time path adjustments in combination with the moving path information of dynamic obstacles. Specific improvements include:

[0095] (1) Based on the predicted obstacle path and the real-time status of the vehicle (such as speed, direction, etc.), the improved Dijkstra algorithm can dynamically adjust the weights of each node in the graph. In particular, when an obstacle is predicted to enter a certain path, the algorithm will increase the weight of the relevant nodes, giving priority to avoiding high-risk areas to ensure safe driving.

[0096] (2) The vehicle status affects the weight distribution, and information such as the vehicle’s speed, direction, and current energy consumption is added to the weight calculation. For example, when the vehicle is moving faster, the algorithm will tend to choose a straighter path, while in complex curves or congested areas, the algorithm will choose a safer route based on the vehicle status and road conditions.

[0097] (3) Real-time update mechanism: The improved Dijkstra algorithm introduces a real-time update mechanism that can dynamically adjust the route as the obstacle movement path changes. By combining with the LSTM network, the algorithm can predict the obstacle path in the next few seconds and prejudge possible dangerous areas, thereby planning a new optimal route in advance to ensure that the vehicle reaches its destination safely and efficiently.

[0098] For example, suppose an unmanned vehicle is driving on a city road. The traffic situation ahead is complex, with several vehicles driving in different directions. Based on the predicted moving path of the obstacle, the improvement process of the Dijkstra algorithm is as follows:

[0099] (1) Obstacle prediction path: Through the prediction of the LSTM network in step S30, the system determines that vehicle A in front will change lanes to the left, vehicle B will turn right 50 meters ahead, and a pedestrian is crossing the road.

[0100] (2) Node weight adjustment: Since both vehicles A and B pose a potential threat to the current vehicle, the algorithm will adjust the weights of the relevant nodes in the driving route:

[0101] The node weight of vehicle A’s lane change area increases;

[0102] The node weight of vehicle B's right-turn area increases;

[0103] The node weights in the areas where pedestrians are located are greatly increased, and pedestrians are given priority.

[0104] (3) Generate the optimal driving route. The improved Dijkstra algorithm combines these weight information and recalculates the optimal driving route. The results show that the optimal path is to drive slightly to the left, avoiding the position of vehicle A and pedestrians, while ensuring the vehicle's driving speed.

[0105] In addition, the present invention also provides a safety control device for unmanned vehicles, please refer to Figure 2 , the safety control device for an unmanned vehicle comprises:

[0106] A feature extraction module, which uses a convolutional neural network to extract key feature data from the environmental perception result data, and identifies and marks obstacles in the vehicle's surrounding environment through an edge detection algorithm based on the road structure;

[0107] The path prediction module is used to combine the vehicle speed data, driving direction data, relative speed data and relative acceleration data with the obstacle, and use the optimized long short-term memory network to predict the moving path of dynamic obstacles within a preset time in the future;

[0108] The path planning module is used to generate an optimal driving route according to the moving path information combined with the improved Dijkstra algorithm.

[0109] The safety control device for an unmanned vehicle provided by the present invention adopts a safety control method for an unmanned vehicle in the above embodiment, and can solve a technical problem of safety control for an unmanned vehicle. Compared with the prior art, the beneficial effects of the safety control device for an unmanned vehicle provided by the present invention are the same as the beneficial effects of the safety control method for an unmanned vehicle provided by the above embodiment, and other technical features in the safety control device for an unmanned vehicle are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0110] The present invention provides a safety control device for an unmanned vehicle, and the safety control device for an unmanned vehicle includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a safety control method for an unmanned vehicle in the above-mentioned embodiment 1. The safety control device for an unmanned vehicle in the embodiment of the present invention may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. A safety control device for an unmanned vehicle is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. A safety control device for an unmanned vehicle may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of a safety control device for an unmanned vehicle are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow a safety control device for an unmanned vehicle to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a safety control device for an unmanned vehicle with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0111] The present invention also provides a computer program product, including a computer program, which implements the steps of a safety control method for an unmanned vehicle as described above when executed by a processor. The computer program product provided by the present invention can solve a technical problem of safety control for an unmanned vehicle. Compared with the prior art, the beneficial effects of the computer program product provided by the present invention are the same as the beneficial effects of the safety control method for an unmanned vehicle provided by the above embodiment, and will not be repeated here.

[0112] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present invention are executed.

[0113] It should be understood that the various parts disclosed in the present invention can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A safety control method for an unmanned vehicle, characterized in that: The method comprises: Step S10: Based on the multi-source real-time data collected by the multi-source sensors, Kalman filtering is used to perform temporal and spatial fusion on the multi-source real-time data, thereby forming a three-dimensional model of the vehicle's surrounding environment and generating environmental perception result data; Step S20: using a convolutional layer in a convolutional neural network to perform a local scan on the environmental perception result data, extracting road edge feature data, road texture feature data, and object shape feature data in the environmental perception result data, and classifying the extracted features through a classifier to obtain information about objects around the vehicle, including pedestrian information, other vehicle information, and traffic sign information; Step S30: using an edge detection algorithm based on road structure, combining vehicle driving trajectory information and road geometry information to detect and mark key road information, including boundary information, lane line information, and lane width information; Step S40: comprehensively judging and confirming obstacle information based on the vehicle surrounding object information and road key information, combining the vehicle's driving speed data, driving direction data, relative speed data and relative acceleration data with the obstacle, and using an optimized long short-term memory network to predict the movement path of the obstacle within a preset time in the future; Step S50: Generate an optimal driving route based on the moving path information and the improved Dijkstra algorithm.

2. A safety control method for an unmanned vehicle according to claim 1, characterized in that: In step S30, the step of using the optimized long short-term memory network to predict the movement path of the obstacle within a preset time in the future includes: Step S301: The vehicle speed data v vehicle (t), driving direction data θ vehicle (t), relative speed data v between the obstacle and the vehicle relative (t) and the relative acceleration data a between the obstacle and the vehicle relative (t) Input into the optimized long short-term memory network; Step S302: In the optimized long short-term memory network, v in step S301 is converted to vehicle (t),θ vehicle (t), v relative (t), a relative (t) as the input data x at each time step t , processed by the following formula: The input gate layer controls the impact of input data on the current state: i t =σ(W i ·[h t-1 , x t +b i ]), The forget gate layer determines the degree to which previous information is forgotten: f t =σ(W f ·[h t-1 , x t +b f ]), The output gate layer controls the output of the current time step: o t =σ(W o ·[h t-1 , x t +b o ]), The cell state update layer generates the current state by memorizing the cell input information: C t =f t ·C t-1 +i t ·tanh(W c ·[h t-1 ,x t ]+b C ), Among them, W i , W f , W o , W c represents the weight matrix of each layer, b i , b f , b s , b C represents the bias term of each layer, σ is the sigmoid activation function, h t-1 is the hidden state output of the previous time step t-1, C t-1 is the cell state at the previous time step, and tanh is the hyperbolic tangent activation function; Step S303: The output of the model is the prediction of the movement path of the obstacle within the preset time in the future, including the position (x out ,y out ) and speed v out , these outputs are calculated by the last layer of LSTM.

3. A safety control method for an unmanned vehicle according to claim 1, characterized in that: In step S10, comprehensively judging and confirming obstacle information based on the vehicle surrounding object information and road key information includes: Use road geometry information and vehicle trajectory to make association judgments; When the objects around the vehicle are static and in the vehicle's driving track, they are judged to be obstacles; When the objects around the vehicle are dynamic objects, they are judged as obstacles regardless of whether they are in the vehicle's driving trajectory.

4. A safety control method for an unmanned vehicle as claimed in claim 1, characterized in that: In step S30, the obstacle information includes the geometric shape, edge contour, material characteristics and spatial movement trend information of the obstacle.

5. A safety control method for an unmanned vehicle according to claim 1, characterized in that: In the step S40, the step of generating the optimal driving route according to the moving path information in combination with the improved Dijkstra algorithm includes dynamically adjusting the node weights, updating the multiple moving paths of the obstacles in real time, and generating the optimal driving route.

6. A safety control method for an unmanned vehicle as claimed in claim 5, characterized in that: The step of dynamically adjusting the node weights is preferentially performed when the distance between the obstacle and the vehicle is less than a preset safety distance, and by increasing the weights of nodes near the obstacle, a driving route away from the obstacle is preferentially selected.

7. A safety control method for an unmanned vehicle according to any one of claims 1 to 6, characterized in that: The method also includes automatically adjusting the speed and direction of the vehicle based on the optimal driving route.

8. A safety control device for an unmanned vehicle, characterized in that: The safety control device comprises: A data fusion module is used to fuse the multi-source real-time data collected by multi-source sensors in time and space using Kalman filtering to form a three-dimensional model of the vehicle's surrounding environment and generate environmental perception result data; A feature extraction module, used to use a convolutional layer in a convolutional neural network to perform a local scan on the environmental perception result data, extract road edge feature data, road texture feature data, and object shape feature data in the environmental perception result data, and classify the extracted features through a classifier to obtain information about objects around the vehicle, including pedestrian information, other vehicle information, and traffic sign information; The road information detection module is used to detect and mark key road information, including boundary information, lane line information, and lane width information, using an edge detection algorithm based on the road structure, combined with vehicle driving trajectory information and road geometry information; The obstacle prediction module is used to comprehensively determine and confirm obstacle information based on the information of objects around the vehicle and key road information, and to use an optimized long short-term memory network to predict the movement path of obstacles within a preset time in the future by combining the vehicle's driving speed data, driving direction data, relative speed data and relative acceleration data with the obstacle; The path planning module is used to generate an optimal driving route according to the moving path information combined with the improved Dijkstra algorithm.

9. A safety control device for an unmanned vehicle, characterized in that: The device includes: a memory, a processor, and a safety control program stored in the memory and executable on the processor. When the program is executed by the processor, the safety control method for an unmanned vehicle as described in any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The computer program product includes a safety control program, and when the program is executed by a processor, the safety control method for an unmanned vehicle according to any one of claims 1 to 7 is implemented.

Images