A Vehicle Autonomous Obstacle Avoidance Method and System Based on Vision and LiDAR

Abstract

The present invention relates to a vehicle autonomous obstacle avoidance method and system based on vision and lidar, belonging to the technical field of intelligent vehicles. Among them, the method includes: acquiring light intensity data, automatically switching the acquisition mode according to the light intensity; acquiring the real-time position data of the vehicle, calculating the optimal driving route of the vehicle, dividing the optimal driving route according to a preset distance threshold to obtain a sequence of target points, and determining the real-time target point according to the sequence of target points; determining the traveling direction of the vehicle and the real-time vehicle safety zone, dividing the recognition area, and acquiring the size of the obstacle, the real-time state quantity, and the obstacle historical trajectory of the obstacle within the recognition area; calculating the predicted point coordinates through a prediction model according to the obstacle historical trajectory, and calculating the avoidance decision through a planning model. It realizes the automatic switching of the vehicle data acquisition mode, and improves the reaction speed and data accuracy of automatic obstacle avoidance.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent vehicles, and particularly relates to a vehicle autonomous obstacle avoidance method and system based on vision and lidar. Background Art

[0002] With the continuous progress of technology, the automation and intelligence levels of vehicles are getting higher and higher. As one of the key technologies for vehicle intelligence, autonomous obstacle avoidance is of great significance for improving the safety and reliability of vehicles.

[0003] During the traditional vehicle driving process, the driver mainly relies on vision and experience to judge the surrounding environment and perform obstacle avoidance operations. However, this method has many limitations. First of all, human vision has certain limitations. In bad weather conditions (such as heavy fog, heavy rain, etc.) or in low light conditions, the driver's line of sight will be seriously affected, making it difficult to accurately judge the surrounding obstacles. Secondly, the driver's reaction time is limited, and in emergency situations, it may not be possible to make the correct obstacle avoidance decision in time. In addition, factors such as driver fatigue and distraction will also increase the risk of vehicle collisions.

[0004] In the existing vehicle autonomous obstacle avoidance, the acquisition unit of the vehicle relies on lidar, vision sensors or a combination of lidar and vision sensors to collect data. However, all three data acquisition modes have certain problems:

[0005] (1) When using lidar to collect data under high light intensity, the data measured by the lidar will be inaccurate due to the influence of external light intensity on the laser.

[0006] (2) When using vision sensors to collect data under poor light intensity, the collected image data will be unclear due to weak light, and obstacles cannot be recognized in time.

[0007] (3) When collecting data through a combination of lidar and vision sensors, although the recognition accuracy of obstacles is improved, more data needs to be processed, the data processing time is longer, and higher-precision software and hardware devices are required. However, in current vehicle driving, the main driving mode is manual driving supplemented by autonomous driving. In this driving mode, high-precision obstacle acquisition devices are not needed. Only the position, size, speed, and movement direction of obstacles need to be measured quickly so that the in-vehicle processor can quickly make obstacle avoidance processing. Summary of the Invention

[0008] To solve the above problems existing in the prior art, the present invention provides a vehicle autonomous obstacle avoidance method and system based on vision and lidar.

[0009] The object of the present invention can be achieved by the following technical solutions:

[0010] Collect the light intensity data through an in-vehicle photosensitive sensor, and the in-vehicle control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode;

[0011] Obtain the real-time position data of the vehicle through the in-vehicle positioning unit, obtain the map data, the driving end point, and the driving starting point. The in-vehicle motion planning unit calculates the optimal path through the Dijkstra algorithm according to the map data, the driving end point, and the driving starting point, preset a distance threshold, divide the optimal path according to the distance threshold to obtain a target point sequence, and determine the real-time target point according to the target point sequence and the real-time position data;

[0012] Determine the vehicle traveling direction according to the real-time position data and the real-time target point, preset a vehicle safety distance, determine the real-time vehicle safety area according to the vehicle safety distance and the real-time position data, establish a coordinate system according to the real-time position data, divide the recognition area according to the coordinate system, obtain the obstacle size and the real-time state quantity of the obstacles in the recognition area through the acquisition unit, add a time stamp to the real-time state quantity to obtain a time-series state quantity, and obtain the obstacle historical trajectory according to the time-series state quantity;

[0013] Calculate the predicted point coordinates through the prediction model according to the obstacle historical trajectory, obtain the real-time grid map through the in-vehicle mapping unit, and calculate the avoidance decision through the planning model according to the real-time grid map, the vehicle traveling direction, the predicted point coordinates, the real-time vehicle safety area, the real-time state quantity, the obstacle size, and the real-time target point.

[0014] Specifically, the acquisition mode includes a lidar acquisition mode and a vision acquisition mode; the real-time target point is the target point with the smallest distance from the real-time position data in the target point sequence; the real-time state quantity includes the real-time coordinates, speed, and motion direction of the obstacle.

[0015] Specifically, the recognition area includes a first area, a second area, a third area, and a fourth area.

[0016] Specifically, the in-vehicle control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode, including:

[0017] Preset a light intensity threshold, judge the light intensity data according to the light intensity threshold. If the light intensity data is greater than the light intensity threshold, the acquisition mode is the vision acquisition mode; if the light intensity data is less than the light intensity threshold, the acquisition mode is the lidar acquisition mode.

[0018] Specifically, the specific calculation steps of the prediction model include:

[0019] S201: Discretize the obstacle historical trajectory to obtain attribute blocks, construct one-hot vectors according to the attribute blocks, preset sparsity constraints, obtain high-dimensional embedding vectors, and associate the one-hot vectors with the high-dimensional embedding vectors according to the sparsity constraints to obtain associated vectors;

[0020] S202: Calculate time features through a temporal convolutional model according to the associated vectors, and calculate spatial features through a Transformer model according to the associated vectors;

[0021] S203: Calculate predicted time features and predicted spatial features through an adaptive pooling layer according to the time features and the spatial features, and perform feature fusion on the predicted time features and the predicted spatial features through a CNN-LSTM-Attention model to obtain predicted spatio-temporal features;

[0022] S204: Perform a pseudo-inverse operation on the predicted spatio-temporal features to obtain the predicted coordinate points.

[0023] Specifically, the specific calculation steps of the planning model are as follows:

[0024] S301: Determine the real-time coordinates according to the real-time state variables, and determine the obstacle moving direction according to the real-time coordinates and the predicted point coordinates;

[0025] S302: Make a judgment according to the predicted point coordinates and the real-time vehicle safety domain. If the predicted point coordinates are within the real-time vehicle safety domain, execute step S303; if the predicted point coordinates are not within the real-time vehicle safety domain, execute the first decision;

[0026] S303: Make a judgment according to the obstacle moving direction and the vehicle moving direction. If the obstacle moving direction is parallel to the vehicle moving direction, execute the first decision; if the obstacle moving direction is not parallel to the vehicle moving direction, execute step S304;

[0027] S304: Obtain the real-time coordinates of the obstacle according to the real-time state variables, determine the avoidance direction according to the real-time coordinates of the obstacle and the real-time vehicle safety domain. If the obstacle coordinates are within the real-time vehicle safety domain, the avoidance direction is the obstacle moving direction; if the obstacle coordinates are not within the real-time vehicle safety domain, the avoidance direction is the opposite direction of the obstacle moving direction;

[0028] S305: Obtain an obstacle - marked map by marking the real - time grid map according to the size of the obstacle and the real - time coordinates of the obstacle. Calculate a traversal result through a path traversal model based on the obstacle - marked map, the real - time target point, the real - time position data, and the traveling direction. The traversal result includes path existence and path non - existence;

[0029] S306: Make a judgment according to the traversal result. If the traversal result is path existence, execute the second decision; if the traversal result is path non - existence, execute the third decision.

[0030] Specifically, the path traversal model specifically includes:

[0031] S401: Initialize the avoidance route, set the real - time target point as the traversal starting point, and set the real - time position data as the traversal ending point;

[0032] S402: Judge the passability factor of the grid in the obstacle - marked map according to the traversal starting point and the avoidance direction through a heuristic algorithm. If the passability factor is 0, store the grid in the avoidance route and execute step S403; if the passability factor is 1, output that the traversal result is path non - existence;

[0033] S403: Judge whether the grid is the traversal ending point. If the grid is the traversal ending point, output the avoidance route and output that the traversal result is path existence; if the grid is not the traversal ending point, repeat steps S401 - S403.

[0034] A vehicle autonomous obstacle avoidance system based on vision and lidar includes: a mode switching module, a data acquisition module, a data processing module, and an avoidance decision - making module;

[0035] The mode switching module is used to obtain light intensity data through an in - vehicle photosensitive sensor, and the in - vehicle control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode;

[0036] The data acquisition module is used to obtain the real - time position data of the vehicle through an in - vehicle positioning unit, obtain map data, a driving end point, and a driving starting point. The in - vehicle motion planning unit calculates the optimal path through Dijkstra's algorithm according to the map data, the driving end point, and the driving starting point, sets a preset distance threshold, divides the optimal path according to the distance threshold to obtain a target point sequence, and determines the real - time target point according to the target point sequence and the real - time position data;

[0037] The data processing module is used to determine the vehicle traveling direction according to the real-time position data and the real-time target point, preset the vehicle safety distance, determine the real-time vehicle safety area according to the vehicle safety distance and the real-time position data, establish a coordinate system according to the real-time position data, divide the recognition area according to the coordinate system, obtain the obstacle size and real-time state quantity of the obstacles in the recognition area through the acquisition unit, add a timestamp to the real-time state quantity to obtain the time-series state quantity, and obtain the obstacle historical trajectory according to the time-series state quantity;

[0038] The avoidance decision-making module is used to calculate the predicted point coordinates through a prediction model according to the obstacle historical trajectory, obtain the real-time grid map through the on-vehicle mapping unit, and calculate the avoidance decision through a planning model according to the real-time grid map, the vehicle traveling direction, the predicted point coordinates, the real-time vehicle safety area, the real-time state quantity, the obstacle size, and the real-time target point.

[0039] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the vehicle autonomous obstacle avoidance method based on vision and lidar as described above.

[0040] A storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute the vehicle autonomous obstacle avoidance method based on vision and lidar as described in any one of the above when executed by a computer processor.

[0041] The beneficial effects of the present invention are as follows:

[0042] By setting an on-vehicle photosensitive sensor to obtain the light intensity data of the external environment in real time and adjusting the on-vehicle acquisition mode in a timely manner according to the light intensity data, data acquisition is realized through a vision sensor when the light intensity is good, and data acquisition is realized through a lidar when the light intensity is weak. This avoids data errors caused by data acquisition through a single acquisition mode in different light environments, and also avoids the problem of slow obstacle avoidance reaction caused by the long data processing time in the data acquisition mode combining lidar and vision sensor, improving the reaction speed of vehicle automatic obstacle avoidance under the existing automotive hardware technology while improving the accuracy of the acquired data. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] For the convenience of those skilled in the art to understand, the present invention will be further described below with reference to the accompanying drawings.

[0044] Figure 1 It is a schematic flow chart of a vehicle autonomous obstacle avoidance method based on vision and lidar of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0045] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, elaborate in detail on the specific implementation manners, structures, features, and effects of the present invention as follows.

[0046] Please refer to Figure 1 , a vehicle autonomous obstacle avoidance method and system based on vision and lidar,

[0047] Obtain the light intensity data through the vehicle-mounted photosensitive sensor, and the vehicle control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode;

[0048] Obtain the real-time position data of the vehicle through the vehicle-mounted positioning unit, obtain the map data, the driving end point, and the driving start point. The vehicle motion planning unit calculates the optimal path through the Dijkstra algorithm according to the map data, the driving end point, and the driving start point, preset a distance threshold, divide the optimal path according to the distance threshold to obtain a target point sequence, and determine the real-time target point according to the target point sequence and the real-time position data;

[0049] Determine the vehicle traveling direction according to the real-time position data and the real-time target point, preset the vehicle safety distance, determine the real-time vehicle safety area according to the vehicle safety distance and the real-time position data, establish a coordinate system according to the real-time position data, divide the recognition area according to the coordinate system, obtain the obstacle size and real-time state quantity of the obstacles in the recognition area through the acquisition unit, add a time stamp to the real-time state quantity to obtain the time-series state quantity, and obtain the obstacle historical trajectory according to the time-series state quantity;

[0050] Calculate the predicted point coordinates through the prediction model according to the obstacle historical trajectory, obtain the real-time grid map through the vehicle-mounted mapping unit, and calculate the avoidance decision through the planning model according to the real-time grid map, the vehicle traveling direction, the predicted point coordinates, the real-time vehicle safety area, the real-time state quantity, the obstacle size, and the real-time target point.

[0051] The acquisition mode includes the lidar acquisition mode and the vision acquisition mode; the real-time target point is the target point with the smallest distance from the real-time position data in the target point sequence; the real-time state quantity includes the real-time coordinates, speed, and motion direction of the obstacle.

[0052] In this embodiment, the vision acquisition mode includes:

[0053] Image data is acquired through a visual sensor, preprocessing is performed according to the image data to obtain preprocessed data, feature extraction is performed according to the preprocessed data by an edge detection method to obtain image features, obstacle recognition is performed according to the image features by a convolutional neural network to obtain an obstacle recognition result, the speed and the moving direction are calculated according to the obstacle recognition result by an optical flow method, obstacle material points are determined according to the obstacle recognition result, pixel coordinates of the obstacle recognition result are converted by coordinates to obtain real-time coordinates of the obstacle, and the obstacle size is calculated according to the edge point coordinates of the obstacle recognition result.

[0054] Specifically, the coordinate transformation calculation formula is:

[0055] S*P i =A[R|T]P w ,

[0056] Where S is the distance between the obstacle and the camera, P i is the pixel coordinate, A is the camera intrinsic matrix, R is the rotation matrix, T is the translation vector, P w Real-time coordinates of obstacles.

[0057] It should be noted that the preprocessing includes noise reduction, data enhancement, grayscale, and binarization.

[0058] Specifically, the laser radar acquisition mode includes:

[0059] Point cloud data is acquired through a laser radar, and de-noised point cloud data is calculated based on the point cloud data through a through-filtering algorithm. A region of interest is calculated based on the de-noised point cloud data through an ROI algorithm. An obstacle point cloud is obtained by performing ground segmentation based on the region of interest through a plane fitting equation. A point cloud cluster is obtained based on the obstacle point cloud through Euclidean clustering. Point cloud features are calculated based on the point cloud cluster through a DGCNN algorithm. Two consecutive frames of obstacle point clouds are associated based on the point cloud features to obtain associated data. The obstacle size, the real-time coordinates of the obstacle, and the movement direction are obtained based on the associated data through an EKF filtering algorithm. The speed is calculated based on the real-time coordinates of the obstacle in two consecutive frames and a frame interval.

[0060] Specifically, the identified partitions include partition No. 1, partition No. 2, partition No. 3, and partition No. 4. Partition No. 1 is the [60°, 120°] interval area on the coordinate system, partition No. 2 is the [120°, 240°] interval area on the coordinate system, partition No. 3 is the [240°, 300°] interval area on the coordinate system, and partition No. 4 is the [300°, 60°] interval area on the coordinate system.

[0061] Specifically, the in-vehicle control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode, including:

[0062] Preset a light intensity threshold, and judge the light intensity data according to the light intensity threshold. If the light intensity data is greater than the light intensity threshold, the acquisition mode is the visual acquisition mode; if the light intensity data is less than the light intensity threshold, the acquisition mode is the lidar acquisition mode.

[0063] It should be noted that both the lidar acquisition mode and the visual acquisition mode are used to acquire the real-time coordinates of the obstacle, and the light intensity threshold is used to judge whether the vehicle driving environment is a bright environment.

[0064] Specifically, the specific calculation steps of the prediction model include:

[0065] S201: Discretize the obstacle historical trajectory to obtain attribute blocks, construct one-hot vectors according to the attribute blocks, preset a sparsity constraint, obtain a high-dimensional embedding vector, and associate the one-hot vector with the high-dimensional embedding vector according to the sparsity constraint to obtain an association vector;

[0066] It should be noted that the attribute blocks include an abscissa attribute block, an ordinate attribute block, a speed attribute block, and a motion direction attribute block;

[0067] S202: Calculate time features according to the association vector through a temporal convolutional model, and calculate spatial features according to the association vector through a Transformer model;

[0068] S203: Calculate predicted time features and predicted spatial features according to the time features and the spatial features through an adaptive pooling layer, and perform feature fusion on the predicted time features and the predicted spatial features through a CNN-LSTM-Attention model to obtain predicted spatio-temporal features;

[0069] S204: Perform a pseudo-inverse operation on the predicted spatio-temporal features to obtain the predicted coordinate points.

[0070] The expression of the pseudo-inverse operation is:

[0071] X = (A T A) -1 A T b,

[0072] where X is the predicted coordinate point, A is the predicted spatio-temporal feature, and b is the mapping relationship vector.

[0073] Specifically, the specific calculation steps of the planning model are:

[0074] S301: Determine the real-time coordinates based on the real-time state variables, and determine the obstacle moving direction based on the real-time coordinates and the predicted point coordinates;

[0075] S302: Make a judgment based on the predicted point coordinates and the real-time vehicle safety region. If the predicted point coordinates are within the real-time vehicle safety region, execute step S303; if the predicted point coordinates are not within the real-time vehicle safety region, execute the first decision;

[0076] S303: Make a judgment based on the obstacle moving direction and the vehicle moving direction. If the obstacle moving direction is parallel to the vehicle moving direction, execute the first decision; if the obstacle moving direction is not parallel to the vehicle moving direction, execute step S304;

[0077] S304: Obtain the obstacle coordinates based on the real-time state variables, determine the avoidance direction based on the obstacle coordinates and the real-time vehicle safety region. If the obstacle coordinates are within the real-time vehicle safety region, the avoidance direction is the obstacle moving direction; if the obstacle coordinates are not within the real-time vehicle safety region, the avoidance direction is the opposite direction of the obstacle moving direction;

[0078] S305: Mark the real-time grid map according to the obstacle size and the real-time coordinates of the obstacle to obtain a marked obstacle map, and calculate the traversal result through a path traversal model based on the marked obstacle map, the real-time target point, the real-time position data, and the moving direction;

[0079] S306: Make a judgment based on the traversal result. If the traversal result is that a path exists, execute the second decision; if the traversal result is that a path does not exist, execute the third decision.

[0080] Specifically, the avoidance decision includes the first decision, the second decision, and the third decision. The first decision is to continue running in the current state, the second decision is to run according to the planned route, and the third decision is to park the vehicle.

[0081] Specifically, the path traversal model specifically includes:

[0082] S401: Initialize the avoidance route, set the real-time target point as the traversal starting point, and set the real-time position data as the traversal ending point;

[0083] S402: Judge the passing factor of the grid in the marked obstacle map based on the traversal starting point and the avoidance direction through a heuristic algorithm. If the passing factor is 0, store the grid in the avoidance route and execute step S403; if the passing factor is 1, output that the traversal result is that a path does not exist;

[0084] S403: Determine whether the grid is the end point of the traversal. If the grid is the end point of the traversal, output the avoidance route and output that the traversal result is that the path exists; if the grid is not the end point of the traversal, repeat steps S401 - S403.

[0085] It should be noted that the heuristic algorithm includes but is not limited to genetic algorithm, particle swarm optimization algorithm, ant colony algorithm, and greedy algorithm.

[0086] A vehicle autonomous obstacle avoidance system based on vision and lidar, comprising: a mode switching module, a data acquisition module, a data processing module, and an avoidance decision module;

[0087] The mode switching module is used to obtain light intensity data through an in - vehicle photosensitive sensor, and the in - vehicle control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode;

[0088] The data acquisition module is used to obtain the real - time position data of the vehicle through an in - vehicle positioning unit, obtain map data, a driving end point, and a driving start point. The in - vehicle motion planning unit calculates the optimal path through Dijkstra's algorithm according to the map data, the driving end point, and the driving start point, sets a preset distance threshold, divides the optimal path according to the distance threshold to obtain a target point sequence, and determines the real - time target point according to the target point sequence and the real - time position data;

[0089] The data processing module is used to determine the vehicle's traveling direction according to the real - time position data and the real - time target point, set a preset vehicle safety distance, determine the real - time vehicle safety area according to the vehicle safety distance and the real - time position data, establish a coordinate system according to the real - time position data, divide the recognition area according to the coordinate system, obtain the obstacle size and real - time state quantity of the obstacles in the recognition area through the acquisition unit, add a time stamp to the real - time state quantity to obtain a time - series state quantity, and obtain the obstacle historical trajectory according to the time - series state quantity;

[0090] The avoidance decision module is used to calculate the predicted point coordinates through a prediction model according to the obstacle historical trajectory, obtain the real - time grid map through the in - vehicle mapping unit, and calculate the avoidance decision through a planning model according to the real - time grid map, the vehicle's traveling direction, the predicted point coordinates, the real - time vehicle safety area, the real - time state quantity, the obstacle size, and the real - time target point.

[0091] An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, it implements the vehicle autonomous obstacle avoidance method based on vision and lidar as described above.

[0092] A storage medium containing computer-executable instructions, which are used to execute the vision- and lidar-based vehicle autonomous obstacle avoidance method as described in any one of the above when executed by a computer processor.

[0093] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0094] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0095] The program code contained on a computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0096] As described above, it is only the preferred embodiment of the present invention, and it does not impose any form of limitation on the present invention. Although the present invention has been disclosed as above with the preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to be equivalent embodiments of equivalent changes within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A vehicle autonomous obstacle avoidance method based on vision and laser radar, characterized in that: include: The light intensity data is collected by the vehicle-mounted photosensor, and the vehicle-mounted control unit generates an adaptive signal according to the light intensity data to control the collection unit to switch the collection mode; The real-time position data of the vehicle is obtained through the vehicle positioning unit, and the map data, the driving end point, and the driving starting point are obtained. The vehicle motion planning unit calculates the optimal path through the Dijkstra algorithm according to the map data, the driving end point, and the driving starting point, presets a distance threshold, divides the optimal path according to the distance threshold to obtain a target point sequence, and determines the real-time target point according to the target point sequence and the real-time position data; Determine the direction of vehicle travel according to the real-time position data and the real-time target point, preset a vehicle safety distance, determine a real-time vehicle safety domain according to the vehicle safety distance and the real-time position data, establish a coordinate system according to the real-time position data, divide the identification partition according to the coordinate system, obtain the obstacle size and real-time state quantity of the obstacle in the identification partition through the acquisition unit, add a timestamp according to the real-time state quantity to obtain a time series state quantity, and obtain the obstacle history trajectory according to the time series state quantity; The prediction point coordinates are calculated by the prediction model according to the historical trajectory of the obstacle, the real-time grid map is obtained by the on-board mapping unit, and the avoidance decision is calculated by the planning model according to the real-time grid map, the vehicle travel direction, the prediction point coordinates, the real-time vehicle safety domain, the real-time state quantity, the obstacle size, and the real-time target point; The specific calculation steps of the prediction model include: Discretize the obstacle history trajectory to obtain an attribute block, construct a one-hot vector according to the attribute block, preset a sparsity constraint, obtain a high-dimensional embedding vector, and associate the one-hot vector with the high-dimensional embedding vector according to the sparsity constraint to obtain an association vector; A temporal feature is calculated by using a temporal convolution model according to the association vector, and a spatial feature is calculated by using a Transformer model according to the association vector; According to the time feature and the space feature, a predicted time feature and a predicted space feature are calculated by an adaptive pooling layer, and according to the predicted time feature and the predicted space feature, a predicted space-time feature is obtained by feature fusion through a CNN-LSTM-Attention model; The predicted spatiotemporal features are subjected to a pseudo-inverse operation to obtain the predicted point coordinates.

2. The vehicle autonomous obstacle avoidance method based on vision and laser radar according to claim 1 is characterized in that: The acquisition mode includes a laser radar acquisition mode and a visual acquisition mode; the real-time target point is the target point with the smallest distance from the real-time position data in the target point sequence; the real-time state quantity includes the real-time coordinates, speed, and movement direction of the obstacle.

3. The vehicle autonomous obstacle avoidance method based on vision and laser radar according to claim 1 is characterized in that: The identified partitions include partition number one, partition number two, partition number three, and partition number four.

4. The vehicle autonomous obstacle avoidance method based on vision and laser radar according to claim 1 is characterized in that: The vehicle-mounted control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode, including: A light intensity threshold is preset, and the light intensity data is judged according to the light intensity threshold. If the light intensity data is greater than the light intensity threshold, the acquisition mode is the visual acquisition mode; if the light intensity data is less than the light intensity threshold, the acquisition mode is the lidar acquisition mode.

5. The vehicle autonomous obstacle avoidance method based on vision and laser radar according to claim 1, characterized in that: The specific calculation steps of the planning model are: S301: determining a real-time coordinate according to the real-time state quantity, and determining an obstacle moving direction according to the real-time coordinate and the predicted point coordinate; S302: judging according to the predicted point coordinates and the real-time vehicle safety domain, if the predicted point coordinates are within the real-time vehicle safety domain, executing step S303; if the predicted point coordinates are not within the real-time vehicle safety domain, executing the first decision; S303: judging according to the obstacle moving direction and the vehicle moving direction, if the obstacle moving direction is parallel to the vehicle moving direction, executing the first decision; if the obstacle moving direction is not parallel to the vehicle moving direction, executing step S304; S304: Obtaining the real-time coordinates of the obstacle according to the real-time state quantity, and determining the avoidance direction according to the real-time coordinates of the obstacle and the real-time vehicle safety domain, if the obstacle coordinates are within the real-time vehicle safety domain, the avoidance direction is the direction of the obstacle; if the obstacle coordinates are not within the real-time vehicle safety domain, the avoidance direction is the opposite direction of the obstacle direction; S305: Marking the real-time grid map according to the obstacle size and the real-time coordinates of the obstacle to obtain a marked obstacle map, and calculating a traversal result through a path traversal model according to the marked obstacle map, the real-time target point, the real-time location data, and the travel direction, wherein the traversal result includes whether a path exists or not; S306: Making a judgment based on the traversal result, if the traversal result is that the path exists, executing the second decision; if the traversal result is that the path does not exist, executing the third decision.

6. The vehicle autonomous obstacle avoidance method based on vision and laser radar according to claim 5 is characterized in that: The path traversal model specifically includes: S401: Initialize the avoidance route, set the real-time target point as the traversal starting point, and set the real-time location data as the traversal end point; S402: judging the pass factor of the grid in the obstacle map by a heuristic algorithm according to the traversal starting point and the avoidance direction; if the pass factor is 0, storing the grid in the avoidance route and executing step S403; if the pass factor is 1, outputting the traversal result as path non-existence; S403: Determine whether the grid is the traversal end point. If the grid is the traversal end point, output the avoidance route and output the traversal result as path existence. If the grid is not the traversal end point, repeat steps S401-S403.

7. A vehicle autonomous obstacle avoidance system based on vision and laser radar, characterized in that: include: Mode switching module, data acquisition module, data processing module, avoidance decision module; The mode switching module is used to obtain light intensity data through the vehicle-mounted photosensor, and the vehicle-mounted control unit generates an adaptive signal according to the light intensity data to control the acquisition unit to switch the acquisition mode; The data acquisition module is used to obtain the real-time position data of the vehicle through the vehicle positioning unit, obtain the map data, the driving end point, and the driving starting point, the vehicle motion planning unit calculates the optimal path through the Dijkstra algorithm according to the map data, the driving end point, and the driving starting point, presets a distance threshold, divides the optimal path according to the distance threshold to obtain a target point sequence, and determines the real-time target point according to the target point sequence and the real-time position data; The data processing module is used to determine the vehicle's direction of travel according to the real-time position data and the real-time target point, preset a vehicle safety distance, determine a real-time vehicle safety domain according to the vehicle safety distance and the real-time position data, establish a coordinate system according to the real-time position data, divide the identification partition according to the coordinate system, obtain the obstacle size and real-time state quantity of the obstacle in the identification partition through the acquisition unit, add a timestamp according to the real-time state quantity to obtain a time series state quantity, and obtain the obstacle history trajectory according to the time series state quantity; The avoidance decision module is used to calculate the predicted point coordinates through the prediction model according to the obstacle historical trajectory, obtain the real-time grid map through the vehicle-mounted mapping unit, and calculate the avoidance decision through the planning model according to the real-time grid map, the vehicle travel direction, the predicted point coordinates, the real-time vehicle safety domain, the real-time state quantity, the obstacle size, and the real-time target point; The specific calculation steps of the prediction model include: Discretize the obstacle history trajectory to obtain an attribute block, construct a one-hot vector according to the attribute block, preset a sparsity constraint, obtain a high-dimensional embedding vector, and associate the one-hot vector with the high-dimensional embedding vector according to the sparsity constraint to obtain an association vector; A temporal feature is calculated by using a temporal convolution model according to the association vector, and a spatial feature is calculated by using a Transformer model according to the association vector; According to the time feature and the space feature, a predicted time feature and a predicted space feature are calculated by an adaptive pooling layer, and according to the predicted time feature and the predicted space feature, a predicted space-time feature is obtained by feature fusion through a CNN-LSTM-Attention model; The predicted spatiotemporal features are subjected to a pseudo-inverse operation to obtain the predicted point coordinates.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the vehicle autonomous obstacle avoidance method based on vision and lidar as described in any one of claims 1-6 is implemented.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the vehicle autonomous obstacle avoidance method based on vision and lidar as described in any one of claims 1 to 6.

Images