Path optimization selection method and system based on time-sharing planning and radar-vision fusion

Abstract

The present invention discloses a path optimization selection method and system based on time-sharing planning and radar-vision fusion. First, time-sharing path planning is performed based on a spatiotemporal dynamic graph model TSDG and time-sharing path planning data is obtained. Then, a graph neural network GNN is used to dynamically adjust the time-sharing path planning data according to real-time traffic conditions and the adjusted time-sharing path planning data is obtained. The present invention realizes the function of time-sharing path optimization selection, and also has the function of fusing visual data with laser radar data, and also has the function of quickly and accurately selecting and executing from executing path optimization and executing trajectory optimization, and also has the function of real-time monitoring and adjustment of optimization results. It can not only adapt to different application scenario requirements, but also use the advantages of path optimization and trajectory optimization to improve the efficiency and safety of path planning, and is suitable for being widely promoted and used.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned driving path optimization selection, and in particular to a path optimization selection method and system based on time-sharing planning and radar-vision fusion. Background Art

[0002] Autonomous driving technology is a car autonomous driving technology achieved through the integration of multiple advanced technologies. As a part of future transportation, autonomous driving technology integrates advanced artificial intelligence, perception computing and data communication technologies. These technologies together provide autonomous vehicles with the ability to perceive the environment, make decisions and perform operations.

[0003] At present, it is difficult for vehicle unmanned driving modules to perform rapid path optimization selection in uncertain environments, resulting in poor path planning efficiency during use of vehicle unmanned driving modules and the inability to adapt to the needs of different application scenarios. For example, when dynamic obstacles are detected, it is impossible to generate an accurate trajectory that meets the vehicle dynamics constraints, which makes the efficiency and safety of path planning low. Therefore, it is necessary to design a path optimization selection method and system based on time-sharing planning and radar-vision fusion. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and to better and effectively solve the problem that the current vehicle unmanned driving module is difficult to perform rapid path optimization selection in an uncertain environment, resulting in poor path planning efficiency during use of the vehicle unmanned driving module and the inability to adapt to the needs of different application scenarios. For example, when a dynamic obstacle is detected, an accurate trajectory that meets the vehicle dynamics constraints cannot be generated, which makes the efficiency and safety of path planning low. A path optimization selection method and system based on time-sharing planning and radar-vision fusion are provided, which realizes the function of time-sharing path optimization selection, and also has the function of fusing visual data with lidar data, and also has the function of quickly and accurately selecting and executing from executing path optimization and executing trajectory optimization, and also has the function of real-time monitoring and adjustment of the optimization results. It can not only adapt to the needs of different application scenarios, but also utilize the advantages of path optimization and trajectory optimization to improve the efficiency and safety of path planning.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] like Figure 1 As shown, the path optimization selection method based on time-sharing planning and radar-vision fusion includes the following steps:

[0007] Step A, performing time-sharing path planning based on the spatiotemporal dynamic graph model TSDG to obtain time-sharing path planning data;

[0008] Step B, using the graph neural network GNN to dynamically adjust the time-sharing path planning data according to the real-time traffic condition changes to obtain the adjusted time-sharing path planning data;

[0009] Step C, using the multi-sensory fusion learning framework MGLF to fuse the visual data and the lidar data to obtain processed visual and lidar fusion data;

[0010] Step D, selecting from performing path optimization and performing trajectory optimization according to the adjusted time-sharing path planning data and the processed vision and lidar fusion data to obtain an optimization result;

[0011] Step E: Use the feedback mechanism to monitor and adjust the optimization results in real time to complete the path optimization selection task.

[0012] The aforementioned path optimization selection method based on time-sharing planning and radar-vision fusion, step A, performs time-sharing path planning based on the spatiotemporal dynamic graph model TSDG to obtain time-sharing path planning data. The specific steps are as follows:

[0013] Step A1, dividing 24 hours a day into multiple time periods, wherein the time periods include morning peak, evening peak and night;

[0014] Step A2, constructing a dynamic spatiotemporal grid map based on historical data and real-time data for each time period;

[0015] Step A3, path search based on the dynamic spatiotemporal grid graph, as shown in formula (1),

[0016] (1)

[0017] in, represents the total estimated cost from the starting point to the end point through node x, represents the actual cost from the starting point to the current point, Indicates the estimated cost from the current point to the end point;

[0018] Step A4, constructing the long-term change pattern of traffic flow and predicting future traffic condition changes based on the graph neural network GNN;

[0019] Step A5, after re-entering a new time period, the dynamic spatiotemporal grid map is updated based on the new data and the path is re-planned.

[0020] The aforementioned path optimization selection method based on time-sharing planning and radar vision fusion, step B, uses the graph neural network GNN to dynamically adjust the time-sharing path planning data according to the real-time traffic conditions, and obtains the adjusted time-sharing path planning data. The specific steps are as follows:

[0021] Step B1, constructing a graph neural network GNN model, specifically constructing a graph neural network GNN model based on the traffic flow, vehicle speed and accident report data detected by the L4 unmanned parking inspection module installed inside the vehicle, wherein the road intersections and road sections are respectively represented as nodes and edges in the graph neural network GNN model, and then setting the node features and edge features in the graph neural network GNN model, the node features include the average vehicle speed and the average traffic density, and the edge features include the travel time, distance and current traffic status between two nodes;

[0022] Step B2, update the node features and edge features in the graph neural network GNN model, as shown in formula (2):

[0023] (2)

[0024] in, represents the node features of the next layer, Representation Node In the The feature vector of the layer, and represents the learned weight and bias parameters, Representation Node The set of neighbor nodes of represents the activation function;

[0025] Step B3, using historical traffic data to train the graph neural network (GNN) model to learn the long-term change pattern of traffic flow, where the long-term change pattern includes the impact of seasonal changes and special events;

[0026] Step B4, use real-time traffic data to update the node features and edge features in the graph neural network GNN model, and perform forward propagation through the graph neural network GNN to predict changes in traffic conditions in the short term in the future.

[0027] The aforementioned path optimization selection method based on time-sharing planning and laser-visual fusion, step C, uses the multi-sensory fusion learning framework MGLF to fuse the visual data and the laser radar data to obtain the processed visual and laser radar fusion data. The specific steps are as follows:

[0028] Step C1, extracting visual data and LiDAR data The features of Adjusting visual data and LiDAR data In the Ray Vision Fusion feature The proportion in is as shown in formula (3),

[0029] (3);

[0030] Step C2, constructing a deep deformable convolutional network in the visual branch, wherein the deep deformable convolutional network is used to detect objects in visual data, specifically, adjusting the shape of the convolution kernel to adapt to the actual shape change of the object, as shown in formula (4),

[0031] (4)

[0032] in, Represents the input data Apply weights The weighted sum of represents the input data at the i-th convolution position, represents the learned weights;

[0033] Step C3, constructing a visual information enhancement model VIME in the laser radar branch. The visual information enhancement model VIME is used to detect and track moving targets based on the laser radar data. Specifically, the detection and tracking capabilities of target motion are enhanced by using time series analysis, as shown in formula (5).

[0034] (5)

[0035] in, Indicates time The target motion model, Represents the lidar data of the current frame, represents the momentum term;

[0036] Step C4, establish the attention fusion mechanism of adaptive transformation learning, specifically based on extracting feature vectors from visual data and lidar data , and then use the weight matrix Calculate the attention weight as shown in formula (6):

[0037] (6)

[0038] in, is the normalized attention coefficient, It is used to convert weights into probability distribution to ensure that the sum of weights of each data source is 1, and then automatically adjust the influence of each perception data according to the actual situation;

[0039] Step C5, using the calculated attention coefficient Dynamically adjust the fusion ratio of visual data and lidar data to obtain processed visual and lidar fusion data.

[0040] The aforementioned path optimization selection method based on time-sharing planning and laser vision fusion, step D, selects from execution path optimization and execution trajectory optimization according to the adjusted time-sharing path planning data and the processed vision and laser radar fusion data to obtain the optimization result. The specific steps are as follows:

[0041] Step D1, data acquisition and analysis, obtaining traffic flow data, road status data and estimated delivery time data from the adjusted time-sharing path planning data, and then obtaining road obstacle data, traffic sign data and pedestrian data from the processed vision and lidar fusion data;

[0042] Step D2, decision combination, constructing a dynamic decision support model DDSS, the dynamic decision support model DDSS is used to use the data obtained in step D1 to analyze and predict the current traffic and environmental conditions in real time. The dynamic decision support model DDSS uses a machine learning algorithm to integrate and analyze the data obtained from step D1 and determine the optimal decision plan under the current conditions, as shown in formula (7).

[0043] (7)

[0044] in, Indicates at time Traffic status score, represents the data obtained from step D1, represents the smoothing coefficient;

[0045] Step D3, select to perform path optimization or trajectory optimization, and use the dynamic decision support model DDSS to evaluate and select to perform path optimization or trajectory optimization, wherein the path optimization is used to change the route to avoid congestion and reduce the execution time, and the trajectory optimization is used to adjust the vehicle's driving behavior, driving speed and obstacle avoidance strategy on the current route to adapt to the current road conditions and traffic conditions, as shown in formula (8),

[0046] (8)

[0047] in, represents the decision value, and the decision value Used to assess the current level of congestion, represents the decision factor weight obtained based on historical data training, Representing real-time environmental and traffic data characteristics;

[0048] Step D4, optimization execution, executes the selected path optimization or trajectory optimization, and the vehicle navigation module is also used for dynamic adjustment during the optimization execution process, as shown in formula (9),

[0049] (9)

[0050] in, represents the priority score of comprehensive path optimization and trajectory optimization, and are the priority scores for path optimization and trajectory optimization, and Represent the weights of path optimization and trajectory optimization respectively.

[0051] In the aforementioned path optimization selection method based on time-sharing planning and radar-vision fusion, step E uses the feedback mechanism to monitor and adjust the optimization results in real time to complete the path optimization selection operation. The feedback loop control formula in the feedback mechanism is shown in formula (10):

[0052] (10)

[0053] in, represents a control input, the control input Used to adjust the speed and distance of the vehicle to adapt to speed changes and route adjustments caused by path optimization or trajectory optimization. Represents the difference between expectation and actual execution. , and They represent proportional gain, integral gain and differential gain respectively.

[0054] The path optimization selection system based on time-sharing planning and radar-vision fusion includes a time-sharing path planning module, a time-sharing path planning adjustment module, a fusion processing module, an optimization selection module and a monitoring adjustment module;

[0055] The time-sharing path planning module is used to perform time-sharing path planning based on the spatiotemporal dynamic graph model TSDG and obtain time-sharing path planning data;

[0056] The time-sharing path planning adjustment module is used to dynamically adjust the time-sharing path planning data according to the real-time traffic condition changes using the graph neural network GNN and obtain the adjusted time-sharing path planning data;

[0057] The fusion processing module is used to use the multi-sensory fusion learning framework MGLF to fuse the visual data and the laser radar data and obtain the processed visual and laser radar fusion data;

[0058] The optimization selection module is used to select and obtain the optimization result from the execution path optimization and the execution trajectory optimization according to the adjusted time-sharing path planning data and the processed vision and laser radar fusion data;

[0059] The monitoring and adjustment module is used to utilize a feedback mechanism to perform real-time monitoring and adjustment on the optimization results and complete the path optimization selection operation.

[0060] The aforementioned path optimization selection system based on time-sharing planning and radar-vision fusion, the specific fusion processing process of the fusion processing module includes extracting features of visual data and lidar data, constructing a deep deformable convolutional network in the visual branch, constructing a visual information enhancement model VIME in the lidar branch, and establishing an attention fusion mechanism for adaptive transformation learning.

[0061] The aforementioned path optimization selection system based on time-sharing planning and radar-vision fusion, the specific optimization selection process of the optimization selection module includes data acquisition and analysis, decision combination, selection of execution path optimization or trajectory optimization and execution of the selected path optimization or trajectory optimization.

[0062] The beneficial effects of the present invention are:

[0063] (1) The present invention can perform time-sharing path planning based on the spatiotemporal dynamic graph model TSDG and obtain time-sharing path planning data, and then use the graph neural network GNN to dynamically adjust the time-sharing path planning data according to real-time traffic conditions and obtain the adjusted time-sharing path planning data, effectively realizing the function of time-sharing path optimization selection of the present invention, which is not only applicable to long-term changes including seasonal changes and special events, but also can predict changes in traffic conditions in the short term in the future, and the path optimization speed is fast and efficient.

[0064] (2) The present invention can utilize the multi-sensory fusion learning framework MGLF to fuse visual data with lidar data and obtain processed visual and lidar fusion data, which effectively realizes the function of the present invention to fuse visual data with lidar data. It can not only adapt to different environmental conditions and perception requirements, but also allow the network to adapt to different target forms. At the same time, it can also balance historical data and current data to improve the consistency and stability of motion prediction. In this way, by dynamically adjusting the fusion ratio of different perception data, the integration of multi-sensory data can be optimized and the perception output can be automatically adjusted according to environmental changes, thereby improving the recognition accuracy and robustness, and ensuring the ability to flexibly adapt to different external conditions such as weather changes and light intensity and the requirements of urban driving and highway driving scenarios.

[0065] (3) The present invention can select and obtain optimization results from executing path optimization and executing trajectory optimization based on the adjusted time-sharing path planning data and the processed vision and lidar fusion data, effectively realizing the function of the present invention to quickly and accurately select and execute from executing path optimization and executing trajectory optimization. It can not only adapt to the current road conditions and traffic conditions, but also ensure driving efficiency and safety in various traffic environments. At the same time, it provides accurate traffic flow prediction and road status update, making it suitable for emergency obstacle avoidance and traffic congestion rapid response scenarios that require rapid response. In this way, by dynamically selecting to execute path optimization or trajectory optimization according to the current road and traffic conditions, the most appropriate driving strategy can be selected in emergencies or special weather conditions.

[0066] (4) The present invention can utilize the feedback mechanism to monitor and adjust the optimization results in real time and complete the path optimization selection operation, which effectively realizes the function of the present invention to monitor and adjust the optimization results in real time. It can not only adjust the speed and distance of the vehicle, but also adapt to the speed change and route adjustment of the vehicle in front. The present invention can plan the optimal geometric path by performing path optimization in the global route planning stage, and switch to the trajectory optimization mode when a dynamic obstacle is detected in the local motion planning stage to generate a fine trajectory that meets the vehicle dynamics constraints. This dynamic selection mechanism enables the present invention to adapt to different application scenario requirements, and utilizes the advantages of path optimization and trajectory optimization to improve the efficiency and safety of path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is an overall flow chart of the path optimization selection method based on time-sharing planning and radar-vision fusion of the present invention;

[0068] Figure 2 It is an overall schematic diagram of the path optimization selection system based on time-sharing planning and radar-vision fusion of the present invention;

[0069] Figure 3 It is a schematic diagram of the spatiotemporal dynamic graph model TSDG of the present invention;

[0070] Figure 4 It is a schematic diagram of the present invention using the multi-sensory fusion learning framework MGLF to fuse visual data and laser radar data;

[0071] Figure 5 It is a schematic diagram of simulation results of dynamic selection path optimization and trajectory optimization of the present invention. DETAILED DESCRIPTION

[0072] The present invention will be further described below in conjunction with the accompanying drawings.

[0073] like Figure 1As shown, the path optimization selection method based on time-sharing planning and radar-vision fusion of the present invention includes the following steps:

[0074] like Figure 3 As shown, step A, based on the spatiotemporal dynamic graph model TSDG, time-sharing path planning is performed to obtain time-sharing path planning data. The specific steps are as follows:

[0075] Step A1, dividing 24 hours a day into multiple time periods, wherein the time periods include morning peak, evening peak and night;

[0076] Step A2, constructing a dynamic spatiotemporal grid map based on historical data and real-time data for each time period;

[0077] Step A3, path search based on the dynamic spatiotemporal grid graph, as shown in formula (1),

[0078] (1)

[0079] in, represents the total estimated cost from the starting point to the end point through node x, represents the actual cost from the starting point to the current point, Indicates the estimated cost from the current point to the end point;

[0080] Step A4, constructing the long-term change pattern of traffic flow and predicting future traffic condition changes based on the graph neural network GNN;

[0081] Step A5, after re-entering a new time period, the dynamic spatiotemporal grid map is updated based on the new data and the path is re-planned.

[0082] Step B: Use the graph neural network GNN to dynamically adjust the time-sharing path planning data according to the real-time traffic conditions to obtain the adjusted time-sharing path planning data. The specific steps are as follows:

[0083] Step B1, constructing a graph neural network GNN model, specifically constructing a graph neural network GNN model based on the traffic flow, vehicle speed and accident report data detected by the L4 unmanned parking inspection module installed inside the vehicle, wherein the road intersections and road sections are respectively represented as nodes and edges in the graph neural network GNN model, and then setting the node features and edge features in the graph neural network GNN model, the node features include the average vehicle speed and the average traffic density, and the edge features include the travel time, distance and current traffic status between two nodes;

[0084] Step B2, update the node features and edge features in the graph neural network GNN model, as shown in formula (2):

[0085] (2)

[0086] in, represents the node features of the next layer, Representation Node In the The feature vector of the layer, and represents the learned weight and bias parameters, Representation Node The set of neighbor nodes of represents the activation function;

[0087] Step B3, using historical traffic data to train the graph neural network (GNN) model to learn the long-term change pattern of traffic flow, where the long-term change pattern includes the impact of seasonal changes and special events;

[0088] Step B4, use real-time traffic data to update the node features and edge features in the graph neural network GNN model, and perform forward propagation through the graph neural network GNN to predict changes in traffic conditions in the short term in the future.

[0089] like Figure 4 As shown, step C uses the multi-sensory fusion learning framework MGLF to fuse the visual data and the lidar data to obtain the processed visual and lidar fusion data. The specific steps are as follows:

[0090] Step C1, extracting visual data and LiDAR data The features of Adjusting visual data and LiDAR data In the Ray Vision Fusion feature The proportion in is as shown in formula (3),

[0091] (3);

[0092] Step C2, constructing a deep deformable convolutional network in the visual branch, wherein the deep deformable convolutional network is used to detect objects in visual data, specifically, adjusting the shape of the convolution kernel to adapt to the actual shape change of the object, as shown in formula (4),

[0093] (4)

[0094] in, Represents the input data Apply weights The weighted sum of represents the input data at the i-th convolution position, represents the learned weights;

[0095] Step C3, constructing a visual information enhancement model VIME in the laser radar branch. The visual information enhancement model VIME is used to detect and track moving targets based on the laser radar data. Specifically, the detection and tracking capabilities of target motion are enhanced by using time series analysis, as shown in formula (5).

[0096] (5)

[0097] in, Indicates time The target motion model, Represents the lidar data of the current frame, represents the momentum term;

[0098] Step C4, establish the attention fusion mechanism of adaptive transformation learning, specifically based on extracting feature vectors from visual data and lidar data , and then use the weight matrix Calculate the attention weight as shown in formula (6):

[0099] (6)

[0100] in, is the normalized attention coefficient, It is used to convert weights into probability distribution to ensure that the sum of weights of each data source is 1, and then automatically adjust the influence of each perception data according to the actual situation;

[0101] Step C5, using the calculated attention coefficient to dynamically adjust the fusion ratio of visual data and lidar data to obtain processed visual and lidar fusion data.

[0102] like Figure 5 As shown, step D, according to the adjusted time-sharing path planning data and the processed vision and lidar fusion data, select from the execution path optimization and execution trajectory optimization to obtain the optimization result. The specific steps are as follows:

[0103] Step D1, data acquisition and analysis, obtaining traffic flow data, road status data and estimated delivery time data from the adjusted time-sharing path planning data, and then obtaining road obstacle data, traffic sign data and pedestrian data from the processed vision and lidar fusion data;

[0104] Step D2, decision combination, constructing a dynamic decision support model DDSS, the dynamic decision support model DDSS is used to use the data obtained in step D1 to analyze and predict the current traffic and environmental conditions in real time. The dynamic decision support model DDSS uses a machine learning algorithm to integrate and analyze the data obtained from step D1 and determine the optimal decision plan under the current conditions, as shown in formula (7).

[0105] (7)

[0106] in, Indicates at time Traffic status score, represents the data obtained from step D1, represents the smoothing coefficient;

[0107] Step D3, select to perform path optimization or trajectory optimization, and use the dynamic decision support model DDSS to evaluate and select to perform path optimization or trajectory optimization, wherein the path optimization is used to change the route to avoid congestion and reduce the execution time, and the trajectory optimization is used to adjust the vehicle's driving behavior, driving speed and obstacle avoidance strategy on the current route to adapt to the current road conditions and traffic conditions, as shown in formula (8),

[0108] (8)

[0109] in, represents the decision value, and the decision value To assess the current level of congestion, represents the decision factor weight obtained based on historical data training, Represent real-time environmental and traffic data characteristics;

[0110] Step D4, optimization execution, executes the selected path optimization or trajectory optimization, and the vehicle navigation module is also used for dynamic adjustment during the optimization execution process, as shown in formula (9),

[0111] (9)

[0112] in, represents the priority score of comprehensive path optimization and trajectory optimization, and are the priority scores for path optimization and trajectory optimization, and Represent the weights of path optimization and trajectory optimization respectively.

[0113] Step E: Use the feedback mechanism to monitor and adjust the optimization results in real time to complete the path optimization selection operation. The feedback loop control formula in the feedback mechanism is shown in formula (10):

[0114] (10)

[0115] in, represents a control input, the control input Used to adjust the speed and distance of the vehicle to adapt to speed changes and route adjustments caused by path optimization or trajectory optimization. Represents the difference between expectation and actual execution. , and They represent proportional gain, integral gain and differential gain respectively.

[0116] like Figure 2 As shown, the path optimization selection system based on time-sharing planning and radar-vision fusion includes a time-sharing path planning module, a time-sharing path planning adjustment module, a fusion processing module, an optimization selection module and a monitoring adjustment module; the time-sharing path planning module is used to perform time-sharing path planning based on the spatiotemporal dynamic graph model TSDG and obtain time-sharing path planning data; the time-sharing path planning adjustment module is used to dynamically adjust the time-sharing path planning data according to real-time traffic conditions using the graph neural network GNN and obtain the adjusted time-sharing path planning data; the fusion processing module is used to use the multi-sensory fusion learning framework MGLF to fuse the visual data with the laser radar data and obtain the processed visual and laser radar fusion data; the optimization selection module is used to select from the execution path optimization and execution trajectory optimization according to the adjusted time-sharing path planning data and the processed visual and laser radar fusion data and obtain the optimization result; the monitoring adjustment module is used to use the feedback mechanism to monitor and adjust the optimization result in real time and complete the path optimization selection operation.

[0117] Specifically, the specific fusion processing process of the fusion processing module includes extracting features of visual data and lidar data, constructing a deep deformable convolutional network in the visual branch, constructing a visual information enhancement model VIME in the lidar branch, and establishing an attention fusion mechanism for adaptive transformation learning.

[0118] Specifically, the specific optimization selection process of the optimization selection module includes data acquisition and analysis, decision combination, selection of execution path optimization or trajectory optimization, and execution of the selected path optimization or trajectory optimization.

[0119] In summary, the path optimization selection method and system based on time-sharing planning and radar-vision fusion of the present invention firstly performs time-sharing path planning based on the spatiotemporal dynamic graph model TSDG and obtains time-sharing path planning data, then uses the graph neural network GNN to dynamically adjust the time-sharing path planning data according to the real-time traffic conditions and obtains the adjusted time-sharing path planning data, then uses the multi-sensory fusion learning framework MGLF to fuse the visual data and the laser radar data and obtain the processed visual and laser radar fusion data, then selects from the execution path optimization and execution trajectory optimization according to the adjusted time-sharing path planning data and the processed visual and laser radar fusion data and obtains the optimization result, and finally uses the feedback mechanism to monitor and adjust the optimization result in real time and complete the path optimization selection operation; the present invention realizes the function of time-sharing path optimization selection, and also has the function of fusing the visual data and the laser radar data, and also has the function of quickly and accurately selecting and executing from the execution path optimization and execution trajectory optimization, and also has the function of real-time monitoring and adjustment of the optimization result, which can not only adapt to the needs of different application scenarios, but also utilize the advantages of path optimization and trajectory optimization to improve the efficiency and safety of path planning.

[0120] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A path optimization selection method based on time-sharing planning and radar-vision fusion, characterized by: The following steps are included: Step A: Perform time-sharing path planning based on the spatiotemporal dynamic graph model TSDG to obtain time-sharing path planning data. The specific steps are as follows: Step A1, dividing 24 hours a day into multiple time periods, wherein the time periods include morning peak, evening peak and night; Step A2, constructing a dynamic spatiotemporal grid map based on historical data and real-time data for each time period; Step A3, performing path search based on the dynamic spatiotemporal grid graph, as shown in formula (1), t(x)=g(x)+h(x)(1) Among them, t(x) represents the total estimated cost from the starting point to the end point through node x, g(x) represents the actual cost from the starting point to the current point, and h(x) represents the estimated cost from the current point to the end point; Step A4, constructing the long-term change pattern of traffic flow and predicting future traffic condition changes based on the graph neural network GNN; Step A5, after re-entering a new time period, the dynamic spatiotemporal grid map is updated based on the new data and the path is re-planned; Step B: Use the graph neural network GNN to dynamically adjust the time-sharing path planning data according to the real-time traffic conditions to obtain the adjusted time-sharing path planning data. The specific steps are as follows: Step B1, constructing a graph neural network GNN model, specifically constructing a graph neural network GNN model based on the traffic flow, vehicle speed and accident report data detected by the L4 unmanned parking inspection module installed inside the vehicle, wherein the road intersections and road sections are respectively represented as nodes and edges in the graph neural network GNN model, and then setting the node features and edge features in the graph neural network GNN model, the node features include the average vehicle speed and the average traffic density, and the edge features include the travel time, distance and current traffic status between two nodes; Step B2, update the node features and edge features in the graph neural network GNN model, as shown in formula (2): in, represents the node features of the next layer, represents the feature vector of node v at layer l, W (l) and B (l) Represents the learned weight and bias parameters, N(v) represents the set of neighbor nodes of node v, and ReLU represents the activation function; Step B3, using historical traffic data to train the graph neural network (GNN) model to learn the long-term change pattern of traffic flow, where the long-term change pattern includes the impact of seasonal changes and special events; Step B4, using real-time traffic data to update the node features and edge features in the graph neural network GNN model, and forward propagating through the graph neural network GNN to predict changes in traffic conditions in the short term in the future; Step C: Use the multi-sensory fusion learning framework MGLF to fuse the visual data and the lidar data to obtain the processed visual and lidar fusion data. The specific steps are as follows: Step C1, extracting visual data F vision and lidar data F lidar The features of the visual data F are then adjusted by the fusion coefficient α1 vision and lidar data F lidar In the Ray Vision Fusion feature F combined The proportion in is as shown in formula (3), F combined =α1·F vision +(1-α1)·F lidar (3); Step C2, constructing a deep deformable convolutional network in the visual branch, wherein the deep deformable convolutional network is used to detect objects in visual data, specifically adjusting the shape of the convolution kernel to adapt to the actual shape change of the object, as shown in formula (4), Among them, f(x) represents the input data x pi Apply weight ω i The weighted sum of pi represents the input data at the i-th convolution position, ω i represents the learned weights; Step C3, constructing a visual information enhancement model VIME in the laser radar branch. The visual information enhancement model VIME is used to detect and track moving targets based on the laser radar data. Specifically, the detection and tracking capabilities of target motion are enhanced by using time series analysis, as shown in formula (5). m(t)=β·m(t-1)+(1-β)·l t (5) Among them, m(t) represents the target motion model at time t, l t represents the lidar data of the current frame, and β represents the momentum term; Step C4, establish the attention fusion mechanism of adaptive transformation learning, specifically, extract the feature vector h from the visual data and the lidar data, and then use the weight matrix ω to calculate the attention weight, as shown in formula (6), α2=softmax(ω·h)(6) Among them, α2 is a normalized attention coefficient, which is used to convert the weight into a probability distribution to ensure that the sum of the weights of each data source is 1, and then automatically adjust the influence of each perception data according to the actual situation; Step C5, dynamically adjusting the fusion ratio of visual data and lidar data using the calculated attention coefficient α2 to obtain processed visual and lidar fusion data; Step D, select from execution path optimization and execution trajectory optimization according to the adjusted time-sharing path planning data and the processed vision and lidar fusion data to obtain the optimization result. The specific steps are as follows: Step D1, data acquisition and analysis, obtaining traffic flow data, road status data and estimated delivery time data from the adjusted time-sharing path planning data, and then obtaining road obstacle data, traffic sign data and pedestrian data from the processed vision and lidar fusion data; Step D2, decision combination, constructing a dynamic decision support model DDSS, the dynamic decision support model DDSS is used to use the data obtained in step D1 to analyze and predict the current traffic and environmental conditions in real time. The dynamic decision support model DDSS uses a machine learning algorithm to integrate and analyze the data obtained from step D1 and determine the optimal decision plan under the current conditions, as shown in formula (7). S(t)=λ·S(t-1)+(1-λ)·Y(t)(7) Wherein, S(t) represents the traffic status score at time t, Y(t) represents the data obtained from step D1, and λ represents the smoothing coefficient; Step D3, select to perform path optimization or trajectory optimization, and use the dynamic decision support model DDSS to evaluate and select to perform path optimization or trajectory optimization, wherein the path optimization is used to change the route to avoid congestion and reduce the execution time, and the trajectory optimization is used to adjust the vehicle's driving behavior, driving speed and obstacle avoidance strategy on the current route to adapt to the current road conditions and traffic conditions, as shown in formula (8), Where D(X) represents the decision value, and the decision value D(X) is used to evaluate the current congestion level, c i represents the decision factor weight obtained based on historical data training, x i Representing real-time environmental and traffic data characteristics; Step D4, optimization execution, executes the selected path optimization or trajectory optimization, and the vehicle navigation module is also used for dynamic adjustment during the optimization execution process, as shown in formula (9), P=ω p ·p path +oh t ·p traj (9) Among them, P represents the priority score of comprehensive path optimization and trajectory optimization, p path and p traj are the priority scores for path optimization and trajectory optimization, ω p and ω t Represent the weights of path optimization and trajectory optimization respectively; Step E: Use the feedback mechanism to monitor and adjust the optimization results in real time to complete the path optimization selection task.

2. The path optimization selection method based on time-sharing planning and radar-visual fusion according to claim 1 is characterized in that: Step E: Use the feedback mechanism to monitor and adjust the optimization results in real time to complete the path optimization selection operation. The feedback loop control formula in the feedback mechanism is shown in formula (10): Wherein, U(t) represents the control input, which is used to adjust the speed and distance of the vehicle to adapt to the speed change and route adjustment caused by path optimization or trajectory optimization, e(t) represents the difference between the expected and actual execution, and K P , K i and K d They represent proportional gain, integral gain and differential gain respectively.

3. A path optimization selection system based on time-sharing planning and radar-vision fusion, wherein the specific operation process of the path optimization selection system is based on the path optimization selection method according to claim 2, characterized in that: It includes a time-sharing path planning module, a time-sharing path planning adjustment module, a fusion processing module, an optimization selection module and a monitoring adjustment module; The time-sharing path planning module is used to perform time-sharing path planning based on the spatiotemporal dynamic graph model TSDG and obtain time-sharing path planning data; The time-sharing path planning adjustment module is used to dynamically adjust the time-sharing path planning data according to the real-time traffic condition changes using the graph neural network GNN and obtain the adjusted time-sharing path planning data; The fusion processing module is used to use the multi-sensory fusion learning framework MGLF to fuse the visual data and the laser radar data and obtain the processed visual and laser radar fusion data; The optimization selection module is used to select and obtain the optimization result from the execution path optimization and the execution trajectory optimization according to the adjusted time-sharing path planning data and the processed vision and laser radar fusion data; The monitoring and adjustment module is used to utilize a feedback mechanism to perform real-time monitoring and adjustment on the optimization results and complete the path optimization selection operation.

4. The path optimization selection system based on time-sharing planning and radar-vision fusion according to claim 3 is characterized in that: The specific fusion processing process of the fusion processing module includes extracting features of visual data and lidar data, constructing a deep deformable convolutional network in the visual branch, constructing a visual information enhancement model VIME in the lidar branch, and establishing an attention fusion mechanism for adaptive transformation learning.

5. The path optimization selection system based on time-sharing planning and radar-vision fusion according to claim 3 is characterized in that: The specific optimization selection process of the optimization selection module includes data acquisition and analysis, decision combination, selection execution optimization and execution of selected optimization.

Images