Inference decision-making method and system based on vehicle-road-network collaborative strategy

Abstract

The invention discloses an inference decision-making method and system based on a vehicle-road-network cooperation strategy, and belongs to the technical field of vehicle-road-network cooperation. Comprising the steps that a vehicle side node collects a vehicle side road traffic image, target detection is carried out on the vehicle side road traffic image through an optimal target detection algorithm deployed in an optimal hardware combination, and a vehicle side target detection result is obtained; the roadside edge node collects a road traffic image of a roadside, and performs target detection on the road traffic image based on the optimal edge reasoning mode to obtain a roadside target detection result; the cloud server gathers the vehicle side target detection result and the road side target detection result so as to carry out reasoning and decision making; the optimal hardware combination, the optimal target detection algorithm and the optimal edge reasoning mode are generated by optimizing a preset search space for the cloud server by taking execution time minimization as a target. Flexible switching and verification of different execution strategies or calculation modes can be realized, and the actual application effect is improved; the problem of low task processing efficiency in existing traffic management is solved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-road-network collaborative technology, and in particular to a reasoning decision method and system based on a vehicle-road-network collaborative strategy. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] With the acceleration of urbanization, the complexity and scale of urban transportation systems continue to expand. Urban transportation is not only an important lifeline for economic and social development, but also a key link in the construction of smart cities. Most of the existing traffic management and optimization technologies rely on simulation environments for testing. Although they can simulate traffic pressure to a certain extent, they have many problems and shortcomings when facing actual traffic scenarios. There is a lack of effective verification of roadside perception units, which limits their effectiveness and scope of application.

[0004] Specifically, the existing vehicle-road-network simulation implementation technology and roadside perception units have the following problems:

[0005] Existing technologies rely on simulation systems for traffic management and optimization. When processing complex traffic data, these simulation systems are prone to problems such as over-idealization of task processing, insufficient optimization space for algorithms for actual reasoning, and unreasonable allocation of computing resources in resource-constrained actual environments due to their single computing units and abundant resources. The computing power of the simulation system is limited, making it difficult to process and analyze large amounts of traffic data in real time. The effect may be good in a simulation environment, but in actual scene applications, there may be an inability to quickly respond to traffic changes, resulting in untimely traffic signal scheduling and emergency response. Summary of the invention

[0006] In order to address the deficiencies in the prior art, the present invention provides an inference decision-making method and system based on a vehicle-road-network collaborative strategy to solve the problems of low task processing efficiency, data transmission congestion and insufficient data storage capacity in existing traffic management systems in actual scenarios.

[0007] In a first aspect, the present invention provides a reasoning decision system based on a vehicle-road-network collaborative strategy;

[0008] A reasoning decision system based on a vehicle-road-network collaborative strategy, comprising:

[0009] The vehicle-side node is used to collect road traffic images on the vehicle side, and perform target detection on the road traffic images on the vehicle side through the optimal target detection algorithm deployed on the optimal hardware combination to obtain the vehicle-side target detection results;

[0010] The roadside edge node is used to collect road traffic images on the roadside, and perform target detection on the road traffic images on the roadside based on the optimal edge inference mode to obtain roadside target detection results;

[0011] The cloud server is used to aggregate the vehicle-side object detection results and the road-side object detection results for reasoning and decision making;

[0012] Among them, the optimal hardware combination, the optimal target detection algorithm and the optimal edge inference mode are generated by optimizing the preset search space for the cloud server with the goal of minimizing execution time.

[0013] In a second aspect, the present invention provides a reasoning decision method based on a vehicle-road-network collaborative strategy;

[0014] A reasoning decision method based on a vehicle-road-network collaborative strategy, comprising:

[0015] Obtaining vehicle-side target detection results and road traffic images on the road side, and performing target detection on the road traffic images on the road side based on the optimal edge inference mode to obtain road side target detection results;

[0016] Transmitting the vehicle-side target detection results and the road-side target detection results to the cloud server to make inference decisions based on the vehicle-side target detection results and the road-side target detection results;

[0017] The vehicle-side target detection result is generated by collecting the road traffic image on the vehicle side by the vehicle-side node and performing target detection on the road traffic image on the vehicle side by using the optimal target detection algorithm deployed on the optimal hardware combination;

[0018] The optimal hardware combination, the optimal target detection algorithm and the optimal edge reasoning mode are generated by optimizing the preset search space for the cloud server with the goal of minimizing execution time.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. The technical solution provided by the present invention is to equip the vehicle-side node with the Ascend computing core, which can provide one or more hardware combinations of CPU, DVPP, and NPU for target detection, and can simulate pure arm computing and multi-core heterogeneous computing. Different hardware combinations correspond to different computing effects, so that the vehicle-side node has hardware selectivity and can be selected according to actual applications, which conforms to the characteristics of actual applications; the roadside edge node is equipped with AMD NPU, which has two reasoning modes of Latency and Throughput, and can provide different preference options for the execution strategy of computing tasks, thereby improving the reliability and practicality of the system and avoiding idealized or single task execution environment.

[0021] 2. The technical solution provided by the present invention constructs the search space and optimizes the execution strategy according to the hardware configuration of the vehicle-side node, the edge reasoning mode of the roadside edge node and the target detection algorithm, which can meet the actual resource requirements of the road traffic management system and be close to the real environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0023] Figure 1 A schematic diagram of the hardware structure of the reasoning decision system based on the vehicle-road-network collaborative strategy provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of an execution strategy of a vehicle-side node provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of an execution strategy of a roadside edge node provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0027] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0028] Embodiment 1

[0029] Next, combine Figure 1-Figure 3 , a reasoning decision system based on the vehicle-road-network collaborative strategy disclosed in this embodiment is described in detail.

[0030] The reasoning decision system based on the vehicle-road-network collaborative strategy includes a vehicle-side node, a roadside edge node and a cloud server. The vehicle-side node includes a first image acquisition module, a vehicle-computer control module, a vehicle-side communication module and a vehicle-side computing module. The vehicle-side computing module includes a CPU, an NPU and a DVPP; the roadside edge node includes a roadside communication module, a second image acquisition module and an inference module. The inference module is an AMD NPU, and the edge inference mode of the inference module includes a Throughput mode and a Latency mode; a cloud resource pool is deployed on the cloud server, and the cloud resource pool includes a cloud communication module, a storage module, a computing module and a data push module.

[0031] The first image acquisition module is used to collect road traffic images on the vehicle side. The vehicle-side computing module performs target detection on the road traffic images on the vehicle side by using the optimal target detection algorithm deployed on the optimal hardware combination, obtains the vehicle-side target detection results and transmits them to the roadside edge node through the vehicle-side communication module and the roadside communication module.

[0032] The second image acquisition module is used to collect road traffic images on the road side. The inference module performs target detection on the road traffic images on the road side based on the optimal edge inference mode to obtain road side target detection results; the vehicle side target detection results are received through the road side communication module and sent to the cloud server together with the road side target detection results.

[0033] The cloud server collects the vehicle-side target detection results and the road-side target detection results through the cloud communication module and stores them in the storage module; processes the vehicle-side target detection results and the road-side target detection results through the calculation module to obtain the inference decision results; and transmits the inference decision results to the vehicle control module through the data push module, and the vehicle control module controls the vehicle according to the inference decision results.

[0034] In this embodiment, the reasoning decision can be prohibiting passing, identifying unsafe behaviors, collaborative lane changing, etc. Its specific implementation depends on existing algorithms such as the YOLO target detection algorithm, etc. This embodiment does not make any improvements to it, so it will not be repeated here.

[0035] Furthermore, the vehicle-side node is equipped with the Ascend series, and the hardware combination includes one or more of CPU, NPU and DVPP.

[0036] The implementation of vehicle-side node target detection includes preprocessing, target detection, and postprocessing. If the above three steps are all processed on the CPU, the advantages are small communication delay and a wide range of tasks that the CPU can process. The disadvantage is that the CPU performance is poor, resulting in a long calculation time. For the NPU+CPU heterogeneous hardware combination, the NPU performs reasoning and the CPU performs the other two steps. The advantage is that the NPU has strong computing power and the CPU is suitable for processing complex postprocessing. The disadvantage is that there is a certain communication time overhead for the data between the NPU and the CPU. For the NPU+CPU+DVPP hardware combination, target detection is performed by the NPU, preprocessing is performed by the DVPP, and postprocessing is performed by the CPU. The advantage is that the DVPP hardware is dedicated to preprocessing and has good performance in preprocessing operations. The NPU has strong reasoning computing power and the CPU is convenient for processing complex data. The disadvantage is that the communication time between the hardware caused by the three hardware heterogeneities is further increased.

[0037] Therefore, considering the inconsistent weights of different algorithm models, the different computing power of different hardware, and the communication delay between different hardware platforms, different hardware combinations will have different computing effects and are difficult to directly quantify and evaluate.

[0038] The roadside computing module is AMD NPU, which has two modes: Latency and Throughput. The Throughput mode supports the parallel execution of multiple inference sessions (up to 4), and the computing power of each session is up to 2TOPS, which is suitable for executing multiple latency-insensitive tasks to maximize throughput; the Latency mode supports the execution of one inference session, and the computing power of the session is up to 10TOPS, which is suitable for executing a single latency-sensitive task; this provides different preference options for the execution strategy of roadside computing tasks.

[0039] Therefore, in order to select a hardware combination, mode and reasoning algorithm that is more suitable for the actual road traffic management scenario, as an implementation method, the preset search space is optimized to generate the optimal hardware combination, optimal target detection algorithm and optimal edge reasoning mode with the goal of minimizing the execution time, specifically including:

[0040] (1) Construct a search space based on the hardware configuration of the vehicle-side node, the edge inference mode of the roadside edge node, and the optional target detection algorithm.

[0041] Exemplarily, in this system, the target detection for road traffic images is specifically performed in the following optional manners:

[0042] ①Target detection algorithm selection: YOLO, SSD, Faster R-CNN;

[0043] ②Choice of edge inference mode for roadside edge nodes: latency mode or throughput mode;

[0044] ③ Implementation forms of heterogeneous computing combinations of vehicle nodes: pure ARM CPU, two computing core heterogeneities of CPU+NPU, and three core heterogeneities of CPU+NPU+DVPP.

[0045] Based on the above optional methods, the following search space is constructed:

[0046] [[yolo,ssd,Faster R-CNN],

[0047] [latency, throughput],

[0048] [cpu,cpu+npu,cpu+npu+dvpp]].

[0049] The search space consists of three dimensions of options. The first dimension includes three algorithm options (YOLO, SSD, and Faster R-CNN). The second dimension includes two performance indicator options (latency and throughput). Latency represents the latency mode, and throughput represents the throughput mode. The third dimension includes three hardware configuration options (CPU, CPU+NPU, and CPU+NPU+DVPP).

[0050] (2) Based on the search space, generate a variety of different execution strategies.

[0051] Specifically, according to the search space in step (1), by combining the options in each dimension, a search space covering multiple possibilities is formed.

[0052] For example, the options of the first dimension and the second dimension are combined to generate 6 combinations; then the options of the first dimension and the third dimension are combined to generate 9 combinations. Finally, these combinations are integrated together to form a search space containing 15 possible combinations. This search space can provide a comprehensive configuration solution for system optimization under multi-dimensional conditions.

[0053] (3) With the goal of minimizing execution time, Monte Carlo tree search is introduced for optimization to determine the optimal execution strategy, namely the optimal hardware combination, the optimal target detection algorithm, and the optimal edge inference mode.

[0054] Vehicle-road-network collaboration needs to achieve early warning as quickly as possible while controlling the accuracy, that is, the minimum time from detecting the target to feedback results, where the accuracy is ≥90% and the overall detection to feedback time is ≤1s.

[0055] The combination in the execution strategy represents the computing method, and the hardware, algorithm, mode, and communication module selected in each combination. Assume that there are n computing methods and each computing method has m modules. In this embodiment, T ij Indicates the calculation time of the jth module under the i-th calculation method.

[0056] Exemplarily, the specific process of step (3) is as follows:

[0057] Step 1: Construct a training set through the collected real road images, and use the training set to train the target detection algorithms YOLO, SSD, and Faster R-CNN until the accuracy rate is ≥ 90%.

[0058] Step 2: Calculate the maximum module execution time (including waiting time and communication time) of each calculation method as the execution time of the calculation method, expressed as:

[0059] For each calculation method Mi , calculate the maximum execution time of all its modules:

[0060] M i =max(T i1 ,T i2 ,...,T im );

[0061] Among them, M i Indicates the maximum module execution time of the i-th calculation method.

[0062] Step 3: Select the calculation method for the minimum execution time, expressed as:

[0063] M min =min(M1,M2,...,M n );

[0064] In summary, the minimum execution time is expressed as:

[0065] M min =min(max(T i1 ,T i2 ,...,T im ),...,max(T n1 ,T n2 ,...,T nm )).

[0066] Step 4: With the goal of minimizing the execution time, the Monte Carlo tree search algorithm is used to iteratively optimize the selection of the optimal execution strategy until the set termination condition is reached. The specific steps are as follows:

[0067] Step 401, starting from the root node (initial state), select layer by layer down along the search tree based on the selection strategy, each layer corresponds to a dimension in the search space (such as algorithm, reasoning mode or hardware configuration).

[0068] In this embodiment, the selection strategy is to use the UCB1 formula and make a trade-off in combination with historical feedback. The formula is as follows:

[0069]

[0070] Where W i is the cumulative reward of the current node (negative value of execution time, the larger the reward, the better), N i Indicates the number of times the node is visited, N p represents the number of visits to the parent node, and C represents the balance factor between exploration and utilization, which is used to control the breadth and depth of the search.

[0071] Through step 401, MCTS selects a currently optimal leaf node, which may represent a certain algorithm (such as YOLO) combined with a certain reasoning mode (such as Throughput) and hardware configuration (such as CPU+NPU).

[0072] Step 402: If the current node has not been fully explored, a new node is expanded. The new node represents the selection of a new hardware configuration or reasoning mode. For example, if the current node selects YOLO and Throughput mode, but the CPU+NPU+DVPP combination has not been explored, then expansion is performed at this stage.

[0073] Each new node generated by the expansion represents an unexplored strategy combination, for example: [YOLO, Throughput, CPU+NPU+DVPP]. At this point, MCTS begins to incorporate this new strategy into the search tree.

[0074] Step 403: After the new strategy combination is expanded, MCTS will simulate the execution of the combination and calculate the execution performance of the combination.

[0075] In this step, MCTS evaluates the execution effect of the new node and obtains a clear execution time as the simulation result.

[0076] Specifically, the execution time of the strategy combination is calculated. The execution time of each strategy combination consists of multiple modules, including reasoning time, communication time, and waiting time. The simulated execution time is used as performance feedback and recorded.

[0077] Step 404: After the simulation is completed, MCTS will backtrack the evaluation results along the path and update the cumulative reward values ​​and visit times of all the nodes passed.

[0078] Specifically, the evaluation result will affect all W passing through the node i and N i By backtracking, MCTS can update the impact of each node on future choices, thereby gradually converging to a better execution strategy in the next round of search.

[0079] By backtracking, MCTS can update the impact of each node on future choices, thereby gradually converging to a better execution strategy in the next round of search.

[0080] In this embodiment, the termination condition of MCTS is that the cumulative reward value of the current node reaches a certain threshold, indicating that the optimal strategy has been found.

[0081] In some embodiments, the termination condition of MCTS may also be reaching a predetermined maximum number of searches.

[0082] Through Monte Carlo tree search, the system can dynamically explore complex search spaces and gradually find the optimal execution strategy combination under various hardware and inference modes, thereby optimizing execution efficiency.

[0083] Embodiment 2

[0084] This embodiment discloses a reasoning decision method based on a vehicle-road-network collaborative strategy, comprising the following steps:

[0085] S1. Obtain vehicle-side target detection results and road traffic images on the road side, and perform target detection on the road traffic images on the road side based on the optimal edge inference mode to obtain road-side target detection results.

[0086] S2. Transmitting the vehicle-side target detection results and the road-side target detection results to the cloud server to make inference decisions based on the vehicle-side target detection results and the road-side target detection results.

[0087] Among them, the vehicle-side target detection result is that the vehicle-side node collects the road traffic image on the vehicle side, and generates target detection for the road traffic image on the vehicle side through the optimal target detection algorithm deployed on the optimal hardware combination; the optimal hardware combination, the optimal target detection algorithm and the optimal edge inference mode are used for the cloud server to optimize and generate the preset search space with the goal of minimizing the execution time.

[0088] As an implementation method, optimizing the preset search space with the goal of minimizing execution time to generate the optimal hardware combination, the optimal target detection algorithm, and the optimal edge reasoning mode specifically includes:

[0089] (1) Construct a search space based on the hardware configuration of the vehicle-side node, the edge reasoning mode of the roadside edge node, and the optional target detection algorithm;

[0090] (2) With the goal of minimizing execution time, the Monte Carlo tree search algorithm is used to optimize and determine the optimal hardware combination, optimal target detection algorithm, and optimal edge inference mode.

[0091] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A reasoning decision system based on vehicle-road-network collaborative strategy, characterized by: include: The vehicle-side node is used to collect road traffic images on the vehicle side, and perform target detection on the road traffic images on the vehicle side through the optimal target detection algorithm deployed on the optimal hardware combination to obtain the vehicle-side target detection results; The roadside edge node is used to collect road traffic images on the roadside, and perform target detection on the road traffic images on the roadside based on the optimal edge inference mode to obtain roadside target detection results; The cloud server is used to aggregate the vehicle-side object detection results and the road-side object detection results for reasoning and decision making; Among them, the optimal hardware combination, the optimal target detection algorithm and the optimal edge inference mode are generated by optimizing the preset search space for the cloud server with the goal of minimizing execution time.

2. The reasoning decision system based on the vehicle-road-network collaborative strategy as claimed in claim 1, characterized in that: Optimizing the preset search space with the goal of minimizing execution time to generate the optimal hardware combination, the optimal target detection algorithm and the optimal edge reasoning mode specifically includes: Construct a search space based on the hardware configuration of the vehicle-side node, the edge reasoning mode of the roadside edge node, and the optional target detection algorithm; With the goal of minimizing execution time, the Monte Carlo tree search algorithm is used for optimization to determine the optimal hardware combination, optimal target detection algorithm, and optimal edge inference mode.

3. The reasoning decision system based on vehicle-road-network collaborative strategy as claimed in claim 2, characterized in that: The execution strategy includes a hardware combination and a corresponding target detection algorithm or an edge reasoning mode and a corresponding target detection algorithm.

4. The reasoning decision system based on vehicle-road-network collaborative strategy as claimed in claim 2, characterized in that: The maximum running time of a single module in each execution strategy is taken as the execution time.

5. The reasoning decision system based on vehicle-road-network collaborative strategy as claimed in claim 1, characterized in that: The hardware combination of the vehicle-side node includes one or more of a CPU, an NPU and a DVPP.

6. The reasoning decision system based on vehicle-road-network collaborative strategy as claimed in claim 1, characterized in that: The edge reasoning modes of the roadside edge nodes include a Throughput mode and a Latency mode.

7. The reasoning decision system based on vehicle-road-network collaborative strategy as claimed in claim 1, characterized in that: Before optimizing the preset search space with the goal of minimizing the execution time, the method also includes: constructing a training set through the collected real road images, and training the target detection algorithm using the training set.

8. The reasoning decision method based on the vehicle-road-network collaborative strategy is characterized by: include: Obtaining vehicle-side target detection results and road traffic images on the road side, and performing target detection on the road traffic images on the road side based on the optimal edge inference mode to obtain road side target detection results; Transmitting the vehicle-side target detection results and the road-side target detection results to the cloud server to make inference decisions based on the vehicle-side target detection results and the road-side target detection results; The vehicle-side target detection result is generated by collecting the road traffic image on the vehicle side by the vehicle-side node and performing target detection on the road traffic image on the vehicle side by using the optimal target detection algorithm deployed on the optimal hardware combination; The optimal hardware combination, the optimal target detection algorithm and the optimal edge reasoning mode are generated by optimizing the preset search space for the cloud server with the goal of minimizing execution time.

9. The reasoning decision method based on the vehicle-road-network collaborative strategy according to claim 8, characterized in that: Optimizing the preset search space with the goal of minimizing execution time to generate the optimal hardware combination, the optimal target detection algorithm and the optimal edge reasoning mode specifically includes: Construct a search space based on the hardware configuration of the vehicle-side node, the edge reasoning mode of the roadside edge node, and the optional target detection algorithm; With the goal of minimizing execution time, the Monte Carlo tree search algorithm is used for optimization to determine the optimal hardware combination, optimal target detection algorithm, and optimal edge inference mode.

10. The reasoning decision method based on vehicle-road-network collaborative strategy according to claim 8, characterized in that: Before optimizing the preset search space with the goal of minimizing the execution time, the method also includes: constructing a training set through the collected real road images, and training the target detection algorithm using the training set.

Images