Vehicle-mounted inference task execution method, device and vehicle

Abstract

This application provides a method, device, and vehicle for executing an inference task. The method includes: responding to an execution request for an inference task sent by the vehicle's application layer, sending task data for the inference task to a target roadside node; the task data includes a model identifier for executing the inference task and inference task input data; the target roadside node is a roadside node pre-determined based on the vehicle's driving behavior data for executing the vehicle's inference task, and the target roadside node pre-loads a target inference model for executing the inference task; and receiving the inference result returned by the target roadside node. This method aims to improve the real-time performance of inference task execution in vehicle-road cooperative inference scenarios.

Description

Technical Field

[0001] This application relates to the field of vehicle data processing, and in particular to a method, device and vehicle for performing in-vehicle inference tasks. Background Technology

[0002] With the rapid development of the Intelligent Connected Vehicle (ICV) industry, in-vehicle artificial intelligence (AI) inference systems face a structural contradiction between limited computing power and high real-time requirements. Vehicle-to-Everything (V2X) communication technologies, represented by cellular vehicle-to-everything (V2X) networks, enable vehicles to establish low-latency, highly reliable communication connections with roadside nodes, providing a physical basis for offloading computationally intensive inference tasks to roadside nodes.

[0003] Currently, when a vehicle is processing an inference task, it typically sends the first frame of real-time perception data processed by the vehicle using that node, along with a model preloading instruction, to the roadside node via the V2X PC5 interface when the vehicle enters the communication range of the nearest roadside node. This allows the roadside node to respond to the instruction, load the model, and then perform the inference task based on the perception data after the model is loaded. Finally, the obtained inference results are returned to the vehicle.

[0004] However, loading parameters for large inference models typically takes over 200ms, while the effective communication window of the V2X PC5 interface is only about 1-3 seconds (taking a vehicle speed of 60km / h and a coverage radius of 300m as an example). This results in existing solutions consuming a significant amount of time for model loading and having insufficient effective inference time, thus affecting the real-time performance of inference tasks. Therefore, a new in-vehicle inference task execution technology is urgently needed to improve the real-time performance of inference tasks. Summary of the Invention

[0005] This application provides a method, device, and vehicle for executing vehicle-mounted inference tasks, which aims to improve the real-time performance of inference tasks in vehicle-road cooperative inference scenarios.

[0006] In a first aspect, embodiments of this application provide a method for executing an in-vehicle inference task, applied to a vehicle control unit, comprising:

[0007] In response to the execution request of the inference task sent by the application layer of the vehicle, the task data of the inference task is sent to the target roadside node;

[0008] Receive the inference results returned by the target roadside node;

[0009] The task data includes the model identifier for performing the inference task and the inference task input data;

[0010] The target roadside node is a roadside node pre-determined based on the vehicle's driving behavior data to perform the vehicle's reasoning task. The target roadside node is pre-loaded with the target reasoning model for performing the reasoning task.

[0011] In one possible implementation, before sending the inference task to the target roadside node, the method further includes:

[0012] During vehicle operation, the target roadside node to be used for reasoning tasks is determined based on the driving behavior data.

[0013] A model preloading instruction is sent to the target roadside node to enable the target roadside node to preload the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model.

[0014] In one possible implementation, determining the target roadside node to be used for the reasoning task based on the driving behavior data includes:

[0015] Based on the driving behavior data, the system predicts multiple roadside nodes that the vehicle will pass through in a future preset time period and the probability of reaching each roadside node; the probability of reaching represents the confidence that the vehicle can reach the corresponding roadside node within the preset time period.

[0016] The roadside node with the highest arrival probability among the plurality of roadside nodes is determined as the target roadside node;

[0017] The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0018] In one possible implementation, sending the task data of the inference task to the target roadside node includes:

[0019] When the arrival probability of the target roadside node reaches a preset threshold, the task data is sent to the target roadside node.

[0020] In one possible implementation, before sending the model preloading instruction to the target roadside node, the method further includes:

[0021] Based on the driving behavior data, obtain the latest time that the vehicle is expected to arrive at the target roadside node;

[0022] The latest time is added to the model preloading instruction; the latest time is used to trigger the target roadside node to delete the preloaded data.

[0023] Secondly, embodiments of this application provide a method for executing an on-board inference task, applied to a roadside node, including:

[0024] Receive task data for an inference task sent by a vehicle, wherein the task data includes a model identifier for performing the inference task and inference task input data;

[0025] The inference task input data is input into the pre-loaded target inference model corresponding to the model identifier to perform inference and obtain the inference result;

[0026] The reasoning result is returned to the vehicle;

[0027] The target inference model is an inference model preloaded under the trigger of a model preload instruction sent before the vehicle sends the task data.

[0028] In one possible implementation, before receiving the task data for the inference task sent by the vehicle, the method further includes:

[0029] Receive a model preloading instruction sent by the vehicle, wherein the model preloading instruction includes the model identifier;

[0030] According to the model preloading instruction, the structure and parameters of the target inference model corresponding to the model identifier are loaded.

[0031] In one possible implementation, the model preloading instruction also includes the latest time that the vehicle is expected to arrive at the target roadside node;

[0032] If the task data is not received before the latest time, the preloaded data corresponding to the target inference model is deleted.

[0033] Thirdly, embodiments of this application provide a method for executing an in-vehicle inference task, applied to a vehicle control unit, comprising:

[0034] In response to the inference task execution request sent by the application layer of the vehicle, the first task data of the inference task is sent to the first roadside node; the first task data includes the vehicle's identifier, the model identifier for executing the inference task, the inference task input data, and the task execution strategy. The task execution strategy includes the node identifiers and execution order of all target roadside nodes executing the inference task, as well as the model fragments to be executed for each target roadside node; the first roadside node is the first of at least two target roadside nodes pre-determined based on the vehicle's driving behavior data for executing the vehicle's inference task, and the first roadside node pre-loads the first model fragment of the target inference model for executing the inference task;

[0035] Receive the inference results returned by the second roadside node or other roadside nodes, wherein the second roadside node preloads the second model fragment of the target inference model for performing the inference task.

[0036] In one possible implementation, before sending the first task data of the inference task to the first roadside node, the method further includes:

[0037] During vehicle operation, the task execution strategy is determined based on the driving behavior data;

[0038] A model preloading instruction is sent to each target roadside node indicated by the task execution strategy, so that each target roadside node preloads different model fragments of the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model and the identifier of the corresponding model fragment.

[0039] In one possible implementation, determining the task execution strategy based on the driving behavior data includes:

[0040] Based on the driving behavior data, the system predicts multiple roadside nodes that the vehicle will pass through within a future preset time period and the probability of reaching each roadside node; the probability of reaching the roadside node represents the confidence level that the vehicle can reach the corresponding roadside node within the future preset time period.

[0041] Based on the arrival probability of each roadside node, at least two target roadside nodes are determined from the plurality of roadside nodes, and the model loading requirements for each target roadside node are determined.

[0042] Based on the model loading requirements of each target roadside node, the task execution strategy is obtained;

[0043] The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0044] In one possible implementation, determining at least two target roadside nodes from the plurality of roadside nodes based on the arrival probability of each roadside node, and determining the model loading requirements for each target roadside node, includes:

[0045] Roadside nodes whose arrival probability is greater than the first preset threshold are identified as intermediate roadside nodes;

[0046] The target inference model is clustered into multiple model subgraphs according to operator type, and each model subgraph includes at least one model piece of the target inference model;

[0047] Based on the node type of each intermediate roadside node, the operator type of each model subgraph, and the preset computing power adaptation mapping table, at least two target roadside nodes and the model subgraph matched by each target roadside node are determined from all intermediate roadside nodes; wherein, the preset computing power adaptation mapping table represents the computational efficiency of different types of roadside nodes when executing different types of operators.

[0048] The model subgraph matched for each target roadside node is determined as the model loading requirement for the target roadside node.

[0049] In one possible implementation, sending the first task data of the inference task to the first roadside node includes:

[0050] When the arrival probability of the first roadside node reaches the second preset threshold, the first task data is sent to the first roadside node.

[0051] In one possible implementation, the method further includes:

[0052] Real-time acquisition of the load of each target roadside node;

[0053] When the load of any target roadside node exceeds the preset load, the inference tasks that are not completed at the target roadside node will be migrated to other roadside nodes for execution.

[0054] In one possible implementation, the method further includes:

[0055] Obtain the semantic summary fingerprint sent by the target roadside node; the semantic summary fingerprint is obtained by statistical feature extraction based on the intermediate inference results output by the model segment corresponding to the target roadside node; the model segment is the inference unit other than the last inference unit in the target inference model;

[0056] Calculate the distance between the semantic summary fingerprint and the pre-stored fingerprint reference distribution;

[0057] If the distance is less than or equal to a preset first distance threshold, a verification pass command is sent to the target roadside node; the verification pass command is used to trigger the target roadside node to continue the execution of the inference task based on the intermediate inference result;

[0058] If the distance is greater than the first distance threshold and less than the preset second distance threshold, a re-inference request is sent to the target roadside node; the re-inference request is used to instruct the target roadside node to re-execute the inference task based on the corresponding task data;

[0059] If the distance is greater than or equal to the second distance threshold, the inference task of the target roadside node will be migrated to other roadside nodes for execution.

[0060] In one possible implementation, the semantic summary fingerprint includes at least one of the mean, variance, quantiles, median, and sparse feature entropy of the intermediate inference results.

[0061] Fourthly, embodiments of this application provide a method for executing an in-vehicle inference task, applied to a first roadside node, including:

[0062] The system receives first task data for an inference task sent by a vehicle. The first task data includes the vehicle's identifier, the model identifier for executing the inference task, inference task input data, and a task execution strategy. The task execution strategy includes the node identifiers and execution order of all target roadside nodes executing the inference task, as well as the model fragments executed for each target roadside node.

[0063] According to the execution order, the input data of the inference task is input into the first model fragment of the target inference model corresponding to the pre-loaded model identifier to obtain the first inference result;

[0064] According to the task execution strategy, the second task data of the inference task is sent to the second roadside node. The second task data includes the vehicle identifier, the first inference result, the model identifier, and the task execution strategy, so that the second roadside node continues to execute the inference task.

[0065] The first model slice is the first slice of the target inference model preloaded under the trigger of a model preload instruction sent before the vehicle sends the first task data.

[0066] In one possible implementation, prior to receiving the first task data sent by the vehicle, the method further includes:

[0067] Receive a model preloading instruction sent by the vehicle, the model preloading instruction including the model identifier and the identifier of the first model segment;

[0068] According to the model preloading instruction, the structure and parameters of the first model segment of the target inference model corresponding to the model identifier are loaded.

[0069] Fifthly, embodiments of this application provide a method for executing an onboard inference task, applied to a second roadside node, comprising:

[0070] The system receives second task data for an inference task sent by a first roadside node. The second task data includes vehicle identification, first inference result, model identification, and execution strategy. The first inference result is the output tensor obtained by executing the inference task on the first model segment of the target inference model corresponding to the model identification. The task execution strategy includes the node identification and execution order of all target roadside nodes executing the inference task, as well as the model segment executed by each target roadside node.

[0071] According to the execution order, the first inference result is input into the second model fragment of the target inference model corresponding to the pre-loaded model identifier to obtain the second inference result;

[0072] If the second model segment is the last reasoning unit of the target reasoning model, then the second reasoning result is sent to the vehicle according to the vehicle's identifier;

[0073] If the second model segment is an intermediate inference unit of the target inference model, then according to the task execution strategy, a third inference task is sent to the third roadside node that has preloaded the next model segment of the target inference model. The third inference task includes the vehicle identifier, the second inference result, the model identifier, and the task execution strategy.

[0074] The second model slice is a second slice of the target inference model that is preloaded under the trigger of a model preloading instruction sent by the vehicle before receiving the second task data.

[0075] In one possible implementation, before receiving the second task data of the inference task sent by the first roadside node, the method further includes:

[0076] Receive a model preloading instruction sent by the vehicle, the model preloading instruction including the model identifier and the identifier of the second model segment;

[0077] According to the model preloading instruction, the structure and parameters of the second model segment of the target inference model corresponding to the model identifier are loaded.

[0078] Sixthly, embodiments of this application provide an execution device for an in-vehicle reasoning task, comprising:

[0079] The first sending module is used to send the task data of the inference task to the target roadside node in response to the execution request of the inference task sent by the application layer of the vehicle.

[0080] The task data includes the model identifier for performing the inference task and the inference task input data; the target roadside node is a roadside node pre-determined based on the vehicle's driving behavior data for performing the vehicle's inference task, and the target roadside node is pre-loaded with the target inference model for performing the inference task.

[0081] The receiving module is used to receive the inference results returned by the target roadside node.

[0082] In one possible implementation, the device further includes:

[0083] The determination module is used to determine the target roadside node to be used for reasoning tasks based on the driving behavior data during vehicle operation.

[0084] The second sending module is used to send a model preloading instruction to the target roadside node so that the target roadside node preloads the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model.

[0085] In one possible implementation, the determining module is specifically used for:

[0086] Based on the driving behavior data, the system predicts multiple roadside nodes that the vehicle will pass through in a future preset time period and the probability of reaching each roadside node; the probability of reaching represents the confidence that the vehicle can reach the corresponding roadside node within the preset time period.

[0087] The roadside node with the highest arrival probability among the plurality of roadside nodes is determined as the target roadside node;

[0088] The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0089] In one possible implementation, the first transmitting module is specifically used for:

[0090] When the arrival probability of the target roadside node reaches a preset threshold, the task data is sent to the target roadside node.

[0091] In one possible implementation, the device further includes:

[0092] The acquisition module is used to acquire the latest time that the vehicle is expected to arrive at the target roadside node based on the driving behavior data;

[0093] An add module is used to add the latest time to the model preloading instruction; the latest time is used to trigger the target roadside node to delete the preloaded data.

[0094] In a seventh aspect, embodiments of this application provide an execution device for an in-vehicle reasoning task, comprising:

[0095] The first receiving module is used to receive task data of the inference task sent by the vehicle. The task data includes the model identifier for performing the inference task and the inference task input data.

[0096] The reasoning module is used to input the input data of the reasoning task into the pre-loaded target reasoning model corresponding to the model identifier to perform reasoning and obtain the reasoning result;

[0097] The return module is used to return the reasoning result to the vehicle;

[0098] The target inference model is an inference model preloaded under the trigger of a model preload instruction sent before the vehicle sends the task data.

[0099] In one possible implementation, the device further includes:

[0100] The second receiving module is used to receive the model preloading instruction sent by the vehicle, wherein the model preloading instruction includes the model identifier;

[0101] The preloading module is used to load the structure and parameters of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0102] In one possible implementation, the model preloading instruction in the second receiving module further includes the latest estimated time for the vehicle to arrive at the target roadside node; the device also includes

[0103] The deletion module is used to delete the preloaded data corresponding to the target inference model if the task data is not received before the latest time.

[0104] Eighthly, embodiments of this application provide an execution device for an in-vehicle reasoning task, comprising:

[0105] The first sending module is used to send the first task data of the inference task to the first roadside node in response to the execution request of the inference task sent by the application layer of the vehicle.

[0106] The first task data includes the vehicle identifier, the model identifier for executing the inference task, the inference task input data, and the task execution strategy. The task execution strategy includes the node identifiers and execution order of all target roadside nodes executing the inference task, as well as the model fragments executed for each target roadside node.

[0107] The first roadside node is the first of at least two target roadside nodes pre-determined based on the vehicle's driving behavior data for performing the vehicle's inference task. The first roadside node pre-loads the first model fragment of the target inference model for performing the inference task.

[0108] The receiving module is used to receive the inference results returned by the second roadside node or other roadside nodes, wherein the second roadside node preloads the second model fragment of the target inference model for performing the inference task.

[0109] In one possible implementation, the device further includes:

[0110] The determination module is used to determine the task execution strategy based on the driving behavior data during vehicle operation;

[0111] The second sending module is used to send a model preloading instruction to each target roadside node indicated by the task execution strategy, so that each target roadside node preloads different model fragments of the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model and the identifier of the corresponding model fragment.

[0112] In one possible implementation, the determining module includes:

[0113] The prediction unit is used to predict, based on the driving behavior data, multiple roadside nodes that the vehicle will pass through in the future within a preset time period and the arrival probability of each roadside node; the arrival probability represents the confidence level that the vehicle can reach the corresponding roadside node in the future preset time period.

[0114] The determining unit is used to determine at least two target roadside nodes from the plurality of roadside nodes based on the arrival probability of each roadside node, and to determine the model loading requirements of each target roadside node.

[0115] The first acquisition unit is used to acquire the task execution strategy based on the model loading requirements of each target roadside node;

[0116] The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0117] In one possible implementation, the determining unit is specifically used for:

[0118] Roadside nodes whose arrival probability is greater than the first preset threshold are identified as intermediate roadside nodes;

[0119] The target inference model is clustered into multiple model subgraphs according to operator type, and each model subgraph includes at least one model piece of the target inference model;

[0120] Based on the node type of each intermediate roadside node, the operator type of each model subgraph, and the preset computing power adaptation mapping table, at least two target roadside nodes and the model subgraph matched by each target roadside node are determined from all intermediate roadside nodes; wherein, the preset computing power adaptation mapping table represents the computational efficiency of different types of roadside nodes when executing different types of operators.

[0121] The model subgraph matched for each target roadside node is determined as the model loading requirement for the target roadside node.

[0122] In one possible implementation, the first transmitting module is specifically used for:

[0123] When the arrival probability of the first roadside node reaches the second preset threshold, the first task data is sent to the first roadside node.

[0124] In one possible implementation, the device further includes:

[0125] The acquisition module is used to acquire the load of each target roadside node in real time;

[0126] The first migration module is used to migrate the inference tasks that have not been completed at any target roadside node to other roadside nodes for execution when the load of any target roadside node exceeds a preset load.

[0127] In one possible implementation, the device further includes:

[0128] The verification module is specifically used for:

[0129] Obtain the semantic summary fingerprint sent by the target roadside node; the semantic summary fingerprint is obtained by statistical feature extraction based on the intermediate inference results output by the model segment corresponding to the target roadside node; the model segment is the inference unit other than the last inference unit in the target inference model;

[0130] Calculate the distance between the semantic summary fingerprint and the pre-stored fingerprint reference distribution;

[0131] If the distance is less than or equal to a preset first distance threshold, a verification pass command is sent to the target roadside node; the verification pass command is used to trigger the target roadside node to continue the execution of the inference task based on the intermediate inference result;

[0132] If the distance is greater than the first distance threshold and less than the preset second distance threshold, a re-inference request is sent to the target roadside node; the re-inference request is used to instruct the target roadside node to re-execute the inference task based on the corresponding task data;

[0133] If the distance is greater than or equal to the second distance threshold, the inference task of the target roadside node will be migrated to other roadside nodes for execution.

[0134] In one possible implementation, the semantic summary fingerprint in the verification unit includes at least one of the mean, variance, quantile, median, and sparse feature entropy of the intermediate inference results.

[0135] Ninthly, embodiments of this application provide an execution device for an in-vehicle reasoning task, including...

[0136] The first receiving module is used to receive the first task data of the inference task sent by the vehicle. The first task data includes the vehicle's identifier, the model identifier that executes the inference task, the inference task input data, and the task execution strategy.

[0137] The task execution strategy includes the node identifiers and execution order of all target roadside nodes executing the inference task, as well as the model slices executed for each target roadside node.

[0138] The inference module is used to input the inference task input data into the first model fragment of the target inference model corresponding to the pre-loaded model identifier according to the execution order, and obtain the first inference result;

[0139] The sending module is used to send the second task data of the inference task to the second roadside node according to the task execution strategy. The second task data includes the vehicle identifier, the first inference result, the model identifier, and the task execution strategy, so that the second roadside node can continue to execute the inference task.

[0140] The first model slice is the first slice of the target inference model preloaded under the trigger of a model preload instruction sent before the vehicle sends the first task data.

[0141] In one possible implementation, the device further includes:

[0142] The second receiving module is used to receive the model preloading instruction sent by the vehicle, wherein the model preloading instruction includes the model identifier and the identifier of the first model segment;

[0143] The preloading module is used to load the structure and parameters of the first model segment of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0144] In a tenth aspect, embodiments of this application provide an execution device for an in-vehicle reasoning task, comprising:

[0145] The first receiving module is used to receive the second task data of the inference task sent by the first roadside node. The second task data includes the vehicle identifier, the first inference result, the model identifier, and the execution strategy. The first inference result is the output tensor obtained by executing the inference task of the first model segment of the target inference model corresponding to the model identifier. The task execution strategy includes the node identifier and execution order of all target roadside nodes executing the inference task, as well as the model segment executed by each target roadside node.

[0146] The inference module is used to input the first inference result into the second model fragment of the target inference model corresponding to the pre-loaded model identifier, according to the execution order, to obtain the second inference result;

[0147] The sending module is configured to send the second inference result to the vehicle based on the vehicle's identifier if the second model fragment is the last inference unit of the target inference model.

[0148] If the second model segment is an intermediate inference unit of the target inference model, then according to the task execution strategy, a third inference task is sent to the third roadside node that has preloaded the next model segment of the target inference model. The third inference task includes the vehicle identifier, the second inference result, the model identifier, and the task execution strategy.

[0149] The second model slice is a second slice of the target inference model that is preloaded under the trigger of a model preloading instruction sent by the vehicle before receiving the second task data.

[0150] In one possible implementation, the device further includes:

[0151] The second receiving module is used to receive the model preloading instruction sent by the vehicle, the model preloading instruction including the model identifier and the identifier of the second model segment;

[0152] The preloading module is used to load the structure and parameters of the second model segment of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0153] Eleventhly, embodiments of this application provide a control unit, including: a memory and a processor;

[0154] The memory stores computer-executed instructions;

[0155] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect, or the third aspect and / or various possible implementations of the third aspect.

[0156] In a twelfth aspect, embodiments of this application provide a roadside node, including: a memory and a processor;

[0157] The memory stores computer-executed instructions;

[0158] The processor executes computer execution instructions stored in the memory, causing the processor to perform the second aspect and / or various possible implementations of the second aspect as described above, or the fourth aspect and / or various possible implementations of the fourth aspect as described above, or the fifth aspect and / or various possible implementations of the fifth aspect as described above.

[0159] In one possible implementation, the node type of the roadside node is a roadside unit, a charging station, or a parking lot gateway.

[0160] In a thirteenth aspect, embodiments of this application provide a vehicle, including a vehicle body and the control unit described in the eleventh aspect.

[0161] In a fifteenth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect, or the third aspect and / or various possible implementations of the third aspect.

[0162] In a sixteenth aspect, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the second aspect and / or various possible implementations of the second aspect, or the fourth aspect and / or various possible implementations of the fourth aspect, or the fifth aspect and / or various possible implementations of the fifth aspect.

[0163] In a seventeenth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect, or the third aspect and / or various possible implementations of the third aspect.

[0164] In the eighteenth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the second aspect and / or various possible implementations of the second aspect, or the fourth aspect and / or various possible implementations of the fourth aspect, or the fifth aspect and / or various possible implementations of the fifth aspect.

[0165] The vehicle-mounted inference task execution method, device, and vehicle provided in this application embodiment first determine the target roadside node to be used to execute its inference task based on driving behavior data, and send an instruction to the target roadside node to preload the target inference model (or model fragments of the target inference model) according to data loading requirements. Then, when the vehicle uses the target roadside node to execute the inference task, only the task data including its inference task input data is sent to the target roadside node, so that the target roadside node can directly use the preloaded model to execute the inference task. This means that when the vehicle uses the target roadside node to execute the inference task, it does not need to wait for the model to load, which greatly shortens the time for the vehicle to process the first frame of task data using the target roadside node, improves the effective inference time for the vehicle to execute the inference task using the node, and thus improves the real-time performance of inference task execution in vehicle-road cooperative inference scenarios. Attached Figure Description

[0166] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0167] Figure 1 A flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 1 of this application;

[0168] Figure 2 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 2 of this application;

[0169] Figure 3 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 3 of this application;

[0170] Figure 4 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 4 of this application;

[0171] Figure 5This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 5 of this application;

[0172] Figure 6 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment Six of this application;

[0173] Figure 7 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 7 of this application;

[0174] Figure 8 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 8 of this application;

[0175] Figure 9 A schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 9 of this application;

[0176] Figure 10 A schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 10 of this application;

[0177] Figure 11 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment Eleven of this application;

[0178] Figure 12 This is a schematic diagram of the control unit provided in Embodiment Twelve of this application;

[0179] Figure 13 This is a schematic diagram of the roadside node provided in Embodiment Thirteen of this application.

[0180] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0181] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0182] To address the problems mentioned in the background technology, the inventors discovered during their research that when a vehicle uses roadside nodes to process inference tasks, it can predict in advance the roadside nodes that the vehicle may pass through in a short period of time based on the vehicle's driving behavior data. While using the currently passed roadside nodes to process inference tasks, a model preloading instruction is sent to a predicted target roadside node at a greater distance, so that the target roadside node can complete the model loading in advance. Based on this, when the vehicle passes the target roadside node, it can send only the input data of the inference task to the roadside node, so that the target roadside node can directly use the preloaded model to perform the inference task.

[0183] As can be seen, this solution can eliminate the waiting delay caused by the loading of the model on the new node when the vehicle switches to a roadside node carrying the inference task, thereby improving the real-time performance of the inference task in the vehicle-road cooperative inference scenario.

[0184] It should be noted that this application applies to collaborative reasoning scenarios between intelligent connected vehicles and roadside nodes. For example, in complex traffic scenarios such as urban arterial roads and intersections, vehicles need to collaborate with roadside nodes through onboard control units to complete AI reasoning tasks such as target detection and trajectory prediction.

[0185] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0186] Embodiment 1 of this application provides an execution system and method for an in-vehicle inference task. The execution system for the inference task includes a vehicle control unit and a target roadside node. The target roadside node is, for example, a roadside unit (RSU), a charging station, or a parking lot gateway. The control unit is, for example, a domain controller in the vehicle or a control module in an on-board unit (OBU). This application does not impose any special limitations on the specific type of roadside node or the specific hardware implementation of the on-board control unit.

[0187] Figure 1 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 1 of this application. This method is applied to the execution system provided in this embodiment. In this embodiment, the system includes a target roadside node, such as... Figure 1 As shown, the method includes:

[0188] S101. In response to the inference task execution request sent by the vehicle's application layer, the task data of the inference task is sent to the target roadside node.

[0189] In practical applications, in order to ensure the stable and safe operation of a vehicle during driving, it is usually necessary to perform a variety of intelligent reasoning tasks, such as object detection, obstacle recognition, lane segmentation, and traffic sign recognition.

[0190] Specifically, vehicles typically receive execution requests for inference tasks triggered by upstream application layers (such as intelligent driving decision-making modules, driving safety assurance modules, etc.) based on driving needs, and under the trigger of these requests, they advance the execution of the inference task (i.e., including the acquisition of task data and the execution of inference).

[0191] In this step, after receiving the inference task execution request from the upstream application layer, the control unit will acquire task data in real time according to the inference requirements and send the real-time task data to the predetermined target roadside node while passing through it. It should be understood that the communication quality with the target roadside node is optimal while the vehicle is passing through it.

[0192] In a specific implementation, data interaction between the vehicle and the target roadside node is achieved, for example, through the V2X PC5 interface, including the transmission of task data and inference results. Optionally, the communication methods between the vehicle and the target roadside node may also include Uu interface cellular communication links, roadside local area network (LAN) links, etc.

[0193] In addition, the control unit can determine whether the vehicle is passing the target roadside node based on the actual distance between the vehicle and the target roadside node. For example, when the actual distance between the vehicle and the target roadside node is less than a preset distance (such as 200m), it is determined that the vehicle is passing the target roadside node. During this period, the vehicle needs to send the currently acquired task data to the target roadside node in real time.

[0194] It should be understood that before the vehicle uses the target roadside node to perform the inference task, the inference task will be performed by other roadside nodes (such as nodes that are closer to the vehicle in actual distance).

[0195] The target roadside node is a roadside node that is pre-determined based on the vehicle's driving behavior data and is used to perform the vehicle's reasoning task. The target roadside node is pre-loaded with the target reasoning model for performing the reasoning task.

[0196] Specifically, a roadside node refers to an edge computing device that can interact with vehicles, has local computing capabilities, can perform model inference and data processing, and is deployed at the edge of the network, such as RSUs, charging piles, and parking lot gateways; a target roadside node is a node that a vehicle is about to pass through, which is determined in advance by the vehicle from multiple roadside nodes that can interact with it based on its driving behavior data.

[0197] The target reasoning model includes the model structure and model parameters, which can perform data reasoning on the input data according to its structure and parameters and output the reasoning results; driving behavior data refers to relevant data that can reflect the vehicle driving status and driver control commands, such as vehicle speed, acceleration, heading angle, driving trajectory, navigation data, etc.

[0198] It should be understood that before receiving the task data sent by the vehicle, the target roadside node has already preloaded the target inference model for performing the inference task, so that when the target roadside node receives the first frame of task data sent by the vehicle, it can directly use the preloaded inference model to perform the inference task, effectively saving the model loading time of the node.

[0199] In addition, the task data includes model identifiers for performing inference tasks and inference task input data.

[0200] Specifically, the inference task input data refers to the real-time perception data collected by the vehicle based on the execution requirements of the inference task; for example, if the inference task is lane line segmentation, the inference task input data includes road environment image data collected by the vehicle camera, as well as point cloud data, obstacle distance and location information, etc. collected by millimeter-wave radar and lidar.

[0201] It should be noted that the input data for this inference task can be the input data required when the vehicle uses the target roadside node to perform the first frame of the inference task, or the input data required when the vehicle uses the target roadside node to perform any frame of the subsequent multi-frame inference task.

[0202] A model identifier refers to the model identifier of the target inference model required to perform the corresponding inference task. This model identifier uniquely identifies the target inference model, enabling the target roadside node to clearly understand the model that should be input into the inference task. Generally speaking, the target inference model is a pre-trained artificial intelligence model used for intelligent analysis and prediction calculations on the input data, such as a convolutional neural network model, a YOLOv8 object detection model, or a semantic segmentation model.

[0203] Optionally, the task data may also include vehicle identifiers, which are used by the target roadside node to return the inference results to the vehicle.

[0204] S102, Receive the task data for the inference task sent by the vehicle.

[0205] In this step, the target roadside node will receive the task data for the inference task sent by the vehicle.

[0206] S103. Input the inference task input data into the target inference model corresponding to the pre-loaded model identifier to perform inference and obtain the inference result.

[0207] In this step, the target roadside node needs to directly input the inference task input data into the target inference model corresponding to the pre-loaded model identifier, so that the target inference model can perform inference according to its model structure and model parameters and output the inference results.

[0208] The target inference model is a preloaded inference model triggered by a model preloading command sent before the vehicle sends task data. The inference result is used by the vehicle's application layer to perform decision-making, control, and other operations on the vehicle. For example, if the inference task is target detection, the inference result may include the target category, detection box coordinates, and confidence score output by the model inference.

[0209] S104. Return the reasoning result to the vehicle.

[0210] In this step, the target roadside node needs to return the obtained inference results to the vehicle.

[0211] S105. Receive the inference results returned by the target roadside node.

[0212] In this step, the control unit will receive the inference results from the target roadside node and use them to return the inference results to the upper application layer to meet the data requirements of the application layer to perform decision-making or control operations.

[0213] The method provided in this embodiment is applied to the execution system and method of vehicle-mounted inference tasks. By sending only the task data, including its inference task input data, to the target roadside node when the vehicle performs the inference task using the target roadside node, the target roadside node can directly use the pre-loaded model to perform the inference task. In the scenario of performing the inference task using a single roadside node, the vehicle does not need to wait for the model to load when performing the inference task using the target roadside node, which greatly shortens the time for the vehicle to process the first frame of task data using the target roadside node and improves the effective inference time for the vehicle to perform the inference task using the node. This achieves the effect of improving the real-time performance of the inference task execution in the vehicle-road cooperative inference scenario.

[0214] Furthermore, Figure 2This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 2 of this application. The method provided in Embodiment 2 is executed before the aforementioned step S101, and is used to introduce the determination of the target roadside node and model preloading, such as... Figure 2 As shown, based on the above embodiments, the method provided in this embodiment further includes...

[0215] S201. During vehicle operation, determine the target roadside node to be used for reasoning tasks based on driving behavior data.

[0216] In this step, while the vehicle is in motion, the vehicle's control unit will acquire, in real time, target roadside nodes determined based on its driving behavior data, which will be used to perform the reasoning task. Specifically, target roadside nodes refer to nodes determined from multiple roadside nodes that the vehicle will pass through in a preset time period.

[0217] For example, the control unit can predict multiple roadside nodes that the vehicle will pass through in a preset time period based on the vehicle's navigation data, and determine the target roadside node whose distance is within a preset distance range (e.g., 200~500m) based on the actual distance between the vehicle and each roadside node.

[0218] In some implementations, vehicles can send real-time driving behavior data to the cloud, which will then determine the target roadside node, and the vehicle will receive the result of the target roadside node determination from the cloud.

[0219] S202, Send a model preloading command to the target roadside node.

[0220] The model preloading instruction includes the model identifier of the target inference model.

[0221] In this step, in order to ensure that the vehicle can perform inference using the pre-loaded target inference model when it passes the target roadside node, the control unit needs to send a model preloading command to the target roadside node after it is identified and before the vehicle passes the target roadside node, so that the target roadside node can perform the preloading of the target inference model according to the model identifier indicated by the command after receiving the command.

[0222] In some implementations, the control unit can send a model preloading command to the target roadside node via the V2X PC5 interface between the vehicle and the target roadside node.

[0223] Alternatively, in some implementations, the vehicle may send a target roadside node model preloading instruction to the cloud, so that the cloud can send the model preloading instruction to the target roadside node through the communication link between the cloud and the target roadside node.

[0224] Alternatively, in some implementations, if the determination of the target roadside node is performed by the cloud, the cloud will directly send a model preloading command to the target roadside node through the communication link between the cloud and the target roadside node after determining the target roadside node. This communication link between the cloud and the target roadside node may include, for example, fiber optic communication links, 5G cellular communication links, or Ethernet wired links.

[0225] S203, Receive the model preloading instruction sent by the vehicle.

[0226] In this step, the target roadside node will receive the model preloading instruction corresponding to the inference task.

[0227] S204. According to the model preloading instruction, load the structure and parameters of the target inference model corresponding to the model identifier.

[0228] In this step, the target roadside node will use the model identifier in the model preloading instruction as an index to load the structure and parameters of the target inference model corresponding to the model identifier, so that when subsequent vehicles pass through the target roadside node, the target roadside node can directly respond to each frame of task data sent by the vehicle and directly use the loaded target inference model to perform inference.

[0229] The structure of the model refers to the hierarchical layout and computational framework of the neural network in the model, which is pre-defined based on the type of reasoning task and is used to determine the processing flow of the input perceived data; the parameters of the model refer to the set of values ​​such as weights and biases learned by the model after training, which are the feature patterns and judgment criteria learned for the corresponding reasoning task.

[0230] In practical applications, the model identifiers, structures, and parameters of each inference model required for performing the vehicle's inference task can be pre-stored locally on each roadside node. This allows each roadside node to find and load the structure and parameters of the target inference model corresponding to the model identifier carried in the instruction after receiving the model preloading instruction.

[0231] In the vehicle-mounted inference task execution method provided in Embodiment 2 of this application, the target roadside node to be used for inference task execution is determined in real time based on driving behavior data during vehicle travel, and a model preloading instruction is sent to it to instruct it to preload the target inference model. This achieves the preloading of the target inference model in the target roadside node in the scenario of executing the inference task at a single node, providing a basis for subsequent vehicles to directly use task data to execute the inference task without waiting for the model to load when passing through the target roadside node.

[0232] Furthermore, Figure 3This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 3 of this application, as shown below. Figure 3 As shown, based on the above embodiments, this application provides a more detailed execution method, specifically including:

[0233] S301. During vehicle operation, based on driving behavior data, predict multiple roadside nodes that the vehicle will pass through in a future preset time period and the probability of arriving at each roadside node.

[0234] The arrival probability represents the confidence level that the vehicle will reach the corresponding roadside node within a preset time period. It should be understood that the higher the probability value, the greater the likelihood that the vehicle will pass through the roadside node within the preset time period.

[0235] In this step, in order to identify the target roadside node that matches the current time and will be used to perform the reasoning task, the control unit needs to predict, based on the vehicle's driving behavior data, the multiple roadside nodes that the vehicle will pass through in the future within a preset time period and the probability of arriving at each roadside node.

[0236] Regarding the determination of multiple roadside nodes, for example, multiple roadside nodes whose actual distance to the vehicle will be less than a preset distance within a preset time period can be predicted based on driving behavior data. The preset time period and preset distance can be determined based on factors such as the optimal communication coverage of conventional roadside nodes, and this application does not impose specific limitations on them. For example, multiple roadside nodes whose actual distance to the vehicle will be less than 200m within the next 2-5 seconds can be predicted based on driving behavior data.

[0237] In addition, for each predicted roadside node, the reachability probability of the roadside node can be determined based on the confidence level that the node can be reached within a preset time period.

[0238] Specifically, driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status, which includes the vehicle's positioning data, speed data, and heading angle data.

[0239] Among them, the historical trajectory sequence refers to the ordered sequence of geographical locations that the vehicle has passed through in a continuous period of time in the past, such as the sequence of latitude and longitude points that the vehicle has passed through in recent seconds or minutes; the location data refers to the real-time geographical coordinates of the vehicle, used to determine the vehicle's current location; the speed data refers to the real-time speed of the vehicle, used to reflect the vehicle's rate of movement; and the heading angle data refers to the current heading angle of the vehicle, used to characterize the vehicle's direction of travel.

[0240] It should be understood that since historical trajectory sequences can reflect the driving path patterns of vehicles, positioning data, speed data, and heading angle data can jointly characterize the vehicle's current real-time position, speed, and direction of travel. The above data can fully represent the vehicle's driving status and movement trend. Therefore, using these data can accurately predict multiple roadside nodes that the vehicle may pass through in a future preset time period, as well as the probability of the vehicle arriving at each roadside node.

[0241] Optionally, driving behavior data may also include vehicle acceleration data and navigation data. Acceleration data refers to the rate of change of vehicle speed over time, used to characterize the vehicle's acceleration and deceleration states and dynamic driving trends; navigation data refers to information such as the target location, driving route, road segments, and key waypoints planned by the vehicle's navigation system, used to reflect the vehicle's preset driving intentions and planned path.

[0242] As a specific example, the control unit can make predictions based on the aforementioned driving behavior data, combined with a pre-built and locally stored roadside node topology map (containing map information such as the geographical coordinates and road connectivity of each roadside node):

[0243] Specifically, based on vehicle driving behavior data and preset time period constraints, accessibility can be filtered using a roadside node relationship topology graph to obtain multiple roadside nodes.

[0244] For example, real-time vehicle positioning data can be used as the geometric starting point. The heading angle is used as the main direction, and the driving trend of historical trajectory sequences is combined to determine the range of the forward fan-shaped angle. At the same time, the maximum achievable driving distance is estimated by multiplying the current speed by the preset time period, which is used as the fan-shaped radius. This delineates the spatiotemporal reachable area of ​​the vehicle within the preset time period. Then, combined with the pre-constructed roadside node topology graph, multiple roadside nodes located within the reachable area and connected to the vehicle's current location by road are selected.

[0245] Next, the aforementioned vehicle driving behavior time series data and the topological feature data (including the distance between the node and the vehicle location, heading deviation, road connectivity, etc.) corresponding to each roadside node selected above are used as model input data and input into the pre-trained time series prediction model to obtain the probability value of the model for each roadside node, and this probability value is used as the arrival probability of the roadside node.

[0246] Among them, the time series prediction model is a Long Short-Term Memory (LSTM) network model, a Transformer model, or a Multi-Layer Perceptron (MLP) model, which are trained in advance using a large amount of historical driving behavior data, roadside node topology data, and actual vehicle passage records. This application does not impose specific restrictions on the selection of models.

[0247] S302. Determine the roadside node with the highest arrival probability among multiple roadside nodes as the target roadside node.

[0248] In this step, in order to ensure the effectiveness of model preloading and the rapid switching of the execution subject of subsequent inference tasks, thereby ensuring the high real-time performance of task execution, the control unit needs to determine the roadside node with the highest arrival probability (meaning that the roadside node is the most likely to be reached by the vehicle in the future within a preset time period) among multiple roadside nodes as the target roadside node.

[0249] S303, Send a model preloading command to the target roadside node.

[0250] S304. Receive the model preloading instruction sent by the vehicle, and load the structure and parameters of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0251] The implementation methods and principles of steps S303 to be executed by the control unit and steps S304 to be executed by the target roadside node can be referred to the aforementioned embodiments, and will not be repeated here.

[0252] S305. In response to the execution request of the inference task sent by the vehicle's application layer, when the arrival probability of the target roadside node reaches a preset threshold, the task data is sent to the target roadside node.

[0253] In practical applications, the arrival probability of the target roadside node will be determined in real time based on the vehicle's driving behavior data. For example, the arrival probability of the target roadside node can be dynamically predicted based on the pre-trained time-series prediction model described above and the real-time changing driving behavior data.

[0254] In this step, upon receiving an execution request, if the arrival probability of the target roadside node reaches a preset threshold, indicating that the vehicle is passing the target roadside node, the roadside node used for inference needs to be switched to that target roadside node. In other words, when the arrival probability of the target roadside node is lower than the preset threshold, the currently used roadside node is maintained for inference; when the arrival probability of the target roadside node is greater than or equal to the preset threshold, the inference will be switched to that roadside node, meaning the task data corresponding to the vehicle's current moment will be sent to that target roadside node.

[0255] The determination of the preset threshold can be based on the actual application of the solution, such as 0.85, 0.87, 0.9, etc. This application does not impose specific restrictions on this.

[0256] S306. Receive the task data for the inference task sent by the vehicle, input the inference task input data into the target inference model corresponding to the pre-loaded model identifier, perform inference, and obtain the inference result.

[0257] S307, The reasoning result is returned to the vehicle.

[0258] S308, Receive the inference results returned by the target roadside node.

[0259] The implementation methods and principles of steps S306 and S307 to be executed by the target roadside node and step S308 to be executed by the control unit can be referred to the aforementioned embodiments, and will not be repeated here.

[0260] The vehicle-mounted inference task execution method provided in this application predicts multiple roadside nodes that the vehicle will pass through in a preset time period and the arrival probability of each roadside node by utilizing the vehicle's historical trajectory sequence and real-time motion state during vehicle travel. The node with the highest arrival probability is determined as the target roadside node. Simultaneously, the method assesses whether the vehicle is currently passing the target roadside node based on the arrival probability of the vehicle and triggers the transmission of task data. This allows for dynamic and accurate assessment of roadside nodes that the vehicle is about to pass or is currently passing through, based on the arrival probability, thus achieving precise control of the task triggering timing. On the one hand, it can improve the accuracy of determining the target roadside node, thereby avoiding the waste of computing resources caused by model misloading in the node. On the other hand, it can improve the accuracy of determining the timing of sending task data to the target roadside node, thereby effectively ensuring the smooth execution of the inference task.

[0261] Furthermore, Figure 4 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 4 of this application, as shown below. Figure 4 As shown, based on the foregoing embodiments, the method provided in this embodiment includes:

[0262] S401. During vehicle operation, determine the target roadside node to be used for reasoning tasks based on driving behavior data.

[0263] The implementation method of this step can be referred to the foregoing embodiments, and will not be repeated here.

[0264] S402. Based on driving behavior data, obtain the latest time that the vehicle is expected to arrive at the target roadside node.

[0265] In this step, the latest time that the vehicle is expected to arrive at the target roadside node will be determined based on driving behavior data.

[0266] The latest time can be represented as a point in time (such as Beijing time) or a relative duration. For example, when the representation is an absolute point in time, the latest time is, for example, X year X month X day X hour X minute X second; or, when the representation is a relative duration, the latest time is 5 seconds, that is, the latest time for the vehicle is 5 seconds after the target roadside node receives the model preloading instruction.

[0267] As a specific example, it can be assumed that the vehicle is traveling at a constant speed, and the theoretical travel time can be calculated based on the current speed data and the road topology distance from the vehicle to the target roadside node. A preset redundant buffer time (such as 1 second) can be added to obtain the latest expected arrival time.

[0268] S403. Add the latest time to the model preloading instruction.

[0269] The latest time is used to trigger the target roadside node to delete the preloaded data.

[0270] In this step, the latest time needs to be added to the model preloading instruction. Accordingly, the model preloading instruction will include the model identifier and the latest time that the vehicle is expected to arrive at the target roadside node.

[0271] S404. Send a model preloading command to the target roadside node.

[0272] S405: Receive the model preloading instruction sent by the vehicle, and load the structure and parameters of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0273] The implementation methods and principles of steps S404 to be executed by the control unit and steps S405 to be executed by the target roadside node can be referred to the aforementioned embodiments, and will not be repeated here.

[0274] S406. If no task data is received before the latest time, delete the preloaded data corresponding to the target inference model.

[0275] In this step, if the target roadside unit determines that it has not received the task data sent by the vehicle before the latest time, it means that the actual trajectory of the vehicle has deviated from the prediction. Therefore, in order to avoid the computing resources in the target roadside node being occupied by useless data, the preloaded data (including model structure and model parameters) corresponding to the target inference model will be deleted.

[0276] The vehicle-mounted inference task execution method provided in this application embodiment adds the latest estimated arrival time of the vehicle to the target roadside node in the model preloading instruction sent to the target roadside node, and deletes the preloaded data corresponding to the target inference model when the target roadside node detects that the node has not received the task data sent by the vehicle before the latest arrival time. This enables the target roadside node to automatically identify the timeout state of the vehicle not arriving, realize the autonomous release and management of preloaded model resources, avoid invalid model data occupying roadside storage and computing resources for a long time, improve the resource utilization rate of roadside nodes, reduce resource idle loss, and ensure the efficient and stable operation of nodes.

[0277] The following describes the execution method of a single roadside node performing an inference task provided in the foregoing embodiments of this application using a specific scenario example. Specifically, this scenario includes:

[0278] A certain vehicle is an intelligent connected vehicle with vehicle-to-everything (V2X) collaborative interaction capabilities. The onboard OBU has a V2X communication module and a control module, which can achieve low-latency wireless communication with roadside nodes and other vehicles. Its onboard Neural Processing Unit (NPU) for AI inference has a computing power of only 8 TOPS, which is limited and makes it difficult to independently complete the complete inference calculation of large-scale, highly complex neural network models locally. The vehicle is driving on an urban main road, where roadside nodes are densely distributed, vehicle travel is highly continuous, and the latency requirements for inference tasks are high. Therefore, it needs to rely on roadside nodes to complete the inference tasks collaboratively.

[0279] In this scenario, during vehicle operation, based on driving behavior data, the control module in the OBU predicts multiple roadside nodes that the vehicle will pass through in the next 2-5 seconds and the arrival probability of each roadside node. These multiple roadside nodes include the roadside unit node RSU-1, the charging pile node CPO-1, and the charging parking lot gateway node PG-1 deployed on the roadside.

[0280] Specifically, RSU-1 is equipped with an NPU, CPO-1 is equipped with a central processing unit (CPU), and PG-1 is equipped with a graphics processing unit (GPU). At the current moment, the roadside node with the highest probability of arrival is RSU-1, with an arrival probability of 0.68 (at this time, the actual distance between the vehicle and RSU-1 is about 500m).

[0281] At this point, the OBU sends a model preloading command to the roadside node RSU-1 via the V2X PC5 interface between the vehicle and the node, so that the roadside node RSU-1 preloads the target inference model. From then on, the control module determines the arrival probability of the roadside node RSU-1 in real time. When the arrival probability reaches 0.85 (at which point the actual distance between the vehicle and RSU-1 is about 200m), it sends the task data acquired at the current moment to the roadside node RSU-1 via the V2X PC5 interface between the vehicle and the node, so that the roadside node RSU-1 uses its preloaded target inference model to directly receive the first frame of task data sent by the vehicle and returns the inference result to the vehicle. Subsequently, the vehicle will continue to use the roadside node RSU-1 to perform inference tasks until the next roadside node for which the vehicle is expected to perform the inference task completes the preloading of the target inference model, and the arrival probability of the vehicle reaching the next roadside node is greater than 0.85. At this time, the vehicle will output the inference data acquired in real time to the next roadside node and start using the next roadside node to perform inference.

[0282] Figure 5 This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment 5 of this application. This method is applied to the execution system provided in Embodiment 1 above. In this embodiment, the system includes at least two target roadside nodes. It should be noted that in practical applications, the second model slice may be the last inference unit of the target inference model, or it may be an intermediate inference unit of the target inference model.

[0283] Figure 5 The provided flowchart uses the scenario of "the second model slice being the last inference unit of the target inference model" as an example to illustrate the interaction logic of each entity in the execution method of the in-vehicle inference task provided in this application. Specifically, the execution method of the in-vehicle inference task provided in this embodiment includes:

[0284] S501. During vehicle operation, determine the task execution strategy based on driving behavior data.

[0285] It should be understood that, considering that in some application scenarios, multiple nodes jointly execute the same inference task (i.e., tasks corresponding to different model fragments of the target inference model are executed by different roadside nodes), the time delay is shorter than that of a single node executing the inference task, which is more conducive to improving the real-time performance of the inference task.

[0286] For this application scenario, this embodiment provides a specific execution scheme. Specifically, in this step, during the vehicle's operation, the vehicle's control unit needs to acquire in real time all target roadside nodes that will be used to execute the inference task, determined based on its driving behavior data, as well as the model fragments that should be loaded for each target roadside node and the execution order of each roadside node, to obtain the task execution strategy.

[0287] Accordingly, the task execution strategy includes the node identifiers and execution order of all target roadside nodes (at least two) that will be used to perform the inference task, as well as the model slices to be executed for each target roadside node.

[0288] It should be noted that for each target roadside node indicated in the task execution strategy, the corresponding model slice to be executed can be one or multiple non-contiguous slices. For example, a target roadside node may only need to execute the first model slice of the target inference model, or it may need to execute the first, third, and sixth model slices of the target inference model.

[0289] Here, model segmentation refers to the independent inference units obtained by dividing the complete target inference model according to the network hierarchy; each model segment contains part of the network structure and corresponding model parameters of the complete target inference model.

[0290] The node identifier of each target roadside node refers to a unique number or code used to uniquely identify the identity of each target roadside node, and is used to clarify the target of instructions and data transmission; the execution order of each target roadside node refers to the order of calculation when multiple nodes work together to complete the complete inference, and is used to provide the basis for data interaction objects for relay-style inference execution; the model fragments executed for each target roadside node are used to indicate the model loading and task execution requirements of each target roadside node.

[0291] Furthermore, each target roadside node is determined from multiple roadside nodes that the vehicle will pass through within a preset time period in the future. These target roadside nodes will jointly perform the inference task.

[0292] As a specific example, the control unit can predict multiple roadside nodes that the vehicle will pass through within a preset time period based on the vehicle's navigation data. Based on the actual distance between the vehicle and each roadside node, it identifies multiple roadside nodes within a preset distance range (e.g., 200-500m), forming a complete set of candidate nodes. Then, a preset strategy formulation method is used to formulate a task execution strategy based on the complete set of candidate nodes. For example, the preset strategy formulation method may involve the control unit selecting different subsets of nodes from the complete set of candidate nodes to form multiple node combinations, configuring different execution orders and model shard loading schemes for each node combination, thus constructing multiple different task execution strategies. Subsequently, the total task execution time delay corresponding to each execution strategy is calculated, and the scheme with the shortest total time delay is determined as the final task execution strategy. The total time delay includes the transmission time of task data, inference results, and intermediate inference data, as well as the task execution time of each node depending on the corresponding model shards.

[0293] In one possible implementation, the task execution strategy can be formulated using the methods described in steps 1.1 to 1.1:

[0294] Step 1.1: Based on driving behavior data, predict the multiple roadside nodes that the vehicle will pass through in the future within a preset time period and the probability of arriving at each roadside node.

[0295] Among them, the arrival probability represents the confidence that the vehicle can reach the corresponding roadside node within a preset time period in the future; driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0296] The implementation method in this step can refer to step S301 in the aforementioned embodiment three, and will not be repeated here.

[0297] Step 1.2: Based on the arrival probability of each roadside node, determine at least two target roadside nodes from multiple roadside nodes, and determine the model loading requirements for each target roadside node.

[0298] In this step, based on the arrival probability of each roadside node, at least two target roadside nodes with higher arrival probabilities and higher execution efficiency can be identified from multiple roadside nodes, and the model loading requirements of each target roadside node when cooperating to perform inference tasks can be determined.

[0299] As a specific implementation scheme, this step can be implemented using steps 1.2.1 to 1.2.4 as follows:

[0300] Step 1.2.1: Identify roadside nodes whose arrival probability is greater than the first preset threshold as intermediate roadside nodes.

[0301] In this step, the roadside nodes with an arrival probability greater than the first preset threshold among the multiple roadside nodes identified in step 1.2 need to be identified as intermediate roadside nodes.

[0302] The number of intermediate roadside nodes may be one or more; the first preset threshold can be determined according to the actual application of the scheme, etc., and this application does not limit it, for example, it can be 0.6, 0.65, etc.

[0303] It should be understood that in practical applications, if there is only one intermediate roadside node in the current scenario, the reasoning can be performed using the methods described in Embodiments 1 to 4 above, that is, the intermediate roadside node is determined as the unique target roadside node; if there are at least two intermediate roadside nodes in the current scenario, the following steps 1.2.2 to 1.2.4 can be executed.

[0304] Step 1.2.2: Cluster the target inference model into multiple model subgraphs according to operator type.

[0305] Each model subgraph includes at least one model slice of the target inference model.

[0306] In this step, considering that different types of roadside nodes are typically equipped with different types of processors—specifically, roadside units typically use NPUs, charging piles typically use CPUs, and parking lot gateways typically use PG-1 processors with GPUs—and also considering that different types of processors have different computational efficiencies for different types of operators (e.g., NPUs excel at convolutional operators, CPUs excel at post-processing, and GPUs excel at attention operators), and each has its own preferences, it is necessary to cluster each level of the target inference model into multiple model subgraphs according to operator type. Each model subgraph includes at least one model slice of the target inference model, and the operator types of each included model slice are similar.

[0307] In detail, all network layers of the target inference model can be traversed, and the network layers can be classified according to the operator type of each network layer. Network layers with similar operator types are aggregated into a model subgraph. In this model subgraph, one or more network layers that were originally continuous in the original target inference model are divided into a model slice.

[0308] For example, the target inference model can be divided into three model subgraphs: a convolutional dense subgraph G_conv, a multi-head attention subgraph G_attn, and a post-processing subgraph G_post.

[0309] Specifically, all convolution-related operators (convolutional layers, batch normalization layers, activation layers, etc.) in the backbone and feature fusion networks of the target inference model are aggregated into a first-class model subgraph. The originally continuous network layers in this subgraph are then divided into a model slice, forming at least one convolution-dense model slice. The multi-head attention-related operators in the detector head of the target inference model are aggregated into a second-class model subgraph. The originally continuous network layers in this subgraph are then divided into a model slice, forming at least one attention-class model slice. Post-processing operators such as non-maximum suppression (NMS) and coordinate transformation in the target inference model are aggregated into a third-class model subgraph. The originally continuous network layers in this subgraph are then divided into a model slice, forming at least one post-processing model slice.

[0310] Step 1.2.3: Based on the node type of each intermediate roadside node, the operator type of each model subgraph, and the preset computing power adaptation mapping table, determine at least two target roadside nodes and the model subgraph matching each target roadside node from all intermediate roadside nodes.

[0311] Among them, the preset computing power adaptation mapping table represents the computing efficiency of different types of roadside nodes when executing different types of operators.

[0312] In this step, based on the node type of each intermediate roadside node and the operator type of the model subgraph, a preset computing power adaptation mapping table is used to determine at least two target roadside nodes with high cooperative execution efficiency and the execution scheme under high cooperative execution efficiency. This achieves the optimal matching of "computing power-nodes", so that each subgraph can run on the node it is best at, thereby improving the overall inference task execution efficiency.

[0313] In detail, for each model subgraph, all roadside nodes corresponding to the corresponding operator type and their efficiency values ​​can be found in the computing power adaptation mapping table. These nodes are then sorted from highest to lowest efficiency to generate a candidate node list for that model subgraph, ensuring that the most efficient node is selected first. Then, starting with the first model subgraph, the most efficient and unoccupied intermediate roadside node is assigned to each subgraph sequentially: if the most efficient node in the current subgraph is not assigned, it is directly selected as the execution node for that subgraph; if the most efficient node has already been assigned to another subgraph, the next most efficient node in the candidate list for that subgraph is selected, until an unoccupied intermediate roadside node is found. This process is repeated until all model subgraphs are assigned to a roadside node, thus obtaining at least two target roadside nodes and their matching model subgraphs. In some scenarios, if all intermediate roadside nodes are occupied but there are still model subgraphs without assigned execution nodes, nodes with lower computational loads or higher matching degrees with the current subgraph's operator type are selected from the already assigned target roadside nodes, and the model subgraph is appended and assigned to the corresponding node for collaborative execution.

[0314] Step 1.2.4: Determine the model subgraph matched for each target roadside node as the model loading requirement for the target roadside node.

[0315] In this step, for each target roadside node, each model fragment in the determined threshold-matched model subgraph needs to be identified as the model loading requirement for that target roadside node.

[0316] For example, if the currently determined intermediate roadside nodes include roadside unit nodes RSU-1, charging pile nodes CPO-1, and parking lot gateway nodes PG-1 deployed on the roadside, then after dividing the target inference model into a convolutional dense subgraph G_conv, a multi-head attention subgraph G_attn, and a post-processing subgraph G_post, at least one model fragment corresponding to the convolutional dense subgraph G_conv is determined as the model loading requirement of RSU-1, at least one model fragment corresponding to the multi-head attention subgraph G_attn is determined as the model loading requirement of PG-1, and at least one model fragment corresponding to the post-processing subgraph G_post is determined as the model loading requirement of CPO-1.

[0317] It should be understood that the methods in steps 1.1.2 to 1.2.4 above first select roadside nodes with an arrival probability higher than a first preset threshold as intermediate roadside nodes, then cluster the target inference model according to operator type into multiple model subgraphs containing at least one model piece, and finally determine at least two target roadside nodes and match them with corresponding model subgraphs based on the intermediate roadside node type, the operator type of the model subgraph, and the preset computing power adaptation mapping table. This clarifies the means of loading the model for each target roadside node, enabling model subgraphs of different operator types to accurately match heterogeneous roadside nodes with corresponding high-efficiency computing capabilities. This achieves operator-hardware affinity orchestration, routing operators with different features such as convolution, attention, and post-processing to the most matched heterogeneous hardware nodes, effectively avoiding the waste of computing resources and the inference efficiency bottleneck caused by operator-hardware mismatch. Compared with the traditional coarse-grained hierarchical segmentation scheme, this scheme ultimately improves the overall computing power utilization of roadside nodes and reduces end-to-end inference latency, further ensuring the stable and efficient execution of inference tasks.

[0318] Step 1.3: Obtain the task execution strategy based on the model loading requirements of each target roadside node.

[0319] In this step, the execution strategy is obtained according to the connection order of each segment in the target inference model; at the same time, the identifier of each target roadside node is obtained, and the model segment to be executed for each target roadside node is determined according to the model loading requirements of each target roadside node, thereby obtaining the task execution strategy.

[0320] This implementation method, as described in steps 1.1 to 1.3 above, first predicts multiple roadside nodes and their corresponding arrival probabilities that the vehicle will pass through in the future based on driving behavior data such as the vehicle's historical trajectory sequence, real-time positioning, speed, and heading angle. Then, it selects at least two target roadside nodes based on the arrival probabilities of each node and clarifies their model loading requirements. Finally, it generates a task execution strategy based on the model loading requirements. This approach ensures that the selection of task execution nodes is highly matched with the confidence level of the vehicle's actual driving, effectively eliminating the risk of strategy failure caused by nodes with low arrival probabilities, avoiding the ineffective consumption of model preloading resources and task interruption, and achieving a task execution strategy that is more closely aligned with the vehicle's actual driving path, thus resulting in greater reliability.

[0321] In some implementations, the vehicle can send real-time driving behavior data to the cloud, where the task execution strategy is determined. Finally, the vehicle receives the result from the cloud. This approach effectively reduces the computational resource consumption on the vehicle, while the higher computing power of the cloud ensures the real-time nature of the strategy formulation.

[0322] S502, send model preloading instructions to the first and second roadside nodes that indicate the task execution strategy.

[0323] The model preloading instruction includes the model identifier of the target inference model and the identifier of the corresponding model fragment.

[0324] In this step, in order to ensure that the vehicle can perform inference using the loaded target inference model when it passes the target roadside node, the control unit needs to send a model preloading instruction to each target roadside node after the target roadside node is determined and before the vehicle passes the target roadside node of the first model segment for executing the target inference model. This will enable each target roadside node to perform the preloading of the target inference model segment according to the model identifier and the identifier of the corresponding model segment indicated by the instruction after receiving the instruction.

[0325] The identifier of the model segment refers to the identifier of the model segment required to perform the corresponding inference task. This identifier can uniquely identify the model segment in the target inference model, so that the target roadside node can clearly know the model segment that should be loaded and executed for the segmented calculation of the inference task.

[0326] In some implementations, the control unit can send model preloading commands to each target roadside node via the V2X PC5 interface between the vehicle and the target roadside node.

[0327] Alternatively, in some implementations, the vehicle can also send a target roadside node model preloading instruction to the cloud, so that the cloud can send the corresponding model preloading instruction to each target roadside node through the communication link between the cloud and each target roadside node.

[0328] Alternatively, in some other implementations, if the task execution strategy is determined by the cloud, the cloud will directly send the corresponding model preloading instruction to each target roadside node indicated by the strategy through the communication link between the cloud and each target roadside node after determining the task execution strategy.

[0329] S503: Receive the model preloading instruction sent by the vehicle, and load the structure and parameters of the first model segment of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0330] The model preloading instruction includes the model identifier and the identifier of the first model fragment.

[0331] In this step, the first roadside node needs to receive the model preloading instruction sent by the vehicle. Using the model identifier and the corresponding model fragment identifier in the model preloading instruction as indexes, it loads the structure and parameters of the model fragment corresponding to the model fragment identifier (i.e., the first model fragment of the target inference model) in the target inference model corresponding to the model identifier. This allows the target roadside node to directly respond to each frame of task data sent by the vehicle as the subsequent vehicle passes through it, and directly use the loaded model fragment to perform inference.

[0332] The first roadside node is the first of at least two target roadside nodes pre-determined based on the vehicle's driving behavior data for performing the vehicle's inference task. In other words, the first roadside node is the target roadside node that indicates the first model segment of the target inference model to be executed in the task execution strategy.

[0333] It should be understood that the structure of a model segment is a part of the neural network layer layout and computational framework in the overall structure of the target inference model. It constitutes the computational logic basis for the model segment as an independent inference unit and is used to realize the data processing and computation of the corresponding stage in the complete inference process. The parameters of the model segment refer to the set of values ​​such as weights and biases in the parameters of the complete target inference model that correspond to the structure of the segment. They provide the corresponding feature rules and judgment basis for the model segment as an independent inference unit, supporting it to complete local inference computation.

[0334] Therefore, the structure of the first model segment of the target inference model refers to the first part of the neural network hierarchy layout and computation framework in the complete target inference model; the parameters of the first model segment refer to the set of partial weights and biases corresponding to the first part of the neural network hierarchy and computation framework.

[0335] S504: Receive the model preloading instruction sent by the vehicle, and load the structure and parameters of the second model segment of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0336] The model preloading instruction includes the model identifier and the identifier of the second model fragment.

[0337] In this step, the second roadside node needs to receive the model preloading instruction sent by the vehicle. Using the model identifier and the corresponding model fragment identifier in the model preloading instruction as indexes, it loads the structure and parameters of the model fragment corresponding to the model fragment identifier (i.e., the second model fragment of the target inference model) in the target inference model corresponding to the model identifier. This allows the target roadside node to directly respond to each frame of task data sent by the vehicle as the vehicle passes through it, and directly use the loaded model fragment to perform inference.

[0338] The second roadside node is the second of at least two target roadside nodes pre-determined based on the vehicle's driving behavior data for performing the vehicle's inference task. In other words, the second roadside node is the target roadside node in the task execution strategy that indicates the second model segment of the target inference model needs to be executed.

[0339] Correspondingly, the structure of the second model segment of the target inference model refers to the second part of the neural network hierarchy layout and computation framework in the complete target inference model; the parameters of the second model segment refer to the set of partial weights and biases corresponding to the second part of the neural network hierarchy and computation framework.

[0340] Furthermore, in practical applications, for scenarios where "the second model segment is an intermediate inference unit of the target inference model," the next model segment of that second model segment in the target inference model will be preloaded by at least two roadside nodes other than the second roadside node. The model preloading methods for other roadside nodes are similar to those for the second roadside node, and will not be listed here.

[0341] S505, in response to the inference task execution request sent by the vehicle's application layer, the first task data of the inference task is sent to the first roadside node.

[0342] The first task data includes vehicle identification, model identification for performing the inference task, inference task input data, and task execution strategy; the first roadside node preloads the first model fragment of the target inference model for performing the inference task.

[0343] In this step, after receiving the execution request of the inference task sent by the upstream application layer, the control unit will acquire the first task data in real time according to the inference requirements, and send the real-time first task data to the first roadside node while passing through the first roadside node.

[0344] It should be understood that before receiving the first task data sent by the vehicle, the first roadside node has already preloaded the first model fragment of the target inference model for performing the inference task, so that when the target roadside node receives the first frame of task data sent by the vehicle, it can directly use the preloaded model fragment to perform the inference task, effectively saving the model loading time of the node.

[0345] In one possible implementation, if the aforementioned task execution strategy is determined based on the arrival probability of each roadside node, then this step can send the first task data to the first roadside node when the arrival probability of the first roadside node reaches the second preset threshold, so as to ensure that the vehicle is passing through the first roadside node, thereby improving the accuracy of determining the timing of sending the first task data to the first roadside node and effectively ensuring the smooth execution of the inference task.

[0346] The second preset threshold will be greater than the first preset threshold. Its specific value can be determined according to the actual application of the scheme. This application does not impose specific restrictions on it. For example, it can be 0.8, 0.85, 0.9, etc.

[0347] S506, Receive the first task data of the inference task sent by the vehicle.

[0348] The first task data includes vehicle identification, model identification for performing the inference task, inference task input data, and task execution strategy.

[0349] In this step, the first roadside unit will receive the first task data of the inference task sent by the vehicle.

[0350] S507. According to the execution order, input the inference task input data into the first model segment of the target inference model corresponding to the pre-loaded model identifier to obtain the first inference result.

[0351] In this step, the first roadside node needs to input the inference task input data into the first model fragment of the target inference model corresponding to the pre-loaded model identifier, according to the execution order of each target roadside node as indicated by the execution order instruction. This allows the first roadside node to perform local inference according to the model fragment structure and parameters, and output the first inference result.

[0352] The first model slice is the first slice of the target inference model preloaded under the trigger of the model preload instruction sent before the vehicle sends the first task data; the first inference result is the output tensor obtained by the first model slice of the target inference model corresponding to the model identifier performing the inference task.

[0353] S508. According to the task execution strategy, send the second task data of the inference task to the second roadside node.

[0354] In this step, the first pathside node needs to determine the second pathside node of the second model fragment of the target inference model according to the execution order indicated in the task execution strategy, and obtain the node identifier corresponding to the second model fragment as indicated by the task execution strategy, and send the second task data of the inference task to the second pathside node corresponding to the identifier.

[0355] The second task data includes vehicle identification, first inference result, model identification, and task execution strategy, so that the second roadside node can continue to perform inference tasks.

[0356] S509. Receive the second task data of the inference task sent by the first roadside node, and input the first inference result into the second model fragment of the target inference model corresponding to the pre-loaded model identifier according to the execution order to obtain the second inference result.

[0357] In this step, the second roadside node needs to receive the second task data of the inference task sent by the first roadside node, and input the first inference into the second model fragment of the target inference model corresponding to the pre-loaded model identifier according to the execution order of each target roadside node indicated by the execution order, so that the second roadside node performs local inference according to the model fragment structure and parameters, and outputs the second inference result.

[0358] The second model slice is the second slice of the target inference model that is preloaded under the trigger of the model preloading instruction sent by the vehicle before receiving the second task data.

[0359] S510. Based on the vehicle's identification, send the second reasoning result to the vehicle.

[0360] In this step, because Figure 5 The demonstration shows the interaction logic of each subject in the execution method under the scenario that "the second model segment is the last inference unit of the target inference model". In view of this, the second inference result obtained by the second roadside node is the final output result of the target inference model. Accordingly, the second roadside unit will directly send the second inference result to the vehicle corresponding to the vehicle's identifier.

[0361] In addition, in practical applications, for the scenario where "the second model segment is an intermediate inference unit of the target inference model", the second roadside node needs to send a third inference task to the third roadside node that has preloaded the next model segment of the target inference model according to the task execution strategy. The third inference task includes the vehicle identifier, the second inference result, the model identifier, and the task execution strategy.

[0362] Specifically, the second roadside node needs to determine the target roadside node for executing the third model segment of the target inference model according to the execution order indicated in the task execution strategy, obtain the node identifier corresponding to the third model segment as indicated by the task execution strategy, send the third task data of the inference task to the target roadside node corresponding to the identifier, and hand it over to the target roadside node to continue executing the inference task until the target roadside node whose loaded model segment is the last inference unit of the target inference model outputs its corresponding inference result and returns the inference result to the vehicle corresponding to the vehicle identifier.

[0363] S512, Receive the second inference result returned by the second roadside node.

[0364] In this step, the vehicle's control unit will receive the second inference result returned by the second roadside node, and use the second inference result to return to the upper application layer to meet the data requirements of the application layer to perform operations such as decision-making or control.

[0365] In addition, in practical applications, for scenarios where "the second model segment is an intermediate inference unit of the target inference model", the vehicle's control unit will receive inference results returned by other roadside nodes.

[0366] The method provided in this embodiment is applied to the execution system and method of in-vehicle inference tasks. The vehicle first pre-determines all target roadside nodes to be used for its inference task based on driving behavior data, and sends model fragments instructing each target roadside node to pre-load the corresponding target inference model. Then, when the vehicle uses these target roadside nodes to perform the inference task, the first task data is sent to the first roadside node. This allows all target roadside nodes to sequentially and locally perform inference using the pre-loaded model fragments until the target roadside node with the last loaded model fragment returns the output inference result to the vehicle. This means that in scenarios where at least two roadside nodes are used for inference tasks, the vehicle does not need to wait for model loading when using each target roadside node, significantly shortening the time for the vehicle to process the first frame of task data and increasing the effective inference time for the vehicle to perform inference tasks using that node. This improves the real-time performance of inference task execution in vehicle-road cooperative inference scenarios. Furthermore, the multi-node collaborative inference method in this solution can further improve the real-time performance of inference task execution.

[0367] Furthermore, in one possible implementation, based on the aforementioned embodiments, after sending the model preloading instruction to the target roadside node, steps a and b can also be performed to further ensure that the inference task can be executed smoothly:

[0368] Step a: Obtain the load of each target roadside node in real time.

[0369] In this step, the control unit in the vehicle will acquire the load of each target roadside node in real time. In practical applications, the control unit can receive broadcast messages from each target roadside node in real time, which will carry the computing resource status of the corresponding node (including the node's load).

[0370] Step b: When the load of any target roadside node exceeds the preset load, the inference tasks that have not been completed at the target roadside node are migrated to other roadside nodes for execution.

[0371] In this step, before the inference task is completed using these target roadside nodes, if the control unit detects that the load of any target roadside node exceeds the preset load, it means that the existing computing resources of the target roadside node will not be able to stably support the normal execution of the remaining inference task, which may easily lead to increased computing latency, task blocking or even execution abnormality. Therefore, it is necessary to promptly migrate the unfinished inference task on the node to other roadside nodes with normal load and sufficient computing resources to continue execution.

[0372] Other roadside nodes may be pre-planned backup nodes or nodes adjacent to the target roadside node.

[0373] As a specific example, when the control unit determines that the load of a target roadside node is greater than a preset load threshold, it immediately issues a task migration command to the overloaded roadside node. This allows the overloaded roadside node to migrate the task execution progress, intermediate computation feature data, and unloaded model resources of the currently incomplete inference task to other roadside nodes with sufficient load, according to the command. The control unit also dynamically updates the global task execution strategy based on the migration results, so that when the control unit issues task data again, the relevant roadside nodes will continue to collaboratively complete the inference task according to the node execution order indicated by the updated task execution strategy.

[0374] For example, if the control unit detects that the load of node RSU-1 currently executing the inference task exceeds the 85% threshold, and the node has only completed the third model slice corresponding to the convolutional dense subgraph, then the remaining model slices in the convolutional dense subgraph need to be migrated to the neighboring node RSU-2 for execution.

[0375] The method provided in this implementation monitors the load status of each target roadside node in real time and promptly migrates unfinished inference tasks to other available roadside nodes when the node load exceeds a preset threshold. This avoids inference blocking, latency spikes, or task interruptions caused by overload of a single node, further ensuring that inference tasks can be executed smoothly and improving the overall computational stability and task execution reliability of the system.

[0376] Furthermore, Figure 6This is a flowchart illustrating an execution method for an in-vehicle inference task provided in Embodiment Six of this application. To further improve the security and reliability of the inference results, the method provided in this embodiment introduces an intermediate inference result verification scheme based on semantic summary fingerprints. The following example illustrates the method provided in this application using the scenario where "two target roadside nodes (including a first roadside node and a second roadside node) jointly execute the inference task, and the second model fragment loaded in the second roadside node is the last inference unit of the target inference model":

[0377] S601, In response to the execution request of the inference task sent by the vehicle's application layer, the first task data of the inference task is sent to the first roadside node.

[0378] The first roadside node is the first of two target roadside nodes pre-determined based on the vehicle's driving behavior data to perform the vehicle's reasoning task. The first roadside node pre-loads the first model fragment of the target reasoning model for performing the reasoning task.

[0379] S602. Receive the first task data of the inference task sent by the vehicle, and according to the execution order, input the inference task input data into the first model fragment of the target inference model corresponding to the pre-loaded model identifier to obtain the first inference result.

[0380] The implementation method and principle of step S602 that the first roadside node needs to execute can be referred to the aforementioned embodiments, and will not be repeated here.

[0381] It should be understood that intermediate inference results refer to the inference results output by the model segment of the last inference unit of the non-target inference model. Therefore, the first inference result can also be called the intermediate inference result.

[0382] It should be noted that, for the scenario where "at least two target roadside nodes jointly execute the inference task and the second model fragment loaded in the second roadside node is not the last inference unit of the target inference model", for each target inference unit, if the model fragment used for its task execution is another inference unit other than the last inference unit in the target inference model, its output inference result can be used as an intermediate inference result, and can be verified based on the semantic summary fingerprint-based execution result provided in this embodiment.

[0383] S603. Statistical feature extraction is performed on the first inference result to obtain the semantic summary fingerprint corresponding to the first inference result.

[0384] In this step, the first path node needs to extract statistical features from the intermediate inference result and calculate a semantic summary fingerprint containing at least one feature indicator.

[0385] In practical applications, the first roadside node can encapsulate the semantic digest fingerprint into a 128-byte fingerprint packet and send the fingerprint packet to the vehicle through its V2X PC5 interface with the vehicle.

[0386] In one possible implementation, the semantic summary fingerprint includes at least one of the mean, variance, quantiles, median, and sparse feature entropy of the intermediate inference results.

[0387] Among them, the mean refers to the arithmetic mean of all elements in the feature tensor corresponding to the intermediate inference result, used to characterize the overall numerical distribution center of the feature data, and can be obtained by summing all elements in the feature tensor and taking the average; the variance refers to the average of the squares of the differences between each element in the feature tensor and the mean, used to characterize the dispersion and distribution fluctuation of the feature data, and can be obtained by calculating the sum of the squares of the differences between each element and the mean and taking the average; the quantile refers to the value at a specific proportion after all elements in the feature tensor are sorted by value, used to characterize the distribution range and extreme value characteristics of the feature data, and can be obtained by sorting the feature elements and selecting the corresponding proportion of values; the median refers to the value in the middle position after the feature tensor elements are sorted by value, a special quantile, characterizing the center trend of the data, and can be obtained by sorting and taking the middle element; the sparse feature entropy is used to characterize the sparse distribution and information redundancy of the feature tensor of the intermediate inference result. The larger the value, the sparser the feature distribution and the more concentrated the information, and can be obtained by statistically analyzing the distribution probability of zero elements and non-zero elements in the feature tensor and calculating the information entropy.

[0388] For example, a semantic summary fingerprint can be represented in the following form:

[0389] F=(μ,σ 2 ,q 0.1 ,q 0.5 ,q 0.9 H sparse )

[0390] Where μ is the mean of the output tensor, σ 2 To output the variance of the tensor, q 0.1 q is the 0.1 quantile of the output tensor. 0.5 q is the 0.5 quantile of the output tensor. 0.9 H is the 0.9 quantile of the output tensor. sparse This is the sparse feature entropy of the output tensor.

[0391] It should be understood that this implementation method uses a semantic summary fingerprint, which includes at least one of the mean, variance, quantiles, median, and sparse feature entropy, to characterize intermediate inference results. This method can comprehensively depict the essential characteristics of the inference results from multiple dimensions, such as numerical center, degree of dispersion, overall distribution range, median trend, and sparse information structure. This makes the description of intermediate inference results more comprehensive, stable, and accurate, providing a reliable data foundation for subsequent verification of intermediate inference results based on semantic summary fingerprints, thereby improving the accuracy of subsequent verification.

[0392] S604. Based on the vehicle's identifier, send the semantic digest fingerprint to the vehicle.

[0393] In this step, the first roadside node will first send the semantic digest fingerprint to the vehicle corresponding to the vehicle's identifier.

[0394] S605. Obtain the semantic digest fingerprint sent by the first roadside node, and calculate the distance between the semantic digest fingerprint and the pre-stored fingerprint reference distribution.

[0395] In this step, the vehicle's control unit will receive and acquire the semantic digest fingerprint sent by the first roadside node, and then calculate the distance between the semantic digest fingerprint and the pre-stored fingerprint reference distribution.

[0396] The fingerprint reference distribution refers to the set of standard fingerprint distributions that are pre-constructed and stored for the model segment corresponding to the intermediate inference result, representing the normal inference output of the model segment. It is formed by statistical analysis of the legal semantic summary fingerprints obtained by the model segment under a large number of standard inputs, and is used as a benchmark for judging whether the inference result of the current roadside node is correct and reliable.

[0397] The distance between the semantic summary fingerprint and the fingerprint reference distribution can be calculated using conventional distance calculation methods, such as Euclidean distance, cosine distance, Manhattan distance, Chebyshev distance, and Mahalanobis distance. This application does not impose specific restrictions on the distance calculation method.

[0398] S606. If the distance is less than or equal to the preset first distance threshold, a verification pass command is sent to the first roadside node.

[0399] The verification pass instruction is used to trigger the target roadside node to continue the execution of the reasoning task based on the intermediate reasoning results; the setting of the first preset distance threshold can be determined according to the actual application situation, and this application does not impose specific restrictions on it.

[0400] In this step, if the distance is less than or equal to the preset first distance threshold, it means that the current inference result conforms to the normal distribution and meets the requirements for safe and accurate verification. At this time, the vehicle's control unit will directly send a verification pass command to the target roadside node.

[0401] S607. Receive the verification pass instruction and continue to advance the execution of the inference task based on the first inference result.

[0402] In this step, after the first pathside node receives the verification pass instruction, it can directly send the first task data to the second pathside node so that the second pathside node can continue to execute the subsequent inference task.

[0403] S608. If the distance is greater than the first distance threshold and less than the preset second distance threshold, a re-inference request is sent to the first roadside node.

[0404] The re-inference request is used to instruct the target roadside node to re-execute the inference task based on the corresponding task data; the setting of the second preset distance threshold can be determined according to the actual application situation, and this application does not impose specific restrictions on it.

[0405] In this step, if the distance is greater than the first distance threshold and less than the preset second distance threshold, it indicates that the first inference result is inaccurate and unsafe. At this time, the vehicle's control unit needs to send a re-inference request to the target roadside node.

[0406] In practical applications, vehicles can perform multiple verifications on the intermediate inference results corresponding to the same model segment until the number of verifications for a single node reaches the preset number of verifications (e.g., 2 times). If the distance corresponding to the latest intermediate inference result still does not meet the requirement of being less than or equal to the preset first distance threshold, the inference task of the target roadside node will be migrated to other roadside nodes for execution.

[0407] S609. Receive the re-inference request and re-execute the inference task based on the first task data.

[0408] In this step, after receiving the re-inference request, the first path node will re-execute the inference task based on the first task data and obtain a new first inference result; then it will perform verification on the new first inference result based on the semantic digest fingerprint until it receives a verification pass instruction.

[0409] S610. If the distance is greater than or equal to the second distance threshold, the inference task of the first roadside node is migrated to other roadside nodes for execution.

[0410] In this step, if the distance is greater than or equal to the second distance threshold, it indicates that the first roadside node has a great safety risk or malfunction. Therefore, the vehicle's control unit needs to transfer the inference task of the target roadside node to other roadside nodes for execution.

[0411] Other roadside nodes may be pre-planned backup nodes or nodes adjacent to the target roadside node.

[0412] As a specific example, if the distance is greater than or equal to the second distance threshold, a task migration instruction is immediately issued to the first roadside node, so that the first roadside node can synchronously migrate the task execution progress, intermediate computation feature data and unloaded model resources of the currently incomplete inference task to other roadside nodes according to the instruction, and dynamically update the global task execution strategy according to the migration result, so that when the control unit issues task data again, each relevant roadside node will continue to collaboratively complete the inference task according to the node execution order indicated by the updated task execution strategy.

[0413] The vehicle-mounted inference task execution method provided in this application first obtains the semantic digest fingerprint of the intermediate inference result output by the model fragment of the non-final inference unit corresponding to the roadside node, then calculates the distance between it and the fingerprint reference distribution, and performs control measures such as verification pass, re-inference or task migration according to the distance size. This achieves fine-grained reliable verification of the inference process, which can not only verify the legality of the node identity, but also effectively detect computational error faults such as hardware failure, firmware anomaly, numerical overflow, and inference deviation that cannot be covered by traditional digital signature schemes, thereby improving the security of inference computation from the root. At the same time, the lightweight semantic digest fingerprint is used to replace the transmission of high-dimensional original features, which greatly reduces the communication overhead. While ensuring the security and reliability of inference, it effectively reduces the pressure and computational overhead of vehicle-road cooperative communication.

[0414] Figure 7 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 7 of this application, as shown below. Figure 7 As shown, the vehicle-mounted inference task execution device 70 provided in this application embodiment includes:

[0415] The first sending module 701 is used to respond to the execution request of the inference task sent by the application layer of the vehicle and send the task data of the inference task to the target roadside node. The task data includes the model identifier for executing the inference task and the input data of the inference task. The target roadside node is a roadside node for executing the vehicle's inference task, which is determined in advance based on the vehicle's driving behavior data. The target roadside node is preloaded with the target inference model for executing the inference task.

[0416] The receiving module 702 is used to receive the inference results returned by the target roadside node.

[0417] In one possible implementation, the device 70 further includes:

[0418] The determination module 703 is used to determine the target roadside node to be used for inference tasks based on driving behavior data during vehicle operation;

[0419] The second sending module 704 is used to send a model preloading instruction to the target roadside node so that the target roadside node preloads the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model.

[0420] In one possible implementation, the determining module 703 is specifically used for:

[0421] Based on driving behavior data, predict multiple roadside nodes that the vehicle will pass through in a future preset time period and the arrival probability of each roadside node; the arrival probability represents the confidence that the vehicle can reach the corresponding roadside node within the preset time period.

[0422] The roadside node with the highest arrival probability among multiple roadside nodes is identified as the target roadside node.

[0423] The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0424] In one possible implementation, the first transmitting module 701 is specifically used for:

[0425] When the arrival probability of the target roadside node reaches a preset threshold, the task data is sent to the target roadside node.

[0426] In one possible implementation, the device 70 further includes:

[0427] The acquisition module 705 is used to obtain the latest time that the vehicle is expected to arrive at the target roadside node based on driving behavior data;

[0428] Add module 706 to add the latest time to the model preloading instruction; the latest time is used to trigger the target roadside node to delete the preloaded data.

[0429] The vehicle-mounted reasoning task execution device 70 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0430] Figure 8 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 8 of this application, as shown below. Figure 8 As shown, the vehicle-mounted inference task execution device 80 provided in this embodiment includes:

[0431] The first receiving module 801 is used to receive task data of the inference task sent by the vehicle. The task data includes the model identifier for performing the inference task and the inference task input data.

[0432] The inference module 802 is used to input the inference task input data into the target inference model corresponding to the pre-loaded model identifier to perform inference and obtain the inference result;

[0433] Return module 803 is used to return the reasoning result to the vehicle;

[0434] The target inference model is an inference model preloaded under the trigger of a model preload command sent before the vehicle sends task data.

[0435] In one possible implementation, the device 80 further includes:

[0436] The second receiving module 804 is used to receive the model preloading instruction sent by the vehicle, the model preloading instruction including the model identifier;

[0437] The preloading module 805 is used to load the structure and parameters of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0438] In one possible implementation, the model preloading instruction in the second receiving module further includes the latest estimated time for the vehicle to arrive at the target roadside node; the device 80 also includes

[0439] The deletion module 806 is used to delete the preloaded data corresponding to the target inference model if no task data is received before the latest time.

[0440] The vehicle-mounted reasoning task execution device 80 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0441] Figure 9 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 9 of this application, as shown below. Figure 9 As shown, the vehicle-mounted inference task execution device 90 provided in this application embodiment includes:

[0442] The first sending module 901 is used to send the first task data of the inference task to the first roadside node in response to the execution request of the inference task sent by the application layer of the vehicle.

[0443] The first task data includes vehicle identification, model identification for performing the inference task, inference task input data, and task execution strategy. The task execution strategy includes node identification and execution order of all target roadside nodes performing the inference task, as well as the model fragments executed for each target roadside node.

[0444] The first roadside node is the first of at least two target roadside nodes that are pre-determined based on the vehicle's driving behavior data to perform the vehicle's reasoning task. The first roadside node preloads the first model fragment of the target reasoning model for performing the reasoning task.

[0445] The receiving module 902 is used to receive the inference results returned by the second roadside node or other roadside nodes. The second roadside node preloads the second model fragment of the target inference model for performing the inference task.

[0446] In one possible implementation, the device 90 further includes:

[0447] The determination module 903 is used to determine the task execution strategy based on driving behavior data during vehicle operation;

[0448] The second sending module 904 is used to send a model preloading instruction to each target roadside node indicated by the task execution strategy, so that each target roadside node preloads different model fragments of the target inference model. The model preloading instruction includes the model identifier of the target inference model and the identifier of the corresponding model fragment.

[0449] In one possible implementation, the determining module 903 includes:

[0450] The prediction unit is used to predict, based on driving behavior data, multiple roadside nodes that the vehicle will pass through in the future within a preset time period and the probability of reaching each roadside node; the probability of reaching represents the confidence that the vehicle can reach the corresponding roadside node in the future within a preset time period.

[0451] The determination unit is used to determine at least two target roadside nodes from multiple roadside nodes based on the arrival probability of each roadside node, and to determine the model loading requirements of each target roadside node.

[0452] The first acquisition unit is used to acquire the task execution strategy based on the model loading requirements of each target roadside node;

[0453] The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

[0454] In one possible implementation, the determining unit is specifically used for:

[0455] Roadside nodes whose arrival probability is greater than the first preset threshold are identified as intermediate roadside nodes;

[0456] The target inference model is clustered into multiple model subgraphs according to operator type, and each model subgraph includes at least one model piece of the target inference model;

[0457] Based on the node type of each intermediate roadside node, the operator type of each model subgraph, and the preset computing power adaptation mapping table, at least two target roadside nodes and the model subgraph matched by each target roadside node are determined from all intermediate roadside nodes; wherein, the preset computing power adaptation mapping table represents the computational efficiency of different types of roadside nodes when executing different types of operators.

[0458] The model subgraph matched for each target roadside node is determined as the model loading requirement for the target roadside node.

[0459] In one possible implementation, the first transmitting module 901 is specifically used for:

[0460] When the arrival probability of the first roadside node reaches the second preset threshold, the first task data is sent to the first roadside node.

[0461] In one possible implementation, the device 90 further includes:

[0462] The acquisition module 905 is used to acquire the load of each target roadside node in real time;

[0463] The first migration module 906 is used to migrate the inference tasks that have not been completed at any target roadside node to other roadside nodes for execution when the load of any target roadside node exceeds the preset load.

[0464] In one possible implementation, the device 90 further includes:

[0465] Verification module 907 is specifically used for:

[0466] Obtain the semantic summary fingerprint sent by the target roadside node; the semantic summary fingerprint is obtained by extracting statistical features from the intermediate inference results output by the model segment corresponding to the target roadside node; the model segment is the inference unit other than the last inference unit in the target inference model;

[0467] Calculate the distance between the semantic summary fingerprint and the pre-stored fingerprint reference distribution;

[0468] If the distance is less than or equal to the preset first distance threshold, a verification pass command is sent to the target roadside node; the verification pass command is used to trigger the target roadside node to continue the execution of the inference task based on the intermediate inference results;

[0469] If the distance is greater than the first distance threshold and less than the preset second distance threshold, a re-inference request is sent to the target roadside node; the re-inference request is used to instruct the target roadside node to re-execute the inference task based on the corresponding task data;

[0470] If the distance is greater than or equal to the second distance threshold, the inference task of the target roadside node will be migrated to other roadside nodes for execution.

[0471] In one possible implementation, the semantic summary fingerprint in the verification unit includes at least one of the mean, variance, quantile, median, and sparse feature entropy of the intermediate inference results.

[0472] The vehicle-mounted reasoning task execution device 90 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0473] Figure 10 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment 10 of this application, as shown below. Figure 10 As shown, the vehicle-mounted inference task execution device 100 provided in this application embodiment includes:

[0474] The first receiving module 1001 is used to receive the first task data of the inference task sent by the vehicle. The first task data includes the vehicle identifier, the model identifier of the inference task, the inference task input data and the task execution strategy. The task execution strategy includes the node identifier and execution order of all target roadside nodes that execute the inference task and the model fragments executed for each target roadside node.

[0475] The inference module 1002 is used to input the inference task input data into the first model fragment of the target inference model corresponding to the pre-loaded model identifier according to the execution order, and obtain the first inference result;

[0476] The sending module 1003 is used to send the second task data of the inference task to the second roadside node according to the task execution strategy. The second task data includes the vehicle identifier, the first inference result, the model identifier and the task execution strategy, so that the second roadside node can continue to execute the inference task.

[0477] The first model slice is the first slice of the target inference model preloaded under the trigger of the model preload instruction sent before the vehicle sends the first task data.

[0478] In one possible implementation, the device 100 further includes:

[0479] The second receiving module 1004 is used to receive the model preloading instruction sent by the vehicle. The model preloading instruction includes the model identifier and the identifier of the first model segment.

[0480] The preloading module 1005 is used to load the structure and parameters of the first model segment of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0481] The vehicle-mounted reasoning task execution device 100 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0482] Figure 11 This is a schematic diagram of the structure of an execution device for an in-vehicle reasoning task provided in Embodiment Eleven of this application, as shown below. Figure 11 As shown, the vehicle-mounted inference task execution device 110 provided in this application embodiment includes:

[0483] The first receiving module 1101 is used to receive the second task data of the inference task sent by the first roadside node. The second task data includes the vehicle identifier, the first inference result, the model identifier, and the execution strategy. The first inference result is the output tensor obtained by executing the inference task of the first model segment of the target inference model corresponding to the model identifier. The task execution strategy includes the node identifier and execution order of all target roadside nodes that execute the inference task, as well as the model segment executed by each target roadside node.

[0484] The reasoning module 1102 is used to input the first reasoning result into the second model fragment of the target reasoning model corresponding to the pre-loaded model identifier according to the execution order, so as to obtain the second reasoning result;

[0485] The sending module 1103 is used to send the second inference result to the vehicle according to the vehicle's identifier if the second model segment is the last inference unit of the target inference model.

[0486] If the second model segment is an intermediate inference unit of the target inference model, then according to the task execution strategy, the third inference task is sent to the third roadside node that has preloaded the next model segment of the target inference model. The third inference task includes the vehicle identifier, the second inference result, the model identifier, and the task execution strategy.

[0487] The second model slice is the second slice of the target inference model that is preloaded under the trigger of the model preloading instruction sent by the vehicle before receiving the second task data.

[0488] In one possible implementation, the device 110 further includes:

[0489] The second receiving module 1104 is used to receive the model preloading instruction sent by the vehicle. The model preloading instruction includes the model identifier and the identifier of the second model segment.

[0490] The preloading module 1105 is used to load the structure and parameters of the second model segment of the target inference model corresponding to the model identifier according to the model preloading instruction.

[0491] The vehicle-mounted reasoning task execution device 100 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0492] Figure 12 This is a schematic diagram of the control unit provided in Embodiment Twelve of this application. Figure 12 As shown, the control unit 120 provided in this embodiment includes at least one processor 1201 and a memory 1202. Optionally, the control unit 120 further includes a communication component 1203. The processor 1201, memory 1202, and communication component 1203 are connected via a bus 1204.

[0493] In a specific implementation, at least one processor 1201 executes computer execution instructions stored in memory 1202, causing at least one processor 1201 to perform the above-described method.

[0494] The specific implementation process of processor 1201 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0495] For example, the control unit may be a domain controller in a vehicle, a control module in an on-board unit (OBU), etc.

[0496] Figure 13 This is a structural schematic diagram of the roadside node provided in Embodiment Thirteen of this application. Figure 13 As shown, the roadside node 130 provided in this embodiment includes at least one processor 1301 and a memory 1302. Optionally, the roadside node 130 further includes a communication component 1303. The processor 1301, memory 1302, and communication component 1303 are connected via a bus 1304.

[0497] In a specific implementation, at least one processor 1301 executes computer execution instructions stored in memory 1302, causing at least one processor 1301 to perform the above-described method.

[0498] The specific implementation process of processor 1301 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0499] For example, roadside nodes are roadside edge nodes such as roadside units, charging stations, or parking gateways.

[0500] In Embodiments 11 and 12 above, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0501] The memory may include read-only memory and random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0502] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0503] This application also provides a vehicle, including a vehicle body and a control unit as described in the foregoing embodiments, the control unit being used to implement the methods described in the foregoing embodiments.

[0504] This application also provides a computer program product, including a computer program that, when executed, implements the above-described method.

[0505] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0506] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as SRAM, EEPROM, EPROM, PROM, ROM, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0507] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside within an ASIC. Alternatively, the processor and the readable storage medium can exist as discrete components in a device.

[0508] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0509] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0510] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0511] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0512] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0513] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for executing an in-vehicle inference task, characterized in that, Control units used in vehicles include: In response to the inference task execution request sent by the application layer of the vehicle, the task data of the inference task is sent to the target roadside node; the task data includes the model identifier for executing the inference task and the inference task input data; the target roadside node is a roadside node pre-determined based on the vehicle's driving behavior data for executing the vehicle's inference task, and the target roadside node is pre-loaded with the target inference model for executing the inference task. Receive the inference results returned by the target roadside node.

2. The method according to claim 1, characterized in that, Before sending the inference task to the target roadside node, the method further includes: During vehicle operation, the target roadside node to be used for reasoning tasks is determined based on the driving behavior data. A model preloading instruction is sent to the target roadside node to enable the target roadside node to preload the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model.

3. The method according to claim 2, characterized in that, The step of determining the target roadside node to be used for the reasoning task based on the driving behavior data includes: Based on the driving behavior data, the system predicts multiple roadside nodes that the vehicle will pass through in a future preset time period and the probability of reaching each roadside node; the probability of reaching represents the confidence that the vehicle can reach the corresponding roadside node within the preset time period. The roadside node with the highest arrival probability among the plurality of roadside nodes is determined as the target roadside node; The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

4. The method according to claim 1, characterized in that, Sending the task data of the inference task to the target roadside node includes: When the arrival probability of the target roadside node reaches a preset threshold, the task data is sent to the target roadside node.

5. The method according to any one of claims 1 to 4, characterized in that, Before sending the model preloading instruction to the target roadside node, the method further includes: Based on the driving behavior data, obtain the latest time that the vehicle is expected to arrive at the target roadside node; The latest time is added to the model preloading instruction; the latest time is used to trigger the target roadside node to delete the preloaded data.

6. A method for executing an in-vehicle inference task, characterized in that, Applied to roadside nodes, including: Receive task data for an inference task sent by a vehicle, wherein the task data includes a model identifier for performing the inference task and inference task input data; The inference task input data is input into the pre-loaded target inference model corresponding to the model identifier to perform inference and obtain the inference result; The reasoning result is returned to the vehicle; The target inference model is an inference model preloaded under the trigger of a model preload instruction sent before the vehicle sends the task data.

7. The method according to claim 6, characterized in that, Before receiving the task data for the inference task sent by the vehicle, the method further includes: Receive a model preloading instruction sent by the vehicle, wherein the model preloading instruction includes the model identifier; According to the model preloading instruction, the structure and parameters of the target inference model corresponding to the model identifier are loaded.

8. The method according to claim 7, characterized in that, The model preloading instruction also includes the latest time that the vehicle is expected to arrive at the target roadside node; If the task data is not received before the latest time, the preloaded data corresponding to the target inference model is deleted.

9. A method for executing an in-vehicle inference task, characterized in that, Control units used in vehicles include: In response to the inference task execution request sent by the application layer of the vehicle, the first task data of the inference task is sent to the first roadside node; the first task data includes the vehicle's identifier, the model identifier for executing the inference task, the inference task input data, and the task execution strategy. The task execution strategy includes the node identifiers and execution order of all target roadside nodes executing the inference task, as well as the model fragments to be executed for each target roadside node; the first roadside node is the first of at least two target roadside nodes pre-determined based on the vehicle's driving behavior data for executing the vehicle's inference task, and the first roadside node pre-loads the first model fragment of the target inference model for executing the inference task; Receive the inference results returned by the second roadside node or other roadside nodes, wherein the second roadside node preloads the second model fragment of the target inference model for performing the inference task.

10. The method according to claim 9, characterized in that, Before sending the first task data of the inference task to the first pathside node, the method further includes: During vehicle operation, the task execution strategy is determined based on the driving behavior data; A model preloading instruction is sent to each target roadside node indicated by the task execution strategy, so that each target roadside node preloads different model fragments of the target inference model, wherein the model preloading instruction includes the model identifier of the target inference model and the identifier of the corresponding model fragment.

11. The method according to claim 10, characterized in that, Determining the task execution strategy based on the driving behavior data includes: Based on the driving behavior data, the system predicts multiple roadside nodes that the vehicle will pass through within a future preset time period and the probability of reaching each roadside node; the probability of reaching the roadside node represents the confidence level that the vehicle can reach the corresponding roadside node within the future preset time period. Based on the arrival probability of each roadside node, at least two target roadside nodes are determined from the plurality of roadside nodes, and the model loading requirements for each target roadside node are determined. Based on the model loading requirements of each target roadside node, the task execution strategy is obtained; The driving behavior data includes the vehicle's historical trajectory sequence and real-time motion status; the real-time motion status includes the vehicle's positioning data, speed data, and heading angle data.

12. The method according to claim 11, characterized in that, The step of determining at least two target roadside nodes from the plurality of roadside nodes based on the arrival probability of each roadside node, and determining the model loading requirements for each target roadside node, includes: Roadside nodes whose arrival probability is greater than the first preset threshold are identified as intermediate roadside nodes; The target inference model is clustered into multiple model subgraphs according to operator type, and each model subgraph includes at least one model piece of the target inference model; Based on the node type of each intermediate roadside node, the operator type of each model subgraph, and the preset computing power adaptation mapping table, at least two target roadside nodes and the model subgraph matched by each target roadside node are determined from all intermediate roadside nodes; wherein, the preset computing power adaptation mapping table represents the computational efficiency of different types of roadside nodes when executing different types of operators. The model subgraph matched for each target roadside node is determined as the model loading requirement for the target roadside node.

13. The method according to claim 9, characterized in that, Sending the first task data of the inference task to the first pathside node includes: When the arrival probability of the first roadside node reaches the second preset threshold, the first task data is sent to the first roadside node.

14. The method according to any one of claims 9 to 13, characterized in that, The method further includes: Real-time acquisition of the load of each target roadside node; When the load of any target roadside node exceeds the preset load, the inference tasks that are not completed at the target roadside node will be migrated to other roadside nodes for execution.

15. The method according to claims 9 to 13, characterized in that, The method further includes: Obtain the semantic summary fingerprint sent by the target roadside node; the semantic summary fingerprint is obtained by statistical feature extraction based on the intermediate inference results output by the model segment corresponding to the target roadside node; the model segment is the inference unit other than the last inference unit in the target inference model; Calculate the distance between the semantic summary fingerprint and the pre-stored fingerprint reference distribution; If the distance is less than or equal to a preset first distance threshold, a verification pass command is sent to the target roadside node; the verification pass command is used to trigger the target roadside node to continue the execution of the inference task based on the intermediate inference result; If the distance is greater than the first distance threshold and less than the preset second distance threshold, a re-inference request is sent to the target roadside node; the re-inference request is used to instruct the target roadside node to re-execute the inference task based on the corresponding task data; If the distance is greater than or equal to the second distance threshold, the inference task of the target roadside node will be migrated to other roadside nodes for execution.

16. The method according to claim 15, characterized in that, The semantic summary fingerprint includes at least one of the mean, variance, quantiles, median, and sparse feature entropy of the intermediate inference results.

17. A method for executing an in-vehicle reasoning task, characterized in that, Applied to the first roadside node, including: The system receives first task data for an inference task sent by a vehicle. The first task data includes the vehicle's identifier, the model identifier for executing the inference task, inference task input data, and a task execution strategy. The task execution strategy includes the node identifiers and execution order of all target roadside nodes executing the inference task, as well as the model fragments executed for each target roadside node. According to the execution order, the input data of the inference task is input into the first model fragment of the target inference model corresponding to the pre-loaded model identifier to obtain the first inference result; According to the task execution strategy, the second task data of the inference task is sent to the second roadside node. The second task data includes the vehicle identifier, the first inference result, the model identifier, and the task execution strategy, so that the second roadside node continues to execute the inference task. The first model slice is the first slice of the target inference model preloaded under the trigger of a model preload instruction sent before the vehicle sends the first task data.

18. The method according to claim 17, characterized in that, Before receiving the first task data sent by the vehicle, the method further includes: Receive a model preloading instruction sent by the vehicle, the model preloading instruction including the model identifier and the identifier of the first model segment; According to the model preloading instruction, the structure and parameters of the first model segment of the target inference model corresponding to the model identifier are loaded.

19. A method for executing an in-vehicle reasoning task, characterized in that, Applied to the second roadside node, including: The system receives second task data for an inference task sent by a first roadside node. The second task data includes vehicle identification, first inference result, model identification, and execution strategy. The first inference result is the output tensor obtained by executing the inference task on the first model segment of the target inference model corresponding to the model identification. The task execution strategy includes the node identification and execution order of all target roadside nodes executing the inference task, as well as the model segment executed by each target roadside node. According to the execution order, the first inference result is input into the second model fragment of the target inference model corresponding to the pre-loaded model identifier to obtain the second inference result; If the second model segment is the last reasoning unit of the target reasoning model, then the second reasoning result is sent to the vehicle according to the vehicle's identifier; If the second model segment is an intermediate inference unit of the target inference model, then according to the task execution strategy, a third inference task is sent to the third roadside node that has preloaded the next model segment of the target inference model. The third inference task includes the vehicle identifier, the second inference result, the model identifier, and the task execution strategy. The second model slice is a second slice of the target inference model that is preloaded under the trigger of a model preloading instruction sent by the vehicle before receiving the second task data.

20. The method according to claim 19, characterized in that, Before receiving the second task data of the inference task sent by the first pathside node, the method further includes: Receive a model preloading instruction sent by the vehicle, the model preloading instruction including the model identifier and the identifier of the second model segment; According to the model preloading instruction, the structure and parameters of the second model segment of the target inference model corresponding to the model identifier are loaded.

21. A control unit, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-5, or the method as described in any one of claims 9-17.

22. A roadside node, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 6-8, or the method as described in any one of claims 17-18, or the method as described in any one of claims 19-20.

23. The roadside node according to claim 22, characterized in that, The node type of the roadside node is a roadside unit, a charging station, or a parking lot gateway.

24. A vehicle, characterized in that, The vehicle includes a vehicle body and the control unit as described in claim 21.

Images