Evaluation system, method, device, equipment and medium for cloud-based control platform regulatory commands

Abstract

This invention discloses an evaluation system, method, apparatus, device, and medium for regulatory control instructions on a cloud-based platform. The system includes a roadside subsystem comprising a roadside time information acquisition module for transmitting acquired roadside time information to an instruction evaluation subsystem; a cloud subsystem comprising a data acquisition module for transmitting cloud time information and external data to the instruction evaluation subsystem; and a vehicle-side subsystem comprising a vehicle-side time information acquisition module for transmitting acquired vehicle-side time information to the instruction evaluation subsystem. The instruction evaluation subsystem acquires roadside time information, cloud time information, and vehicle-side time information, aligns these with the time information of the instruction evaluation subsystem, and calculates the value of the regulatory control instruction evaluation index based on the aligned time information of each subsystem. By adopting the above technical solution, the reliability of the regulatory control instruction evaluation results is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of autonomous driving technology, and more specifically, to an evaluation system, method, apparatus, device, and medium for control instructions of a cloud-based control platform. Background Technology

[0002] Traditional driver assistance and autonomous driving technologies refer to single-vehicle assisted driving and single-vehicle autonomous driving. Single-vehicle intelligent driving faces several pressing issues, such as the inability to perceive beyond line of sight and low synergy in group decision-making and control. To address these problems, academia and industry have proposed intelligent driving solutions based on vehicle-road-cloud integration. This solution, building upon single-vehicle intelligence, integrates vehicle-side perception, roadside perception, and cloud information to empower the vehicle, achieving enhanced perception and decision-making / control suggestions. These intelligent driving solutions require an evaluation mechanism to verify and assess their effectiveness.

[0003] In evaluating the effectiveness of vehicle-road-cloud integrated intelligent driving, current focus is primarily on system design verification and fusion perception testing. Traditional testing schemes generally utilize truth-based comparisons to test fusion perception, failing to propose clear evaluation models and methods from the perspective of business execution efficiency. Furthermore, in vehicle-road-cloud integrated intelligent driving solutions, perception, decision-making, control, and execution are implemented in different systems, requiring data transmission between systems through various network types. Traditional evaluation methods do not consider the impact of instruction latency during data transmission between different systems on the final performance of the vehicle, resulting in poor feasibility and reliability of the evaluation results. Summary of the Invention

[0004] This invention provides an evaluation system, method, apparatus, equipment, and medium for regulatory control instructions on a cloud-based control platform, in order to improve the feasibility and reliability of the evaluation results of regulatory control instructions.

[0005] The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide an evaluation system for control commands on a cloud-based control platform. This system includes: a vehicle-level terminal system, a road-level terminal system, a cloud-based subsystem, and a command evaluation subsystem.

[0007] The roadside terminal system includes a roadside time information acquisition module, which is used to acquire roadside time information and transmit the roadside time information to the instruction evaluation subsystem. The roadside time information includes: the exposure time point of the roadside sensor and the time point of the roadside output sensing result.

[0008] The cloud subsystem includes a data acquisition module, which is used to collect cloud time information and external data sent by vehicle terminal system and road terminal system, and transmit cloud time information and external data to instruction evaluation subsystem. The cloud time information includes: the time point when the cloud receives the road terminal perception result and the time point when the cloud issues the instruction.

[0009] The vehicle terminal system includes a vehicle terminal time information acquisition module, which is used to collect vehicle terminal time information and transmit the vehicle terminal time information to the instruction evaluation subsystem. The vehicle terminal time information includes: the time when the vehicle terminal receives the instruction from the cloud and the time when the vehicle terminal completes the execution of the cloud instruction.

[0010] The instruction evaluation subsystem is used to acquire roadside time information, cloud-based time information, and vehicle-side time information, and align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the instruction evaluation subsystem. Based on the aligned time information of each subsystem, the system calculates the value of the regulatory instruction evaluation index. Among them, the regulatory instruction evaluation index includes the instruction delay index, which is used to evaluate the rationality of the instruction delay.

[0011] The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

[0012] Optionally, the evaluation indicators for regulatory control instructions also include instruction execution indicators, which are used to evaluate the reliability of the execution of regulatory control instructions. Among them, the instruction execution indicators include instruction feedback rate, instruction execution rate, instruction execution success rate, and instruction execution efficiency.

[0013] Among them, the instruction feedback rate index represents the ratio of the first value of the instruction received by the cloud subsystem with feedback information from the vehicle subsystem to the second value of the instruction issued by the cloud subsystem.

[0014] The instruction execution rate metric represents the ratio of the third value of instructions that have feedback information from the vehicle subsystem and are adopted and executed by the vehicle subsystem to the second value of instructions issued by the cloud subsystem.

[0015] The instruction execution success rate metric represents the ratio of the fourth value of instructions successfully executed by the vehicle-to-vehicle subsystem with feedback information to the second value of instructions issued by the cloud subsystem.

[0016] The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem, when the vehicle's motion state information after successfully executing the instruction meets the expected motion state.

[0017] Correspondingly, the instruction evaluation subsystem is also configured as follows:

[0018] Within a set time period, the first, second, third, and fourth values ​​are statistically analyzed, and the ratios of the first and second values, the third and second values, the fourth and second values, and the fifth and second values ​​are calculated.

[0019] Secondly, embodiments of the present invention also provide an evaluation method for control instructions on a cloud control infrastructure platform, applied to an instruction evaluation subsystem, the method comprising:

[0020] The system acquires roadside time information, cloud-based time information, and vehicle-side time information. Roadside time information is acquired through the roadside time information acquisition module in the roadside subsystem; cloud-based time information is acquired through the data acquisition module in the cloud subsystem; and vehicle-side time information is acquired through the vehicle-side time information acquisition module in the vehicle-side subsystem. Roadside time information includes: the exposure time of the roadside sensor and the time of the roadside sensor outputting the sensing result. Cloud-based time information includes: the time the cloud receives the roadside sensing result and the time the cloud issues the command. Vehicle-side time information includes: the time the vehicle receives the cloud command and the time the vehicle completes executing the cloud command.

[0021] Align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the current instruction evaluation subsystem;

[0022] The values ​​of the planning and control command evaluation indicators are calculated based on the time information of each aligned subsystem. Among them, the planning and control command evaluation indicators include the command delay indicator, which is used to evaluate the rationality of the command delay.

[0023] The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

[0024] Optionally, the roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem, including:

[0025] Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem;

[0026] Determine the time difference information between each subsystem and the current instruction evaluation subsystem based on network transmission delay information;

[0027] Based on roadside time information, cloud-based time information, vehicle-side time information, and time difference information, the roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem.

[0028] Optionally, calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem, including:

[0029] Based on the current instruction evaluation subsystem time, heartbeat packets are sent to the time information acquisition module of the road terminal subsystem, the cloud data acquisition module of the cloud subsystem, and the vehicle terminal time information acquisition module of the vehicle terminal subsystem at set intervals, and the time of sending the heartbeat packets is recorded.

[0030] The system receives heartbeat data returned by the time information collection module, the cloud data collection module, and the vehicle-side time information collection module, and records the time when each heartbeat data is received.

[0031] For each subsystem other than the current instruction evaluation subsystem, the network transmission delay information between that subsystem and the current instruction evaluation subsystem is calculated based on the time difference between when the current instruction evaluation subsystem sends a heartbeat packet to it and when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

[0032] Optionally, the time difference information between each subsystem and the current instruction evaluation subsystem is determined based on network transmission delay information, including:

[0033] The time difference information between each subsystem and the current instruction evaluation subsystem is determined according to the following formula:

[0034] △t c-ts =t ts -t client -TD c2ts

[0035] Among them, t client t represents the current time information collected by the subsystem. ts This represents the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts This indicates the network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0036] Optionally, based on roadside time information, cloud-based time information, vehicle-side time information, and time difference information, the roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem, including:

[0037] The roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem according to the following formula to obtain the aligned time information:

[0038] t r-exp =t r-exp +△t r-c-ts

[0039] t r-out =t r-out +△t r-c-ts

[0040] t c-in =t c-in +△t c-c-ts

[0041] t c-out =t c-out +△t c-c-ts

[0042] t v-in =t v-in +△t v-c-ts

[0043] t v-finish =t v-finish +△t v-c-ts

[0044] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out t represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out Indicates the time point at which the command is issued from the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; t v-finish Indicates the time point at which the vehicle completes the execution of the instruction; Δt r-c-ts Δt represents the time difference between the terminal system and the current instruction evaluation subsystem. v-c-ts Δt represents the time difference between the vehicle terminal system and the current instruction evaluation subsystem. c-c-ts This represents the time difference between the cloud subsystem and the current instruction evaluation subsystem.

[0045] Thirdly, embodiments of the present invention also provide an evaluation device for control instructions of a cloud-based control platform, comprising:

[0046] The data access module is configured to acquire roadside time information, cloud-based time information, and vehicle-side time information. The roadside time information is acquired by a roadside time information acquisition module in the roadside subsystem; the cloud-based time information is acquired by a data acquisition module in the cloud subsystem; and the vehicle-side time information is acquired by a vehicle-side time information acquisition module in the vehicle-side subsystem. The roadside time information includes: the exposure time of the roadside sensor and the time of the roadside sensor outputting the sensing result. The cloud-based time information includes: the time of the cloud receiving the roadside sensing result and the time of the cloud issuing the command. The vehicle-side time information includes: the time of the vehicle receiving the cloud command and the time of the vehicle completing the cloud command.

[0047] The time alignment module is configured to align the roadside time information, the cloud-based time information, and the vehicle-side time information with the time information of the current instruction evaluation subsystem.

[0048] The indicator calculation module is configured to calculate the value of the regulatory control instruction evaluation indicator based on the time information of each aligned subsystem. The regulatory control instruction evaluation indicator includes an instruction delay indicator, which is used to evaluate the rationality of the instruction delay.

[0049] The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

[0050] Optional time alignment modules include:

[0051] The network transmission delay calculation unit is configured to calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem.

[0052] The time difference determination unit is configured to determine the time difference information between each subsystem and the current instruction evaluation subsystem based on the network transmission delay information.

[0053] The time alignment unit is configured to align the roadside time information, cloud-based time information, vehicle-side time information, and time difference information with the time information of the instruction evaluation subsystem.

[0054] Optionally, the network transmission delay calculation unit is specifically configured as follows:

[0055] Based on the current instruction evaluation subsystem time, heartbeat packets are sent to the time information acquisition module of the road terminal subsystem, the cloud data acquisition module of the cloud subsystem, and the vehicle terminal time information acquisition module of the vehicle terminal subsystem at set intervals, and the time of sending the heartbeat packets is recorded.

[0056] Receive heartbeat data returned by the time information acquisition module, the cloud data acquisition module, and the vehicle-side time information acquisition module, and record the time when each heartbeat data is received;

[0057] For each subsystem other than the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends the heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

[0058] Optional, the time difference determination unit is specifically configured as follows:

[0059] The time difference information between each subsystem and the current instruction evaluation subsystem is determined according to the following formula:

[0060] △t c-ts =t ts -t client -TD c2ts

[0061] Among them, t client t represents the current time information collected by the subsystem. ts Δt represents the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts This indicates the network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0062] Optional, time alignment unit, specifically configured as follows:

[0063] The roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem according to the following formula to obtain the aligned time information:

[0064] t r-exp =t r-exp +△t r-c-ts

[0065] t r-out =t r-out +△t r-c-ts

[0066] t c-in =t c-in +△t c-c-ts

[0067] t c-out =t c-out +△t c-c-ts

[0068] t v-in =t v-in +△t v-c-ts

[0069] t v-finish =t v-finish +△t v-c-ts

[0070] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out t represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out Indicates the time point at which the command is issued from the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; t v-finish Indicates the time point at which the vehicle completes the execution of the instruction; Δt r-c-ts Δt represents the time difference between the terminal system and the current instruction evaluation subsystem. v-c-ts Δt represents the time difference between the vehicle terminal system and the current instruction evaluation subsystem. c-c-ts This represents the time difference between the cloud subsystem and the current instruction evaluation subsystem.

[0071] Optionally, the evaluation indicators for regulatory control instructions also include instruction execution indicators, which are used to evaluate the reliability of the execution of regulatory control instructions. The instruction execution indicators include instruction feedback rate, instruction execution rate, instruction execution success rate, and instruction execution efficiency.

[0072] The instruction feedback rate index represents the ratio of a first value of an instruction received by the cloud subsystem with feedback information from the vehicle terminal system to a second value of an instruction issued by the cloud subsystem.

[0073] The instruction execution rate metric represents the ratio of the third value of the instructions that have feedback information from the vehicle terminal system and are adopted and executed by the vehicle terminal system to the second value of the instructions issued by the cloud subsystem.

[0074] The instruction execution success rate index represents the ratio of the fourth value of the instruction successfully executed by the vehicle terminal system with feedback information to the second value of the instruction issued by the cloud subsystem.

[0075] The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem, when the vehicle's motion state information after successfully executing the instruction meets the expected motion state.

[0076] Accordingly, the apparatus provided in the embodiments of the present invention further includes:

[0077] The instruction execution index value calculation module is configured to, within a set time period, statistically analyze the first value, the second value, the third value, and the fourth value, and calculate the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value.

[0078] Fourthly, embodiments of the present invention provide a computing device, the computing device comprising:

[0079] At least one processor is provided, and the processor is coupled to a memory. The memory stores a program or instructions that run on the processor. When the program or instructions are executed by the processor, they implement the evaluation method for cloud control infrastructure platform regulation instructions as provided in any embodiment of the present invention.

[0080] Fifthly, embodiments of the present invention provide a readable storage medium storing a program or instructions thereon, which, when executed by a processor, implements the evaluation method for cloud control infrastructure platform regulation instructions as provided in any embodiment of the present invention.

[0081] Sixthly, embodiments of the present invention provide a computer program, the computer program including program instructions, which, when executed by a computer, implement the evaluation method for cloud control infrastructure platform regulatory instructions as provided in any embodiment of the present invention.

[0082] In the technical solution provided by this invention, at the level of designing evaluation indicators for regulatory control instructions, two types of evaluation indicators are proposed based on the entire lifecycle of the instruction: instruction latency indicators and instruction execution indicators. Their definitions and calculation methods are clearly defined, filling the gap in the current lack of a clear testing and evaluation model for cloud-based instructions. Regarding evaluation technology, this invention proposes a distributed evaluation system architecture. Through the cooperation of the server and various clients, the statistical calculation of the evaluation indicator values ​​for each instruction is comprehensively realized. The calculation process considers the potential latency during the instruction's generation, transmission, and execution processes, as well as the instruction's execution status, effectively improving the feasibility and reliability of improving the evaluation results of regulatory control instructions.

[0083] The innovative aspects of this invention include:

[0084] 1. In terms of the design of evaluation indicators for regulatory commands, two types of evaluation indicators are proposed based on the entire life cycle of the command, including command latency indicators and command execution indicators. Their definitions and calculation methods are clarified, which fills the gap in the current lack of a clear test and evaluation model for cloud commands and is one of the innovations of this application.

[0085] 2. Regarding the evaluation technology, this invention proposes a distributed evaluation system architecture. Through the cooperation of the server and various clients, the statistical calculation of the evaluation index values ​​of each instruction is comprehensively realized. In the calculation process, the potential time delays and execution status of the instructions during the generation, transmission and execution processes are taken into account, which effectively improves the feasibility and reliability of improving the evaluation results of regulatory instructions. This is one of the innovations of this application.

[0086] 3. By utilizing the time difference between each subsystem and the instruction evaluation subsystem for time alignment, the problem of needing to mechanically modify the system clock is successfully avoided. This avoids the problem of the system time being inconsistent with its actual time due to mechanical modification of the system clock, which is one of the innovations of this application. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1a This is an instruction lifecycle diagram provided in Embodiment 1 of the present invention;

[0089] Figure 1b This is a structural block diagram of an evaluation system for control instructions on a cloud-based control platform provided in Embodiment 1 of the present invention;

[0090] Figure 2 A flowchart illustrating an evaluation method for control instructions of a cloud-based control platform provided in Embodiment 2 of the present invention;

[0091] Figure 3 This is a structural block diagram of an evaluation device for control instructions of a cloud-based control platform provided in Embodiment 3 of the present invention;

[0092] Figure 4 This is a structural block diagram of a computing device provided in Embodiment 4 of the present invention. Detailed Implementation

[0093] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0094] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0095] This invention discloses an evaluation system, method, apparatus, device, and medium for control instructions on a cloud-based control platform. These are described in detail below.

[0096] Example 1

[0097] To explain the various embodiments more clearly, the evaluation index model of the cloud control infrastructure platform's regulatory commands will be described in detail below. This evaluation index model is implemented through regulatory command evaluation indicators, which are designed and defined based on the command's lifecycle.

[0098] Please participate Figure 1a , Figure 1a The instruction lifecycle diagram provided in Embodiment 1 of the present invention is as follows: Figure 1a As shown, the lifecycle of an instruction consists of 7 processes and 3 stages. The 7 processes are roadside perception, road-to-cloud transmission, vehicle-side perception, vehicle-to-cloud transmission, cloud-to-vehicle processing, cloud-to-vehicle transmission, and vehicle-side processing. The 3 stages include instruction generation, instruction transmission, and instruction execution. The instruction generation stage includes 5 processes: roadside perception, vehicle-side perception, roadside transmission, vehicle-to-cloud transmission, and cloud-to-cloud processing. The instruction transmission stage includes the cloud-to-vehicle transmission process. The instruction execution stage includes the vehicle processing process.

[0099] In this process, vehicle-side perception and roadside perception run in parallel, converging in the cloud via vehicle-to-cloud and road-to-cloud transmissions respectively. Cloud-based processing enables fused perception. Due to the stability and controllability of the roadside sensors, the start time of the command's lifecycle is determined by the exposure time of the roadside sensors.

[0100] like Figure 1a As shown, there are six important time points in the entire lifecycle of an instruction. r-exp This indicates the exposure time of the roadside sensor, which is also the start point of the command lifecycle; t r-outt represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out This indicates the time point at which the command is issued after it is generated in the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; t v-finish This indicates the point in time when the vehicle finishes executing the instruction, which is also the point at which the instruction's lifecycle ends.

[0101] Based on the complete lifecycle of instructions, this invention constructs an evaluation index model for collaborative perception and decision-making instructions in a vehicle-road-cloud integrated system. This model includes two main categories of indicators: instruction latency and instruction execution, defined as shown in Table 1 below.

[0102] Table 1 Definition of Evaluation Indicators for Regulatory Control Commands

[0103]

[0104]

[0105] Based on the above evaluation index model, this embodiment provides an evaluation system for the control instructions of a cloud-based control platform, such as... Figure 1b As shown, the evaluation system includes: a vehicle terminal system 110, a road terminal system 120, a cloud subsystem 130, and a command evaluation subsystem 140, wherein...

[0106] The roadside terminal system 120 includes a roadside time information acquisition module 121, used to acquire roadside time information and transmit it to the instruction evaluation subsystem 140. The roadside time information includes: the exposure time point of the roadside sensor (t...). r-exp ), the time point of the roadside output sensing results (t) r-out );

[0107] The cloud subsystem 130 includes a data acquisition module 131, used to acquire cloud time information and external data sent by the vehicle terminal system 110 and the road terminal system 120, and transmit the cloud time information and external data to the instruction evaluation subsystem 140. The cloud time information includes the time point at which the cloud receives the road terminal sensing result (t). c-in ), the time point at which the command is issued from the cloud (t) c-out );

[0108] The vehicle terminal system 110 includes a vehicle terminal time information acquisition module 111, used to collect vehicle terminal time information and transmit it to the instruction evaluation subsystem 140. The vehicle terminal time information includes the time point at which the vehicle terminal receives the instruction from the cloud (t...). v-in ), the time point at which the vehicle completes the cloud command (t) v-finish );

[0109] The instruction evaluation subsystem 140 is used to acquire roadside time information, cloud-based time information, and vehicle-side time information, and to align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the instruction evaluation subsystem 140, and to calculate the value of the regulatory instruction evaluation index based on the aligned time information of each subsystem.

[0110] In this embodiment, the evaluation system for the cloud control platform's regulatory instructions is a distributed architecture, comprising a server and a client.

[0111] The server-side is the instruction evaluation subsystem, which includes five modules: data access, data preprocessing, data storage, time alignment, and indicator calculation. The data access module is responsible for connecting and communicating with the client. The data preprocessing module is responsible for cleaning and labeling the collected raw data. The data storage module is responsible for storing data from each stage and supporting various calculations. The time difference calculation uses the machine clock of the test server (which is set as the sole clock synchronization source) as the benchmark to calculate the time difference between the vehicle, the road, and the cloud. The indicator calculation module evaluates various indicators related to the instruction based on data storage and time difference calculation.

[0112] The client-side is divided into two types: a data acquisition client, which is the data acquisition module belonging to the cloud subsystem, and a time acquisition client, which is the time information acquisition module. The data acquisition client implements three functions: time acquisition, data acquisition, and transmission, and is installed in the cloud. The time acquisition client implements two functions: time acquisition and transmission, and is installed on both the vehicle and roadside. The one installed on the vehicle is the vehicle-side time information acquisition module, and the one installed on the roadside is the roadside time information acquisition module. The time acquisition function implemented on the client-side supports time difference calculation on the server side, but does not affect the machine clocks of the host (vehicle, road, and cloud).

[0113] In this embodiment, the evaluation index for regulatory commands includes a command latency index, which is used to represent the reasonableness of command latency. The command latency index includes: command generation latency index (TD). cmd-g ), Command Transmission Latency Index (TD) cmd-t ) and instruction execution latency metrics (TD) cmd-e ),in,

[0114] The instruction generation latency metric represents the time interval between the time the cloud issues the instruction and the exposure time of the on-site sensor, i.e., TD. cmd-g =t c-out -t r-exp ;

[0115] The command transmission latency metric represents the time interval between the time the vehicle receives the command and the time the cloud sends the command, i.e., TD. cmd-t =t v-in -t c-out ;

[0116] The instruction execution latency metric represents the time interval between the time when the vehicle completes the instruction execution and the time when the vehicle receives the instruction, i.e., TD. cmd-e =t v-finish -t v-in .

[0117] The time information used in the calculation of the above-mentioned instruction delay index (i.e., the parameters on the right side of the equation in the formula) is obtained by aligning the road segment time information of the road segment subsystem, the cloud time information of the cloud subsystem, and the vehicle-end time information of the vehicle terminal subsystem through the instruction evaluation subsystem. In this embodiment, the instruction evaluation subsystem aligns the time information of each subsystem through the following steps:

[0118] 1. Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem.

[0119] Specifically, for any target subsystem, the network transmission delay information between the target subsystem and the instruction evaluation subsystem can be obtained through the following process:

[0120] Based on the time of the current instruction evaluation subsystem, the current instruction evaluation subsystem sends heartbeat packets to the time information acquisition module of the road terminal subsystem, the cloud data acquisition module of the cloud subsystem, and the vehicle terminal time information acquisition module of the vehicle terminal subsystem at set intervals, and records the time of sending the heartbeat packets.

[0121] The current instruction evaluation subsystem receives heartbeat data returned by the time information acquisition module, the cloud data acquisition module, and the vehicle-side time information acquisition module, and records the time of receiving each heartbeat data. In this way, the time information acquisition module, the cloud data acquisition module, and the vehicle-side time information acquisition module maintain a continuously increasing transmission delay sequence within the current instruction evaluation subsystem. A specific delay value in the sequence represents the one-way network delay between that module and the current instruction evaluation subsystem at a given minute. That is, for each subsystem other than the current instruction evaluation subsystem, the network transmission delay information between that subsystem and the current instruction evaluation subsystem can be calculated based on the difference between the time the current instruction evaluation subsystem sends the heartbeat packet to it and the time the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem. This can be expressed by the following formula:

[0122]

[0123] Among them, TD c2ts t represents the network transmission delay between a target subsystem and the current instruction evaluation subsystem. ts2c-h This indicates the time t represents when the current instruction evaluation subsystem sends a heartbeat packet to the target subsystem. c2ts-h This indicates the time when the current instruction evaluation subsystem receives the heartbeat data returned by the target subsystem. The target subsystem can be any one of the cloud subsystem, road segment subsystem, or vehicle terminal subsystem.

[0124] 2. Determine the time difference information between each subsystem and the current instruction evaluation subsystem based on the network transmission delay information.

[0125] For a given target subsystem, the specific method for calculating the time difference information between the target subsystem and the current instruction evaluation subsystem can be as follows:

[0126] The client collects the machine time point of the host (vehicle, road, or cloud) and sends it to the instruction evaluation subsystem. After receiving the time, the instruction evaluation subsystem collects its own local time point. The calculation formula for the time difference (which may be positive or negative) is as follows:

[0127] △t c-ts =t ts -t client -TD c2ts

[0128] Among them, t client t represents the current time information collected by the subsystem. ts This represents the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts This indicates the network transmission latency between the subsystem and the current instruction evaluation subsystem.

[0129] In this embodiment, the time difference information can be used for the subsequent alignment process of different subsystems. By employing a high-density time error sequence to align time, the adaptability and accuracy of the algorithm are greatly improved.

[0130] 3. Align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the instruction evaluation subsystem. Specifically, the aligned time information can be calculated using the following formula:

[0131] t r-exp =t r-exp +△t r-c-ts

[0132] t r-out =t r-out+△t r-c-ts

[0133] t c-in =t c-in +△t c-c-ts

[0134] t c-out =t c-out +△t c-c-ts

[0135] t v-in =t v-in +△t v-c-ts

[0136] t v-finish =t v-finish +△t v-c-ts

[0137] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out t represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out Indicates the time point at which the command is issued from the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; t v-finish Indicates the time point at which the vehicle completes the execution of the instruction; Δt r-c-ts Δt represents the time difference between the terminal system and the current instruction evaluation subsystem. v-c-ts Δt represents the time difference between the vehicle terminal system and the current instruction evaluation subsystem. c-c-ts This represents the time difference between the cloud subsystem and the current instruction evaluation subsystem. By aligning the time according to the above calculation formula, all the time information on the left side of the equal sign is on the same time axis as the time information of the instruction evaluation subsystem.

[0138] In related technologies, aligning the time of other subsystems (vehicle terminal system, road segment subsystem, and cloud subsystem) by the instruction evaluation subsystem requires embedded modification of the clock information of other subsystems. In this embodiment, by utilizing the time difference between each subsystem and the instruction evaluation subsystem for time alignment, the problem of needing to mechanically modify the system clock is successfully avoided, thus preventing inconsistencies between the system time and the actual time caused by mechanical clock modifications. Furthermore, during the time alignment process, using a high-density time difference sequence to align time greatly improves the adaptability and accuracy of the algorithm, thereby effectively improving the accuracy of subsequent instruction delay index results.

[0139] In this embodiment, after calculating the aligned time information, the obtained time information can be substituted into the calculation formulas of each instruction latency index mentioned above to obtain the values ​​of each instruction latency index.

[0140] Furthermore, to more accurately and objectively evaluate the control commands of the cloud-based control platform, an aggregate value of the command latency index can be calculated based on the calculated command latency index value at a certain moment. This aggregate value is then used to evaluate the control commands of the cloud-based control platform. The aggregate value can be calculated by statistically analyzing the command latency index values ​​over a period of time. For example, it can be calculated according to time periods such as minutes, hours, or days, yielding the following percentile values: the maximum value (max), minimum value (min), mean value (mean), median value (P50), and P99 value (99% of the observations are lower than a certain command latency index value, and only 1% of the observations are higher than that command latency index value).

[0141] Furthermore, after obtaining the aggregated value of the command latency index, the magnitude of this aggregated value can be used to evaluate whether there are problems in each stage of the command, thereby assisting staff in troubleshooting related issues. For example, if the command generation latency index value is too large, such as exceeding a certain first threshold, it indicates that the command generation latency is too long and there is an unreasonable problem. In this case, it is possible to deduce that there is latency in the road segment perception process, road-to-cloud transmission process, vehicle-side perception process, vehicle-to-cloud transmission process, or cloud processing process related to the command generation latency index. Staff can then use this command generation latency index value to investigate each of these processes. Similarly, if the command transmission latency value is too large, such as exceeding a certain second threshold, it indicates that the command transmission latency is too long and there is an unreasonable problem in the command transmission process. In this case, it is possible to deduce that there is latency in the cloud-to-vehicle transmission process related to the command transmission latency index, and staff can then use this command transmission latency index value to investigate the cloud-to-vehicle transmission process. For example, if the instruction execution delay is too large, such as exceeding a certain third threshold, it indicates that there is an unreasonable problem in the instruction execution process on the vehicle side. In this case, it can be deduced that there is a delay in the vehicle side processing, and staff can check each link of the vehicle side processing based on the instruction execution delay index value.

[0142] Furthermore, in this embodiment, the evaluation index for regulatory control commands also includes command execution indicators, used to evaluate the reliability of regulatory control command execution. These command execution indicators include command feedback rate, command execution rate, command execution success rate, and command execution efficiency. As shown in Table 1 above, the command feedback rate is the ratio of the first value of the command received by the cloud subsystem with feedback information from the vehicle subsystem to the second value of the command issued by the cloud subsystem; the command execution rate is the ratio of the third value of the command adopted and executed by the vehicle subsystem with feedback information from the vehicle subsystem to the second value of the command issued by the cloud subsystem; the command execution success rate is the ratio of the fourth value of the command successfully executed by the vehicle subsystem with feedback information from the vehicle subsystem to the second value of the command issued by the cloud subsystem; and the command execution efficiency is the ratio of the fifth value of the command successfully executed by the vehicle to the second value of the command issued by the cloud subsystem when the vehicle's motion state information after successful command execution matches the expected motion state.

[0143] In this embodiment, the instruction evaluation subsystem, in addition to calculating the latency indicators of each instruction, is also used to calculate the values ​​of the instruction execution indicators. Specifically, the instruction evaluation subsystem is configured to: within a set time period, statistically analyze the following: a first value of instructions received by the cloud subsystem with feedback information from the vehicle terminal system; a second value of instructions issued by the cloud subsystem; a third value of instructions with feedback information from the vehicle terminal system and adopted and executed by the vehicle terminal system; and a fourth value of instructions with feedback information from the vehicle terminal system and successfully executed by the vehicle terminal system. The ratio of the first value to the second value is then calculated to obtain the instruction feedback rate indicator; the ratio of the third value to the second value is calculated to obtain the instruction execution rate indicator; and the ratio of the fourth value to the second value is calculated to obtain the instruction execution success rate indicator.

[0144] In addition, when calculating the efficiency index of instruction execution, for instructions successfully executed by the vehicle, vehicle status and performance information (such as vehicle speed, trajectory, etc.) within the most recent set time period (e.g., 20 seconds) after the instruction is successfully executed can be collected from the vehicle and the roadside. Then, machine learning, manual annotation, or a combination of machine learning and manual annotation can be used to determine whether the vehicle's motion state meets the instruction expectation, and the fifth value of instructions that meet the expected performance is counted. Then, the ratio of the fifth value and the second value is calculated to obtain the instruction execution efficiency index.

[0145] In this embodiment, to more accurately and objectively evaluate the control commands of the cloud control platform, an aggregated value of the command execution index can be calculated. This aggregated value is then used to evaluate the reliability of the cloud control platform's control commands, thereby reflecting the reliability of the cloud control platform model. For command execution index values ​​that do not conform to the specifications, this can assist staff in troubleshooting the causes. Adjustments can be made to the corresponding algorithms or hardware devices on the cloud, vehicle, or road segment to ensure the command execution index values ​​conform to the specifications, thereby improving the reliability of the cloud control platform.

[0146] In this embodiment, at the level of designing evaluation indicators for regulatory control instructions, two types of evaluation indicators are proposed based on the entire lifecycle of the instructions: instruction latency indicators and instruction execution indicators. Their definitions and calculation methods are clearly defined, filling the gap in the current lack of a clear testing and evaluation model for cloud-based instructions. Regarding evaluation technology, this embodiment proposes a distributed evaluation system architecture. Through the cooperation of the server and various clients, the statistical calculation of the evaluation indicator values ​​for each instruction is comprehensively realized. The calculation process takes into account the potential latency during the instruction's generation, transmission, and execution processes, as well as the instruction's execution status, effectively improving the feasibility and reliability of improving the evaluation results of regulatory control instructions.

[0147] Example 2

[0148] Figure 2 This is a flowchart of an evaluation method for regulatory commands on a cloud-based control platform, provided in Embodiment 2 of the present invention. This method can be executed by the command evaluation subsystem provided in the above embodiments. Specifically, the method provided in this embodiment details the calculation process of regulatory command evaluation indicators from the perspective of the command evaluation subsystem. Figure 2 As shown, the method provided in this embodiment specifically includes:

[0149] S210: Obtain time information from the roadside, cloud, and vehicle.

[0150] The roadside time information is collected by the roadside time information acquisition module in the roadside subsystem, the cloud-based time information is collected by the data acquisition module in the cloud-based subsystem, and the vehicle-side time information is collected by the vehicle-side time information acquisition module in the vehicle-side subsystem. The cloud-based time information includes: the time when the cloud receives the roadside sensing result and the time when the cloud issues the instruction; the vehicle-side time information includes: the time when the vehicle receives the cloud instruction and the time when the vehicle completes the execution of the cloud instruction. For details regarding the time points of each instruction, please refer to the relevant content provided in the above embodiments; further elaboration is not required here.

[0151] S220. Align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the current instruction evaluation subsystem.

[0152] In this embodiment, step S220 can be implemented through the following steps S221 to S223:

[0153] S221. Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem.

[0154] Specifically, step S221 can be implemented through the following steps A to C:

[0155] A. Based on the current instruction evaluation subsystem time, send heartbeat packets to the time information acquisition module of the road terminal subsystem, the cloud data acquisition module of the cloud subsystem, and the vehicle terminal time information acquisition module of the vehicle terminal subsystem at set intervals, and record the time of sending the heartbeat packets.

[0156] B. Receive heartbeat data returned by the time information collection module, the cloud data collection module, and the vehicle-side time information collection module, and record the time when each heartbeat data is received;

[0157] C. For each subsystem other than the current instruction evaluation subsystem, calculate the network transmission delay information between that subsystem and the current instruction evaluation subsystem based on the time difference between when the current instruction evaluation subsystem sends the heartbeat packet to it and when the current instruction evaluation subsystem receives the heartbeat data returned by that subsystem. This can be expressed by the following formula:

[0158]

[0159] Among them, TD c2ts t represents the network transmission delay between a target subsystem and the current instruction evaluation subsystem. ts2c-h This indicates the time t represents when the current instruction evaluation subsystem sends a heartbeat packet to the target subsystem. c2ts-h This indicates the time when the current instruction evaluation subsystem receives the heartbeat data returned by the target subsystem. The target subsystem can be any one of the cloud subsystem, road segment subsystem, or vehicle terminal subsystem.

[0160] S222. Determine the time difference information between each subsystem and the current instruction evaluation subsystem based on the network transmission delay information.

[0161] Specifically, for each subsystem in the vehicle terminal system, road terminal system, and cloud subsystem, the time difference information between that subsystem and the current instruction evaluation subsystem can be determined according to the following formula:

[0162] △t c-ts =tts -t client -TD c2ts

[0163] Among them, t client t represents the current time information collected by the subsystem. ts This represents the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts This indicates the network transmission latency between the subsystem and the current instruction evaluation subsystem.

[0164] S223. Based on roadside time information, cloud-based time information, vehicle-side time information, and time difference information, align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the instruction evaluation subsystem.

[0165] Specifically, the roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem according to the following formula:

[0166] t r-exp =t r-exp +△t r-c-ts

[0167] t r-out =t r-out +△t r-c-ts

[0168] t c-in =t c-in +△t c-c-ts

[0169] t c-out =t c-out +△t c-c-ts

[0170] t v-in =t v-in +△t v-c-ts

[0171] t v-finish =t v-finish +△t v-c-ts

[0172] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out t represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out Indicates the time point at which the command is issued from the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; tv-finish Indicates the time point at which the vehicle completes the execution of the instruction; Δt r-c-ts Δt represents the time difference between the terminal system and the current instruction evaluation subsystem. v-c-ts Δt represents the time difference between the vehicle terminal system and the current instruction evaluation subsystem. c-c-ts This represents the time difference between the cloud subsystem and the current instruction evaluation subsystem. By aligning the time according to the above calculation formula, all the time information on the left side of the equal sign is on the same time axis as the time information of the instruction evaluation subsystem.

[0173] In this embodiment, the network transmission delay information between each subsystem and the current instruction evaluation subsystem, the time difference information between each subsystem and the current instruction evaluation subsystem, and the calculation principle and effect of time alignment can be referred to the description of the above embodiment, and will not be repeated here.

[0174] S230. Calculate the value of the control instruction evaluation index based on the time information of each aligned subsystem.

[0175] The design of the regulatory control command evaluation index can be found in the description of the above embodiments, and will not be repeated here. In this embodiment, the regulatory control command evaluation index includes a command delay index, which is used to evaluate the rationality of the command delay. The command delay index includes: command generation delay index, command transmission delay index, and command execution delay index. The command generation delay index represents the time interval between the time the cloud sends the command and the exposure time of the roadside sensor; the command transmission delay index represents the time interval between the time the vehicle receives the command and the time the cloud sends the command; and the command execution delay index represents the time interval between the time the vehicle completes the command and the time the vehicle receives the command. Specifically, the values ​​of each command delay index can be calculated based on the time information of each aligned subsystem according to the following formula:

[0176] The instruction generation latency metric can be calculated using the following formula:

[0177] TD cmd-g =t c-out -t r-exp

[0178] The instruction transmission latency metric can be calculated using the following formula:

[0179] TD cmd-t =t v-in -t c-out

[0180] The instruction execution latency metric can be calculated using the following formula:

[0181] TDcmd-e =t v-finish -t v-in

[0182] In this embodiment, by calculating the instruction latency index value, the latency of the control instructions of the cloud control infrastructure platform can be effectively evaluated, thereby improving the accuracy and reliability of the evaluation results. Furthermore, to accurately and objectively evaluate the control instructions of the cloud control infrastructure platform, an aggregate value of the instruction latency index value can be calculated, and this aggregate value can be used to evaluate the control instructions of the cloud control infrastructure platform. The calculation of the aggregate value can be referred to the content of the above embodiment, and will not be repeated here.

[0183] In this embodiment, the evaluation index for regulatory control instructions also includes instruction execution index, which is used to evaluate the reliability of the execution of regulatory control instructions. The instruction execution index includes instruction feedback rate index, instruction execution rate index, instruction execution success rate index, and instruction execution efficiency index.

[0184] Among them, the instruction feedback rate index represents the ratio of the first value of the instruction received by the cloud subsystem with feedback information from the vehicle subsystem to the second value of the instruction issued by the cloud subsystem.

[0185] The instruction execution rate metric represents the ratio of the third value of instructions that have feedback information from the vehicle subsystem and are adopted and executed by the vehicle subsystem to the second value of instructions issued by the cloud subsystem.

[0186] The instruction execution success rate metric represents the ratio of the fourth value of instructions successfully executed by the vehicle-to-vehicle subsystem with feedback information to the second value of instructions issued by the cloud subsystem.

[0187] The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem, when the vehicle's motion state information after successfully executing the instruction meets the expected motion state.

[0188] Accordingly, the evaluation method for cloud control infrastructure platform control instructions provided in this embodiment also includes:

[0189] Within a set time period, the first, second, third, and fourth values ​​are statistically analyzed, and the ratios of the first and second values, the third and second values, the fourth and second values, and the fifth and second values ​​are calculated.

[0190] In this embodiment, by calculating the instruction execution index value, the vehicle-side execution of instructions on the cloud control infrastructure platform can be effectively evaluated, further improving the accuracy and reliability of the regulatory control instruction evaluation results. Furthermore, to accurately and objectively evaluate the regulatory control instructions of the cloud control infrastructure platform, an aggregate value of each instruction execution index value can be calculated, and the reliability of the regulatory control instructions of the cloud control infrastructure platform can be evaluated using this aggregate value. The calculation of the aggregate value can be referred to the content of the above embodiment, and will not be repeated here.

[0191] The technical solution provided in this embodiment proposes two types of evaluation indicators based on the entire lifecycle of the regulatory control command, namely, command latency indicators and command execution indicators, and clarifies their definitions and calculation methods, filling the gap in the current lack of a clear test and evaluation model for cloud-based commands. Furthermore, the calculation of the regulatory control command evaluation indicators takes into account the potential latency during command generation, transmission, and execution, as well as the vehicle-side execution of the commands, effectively improving the feasibility and reliability of improving the evaluation results of regulatory control commands.

[0192] Example 3

[0193] Figure 3 This is a structural block diagram of an evaluation device for control instructions on a cloud-based basic platform provided in Embodiment 3 of the present invention, as shown below. Figure 3 As shown, the device includes: a data access module 310, a time alignment module 320, and an indicator calculation module 330, wherein,

[0194] The data access module 310 is configured to acquire roadside time information, cloud-based time information, and vehicle-side time information. Roadside time information is acquired through a roadside time information acquisition module in the roadside subsystem; cloud-based time information is acquired through a data acquisition module in the cloud-based subsystem; and vehicle-side time information is acquired through a vehicle-side time information acquisition module in the vehicle-side subsystem. Specifically, cloud-based time information includes: the time the cloud receives the roadside sensing result and the time the cloud issues the command; vehicle-side time information includes: the time the vehicle receives the cloud command and the time the vehicle completes the execution of the cloud command.

[0195] The time alignment module 320 is configured to align the roadside time information, cloud-based time information, and vehicle-side time information with the time information of the current instruction evaluation subsystem.

[0196] The indicator calculation module 330 is configured to calculate the value of the regulatory control instruction evaluation indicator based on the time information of each aligned subsystem. The regulatory control instruction evaluation indicator includes the instruction delay indicator, which is used to evaluate the rationality of the instruction delay.

[0197] The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

[0198] Optional time alignment modules include:

[0199] The network transmission delay calculation unit is configured to calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem.

[0200] The time difference determination unit is configured to determine the time difference information between each subsystem and the current instruction evaluation subsystem based on network transmission delay information.

[0201] The time alignment unit is configured to align the roadside time information, cloud-based time information, vehicle-side time information, and time difference information with the time information of the instruction evaluation subsystem based on the roadside time information, cloud-based time information, vehicle-side time information, and time difference information.

[0202] Optionally, the network transmission delay calculation unit is specifically configured as follows:

[0203] Based on the current instruction evaluation subsystem time, heartbeat packets are sent to the time information acquisition module of the road terminal subsystem, the cloud data acquisition module of the cloud subsystem, and the vehicle terminal time information acquisition module of the vehicle terminal subsystem at set intervals, and the time of sending the heartbeat packets is recorded.

[0204] The system receives heartbeat data returned by the time information collection module, the cloud data collection module, and the vehicle-side time information collection module, and records the time when each heartbeat data is received.

[0205] For each subsystem, the network transmission delay information between the current instruction evaluation subsystem and the current instruction evaluation subsystem is calculated based on the time difference between when the current instruction evaluation subsystem sends a heartbeat packet to it and when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

[0206] Optional, the time difference determination unit is specifically configured as follows:

[0207] The time difference information between each subsystem and the current instruction evaluation subsystem is determined using the following formula:

[0208] △t c-ts =t ts-t client -TD c2ts

[0209] Among them, t client t represents the current time information collected by the subsystem. ts This represents the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts This indicates the network transmission latency between the subsystem and the current instruction evaluation subsystem.

[0210] Optional, time alignment unit, specifically configured as follows:

[0211] The roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem according to the following formula to obtain the aligned time information:

[0212] t r-exp =t r-exp +△t r-c-ts

[0213] t r-out =t r-out +△t r-c-ts

[0214] t c-in =t c-in +△t c-c-ts

[0215] t c-out =t c-out +△t c-c-ts

[0216] t v-in =t v-in +△t v-c-ts

[0217] t v-finish =t v-finish +△t v-c-ts

[0218] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out t represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out Indicates the time point at which the command is issued from the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; t v-finish Indicates the time point at which the vehicle completes the execution of the instruction; Δt r-c-tsΔt represents the time difference between the terminal system and the current instruction evaluation subsystem. v-c-ts Δt represents the time difference between the vehicle terminal system and the current instruction evaluation subsystem. c-c-ts This represents the time difference between the cloud subsystem and the current instruction evaluation subsystem.

[0219] Optionally, the evaluation indicators for regulatory control instructions also include instruction execution indicators, which are used to evaluate the reliability of the execution of regulatory control instructions. Among them, the instruction execution indicators include instruction feedback rate, instruction execution rate, instruction execution success rate, and instruction execution efficiency.

[0220] Among them, the instruction feedback rate index represents the ratio of the first value of the instruction received by the cloud subsystem with feedback information from the vehicle subsystem to the second value of the instruction issued by the cloud subsystem.

[0221] The instruction execution rate metric represents the ratio of the third value of instructions that have feedback information from the vehicle subsystem and are adopted and executed by the vehicle subsystem to the second value of instructions issued by the cloud subsystem.

[0222] The instruction execution success rate metric represents the ratio of the fourth value of instructions successfully executed by the vehicle-to-vehicle subsystem with feedback information to the second value of instructions issued by the cloud subsystem.

[0223] The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem, when the vehicle's motion state information after successfully executing the instruction meets the expected motion state.

[0224] Accordingly, the apparatus provided in the embodiments of the present invention further includes:

[0225] The instruction execution index value calculation module is configured to, within a set time period, statistically analyze the first value, the second value, the third value, and the fourth value, and calculate the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value.

[0226] The cloud-based control platform regulation and control instruction evaluation device provided in this embodiment of the invention can execute the cloud-based control platform regulation and control instruction evaluation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in the above embodiments can be found in the cloud-based control platform regulation and control instruction evaluation method provided in any embodiment of the invention.

[0227] Example 4

[0228] Figure 4 This is a structural block diagram of a computing device provided in Embodiment 4 of the present invention, as shown below. Figure 4 As shown, the computing device includes:

[0229] At least one processor Figure 4 The image shows a processor 520.

[0230] The processor 520 is coupled to the memory 510, which stores a program or instruction that runs on the processor 520. When the program or instruction is executed by the processor 520, it implements the evaluation method of the cloud control basic platform regulation instruction provided in any embodiment of the present invention.

[0231] Based on the above method embodiments, another embodiment of the present invention provides a readable storage medium storing a program or instructions thereon, which, when executed by a processor, causes the processor to implement the evaluation method of cloud control infrastructure platform regulation instructions as described in any of the above embodiments.

[0232] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0233] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An evaluation system for control commands on a cloud-based control platform, characterized in that, include: The system comprises a vehicle terminal system, a road terminal system, a cloud-based subsystem, and a command evaluation subsystem. The roadside terminal system includes a roadside time information acquisition module, which is used to acquire roadside time information and transmit the roadside time information to the instruction evaluation subsystem. The roadside time information includes: the exposure time point of the roadside sensor and the time point of the roadside output sensing result. The cloud subsystem includes a data acquisition module, which is used to collect cloud time information and external data sent by the vehicle terminal system and the road terminal system, and transmit the cloud time information and the external data to the instruction evaluation subsystem. The cloud time information includes the time point at which the cloud receives the road terminal perception result and the time point at which the cloud issues the instruction. The vehicle terminal system includes a vehicle terminal time information acquisition module, which is used to collect vehicle terminal time information and transmit the vehicle terminal time information to the instruction evaluation subsystem. The vehicle terminal time information includes: the time when the vehicle terminal receives the instruction from the cloud and the time when the vehicle terminal completes the execution of the cloud instruction. The instruction evaluation subsystem is used to acquire the roadside time information, the cloud-based time information, and the vehicle-side time information, and align the roadside time information, the cloud-based time information, and the vehicle-side time information with the time information of the instruction evaluation subsystem. The subsystem then calculates the value of the regulatory instruction evaluation index based on the aligned time information of each subsystem. The regulatory instruction evaluation index includes an instruction delay index, which is used to evaluate the rationality of the regulatory instruction delay. The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

2. The system according to claim 1, characterized in that, The evaluation indicators for regulatory control instructions also include instruction execution indicators, which are used to evaluate the reliability of the execution of regulatory control instructions. The instruction execution indicators include instruction feedback rate, instruction execution rate, instruction execution success rate, and instruction execution efficiency. The instruction feedback rate index represents the ratio of a first value of an instruction received by the cloud subsystem with feedback information from the vehicle terminal system to a second value of an instruction issued by the cloud subsystem. The instruction execution rate metric represents the ratio of the third value of the instructions that have feedback information from the vehicle terminal system and are adopted and executed by the vehicle terminal system to the second value of the instructions issued by the cloud subsystem. The instruction execution success rate index represents the ratio of the fourth value of the instruction successfully executed by the vehicle terminal system with feedback information to the second value of the instruction issued by the cloud subsystem. The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem, when the vehicle's motion state information after successfully executing the instruction meets the expected motion state. Accordingly, the instruction evaluation subsystem is further configured as follows: Within a set time period, the first, second, third, and fourth values ​​are statistically analyzed, and the ratios of the first and second values, the third and second values, the fourth and second values, and the fifth and second values ​​are calculated.

3. A method for evaluating control commands on a cloud-based control platform, applied to a command evaluation subsystem, characterized in that, The method includes: The system acquires roadside time information, cloud-based time information, and vehicle-side time information. The roadside time information is acquired through a roadside time information acquisition module in the roadside subsystem; the cloud-based time information is acquired through a data acquisition module in the cloud subsystem; and the vehicle-side time information is acquired through a vehicle-side time information acquisition module in the vehicle-side subsystem. The roadside time information includes: the exposure time of the roadside sensor and the time of the roadside sensor outputting the sensing result. The cloud-based time information includes: the time of the cloud receiving the roadside sensing result and the time of the cloud issuing the command. The vehicle-side time information includes: the time of the vehicle receiving the cloud command and the time of the vehicle completing the cloud command. Align the roadside time information, the cloud-based time information, and the vehicle-side time information with the time information of the current instruction evaluation subsystem; The values ​​of the planning and control command evaluation indicators are calculated based on the time information of each aligned subsystem. The planning and control command evaluation indicators include command delay indicators, which are used to evaluate the rationality of the planning and control command delay. The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

4. The method according to claim 3, characterized in that, Aligning the roadside time information, the cloud-based time information, and the vehicle-side time information with the time information of the instruction evaluation subsystem includes: Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem; The time difference information between each subsystem and the current instruction evaluation subsystem is determined based on the network transmission delay information. Based on the roadside time information, the cloud-based time information, the vehicle-side time information, and the time difference information, the roadside time information, the cloud-based time information, and the vehicle-side time information are aligned with the time information of the instruction evaluation subsystem.

5. The method according to claim 4, characterized in that, The calculation of network transmission delay information between each subsystem and the current instruction evaluation subsystem includes: Based on the current instruction evaluation subsystem time, heartbeat packets are sent to the time information acquisition module of the road terminal subsystem, the cloud data acquisition module of the cloud subsystem, and the vehicle terminal time information acquisition module of the vehicle terminal subsystem at set intervals, and the time of sending the heartbeat packets is recorded. Receive heartbeat data returned by the time information acquisition module, the cloud data acquisition module, and the vehicle-side time information acquisition module, and record the time when each heartbeat data is received; For each subsystem other than the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends the heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

6. The method according to claim 4, characterized in that, The step of determining the time difference information between each subsystem and the current instruction evaluation subsystem based on the network transmission delay information includes: The time difference information between each subsystem and the current instruction evaluation subsystem is determined according to the following formula: △t c-ts =t ts -t client -TD c2ts Among them, t client t represents the current time information collected by the subsystem. ts Δt represents the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts This indicates the network transmission delay between the subsystem and the current instruction evaluation subsystem.

7. The method according to claim 4, characterized in that, The step of aligning the roadside time information, cloud-based time information, vehicle-side time information, and time difference information with the time information of the instruction evaluation subsystem includes: The roadside time information, cloud-based time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem according to the following formula to obtain the aligned time information: t r-exp =t r-exp +△t r-c-ts t r-out =t r-out +△t r-c-ts t c-in =t c-in +△t c-c-ts t c-out =t c-out +△t c-c-ts t v-in =t v-in +△t v-c-ts t v-finish =t v-finish +△t v-c-ts Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out t represents the time point at which the sensing results are output from the roadside; c-in This indicates the time point at which the cloud receives the roadside sensing results; t c-out Indicates the time point at which the command is issued from the cloud; t v-in Indicates the time point at which the vehicle receives the instruction; t v-finish Indicates the time point at which the vehicle completes the execution of the instruction; Δt r-c-ts Δt represents the time difference between the terminal system and the current instruction evaluation subsystem. v-c-ts Δt represents the time difference between the vehicle terminal system and the current instruction evaluation subsystem. c-c-ts This represents the time difference between the cloud subsystem and the current instruction evaluation subsystem.

8. An evaluation device for control commands of a cloud-based control platform, characterized in that, include: The data access module is configured to acquire roadside time information, cloud-based time information, and vehicle-side time information. The roadside time information is acquired by a roadside time information acquisition module in the roadside subsystem; the cloud-based time information is acquired by a data acquisition module in the cloud subsystem; and the vehicle-side time information is acquired by a vehicle-side time information acquisition module in the vehicle-side subsystem. The roadside time information includes: the exposure time of the roadside sensor and the time of the roadside sensor outputting the sensing result. The cloud-based time information includes: the time of the cloud receiving the roadside sensing result and the time of the cloud issuing the command. The vehicle-side time information includes: the time of the vehicle receiving the cloud command and the time of the vehicle completing the cloud command. The time alignment module is configured to align the roadside time information, the cloud-based time information, and the vehicle-side time information with the time information of the current instruction evaluation subsystem. The indicator calculation module is configured to calculate the value of the regulatory control instruction evaluation indicator based on the time information of each aligned subsystem. The regulatory control instruction evaluation indicator includes an instruction delay indicator, which is used to evaluate the rationality of the regulatory control instruction delay. The instruction latency metrics include: instruction generation latency, instruction transmission latency, and instruction execution latency. The instruction generation latency represents the time interval between the time the cloud sends the instruction and the exposure time of the roadside sensor. The instruction transmission latency represents the time interval between the time the vehicle receives the instruction and the time the cloud sends the instruction. The instruction execution latency represents the time interval between the time the vehicle completes the instruction and the time the vehicle receives the instruction.

9. A computing device, characterized in that, It includes at least one processor coupled to a memory, the memory storing a program or instructions that run on the processor, the program or instructions being executed by the processor to implement the steps of the evaluation method for cloud control infrastructure platform regulatory instructions as described in any one of claims 3 to 7.

10. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instruction is executed by the processor, it implements the steps of the evaluation method for the cloud control basic platform regulation and control instructions as described in any one of claims 3 to 7.

Images