Evaluation system, method, device and equipment for regulation and control instruction of cloud control basic platform and medium

Abstract

The embodiment of the invention discloses an evaluation system, method, device, equipment and medium for a regulation and control instruction of a cloud control basic platform, in the system, a road end subsystem comprises a road end time information acquisition module used for transmitting acquired road end time information to an instruction evaluation subsystem; the cloud subsystem comprises a data acquisition module and is used for transmitting cloud time information and external data to the instruction evaluation subsystem; the vehicle-end subsystem comprises a vehicle-end time information acquisition module which is used for transmitting acquired vehicle-end time information to the instruction evaluation subsystem; the instruction evaluation subsystem is used for acquiring road end time information, cloud end time information and vehicle end time information, aligning the road end time information, the cloud end time information and the vehicle end time information with the time information of the instruction evaluation subsystem, and calculating the value of a regulation and control instruction evaluation index according to the aligned time information of each subsystem; by adopting the technical scheme, the reliability of the regulation and control instruction evaluation result is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of autonomous driving technology, and more specifically, to a system, method, device, equipment and medium for evaluating regulatory control instructions of a cloud control basic platform. Background Art

[0002] Traditional assisted driving and autonomous driving technologies refer to single-vehicle assisted driving and single-vehicle autonomous driving. Single-vehicle intelligent driving has many problems that need to be solved, such as the inability to perceive beyond visual range, low group decision-making and regulatory coordination effects, etc. In response to such problems, academia and industry have proposed an intelligent driving solution based on vehicle-road-cloud integration. Based on the intelligence of a single vehicle, this solution integrates vehicle-side perception, road-side perception and cloud information, empowers a single vehicle, and realizes perception enhancement and decision-making and regulatory suggestions. The above intelligent driving solution requires a set of evaluation mechanisms to verify and evaluate the effect of intelligent driving.

[0003] In terms of the evaluation of the effect of vehicle-road-cloud integrated intelligent driving, the current focus is on system design verification and fusion perception testing. Traditional testing schemes generally use the method of true value comparison to test fusion perception, and no clear evaluation model and evaluation method are proposed from the perspective of business execution performance. In addition, in the vehicle-road-cloud integrated intelligent driving solution, perception, decision-making, regulation and execution are implemented in different systems, and data between systems needs to be transmitted through multiple types of networks. However, traditional evaluation methods do not consider the impact of instruction delays when different systems transmit data on the final execution performance of the vehicle, and the feasibility and reliability of their evaluation results are poor. Summary of the invention

[0004] Embodiments of the present invention provide a system, method, apparatus, device and medium for evaluating regulatory control instructions of a cloud control basic platform to improve the feasibility and reliability of regulatory control instruction evaluation results.

[0005] The specific technical solutions are as follows:

[0006] In a first aspect, an embodiment of the present invention provides an evaluation system for regulation and control instructions of a cloud control basic platform, the system comprising: a vehicle subsystem, a road subsystem, a cloud subsystem and an instruction evaluation subsystem, wherein:

[0007] The roadside subsystem includes a roadside time information collection module, which is used to collect roadside time information and transmit the roadside time information to the instruction evaluation subsystem, wherein the roadside time information includes: the exposure time point of the roadside sensor and the time point when the roadside outputs the perception result;

[0008] The cloud subsystem includes a data collection module, which is used to collect cloud time information and external data sent by the vehicle-side subsystem and the road-side subsystem, and transmit the cloud time information and external data to the instruction evaluation subsystem, wherein the cloud time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction;

[0009] The vehicle-side subsystem includes a vehicle-side time information collection module, which is used to collect vehicle-side time information and transmit the vehicle-side time information to the instruction evaluation subsystem, wherein the vehicle-side time information includes: the time point when the vehicle-side receives the cloud instruction and the time point when the vehicle-side completes the execution of the cloud instruction;

[0010] The command evaluation subsystem is used to obtain the road-side time information, the cloud-side time information and the vehicle-side time information, and align the road-side time information, the cloud-side time information and the vehicle-side time information with the time information of the command evaluation subsystem, and calculate the value of the regulatory control command evaluation index according to the aligned time information of each subsystem, wherein the regulatory control command evaluation index includes the command delay index, which is used to evaluate the rationality of the command delay;

[0011] Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index. Among them, the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side completes the execution of the instruction and the time point when the vehicle-side receives the instruction.

[0012] Optionally, the regulatory control instruction evaluation index further includes an instruction execution index, which is used to evaluate the reliability of the execution of the regulatory control instruction, wherein the instruction execution index includes an instruction feedback rate index, an instruction execution rate index, an instruction execution success rate index, and an instruction execution efficiency index;

[0013] The command feedback rate indicator represents the ratio of a first value of the command with feedback information from the vehicle-side subsystem received by the cloud subsystem to a second value of the command issued by the cloud subsystem;

[0014] The command execution rate indicator represents the ratio of the third value of the command with feedback information from the vehicle-side subsystem and adopted and executed by the vehicle-side subsystem to the second value of the command issued by the cloud subsystem;

[0015] The command execution success rate indicator represents the ratio of the fourth value of the command with feedback information from the vehicle-side subsystem and successfully executed by the vehicle-side subsystem to the second value of the command issued by the cloud subsystem;

[0016] The command execution efficiency index represents the ratio of the fifth value of the command successfully executed by the vehicle to the second value of the command issued by the cloud subsystem when the motion state information after the vehicle successfully executes the command meets the expected motion state;

[0017] Correspondingly, the execution evaluation subsystem is also configured as follows:

[0018] Within a set time period, the first value, the second value, the third value and the fourth value are counted, and the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value are calculated.

[0019] In a second aspect, an embodiment of the present invention further provides a method for evaluating a cloud control basic platform regulation and control instruction, which is applied to an instruction evaluation subsystem. The method includes:

[0020] Obtain road-side time information, cloud-side time information and vehicle-side time information, wherein the road-side time information is collected by the road-side time information collection module in the road-side subsystem, the cloud-side time information is collected by the data collection module in the cloud-side subsystem, and the vehicle-side time information is collected by the vehicle-side time information collection module in the vehicle-side subsystem, wherein the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the vehicle-side time information includes: the time point when the vehicle receives the cloud-side instruction and the time point when the vehicle completes the cloud-side instruction;

[0021] Align the road-side time information, cloud-side time information and vehicle-side time information with the time information of the current command evaluation subsystem;

[0022] Calculate the value of the regulatory control instruction evaluation index according to the aligned time information of each subsystem, wherein the regulatory control instruction evaluation index includes an instruction delay index, which is used to evaluate the rationality of the instruction delay;

[0023] Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index. Among them, the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side completes the execution of the instruction and the time point when the vehicle-side receives the instruction.

[0024] Optionally, aligning the road-side time information, the cloud-side time information, and the vehicle-side time information with the time information of the instruction evaluation subsystem includes:

[0025] Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem respectively;

[0026] Determine the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information;

[0027] Based on the road-side time information, cloud-side time information, vehicle-side time information and time difference information, the road-side time information, cloud-side time information and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem.

[0028] Optionally, the network transmission delay information between each subsystem and the current instruction evaluation subsystem is calculated separately, including:

[0029] Based on the time of the current instruction evaluation subsystem, a heartbeat packet is sent to the time information collection module of the roadside subsystem, the cloud data collection module of the cloud subsystem, and the vehicle-side time information collection module of the vehicle-side subsystem at set intervals, and the time of sending the heartbeat packet is recorded;

[0030] Receive the heartbeat data returned by the time information collection module, the cloud data collection module and the vehicle-side time information collection module respectively, and record the time when each heartbeat data is received;

[0031] For each subsystem except the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends a heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

[0032] Optionally, determining the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information includes:

[0033] The time difference information between each subsystem and the current instruction evaluation subsystem is determined according to the following formula:

[0034] △t c-ts =t ts -t client -TD c2ts

[0035] Among them, t client Indicates the current time information collected by the subsystem, t ts Indicates the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts The network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0036] Optionally, based on the road-side time information, the cloud-side time information, the vehicle-side time information and the time difference information, the road-side time information, the cloud-side time information and the vehicle-side time information are aligned with the time information of the instruction evaluation subsystem, including:

[0037] According to the following formula, the road-side time information, cloud-side time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem to obtain the aligned time information:

[0038] t r-exp =t r-exp +△t r-c-ts

[0039] t r-out =t r-out +△t r-c-ts

[0040] t c-in =t c-in +△t c-c-ts

[0041] t c-out =t c-out +△t c-c-ts

[0042] t v-in =t v-in +△t v-c-ts

[0043] t v-finish =t v-finish +△t v-c-ts

[0044] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out Indicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the cloud sends the command; t v-in Indicates the time when the vehicle receives the command; t v-finish Indicates the time point when the vehicle completes the execution of the command; Δt r-c-ts Indicates the time difference between the subsystem of the route and the current instruction evaluation subsystem; Δt v-c-ts Indicates the time difference between the vehicle subsystem and the current instruction evaluation subsystem; Δt c-c-ts Indicates the time difference between the cloud subsystem and the current instruction evaluation subsystem.

[0045] In a third aspect, an embodiment of the present invention further provides an evaluation device for a cloud control basic platform regulation and control instruction, comprising:

[0046] The data access module is configured to obtain road-side time information, cloud-side time information and vehicle-side time information, wherein the road-side time information is collected by the road-side time information collection module in the road-side subsystem, the cloud-side time information is collected by the data collection module in the cloud-side subsystem, and the vehicle-side time information is collected by the vehicle-side time information collection module in the vehicle-side subsystem, wherein the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the vehicle-side time information includes: the time point when the vehicle receives the cloud-side instruction and the time point when the vehicle executes the cloud-side instruction;

[0047] A time alignment module, configured to align the road-side time information, the cloud-side time information and the vehicle-side time information with the time information of the current instruction evaluation subsystem;

[0048] An indicator calculation module is configured to calculate a value of a regulatory control instruction evaluation indicator according to the aligned time information of each subsystem, wherein the regulatory control instruction evaluation indicator includes an instruction delay indicator for evaluating the rationality of the instruction delay;

[0049] Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index, wherein the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side executes the instruction and the time point when the vehicle-side receives the instruction.

[0050] Optional, time alignment module, including:

[0051] A network transmission delay calculation unit is configured to respectively calculate network transmission delay information between each subsystem and the current instruction evaluation subsystem;

[0052] A time difference determination unit, configured to determine the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information;

[0053] The time alignment unit is configured to align the road-side time information, the cloud-side time information, the vehicle-side time information and the time difference information with the time information of the instruction evaluation subsystem.

[0054] Optionally, the network transmission delay calculation unit is specifically configured as follows:

[0055] Taking the time of the current instruction evaluation subsystem as a reference, heartbeat packets are sent to the time information collection module of the road subsystem, the cloud data collection module of the cloud subsystem, and the vehicle-side time information collection module of the vehicle-side subsystem at set intervals, and the time of sending the heartbeat packets is recorded;

[0056] Receive the heartbeat data returned by the time information acquisition module, the cloud data acquisition module and the vehicle-side time information acquisition module respectively, and record the time when each heartbeat data is received;

[0057] For each subsystem except the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends a heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

[0058] Optionally, the time difference determination unit is specifically configured to:

[0059] The time difference information between each subsystem and the current instruction evaluation subsystem is determined according to the following formula:

[0060] △t c-ts =t ts -t client -TD c2ts

[0061] Among them, t client Indicates the current time information collected by the subsystem, t ts Indicates the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts The time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts The network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0062] Optionally, the time alignment unit is specifically configured as follows:

[0063] According to the following formula, the road-side time information, the cloud-side time information, the vehicle-side time information and the time information of the instruction evaluation subsystem are aligned to obtain the aligned time information:

[0064] t r-exp =t r-exp +△t r-c-ts

[0065] t r-out =t r-out +△t r-c-ts

[0066] t c-in =t c-in +△t c-c-ts

[0067] t c-out =t c-out +△t c-c-ts

[0068] t v-in =t v-in +△t v-c-ts

[0069] t v-finish =t v-finish +△t v-c-ts

[0070] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out Indicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the cloud sends the command; t v-in Indicates the time when the vehicle receives the command; t v-finish Indicates the time point when the vehicle completes the execution of the command; Δt r-c-ts Indicates the time difference between the subsystem of the route and the current instruction evaluation subsystem; Δt v-c-ts Indicates the time difference between the vehicle subsystem and the current instruction evaluation subsystem; Δt c-c-ts Indicates the time difference between the cloud subsystem and the current instruction evaluation subsystem.

[0071] Optionally, the regulatory control instruction evaluation index further includes an instruction execution index, which is used to evaluate the reliability of the execution of the regulatory control instruction, wherein the instruction execution index includes an instruction feedback rate index, an instruction execution rate index, an instruction execution success rate index, and an instruction execution efficiency index;

[0072] The command feedback rate indicator represents the ratio of a first value of the command with feedback information from the vehicle-side subsystem received by the cloud subsystem to a second value of the command issued by the cloud subsystem;

[0073] The instruction execution rate indicator represents the ratio of a third value of instructions with feedback information from the vehicle-side subsystem and adopted and executed by the vehicle-side subsystem to a second value of instructions issued by the cloud subsystem;

[0074] The instruction execution success rate indicator represents the ratio of a fourth value of instructions having feedback information from the vehicle-side subsystem and successfully executed by the vehicle-side subsystem to a second value of instructions issued by the cloud subsystem;

[0075] The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem when the motion state information after the vehicle successfully executes the instruction meets the expected motion state;

[0076] Accordingly, the device provided by the embodiment of the present invention further includes:

[0077] The instruction execution index value calculation module is configured to count the first value, the second value, the third value and the fourth value within a set time period, and calculate the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value.

[0078] In a fourth aspect, an embodiment of the present invention provides a computing device, the computing device comprising:

[0079] At least one processor is coupled to a memory, and the memory stores programs or instructions running on the processor. When the programs or instructions are executed by the processor, a method for evaluating the regulation and control instructions of a cloud control basic platform as provided in any embodiment of the present invention is implemented.

[0080] In a fifth aspect, an embodiment of the present invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, a method for evaluating the regulation and control instructions of a cloud control basic platform as provided in any embodiment of the present invention is implemented.

[0081] In a sixth aspect, an embodiment of the present invention provides a computer program, which includes program instructions, and when the program instructions are executed by a computer, implements an evaluation method for cloud control basic platform regulation and control instructions as provided in any embodiment of the present invention.

[0082] In the technical solution provided by the embodiment of the present invention, at the level of designing evaluation indicators for regulatory control instructions, two types of evaluation indicators are proposed according to the entire life cycle of the instruction, including instruction delay indicators and instruction execution indicators, and their definitions and calculation methods are clarified, filling the gap that there is currently no clear test and evaluation model for cloud-based instructions. In terms of evaluation technology, the embodiment of the present invention proposes a distributed evaluation system architecture, which comprehensively realizes the statistical calculation of the evaluation index values ​​of each instruction through the cooperation of the server and each client. In the calculation process, the delay that may exist in the generation process, transmission process and execution process of the instruction and the execution of the instruction are taken into account, which effectively improves the feasibility and reliability of the evaluation results of regulatory control instructions.

[0083] The innovative features of the embodiments of the present invention include:

[0084] 1. In terms of the design of regulatory instruction evaluation indicators, two types of evaluation indicators are proposed based on the entire life cycle of the instruction, including instruction latency indicators and instruction execution indicators, and their definitions and calculation methods are clarified, which fills the gap that there is currently no clear test and evaluation model for cloud instructions, and is one of the innovations of this application.

[0085] 2. In terms of evaluation technology, the embodiment of the present invention proposes a distributed evaluation system architecture, which, through the cooperation of the server and each client, comprehensively realizes the statistical calculation of the evaluation index value of each instruction. During the calculation process, the possible delays in the generation process, transmission process and execution process of the instruction as well as the execution status of the instruction are taken into account, which effectively improves the feasibility and reliability of the evaluation results of the regulatory control instructions, which is one of the innovations of the present application.

[0086] 3. By utilizing the time difference between each subsystem and the instruction evaluation subsystem for time alignment, the problem of needing to mechanically modify the system clock is successfully avoided, thereby avoiding the problem of inconsistency between the system time and its actual time caused by mechanically modifying the system clock, which is one of the innovations of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0088] Figure 1a The instruction life cycle diagram provided by the first embodiment of the present invention;

[0089] Figure 1b A structural block diagram of an evaluation system for cloud control basic platform regulation and control instructions provided in the first embodiment of the present invention;

[0090] Figure 2 A flowchart of a method for evaluating a cloud control basic platform regulation and control instruction provided in Embodiment 2 of the present invention;

[0091] Figure 3 A structural block diagram of a cloud control basic platform regulatory control instruction evaluation device provided in Embodiment 3 of the present invention;

[0092] Figure 4 A structural block diagram of a computing device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0093] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0094] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0095] The embodiments of the present invention disclose a system, method, device, equipment and medium for evaluating regulation and control instructions of a cloud control basic platform, which are described in detail below.

[0096] Embodiment 1

[0097] In order to explain the contents of each embodiment more clearly and understandably, the evaluation index model of the cloud control basic platform regulation and control instructions is first introduced in detail below. The evaluation index model is implemented through the regulation and control instruction evaluation index, which is designed and defined according to the life cycle of the instruction.

[0098] Please attend Figure 1a , Figure 1a The instruction life cycle diagram provided in the first embodiment of the present invention is as follows: Figure 1a As shown in the figure, the life cycle of an instruction includes 7 processes and 3 stages. The 7 processes are road-side perception, road-cloud transmission, vehicle-side perception, vehicle-cloud transmission, cloud-processing, cloud-vehicle transmission and vehicle-side processing; the 3 stages include instruction generation, instruction transmission and instruction execution. The instruction generation stage includes 5 processes: road-side perception, vehicle-side perception, road-side transmission, vehicle-cloud transmission and cloud-processing. The instruction transmission stage includes the cloud-vehicle transmission process. The instruction execution stage includes the vehicle processing process.

[0099] Among them, the perception of the vehicle side and the road side are parallel, and they are respectively transmitted to the cloud through the vehicle-to-cloud transmission and the road-to-cloud transmission, and are processed through the cloud to achieve fusion perception. Due to the stability and controllability of the road side, the start time of the life cycle of the command adopts the exposure time of the road side sensor.

[0100] like Figure 1a As shown in the figure, there are 6 important time points in the entire life cycle of the instruction. r-exp Indicates the exposure time of the roadside sensor, which is also the starting time of the instruction life cycle; t r-outIndicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the command is issued after the cloud generates the command; t v-in Indicates the time when the vehicle receives the command; t v-finish Indicates the time point when the vehicle completes executing the instruction, which is also the end point of the instruction life cycle.

[0101] Based on the complete life cycle of the instruction, the present invention constructs a vehicle-road-cloud integrated system collaborative perception and decision-making instruction evaluation index model, which includes two major indicators: instruction latency and instruction execution, and their definitions are shown in Table 1 below:

[0102] Table 1 Definition of evaluation indicators for regulatory directives

[0103]

[0104]

[0105] Based on the above evaluation index model, this embodiment provides an evaluation system for cloud control basic platform regulation and control instructions, such as Figure 1b As shown, the evaluation system includes: a vehicle subsystem 110, a road subsystem 120, a cloud subsystem 130 and a command evaluation subsystem 140, wherein:

[0106] The roadside subsystem 120 includes a roadside time information collection module 121, which is used to collect roadside time information and transmit the roadside time information to the instruction evaluation subsystem 140, wherein the roadside time information includes: the exposure time point (t r-exp ), the time point when the road end outputs the perception result (t r-out );

[0107] The cloud subsystem 130 includes a data collection module 131, which is used to collect cloud time information and external data sent by the vehicle-side subsystem 110 and the road-side subsystem 120, and transmit the cloud time information and external data to the instruction evaluation subsystem 140, wherein the cloud time information includes: the time point (t c-in ), the time when the cloud sends the command (t c-out );

[0108] The vehicle-side subsystem 110 includes a vehicle-side time information collection module 111, which is used to collect vehicle-side time information and transmit the vehicle-side time information to the instruction evaluation subsystem 140, wherein the vehicle-side time information includes: the time point (t v-in ), the time when the vehicle completes the execution of the cloud command (t v-finish );

[0109] The instruction evaluation subsystem 140 is used to obtain road-side time information, cloud-side time information and vehicle-side time information, and align the road-side time information, cloud-side time information and vehicle-side time information with the time information of the instruction evaluation subsystem 140, and calculate the value of the regulatory control instruction evaluation index based on the aligned time information of each subsystem.

[0110] In this embodiment, the evaluation system of the cloud control basic platform regulation and control instructions is a distributed architecture, including a server and a client.

[0111] Among them, the server side is the instruction evaluation subsystem, which includes five modules: data access, data preprocessing, data storage, time alignment and indicator calculation. The data access module is responsible for connecting and communicating with the client. The data preprocessing is responsible for cleaning and labeling the collected raw data. The data storage module is responsible for storing data in each link and supporting various calculations. The time difference calculation uses the machine clock of the test server (the test server sets a unique clock synchronization source) as the benchmark to calculate the time difference between the vehicle, road and cloud. The indicator calculation module evaluates various indicators related to the instruction based on data storage and time difference calculation.

[0112] There are two types of clients: data collection client, which is the data collection module of the cloud subsystem, and time collection client, which is the time information collection module. The data collection client implements three functions: time collection, data collection and transmission, and is installed on the cloud. The time collection client implements two functions: time collection and transmission, and is installed on the vehicle side and the road side. The vehicle side time information collection module is installed on the vehicle side, and the road section time information collection module is installed on the road section. The time collection function implemented on the client side supports the server side time difference calculation, but does not affect the machine clock of the host (vehicle, road and cloud).

[0113] In this embodiment, the control instruction evaluation index includes an instruction delay index, which is used to indicate the rationality of the instruction delay. The instruction delay index includes: an instruction generation delay index (TD cmd-g ), instruction transmission delay index (TD cmd-t ) and instruction execution delay indicator (TD cmd-e ),in,

[0114] The command generation delay indicator represents the time interval between the time when the cloud sends the command and the exposure time of the road-side sensor, that is: TD cmd-g =t c-out -t r-exp ;

[0115] The command transmission delay index represents the time interval between the time when the vehicle receives the command and the time when the cloud sends the command, that is: TD cmd-t =t v-in -t c-out ;

[0116] The instruction execution delay index represents the time interval between the time when the vehicle end executes the instruction and the time when the vehicle end receives the instruction, that is: TD cmd-e =t v-finish -t v-in .

[0117] Among them, the various time information used in the calculation process of the above instruction delay index (i.e., the various parameters on the right side of the equal sign in the formula) is the time information after the instruction evaluation subsystem aligns the section time information of the section subsystem, the cloud time information of the cloud subsystem, and the vehicle-side time information of the vehicle-side subsystem. In this embodiment, when the instruction evaluation subsystem aligns the time information of each subsystem, it can be specifically implemented through the following steps:

[0118] 1. Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem.

[0119] Specifically, for any target subsystem, the network transmission delay information between the target subsystem and the instruction evaluation subsystem can be realized through the following process:

[0120] The time of the current instruction evaluation subsystem can be used as a benchmark. The current instruction evaluation subsystem sends a heartbeat packet to the time information collection module of the road subsystem, the cloud data collection module of the cloud subsystem, and the vehicle-side time information collection module of the vehicle-side subsystem at set intervals, and records the time of sending the heartbeat packet;

[0121] The current instruction evaluation subsystem receives the heartbeat data returned by the time information collection module, the cloud data collection module and the vehicle-side time information collection module respectively, and records the time of receiving each heartbeat data. In this way, the time information collection module, the cloud data collection module and the vehicle-side time information collection module will maintain an increasing transmission delay sequence in the current instruction evaluation subsystem. A delay value in the sequence represents the one-way network delay between the module and the current instruction evaluation subsystem at a certain minute, that is: for each subsystem except the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem can be calculated based on the time difference between the time when the current instruction evaluation subsystem sends a heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem, which can be specifically expressed by the following formula:

[0122]

[0123] Among them, TD c2ts represents the network transmission delay between a target subsystem and the current instruction evaluation subsystem, t ts2c-h Indicates the time when the current instruction evaluation subsystem sends a heartbeat packet to the target subsystem, t c2ts-h Indicates the time when the current instruction evaluation subsystem receives the heartbeat data returned by the target subsystem, where the target subsystem can be any one of the cloud subsystem, road segment subsystem, and vehicle subsystem.

[0124] 2. Determine the time difference information between each subsystem and the current instruction evaluation subsystem based on the network delay information.

[0125] For a certain target subsystem, the specific calculation method of the time difference information between the target subsystem and the current instruction evaluation subsystem can be:

[0126] The client collects the machine time point of the host (vehicle side, road side or cloud side) and sends it to the instruction evaluation subsystem. After receiving the time, the instruction evaluation subsystem collects the local time point. The time difference (which may be positive or negative) is calculated as follows:

[0127] △t c-ts =t ts -t client -TD c2ts

[0128] Among them, t client Indicates the current time information collected by the subsystem, t ts Indicates the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts Indicates the network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0129] In this embodiment, the time difference information can be used for the subsequent alignment process of different subsystem times. By using a high-density time error sequence to align time, the adaptability and accuracy of the algorithm are greatly improved.

[0130] 3. Align the road-side time information, cloud-side time information, and vehicle-side time information with the time information of the instruction evaluation subsystem. Specifically, the aligned time information can be calculated using the following formula:

[0131] t r-exp =t r-exp +△t r-c-ts

[0132] t r-out =t r-out +△tr-c-ts

[0133] t c-in =t c-in +△t c-c-ts

[0134] t c-out =t c-out +△t c-c-ts

[0135] t v-in =t v-in +△t v-c-ts

[0136] t v-finish =t v-finish +△t v-c-ts

[0137] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out Indicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the cloud sends the command; t v-in Indicates the time when the vehicle receives the command; t v-finish Indicates the time point when the vehicle completes the execution of the command; Δt r-c-ts Indicates the time difference between the subsystem of the route and the current instruction evaluation subsystem; Δt v-c-ts Indicates the time difference between the vehicle subsystem and the current instruction evaluation subsystem; Δt c-c-ts Represents the time difference between the cloud subsystem and the current instruction evaluation subsystem. By performing time alignment according to the above calculation formula, the various time information on the left side of the equal sign are all on the same time axis as the time information of the instruction evaluation subsystem.

[0138] In the related art, the instruction evaluation subsystem needs to embeddably modify the clock information of other subsystems to align the time of other subsystems (sub-vehicle subsystem, road section subsystem and cloud subsystem). In this embodiment, by using the time difference between each subsystem and the instruction evaluation subsystem for time alignment, the problem of needing to mechanically modify the system clock is successfully avoided, thereby avoiding the problem of inconsistency between the system time and its actual time caused by mechanically modifying the system clock. In addition, in the process of time alignment, by using a high-density time difference sequence to align time, the adaptability and accuracy of the algorithm are greatly improved, thereby effectively improving the accuracy of subsequent instruction delay index value results.

[0139] In this embodiment, after the aligned time information is calculated, the obtained time information may be brought into the calculation formula of each instruction latency index in the above text, so as to obtain each instruction latency index value.

[0140] Furthermore, in order to more accurately and objectively evaluate the cloud control basic platform regulation and control instructions, the aggregation value of the instruction delay index value can be calculated based on the instruction delay index value calculated at a certain moment, and the cloud control basic platform regulation and control instructions can be evaluated by the aggregation value. Among them, the aggregation value can be calculated by counting the instruction delay index value over a period of time. For example, the following quantile values ​​can be calculated according to time periods such as minutes, hours, and days: the maximum value (max), minimum value (min), average value (mean), median value (P50), and P99 (99% of the observed values ​​are lower than a certain instruction delay index value, and only 1% of the observed values ​​are higher than the instruction delay index value) of the instruction delay index value in a certain period of time.

[0141] Furthermore, after obtaining the aggregate value of the instruction delay index value, it is possible to evaluate whether there are problems in each stage of the instruction based on the size of the aggregate value, thereby assisting the staff in troubleshooting related problems. For example, if the instruction generation delay index value is too large, for example, when it is greater than a first threshold, it means that the instruction generation delay is too long and there are unreasonable problems. At this point, it can be reversely deduced that there is a delay in the road section perception process or the road-cloud transmission process or the vehicle-side perception process or the vehicle-cloud transmission process or the cloud-end processing process related to the instruction generation delay index. The staff can troubleshoot the above processes based on the instruction generation delay index value. For another example, if the instruction transmission delay value is too large, for example, greater than a second threshold, it means that the instruction transmission delay is too long and there are unreasonable problems in the instruction transmission process. At this point, it can be reversely deduced that there is a delay in the cloud-to-vehicle transmission process related to the instruction transmission delay index, and the staff can troubleshoot the cloud-to-vehicle transmission process based on the instruction transmission delay index value. For another example, if the instruction execution delay is too large, for example, greater than a third threshold, it indicates a delay in the instruction execution process and there are unreasonable problems in the vehicle-side instruction execution process. At this time, it can be reversely deduced that there is a delay in the vehicle-side processing process. The staff can check each link of the vehicle-side processing process based on the instruction execution delay index value.

[0142] In addition, in this embodiment, the regulatory control instruction evaluation index also includes an instruction execution index, which is used to evaluate the reliability of the execution of the regulatory control instruction, wherein the instruction execution index includes an instruction feedback rate index, an instruction execution rate index, an instruction execution success rate index, and an instruction execution efficiency index. As shown in Table 1 above, the instruction feedback rate index represents the ratio of the first value of the instruction with feedback information from the vehicle-to-subsystem received by the cloud subsystem to the second value of the instruction issued by the cloud subsystem; the instruction execution rate index represents the ratio of the third value of the instruction with feedback information from the vehicle-to-subsystem and adopted and executed by the vehicle-to-subsystem to the second value of the instruction issued by the cloud subsystem; the instruction execution success rate index represents the ratio of the fourth value of the instruction with feedback information from the vehicle-to-subsystem and successfully executed by the vehicle-to-subsystem to the second value of the instruction issued by the cloud subsystem; the instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem when the motion state information after the vehicle successfully executes the instruction meets the expected motion state.

[0143] In this embodiment, the instruction evaluation subsystem is used to calculate the value of the instruction execution index in addition to calculating the delay index of each instruction. Specifically, the instruction evaluation subsystem is specifically configured to: within a set time period, count the first value of the instructions with feedback information from the vehicle-side subsystem received by the cloud subsystem, the second value of the instructions issued by the cloud subsystem, the third value of the instructions with feedback information from the vehicle-side subsystem and adopted and executed by the vehicle-side subsystem, and the fourth value of the instructions with feedback information from the vehicle-side subsystem and successfully executed by the vehicle-side subsystem, and calculate the ratio of the first value to the second value to obtain the instruction feedback rate index value; calculate the ratio of the third value to the second value to obtain the instruction execution rate index value; calculate the ratio of the fourth value to the second value to obtain the instruction execution success rate index.

[0144] In addition, when calculating the instruction execution efficiency index value, for instructions successfully executed on the vehicle side, the vehicle status and performance information (such as vehicle speed, trajectory, etc.) within the most recent set time period (for example, 20 seconds) after the instruction is successful can be collected from the vehicle side and the roadside, and then machine learning or manual labeling, or a combination of machine learning and manual labeling can be used to determine whether the vehicle's movement state meets the instruction expectations, and the fifth value of the instructions that meet the expected performance can be counted, and then the ratio of the fifth value to the second value can be calculated to obtain the instruction execution efficiency index value.

[0145] In this embodiment, in order to more accurately and objectively evaluate the cloud control basic platform regulation and control instructions, the aggregate value of the instruction execution index value can be calculated, and the reliability of the cloud control basic platform regulation and control instructions can be evaluated by the aggregate value, thereby reflecting the reliability of the cloud control basic platform model. For instruction execution index values ​​that do not meet the specifications, the staff can be assisted in investigating the reasons, and the corresponding algorithms or hardware equipment on the cloud, vehicle or road section can be adjusted to make the instruction execution index values ​​meet the specifications, thereby improving the reliability of the cloud control basic platform.

[0146] In this embodiment, at the level of designing evaluation indicators for regulatory control instructions, two types of evaluation indicators are proposed based on the entire life cycle of instructions, including instruction latency indicators and instruction execution indicators, and their definitions and calculation methods are clarified, filling the gap that there is currently no clear test and evaluation model for cloud-based instructions. In terms of evaluation technology, this embodiment proposes a distributed evaluation system architecture, which comprehensively realizes the statistical calculation of the evaluation indicator values ​​of each instruction through the cooperation of the server and each client. In the calculation process, the delay that may exist in the generation process, transmission process, and execution process of the instruction as well as the execution of the instruction are taken into account, effectively improving the feasibility and reliability of the evaluation results of regulatory control instructions.

[0147] Embodiment 2

[0148] Figure 2 This is a flow chart of a method for evaluating control instructions of a cloud control basic platform provided in the second embodiment of the present invention. This method can be executed by the instruction evaluation subsystem provided in the above embodiment, that is, the method provided in this embodiment introduces in detail the calculation process of the control instruction evaluation index from the perspective of the instruction evaluation subsystem. Figure 2 As shown, the method provided in this embodiment specifically includes:

[0149] S210, obtaining road-side time information, cloud-side time information, and vehicle-side time information.

[0150] Among them, the road-side time information is collected by the road-side time information collection module in the road-side subsystem, the cloud-side time information is collected by the data collection module in the cloud-side subsystem, and the vehicle-side time information is collected by the vehicle-side time information collection module in the vehicle-side subsystem, wherein the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the vehicle-side time information includes: the time point when the vehicle receives the cloud-side instruction and the time point when the vehicle executes the cloud-side instruction. Among them, the introduction of each instruction time point can refer to the relevant content provided in the above embodiment, which will not be repeated here.

[0151] S220, aligning the road-side time information, the cloud-side time information and the vehicle-side time information with the time information of the current instruction evaluation subsystem.

[0152] In this embodiment, step S220 can be implemented by the following steps S221 to S223:

[0153] S221. Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem.

[0154] Wherein, step S221 can be specifically implemented through the following steps A to C:

[0155] A. Based on the time of the current instruction evaluation subsystem, a heartbeat packet is sent to the time information collection module of the roadside subsystem, the cloud data collection module of the cloud subsystem, and the vehicle-side time information collection module of the vehicle-side subsystem at set intervals, and the time of sending the heartbeat packet is recorded;

[0156] B. Receive the heartbeat data returned by the time information collection module, the cloud data collection module and the vehicle-side time information collection module respectively, and record the time when each heartbeat data is received;

[0157] C. For each subsystem other than the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends a heartbeat packet to the subsystem and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem. The specific information can be expressed by the following formula:

[0158]

[0159] Among them, TD c2ts represents the network transmission delay between a target subsystem and the current instruction evaluation subsystem, t ts2c-h Indicates the time when the current instruction evaluation subsystem sends a heartbeat packet to the target subsystem, t c2ts-h Indicates the time when the current instruction evaluation subsystem receives the heartbeat data returned by the target subsystem, where the target subsystem can be any of the cloud subsystem, road section subsystem, and vehicle subsystem.

[0160] S222. Determine the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information.

[0161] Specifically, for each subsystem in the vehicle-side subsystem, the road-side subsystem, and the cloud-side subsystem, the time difference information between the subsystem and the current instruction evaluation subsystem can be determined according to the following formula:

[0162] △t c-ts =tts -t client -TD c2ts

[0163] Among them, t client Indicates the current time information collected by the subsystem, t ts Indicates the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts Indicates the network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0164] S223. Based on the road-side time information, cloud-side time information, vehicle-side time information and time difference information, align the road-side time information, cloud-side time information and vehicle-side time information with the time information of the instruction evaluation subsystem.

[0165] Specifically, the road-side time information, cloud-side time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem according to the following formula:

[0166] t r-exp =t r-exp +△t r-c-ts

[0167] t r-out =t r-out +△t r-c-ts

[0168] t c-in =t c-in +△t c-c-ts

[0169] t c-out =t c-out +△t c-c-ts

[0170] t v-in =t v-in +△t v-c-ts

[0171] t v-finish =t v-finish +△t v-c-ts

[0172] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out Indicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the cloud sends the command; t v-in Indicates the time when the vehicle receives the command; tv-finish Indicates the time point when the vehicle completes the execution of the command; Δt r-c-ts Indicates the time difference between the subsystem of the route and the current instruction evaluation subsystem; Δt v-c-ts Indicates the time difference between the vehicle subsystem and the current instruction evaluation subsystem; Δt c-c-ts Represents the time difference between the cloud subsystem and the current instruction evaluation subsystem. By performing time alignment according to the above calculation formula, the various time information on the left side of the equal sign are all on the same time axis as the time information of the instruction evaluation subsystem.

[0173] In this embodiment, the network transmission delay information between each subsystem and the current instruction evaluation subsystem, the time difference information between each subsystem and the current instruction evaluation subsystem, and the calculation principle and effect of time alignment can be referred to the description of the above embodiment and will not be repeated here.

[0174] S230: Calculate the value of the regulatory control instruction evaluation index according to the aligned time information of each subsystem.

[0175] Among them, the design of the regulatory control instruction evaluation index can be specifically referred to the description of the above embodiment, and will not be repeated here. In this embodiment, the regulatory control instruction evaluation index includes an instruction delay index, which is used to evaluate the rationality of the instruction delay. Among them, the instruction delay index includes: an instruction generation delay index, an instruction transmission delay index and an instruction execution delay index, wherein the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle side receives the instruction and the time point when the cloud side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle side executes the instruction and the time point when the vehicle side receives the instruction. Specifically, the value of each instruction delay index can be calculated according to the time information of each subsystem after alignment according to the following formula:

[0176] The instruction generation delay index value can be calculated according to the following formula:

[0177] TD cmd-g =t c-out -t r-exp

[0178] The instruction transmission delay index value can be calculated according to the following formula:

[0179] TD cmd-t =t v-in -t c-out

[0180] The instruction execution delay index value can be calculated according to the following formula:

[0181] TDcmd-e =t v-finish -t v-in

[0182] In this embodiment, by calculating the instruction latency index value, the obtained instruction latency index value can be used to effectively evaluate the latency of the cloud control basic platform's regulatory control instructions, thereby improving the accuracy and reliability of the regulatory control instruction evaluation results. Furthermore, in order to accurately and objectively evaluate the cloud control basic platform's regulatory control instructions, the aggregate value of the instruction latency index value can be calculated, and the cloud control basic platform's regulatory control instructions can be evaluated by the aggregate value. The calculation of the aggregate value can refer to the contents of the above embodiment, and will not be repeated here.

[0183] In this embodiment, the regulatory control instruction evaluation index also includes an instruction execution index, which is used to evaluate the reliability of the execution of the regulatory control instruction, wherein the instruction execution index includes an instruction feedback rate index, an instruction execution rate index, an instruction execution success rate index, and an instruction execution efficiency index;

[0184] The command feedback rate indicator represents the ratio of a first value of the command with feedback information from the vehicle-side subsystem received by the cloud subsystem to a second value of the command issued by the cloud subsystem;

[0185] The command execution rate indicator represents the ratio of the third value of the command with feedback information from the vehicle-side subsystem and adopted and executed by the vehicle-side subsystem to the second value of the command issued by the cloud subsystem;

[0186] The command execution success rate indicator represents the ratio of the fourth value of the command with feedback information from the vehicle-side subsystem and successfully executed by the vehicle-side subsystem to the second value of the command issued by the cloud subsystem;

[0187] The command execution efficiency index represents the ratio of the fifth value of the command successfully executed by the vehicle to the second value of the command issued by the cloud subsystem when the motion state information after the vehicle successfully executes the command meets the expected motion state;

[0188] Accordingly, the evaluation method of the cloud control basic platform regulation and control instructions provided in this embodiment also includes:

[0189] Within a set time period, the first value, the second value, the third value and the fourth value are counted, and the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value are calculated.

[0190] In this embodiment, by calculating the instruction execution index value, the obtained instruction execution index value can be used to effectively evaluate the execution of the instruction by the vehicle side in the cloud control basic platform, further improving the accuracy and reliability of the evaluation results of the control instruction. Furthermore, in order to accurately and objectively evaluate the control instructions of the cloud control basic platform, the aggregate value of each instruction execution index value can be calculated, and the reliability of the control instructions of the cloud control basic platform can be evaluated by the aggregate value. Among them, the calculation of the aggregate value can refer to the content of the above embodiment, which will not be repeated here.

[0191] The technical solution provided in this embodiment proposes two types of evaluation indicators, including instruction delay indicators and instruction execution indicators, based on the entire life cycle of the instruction, at the level of regulatory control instruction evaluation indicator design, and clarifies their definitions and calculation methods, filling the gap that there is currently no clear test and evaluation model for cloud instructions. In addition, in the calculation process of regulatory control instruction evaluation indicators, the delay that may exist in the generation process, transmission process, and execution process of the instruction, as well as the execution of the instruction by the vehicle side, are taken into account, effectively improving the feasibility and reliability of the evaluation results of regulatory control instructions.

[0192] Embodiment 3

[0193] Figure 3 A structural block diagram of a cloud control basic platform regulation and control instruction evaluation device provided in Embodiment 3 of the present invention, such as Figure 3 As shown, the device includes: a data access module 310, a time alignment module 320 and an indicator calculation module 330, wherein:

[0194] The data access module 310 is configured to obtain road-side time information, cloud-side time information and vehicle-side time information, wherein the road-side time information is collected by the road-side time information collection module in the road-side subsystem, the cloud-side time information is collected by the data collection module in the cloud-side subsystem, and the vehicle-side time information is collected by the vehicle-side time information collection module in the vehicle-side subsystem, wherein the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the vehicle-side time information includes: the time point when the vehicle receives the cloud-side instruction and the time point when the vehicle executes the cloud-side instruction;

[0195] The time alignment module 320 is configured to align the road-side time information, the cloud-side time information and the vehicle-side time information with the time information of the current instruction evaluation subsystem;

[0196] The indicator calculation module 330 is configured to calculate the value of the regulatory control instruction evaluation indicator according to the aligned time information of each subsystem, wherein the regulatory control instruction evaluation indicator includes an instruction delay indicator, which is used to evaluate the rationality of the instruction delay;

[0197] Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index. Among them, the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side completes the execution of the instruction and the time point when the vehicle-side receives the instruction.

[0198] Optional, time alignment module, including:

[0199] A network transmission delay calculation unit is configured to respectively calculate network transmission delay information between each subsystem and the current instruction evaluation subsystem;

[0200] A time difference determination unit, configured to determine the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information;

[0201] The time alignment unit is configured to align the road-side time information, the cloud-side time information, the vehicle-side time information and the time difference information with the time information of the instruction evaluation subsystem.

[0202] Optionally, the network transmission delay calculation unit is specifically configured as follows:

[0203] Based on the time of the current instruction evaluation subsystem, a heartbeat packet is sent to the time information collection module of the roadside subsystem, the cloud data collection module of the cloud subsystem, and the vehicle-side time information collection module of the vehicle-side subsystem at set intervals, and the time of sending the heartbeat packet is recorded;

[0204] Receive the heartbeat data returned by the time information collection module, the cloud data collection module and the vehicle-side time information collection module respectively, and record the time when each heartbeat data is received;

[0205] For each subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends a heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

[0206] Optionally, the time difference determination unit is specifically configured to:

[0207] According to the following formula, determine the time difference information between each subsystem and the current instruction evaluation subsystem:

[0208] △t c-ts =t ts -tclient -TD c2ts

[0209] Among them, t client Indicates the current time information collected by the subsystem, t ts Indicates the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts Indicates the time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts Indicates the network transmission delay between the subsystem and the current instruction evaluation subsystem.

[0210] Optionally, the time alignment unit is specifically configured as follows:

[0211] According to the following formula, the road-side time information, cloud-side time information, and vehicle-side time information are aligned with the time information of the instruction evaluation subsystem to obtain the aligned time information:

[0212] t r-exp =t r-exp +△t r-c-ts

[0213] t r-out =t r-out +△t r-c-ts

[0214] t c-in =t c-in +△t c-c-ts

[0215] t c-out =t c-out +△t c-c-ts

[0216] t v-in =t v-in +△t v-c-ts

[0217] t v-finish =t v-finish +△t v-c-ts

[0218] Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out Indicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the cloud sends the command; t v-in Indicates the time when the vehicle receives the command; t v-finish Indicates the time point when the vehicle completes the execution of the command; Δt r-c-tsIndicates the time difference between the subsystem of the route and the current instruction evaluation subsystem; Δt v-c-ts Indicates the time difference between the vehicle subsystem and the current instruction evaluation subsystem; Δt c-c-ts Indicates the time difference between the cloud subsystem and the current instruction evaluation subsystem.

[0219] Optionally, the regulatory control instruction evaluation index further includes an instruction execution index, which is used to evaluate the reliability of the execution of the regulatory control instruction, wherein the instruction execution index includes an instruction feedback rate index, an instruction execution rate index, an instruction execution success rate index, and an instruction execution efficiency index;

[0220] The command feedback rate indicator represents the ratio of a first value of the command with feedback information from the vehicle-side subsystem received by the cloud subsystem to a second value of the command issued by the cloud subsystem;

[0221] The command execution rate indicator represents the ratio of the third value of the command with feedback information from the vehicle-side subsystem and adopted and executed by the vehicle-side subsystem to the second value of the command issued by the cloud subsystem;

[0222] The command execution success rate indicator represents the ratio of the fourth value of the command with feedback information from the vehicle-side subsystem and successfully executed by the vehicle-side subsystem to the second value of the command issued by the cloud subsystem;

[0223] The command execution efficiency index represents the ratio of the fifth value of the command successfully executed by the vehicle to the second value of the command issued by the cloud subsystem when the motion state information after the vehicle successfully executes the command meets the expected motion state;

[0224] Accordingly, the device provided by the embodiment of the present invention further includes:

[0225] The instruction execution index value calculation module is configured to count the first value, the second value, the third value and the fourth value within a set time period, and calculate the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value.

[0226] The evaluation device for cloud control basic platform regulation and control instructions provided in the embodiment of the present invention can execute the evaluation method for cloud control basic platform regulation and control instructions provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in the above embodiments, please refer to the evaluation method for cloud control basic platform regulation and control instructions provided in any embodiment of the present invention.

[0227] Embodiment 4

[0228] Figure 4 A structural block diagram of a computing device provided in Embodiment 4 of the present invention, such as Figure 4 As shown, the computing device includes:

[0229] At least one processor ( Figure 4 A processor 520 is shown,

[0230] The processor 520 is coupled to the memory 510, and the memory 510 stores a program or instruction running on the processor 520. When the program or instruction is executed by the processor 520, the evaluation method of the cloud control basic platform regulation and control instructions provided by any embodiment of the present invention is implemented.

[0231] Based on the above method embodiments, another embodiment of the present invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the processor implements the evaluation method of the cloud control basic platform regulation and control instructions as described in any of the above embodiments.

[0232] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0233] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0234] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cloud control basic platform regulatory control instruction evaluation system, characterized in that: include: Vehicle subsystem, road subsystem, cloud subsystem and command evaluation subsystem, among which: The roadside subsystem includes a roadside time information collection module, which is used to collect roadside time information and transmit the roadside time information to the instruction evaluation subsystem, wherein the roadside time information includes: the exposure time point of the roadside sensor and the time point when the roadside outputs the perception result; The cloud subsystem includes a data collection module, which is used to collect cloud time information and external data sent by the vehicle-side subsystem and the road-side subsystem, and transmit the cloud time information and the external data to the instruction evaluation subsystem, wherein the cloud time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; The vehicle-side subsystem includes a vehicle-side time information collection module, which is used to collect vehicle-side time information and transmit the vehicle-side time information to the instruction evaluation subsystem, wherein the vehicle-side time information includes: the time point when the vehicle-side receives the cloud instruction and the time point when the vehicle-side completes executing the cloud instruction; The instruction evaluation subsystem is used to obtain the road-side time information, the cloud-side time information and the vehicle-side time information, and align the road-side time information, the cloud-side time information and the vehicle-side time information with the time information of the instruction evaluation subsystem, and calculate the value of the regulatory control instruction evaluation index according to the aligned time information of each subsystem, wherein the regulatory control instruction evaluation index includes an instruction delay index, which is used to evaluate the rationality of the regulatory control instruction delay; Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index, wherein the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side executes the instruction and the time point when the vehicle-side receives the instruction.

2. The system according to claim 1, characterized in that The regulatory control instruction evaluation index also includes an instruction execution index, which is used to evaluate the reliability of the execution of the regulatory control instruction, wherein the instruction execution index includes an instruction feedback rate index, an instruction execution rate index, an instruction execution success rate index, and an instruction execution efficiency index; The command feedback rate indicator represents the ratio of a first value of the command with feedback information from the vehicle-side subsystem received by the cloud subsystem to a second value of the command issued by the cloud subsystem; The instruction execution rate indicator represents the ratio of a third value of instructions with feedback information from the vehicle-side subsystem and adopted and executed by the vehicle-side subsystem to a second value of instructions issued by the cloud subsystem; The instruction execution success rate indicator represents the ratio of a fourth value of instructions having feedback information from the vehicle-side subsystem and successfully executed by the vehicle-side subsystem to a second value of instructions issued by the cloud subsystem; The instruction execution efficiency index represents the ratio of the fifth value of the instruction successfully executed by the vehicle to the second value of the instruction issued by the cloud subsystem when the motion state information after the vehicle successfully executes the instruction meets the expected motion state; Accordingly, the execution evaluation subsystem is further configured as follows: Within a set time period, the first value, the second value, the third value and the fourth value are counted, and the ratio of the first value to the second value, the ratio of the third value to the second value, the ratio of the fourth value to the second value, and the ratio of the fifth value to the second value are calculated.

3. A method for evaluating control instructions of a cloud control basic platform, applied to an instruction evaluation subsystem, characterized in that: The method comprises: Obtain road-side time information, cloud-side time information and vehicle-side time information, wherein the road-side time information is collected by the road-side time information collection module in the road-side subsystem, the cloud-side time information is collected by the data collection module in the cloud-side subsystem, and the vehicle-side time information is collected by the vehicle-side time information collection module in the vehicle-side subsystem, wherein the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the vehicle-side time information includes: the time point when the vehicle receives the cloud-side instruction and the time point when the vehicle completes the cloud-side instruction; Aligning the road-side time information, the cloud-side time information, and the vehicle-side time information with the time information of the current instruction evaluation subsystem; Calculating the value of the regulatory control instruction evaluation index according to the aligned time information of each subsystem, wherein the regulatory control instruction evaluation index includes an instruction delay index, which is used to evaluate the rationality of the regulatory control instruction delay; Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index, wherein the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side executes the instruction and the time point when the vehicle-side receives the instruction.

4. The method according to claim 3, characterized in that The aligning the road-side time information, the cloud-side time information, the vehicle-side time information, and the time information of the instruction evaluation subsystem includes: Calculate the network transmission delay information between each subsystem and the current instruction evaluation subsystem respectively; Determine the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information; Based on the road-side time information, the cloud-side time information, the vehicle-side time information and the time difference information, the road-side time information, the cloud-side time information and the vehicle-side time information are aligned with the time information of the instruction evaluation subsystem.

5. The method according to claim 4, characterized in that The respectively calculating network transmission delay information between each subsystem and the current instruction evaluation subsystem includes: Taking the time of the current instruction evaluation subsystem as a reference, heartbeat packets are sent to the time information collection module of the road subsystem, the cloud data collection module of the cloud subsystem, and the vehicle-side time information collection module of the vehicle-side subsystem at set intervals, and the time of sending the heartbeat packets is recorded; Receive the heartbeat data returned by the time information acquisition module, the cloud data acquisition module and the vehicle-side time information acquisition module respectively, and record the time when each heartbeat data is received; For each subsystem except the current instruction evaluation subsystem, the network transmission delay information between the subsystem and the current instruction evaluation subsystem is calculated based on the time difference between the time when the current instruction evaluation subsystem sends a heartbeat packet to it and the time when the current instruction evaluation subsystem receives the heartbeat data returned by the subsystem.

6. The method according to claim 4, characterized in that The determining of the time difference information between each subsystem and the current instruction evaluation subsystem according to the network delay information includes: The time difference information between each subsystem and the current instruction evaluation subsystem is determined according to the following formula: △t c-ts =t ts -t client -TD c2ts Among them, t client Indicates the current time information collected by the subsystem, t ts Indicates the local time when the current instruction evaluation subsystem receives the time information sent by the subsystem; Δt c-ts The time difference between the subsystem and the current instruction evaluation subsystem; TD c2ts The network transmission delay between the subsystem and the current instruction evaluation subsystem.

7. The method according to claim 4, characterized in that The step of aligning the road-side time information, the cloud-side time information, the vehicle-side time information and the time difference information with the time information of the instruction evaluation subsystem includes: According to the following formula, the road-side time information, the cloud-side time information, the vehicle-side time information and the time information of the instruction evaluation subsystem are aligned to obtain the aligned time information: t r-exp =t r-exp +△t r-c-ts t r-out =t r-out +△t r-c-ts t c-in =t c-in +△t c-c-ts t c-out =t c-out +△t c-c-ts t v-in =t v-in +△t v-c-ts t v-finish =t v-finish +△t v-c-ts Among them, t r-exp Indicates the exposure time point of the roadside sensor; t r-out Indicates the time point when the road end outputs the perception result; t c-in Indicates the time point when the cloud receives the road-side perception result; t c-out Indicates the time when the cloud sends the command; t v-in Indicates the time when the vehicle receives the command; t v-finish Indicates the time point when the vehicle completes the execution of the command; Δt r-c-ts Indicates the time difference between the subsystem of the route and the current instruction evaluation subsystem; Δt v-c-ts Indicates the time difference between the vehicle subsystem and the current instruction evaluation subsystem; Δt c-c-ts Indicates the time difference between the cloud subsystem and the current instruction evaluation subsystem.

8. A cloud control basic platform regulatory control instruction evaluation device, characterized in that: include: The data access module is configured to obtain road-side time information, cloud-side time information and vehicle-side time information, wherein the road-side time information is collected by the road-side time information collection module in the road-side subsystem, the cloud-side time information is collected by the data collection module in the cloud-side subsystem, and the vehicle-side time information is collected by the vehicle-side time information collection module in the vehicle-side subsystem, wherein the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the cloud-side time information includes: the time point when the cloud receives the road-side perception result and the time point when the cloud issues the instruction; the vehicle-side time information includes: the time point when the vehicle receives the cloud-side instruction and the time point when the vehicle executes the cloud-side instruction; A time alignment module, configured to align the road-side time information, the cloud-side time information and the vehicle-side time information with the time information of the current instruction evaluation subsystem; An indicator calculation module is configured to calculate a value of a regulatory control instruction evaluation indicator according to the aligned time information of each subsystem, wherein the regulatory control instruction evaluation indicator includes an instruction delay indicator for evaluating the rationality of the regulatory control instruction delay; Among them, the instruction delay index includes: instruction generation delay index, instruction transmission delay index and instruction execution delay index, wherein the instruction generation delay index represents the time interval between the time point when the cloud sends the instruction and the exposure time point of the road-side sensor, the instruction transmission delay index represents the time interval between the time point when the vehicle-side receives the instruction and the time point when the cloud-side sends the instruction, and the instruction execution delay index represents the time interval between the time point when the vehicle-side executes the instruction and the time point when the vehicle-side receives the instruction.

9. A computing device, characterized in that It includes at least one processor, which is coupled to a memory, and the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, it implements the steps of the evaluation method of the cloud control basic platform regulation and control instructions as described in any one of claims 3 to 7.

10. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by the processor, the steps of the method for evaluating the cloud control basic platform regulation and control instructions as described in any one of claims 3 to 7 are implemented.

Images