Evaluation method and device of intelligent system, vehicle, equipment and medium

Abstract

The invention relates to an evaluation method and device of an intelligent system, a vehicle, equipment and a medium, and the method comprises the steps: constructing an associated data structure which is used for representing an association relationship between an evaluation index of the intelligent system and a basic data field and an intermediate data field involved in index operation; constructing an index structure according to the associated data structure; the index structure is used for storing and searching data values corresponding to the basic data fields and the intermediate data fields; in the process of performing the evaluation index operation according to the evaluation algorithm, determining a required data field according to the associated data structure, and querying a data value corresponding to the required data field according to the index structure; wherein the data value of the intermediate data field obtained through the first operation is stored based on the index structure, and when the data value of the intermediate data field is not changed, the data value is searched and reused through the index structure in the subsequent use process, and repeated calculation is not carried out. And improvement of evaluation calculation efficiency is promoted.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicles and performance evaluation, and in particular to an evaluation method, device, vehicle, equipment and medium for an intelligent system. Background Art

[0002] In related technologies, various intelligent systems need to make reaction states, reaction actions or other feedback results in adaptation scenarios according to the actual environment. During production testing, use or operation and maintenance, the performance of intelligent systems is currently mainly evaluated through rehearsal or simulation in various environments.

[0003] Taking intelligent driving vehicles as an example, to ensure the reliability and safety of the autonomous driving system in various complex road conditions and unpredictable road environments, a lot of testing and evaluation is required. For example, when evaluating the autonomous driving decision-making performance of the test vehicle at complex intersections, it is necessary to collect the autonomous driving data of the test vehicle at various complex intersections, and analyze the autonomous driving decision-making performance of the test vehicle by calculating the evaluation indicators using these autonomous driving data. However, in the process of realizing the concept of the present disclosure, it is found that the current evaluation method has the following technical problems: when evaluating the intelligent system, it is impossible to intuitively present the complex correlation between the indicators and the basic data, and the evaluation calculation efficiency needs to be improved. Summary of the invention

[0004] To overcome the problems existing in the related art, the embodiments of the present disclosure provide an evaluation method, device, vehicle, equipment and medium for an intelligent system.

[0005] According to the first aspect of the embodiment of the present disclosure, a method for evaluating an intelligent system is provided. The method includes: constructing an associated data structure, the associated data structure is used to represent the association relationship between the evaluation index of the intelligent system and the basic data fields and intermediate data fields involved in the index calculation; according to the associated data structure, constructing an index structure about the basic data fields and the intermediate data fields; the index structure is used to store and search for data values ​​corresponding to the basic data fields and the intermediate data fields; in the process of performing the evaluation index calculation according to the evaluation algorithm, determining the required data fields according to the associated data structure, and querying the data values ​​corresponding to the required data fields according to the index structure; wherein the data values ​​of the intermediate data fields obtained by the first calculation are stored based on the index structure, and when the data values ​​of the intermediate data fields are unchanged, they are searched and reused through the index structure in subsequent use without repeated calculation.

[0006] In some embodiments, based on the above-mentioned associated data structure, constructing an index structure about the above-mentioned basic data field and the above-mentioned intermediate data field includes: constructing an index hierarchy of the index structure according to the computational association relationship between the nodes in the above-mentioned associated data structure; wherein, the higher the degree of association between a certain node and other nodes in the above-mentioned associated data structure, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster. Alternatively, in some embodiments, based on the above-mentioned associated data structure, constructing an index structure about the above-mentioned basic data field and the above-mentioned intermediate data field includes: constructing an index hierarchy of the index structure according to the computational association relationship between the nodes in the above-mentioned associated data structure and the execution logic of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation; wherein, the higher the degree of association between a certain node and other nodes in the above-mentioned associated data structure, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster; the higher the frequency of use of a certain node during the execution of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster.

[0007] In some embodiments, based on the above-mentioned associated data structure, an index structure about the above-mentioned basic data field and the above-mentioned intermediate data field is constructed, including: constructing an underlying single linked list in sequence according to the node numbers corresponding to the above-mentioned basic data field and the above-mentioned intermediate data field in the above-mentioned associated data structure; determining the number of times the nodes corresponding to the above-mentioned basic data field and the above-mentioned intermediate data field are referenced according to the operational association relationship between the nodes in the above-mentioned associated data structure; or, determining the number of times the nodes corresponding to the above-mentioned basic data field and the above-mentioned intermediate data field are referenced according to the operational association relationship between the nodes in the above-mentioned associated data structure and the execution logic of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation; the number of times the above-mentioned node is referenced indicates the number of times the current node participates in the result operation of the referenced node; determining the target data nodes at each index level according to the matching relationship between the above-mentioned node reference number and the preset grading interval; integrating the above-mentioned underlying single linked list and the target data nodes at each index level to obtain a multi-layer index structure.

[0008] In some embodiments, the preset grading interval includes a range of citation counts corresponding to each index level; the higher the index level, the larger the value of the corresponding range of citation counts. According to the matching relationship between the node citation counts and the preset grading intervals, the target data nodes at each index level are determined, including: for each basic data field or each intermediate data field corresponding to the current node, the node citation count of the current node is matched with the citation count range interval in the preset grading interval; in response to the node citation count of the current node matching the target citation count range interval, the target index level corresponding to the target citation count range interval is determined; and the current node is determined as the target data node at the target index level.

[0009] In some embodiments, the above-mentioned associated data structure is a graph structure, and the above-mentioned graph structure includes nodes and edges, and the above-mentioned nodes include: basic nodes corresponding to the above-mentioned basic data fields, intermediate nodes corresponding to the above-mentioned intermediate data fields, and evaluation indicator nodes corresponding to the above-mentioned evaluation indicators. The two ends of the above-mentioned edges are connected with associated nodes with an operation association relationship, and the above-mentioned operation association relationship refers to the corresponding relationship between input and output in each operation link of the above-mentioned evaluation indicator operation based on the above-mentioned evaluation algorithm.

[0010] In some embodiments, the graph structure is a directed topological graph, and the index level of the index structure is determined based on the node out-degree; or, the index level of the index structure is determined by the execution logic of the evaluation index calculation based on the node out-degree and the evaluation algorithm.

[0011] In some embodiments, the evaluation index calculation performed according to the evaluation algorithm corresponds to one of the following evaluation situations: the intelligent system performs a post-evaluation after the simulation is completed; the intelligent system performs a real-time evaluation during the simulation. Among them, for the post-evaluation, the basic data fields and intermediate data fields involved in the index calculation are determined based on all the operation links involved in the entire simulation process; for the real-time evaluation, the basic data fields and intermediate data fields involved in the index calculation are dynamically updated based on the existing operation links in the real-time simulation process, and the existing operation links are dynamically updated as the real-time simulation process progresses; the associated data structure and the index structure are dynamically updated as at least one of the basic data fields and the intermediate data fields is updated.

[0012] In some embodiments, the intelligent system includes one or a combination of the following entities: autonomous vehicles, drones, intelligent robots, and wearable devices. In the case where the intelligent system includes autonomous vehicles, the evaluation indicators include at least one type of indicators in the following evaluation scenarios: task execution status evaluation indicators corresponding to a task execution system consisting of one or more autonomous vehicles; or autonomous driving decision-making performance evaluation indicators of autonomous vehicles at complex intersections; or adaptability evaluation indicators of autonomous vehicles under different terrain and road conditions; or interactive performance evaluation of autonomous vehicles and other intelligent devices in driving status.

[0013] According to the second aspect of the embodiment of the present disclosure, an evaluation device for an intelligent system is provided. The device includes: an associated data structure construction module, an index structure construction module and an evaluation module. The associated data structure construction module is used to construct an associated data structure, and the associated data structure is used to represent the association relationship between the evaluation index of the intelligent system and the basic data field and the intermediate data field involved in the index operation. The index structure construction module is used to construct an index structure about the basic data field and the intermediate data field according to the associated data structure; the index structure is used to store and search for data values ​​corresponding to the basic data field and the intermediate data field. The evaluation module is used to determine the required data field according to the associated data structure in the process of performing the evaluation index operation according to the evaluation algorithm, and query the data value corresponding to the required data field according to the index structure; wherein the data value of the intermediate data field obtained by the first operation is stored based on the index structure, and when the data value of the intermediate data field remains unchanged, it is searched and reused through the index structure in subsequent use without repeated calculation.

[0014] According to a third aspect of an embodiment of the present disclosure, a vehicle is provided, storing a set of instruction sets, wherein the instruction sets are executed by the vehicle to implement the intelligent system evaluation method provided by the first aspect of the present disclosure.

[0015] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the evaluation method of the intelligent system provided in the first aspect of the present disclosure.

[0016] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the evaluation method of the intelligent system provided in the first aspect of the present disclosure are implemented.

[0017] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0018] By constructing an associated data structure between basic data fields, intermediate data fields and evaluation indicators, the association relationship between each evaluation indicator in the intelligent system and the basic data fields and intermediate data fields can be clearly and accurately presented; at the same time, by constructing an index structure for storing and searching data values ​​corresponding to basic data fields and intermediate data fields according to the above-mentioned associated data structure, the data value of the intermediate data field obtained by the first calculation is stored based on the above-mentioned index structure. In the process of calculating the evaluation indicator according to the evaluation algorithm, the required data field can be determined according to the above-mentioned associated data structure. For the data value corresponding to the intermediate data field that has been stored in the index structure, when the data value of the intermediate data field remains unchanged, it can be searched and reused through the above-mentioned index structure in subsequent use without repeated calculation, which saves the time cost and computing resources required for recalculating the data value corresponding to the intermediate data field and improves the evaluation calculation efficiency.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] Figure 1 The figure is a flow chart of an evaluation method of an intelligent system according to an exemplary embodiment.

[0022] Figure 2 It is a schematic diagram of basic data fields and intermediate data fields involved in indicator calculation obtained by analyzing different evaluation indicators according to an exemplary embodiment, wherein (a) is a schematic diagram of the process of analyzing the evaluation indicator of braking decision performance at an intersection, and (b) is a schematic diagram of the process of analyzing the evaluation indicator of safe starting acceleration performance at an intersection.

[0023] Figure 3 The present invention is a schematic diagram of using a graph structure to represent an associated data structure according to an exemplary embodiment, wherein (a) is a schematic diagram of the analysis results of basic data fields and intermediate data fields involved in evaluation indicators, and (b) is an associated data structure represented in the form of a graph structure based on the analysis results.

[0024] Figure 4 is a detailed implementation flowchart of step S120 according to an exemplary embodiment.

[0025] Figure 5 is a schematic diagram showing a multi-layer index structure according to an exemplary embodiment.

[0026] Figure 6 The figure is a block diagram of an evaluation device for an intelligent system according to an exemplary embodiment.

[0027] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment.

[0028] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail below with reference to the accompanying drawings.

[0030] It should be pointed out that the relevant embodiments and drawings are only for describing exemplary embodiments provided by the present disclosure, rather than all embodiments of the present disclosure, and it should not be understood that the present disclosure is limited to the relevant exemplary embodiments.

[0031] It should be noted that the terms "first", "second", etc. used in the present disclosure are only used to distinguish different steps, devices or modules, etc. The related terms neither represent any specific technical meanings nor indicate the order or interdependence between them.

[0032] It should be noted that the modifications of the terms "one", "multiple", and "at least one" used in the present disclosure are illustrative rather than restrictive. Unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] It should be noted that the term "and / or" used in this disclosure is used to describe the association relationship between associated objects, and generally indicates that there are at least three association relationships. For example, A and / or B can at least indicate the existence of three association relationships: A exists alone, A and B exist at the same time, and B exists alone.

[0034] It should be noted that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of the present disclosure is not limited by the order of description of the steps in the relevant embodiments.

[0035] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0036] During the research and development, it was found that when conducting intelligent system evaluation in related technologies, the complex correlation between indicators and basic data indicators cannot be intuitively presented; at the same time, more data needs to be called during the evaluation process and the same data may be called for repeated calculations. For example, the intermediate data calculated using the basic data may be discarded after one use, resulting in the need to recalculate the intermediate data in subsequent similar evaluations, wasting computing resources and time, and the evaluation calculation efficiency is low.

[0037] In view of this, the embodiments of the present disclosure provide an evaluation method, device, vehicle, equipment and medium for an intelligent system. The above method includes: constructing an associated data structure, the above associated data structure is used to represent the association relationship between the evaluation index of the intelligent system and the basic data fields and intermediate data fields involved in the index calculation; according to the above associated data structure, constructing an index structure about the above basic data fields and the above intermediate data fields; the above index structure is used to store and search for data values ​​corresponding to the above basic data fields and the above intermediate data fields; in the process of performing the above evaluation index calculation according to the evaluation algorithm, determining the required data field according to the above associated data structure, and querying the data value corresponding to the above required data field according to the above index structure; wherein, the data value of the intermediate data field obtained by the first calculation is stored based on the above index structure, and when the data value of the intermediate data field remains unchanged, it is searched and reused through the above index structure in subsequent use without repeated calculation.

[0038] By constructing an associated data structure between basic data fields, intermediate data fields and evaluation indicators, the association relationship between each evaluation indicator in the intelligent system and the basic data fields and intermediate data fields can be clearly and accurately presented; at the same time, by constructing an index structure for storing and searching data values ​​corresponding to basic data fields and intermediate data fields according to the above-mentioned associated data structure, the data value of the intermediate data field obtained by the first calculation is stored based on the above-mentioned index structure. In the process of calculating the evaluation indicator according to the evaluation algorithm, the required data field can be determined according to the above-mentioned associated data structure. For the data value corresponding to the intermediate data field that has been stored in the index structure, when the data value of the intermediate data field remains unchanged, it can be searched and reused through the above-mentioned index structure in subsequent use without repeated calculation, which saves the time cost and computing resources required for recalculating the data value corresponding to the intermediate data field and improves the evaluation calculation efficiency.

[0039] In some embodiments, when constructing an index structure based on an associated data structure, the higher the degree of association between a node in the associated data structure and other nodes, the higher the index level at which the node is located, and the faster the corresponding query efficiency; this helps to speed up the query efficiency corresponding to data with a higher reference frequency, thereby further improving the evaluation efficiency of the intelligent system.

[0040] In some embodiments, not only the operational association relationship between each node in the associated data structure is considered, but also the execution logic of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation is synchronously considered, and at the same time, the index level of the index structure is constructed according to the operational association relationship between each node in the associated data structure and the above-mentioned execution logic; the higher the degree of association between a node and other nodes in the associated data structure, the higher the index level of the node is, and the faster the corresponding query efficiency is; the higher the frequency of use of a node during the execution of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation, the higher the index level of the node is, and the faster the corresponding query efficiency is.

[0041] Exemplary Methods

[0042] Figure 1 The figure is a flow chart of an evaluation method of an intelligent system according to an exemplary embodiment.

[0043] Reference Figure 1 As shown, the first embodiment of the present disclosure provides an evaluation method for an intelligent system, and the method includes the following steps: S110, S120 and S130.

[0044] In step S110, a correlation data structure is constructed, where the correlation data structure is used to represent the correlation relationship between the evaluation index of the intelligent system and the basic data fields and intermediate data fields involved in the index calculation.

[0045] In some embodiments, the above-mentioned intelligent system includes but is not limited to one or a combination of the following entities: an autonomous driving vehicle, a drone, an intelligent robot, and a wearable device.

[0046] In the case where the intelligent system includes an autonomous vehicle, the evaluation indicators include at least one type of indicator in the following evaluation scenarios:

[0047] A task execution status evaluation indicator corresponding to a task execution system consisting of one or more autonomous vehicles; or

[0048] Evaluation indicators of autonomous driving decision-making performance of autonomous vehicles at complex intersections; or,

[0049] An evaluation index of the adaptability of autonomous vehicles in different terrain and road conditions; or,

[0050] Evaluation of the interactive performance between autonomous vehicles and other intelligent devices while driving.

[0051] In some embodiments, the basic data fields and intermediate data fields involved in the evaluation indicator operation can be determined by analyzing the input data and output results required for each evaluation indicator of the intelligent system: the input data required for the evaluation indicator is analyzed according to the evaluation algorithm corresponding to the evaluation indicator, and the key information corresponding to the input data is used as the basic data field; starting from the input data, the data processing process based on the intelligent system is sorted out to obtain the intermediate data; the key information corresponding to the intermediate data is used as the intermediate data field.

[0052] Figure 2 It is a schematic diagram of basic data fields and intermediate data fields involved in indicator calculation obtained by analyzing different evaluation indicators according to an exemplary embodiment, wherein (a) is a schematic diagram of the process of analyzing the evaluation indicator of braking decision performance at an intersection, and (b) is a schematic diagram of the process of analyzing the evaluation indicator of safe starting acceleration performance at an intersection.

[0053] Reference Figure 2 As shown in (a) and (b), taking an autonomous driving vehicle as an example of an intelligent system, when evaluating the autonomous driving decision-making performance of an autonomous driving vehicle at a complex intersection, specific evaluation indicators include: the vehicle's braking decision performance PG1 at the intersection and the safe starting acceleration performance PG2.

[0054] When evaluating the braking decision performance PG1, the required input data include, for example: the vehicle speed v1 corresponding to the first recognition moment t1 when vehicle A recognizes the red light at the intersection, the first acceleration a1 corresponding to the vehicle deceleration period after the first recognition moment t1, the information T on whether vehicle A changes lanes, the recognized vehicle distance D (indicating Distance) between vehicle A and the vehicle in front of it in the lane at each driving moment, etc.

[0055] In the evaluation algorithm of the braking decision performance PG1, the braking distance s1 of the vehicle is calculated based on the vehicle speed v1 corresponding to the recognition time t1 of the vehicle A and the first acceleration value a1 corresponding to the vehicle deceleration (here is an example of an operation link). The closest distance D between the vehicle A and the preceding vehicle in the same lane is determined based on the information T of whether the vehicle A changes lanes and the recognition distance D between the vehicle A and the preceding vehicle in the lane from the first recognition time t1 to the time t2 when the vehicle A stops. nearest (This is also an example of a calculation link). According to the braking distance s1 and the closest distance D nearest To evaluate the braking decision performance PG1 (here is still an example of an operation link).

[0056] Reference Figure 2As shown in (a), through the analysis of the above evaluation algorithm, it can be known that the basic data fields Z1~Z4 corresponding to the evaluation index of braking decision performance PG1 correspond to: the vehicle speed v1 corresponding to the first recognition time t1, the first acceleration a1 corresponding to the vehicle deceleration period after the first recognition time t1, the information T on whether vehicle A changes lanes, and the recognition distance D between vehicle A and the vehicle in front of it in the lane at each time of driving. The intermediate data fields ZM1~ZM2 correspond to: the braking distance s1, the closest distance D between vehicle A and the vehicle in front of it in the same lane nearest .

[0057] For the evaluation indicator PG2 of the vehicle's safe starting acceleration performance at an intersection, the required input data include, for example: the second recognition time t2 when vehicle A recognizes that the red light turns green at the intersection, the initial time t3 when the vehicle starts to accelerate from a stopped state, the second acceleration a2 of the vehicle's starting stage, whether the crossing information HC of pedestrians or electric vehicles crossing the road is detected during the vehicle's acceleration process (if a crossing situation is detected, the crossing time t4 and crossing trajectory HC1 corresponding to the pedestrian or electric vehicle crossing the road are recorded in real time), the initial time t5 when vehicle A takes the approach of deceleration or even emergency braking, the third acceleration a3 corresponding to the deceleration or emergency braking, and the real-time position WZ1 corresponding to the deceleration or emergency braking process.

[0058] In the evaluation algorithm of the safe starting acceleration performance PG2, the time difference between the second recognition time t2 when vehicle A recognizes that the red light turns green and the initial time t3 when it starts to accelerate is calculated to obtain the starting reaction time Rt1 (here is an example of a calculation link). The starting acceleration characteristic AT1 is determined according to the second acceleration a2 in the starting stage of the vehicle and the starting reaction time Rt1 (here is also an example of a calculation link, which will be referred to for understanding later and will not be explained again). The reaction time Rt2 of vehicle A to deal with the crossing is determined according to the detection time t4 when a pedestrian or electric vehicle is detected crossing the road in the crossing information HC and the initial time t5 when deceleration or even emergency braking is taken. The safe emergency obstacle avoidance characteristic BZ1 of vehicle A is determined according to the reaction time Rt2, the crossing trajectory HC1, the third acceleration a3 corresponding to deceleration or emergency braking, and the real-time position WZ1 of vehicle A during the deceleration or emergency braking process. The safe starting acceleration performance PG2 of vehicle A is determined according to the starting acceleration characteristic AT1 and the safe emergency obstacle avoidance characteristic BZ1.

[0059] Reference Figure 2As shown in (b), the basic data fields Z5 to Z11 corresponding to the evaluation index of safe starting acceleration performance PG2 correspond to: the second recognition time t2 when the red light turns green, the initial time t3 when the vehicle starts to accelerate from a stopped state, the second acceleration a2 of the vehicle starting stage, whether the crossing information HC (including the crossing time t4 and the crossing trajectory HC1) of whether pedestrians or electric vehicles are detected crossing the road during the vehicle acceleration process, the initial time t5 when the vehicle A takes the response of deceleration or even emergency braking, the third acceleration a3 corresponding to deceleration or emergency braking, and the real-time position WZ1 corresponding to deceleration or emergency braking. The intermediate data fields ZM3 to ZM6 correspond to: starting reaction time Rt1, starting acceleration characteristics AT1, response time to crossing Rt2, and safety emergency obstacle avoidance characteristics BZ1.

[0060] According to the actual needs of the evaluation scenario, more evaluation indicators can be provided, such as the performance of coping with the front vehicle's lane change and insertion when starting at a complex intersection. The process of the basic data fields and intermediate data fields involved in the specific indicators can refer to the aforementioned examples and will not be elaborated here.

[0061] Figure 3 The present invention is a schematic diagram of using a graph structure to represent an associated data structure according to an exemplary embodiment, wherein (a) is a schematic diagram of the analysis results of basic data fields and intermediate data fields involved in evaluation indicators, and (b) is an associated data structure represented in the form of a graph structure based on the analysis results.

[0062] In some embodiments, reference Figure 3 As shown in (a), the basic data fields involved in the index calculation analysis for evaluation index 1 include the following fields: X, Y, and Z. The basic data field X and the basic data field Y are calculated to obtain the intermediate data field M, and the basic data field Y and the basic data field Z are calculated to obtain the intermediate data field N. The intermediate data field M and the intermediate data field N are calculated to obtain the evaluation index 1. Figure 3 As shown in (b), the indicator calculation analysis for evaluation indicator 2 shows that the basic data fields involved in the indicator calculation include the following fields: X, Y, and R. The basic data field X and the basic data field Y are calculated to obtain the intermediate data field M. The basic data field Y and the basic data field R are calculated to obtain the intermediate data field P. The intermediate data field M and the intermediate data field P are calculated to obtain the evaluation indicator 2.

[0063] In some embodiments, reference Figure 3 As shown in (b), the above-mentioned association data structure is a graph structure, and the above-mentioned graph structure includes nodes and edges.

[0064] The nodes include: basic nodes corresponding to the basic data fields, intermediate nodes corresponding to the intermediate data fields, and evaluation indicator nodes corresponding to the evaluation indicators. Figure 3 (b) shows four basic nodes 1 to 4, which correspond to the basic data fields X, Y, Z and R respectively; it also shows three intermediate nodes and two evaluation index nodes. The three intermediate nodes correspond to the intermediate data fields M, N and P respectively, and the two evaluation index nodes are the nodes corresponding to the evaluation indexes 1 and 2.

[0065] Reference Figure 3 As shown in (b), directed arrows are used to illustrate the edges. The two ends of the above-mentioned edges are connected to associated nodes with an operation association relationship. The above-mentioned operation association relationship refers to the corresponding relationship between the input and output in each operation link of the above-mentioned evaluation index operation based on the above-mentioned evaluation algorithm. For example, since the basic node 1 and the basic node 2 are operated to obtain the intermediate node 1, there is an edge between the basic node 1 and the intermediate node 1. The basic node 1 serves as the input of the corresponding operation link, and the intermediate node 1 serves as the output. In order to indicate the corresponding direction of the input and output, a directed arrow is used to indicate the direction from the input to the output. Similarly, there is also an edge between the basic node 2 and the intermediate node 1. The edges between other basic nodes and intermediate nodes, and the edges between intermediate nodes and evaluation index nodes can be understood by reference.

[0066] In some embodiments, the above graph structure may be a directed topological graph.

[0067] In step S120, an index structure for the basic data field and the intermediate data field is constructed according to the associated data structure; the index structure is used to store and search for data values ​​corresponding to the basic data field and the intermediate data field.

[0068] By constructing an index structure for basic data fields and intermediate data fields, the data value of the intermediate data field is obtained after the first calculation and stored accordingly based on the index structure. In subsequent use, it can be searched and reused through the above index structure without repeated calculation, thus saving a lot of time cost and computing resources caused by frequent calculations on intermediate data when performing evaluation operations on a large number of evaluation indicators.

[0069] In some embodiments, based on the above-mentioned associated data structure, an index structure about the above-mentioned basic data field and the above-mentioned intermediate data field is constructed, including: based on the operational association relationship between the nodes in the above-mentioned associated data structure, an index hierarchy of the index structure is constructed; wherein, the higher the degree of association between a node and other nodes in the above-mentioned associated data structure, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster.

[0070] This embodiment can be applied to the post-evaluation scenario. The computational association relationship between nodes in the associated data structure can represent the number of citations during the evaluation index calculation period. By matching the computational association relationship between nodes in the associated data structure with the index level of the index structure, it helps to speed up the query efficiency corresponding to data with a high number of citations, thereby improving the evaluation efficiency of the intelligent system. That is, the higher the index level corresponding to the node with a higher degree of association with other nodes in the above index structure, the faster the corresponding query efficiency.

[0071] Alternatively, in other embodiments, an index structure regarding the above-mentioned basic data field and the above-mentioned intermediate data field is constructed based on the above-mentioned associated data structure, including: constructing an index hierarchy of the index structure based on the operational association relationship between the nodes in the above-mentioned associated data structure and the execution logic of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation; wherein, the higher the degree of association between a certain node and other nodes in the above-mentioned associated data structure, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster; the higher the frequency of use of a certain node during the execution of the above-mentioned evaluation algorithm for performing the above-mentioned evaluation index operation, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster.

[0072] In this embodiment, the frequency of node citation (which may also be described as the number of times the node is cited) not only considers the operational association relationship between nodes in the associated data structure in the above embodiment (which is a static association relationship that can reflect whether it is needed by other nodes at the operational level, but does not reflect the number of times it is actually called during the actual execution phase), but also considers the execution logic of the actual execution of the evaluation indicator operation. Specifically, considering that the operation association relationship is a static association relationship, it can reflect the dependency of a certain data at the operation level and whether a certain data is needed by other nodes, but will not reflect the number of times a certain data is actually called in the actual execution stage. Therefore, by introducing the execution logic, considering that the execution logic can reflect the dynamic process of the actual execution of the evaluation algorithm, the dynamic process can reflect the situation where a certain basic data is actually called multiple times and a certain intermediate data is called multiple times (for example, for real-time evaluation scenarios, there may be scenarios where the data value of a basic data field changes. Therefore, when the data value of a certain intermediate data field has been calculated in advance, due to the change in the data value of the basic data field, when calling the data value of the intermediate data field, the old data value will not be reused, but the updated basic data needs to be called again to calculate the intermediate data and the calculated new data value of the intermediate data field is used as the output; when performing the update operation of the evaluation indicator, the updated data value of the intermediate data field also needs to be called again). Therefore, combined with the specific calling relationship reflected in the associated data structure and the actual number of calls reflected in the execution logic, the number of times a node is referenced in the indicator evaluation process can be determined more accurately, thereby determining the index level corresponding to the node.

[0073] Figure 4 is a detailed implementation flowchart of step S120 according to an exemplary embodiment. Figure 5 is a schematic diagram showing a multi-layer index structure according to an exemplary embodiment.

[0074] In some embodiments, reference Figure 4 As shown, in the above step S120, according to the above associated data structure, an index structure about the above basic data field and the above intermediate data field is constructed, including the following steps: S410, S420a, S430 and S440; or, referring to Figure 4 As shown in the dotted box, step S420a can be replaced by step S420b.

[0075] In step S410, an underlying singly linked list is constructed in order according to the node numbers corresponding to the basic data fields and the intermediate data fields in the associated data structure.

[0076] Reference Figure 3 As shown in (b), corresponding node numbers ① to ⑦ are pre-set for the basic data fields and intermediate data fields. The above node numbers can be assigned in the order in which the data appears when constructing the associated data structure; for example, the node number is first assigned to the basic data field, and then the corresponding node number is assigned in sequence for each intermediate data field that is determined. Figure 4 In order to intuitively reflect the object content stored in each node in the index structure, the data fields corresponding to each basic node and intermediate node are used to illustrate. The basic nodes 1 to 4 (corresponding to the storage of basic data fields X, Y, Z and R, respectively) and the intermediate nodes 1 to 3 (corresponding to the storage of intermediate data fields M, N and P, respectively) are arranged in sequence according to the corresponding node numbers ① to ⑦ to construct the underlying single linked list, see Figure 4 The bottom-level data structure is shown in the figure.

[0077] In step S420a, the number of times the nodes corresponding to the basic data field and the intermediate data field are referenced is determined according to the computational association relationship between the nodes in the association data structure.

[0078] The number of times the node is cited indicates the number of times the current node participates in the result calculation of the referenced node; in this embodiment, for example, it can be applied to the scenario of post-evaluation, and the statistical result of whether the node in the associated data structure participates in the calculation of other nodes (this statistical result means: if the current node participates in the calculation of other nodes, no matter how many times it participates, the statistical value corresponding to the same other node is only added by 1; if it participates in the calculation of multiple other nodes, the statistical value will be accumulated) is used as the number of times the node is cited. For example, in most post-evaluation scenarios, the basic data is fixed, so the data value of the basic data field is unchanged. When calculating the data value of the intermediate data field based on the data value of the basic data field, it participates in the calculation according to the calculation association relationship in the associated data structure. Therefore, the number of times the node is cited can be determined according to the calculation association relationship between the nodes in the associated data structure. For the associated data structure in the form of a directed topological graph, the node out-degree is equal to the number of times the node is cited. The higher the correlation between the current node and other nodes, the more times the data corresponding to the current node is referenced by other nodes in the evaluation index calculation. By setting the current node to a higher index level, it helps to improve the query efficiency corresponding to data with a higher number of references, thereby further improving the evaluation efficiency of the intelligent system.

[0079] In step S420b, the number of times the nodes corresponding to the basic data field and the intermediate data field are referenced is determined according to the computational association relationship between the nodes in the association data structure and the execution logic of the evaluation algorithm for performing the evaluation index calculation.

[0080] The above-mentioned number of node citations indicates the number of times the current node participates in the result calculation of the referenced node; in this embodiment, applied to the scenario of real-time evaluation, the statistical result of whether the node in the associated data structure participates in the calculation of other nodes (this statistical result means: if the current node participates in the calculation of other nodes, no matter how many times it participates, the statistical value corresponding to the same other node is only increased by 1; if it participates in the calculation of multiple other nodes, the statistical value will be accumulated) and the statistical result of the actual usage frequency during the execution of the actual evaluation index calculation (this statistical result means: a certain node, such as X, participates in the calculation of another node Y, and the actual number of participations is several times, which is counted as the number of times node X is cited by node Y; similarly, if it participates in the calculation of multiple other nodes, the statistical value of the number of participations will also be accumulated) are combined as the number of node citations.

[0081] In some embodiments, when the above-mentioned graph structure is a directed topological graph, the number of times a node is referenced can be represented based on the node out-degree; the index level of the index structure is determined based on the node out-degree. Alternatively, when the above-mentioned graph structure is a directed topological graph, the actual number of times the node is used in the execution logic of the above-mentioned evaluation index calculation based on the node out-degree and the evaluation algorithm (for example, this frequency of use can be that a node with a small node out-degree is frequently used many times; it can also be that a node with a large node out-degree is frequently used many times, reflecting the actual number of calls) is weighted and calculated (for example, the weight of the node out-degree is set to 80% to 90%, and the weight of the actual number of uses is 20% to 10%, and the specific weight can be adjusted according to the actual situation) to obtain the number of times the node is referenced; the index level of the index structure is further determined based on the number of times the node is referenced. Both of the above-mentioned schemes for determining the number of times a node is referenced can be applied to post-evaluation scenarios. Both of the above-mentioned schemes for determining the number of times a node is referenced can also be applied to real-time evaluation scenarios. The latter method of determining the number of times a node is referenced is more comprehensive, and data search efficiency is higher for some high-frequency referenced nodes. The former method of determining the number of times a node is referenced does not require frequent adjustment of the level of the node in the multi-layer index structure, thus saving more related resources.

[0082] In the above embodiment, the number of times a node is referenced not only considers the computational association relationship between nodes in the associated data structure, but also considers the actual frequency of data use caused by the actual execution logic of the evaluation process. That is, according to the above associated data structure, the number of times the nodes corresponding to the above basic data field and the above intermediate data field are referenced is determined, including: according to the execution logic of the above evaluation index calculation performed according to the above associated data structure and the evaluation algorithm, the number of times the nodes corresponding to the above basic data field and the above intermediate data field are referenced is determined. This execution logic is dynamically advanced along with step S130, that is, step S120 and step S130 are implemented in close coordination.

[0083] For example, in real-time evaluation or some post-evaluation scenarios, the number of times a node is referenced will also be adjusted dynamically as the evaluation process progresses. The evaluation indicators may correspond to different operation logics during the calculation process, which may cause the reference process and order of the nodes to change. In this case, the number of times a node is referenced will also be adjusted dynamically as the evaluation process progresses. It is not limited to the node out-degree reflected by the associated data structure, but can also add the data usage frequency reflected by the execution logic during the dynamic evaluation period.

[0084] In terms of graph structure, it is divided into undirected topology graph and directed topology graph. In undirected topology graph, the degree of a node refers to the number of edges directly connected to the node. In directed topology graph, it includes node in-degree and node out-degree. Node in-degree refers to the number of edges from other nodes to the current node for a current node. Node out-degree refers to the number of edges from the current node to other nodes for a current node.

[0085] Continue to refer to Figure 4 As shown, taking the node out-degree as an example of the number of times a node is referenced, the number of times the basic node 1 (corresponding to the basic data field X) corresponds to is 1, the number of times the basic node 2 (corresponding to the basic data field Y) corresponds to is 3, the number of times the basic node 3 (corresponding to the basic data field Z) corresponds to is 1, and the number of times the basic node 4 (corresponding to the basic data field R) corresponds to is 1. The number of times the intermediate node 1 (corresponding to the intermediate data field M) corresponds to is 2, the number of times the intermediate node 2 (corresponding to the intermediate data field N) corresponds to is 1, and the number of times the intermediate node 3 (corresponding to the intermediate data field P) corresponds to is 1.

[0086] In step S430, the target data nodes at each index level are determined according to the matching relationship between the number of times the node is referenced and the preset grading interval.

[0087] In the embodiment of the present disclosure, the underlying single linked list is arranged from low to high according to the index level, which are respectively the 1st index level to the kth index level, where k≥2 and is a positive integer. In step S430, the target data nodes at each index level are determined, that is, the process of determining which data nodes are included in each level index (e.g., the 1st level index, the 2nd level index, ... the kth level index).

[0088] In some embodiments, the preset classification intervals include the range of citation counts corresponding to each index level; the higher the index level, the larger the value of the corresponding range of citation counts. The following examples illustrate the range of citation counts corresponding to each index level.

[0089] The preset classification intervals are as follows:

[0090] The citation count range is in the interval (1, 2] (indicating a left-open and right-closed interval), corresponding to the first index level (or described as a level 1 index);

[0091] The citation count range is in the interval [3, 4], corresponding to the second index level (or described as a 2-level index);

[0092] The range interval of the number of citations is in the interval [5, 8], corresponding to the third index level (or described as a third-level index), for example, the third-level index is the highest-level index.

[0093] In some embodiments, the total number of index levels corresponding to the preset classification interval is customized according to index efficiency and storage space limit. That is, the total number of index levels corresponding to the preset classification interval can take into account both index efficiency and storage space limit.

[0094] According to the matching relationship between the number of times the node is referenced and the preset grading interval, the target data nodes at each index level are determined, including:

[0095] For each basic data field or each intermediate data field corresponding to the current node, the number of citations of the current node is matched with the range of citations in the above-mentioned preset classification interval;

[0096] In response to the number of times a node is cited of the current node matching the target number of times range, determining a target index level corresponding to the target number of times range;

[0097] The current node is determined as a target data node at the target index level.

[0098] For example, refer to Figure 4 As shown, the number of citations corresponding to the basic node 2 (corresponding to the basic data field Y) is 3, which falls within the target citation count range interval [3, 4], and the corresponding target index level is the 2nd-level index; the basic node 2 (corresponding to the basic data field Y) is determined as the target data node corresponding to the 2nd-level index level. Similarly, traversal is performed for other nodes, where the number of citations corresponding to the intermediate node 1 (corresponding to the intermediate data field M) is 2, which falls within the target citation count range interval (1, 2], and the corresponding target index level is the 1st-level index; the intermediate node 1 (corresponding to the intermediate data field M) is determined as the target data node corresponding to the 1st-level index level.

[0099] In step S440, the underlying single linked list and the target data nodes of each index level are integrated to obtain a multi-layer index structure.

[0100] Reference Figure 4 As shown, the multi-layer index structure corresponding to the underlying single linked list and multiple index levels (for example, the intermediate node 1 at the 1st index level (corresponding to the intermediate data field M) and the basic node 2 at the 2nd index level (corresponding to the basic data field Y)) is the multi-layer index structure regarding the basic data field and the intermediate data field constructed in the embodiment of the present disclosure.

[0101] In the embodiment including steps S410 to S440, there is a difference in the idea of ​​constructing a multi-layer index structure (which can be compared to a skip list structure) and a traditional skip list structure. The original skip list structure can speed up the query efficiency of each query by occupying a part of the space in exchange for time efficiency, and there is no correlation between multiple queries, or the probability of each data being found is equal. In contrast, in the embodiments of the present disclosure, a multi-layer index structure is constructed based on an associated data structure, taking into account that the frequencies at which each basic data and intermediate data are queried or used during the evaluation index calculation are inconsistent. Therefore, the following index construction logic is set: the higher the correlation between a node and other nodes in the associated data structure, the higher the index level of the node; the index level of the node is even determined by comprehensively considering the correlation between the node and other nodes and the actual frequency of use of the node during the actual execution of the evaluation index calculation; that is, by setting the data with a higher number of references to a higher level in the index structure, the corresponding query efficiency is faster; this not only saves the time cost and computing resources required for repeated calculation of intermediate data, but also improves the data query efficiency corresponding to frequently used intermediate data, basic data, etc., which helps to further improve the evaluation efficiency of the intelligent system.

[0102] In step S130, in the process of performing the evaluation index calculation according to the evaluation algorithm, the required data field is determined according to the associated data structure, and the data value corresponding to the required data field is searched according to the index structure.

[0103] The data value of the intermediate data field obtained by the first calculation is stored based on the above index structure. When the data value of the intermediate data field remains unchanged, it is searched and reused through the above index structure in subsequent use without repeated calculation. In the case where the data value of the basic data field is updated and the data value of the intermediate data field needs to be updated, the data value of the basic data field is updated and stored based on the above index structure, and the intermediate data is calculated based on the updated data value of the basic data field found, and the updated data value of the intermediate data field is updated and stored in the above index structure.

[0104] For example, in the process of evaluating the index, refer to Figure 3 As shown in (b), for evaluation indicator 1, the required data fields are determined to include: intermediate data fields: field M and field N; basic data fields: field X, field Y and field Z.

[0105] Assume that the data values ​​corresponding to fields M and N have not yet been stored in the index structure during this query. Figure 4As shown, according to the index structure, the basic data field X and the basic data field Y are first queried, and the data value corresponding to the intermediate data field M is calculated according to the queried data values ​​corresponding to the basic data fields X and Y, and stored in the index structure. When the data value of the intermediate data field remains unchanged, the intermediate data can be reused only based on the index structure when it is used later.

[0106] In some embodiments, the above index structure is an index level generated by taking the node out-degree of the directed topological graph as the number of times the node is referenced. In other embodiments, the index level in the above index structure is also dynamically adjusted as the evaluation logic is executed, for example, dynamically adjusted according to the node out-degree of the directed topological graph and the number of queries in the corresponding evaluation process. The larger the node out-degree, the higher the corresponding index level; the more queries in the evaluation process, the higher the corresponding index level. During the evaluation process, the data value corresponding to the required data field is queried based on the dynamically changing index structure.

[0107] In each of the above embodiments, performing the above evaluation index calculation according to the above evaluation algorithm corresponds to one of the following evaluation situations: performing a post-evaluation after the above intelligent system completes the simulation; performing a real-time evaluation during the above intelligent system performs the simulation.

[0108] Among them, for the above-mentioned post-evaluation, the basic data fields and intermediate data fields involved in the indicator calculation are determined based on all the calculation links involved in the entire simulation process. That is, in the post-evaluation scenario, the basic data fields and intermediate data fields involved in the evaluation indicators are determined globally and statically, and all the basic data fields and intermediate data fields can be determined based on all the calculation links involved in the entire simulation process, and the associated data structure and the corresponding index structure are determined accordingly.

[0109] For the above-mentioned real-time evaluation, the basic data fields and intermediate data fields involved in the indicator calculation are obtained by dynamic updating based on the existing calculation links in the real-time simulation process, and the above-mentioned existing calculation links are dynamically updated as the real-time simulation process proceeds; the above-mentioned associated data structure and the above-mentioned index structure are dynamically updated as at least one of the above-mentioned basic data fields and the above-mentioned intermediate data fields is updated.

[0110] For example, the last moment is represented as t1, and the basic data fields and intermediate data fields of the existing operation links corresponding to the last moment are set J1; according to set J1 and the corresponding evaluation index, the associated data structure GL1 can be constructed, and at the same time, the multi-layer index structure SY1 of different levels can be constructed based on the difference in the reference frequency (or described as the number of times referenced) of the nodes in the associated data structure GL1. As the real-time simulation process develops, assuming that the current moment is t1+δt, during the δt (time microelement) period, as the dynamic simulation process proceeds, at least one of the basic data fields or intermediate data fields involved is updated (for example, newly added, or the association relationship has changed), and the update situation is represented as δJ, then at the current moment t1+δt, the associated data structure GL1 will be dynamically updated according to δJ to obtain the updated associated data structure GL2; at the same time, adapted to the update of the associated data structure, the multi-layer index structure SY1 is updated to obtain the multi-layer index structure SY2 adapted to the associated data structure GL2.

[0111] In an embodiment including the above steps S110 to S130, by constructing an associated data structure between basic data fields, intermediate data fields and evaluation indicators, the association relationship between each evaluation indicator in the intelligent system and the basic data fields and intermediate data fields can be clearly and accurately presented; at the same time, by constructing an index structure for storing and searching data values ​​corresponding to basic data fields and intermediate data fields according to the above associated data structure, the data value of the intermediate data field obtained by the first calculation is stored based on the above index structure. In the process of calculating the evaluation indicator according to the evaluation algorithm, the required data field can be determined according to the above associated data structure. For the data value corresponding to the intermediate data field that has been stored in the index structure, when the data value of the intermediate data field remains unchanged, it can be searched and reused through the above index structure in subsequent use without repeated calculation, which saves the time cost and computing resources required for recalculating the data value corresponding to the intermediate data field and improves the evaluation calculation efficiency.

[0112] Exemplary Devices

[0113] Figure 6 The figure is a block diagram of an evaluation device for an intelligent system according to an exemplary embodiment.

[0114] Reference Figure 6 As shown, the second exemplary embodiment of the present disclosure provides an evaluation device 600 for an intelligent system. The device 600 includes: an associated data structure building module 610 , an index structure building module 620 and an evaluation module 630 .

[0115] The above-mentioned associated data structure construction module 610 is used to construct an associated data structure, and the above-mentioned associated data structure is used to represent the association relationship between the evaluation index of the intelligent system and the basic data fields and intermediate data fields involved in the index calculation.

[0116] The index structure building module 620 is used to build an index structure about the basic data field and the intermediate data field according to the associated data structure; the index structure is used to store and search for data values ​​corresponding to the basic data field and the intermediate data field.

[0117] The above-mentioned evaluation module 630 is used to determine the required data fields according to the above-mentioned associated data structure and query the data values ​​corresponding to the above-mentioned required data fields according to the above-mentioned index structure during the process of performing the above-mentioned evaluation index calculation according to the evaluation algorithm; wherein the data value of the intermediate data field obtained by the first calculation is stored based on the above-mentioned index structure, and when the data value of the above-mentioned intermediate data field remains unchanged, it is searched and reused through the above-mentioned index structure during subsequent use without repeated calculation.

[0118] For more details and beneficial effects of this embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0119] Example Vehicles

[0120] Figure 7 is a block diagram of a vehicle 700 according to an exemplary embodiment. The vehicle 700 may be a fuel vehicle, a hybrid vehicle, an electric vehicle, a fuel cell vehicle or other types of vehicles. Figure 7 As shown, vehicle 700 may include multiple subsystems, for example, a drive system 710, a control system 720, a perception system 730, a communication system 740, an information display system 750, and a computing system 760. Vehicle 700 may also include more or fewer subsystems, and each subsystem may also include multiple components, which are not described one by one here.

[0121] The driving system 710 includes components that provide power movement for the vehicle 700, such as an engine, an energy source, a transmission device, etc.

[0122] The control system 720 includes components that provide control for the vehicle 700, such as vehicle control, cockpit equipment control, and driving assistance control.

[0123] The perception system 730 includes components that provide surrounding environment perception for the vehicle 700, such as a vehicle positioning system, a laser sensor, a voice sensor, an ultrasonic sensor, a camera device, etc.

[0124] The communication system 740 includes components that provide communication connections for the vehicle 700, such as mobile communication networks (such as 3G, 4G, 5G networks, etc.), WiFi, Bluetooth, and Internet of Vehicles.

[0125] The information display system 750 includes components that provide various information displays for the vehicle 700, such as vehicle information display, navigation information display, entertainment information display, etc.

[0126] The computing and processing system 760 includes components that provide data computing and processing capabilities for the vehicle 700. The computing and processing system 760 may include at least one processor 761 and a memory 762. The processor 761 may execute instructions stored in the memory 762.

[0127] The processor 761 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0128] Memory 762 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0129] In the embodiment of the present disclosure, a set of instruction sets is stored in the memory 762, and the processor 761 can execute the instruction set to implement all or part of the steps of the evaluation method described in any of the above exemplary embodiments.

[0130] Exemplary Electronic Devices

[0131] Figure 8 1 is a block diagram of an electronic device 800 according to an exemplary embodiment. The electronic device 800 may be a vehicle controller, a vehicle terminal, a vehicle computer or other types of electronic devices.

[0132] Reference Figure 8As shown, the electronic device 800 may include at least one processor 810 and a memory 820. The processor 810 may execute instructions stored in the memory 820. The processor 810 is communicatively connected to the memory 820 via a data bus. In addition to the memory 820, the processor 810 may also be communicatively connected to an input device 830, an output device 840, and a communication device 850 via a data bus.

[0133] The processor 810 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0134] The memory 820 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0135] In the embodiment of the present disclosure, executable instructions are stored in the memory 820, and the processor 810 can read the executable instructions from the memory 820 and execute the instructions to implement all or part of the steps of the evaluation method described in any of the above exemplary embodiments.

[0136] Exemplary computer-readable storage media

[0137] In addition to the above methods and devices, the exemplary embodiments of the present disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to implement all or part of the steps described in any method in the above exemplary embodiments.

[0138] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and also conventional procedural programming languages, such as "C" language or similar programming languages ​​and scripting languages ​​(e.g., Python). The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on the remote computing device, or entirely on the remote computing device or server.

[0139] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of readable storage media include: a static random access memory (SRAM) with one or more wires electrically connected, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk, or any suitable combination of the above.

[0140] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0141] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for evaluating an intelligent system, characterized in that: include: Constructing a correlation data structure, wherein the correlation data structure is used to represent the correlation relationship between the evaluation index of the intelligent system and the basic data fields and the intermediate data fields involved in the index calculation; According to the associated data structure, construct an index structure about the basic data field and the intermediate data field; the index structure is used to store and search for data values ​​corresponding to the basic data field and the intermediate data field; In the process of performing the evaluation index calculation according to the evaluation algorithm, the required data field is determined according to the associated data structure, and the data value corresponding to the required data field is queried according to the index structure; wherein the data value of the intermediate data field obtained by the first calculation is stored based on the index structure, and when the data value of the intermediate data field remains unchanged, it is searched and reused through the index structure in subsequent use without repeated calculation.

2. The method according to claim 1, characterized in that According to the associated data structure, constructing an index structure about the basic data field and the intermediate data field, including: According to the computational association relationship between the nodes in the association data structure, an index level of the index structure is constructed; wherein the higher the association degree between a node and other nodes in the association data structure, the higher the index level of the node is, and the corresponding query efficiency is faster; or, An index hierarchy of the index structure is constructed according to the operational association relationship between the nodes in the associated data structure and the execution logic of the evaluation algorithm for performing the above-mentioned evaluation index operation; wherein, the higher the degree of association between a certain node and other nodes in the associated data structure, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster; the higher the frequency of use of a certain node during the execution of the evaluation algorithm for performing the above-mentioned evaluation index operation, the higher the index hierarchy of the node is, and the corresponding query efficiency is faster.

3. The method according to claim 2, characterized in that According to the associated data structure, constructing an index structure about the basic data field and the intermediate data field, including: Constructing an underlying singly linked list in order according to the node numbers corresponding to the basic data fields and the intermediate data fields in the associated data structure; According to the computational association relationship between the nodes in the association data structure, the number of times the nodes corresponding to the basic data field and the intermediate data field are referenced is determined; or, according to the computational association relationship between the nodes in the association data structure and the execution logic of the evaluation algorithm for performing the above evaluation index calculation, the number of times the nodes corresponding to the basic data field and the intermediate data field are referenced is determined; the number of times the node is referenced indicates the number of times the current node participates in the result calculation of the referenced node; Determine the target data nodes at each index level according to the matching relationship between the number of times the node is referenced and the preset grading interval; The underlying single linked list and the target data nodes of each index level are integrated to obtain a multi-layer index structure.

4. The method according to claim 3, characterized in that The preset classification interval includes the range of citation times corresponding to each index level; the higher the index level, the larger the value of the corresponding range of citation times; Wherein, according to the matching relationship between the number of times the node is referenced and the preset grading interval, determining the target data node at each index level includes: For each basic data field or each intermediate data field corresponding to the current node, matching the node citation count of the current node with the citation count range interval in the preset grading interval; In response to the number of node citations of the current node matching the target citation number range interval, determining a target index level corresponding to the target citation number range interval; The current node is determined as a target data node at the target index level.

5. The method according to any one of claims 1 to 4, characterized in that The associated data structure is a graph structure, the graph structure includes nodes and edges, the nodes include: basic nodes corresponding to the basic data fields, intermediate nodes corresponding to the intermediate data fields, and evaluation indicator nodes corresponding to the evaluation indicators; Both ends of the edge are connected with associated nodes having an operation association relationship, and the operation association relationship refers to the corresponding relationship between input and output in each operation link of the evaluation index operation based on the evaluation algorithm.

6. The method according to claim 5, characterized in that The graph structure is a directed topological graph, and the index level of the index structure is determined based on the node out-degree; or, The index level of the index structure is determined based on the node out-degree and the execution logic of the evaluation algorithm for performing the above evaluation index calculation.

7. The method according to claim 1, characterized in that Performing the evaluation index calculation according to the evaluation algorithm corresponds to one of the following evaluation situations: After the intelligent system has completed the simulation, it performs a post-evaluation; The intelligent system performs real-time evaluation during simulation execution; Wherein, for the post-evaluation, the basic data fields and intermediate data fields involved in the indicator calculation are determined based on all the calculation links involved in the entire simulation process; For the real-time evaluation, the basic data fields and intermediate data fields involved in the indicator calculation are dynamically updated based on the existing calculation links in the real-time simulation process, and the existing calculation links are dynamically updated as the real-time simulation process progresses; the associated data structure and the index structure are dynamically updated as at least one of the basic data fields and the intermediate data fields is updated.

8. The method according to claim 1, characterized in that The intelligent system includes one or a combination of the following entities: Autonomous vehicles, drones, smart robots, wearable devices; In the case where the intelligent system includes an autonomous driving vehicle, the evaluation indicators include at least one type of indicators in the following evaluation scenarios: A task execution status evaluation indicator corresponding to a task execution system consisting of one or more autonomous vehicles; or Evaluation indicators of autonomous driving decision-making performance of autonomous vehicles at complex intersections; or, An evaluation index of the adaptability of autonomous vehicles in different terrain and road conditions; or, Evaluation of the interactive performance between autonomous vehicles and other intelligent devices while driving.

9. An evaluation device for an intelligent system, characterized in that: include: An associated data structure building module, used to build an associated data structure, wherein the associated data structure is used to represent the association relationship between the evaluation index of the intelligent system and the basic data fields and intermediate data fields involved in the index calculation; An index structure building module, used to build an index structure about the basic data field and the intermediate data field according to the associated data structure; the index structure is used to store and search for data values ​​corresponding to the basic data field and the intermediate data field; An evaluation module is used to determine the required data fields according to the associated data structure and query the data values ​​corresponding to the required data fields according to the index structure during the process of performing the evaluation index calculation according to the evaluation algorithm; wherein the data values ​​of the intermediate data fields obtained by the first calculation are stored based on the index structure, and when the data values ​​of the intermediate data fields remain unchanged, they are searched and reused through the index structure in subsequent use without repeated calculation.

10. A vehicle, characterized in that: The vehicle stores a set of instruction sets, and the instruction sets are executed by the vehicle to implement the method described in any one of claims 1-8.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-8.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of any method described in claims 1-8 are implemented.

Images