A scenario testing method, device, medium and equipment for driving control system

Abstract

The present invention discloses a scenario testing method, apparatus, medium, and equipment for a driving control system. The method inputs a state transition diagram and then, based on the state transition diagram, gradually achieves test coverage for each state transition in the state transition diagram through cyclic iterative mutation of scenario samples. The simulation output of the scenario samples is used as a reference during the cyclic iterative mutation, allowing the iterative process to quickly converge to the corresponding state transition and complete the coverage test. The present invention can intuitively test the driving control system's ability and level of handling the driving environment, with high testing efficiency.

Description

Technical Field

[0001] The invention relates to a simulation test of a driving control system. Background Art

[0002] Because intelligent driving systems need to analyze and solve real-world driving scenarios, they typically apply different route planning strategies to different scenarios. Therefore, scenario modeling is an effective means of analyzing the system's internal architecture, and scenario-based testing of intelligent driving systems is a more accurate and objective approach. Intelligent driving systems treat a series of specific tasks during vehicle operation as specific problems, such as left and right turns, as well as specific driving scenarios. Whenever a vehicle is in a scenario, it will perform the same analysis and judgment operations in the same state. Therefore, scenario modeling and scenario construction are a feasible and effective approach for intelligent driving systems. In this approach, test cases are explicitly represented as simulated driving scenarios. By converting system code into a scenario model, this method not only clearly represents the entire sequential flow structure of system states, allowing for analysis and testing of this architecture, but also significantly reduces complexity compared to direct code testing.

[0003] During the path planning and driving control process of an intelligent driving system, the vehicle, starting from startup, solves various specific tasks required by the path at each time period based on various external information. For example, navigation analysis determines that a certain intersection is the only way to the destination. As the vehicle approaches the intersection, it determines the next task to solve based on its current position: how to pass through the intersection. After passing the intersection, the vehicle returns to normal driving mode. The entire driving process is a sequential process, and at each point in time, the vehicle is in a specific state to handle the specific scenario at that moment. Therefore, the path planning module of the intelligent driving system can be modeled as a transition system model. The transformation of the scenarios handled by the vehicle during operation can be abstracted as the transformation of the vehicle state. The different logic of the vehicle in different states represents the different methods used by the vehicle to handle different scenarios.

[0004] When testing intelligent driving systems, constructing a test suite that encompasses the vast majority of real-world driving scenarios is crucial for verifying code correctness. Testing more driving scenarios means a greater likelihood of discovering existing vulnerabilities. Scenario modeling can help reduce the complexity of static analysis, but analyzing large systems like intelligent driving systems still requires significant manpower. Fuzz testing is an effective testing method to address this issue. Starting with random test cases, it continuously tests the system using generated samples. Based on the test results, effective fuzz testing strategies are used to generate new samples that are more likely to be effective than previous ones for further testing. This automates the entire process and, when guided by a sound strategy, can achieve excellent results. Summary of the Invention

[0005] Problem to be solved by the present invention: testing and verification of a driving control system.

[0006] To solve the above problems, the present invention adopts the following solutions:

[0007] According to the present invention, a scenario testing method for a driving control system involves a simulation system loaded with a driving control system program; the driving control system program is plugged with plugged program instructions; the plugged program instructions are used to output the current control state of the driving control system and the value of the corresponding state attribute; the method comprises the following steps:

[0008] Step S1: Obtain state control diagram information;

[0009] The state control diagram information includes a set of control states, a set of state transitions, and a set of transition conditions;

[0010] The control state is used to represent the control state of the driving control system, including a set of state attributes;

[0011] The state transition includes a first control state and a second control state, which is used to indicate the action of the driving control system transitioning from the first control state to the second control state;

[0012] The transition condition corresponds to the state transition, and is used to indicate the condition that needs to be satisfied for the driving control system to transition from the first control state corresponding to the state transition to the second control state corresponding to the state transition, and is a state attribute Boolean expression;

[0013] The state attribute Boolean expression is a simple Boolean expression or a compound Boolean expression formed by combining several simple Boolean expressions through conjunction and / or disjunction operations;

[0014] In the state attribute Boolean expression, the simple Boolean expression is a comparison expression about the state attribute, and the state attribute comes from the first control state of the corresponding state transition;

[0015] Step S2: constructing an initial scene sample set by randomly generating scene sample information, and initializing a second state transition set with the state transition set in the state control diagram information;

[0016] The scene sample includes current scene sample information; the current scene sample information corresponds to the current scene sample and is scene sample information; in the initial set of scene samples, the current scene sample information is randomly generated scene sample information;

[0017] Step S3: Using the current scenario sample information of the scenario sample in the scenario sample set as simulation data for simulating a real scenario, the driving control system program is simulated and tested through a simulation system. During the simulation test, the log output of the driving control system program and the output of its instrumented program instructions are collected to form a sample output sequence corresponding to the scenario sample; the sample output sequence is a sequence composed of sample outputs; the sample output includes a control state and a value of its corresponding state attribute;

[0018] Step S4: extracting a control state transition from the simulation result output information sequence, and removing the control state transition as a state transition from the second state transition set; then determining whether to terminate the test based on whether the second state transition set is empty as a test termination condition;

[0019] Step S5: Traversing the sample outputs in the sample output sequence, if the control state of the traversed sample output is the first control state of a state transition in the second state transition set, then combining the scene sample corresponding to the sample output sequence, the sample output, and the corresponding state transition into scene sample state expected transition information, thereby obtaining a scene sample state expected transition information set;

[0020] The expected state transition information of the scenario sample includes the scenario sample, sample output and state migration; the sample output is the simulation result output information;

[0021] Step S6: Calculate the distance between the sample output of each scene sample expected state transition information and the state transition in the scene sample expected state transition information set, and extract the scene sample with the smallest distance between the sample output corresponding to the state transition and the state transition from the scene sample expected state transition information set to form a seed corresponding to the state transition, thereby forming a seed set; the seed includes the state transition and the current scene sample information;

[0022] The distance between the sample output and the state transition is the distance between the state attribute Boolean expression of the transition condition corresponding to the state transition and the sample output;

[0023] Step S7: performing mutation processing on the current scene sample information of each seed in the seed set to obtain new current scene sample information, forming a scene sample set, and then returning to step S3;

[0024] In step S6, the distance between the state attribute Boolean expression and the sample output is calculated as follows:

[0025] For simple Boolean expressions, calculate the distance between the simple Boolean expression and the sample output;

[0026] For conjunction operation, calculate the distance of each sub-item and calculate the sum of the sub-item distances;

[0027] For the disjunction operation, calculate the distance of each sub-item and take the minimum value of the sub-item distance;

[0028] The sub-item distance is the distance between the sub-state attribute Boolean expression of the state attribute Boolean expression and the sample output;

[0029] The distance between the simple Boolean expression and the sample output is calculated by calculating the distance between the size comparison expression of the state attribute and the sample output. The distance between the size comparison expression F(op, sp, v) of the state attribute and the sample output is calculated using the following formula:

[0030] If op is equal to or not equal to, then D(F(op,sp,v))=K; otherwise D(F(op,sp,v))=K+abs(v-out(sp));

[0031] Where op is the operator of the size comparison expression, sp is the state attribute, v is the comparison value of the state attribute, F(op, sp, v) is expressed as the size comparison expression: sp op v or v op sp, D(F(op, sp, v)) is the distance between the size comparison expression and the sample output, K is a preset constant, abs is the absolute value function, and out(sp) is the value of the state attribute sp in the sample output.

[0032] Furthermore, according to the scenario testing method of the driving control system of the present invention, in step S7, a preset mutation mode is randomly selected from the preset mutation mode list according to the mutation mode selection probability table for mutation.

[0033] Furthermore, according to the scenario testing method for a driving control system of the present invention, the scenario sample also includes seed scenario sample information and variation information; the seed scenario sample information corresponds to the seed scenario sample and is scenario sample information; the variation information is information about the variation of the seed scenario sample to the current scenario sample, including state transition, the distance between the sample output and the state transition, the probability corresponding to each preset variation mode, and the preset variation mode selected for the variation;

[0034] In step S2, in the initial set of scene samples, both the seed scene sample information and the variation information are empty;

[0035] The seed also includes the distance between the sample output and the state transition, the original seed scene sample information and the original seed variation information;

[0036] In step S6, when forming the seed, the seed scene sample information and variation information in the scene example are used as the original seed scene sample information and original seed variation information of the seed respectively;

[0037] In step S7, when mutating the current scene sample information of the seed, first determine whether the original seed scene sample information of the seed is empty and whether the state transition is the same as the state transition in the original seed mutation information; if the original seed scene sample of the seed is empty or the state transition is different from the state transition in the original seed mutation information, initialize the mutation mode selection probability table so that the selection probabilities of each preset mutation mode in the mutation mode selection probability table are the same; otherwise, extract the probabilities corresponding to each preset mutation mode in the original seed mutation information of the seed to form a mutation mode selection probability table, and extract the preset mutation mode selected for mutation in the original seed mutation information of the seed as the original mutation mode, and then adjust the probability corresponding to the original mutation mode in the mutation mode selection probability table according to the comparison between the distance between the sample output of the seed and the state transition and the distance between the sample output and the state transition in the original seed mutation information, and finally randomly select a preset mutation mode from the preset mutation mode list according to the adjusted mutation mode selection probability table for mutation.

[0038] Furthermore, according to the scenario testing method of the driving control system of the present invention, the preset variation mode includes: randomly changing the starting and ending points of the vehicle, randomly changing the signal light sequence, randomly changing the NPC vehicle, and adjusting the interference intensity of the NPC around the controlled vehicle.

[0039] According to the present invention, a scenario testing device for a driving control system is connected to a simulation system loaded with a driving control system program; the driving control system program is plugged with plugged program instructions; the plugged program instructions are used to output the current control state of the driving control system and the value of the corresponding state attribute; the device includes the following modules:

[0040] Module M1 is used to: obtain state control diagram information;

[0041] The state control diagram information includes a set of control states, a set of state transitions, and a set of transition conditions;

[0042] The control state is used to represent the control state of the driving control system, including a set of state attributes;

[0043] The state transition includes a first control state and a second control state, which is used to indicate the action of the driving control system transitioning from the first control state to the second control state;

[0044] The transition condition corresponds to the state transition, and is used to indicate the condition that needs to be satisfied for the driving control system to transition from the first control state corresponding to the state transition to the second control state corresponding to the state transition, and is a state attribute Boolean expression;

[0045] The state attribute Boolean expression is a simple Boolean expression or a compound Boolean expression formed by combining several simple Boolean expressions through conjunction and / or disjunction operations;

[0046] In the state attribute Boolean expression, the simple Boolean expression is a comparison expression about the state attribute, and the state attribute comes from the first control state of the corresponding state transition;

[0047] Module M2 is used to: construct an initial scene sample set by randomly generating scene sample information, and initialize a second state transition set with the state transition set in the state control diagram information;

[0048] The scene sample includes current scene sample information; the current scene sample information corresponds to the current scene sample and is scene sample information; in the initial set of scene samples, the current scene sample information is randomly generated scene sample information;

[0049] Module M3 is configured to: use the current scenario sample information of the scenario sample in the scenario sample set as simulation data for simulating a real scenario, perform simulation testing on the driving control system program through a simulation system, and during the simulation testing, collect the log output of the driving control system program and the output of its instrumented program instructions to form a sample output sequence corresponding to the scenario sample; the sample output sequence is a sequence composed of sample outputs; the sample outputs include a control state and the value of its corresponding state attribute;

[0050] Module M4 is configured to extract a control state transition from the simulation result output information sequence and remove the control state transition as a state transition from the second state transition set; and then determine whether to terminate the test based on whether the second state transition set is empty as a test termination condition;

[0051] Module M5 is configured to: traverse the sample outputs in the sample output sequence, and if the control state of the traversed sample output is the first control state of a state transition in the second state transition set, combine the scenario sample corresponding to the sample output sequence, the sample output, and the corresponding state transition into scenario sample state expected transition information, thereby obtaining a scenario sample state expected transition information set;

[0052] The expected state transition information of the scenario sample includes the scenario sample, sample output and state migration; the sample output is the simulation result output information;

[0053] Module M6 is configured to calculate the distance between the sample output of each scene sample expected state transition information and the state transition in the scene sample expected state transition information set, and extract the scene sample with the smallest distance between the sample output corresponding to the state transition and the state transition from the scene sample expected state transition information set to form a seed corresponding to the state transition, thereby forming a seed set; the seed includes the state transition and the current scene sample information;

[0054] The distance between the sample output and the state transition is the distance between the state attribute Boolean expression of the transition condition corresponding to the state transition and the sample output;

[0055] Module M7 is used to perform mutation processing on the current scene sample information of each seed in the seed set to obtain new current scene sample information, form a scene sample set, and then return to module M3;

[0056] In the module M6, the distance between the state attribute Boolean expression and the sample output is calculated as follows:

[0057] For simple Boolean expressions, calculate the distance between the simple Boolean expression and the sample output;

[0058] For conjunction operation, calculate the distance of each sub-item and calculate the sum of the sub-item distances;

[0059] For the disjunction operation, calculate the distance of each sub-item and take the minimum value of the sub-item distance;

[0060] The sub-item distance is the distance between the sub-state attribute Boolean expression of the state attribute Boolean expression and the sample output;

[0061] The distance between the simple Boolean expression and the sample output is calculated by calculating the distance between the size comparison expression of the state attribute and the sample output. The distance between the size comparison expression F(op, sp, v) of the state attribute and the sample output is calculated using the following formula:

[0062] If op is equal to or not equal to, then D(F(op,sp,v))=K; otherwise D(F(op,sp,v))=K+abs(v-out(sp));

[0063] Where op is the operator of the size comparison expression, sp is the state attribute, v is the comparison value of the state attribute, F(op, sp, v) is expressed as the size comparison expression: sp op v or v op sp, D(F(op, sp, v)) is the distance between the size comparison expression and the sample output, K is a preset constant, abs is the absolute value function, and out(sp) is the value of the state attribute sp in the sample output.

[0064] Furthermore, according to the scenario testing device of the driving control system of the present invention, in the module M7, a preset mutation mode is randomly selected from the preset mutation mode list for mutation according to the mutation mode selection probability table.

[0065] Furthermore, the scenario testing device for the driving control system according to the present invention is characterized in that the scenario example further includes seed scenario sample information and variation information; the seed scenario sample information corresponds to the seed scenario sample and is the scenario sample information; the variation information is information about the variation of the seed scenario sample to the current scenario sample, including state transition, the distance between the sample output and the state transition, the probability corresponding to each preset variation mode, and the preset variation mode selected for the variation;

[0066] In the module M2, in the initial set of scene samples, both the seed scene sample information and the variation information are empty;

[0067] The seed also includes the distance between the sample output and the state transition, the original seed scene sample information and the original seed variation information;

[0068] In the module M6, when forming the seed, the seed scene sample information and variation information in the scene example are used as the original seed scene sample information and original seed variation information of the seed respectively;

[0069] In the module M7, when mutating the current scene sample information of the seed, first determine whether the original seed scene sample information of the seed is empty and whether the state transition is the same as the state transition in the original seed mutation information; if the original seed scene sample of the seed is empty or the state transition is different from the state transition in the original seed mutation information, initialize the mutation mode selection probability table so that the selection probabilities of each preset mutation mode in the mutation mode selection probability table are the same; otherwise, extract the probabilities corresponding to each preset mutation mode in the original seed mutation information of the seed to form a mutation mode selection probability table, and extract the preset mutation mode selected for mutation in the original seed mutation information of the seed as the original mutation mode, and then adjust the probability corresponding to the original mutation mode in the mutation mode selection probability table according to the comparison between the distance between the sample output of the seed and the state transition and the distance between the sample output and the state transition in the original seed mutation information, and finally randomly select a preset mutation mode from the preset mutation mode list according to the adjusted mutation mode selection probability table for mutation.

[0070] Furthermore, according to the scenario testing device of the driving control system of the present invention, the preset variation mode includes: randomly changing the starting and ending points of the vehicle, randomly changing the signal light sequence, randomly changing the NPC vehicle, and adjusting the interference intensity of the NPC around the controlled vehicle.

[0071] According to a medium of the present invention, the medium stores a program instruction set that can be read by a machine; when the program instruction set stored in the medium is read and executed by a machine, the above-mentioned scenario testing method of the driving control system can be implemented.

[0072] Furthermore, according to a device of the present invention, the device includes a processor and a memory; a program instruction set is stored in the memory; when the program instruction set stored in the memory is loaded and executed by the processor, the above-mentioned scenario testing method of the driving control system can be implemented.

[0073] The technical effects of the present invention are as follows:

[0074] The present invention realizes its test target for state transition test coverage by constructing scenario parameters, can intuitively test the processing capability and level of the driving control system for the driving environment, and has high test efficiency.

[0075] The present invention uses the simulation output of the scene sample as a reference when performing loop iterative variation, so that the iteration process can quickly converge to its corresponding state migration to complete the goal of coverage testing, and the testing efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a test control flow chart of an embodiment of the present invention.

[0077] Figure 2It is a structural connection diagram of an embodiment of a test control system of the present invention.

[0078] Figure 3 It is a structural schematic diagram of an embodiment of the device of the present invention. DETAILED DESCRIPTION

[0079] The present invention will be further described in detail below with reference to the accompanying drawings.

[0080] Reference Figure 2 This embodiment relates to a test control system 800. The test control system 800 is connected to a simulation system 900. The simulation system 900 is loaded with a driving control system 910. The test control system 800, the simulation system 900, and the driving control system 910 are all implemented by devices executing computer programs. The test control system 800 receives a state control diagram as input. By executing the computer program, scenario parameters are generated based on the state control diagram and input into the simulation system 900. The simulation system 900, by executing the computer program, receives the scenario parameters and then simulates a driving environment for the driving control system 910 based on the scenario parameters. The driving control system 910, by executing the computer program, controls vehicle driving according to the driving environment simulated by the simulation system 900. During the vehicle driving control, it returns status information to the test control system 800 in real time. The test control system 800 performs test coverage analysis based on the returned status information. If the test coverage target is met, the simulation test process ends. Otherwise, new scenario parameters are generated and input into the simulation system 900 for the next simulation step. The test control system 800, the simulation system 900, and the driving control system 910 can be configured on the same device or on different devices. If configured on multiple devices, these devices are connected via a network. Figure 3 The device of configuring the test control system 800 is exemplified. The device is specifically an electronic device, referring to Figure 2, comprising at least a connected processor 100 and memory 200. The processor 100 is typically a general-purpose computer processor capable of executing computer program instructions, and the memory 200 is typically a medium that can store data without loss after a power outage, including but not limited to a disk, tape, flash memory, etc. This medium is also referred to as the medium in the aforementioned disclosure. The memory 200 is typically used to store computer program instruction sets and data. The processor 100 implements its corresponding automation functions by loading the program instruction set stored in the memory 200. Specifically, in this embodiment, the processor 100 implements the aforementioned test control system 800 by loading and executing the program instruction set stored in the memory 200. When the processor 100 executes the program instruction set to implement the test control system 800, the process performed by the processor 100 is the scenario testing method for the driving control system referred to in the present disclosure. Similarly, the driving control system 910 is implemented by a machine or device executing the driving control system program referred to in the present disclosure, and the simulation system 900 is implemented by a machine or device executing a simulator program. Examples of simulator programs include LGSVL and Carla. Examples of driving control system programs include Apollo and Autoware. Specifically, in this embodiment, the simulator program uses the LGSVL simulator, and the driving control system program uses Baidu Apollo 6.0. More specifically, the driving control system referred to in this embodiment is the control planning module of Baidu Apollo 6.0's autonomous driving system. To achieve the testing objectives of the present invention, this embodiment instrumented the control planning module of Baidu Apollo 6.0 by inserting instrumentation program instructions into the control planning module of Baidu Apollo 6.0. In this embodiment, the instrumentation program instructions are combined with Baidu Apollo 6.0's own log output, enabling the instrumentation program instructions to output status information.

[0081] Figure 2 In the embodiment, the state control diagram input by the test control system 800 corresponds to the scenario test method of the driving control system of the present invention, that is, the state control diagram information; the scenario parameters output by the test control system 800 correspond to the scenario test method of the driving control system of the present invention, that is, the scenario sample information; the state information interactively received by the test control system 800 corresponds to the scenario test method of the driving control system of the present invention, that is, the current control state of the driving control system 910 and the value of the corresponding state attribute.

[0082] The state control graph is a directed graph consisting of nodes and directed edges. A directed edge connects two nodes and points from one node to another node. The former node is the starting node of the directed edge, and the latter node is the end node of the directed edge. Corresponding to the state control graph information, the node corresponds to the control state, the directed edge corresponds to the state transition and the migration condition corresponding to the state transition. Specifically, the state control graph information includes a set of control states, a set of state transitions and a set of migration conditions. The control state is used to represent the control state of the driving control system, including a control state identification code and a set of state attributes. Simply put, the state control graph information of the present invention is a directed graph with control states as nodes and state transitions as edges.

[0083] State attributes are parameters used by the vehicle driving control system to characterize the current control state, including system parameters and external environment parameters. System parameters include vehicle speed, acceleration, throttle level, brake strength, steering wheel angle, etc.; external environment parameters are used to characterize the data sensed by the vehicle sensors, such as traffic light color, surrounding vehicles and their positions, etc. Since there are many types of state attributes, there are also many state attributes required to characterize the control state, but not all of them are needed. Therefore, the state attributes of the control state need to be represented by a set, that is, a set of state attributes. In addition, the sets of state attributes of different control states are usually different, but of course they may be the same. In addition, it should be pointed out that the state attribute in the control state is the name of the state attribute rather than the value of the state attribute.

[0084] State transition includes a first control state and a second control state, which are used to represent the action of the driving control system transitioning from the first control state to the second control state. The first control state and the second control state are both the aforementioned control states. Among them, the first control state corresponds to the starting node of the directed edge, or can also be called the starting control state; the second control state corresponds to the end node of the directed edge, or can also be called the end control state. In state transition, the first control state and the second control state are usually represented by control state identification codes. The transition condition corresponds to the state transition, and is used to represent the conditions that need to be met for the driving control system to transition from the first control state corresponding to the state transition to the second control state corresponding to the state transition, which is a state attribute Boolean expression. It should be pointed out that in the specific implementation of this embodiment, state transition and transition conditions are represented by state transition information. The state transition information includes the first control state, the second control state and the state attribute Boolean expression. The reason why the present invention divides the state transition information into state transition and transition conditions is simply to facilitate subsequent description.

[0085] The state attribute Boolean expression is a simple Boolean expression or a compound Boolean expression composed of several simple Boolean expressions through conjunction and / or disjunction operations. In the state attribute Boolean expression, the simple Boolean expression is a size comparison expression about the state attribute, and the state attribute comes from the first control state of the corresponding state transition. The size comparison expression here can be expressed as F(op, sp, v). Among them, op is the operator of the size comparison expression, sp is the state attribute, and v is the comparison value of the state attribute. Specifically in the actual expression, the size comparison expression F(op, sp, v) is equivalent to sp op v, or equivalent to v op sp. The operator op of the size comparison expression includes 6 types: equal, not equal, greater than, greater than or equal to, less than, and less than or equal to. Corresponding to the specific size comparison expression F(op, sp, v) is divided into 6 situations:

[0086] sp == v;

[0087] sp!=v;

[0088] sp>v;

[0089] sp>=v;

[0090] sp <v;

[0091] sp<=v.

[0092] Compound Boolean expressions can be expressed as:

[0093] Conjunctive normal form, b_1∧b_2∧...∧b_n, or

[0094] Disjunctive normal form, b_1∨b_2∨...∨b_n.

[0095] Among them, b_1, b_2, ..., b_n are sub-terms or sub-expressions of the compound Boolean expression; the sub-terms or sub-expressions of the compound Boolean expression can be simple Boolean expressions, or a compound Boolean expression composed of simple Boolean expressions.

[0096] In addition, it should be noted that, in the specific implementation of this embodiment, the control state is usually represented only by the state identification code, and the state attributes of the control state are actually represented by the state transition information. Specifically, for a certain control state, the state attributes come from the state attribute Boolean expression of each state transition information of the control state as the first control state, and the state attributes involved in the state attribute Boolean expression of each state transition information constitute the state attribute set of the control state. Those skilled in the art will understand that the difference between the specific implementation of this embodiment and the aforementioned invention lies only in the different representation of the data structure, and the essence is the same.

[0097] In addition, it should be pointed out that the aforementioned state control diagram or state control diagram information corresponds to a migration system model based on scenario classification. The model is obtained by code analysis, code abstraction and modeling of the planning and control module of the intelligent driving system based on scenario classification. Specifically in this embodiment, it is obtained by code abstraction and modeling of the planning and control module of Baidu Apollo 6.0. The "obtaining state control diagram information" in the aforementioned step S1 indicates that the state control diagram information is the input of the scenario testing method of the driving control system of the present invention. How to obtain it or the method used to obtain it and how to model it are not within the scope of the present invention and need not be elaborated.

[0098] In addition, the aforementioned instrumentation of the driving control system program is based on the migration system model, so the state information output by the aforementioned instrumentation program instructions corresponds to the state control diagram information. The state information includes the control state and the value of the state attribute. That is, the instrumentation program instructions are used to output the current control state of the driving control system and the value of the corresponding state attribute. In addition, the value of the state attribute here needs to correspond to the name of the state attribute. Therefore, in the specific implementation of this embodiment, the value of the state attribute is represented by a tuple of the state attribute and the value of the state attribute. In addition, there are multiple state attributes corresponding to the control state. Therefore, there are also multiple values ​​of the state attributes in the state information. In addition, for the convenience of representation, the control state in the state information is represented by a control state identification code.

[0099] Reference Figure 1 The scenario testing method for a driving control system of the present invention includes an initialization step, a scenario simulation test step, a migration coverage analysis step, a test migration collection step, a mutation seed selection step, and a seed scenario mutation step. The scenario simulation test step, migration coverage analysis step, test migration collection step, mutation seed selection step, and seed scenario mutation step form an iterative process. The termination condition of this iterative process depends on the test coverage analysis of state transitions performed in the migration coverage analysis step.

[0100] The initialization step, also known as step S2, constructs an initial set of scenario samples by randomly generating scenario sample information. The second set of state transitions is initialized using the set of state transitions in the state control diagram information. Simply put, the scenario sample set and the second set of state transitions are initialized. The scenario sample set is a collection of scenario samples. A scenario sample represents a scenario sample. The scenario sample information is the scenario parameters input from the test control system 800 to the simulation system 900. It is equivalent to a test case and is a collection of scenario parameters, including, for example, road layout data, vehicle starting and ending points, traffic light locations and their changing sequences, and the road and surrounding NPCs. The scenario parameters included in the scenario sample information depend on the desired driving environment. The second set of state transitions is used to store untested state transitions. During initialization, all state transitions in the state control diagram information are untested. Therefore, the second set of state transitions is a copy of the set of state transitions in the state control diagram information. In this embodiment, to facilitate subsequent variation of scenario samples, the scenario samples in the scenario sample are divided into current scenario samples and seed scenario samples. Specifically, the scene sample includes current scene sample information, seed scene sample information and mutation information. The current scene sample information and seed scene sample information correspond to the current scene sample and seed scene sample respectively, and both are scene sample information. In the initial set of scene samples, the current scene sample information is randomly generated scene sample information, and the seed scene sample information and mutation information are empty. In subsequent processing, the current scene sample information in the scene sample is obtained after mutation based on the seed scene sample information in the scene sample. The mutation information saves the information of the seed scene sample mutation to the current scene sample, including state transition, the distance between the sample output and the state transition, the probability corresponding to each preset mutation method, and the preset mutation method selected for mutation. In addition, the randomly generated scene sample information here specifically randomly generates a specific scene parameter value for each scene parameter of the scene sample information.

[0101] The scenario simulation test step, that is, the aforementioned step S3, uses the current scenario sample information of the scenario sample in the scenario sample set as the simulation data for simulating the real scenario, and performs simulation testing on the driving control system program through the simulation system. During the simulation test, the log output of the driving control system program and the output of its plug-in program instructions are collected to form a sample output sequence corresponding to the scenario sample. The sample output sequence is a sequence composed of sample outputs. The sample output includes the control state and the value of its corresponding state attribute. In other words, the control state output by the aforementioned plug-in program instruction and the value of its corresponding state attribute are the sample outputs referred to in the present invention.

[0102] This step corresponds to the aforementioned test control system 800 inputting the current scene sample information as scene parameters into the simulation system 900, the simulation system 900 simulating a simulated driving environment for the driving control system 910 based on the scene parameters, and the driving control system 910 regulating the vehicle driving according to the driving environment simulated by the simulation system 900. How the simulation system 900 simulates the corresponding simulated driving environment based on the scene parameters, and how the driving control system 910 regulates the vehicle driving according to the driving environment are not the scope of discussion of the present invention and need not be elaborated. In addition, it should be pointed out that the sample output sequence here corresponds to the scene sample information, and there are multiple scene sample information for simulation testing in this step, and correspondingly, there are also multiple sample output sequences, that is, this step actually outputs a set of sample output sequences.

[0103] The migration coverage analysis step, that is, the aforementioned step S4, extracts the control state transformation in the simulation result output information sequence, and removes the control state transformation as a state transition from the second state transition set; then, based on whether the second state transition set is empty as the test end condition, determines whether to end the test. Here, the control state transformation is to change from one control state to another control state, which is equivalent to the aforementioned state transition. Here, the second state transition set is empty, indicating that all state transitions have been tested. In the specific implementation of this embodiment, there are two test end conditions for determining whether to end the test: the first is whether the second state transition set is empty, and the second is whether the number of loop iterations exceeds a preset threshold. Specifically, if the second state transition set is empty or the number of loop iterations exceeds a preset threshold, the test process is terminated. In addition, the above-mentioned test end condition implies that one of the test goals of the scenario test method of the driving control system of the present invention is to complete coverage testing of all state transitions.

[0104] The step of collecting the transitions to be tested, that is, the aforementioned step S5, traverses the sample outputs in the sample output sequence. If the control state of the traversed sample output is the first control state of a state transition in the second state transition set, the scenario sample corresponding to the sample output sequence, the sample output, and the corresponding state transition are combined into the scenario sample state expected transition information, thereby obtaining the scenario sample state expected transition information set. The scenario sample state expected transition information includes the scenario sample, the sample output, and the state transition. Simply put, this step determines the state transition of the target to be tested for the next round of iteration.

[0105] Assume that the control states in the state control diagram are: c1, c2, c3, c4, c5, c6, and the corresponding state transitions are as follows: c1->c2, c1->c3, c1->c4, c2->c3, c2->c5, c2->c6, c3->c4, c3->c6, c4->c5, c4->c6, c5->c6. Assume that there are four sample output sequences output in the aforementioned step S3, namely ss1, ss2, ss3, and ss4. Among them, the sample outputs in the sample output sequence ss1 are ss1_c1 and ss1_c2, the sample outputs in the sample output sequence ss2 are ss2_c1 and ss2_c3, the sample outputs in the sample output sequence ss3 are ss3_c1 and ss3_c4, and the sample outputs in the sample output sequence ss4 are ss4_c1, ss4_c2, and ss4_c3. Among them, sample outputs ss1_c1, ss2_c1, ss3_c1, and ss4_c1 all correspond to control state c1, sample outputs ss1_c2 and ss4_c2 correspond to control state c2, sample output ss2_c3 corresponds to control state c3, and sample output ss3_c4 corresponds to control state c4. In other words, the sample output sequence ss1 contains control states c1 and c2, and the corresponding control state transitions are c1->c2; the sample output sequence ss2 contains control states c1 and c3, and the corresponding control state transitions are c1->c3; the sample output sequence ss3 contains control states c1 and c4, and the corresponding control state transitions are c1->c4; the sample output sequence ss4 contains control states c1, c2, and c3, and the corresponding control state transitions are c1->c2 and c2->c3. That is, the state transitions c1->c2, c1->c3, c2->c3, and c1->c4 have been tested. After processing in step S4, the remaining state transitions in the second state transition set are: c2->c5, c2->c6, c3->c4, c3->c6, c4->c5, c4->c6, and c5->c6.

[0106] Therefore, in this step, the traversal of the sample outputs in the sample output sequence ss1 yields the corresponding control states c1 and c2. The state transitions in the second state transition set where the first control state is c1 or c2 are c2->c5, c2->c6. The sample output corresponding to the first control state c2 in the state transitions c2->c5, c2->c6 is ss1_c2. Thus, two expected state transition information for the scenario samples can be formed: espt_1 and espt_2.

[0107] espt_1={scenario example corresponding to sample output sequence ss1, sample output ss1_c2, state transition c2->c5},

[0108] espt_2={scene example corresponding to sample output sequence ss1, sample output ss1_c2, state transition c2->c6}.

[0109] Traversing the sample outputs in the sample output sequence ss2 yields the corresponding control states c1 and c3. The state transitions in the second state transition set where the first control state is c1 or c3 are c3->c4, c3->c6. The sample output corresponding to the first control state c3 in the state transitions c3->c4, c3->c6 is ss2_c3. Thus, two expected transition information for the scenario sample states can be formed: espt_3 and espt_4. Among them,

[0110] espt_3={scenario example corresponding to sample output sequence ss2, sample output ss2_c3, state transition c3->c4},

[0111] espt_4={scenario example corresponding to sample output sequence ss2, sample output ss2_c3, state transition c3->c6}.

[0112] Traversing the sample outputs in the sample output sequence ss3 yields the corresponding control states c1 and c4. In the second state transition set, the state transitions whose first control state is c1 or c4 are c4->c5, c4->c6. The sample output corresponding to the first control state c4 in the state transitions c4->c5, c4->c6 is ss3_c4. Thus, two expected transition information for the scenario sample states can be formed: espt_5 and espt_6.

[0113] espt_5={scenario example corresponding to sample output sequence ss3, sample output ss3_c4, state transition c4->c5},

[0114] espt_6={scenario example corresponding to sample output sequence ss3, sample output ss3_c4, state transition c4->c6}.

[0115] Traversing the sample outputs in the sample output sequence ss4, we can obtain the corresponding control states c1, c2, and c3. In the second state transition set, the state transitions with the first control state of c1, c2, or c3 are c2->c5, c2->c6, and c3->c4, c3->c6. Among them, the sample output corresponding to the first control state c2 in the state transitions c2->c5, c2->c6 is ss4_c2; the sample output corresponding to the first control state c3 in the state transitions c3->c4, c3->c6 is ss4_c3. The four expected transition information of the scene sample states that can be formed are: espt_7, espt8, espt9, and espt_10. Among them,

[0116] espt_7={scenario example corresponding to sample output sequence ss4, sample output ss4_c2, state transition c2->c5},

[0117] espt_8={scenario example corresponding to sample output sequence ss4, sample output ss4_c2, state transition c2->c6},

[0118] espt_9={scenario example corresponding to sample output sequence ss4, sample output ss4_c3, state transition c3->c4},

[0119] espt_10={Scene example corresponding to sample output sequence ss4, sample output ss4_c3, state transition c3->c6}.

[0120] The expected state transition information set for the scenario example obtained in step S5 is: {espt_1, espt_2, espt_3, espt_4, espt_5, espt_6, espt_7, espt_8, espt_9, espt_10}. This expected state transition information set for the scenario example includes the state transitions of the target to be tested in the next iteration as c2->c5, c2->c6, c3->c4, c3->c6, c4->c5, and c4->c6.

[0121] The mutation seed selection step, namely, the aforementioned step S6, calculates the distance between the sample output and the state transition of each scenario example expected state transition information in the scenario example expected state transition information set. The scenario example with the smallest distance between the sample output and the state transition corresponding to the state transition is extracted from the scenario example expected state transition information set to form the seed corresponding to the state transition, thereby forming a seed set. According to the example in the aforementioned step S5, the number of state transitions included in the scenario example expected state transition information set outputted in step S5 differs from the number of elements in the scenario example expected state transition information set. That is, in the state transitions of the target to be tested in the next iteration outputted in step S5, each state transition may correspond to multiple scenario example expected state transition information. For example, according to the example in the aforementioned step S5, the scenario example expected state transition information corresponding to the state transition c2->c5 is espt_1 and espt_7. Simply put, the purpose of this step is to select one of the multiple scenario example expected state transition information corresponding to a particular state transition in the scenario example expected state transition information set as the mutation seed. For example, one of the two scenario samples corresponding to the state expected transition information espt_1 and espt_7 of the aforementioned state transition c2->c5 is selected as the seed. The seed may be the scenario sample corresponding to the sample output sequence ss1, or the scenario sample corresponding to the sample output sequence ss4. The criterion for selecting the seed is based on the distance between the sample output and the state transition, which corresponds to the scenario sample state expected transition information espt_1, that is, the distance between the sample output ss1_c2 and the state transition c2->c5, and corresponds to the scenario sample state expected transition information espt_7, the distance between the sample output ss4_c2 and the state transition c2->c5. The distance between the sample output and the state transition is the distance between the state attribute Boolean expression of the transition condition corresponding to the state transition and the sample output. The distance between the state attribute Boolean expression and the sample output is calculated as follows:

[0122] For simple Boolean expressions, calculate the distance between the simple Boolean expression and the sample output;

[0123] For conjunction operation, calculate the distance of each sub-item and calculate the sum of the sub-item distances;

[0124] For the disjunction operation, calculate the distance of each sub-item and take the minimum value of the sub-item distance;

[0125] The child distance is the distance between the child state attribute Boolean expression of the state attribute Boolean expression and the sample output.

[0126] According to the above calculation method, for more complex state attribute Boolean expressions, their distance from the sample output can be obtained by recursive calculation. This is because the sub-state attribute Boolean expression of the state attribute Boolean expression is also a state attribute Boolean expression. In addition, according to the above, the simple Boolean expression that constitutes the state attribute Boolean expression is a size comparison expression about the state attribute, which can be expressed as F(op, sp, v). Therefore, calculating the distance between the simple Boolean expression and the sample output is to calculate the distance between the size comparison expression F(op, sp, v) about the state attribute and the sample output. The distance between F(op, sp, v) and the sample output is calculated as follows:

[0127] If op is equal to or not equal to, then D(F(op,sp,v))=K; otherwise D(F(op,sp,v))=K+abs(v-out(sp));

[0128] Where D(F(op, sp, v)) is the distance between the size comparison expression and the sample output, K is a preset constant, abs is the absolute value function, and out(sp) is the value of the state attribute sp in the sample output.

[0129] In addition, in this embodiment, the seed includes state transitions, current scene sample information, original seed scene sample information, original seed variation information, and the distance between the sample output and the state transition. The seed's state transitions are derived from the state transitions in the scene sample's expected state transition information, while the seed's current scene sample information, original seed scene sample information, and original seed variation information are derived from the current scene sample information, seed scene sample information, and variation information of the scene sample in the scene sample's expected state transition information, respectively. The distance between the sample output and the state transition is the distance between the sample output and the state transition in the scene sample's expected state transition information.

[0130] The seed scene variation step, that is, the aforementioned step S7, performs variation processing on the current scene sample information of each seed in the seed set to obtain new current scene sample information, forming a scene sample set, and then returns to step S3.

[0131] There are many ways to mutate the current scene sample information in the seed. The simplest implementation involves randomly selecting a scene parameter in the current scene sample information and randomly generating a specific value for that scene parameter. Another simple implementation involves randomly mutating a specific project, such as randomly changing the starting and ending points of vehicles, randomly changing traffic light sequences, randomly changing NPC vehicles, or adjusting the interference intensity of NPCs around the controlled vehicle.

[0132] Another simple implementation method is to pre-set a number of mutation methods and then randomly select one of these pre-set mutation methods for mutation. Pre-set mutation methods can include, for example, randomly changing the starting and ending points of vehicles, randomly changing the signal light sequence, randomly changing NPC vehicles, or adjusting the interference intensity of NPCs around the controlled vehicle. Of course, a corresponding probability can also be preset for each pre-set mutation method, thereby constructing a mutation method selection probability table. When randomly selecting a pre-set mutation method, the selection can be made based on this mutation method selection probability table.

[0133] Obviously, the overall testing efficiency is low under the three aforementioned implementations. Furthermore, under these simple implementations, the variation of the current scene sample information does not utilize the original seed scene sample information, the original seed variation information, and the distance between the sample output and the state transition contained in the seed. Therefore, this information is not necessary under these simple implementations. Similarly, the aforementioned scene examples only require the current scene sample information; the seed scene sample information and variation information in the scene examples can be omitted.

[0134] In order to improve the test efficiency and enable the new current scene sample information generated by the mutation to achieve the purpose of corresponding state transition test coverage, this embodiment adopts the following steps to mutate the current scene sample information of the seed:

[0135] Step S71: First, determine whether the original seed scene sample information of the seed is empty and whether the state transition is the same as the state transition in the original seed variation information; if the original seed scene sample information is empty or the state transition of the seed is different from the state transition in the original seed variation information, then go to step S72; otherwise, go to step S73;

[0136] Step S72: Initializing the mutation mode selection probability table so that the selection probabilities of the preset mutation modes in the mutation mode selection probability table are the same, and then randomly selecting one of the preset mutation modes from the preset mutation modes according to the mutation mode selection probability table for mutation;

[0137] Step S73: extracting the probabilities corresponding to the preset mutation modes in the original seed mutation information of the seed to form a mutation mode selection probability table, and extracting the preset mutation mode selected for mutation in the original seed mutation information of the seed as the original mutation mode;

[0138] Step S74: Adjust the probability corresponding to the original mutation method in the mutation method selection probability table based on the comparison between the distance between the sample output of the seed and the state transition and the distance between the sample output and the state transition in the original seed mutation information: if the distance between the sample output of the seed and the state transition is larger, reduce the probability corresponding to the original mutation method; otherwise, increase the probability corresponding to the mutation method.

[0139] Step S75: randomly selecting a preset mutation method from the preset mutation method list according to the adjusted mutation method selection probability table for mutation.

[0140] Simply put, steps S71 to S75 above are an optimization of the second simple implementation method described above. That is, in the specific implementation of this embodiment, the mutation of the current scene sample information of the seed is also performed by randomly selecting a preset mutation method from the list of preset mutation methods according to the mutation method selection probability table. However, in the specific implementation of this embodiment, the mutation method selection probability is iteratively optimized and adjusted.

[0141] Furthermore, the preset variation modes in this embodiment include randomly changing vehicle starting and ending points, randomly changing signal light sequences, randomly changing NPC vehicles, and adjusting the interference intensity of NPCs surrounding the controlled vehicle. Those skilled in the art will appreciate that the preset variation modes can also be based on the scene parameters themselves. For example, randomly increasing the scene parameter value is one preset variation mode, while randomly decreasing the scene parameter value is another preset variation mode. Thus, each of the N scene parameters in the scene sample information corresponds to 2N preset variation modes.

[0142] In addition, in the scene samples generated by mutation in this embodiment, the current scene sample information is the new current scene sample information generated by mutation, the seed scene sample information is the current scene sample information corresponding to the seed, and the mutation information is the mutation parameter information recorded when the new current scene sample information is generated by mutation of the current scene sample information of the seed.

[0143] Furthermore, it should be noted that in this embodiment, by manually instrumenting the original driving control system program and inserting instrumentation program instructions, the driving control system is enabled to output the current control state and the corresponding state attribute values. If the original driving control system program is capable of outputting the current control state and the corresponding state attribute values ​​after execution, this situation can be considered as the original driving control system program being instrumented with instrumentation program instructions as referred to in the present invention.

[0144] In addition, in this embodiment, in order to facilitate the subsequent regression test, all the scene sample information that has completed the test is retained and output.

[0145] In addition, it should be noted that the aforementioned modules referred to in the present invention are virtual devices corresponding to the steps in the method, and will not be described in detail.

Claims

1. A scenario testing method for a driving control system, the method involving a simulation system loaded with a driving control system program; characterized in that: The driving control system program plug-in includes plug-in program instructions; the plug-in program instructions are used to output the current control state of the driving control system and the value of the corresponding state attribute; the method includes the following steps: Step S1: Obtain state control diagram information; The state control diagram information includes a set of control states, a set of state transitions, and a set of transition conditions; The control state is used to represent the control state of the driving control system, including a set of state attributes; The state transition includes a first control state and a second control state, which is used to indicate the action of the driving control system transitioning from the first control state to the second control state; The transition condition corresponds to the state transition, and is used to indicate the condition that needs to be satisfied for the driving control system to transition from the first control state corresponding to the state transition to the second control state corresponding to the state transition, and is a state attribute Boolean expression; The state attribute Boolean expression is a simple Boolean expression or a compound Boolean expression formed by combining several simple Boolean expressions through conjunction and / or disjunction operations; In the state attribute Boolean expression, the simple Boolean expression is a comparison expression about the state attribute, and the state attribute comes from the first control state of the corresponding state transition; Step S2: constructing an initial scene sample set by randomly generating scene sample information, and initializing a second state transition set with the state transition set in the state control diagram information; The scene sample includes current scene sample information; the current scene sample information corresponds to the current scene sample and is scene sample information; in the initial set of scene samples, the current scene sample information is randomly generated scene sample information; Step S3: Using the current scenario sample information of the scenario sample in the scenario sample set as simulation data for simulating a real scenario, the driving control system program is simulated and tested through a simulation system. During the simulation test, the log output of the driving control system program and the output of its instrumented program instructions are collected to form a sample output sequence corresponding to the scenario sample; the sample output sequence is a sequence composed of sample outputs; the sample output includes a control state and a value of its corresponding state attribute; Step S4: extracting a control state transition from the sample output sequence and removing the control state transition as a state transition from the second state transition set; then determining whether to terminate the test based on whether the second state transition set is empty as a test termination condition; Step S5: Traversing the sample outputs in the sample output sequence, if the control state of the traversed sample output is the first control state of a state transition in the second state transition set, then combining the scene sample corresponding to the sample output sequence, the sample output, and the corresponding state transition into scene sample state expected transition information, thereby obtaining a scene sample state expected transition information set; The expected state transition information of the scenario sample includes the scenario sample, sample output and state migration; the sample output is the simulation result output information; Step S6: Calculate the distance between the sample output of each scene sample expected state transition information and the state transition in the scene sample expected state transition information set, and extract the scene sample with the smallest distance between the sample output corresponding to the state transition and the state transition from the scene sample expected state transition information set to form a seed corresponding to the state transition, thereby forming a seed set; the seed includes the state transition and the current scene sample information; The distance between the sample output and the state transition is the distance between the state attribute Boolean expression of the transition condition corresponding to the state transition and the sample output; Step S7: performing mutation processing on the current scene sample information of each seed in the seed set to obtain new current scene sample information, forming a scene sample set, and then returning to step S3; In step S6, the distance between the state attribute Boolean expression and the sample output is calculated as follows: For simple Boolean expressions, calculate the distance between the simple Boolean expression and the sample output; For conjunction operation, calculate the distance of each sub-item and calculate the sum of the sub-item distances; For the disjunction operation, calculate the distance of each sub-item and take the minimum value of the sub-item distance; The sub-item distance is the distance between the sub-state attribute Boolean expression of the state attribute Boolean expression and the sample output; The distance between the simple Boolean expression and the sample output is calculated by calculating the distance between the size comparison expression of the state attribute and the sample output. The distance between the size comparison expression F(op, sp, v) of the state attribute and the sample output is calculated using the following formula: If op is equal to or not equal to, then D(F(op,sp,v))=K; otherwise D(F(op,sp,v))=K+abs(v-out(sp)); Where op is the operator of the size comparison expression, sp is the state attribute, v is the comparison value of the state attribute, F(op, sp, v) is expressed as the size comparison expression: sp op v or v op sp, D(F(op, sp, v)) is the distance between the size comparison expression and the sample output, K is a preset constant, abs is the absolute value function, and out(sp) is the value of the state attribute sp in the sample output.

2. The scenario testing method for a driving control system according to claim 1, characterized in that: In step S7, a preset mutation mode is randomly selected from the preset mutation mode list according to the mutation mode selection probability table for mutation.

3. The scenario testing method for a driving control system according to claim 2, characterized in that: The scene sample also includes seed scene sample information and variation information; the seed scene sample information corresponds to the seed scene sample and is scene sample information; the variation information is information about the mutation of the seed scene sample to the current scene sample, including state transition, the distance between the sample output and the state transition, the probability corresponding to each preset variation method, and the preset variation method selected for the mutation; In step S2, in the initial set of scene samples, both the seed scene sample information and the variation information are empty; The seed also includes the distance between the sample output and the state transition, the original seed scene sample information and the original seed variation information; In step S6, when forming the seed, the seed scene sample information and variation information in the scene example are used as the original seed scene sample information and original seed variation information of the seed respectively; In step S7, when mutating the current scene sample information of the seed, first determine whether the original seed scene sample information of the seed is empty and whether the state transition is the same as the state transition in the original seed mutation information; if the original seed scene sample of the seed is empty or the state transition is different from the state transition in the original seed mutation information, initialize the mutation mode selection probability table so that the selection probabilities of each preset mutation mode in the mutation mode selection probability table are the same; otherwise, extract the probabilities corresponding to each preset mutation mode in the original seed mutation information of the seed to form a mutation mode selection probability table, and extract the preset mutation mode selected for mutation in the original seed mutation information of the seed as the original mutation mode, and then adjust the probability corresponding to the original mutation mode in the mutation mode selection probability table according to the comparison between the distance between the sample output of the seed and the state transition and the distance between the sample output and the state transition in the original seed mutation information, and finally randomly select a preset mutation mode from the preset mutation mode list according to the adjusted mutation mode selection probability table for mutation.

4. The scenario testing method for a driving control system according to claim 2, characterized in that: The preset variation methods include: randomly changing the starting and ending points of vehicles, randomly changing the signal light sequence, randomly changing NPC vehicles, and adjusting the interference intensity of NPCs around the controlled vehicle.

5. A scenario testing device for a driving control system, which relates to a simulation system loaded with a driving control system program; characterized in that: The driving control system program plug-in includes plug-in program instructions; the plug-in program instructions are used to output the current control state of the driving control system and the value of the corresponding state attribute; the device includes the following modules: Module M1 is used to: obtain state control diagram information; The state control diagram information includes a set of control states, a set of state transitions, and a set of transition conditions; The control state is used to represent the control state of the driving control system, including a set of state attributes; The state transition includes a first control state and a second control state, which is used to indicate the action of the driving control system transitioning from the first control state to the second control state; The transition condition corresponds to the state transition, and is used to indicate the condition that needs to be satisfied for the driving control system to transition from the first control state corresponding to the state transition to the second control state corresponding to the state transition, and is a state attribute Boolean expression; The state attribute Boolean expression is a simple Boolean expression or a compound Boolean expression formed by combining several simple Boolean expressions through conjunction and / or disjunction operations; In the state attribute Boolean expression, the simple Boolean expression is a comparison expression about the state attribute, and the state attribute comes from the first control state of the corresponding state transition; Module M2 is used to: construct an initial scene sample set by randomly generating scene sample information, and initialize a second state transition set with the state transition set in the state control diagram information; The scene sample includes current scene sample information; the current scene sample information corresponds to the current scene sample and is scene sample information; in the initial set of scene samples, the current scene sample information is randomly generated scene sample information; Module M3 is configured to: use the current scenario sample information of the scenario sample in the scenario sample set as simulation data for simulating a real scenario, perform simulation testing on the driving control system program through a simulation system, and during the simulation testing, collect the log output of the driving control system program and the output of its instrumented program instructions to form a sample output sequence corresponding to the scenario sample; the sample output sequence is a sequence composed of sample outputs; the sample outputs include a control state and the value of its corresponding state attribute; Module M4 is configured to extract a control state transition from the sample output sequence and remove the control state transition as a state transition from the second state transition set; and then determine whether to terminate the test based on whether the second state transition set is empty as a test termination condition; Module M5 is configured to: traverse the sample outputs in the sample output sequence, and if the control state of the traversed sample output is the first control state of a state transition in the second state transition set, combine the scenario sample corresponding to the sample output sequence, the sample output, and the corresponding state transition into scenario sample state expected transition information, thereby obtaining a scenario sample state expected transition information set; The expected state transition information of the scenario sample includes the scenario sample, sample output and state migration; the sample output is the simulation result output information; Module M6 is configured to calculate the distance between the sample output of each scene sample expected state transition information and the state transition in the scene sample expected state transition information set, and extract the scene sample with the smallest distance between the sample output corresponding to the state transition and the state transition from the scene sample expected state transition information set to form a seed corresponding to the state transition, thereby forming a seed set; the seed includes the state transition and the current scene sample information; The distance between the sample output and the state transition is the distance between the state attribute Boolean expression of the transition condition corresponding to the state transition and the sample output; Module M7 is used to perform mutation processing on the current scene sample information of each seed in the seed set to obtain new current scene sample information, form a scene sample set, and then return to module M3; In the module M6, the distance between the state attribute Boolean expression and the sample output is calculated as follows: For simple Boolean expressions, calculate the distance between the simple Boolean expression and the sample output; For conjunction operation, calculate the distance of each sub-item and calculate the sum of the sub-item distances; For the disjunction operation, calculate the distance of each sub-item and take the minimum value of the sub-item distance; The sub-item distance is the distance between the sub-state attribute Boolean expression of the state attribute Boolean expression and the sample output; The distance between the simple Boolean expression and the sample output is calculated by calculating the distance between the size comparison expression of the state attribute and the sample output. The distance between the size comparison expression F(op, sp, v) of the state attribute and the sample output is calculated using the following formula: If op is equal to or not equal to, then D(F(op,sp,v))=K; otherwise D(F(op,sp,v))=K+abs(v-out(sp)); Where op is the operator of the size comparison expression, sp is the state attribute, v is the comparison value of the state attribute, F(op, sp, v) is expressed as the size comparison expression: sp op v or v op sp, D(F(op, sp, v)) is the distance between the size comparison expression and the sample output, K is a preset constant, abs is the absolute value function, and out(sp) is the value of the state attribute sp in the sample output.

6. The scenario testing device for a driving control system according to claim 5, characterized in that: In the module M7, a preset mutation mode is randomly selected from the preset mutation mode list according to the mutation mode selection probability table for mutation.

7. The scenario testing device for a driving control system according to claim 6, characterized in that: The scene sample also includes seed scene sample information and variation information; the seed scene sample information corresponds to the seed scene sample and is scene sample information; the variation information is information about the mutation of the seed scene sample to the current scene sample, including state transition, the distance between the sample output and the state transition, the probability corresponding to each preset variation method, and the preset variation method selected for the mutation; In the module M2, in the initial set of scene samples, both the seed scene sample information and the variation information are empty; The seed also includes the distance between the sample output and the state transition, the original seed scene sample information and the original seed variation information; In the module M6, when forming the seed, the seed scene sample information and variation information in the scene example are used as the original seed scene sample information and original seed variation information of the seed respectively; In the module M7, when mutating the current scene sample information of the seed, first determine whether the original seed scene sample information of the seed is empty and whether the state transition is the same as the state transition in the original seed mutation information; if the original seed scene sample of the seed is empty or the state transition is different from the state transition in the original seed mutation information, initialize the mutation mode selection probability table so that the selection probabilities of each preset mutation mode in the mutation mode selection probability table are the same; otherwise, extract the probabilities corresponding to each preset mutation mode in the original seed mutation information of the seed to form a mutation mode selection probability table, and extract the preset mutation mode selected for mutation in the original seed mutation information of the seed as the original mutation mode, and then adjust the probability corresponding to the original mutation mode in the mutation mode selection probability table according to the comparison between the distance between the sample output of the seed and the state transition and the distance between the sample output and the state transition in the original seed mutation information, and finally randomly select a preset mutation mode from the preset mutation mode list according to the adjusted mutation mode selection probability table for mutation.

8. The scenario testing device for a driving control system according to claim 6, characterized in that: The preset variation methods include: randomly changing the starting and ending points of vehicles, randomly changing the signal light sequence, randomly changing NPC vehicles, and adjusting the interference intensity of NPCs around the controlled vehicle.

9. A medium storing a program instruction set readable by a machine; characterized in that: When the program instruction set stored in the medium is read and executed by a machine, the scenario testing method for the driving control system according to any one of claims 1 to 4 can be implemented.

10. A device comprising a processor and a memory; wherein the memory stores a program instruction set; characterized in that: When the program instruction set stored in the memory is loaded and executed by the processor, the scenario testing method for the driving control system according to any one of claims 1 to 4 can be implemented.

Images