A test and evaluation method and device for intelligent driving function

Abstract

The present application provides a test and evaluation method and device for intelligent driving functions, the method comprising: obtaining a logical state quantity of a test case during a test process, the logical state quantity being able to characterize a finite state machine state in a continuous time; extracting a state sequence of the logical state quantity, the state sequence being able to characterize a finite state machine state in a continuous time period, one time period corresponding to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods being different; determining a preceding finite state machine state represented by the state sequence that is one time period earlier than a preset critical finite state machine state, and obtaining a transfer condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and a variable parameter corresponding to the transfer condition; and processing the variable parameter based on the transfer condition to obtain a test result of the test case.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving test technology, and more specifically, to a test and evaluation method and device for an intelligent driving function implemented based on a finite state machine concept. Background Art

[0002] The finite state machine concept decomposes complex logic into a finite number of stable states, and then performs corresponding operations on each state. This discrete processing method is beneficial to engineering implementation, so it is widely used in the development of intelligent driving functions at conditional autonomous driving (CA) and below.

[0003] Therefore, how to test the intelligent driving functions implemented based on the finite state machine concept is an urgent problem that needs to be solved at this stage. Summary of the invention

[0004] In view of this, in order to solve the above problems, the present invention provides a test and evaluation method and device for intelligent driving function, and the technical solution is as follows:

[0005] A test and evaluation method for an intelligent driving function, the method comprising:

[0006] Acquire the logical state quantity of the test case during the test process, wherein the logical state quantity can represent the state of the finite state machine in continuous time;

[0007] Extracting the state sequence of the logic state quantity, wherein the state sequence can represent the finite state machine state in continuous time periods, one time period corresponds to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods are different;

[0008] Determine a preceding finite state machine state represented by the state sequence that is one period earlier than a preset critical finite state machine state, and obtain a transition condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and a variable parameter corresponding to the transition condition;

[0009] The variable parameters are processed based on the transfer conditions to obtain a test result of the test case.

[0010] Preferably, the method further comprises:

[0011] Converting the transfer condition into a corresponding judgment sub-function, wherein the judgment sub-function is used to judge whether the function input meets the transfer condition;

[0012] The variable parameters corresponding to the transfer condition include the first variable parameters when the test case is transferred from the pre-finite state machine state to the key finite state machine state;

[0013] The step of processing the variable parameter based on the transfer condition to obtain the test result of the test case includes:

[0014] Inputting the first variable parameter into the judgment sub-function, and obtaining the first flag bit output by the judgment sub-function;

[0015] If the first flag bit indicates that the first variable parameter meets the transfer condition, determining that the test result of the test case is passed;

[0016] If the first flag bit indicates that the first variable parameter does not meet the transfer condition, the test result of the test case is determined to be failed.

[0017] Preferably, the variable parameters corresponding to the transfer condition also include a second variable parameter of the test case in the time period corresponding to the state of the preceding finite state machine;

[0018] The step of processing the variable parameters based on the transfer condition to obtain the test result of the test case further includes:

[0019] If the test result of the test case determined based on the first flag is passed, inputting the second variable parameter into the judgment sub-function, and obtaining the second flag output by the judgment sub-function;

[0020] If the second flag bit indicates that the second variable parameter meets the transfer condition, determining that the test result of the test case is failed;

[0021] If the second flag bit indicates that the second variable parameter does not meet the transfer condition, the test result of the test case is determined to be passed.

[0022] Preferably, the method further comprises:

[0023] Obtaining the number of occurrences of the key finite state machine state in the state sequence;

[0024] Determine whether the number of occurrences of the key finite state machine state is equal to a preset number threshold;

[0025] If yes, the step of obtaining the transition condition of the test case from the pre-finite state machine state to the key finite state machine state, and the variable parameters corresponding to the transition condition is executed;

[0026] If not, it is determined that the test result of the test case is failed.

[0027] Preferably, the method further comprises:

[0028] If the test result of the test case obtained by processing the variable parameter based on the transfer condition is passed, then obtaining a performance state quantity of the test case during the test process, wherein the performance state quantity can represent the performance state in a continuous time;

[0029] For each finite state machine state represented by the state sequence, obtain the value range corresponding to the performance state quantity in the finite state machine state and the actual value of the performance state quantity in the test case in the time period corresponding to the finite state machine state;

[0030] Determining whether the actual value of the performance variable is within the value range corresponding to the performance state quantity;

[0031] If yes, determining that the test result of the test case is passed;

[0032] If not, the test result of the test case is determined to be failed.

[0033] Preferably, the method further comprises:

[0034] If it is determined that the test result of the test case is failed, the corresponding failure reason is obtained and outputted.

[0035] A test and evaluation device for intelligent driving function, the device comprising:

[0036] An acquisition module is used to acquire the logical state quantity of the test case during the test process, wherein the logical state quantity can represent the state of the finite state machine in continuous time;

[0037] An extraction module, used to extract the state sequence of the logic state quantity, wherein the state sequence can represent the finite state machine state in continuous time periods, one time period corresponds to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods are different;

[0038] A test module is used to determine a preceding finite state machine state represented by the state sequence that is one period earlier than a preset critical finite state machine state, and obtain a transfer condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and variable parameters corresponding to the transfer condition; and process the variable parameters based on the transfer condition to obtain a test result of the test case.

[0039] Preferably, the test module is also used for:

[0040] Converting the transfer condition into a corresponding judgment sub-function, wherein the judgment sub-function is used to judge whether the function input meets the transfer condition;

[0041] The variable parameters corresponding to the transfer condition include the first variable parameters when the test case is transferred from the pre-finite state machine state to the key finite state machine state;

[0042] The test module for processing the variable parameters based on the transfer condition to obtain the test result of the test case is specifically used for:

[0043] The first variable parameter is input into the judgment sub-function, and the first flag bit output by the judgment sub-function is obtained; if the first flag bit indicates that the first variable parameter meets the transfer condition, the test result of the test case is determined to be passed; if the first flag bit indicates that the first variable parameter does not meet the transfer condition, the test result of the test case is determined to be failed.

[0044] Preferably, the variable parameters corresponding to the transfer condition also include a second variable parameter of the test case in the time period corresponding to the state of the preceding finite state machine;

[0045] The test module for processing the variable parameters based on the transfer condition to obtain the test result of the test case is further used to:

[0046] If the test result of the test case determined based on the first flag bit is passed, the second variable parameter is input into the judgment sub-function, and the second flag bit output by the judgment sub-function is obtained; if the second flag bit indicates that the second variable parameter meets the transfer condition, the test result of the test case is determined to be failed; if the second flag bit indicates that the second variable parameter does not meet the transfer condition, the test result of the test case is determined to be passed.

[0047] Preferably, the test module is also used for:

[0048] Obtain the number of occurrences of the key finite state machine state in the state sequence; determine whether the number of occurrences of the key finite state machine state is equal to a preset number threshold; if so, execute the step of obtaining the transfer condition of the test case from the preceding finite state machine state to the key finite state machine state, and the variable parameters corresponding to the transfer condition; if not, determine that the test result of the test case is failed.

[0049] Preferably, the test module is also used for:

[0050] If the test result of the test case obtained by processing the variable parameters based on the transfer condition is passed, then the performance state quantity of the test case during the test process is obtained, and the performance state quantity can represent the performance state in continuous time; for each finite state machine state represented by the state sequence, the value range corresponding to the performance state quantity under the finite state machine state and the actual value of the performance state quantity of the test case in the time period corresponding to the finite state machine state are obtained; it is determined whether the actual value of the performance variable is within the value range corresponding to the performance state quantity; if yes, it is determined that the test result of the test case is passed; if not, it is determined that the test result of the test case is failed.

[0051] Preferably, it is characterized in that the test module is also used for:

[0052] If it is determined that the test result of the test case is failed, the corresponding failure reason is obtained and outputted.

[0053] The present invention provides a test evaluation method and device for intelligent driving functions, which can automatically evaluate the test results of test cases and make the test evaluation steps specific and executable, thereby reducing the impact of software requirement changes on test evaluation tasks. At the same time, for tasks that require repeated testing, it will also significantly reduce the manpower and time costs of related work and improve the efficiency of testing work. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0055] Figure 1 A method flow chart of a method for testing and evaluating an intelligent driving function provided by an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of a finite state machine provided for an embodiment of the present invention;

[0057] Figure 3 Examples of logical state quantities provided for embodiments of the present invention;

[0058] Figure 4 A schematic diagram of dividing the logic state quantity provided by an embodiment of the present invention;

[0059] Figure 5 A schematic diagram of the structure of a testing and evaluation device for intelligent driving functions provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] To facilitate understanding of this application, the relevant technologies are introduced here:

[0063] According to the "Energy-saving and New Energy Vehicle Technology Roadmap" released in 2016, the level of automobile intelligence can be divided into driving assistance (DA), partial automation (PA), conditional automation (CA), highly automated driving (HA), and fully automated driving (FA). In recent years, academic research and engineering development work around intelligent driving has been in the ascendant, and mass-produced cars with PA and even CA levels have been launched on the market.

[0064] The finite state machine concept decomposes complex logic into a finite number of stable states, and then performs corresponding operations on each state. This discrete processing method is beneficial to engineering implementation, so it is widely used in the development of intelligent driving functions at CA and below autonomous driving levels.

[0065] Test verification is an essential part of the intelligent driving function development process. For intelligent driving function testing based on the finite state machine concept, it is necessary to focus on whether the jump of each state is consistent with expectations. However, tests for intelligent driving functions, such as software-in-the-loop (SiL), hardware-in-the-loop (HiL), and vehicle-in-the-loop (ViL) tests, often contain a large number of test case sets, and will inevitably undergo multiple rounds of testing as the model code is iterated.

[0066] If the test results of each test case are manually judged one by one, the manpower and time costs consumed are huge, and the naked eye is sometimes unable to distinguish the subtle differences in the test report, that is, it is difficult to ensure that the judgment results are completely accurate. However, if the actual and expected start and end times of each state are compared through a script, the test results can be automatically judged to a certain extent. However, the defect of this judgment method is that if the calibration value in the test case scenario configuration or the model code changes, the expected results set in advance will no longer be valid and need to be readjusted. If the number of test cases involved is large, then the modification work will bring a lot of work. Therefore, it is necessary to develop a method to automatically complete the evaluation of test results for intelligent driving functions based on the idea of ​​finite state machines, reduce manpower and time costs, and improve test efficiency.

[0067] It is a consensus in many industries to use automation to implement some or all of the testing links. For example, the patents "A method for automated testing of battery life based on smart terminals", "A method for automated testing of bank mobile applications", and "Automated testing system for vehicle-mounted multimedia" respectively designed automated testing processes and methods suitable for smart terminals, bank mobile applications, vehicle-mounted multimedia and other fields, but they did not mention the relevant content of test result evaluation. For another example, the patents "An automated testing system and method for communication control devices", "A method and device for automated testing of websites", "A cross-platform automated testing method and system", and "A method and device for automated testing of virtualized platforms" not only proposed automated testing methods or systems in their respective fields, but also involved the steps of judging whether the test case has passed and generating the judgment result according to preset rules or logic. However, they still did not elaborate on the specific implementation methods.

[0068] In the patents related to intelligent driving testing, "An objective mapping test method for intelligent vehicle traffic coordination" designs an objective mapping evaluation model for traffic coordination based on BP neural network, which is similar to an expert system and can reduce the uncertainty caused by inconsistent standards of evaluators to a certain extent. The patent "A performance test method and device for an automatic parking system" discloses a method for testing the performance of an automatic parking system, but its evaluation of the test results only refers to the position of the vehicle relative to the parking space at the end of parking, and does not consider the parking process. "A method for constructing test scenarios for autonomous driving vehicles based on field driving" sets specific evaluation rules for typical scenarios such as side parking, reversing into a garage, right-angle turning, and curve driving, and judges whether the vehicle passes or fails based on the electronic trajectory of the vehicle. The patents "A test method and device for an embedded control algorithm" and "A system and method for evaluating autonomous driving vehicles" propose to constrain the test results in combination with performance index requirements. Among them, "A system and method for evaluating autonomous driving vehicles" quantitatively analyzes the test results from the perspectives of safety, intelligence, comfort, and agility.

[0069] In summary, the patents that have been searched lack the discussion of specific implementation methods in the following two aspects: First, if the evaluation rules change, whether the test cases and result evaluations need to be adjusted, and whether the adjustment will bring new cost investment; second, how to determine whether the test cases pass or fail based on the preset rules and test results.

[0070] To solve the above problems, the present invention realizes the test result evaluation of the intelligent driving function designed based on the finite state machine concept in an automated manner. It has the following two advantages: First, it effectively avoids the deviation caused by manually judging the test case; second, even if the state transition conditions or calibration quantities in the model code change, it is only necessary to update the corresponding judgment subfunction according to the software requirements (assuming that the model code to be tested is consistent with the software requirements), without increasing other workloads. Third, it supports the rapid location of the reasons for the failure of the test, which facilitates problem finding and confirmation.

[0071] The present invention provides a method for testing and evaluating intelligent driving functions. The method flow chart of the method is as follows: Figure 1 As shown, the following steps are included:

[0072] S10, obtaining the logical state quantity of the test case during the test process, where the logical state quantity can represent the state of the finite state machine in continuous time.

[0073] See also Figure 2The finite state machine diagram is shown. The finite state machine states are divided into three states: State_1, State_2 and State_3. The transition condition from State_1 to State_2 is Transition_1, the transition condition from State_2 to State_1 is Transition_2, the transition condition from State_2 to State_3 is Transition_3, and the transition condition from State_3 to State_1 is Transition_4. It should be noted that for different test cases, the specific contents of the finite state machine states and transition conditions are different, which are not limited here.

[0074] In addition, in the embodiment of the present invention, a logical state quantity is used to characterize the change of the finite state machine state over time, and we assume that the data type of the logical state quantity is an enumeration quantity. As shown in Table 1, different finite state machine states are mapped to corresponding digital identifiers, that is, the logical state quantity records a digital sequence that changes over time. Figure 3 This is an example of the logical state quantity in the coordinate system of the time-finite state machine state.

[0075] Enumeration constants The state contained in the logical state quantity The corresponding value of each state Enum_State_1 State_1 0x1 Enum_State_2 State_2 0x2 Enum_State_3 State_3 0x3 …… …… ……

[0076] Table 1

[0077] S20, extracting the state sequence of the logic state quantity, where the state sequence can represent the finite state machine state in continuous time periods, one time period corresponds to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods are different.

[0078] For the convenience of understanding the present invention, continue to refer to Figure 3 The logical state quantity shown, and the state sequence extracted from the logical state quantity is (State_1, State_2, State_3, State_1).

[0079] In the specific implementation process, since the logic state quantity is a digital sequence that changes with time, the position where the digital jump occurs in the logic state quantity is identified and the position is used as the dividing point to divide the logic state quantity, and the digital sequence of the logic state quantity is divided into several subsequences, and each word sequence corresponds to a finite state machine state in the state sequence. Figure 3 The logical state quantity shown in the figure identifies its segmentation point (Check_Point_x, x is 1, 2, 3, etc.), and the segmentation diagram is as follows: Figure 4 shown.

[0080] S30, determining a preceding finite state machine state represented by the state sequence that is one period earlier than a preset critical finite state machine state, and obtaining a transition condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and a variable parameter corresponding to the transition condition.

[0081] In an embodiment of the present invention, for a test case, key finite state machine states are pre-marked, for example, State_2 and State_3 are used as key finite state machine states. Therefore, for (State_1, State_2, State_3, State_1), the predecessor finite state machine state corresponding to the key finite state machine state State_2 is State_1, and the predecessor finite state machine state corresponding to the key finite state machine state State_3 is State_2.

[0082] Taking the finite state machine state State_2 and the preceding finite state machine state State_1 as an example, the transition condition Transition_1 of the test case from State_1 to State_2 and the variable parameters corresponding to the transition condition Transition_1 are obtained. The variable parameters can be parameters related to the transition condition Transition_1 generated by the test case during the transfer process, or can be parameters related to the transition condition Transition_1 generated by the test case during the period from 0 to t1.

[0083] In some other embodiments, to improve the efficiency of test evaluation, the method further includes the following steps:

[0084] Obtain the number of occurrences of the key finite state machine state in the state sequence; determine whether the number of occurrences of the key finite state machine state is equal to a preset number threshold; if so, execute to obtain the transfer condition of the test case from the preceding finite state machine state to the key finite state machine state, and the variable parameters corresponding to the transfer condition; if not, determine that the test result of the test case is failed.

[0085] In the embodiment of the present invention, for the test case, the threshold of the number of critical finite state machine states is marked in advance, and a corresponding relationship is established with the test case, which is convenient for reading through a script, as shown in Table 2, the preset threshold of the number of critical finite state machine states State_2 and State_3. Specifically:

[0086] If the number of occurrences of the critical finite state machine state in the state sequence is equal to the preset number threshold, the subsequent step of "obtaining the transfer condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and the variable parameters corresponding to the transfer condition" is executed; if the number of occurrences of the critical finite state machine state in the state sequence is not equal to the preset number threshold, the test result of the test case is determined to be failed. At this time, the corresponding failure reason is obtained, and the corresponding Fail_ID is output. The output content includes the name of the critical finite state machine state whose number of occurrences is different from the corresponding preset number threshold.

[0087] Test Case ID State_2 preset times threshold State_3 preset number threshold No.1 0 0 No.2 1 1 No.3 2 0 No.4 2 2 …… …… ……

[0088] Table 2

[0089] S40, processing the variable parameters based on the transfer condition to obtain the test result of the test case.

[0090] In the embodiment of the present invention, the test result of the test case can be determined by judging whether the variable parameter meets the transfer condition.

[0091] In the specific implementation process, the embodiment of the present invention converts the transfer condition into a judgment sub-function according to the software requirements, and continues to Figure 2 Taking the finite state machine shown in FIG. 1 as an example, the corresponding relationship between its transfer condition and the judgment sub-function is shown in Table 3. The input of the judgment sub-function is the variable parameter involved in the corresponding transfer condition, and the output is the flag bit indicating whether the transfer condition is met.

[0092] Transfer conditions Status before transfer Post-transfer status Determine the name of the sub-function Transition_1 State_1 State_2 Subfunction_1 Transition_2 State_2 State_1 Subfunction_2 Transition_3 State_2 State_3 Subfunction_3 Transition_4 State_3 State_1 Subfunction_4 …… ……

[0093] Table 3

[0094] In addition, if the variable parameters corresponding to the transfer condition include the first variable parameters when the test case is transferred from the pre-finite state machine state to the key finite state machine state, then step S40 "processing the variable parameters based on the transfer condition to obtain the test result of the test case" can adopt the following steps:

[0095] The first variable parameter is input into the judgment sub-function, and the first flag bit output by the judgment sub-function is obtained; if the first flag bit indicates that the first variable parameter meets the transfer condition, the test result of the test case is determined to be passed; if the first flag bit indicates that the first variable parameter does not meet the transfer condition, the test result of the test case is determined to be failed.

[0096] Continuing to use the finite state machine state State_2 and the preceding finite state machine state State_1 as examples, the judgment subfunction Subfunction_1 is called. If the first variable parameter generated during the transition of the test case from State_1 to State_2 can meet the transition condition Transition_1, the judgment subfunction Subfunction_1 determines that the state transition of the test case is correct and outputs the flag bit True, indicating that the test result of the test case is passed. On the contrary, the judgment subfunction Subfunction_1 outputs the flag bit False, indicating that the test result of the test case is not passed. At this time, the corresponding failure reason is obtained and the corresponding Fail_ID is output. The output content includes the judgment subfunction with the output flag bit False and the name of the corresponding segmentation point.

[0097] It should be noted that if the state sequence contains multiple transitions from State_1 to State_2, or other transitions such as from State_3 to State_1, the test result of the test case can be determined to be passed only when all state transitions are correct. If one state transition is wrong, the test result of the test case is determined to be failed.

[0098] On this basis, in order to improve the accuracy of the test evaluation, if the variable parameters corresponding to the transfer condition also include the second variable parameters of the test case in the period corresponding to the previous finite state machine state, then step S40 "obtaining the test result of the test case by processing the variable parameters based on the transfer condition" also includes the following steps:

[0099] If the test result of the test case determined based on the first flag is passed, the second variable parameter is input into the judgment sub-function, and the second flag output by the judgment sub-function is obtained; if the second flag indicates that the second variable parameter meets the transfer condition, the test result of the test case is determined to be failed; if the second flag indicates that the second variable parameter does not meet the transfer condition, the test result of the test case is determined to be passed.

[0100] Continuing to use the finite state machine state State_2 and the preceding finite state machine state State_1 as examples, the second variable parameter is the variable parameter generated by the test case in the period of 0 to t1, and the judgment subfunction Subfunction_1 is called. If the second variable parameter generated by the test case in State_1 does not meet the transition condition Transition_1, the judgment subfunction Subfunction_1 determines that the state transition of the test case is correct, and outputs the flag bit True, indicating that the test result of the test case is passed. On the contrary, the judgment subfunction Subfunction_1 outputs the flag bit False, indicating that the test result of the test case is not passed. At this time, the corresponding failure reason is obtained, and the corresponding Fail_ID is output. The output content includes the judgment subfunction with the output flag bit False and the name of the corresponding segmentation point.

[0101] Of course, the second variable parameter includes variable parameters at multiple sampling time points, so the second variable parameter can be determined to be inconsistent with the transition condition Transition_1 only if the variable parameters at all sampling time points do not meet the transition condition Transition_1. Conversely, if the variable parameter at one sampling time point meets the transition condition Transition_1, the second variable parameter can be determined to be inconsistent with the transition condition Transition_1.

[0102] In some other embodiments, considering the performance of the test case, in order to improve the accuracy of the test evaluation, Figure 1 Based on the test and evaluation method of the intelligent driving function shown in the figure, the following steps are also included:

[0103] If the test result of the test case obtained by processing the variable parameters based on the transfer condition is passed, then the performance state quantity of the test case during the test process is obtained, and the performance state quantity can characterize the performance state in continuous time; for each finite state machine state represented by the state sequence, the value range corresponding to the performance state quantity under the finite state machine state and the actual value of the performance state quantity of the test case in the time period corresponding to the finite state machine state are obtained; it is determined whether the actual value of the performance variable is within the value range corresponding to the performance state quantity; if so, the test result of the test case is determined to be passed; if not, the test result of the test case is determined to be failed.

[0104] In the embodiment of the present invention, the performance state quantity is used to characterize the change of the performance-related variables (such as speed, acceleration, jerk, the distance of the vehicle's center of mass from the lane center line, etc.) involved in the test case over time, and the performance state quantity also records a digital sequence that changes over time.

[0105] In addition, for the test case, the value range corresponding to the performance state quantity is marked in advance. Of course, the value range of the performance state quantity can be set uniformly or segmented according to the actual situation. For example, the corresponding value range of the performance state quantity is set for each finite state machine state represented by the state sequence, as shown in Table 4, and a corresponding relationship is established between it and the test case so that it can be read through a script.

[0106]

[0107] Table 4

[0108] For each finite state machine state represented by the state sequence, if the actual value of the performance state quantity of the test case in the corresponding time period is within the corresponding value range, the test result of the test case is determined to be passed; otherwise, the test result of the test case is determined to be failed. At this time, the corresponding failure reason is obtained, and the corresponding Fail_ID is output. The output content includes the name of the performance state quantity whose actual value is not within the value range, and the time period in which it is located.

[0109] An embodiment of the present invention provides a test and evaluation method for intelligent driving functions, which can automatically evaluate the test results of test cases and make the test and evaluation steps specific and executable. This can reduce the impact of software requirement changes on test and evaluation tasks. At the same time, for tasks that require repeated testing, it will also significantly reduce the manpower and time costs of related work and improve the efficiency of testing work.

[0110] Based on the test and evaluation method for the intelligent driving function provided in the above embodiment, an embodiment of the present invention provides a device for executing the test and evaluation method for the intelligent driving function. The structural diagram of the device is shown in FIG. Figure 5 As shown, including:

[0111] An acquisition module 10 is used to acquire the logical state quantity of the test case during the test process, and the logical state quantity can represent the state of the finite state machine in continuous time;

[0112] An extraction module 20 is used to extract the state sequence of the logic state quantity, where the state sequence can represent the finite state machine state in continuous time periods, one time period corresponds to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods are different;

[0113] The test module 30 is used to determine the preceding finite state machine state represented by the state sequence that is one period earlier than the preset critical finite state machine state, and obtain the transfer condition of the test case from the preceding finite state machine state to the critical finite state machine state, and the variable parameters corresponding to the transfer condition; and obtain the test result of the test case by processing the variable parameters based on the transfer condition.

[0114] Optionally, the testing module 30 is further used for:

[0115] The transfer condition is converted into a corresponding judgment sub-function, which is used to judge whether the function input meets the transfer condition;

[0116] The variable parameters corresponding to the transfer condition include the first variable parameter when the test case transfers from the pre-finite state machine state to the key finite state machine state;

[0117] The test module 30 for obtaining the test result of the test case by processing the variable parameters based on the transfer condition is specifically used for:

[0118] The first variable parameter is input into the judgment sub-function, and the first flag bit output by the judgment sub-function is obtained; if the first flag bit indicates that the first variable parameter meets the transfer condition, the test result of the test case is determined to be passed; if the first flag bit indicates that the first variable parameter does not meet the transfer condition, the test result of the test case is determined to be failed.

[0119] Optionally, the variable parameters corresponding to the transfer condition also include a second variable parameter of the test case in the time period corresponding to the state of the preceding finite state machine;

[0120] The test module 30 for obtaining the test result of the test case by processing the variable parameters based on the transfer condition is also used for:

[0121] If the test result of the test case determined based on the first flag is passed, the second variable parameter is input into the judgment sub-function, and the second flag output by the judgment sub-function is obtained; if the second flag indicates that the second variable parameter meets the transfer condition, the test result of the test case is determined to be failed; if the second flag indicates that the second variable parameter does not meet the transfer condition, the test result of the test case is determined to be passed.

[0122] Optionally, the testing module 30 is further used for:

[0123] Obtain the number of occurrences of the key finite state machine state in the state sequence; determine whether the number of occurrences of the key finite state machine state is equal to a preset number threshold; if so, execute the step of obtaining the transfer condition of the test case from the preceding finite state machine state to the key finite state machine state, and the variable parameters corresponding to the transfer condition; if not, determine that the test result of the test case is failed.

[0124] Optionally, the testing module 30 is further used for:

[0125] If the test result of the test case obtained by processing the variable parameters based on the transfer condition is passed, then the performance state quantity of the test case during the test process is obtained, and the performance state quantity can characterize the performance state in continuous time; for each finite state machine state represented by the state sequence, the value range corresponding to the performance state quantity under the finite state machine state and the actual value of the performance state quantity of the test case in the time period corresponding to the finite state machine state are obtained; it is determined whether the actual value of the performance variable is within the value range corresponding to the performance state quantity; if so, the test result of the test case is determined to be passed; if not, the test result of the test case is determined to be failed.

[0126] Optionally, the testing module 30 is further used to:

[0127] If the test result of the test case is determined to be failed, the corresponding failure reason is obtained and output.

[0128] An embodiment of the present invention provides a test and evaluation device for intelligent driving functions, which can automatically evaluate the test results of test cases and make the test and evaluation steps specific and executable. This can reduce the impact of software requirement changes on test and evaluation tasks. At the same time, for tasks that require repeated testing, it will also significantly reduce the manpower and time costs of related work and improve the efficiency of testing work.

[0129] The above is a detailed introduction to a test and evaluation method and device for an intelligent driving function provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

[0130] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0131] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device that includes a series of elements is inherent to the elements, or also includes elements inherent to these processes, methods, articles or devices. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0132] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A test and evaluation method for intelligent driving function, characterized in that: The method comprises: Acquire the logical state quantity of the test case during the test process, wherein the logical state quantity can represent the state of the finite state machine in continuous time; Extracting the state sequence of the logic state quantity, wherein the state sequence can represent the finite state machine state in continuous time periods, one time period corresponds to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods are different; Determine a preceding finite state machine state represented by the state sequence that is one period earlier than a preset critical finite state machine state, and obtain a transition condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and a variable parameter corresponding to the transition condition; Processing the variable parameters based on the transfer condition to obtain a test result of the test case; The method further comprises: Converting the transfer condition into a corresponding judgment sub-function, wherein the judgment sub-function is used to judge whether the function input meets the transfer condition; The variable parameters corresponding to the transfer condition include the first variable parameters when the test case is transferred from the pre-finite state machine state to the key finite state machine state; The step of processing the variable parameter based on the transfer condition to obtain the test result of the test case includes: Inputting the first variable parameter into the judgment sub-function, and obtaining the first flag bit output by the judgment sub-function; If the first flag bit indicates that the first variable parameter meets the transfer condition, determining that the test result of the test case is passed; If the first flag bit indicates that the first variable parameter does not meet the transfer condition, determining that the test result of the test case is failed; The variable parameters corresponding to the transfer condition also include a second variable parameter of the test case in the period corresponding to the state of the preceding finite state machine; The step of processing the variable parameters based on the transfer condition to obtain the test result of the test case further includes: If the test result of the test case determined based on the first flag is passed, inputting the second variable parameter into the judgment sub-function, and obtaining the second flag output by the judgment sub-function; If the second flag bit indicates that the second variable parameter meets the transfer condition, determining that the test result of the test case is failed; If the second flag bit indicates that the second variable parameter does not meet the transfer condition, the test result of the test case is determined to be passed.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining the number of occurrences of the key finite state machine state in the state sequence; Determine whether the number of occurrences of the key finite state machine state is equal to a preset number threshold; If yes, the step of obtaining the transition condition of the test case from the pre-finite state machine state to the key finite state machine state, and the variable parameters corresponding to the transition condition is executed; If not, it is determined that the test result of the test case is failed.

3. The method according to claim 1, characterized in that The method further comprises: If the test result of the test case obtained by processing the variable parameter based on the transfer condition is passed, then obtaining a performance state quantity of the test case during the test process, wherein the performance state quantity can represent the performance state in a continuous time; For each finite state machine state represented by the state sequence, obtain the value range corresponding to the performance state quantity in the finite state machine state and the actual value of the performance state quantity in the test case in the time period corresponding to the finite state machine state; Determine whether the actual value of the performance status quantity is within the value range corresponding to the performance status quantity; If yes, determining that the test result of the test case is passed; If not, the test result of the test case is determined to be failed.

4. The method according to any one of claims 2 to 3, characterized in that: The method further comprises: If it is determined that the test result of the test case is failed, the corresponding failure reason is obtained and outputted.

5. A test and evaluation device for intelligent driving function, characterized in that: The device comprises: An acquisition module is used to acquire the logical state quantity of the test case during the test process, wherein the logical state quantity can represent the state of the finite state machine in continuous time; An extraction module, used to extract the state sequence of the logic state quantity, wherein the state sequence can represent the finite state machine state in continuous time periods, one time period corresponds to one finite state machine state, and two finite state machine states corresponding to two adjacent time periods are different; A test module, used to determine a preceding finite state machine state represented by the state sequence that is one period earlier than a preset critical finite state machine state, and obtain a transition condition for the test case to transfer from the preceding finite state machine state to the critical finite state machine state, and a variable parameter corresponding to the transition condition; and process the variable parameter based on the transition condition to obtain a test result of the test case; The test module is also used for: Converting the transfer condition into a corresponding judgment sub-function, wherein the judgment sub-function is used to judge whether the function input meets the transfer condition; The variable parameters corresponding to the transfer condition include the first variable parameters when the test case is transferred from the pre-finite state machine state to the key finite state machine state; The test module for processing the variable parameters based on the transfer condition to obtain the test result of the test case is specifically used for: Inputting the first variable parameter into the judgment sub-function, and obtaining the first flag bit output by the judgment sub-function; if the first flag bit indicates that the first variable parameter meets the transfer condition, determining that the test result of the test case is passed; if the first flag bit indicates that the first variable parameter does not meet the transfer condition, determining that the test result of the test case is failed; The variable parameters corresponding to the transfer condition also include a second variable parameter of the test case in the period corresponding to the state of the preceding finite state machine; The test module for processing the variable parameters based on the transfer condition to obtain the test result of the test case is further used to: If the test result of the test case determined based on the first flag bit is passed, the second variable parameter is input into the judgment sub-function, and the second flag bit output by the judgment sub-function is obtained; if the second flag bit indicates that the second variable parameter meets the transfer condition, the test result of the test case is determined to be failed; if the second flag bit indicates that the second variable parameter does not meet the transfer condition, the test result of the test case is determined to be passed.

6. The device according to claim 5, characterized in that The test module is also used for: Obtaining the number of occurrences of the key finite state machine state in the state sequence; determining whether the number of occurrences of the key finite state machine state is equal to a preset number threshold; If yes, the step of obtaining the transfer condition of the test case from the preceding finite state machine state to the key finite state machine state and the variable parameters corresponding to the transfer condition is executed; if no, the test result of the test case is determined to be failed.

7. The device according to claim 5, characterized in that The test module is also used for: If the test result of the test case obtained by processing the variable parameter based on the transfer condition is passed, then obtaining a performance state quantity of the test case during the test process, wherein the performance state quantity can represent the performance state in a continuous time; For each finite state machine state represented by the state sequence, obtain the value range corresponding to the performance state quantity in the finite state machine state and the actual value of the performance state quantity in the test case in the time period corresponding to the finite state machine state; Determine whether the actual value of the performance status quantity is within the value range corresponding to the performance status quantity; If yes, determining that the test result of the test case is passed; If not, the test result of the test case is determined to be failed.

8. The device according to any one of claims 6 to 7, characterized in that: The test module is also used for: If it is determined that the test result of the test case is failed, the corresponding failure reason is obtained and outputted.

Images