Method and system for automatically generating anticipated function security scenarized test case

Abstract

The invention provides an automatic generation method and system for an expected function security scenarized test case, and the method comprises the steps: constructing an SOTIF real scene library based on an AI model, and the real scene library comprises scene metadata, an environment parameter template, an interference variable template and a security criterion template; a scene keyword input by a user is received, keyword analysis, scene library matching, parameter instantiation and case assembly are carried out through an AI model, and a scene test case is automatically generated; and scene logic verification is carried out according to the generated test cases, in response to meeting the scene logic verification, the priority score of each test case is calculated based on a pre-constructed priority ranking model, and the sequence of the test cases is determined according to the priority scores. According to the method, the test coverage rate is improved, the research and development period is shortened, the generated use case meets the ISO21448 standard, the method can be directly used for expected function safety test verification, and safe development and landing of intelligent driving products are supported.

Description

Technical Field

[0001] This application belongs to the field of expected functional safety testing and verification technology for intelligent driving of automobiles, and in particular relates to a method and system for automatically generating expected functional safety scenario-based test cases. Background Technology

[0002] With the widespread application of AI models in various fields, intelligent driving also faces severe challenges: AI models are prone to decision-making biases or functional failures under "unseen edge scenarios" (such as reflective ground in rainy weather), "incomplete data input" (such as loss of radar point clouds) or "external disturbances" (such as electromagnetic interference), which may lead to safety risks such as collisions.

[0003] Scenario-based testing is the core means of verifying whether intelligent driving functions can cause harmful behaviors due to functional deficiencies. However, existing test case generation methods have at least the following technical problems: high dependence on manual labor and low efficiency; lack of low-probability, high-risk edge scenarios; lack of test case priority ranking, resulting in wasted testing resources; and manually written test cases do not conform to the logic of real scenarios. Summary of the Invention

[0004] In view of this, this application aims to propose a method and system for automatically generating scenario-based test cases for expected functional safety, in order to solve at least one of the above problems.

[0005] To achieve the above objectives, the technical solution of this application is implemented as follows: Firstly, this application provides a method for automatically generating scenario-based test cases for anticipated functional safety, including: Construct a real-world scenario library based on an AI model, which includes scenario metadata, environmental parameter templates, interference variable templates, and security criterion templates. It receives scenario keywords input by users and uses an AI model to perform keyword parsing, scenario library matching, parameter instantiation, and test case assembly to automatically generate scenario-based test cases. The generated test cases are used to perform scenario logic verification. In response to satisfying the scenario logic verification, the priority score of each test case is calculated based on the pre-built priority ranking model, and the order of the test cases is determined according to the priority score.

[0006] Secondly, based on the same inventive concept, this application also provides an automatic generation system for expected functional safety scenario-based test cases, including: The scenario library construction module is configured to build a real scenario library based on AI models, which includes scenario metadata, environmental parameter templates, interference variable templates, and security criterion templates. The scenario library matching module is configured to receive scenario keywords input by the user and automatically generate scenario-based test cases by parsing keywords, matching scenarios, instantiating parameters, and assembling test cases through an AI model. The priority calculation module is configured to perform scenario logic verification based on the generated test cases. In response to satisfying the scenario logic verification, it calculates the priority score of each test case based on a pre-built priority ranking model and determines the order of the test cases based on the priority scores.

[0007] Thirdly, based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0008] Fourthly, based on the same inventive concept, this application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method as described in the first aspect.

[0009] Compared with existing technologies, the method and system for automatically generating scenario-based test cases for expected functional safety described in this application have the following advantages: The method for automatically generating scenario-based test cases for expected functional safety described in this application solves the problems of "low efficiency, low coverage, missing priorities, and unreasonable parameters" in traditional test case generation, thereby improving test coverage, shortening the development cycle, and generating test cases that conform to the ISO 21448 standard. These test cases can be directly used for expected functional safety testing and verification, supporting the safe development and deployment of intelligent driving products. Attached Figure Description

[0010] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating an automatic generation method for scenario-based test cases for expected functional safety, as described in an embodiment of this application. Figure 2 This is a flowchart illustrating a specific example of an automatic generation method for scenario-based test cases for expected functional safety as described in an embodiment of this application. Figure 3 This is a schematic diagram of the system structure for automatically generating scenario-based test cases for expected functional safety, as described in an embodiment of this application. Figure 4 This is a schematic diagram of the hardware structure of the electronic device described in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0012] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0013] As described in the background section above, existing methods for generating expected functional safety test cases require testers to manually design scenario parameters (such as environment, vehicle, and obstacle states) based on experience. For complex intelligent driving scenarios (such as multi-vehicle interaction and extreme weather), designing a single test case through brainstorming is extremely time-consuming and cannot meet the current demand for "massive scenario coverage" in intelligent driving. Furthermore, manual design is prone to overlooking "low-probability, high-risk" edge scenarios. This embodiment aims to solve the problems of "low efficiency, low coverage, and unreasonable parameters" in the existing expected functional safety test case generation by constructing an automatic generation method for scenario-based expected functional safety test cases.

[0014] Secondly, existing methods generate test cases without prioritization, requiring sequential execution during testing. This leads to high-risk scenarios (such as high-speed collision risk scenarios) and low-risk scenarios (such as low-speed parking scenarios) consuming the same testing resources, resulting in slow improvement in test coverage and an inability to quickly identify safety vulnerabilities in intelligent driving scenarios. This embodiment aims to address the "lack of priority" problem in existing expected functional safety test case generation by constructing an automatic generation method for scenario-based expected functional safety test cases.

[0015] This embodiment uses keyword-driven scenario automatic generation and constructs a two-dimensional test case priority ranking model of "scenario occurrence probability + risk consequences" to achieve "high efficiency, high coverage and standardization" of test cases, supporting the verification activities of expected functional safety.

[0016] The embodiments of this application are described in detail below with reference to the accompanying drawings.

[0017] Please seeFigure 1 As shown, this embodiment provides a method for automatically generating expected functional safety scenario-based test cases, which specifically includes the following steps: Step S101: Construct a real-world scenario library based on an AI model. The real-world scenario library includes scenario metadata, environmental parameter templates, interference variable templates, and security criterion templates.

[0018] Specifically, in this embodiment, the "real-world scenario library" in step S101 is the foundation for automatic use case generation. The real-world scenario library includes scenario metadata, environmental parameter templates, interference variable templates, and safety criterion templates. Furthermore, the data sources for the real-world scenario library include intelligent driving real-vehicle road test data, intelligent driving accident statistics, and scenarios recommended by the ISO 21448 standard. Specifically, the "Scene Metadata" section includes data categorized according to intelligent driving scenarios (e.g., city automatic following, highway cruise, automatic parking); the "Environmental Parameter Template" includes weather parameters (temperature, visibility, precipitation, snow thickness), illumination parameters (solar altitude angle, light intensity), and road parameters (road type, slope, traffic signs, number of lanes, road curvature, speed limit, road surface conditions), with the value range of each parameter determined based on real intelligent driving scenario statistics; the "Interference Variable Template" includes data input disturbances (camera occlusion area ratio, radar point cloud loss rate, etc.) and external object disturbances (target obstacle type, location, speed, etc.); and the "Safety Criterion Template" is based on the ISO 21448 standard, including the detection accuracy / missed detection rate / false detection rate of perception-type AI, and the response time / path deviation / collision warning time of planning and decision-making AI.

[0019] It should be noted that in this step, the real-world scenario library supports dynamic updates and can be integrated into new scenarios through incremental learning algorithms (the reinforcement learning algorithm described in this embodiment is a mature technology in this field, and will not be described in further detail in this embodiment).

[0020] Step S102: Receive scenario keywords input by the user, and automatically generate scenario-based test cases by parsing keywords, matching scenario libraries, instantiating parameters, and assembling test cases through an AI model.

[0021] Specifically, in this embodiment, keyword parsing uses an entity recognition algorithm from natural language processing to extract scene type, environmental elements, and disturbance elements from scene keywords; scene library matching locates the corresponding environmental parameter template, disturbance variable template, and safety criterion template in the real scene library based on the extracted elements; parameter instantiation is automatically generated by the AI ​​model based on the scene logical correlation, and the parameter values ​​conform to the logical consistency of real intelligent driving scenarios (e.g., the correlation between rainfall and visibility).

[0022] Step S103: Perform scenario logic verification based on the generated test cases. In response to satisfying the scenario logic verification, calculate the priority score of each test case based on the pre-built priority ranking model, and determine the order of the test cases based on the priority scores.

[0023] Specifically, in this embodiment, the test cases generated in step S102 need to pass the scenario logic verification. The verification rules include: the correlation between environmental parameters and interference variables, the compatibility between security criteria and AI modules, and the test cases that fail the verification are re-instantiated by AI.

[0024] The priority ranking model is configured with a weighted scoring method, and the weighted scoring method formula is as follows: Score=α×P+β×S, where α and β are weight coefficients, α∈[0.3, 0.6], β∈[0.4, 0.7], and α+β=1; during testing, high-priority test cases with Score≥0.7 are executed first.

[0025] Among them, the probability of scene occurrence (P) is based on the statistical frequency and regional adaptability of this type of scene in the real scene library, and is quantified as a value of 0~100%. Risk consequences (S) are based on the severity levels of ISO 26262, combined with the collision speed obtained from the collision simulation model, and quantified as a risk score of 0-1.

[0026] The method described in this embodiment first constructs a real-world scenario library containing scenario metadata, environmental parameter templates, interference variable templates, and safety criterion templates. Then, after receiving user scenario keywords, AI is used to perform keyword parsing, scenario matching, parameter instantiation, and test case assembly, automatically generating test cases containing "environmental parameters + interference variables + safety criteria." Finally, based on a dual-dimensional model of "scenario occurrence probability + risk consequences," test cases are prioritized, with high-risk, high-probability test cases executed first. This invention solves the problems of "low efficiency, low coverage, missing priorities, and unreasonable parameters" in traditional test case generation, improving test coverage, shortening the development cycle, and generating test cases that conform to the ISO 21448 standard. These test cases can be directly used for expected functional safety testing and verification, supporting the safe development and deployment of intelligent driving products.

[0027] like Figure 2 As shown, this embodiment applies the above method, taking "test cases of sudden obstacles on highways in rainy weather under the intelligent driving perception, decision-making and planning model" as an example, and provides the following method steps: Step S1: The user enters keywords; It should be noted that the example input is "rainy day, highway, sudden obstacle, decision model"; Step S2: The AI ​​model performs keyword parsing, identifies keywords, and extracts elements; It should be noted that, taking the user input of "rainy day, highway, sudden obstacle, decision model" in the previous step as an example; the scenario type = highway cruise, the weather = rainy day, the road = highway, the disturbance variable = sudden obstacle, and the applicable AI module = decision planning model; Step S3: The AI ​​model feeds the elements extracted from the keywords into the pre-trained scene library for matching; It should be noted that, in the above example, this parsed element will match "Highway Environmental Parameter Template", "Sudden Obstacle Interference Variable Template", and "Decision Model Safety Criterion Template". Step S4: Instantiate the parameters of the scene logic correlation based on the AI ​​model to generate test scene instances; It should be noted that the above example will be further refined within the model. Environmental parameters: rainfall 20mm / 24h, visibility approximately 5km, highway with 3 lanes, vehicle speed 110kph, road surface friction coefficient 0.5; disturbance variable: a sudden obstacle – a child (120cm tall, 45kg), located 10m to the right front of the vehicle, crossing laterally at 3m / s; safety criteria: decision response time <0.3s, collision warning time >0.4s; Step S5: Assemble test cases based on the AI ​​model to form complete test cases; Step S6: Prioritize all test cases based on the AI ​​model.

[0028] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0029] Based on the same inventive concept, and corresponding to the methods of any of the above embodiments, the embodiments of this application also provide an automatic generation device for expected functional safety scenario-based test cases.

[0030] like Figure 3 As shown, the device for automatically generating expected functional safety scenario-based test cases includes: The scenario library construction module 11 is configured to build a real scenario library based on AI models. The real scenario library includes scenario metadata, environmental parameter templates, interference variable templates and security criterion templates. The scenario library matching module 12 is configured to receive scenario keywords input by the user and automatically generate scenario-based test cases by parsing keywords, matching scenario libraries, instantiating parameters, and assembling test cases through an AI model. The priority calculation module 13 is configured to perform scenario logic verification based on the generated test cases. In response to the scenario logic verification being satisfied, it calculates the priority score of each test case based on a pre-built priority sorting model and determines the order of the test cases based on the priority scores.

[0031] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0032] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0033] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the methods described in any of the above embodiments.

[0034] Figure 4 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0035] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0036] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0037] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0038] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0039] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0040] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0041] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0042] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0043] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0044] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0045] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0046] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0047] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0048] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for automatically generating scenario-based test cases for expected functional safety, characterized in that, include: Construct a real-world scenario library based on an AI model, which includes scenario metadata, environmental parameter templates, interference variable templates, and security criterion templates. It receives scenario keywords input by users and uses an AI model to perform keyword parsing, scenario library matching, parameter instantiation, and test case assembly to automatically generate scenario-based test cases. The generated test cases are used to perform scenario logic verification. In response to satisfying the scenario logic verification, the priority score of each test case is calculated based on the pre-built priority ranking model, and the order of the test cases is determined according to the priority score.

2. The method according to claim 1, characterized in that: The environmental parameter template includes weather parameters, lighting parameters, and road parameters, and the value range of each parameter is determined based on real intelligent driving scenario statistical data. The disturbance variable template includes data input disturbances and external object disturbances; The safety criterion template includes the detection accuracy / missed detection rate / false detection rate of perception-type AI, and the response time / path deviation / collision warning time of planning and decision-making AI.

3. The method according to claim 1, characterized in that: The keyword parsing is based on an entity recognition algorithm using natural language processing to extract scene type, environmental elements, and disturbance elements from scene keywords. The scene library matching locates the corresponding environmental parameter template, interference variable template, and safety criterion template in the real scene library based on the extracted elements. The parameter instantiation is automatically generated by the AI ​​model based on the logical correlation of the scene, and the parameter values ​​conform to the logical consistency of real intelligent driving scenarios.

4. The method according to claim 1, characterized in that: The priority ranking model adopts a two-dimensional use case priority ranking model based on "probability of scenario occurrence + risk consequences"; The probability of a scenario occurring is based on the statistical frequency and regional adaptability of such scenarios in a real-world scenario library, and is quantified to obtain a probability value of 0 to 100%. The risk consequences are based on the severity levels of ISO 26262, combined with the collision speed obtained from the collision simulation model, and quantified to obtain a risk score of 0-1.

5. The method according to claim 4, characterized in that: The priority score is calculated using a weighted scoring method, the specific formula of which is as follows: Score = α × P + β × S; In the formula, α and β are weighting coefficients, and α+β=1, P is the probability value, S is the risk score, and Score is the priority score.

6. The method according to claim 1, characterized in that: The scenario logic verification rules include the correlation between environmental parameters and interference variables, and the compatibility of security criteria with AI modules.

7. The method according to claim 1, characterized in that: The data sources of the real-world scenario library include real-vehicle road test data for intelligent driving, statistical data on intelligent driving accidents, and scenarios recommended by the ISO 21448 standard. It also supports the integration of new scenarios through incremental learning algorithms.

8. A system for automatically generating scenario-based test cases for expected functional safety, characterized in that, include: The scenario library construction module is configured to build a real scenario library based on AI models, which includes scenario metadata, environmental parameter templates, interference variable templates, and security criterion templates. The scenario library matching module is configured to receive scenario keywords input by the user and automatically generate scenario-based test cases by parsing keywords, matching scenarios, instantiating parameters, and assembling test cases through an AI model. The priority calculation module is configured to perform scenario logic verification based on the generated test cases. In response to satisfying the scenario logic verification, it calculates the priority score of each test case based on a pre-built priority ranking model and determines the order of the test cases based on the priority scores.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1-7.

10. A non-transitory computer-readable storage medium, characterized in that, in, The non-transitory computer-readable storage medium stores computer instructions for causing a computer to perform the method described in any one of claims 1-7.

Images