Electric control firmware self-adaptive testing method and system based on deep reinforcement learning

Abstract

The invention relates to the technical field of electronic control firmware testing, and provides an electronic control firmware self-adaptive testing method and system based on deep reinforcement learning, and the method comprises the steps: constructing a virtual model of target hardware equipment; the virtual model comprises to-be-tested electronic control firmware, and the to-be-tested electronic control firmware controls the operation of the target hardware equipment; obtaining working environment parameter information, operation parameter threshold information and component information of the target hardware equipment; learning the operation parameter threshold information based on a preset learning rule, constructing an operation parameter exploration agent, learning the component information based on a preset learning rule, and constructing a component fault exploration agent; and testing the to-be-tested electronic control firmware based on the working environment parameter information, the operation parameter exploration agent and the component fault exploration agent to obtain a test result. According to the method, automatic detection of the electronic control firmware is realized, and the comprehensiveness and the accuracy of a detection result are improved.

Description

Technical Field

[0001] The present application relates to the technical field of electronic control firmware testing, and in particular to an electronic control firmware adaptive testing method and system based on deep reinforcement learning. Background Art

[0002] With the technological advancement in the fields of automobile, aviation and industrial automation, the complexity and functionality of electronic control systems are constantly increasing. As the core of electronic control systems, the stability and performance of electronic control firmware directly affect the reliability and safety of the entire system. Therefore, testing of electronic control firmware is extremely important.

[0003] Traditional electronic control firmware testing methods usually rely on manually setting test scenarios and test parameters. This method is not only time-consuming and labor-intensive, but also difficult to fully cover the operation of the electronic control firmware, resulting in inaccurate test results. Summary of the invention

[0004] The present application provides an electric control firmware adaptive testing method and system based on deep reinforcement learning to solve the problems raised by the above background technology.

[0005] In a first aspect, the present application provides an electronic control firmware adaptive testing method based on deep reinforcement learning, comprising: Constructing a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device; Obtaining working environment parameter information, operating parameter threshold information and component information of the target hardware device; The operating parameter threshold information is learned based on a preset learning rule to construct an operating parameter exploration agent, and the component information is learned based on a preset learning rule to construct a component fault exploration agent; The electronic control firmware to be tested is tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

[0006] In a possible implementation, building a virtual model of the target hardware device includes: Acquire hardware parameter information of the target hardware device, and construct a virtual framework corresponding to the target hardware device based on the hardware parameter information; Parsing the source code of the electronic control firmware to be tested to obtain a digital twin model of the electronic control firmware to be tested; the digital twin model includes multiple functional modules; The digital twin model is embedded in the virtual framework based on the embedding manner of the electronic control firmware to be tested in the target hardware device to obtain the virtual model.

[0007] In a possible implementation, the learning of the operating parameter threshold information based on a preset learning rule to construct an operating parameter exploration agent, and the learning of the component information based on a preset learning rule to construct a component fault exploration agent include: Generate first test case information based on the operation parameter threshold information; the first test case information includes multiple first test cases, each of which includes parameter operation information and first test result processing method information; Inputting the first test case information into a preset first deep network learning model, so that the first deep network learning model learns the first test case information to obtain the operating parameter exploration agent; Generate second test case information based on the component information; the second test case information includes multiple second test cases, each of which includes component failure strategy and second test result processing method information; The second test case information is input into a preset second neural network learning model so that the second neural network learning model learns the second test case information to obtain the component fault exploration agent.

[0008] In a possible implementation manner, the operating parameter threshold information includes parameter operating threshold ranges of multiple parameters, and the generating of the first test case information based on the operating parameter threshold information includes: For each of the parameters, the parameter operation threshold range of the parameter is expanded based on a preset parameter threshold expansion rule to obtain a parameter test threshold range of the parameter; the parameter operation threshold range is included in the parameter test threshold range, and the parameter operation threshold range and the parameter test threshold range do not have a common endpoint value; For each of the parameters, sampling is performed within the parameter test threshold range of the parameter based on a preset sampling rule to obtain multiple sampling values ​​of the parameter, and the multiple sampling values ​​constitute a test set corresponding to the parameter; wherein after the sampling values ​​are arranged in order from small to large, the difference between two adjacent sampling values ​​is consistent; Performing a Cartesian product operation on the test sets corresponding to the parameters to obtain multiple parameter operation information; For each of the parameter operation information, determine whether the parameter operation information meets the operation standard, and generate first test result processing method information corresponding to the parameter operation information based on the determination result; the first test result processing method information is: if it meets the operation standard, determine when the electronic control firmware to be tested issues an alarm message or controls the virtual model to stop running, store the parameter operation information in a preset experience replay pool; if it does not meet the operation standard, determine when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, store the parameter operation information in a preset experience replay pool; For each of the parameters, a group of fluctuation values ​​are generated for the parameter based on the parameter test threshold range of the parameter, a target parameter operation information that meets the operation standard is randomly selected from each of the parameter operation information, and a group of fluctuation values ​​of the parameter are used to replace the sampling value of the parameter in the target parameter operation information to obtain the fluctuation parameter operation information of the parameter; wherein, the first test result processing method information corresponding to the fluctuation parameter operation information is to control the operation of the virtual model with the fluctuation parameter operation information, and when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue to run, the parameter operation information is stored in a preset experience replay pool.

[0009] In a possible implementation, the component information includes multiple components of the target hardware device and function information of each component, and generating the second test case information based on the component information includes: For each of the components, component failure strategy information of the component is generated based on the functional information of the component, and the component failure strategy information includes at least one component failure strategy; wherein, for each of the component failure strategies, the second test result processing method information corresponding to the component failure strategy is that when the component failure strategy exists, when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, the component failure strategy is stored in a preset experience replay pool.

[0010] In a possible implementation, the working environment parameter information includes environment parameter thresholds of multiple environment parameters, and the testing of the electronic control firmware to be tested based on the working environment parameter information, the operating parameter exploration agent, and the component fault exploration agent to obtain a test result includes: For each of the environmental parameters, sampling is performed within the environmental parameter threshold of the environmental parameter based on a preset environmental parameter sampling rule to obtain a plurality of sampling values ​​of the environmental parameter, and the plurality of sampling values ​​constitute a sampling set of the environmental parameter; wherein after the sampling values ​​are arranged in order from small to large, the difference between two adjacent sampling values ​​is consistent; Performing a Cartesian product operation on the sample sets corresponding to the environmental parameters to obtain a plurality of environmental parameter setting information; For each of the environmental parameter setting information, a test environment of the virtual model is constructed based on the environmental parameter setting information, and the electronic control firmware to be tested is tested based on the operating parameter exploration agent and the component fault exploration agent in the test environment to obtain an initial test result corresponding to the environmental parameter setting information; The initial test results are integrated to obtain the test result.

[0011] In a possible implementation, the testing the electronic control firmware to be tested based on the operating parameter exploration agent and the component fault exploration agent in the test environment to obtain an initial test result includes: A first abnormal information set and a second abnormal information set corresponding to the test environment are constructed in a preset experience replay pool; when the operating parameter exploration agent detects an abnormal state of the electronic control firmware to be tested, the parameter operating information corresponding to the abnormal state is stored in the first abnormal information set; and when the component fault exploration agent detects an abnormal state of the electronic control firmware to be tested, the component fault strategy corresponding to the abnormal state is stored in the second abnormal information set; the first abnormal information set and the second abnormal information set constitute the initial test result.

[0012] In a second aspect, the present application provides an electronic control firmware adaptive testing system based on deep reinforcement learning, comprising: A first construction module is used to construct a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device; An acquisition module, used to acquire working environment parameter information, operating parameter threshold information and component information of the target hardware device; A second construction module is used to learn the operating parameter threshold information based on a preset learning rule to construct an operating parameter exploration agent, and to learn the component information based on a preset learning rule to construct a component fault exploration agent; The test module is used to test the electronic control firmware to be tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

[0013] The present application provides an electric control firmware adaptive testing method and system based on deep reinforcement learning, the method comprising: constructing a virtual model of a target hardware device; the virtual model comprises the electric control firmware to be tested, the electric control firmware to be tested controls the operation of the target hardware device; obtaining the working environment parameter information, operating parameter threshold information and component information of the target hardware device; learning the operating parameter threshold information based on preset learning rules, constructing an operating parameter exploration agent, and learning the component information based on preset learning rules, constructing a component fault exploration agent; testing the electric control firmware to be tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent, and obtaining the test results. The method realizes the automated detection of the electric control firmware and improves the comprehensiveness and accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0015] Figure 1 A schematic diagram of a flow chart of an electric control firmware adaptive testing method based on deep reinforcement learning provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of an electronic control firmware adaptive testing system based on deep reinforcement learning provided in an embodiment of the present application; Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0018] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0019] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0020] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0021] See also Figure 1 , Figure 1 A flowchart of an electric control firmware adaptive testing method based on deep reinforcement learning provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the electric control firmware adaptive testing method based on deep reinforcement learning provided in an embodiment of the present application includes steps S1 to S4.

[0022] Step S1, constructing a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device.

[0023] Specifically, step S1 includes the following steps: Acquire hardware parameter information of the target hardware device, and construct a virtual framework corresponding to the target hardware device based on the hardware parameter information; Parsing the source code of the electronic control firmware to be tested to obtain a digital twin model of the electronic control firmware to be tested; the digital twin model includes multiple functional modules; The digital twin model is embedded in the virtual framework based on the embedding manner of the electronic control firmware to be tested in the target hardware device to obtain the virtual model.

[0024] Step S2: Obtain working environment parameter information, operating parameter threshold information and component information of the target hardware device.

[0025] The working environment parameter information includes environment parameter thresholds of multiple environment parameters, the operation parameter threshold information includes parameter operation threshold ranges of multiple parameters, and the component information includes multiple components of the target hardware device and function information of each component.

[0026] Step S3: learning the operating parameter threshold information based on preset learning rules to construct an operating parameter exploration agent, and learning the component information based on preset learning rules to construct a component fault exploration agent.

[0027] Specifically, step S3 includes the following steps: Generate first test case information based on the operation parameter threshold information; the first test case information includes multiple first test cases, each of which includes parameter operation information and first test result processing method information; Inputting the first test case information into a preset first deep network learning model, so that the first deep network learning model learns the first test case information to obtain the operating parameter exploration agent; Generate second test case information based on the component information; the second test case information includes multiple second test cases, each of which includes component failure strategy and second test result processing method information; The second test case information is input into a preset second neural network learning model so that the second neural network learning model learns the second test case information to obtain the component fault exploration agent.

[0028] It can be understood that step S3, on the one hand, improves the intelligence of the testing process by automatically generating test cases, and on the other hand, generates first test case information based on the operating parameter threshold information and generates second test case information based on the component information, which helps to improve test coverage and thus improve the accuracy of the test results.

[0029] The step of generating the first test case information based on the operating parameter threshold information comprises the following steps: For each of the parameters, the parameter operation threshold range of the parameter is expanded based on a preset parameter threshold expansion rule to obtain a parameter test threshold range of the parameter; the parameter operation threshold range is included in the parameter test threshold range, and the parameter operation threshold range and the parameter test threshold range do not have a common endpoint value; For each of the parameters, sampling is performed within the parameter test threshold range of the parameter based on a preset sampling rule to obtain multiple sampling values ​​of the parameter, and the multiple sampling values ​​constitute a test set corresponding to the parameter; wherein after the sampling values ​​are arranged in order from small to large, the difference between two adjacent sampling values ​​is consistent; Cartesian product operations are performed on the test sets corresponding to the parameters to obtain multiple parameter operation information; wherein the Cartesian product is used to create a product set between two or more sets, and the result is all possible ordered pair combinations, and each ordered pair combination is formed by selecting an element from each set participating in the Cartesian product operation; For each of the parameter operation information, determine whether the parameter operation information meets the operation standard, and generate the first test result processing method information corresponding to the parameter operation information based on the determination result; the first test result processing method information is that if it meets the operation standard, it is determined that when the electronic control firmware to be tested issues an alarm message or controls the virtual model to stop running, the parameter operation information is stored in a preset experience replay pool; if it does not meet the operation standard, it is determined that when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, the parameter operation information is stored in a preset experience replay pool; wherein, the determination of whether the parameter operation information meets the operation standard includes: for each parameter value of the parameter operation information, determining whether the parameter value exists in the corresponding parameter operation threshold range; if each parameter value of the parameter operation information exists in the corresponding parameter operation threshold range, it is determined that the parameter operation information meets the operation standard; if there is any parameter value of the parameter operation information that does not exist in the corresponding parameter operation threshold range, it is determined that the parameter operation information does not meet the operation standard; For each of the parameters, a group of fluctuation values ​​are generated for the parameter based on the parameter test threshold range of the parameter, a target parameter operation information that meets the operation standard is randomly selected from each of the parameter operation information, and a group of fluctuation values ​​of the parameter are used to replace the sampling value of the parameter in the target parameter operation information to obtain the fluctuation parameter operation information of the parameter; wherein, the first test result processing method information corresponding to the fluctuation parameter operation information is to control the operation of the virtual model with the fluctuation parameter operation information, and when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue to run, the parameter operation information is stored in a preset experience replay pool; it can be understood that the fluctuation parameter operation information is also the parameter operation information mentioned above.

[0030] It can be understood that the above-mentioned method of generating the first test case information based on the operating parameter threshold information, on the one hand, expands the parameter operating threshold range, uniformly samples within the expanded parameter test threshold range, and performs Cartesian product operations on the test sets corresponding to each of the parameters to obtain multiple parameter operating information, which helps to conduct comprehensive testing of the electronic control firmware. On the other hand, the entire scheme helps to improve the test quality of the electronic control firmware and help to discover potential defects.

[0031] The step of generating the second test case information based on the component information comprises the following steps: For each of the components, component failure strategy information of the component is generated based on the functional information of the component, and the component failure strategy information includes at least one component failure strategy; wherein, for each of the component failure strategies, the second test result processing method information corresponding to the component failure strategy is that when the component failure strategy exists, when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, the component failure strategy is stored in a preset experience playback pool. For example, when the component is a sensor, the component failure strategy information of the component includes network disconnection, data drift, and data mutation; when the component is a communication device, the component failure strategy information of the component includes communication protocol change, network instability, and network disconnection.

[0032] Step S4: testing the electronic control firmware to be tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

[0033] Specifically, step S4 includes the following steps: For each of the environmental parameters, sampling is performed within the environmental parameter threshold of the environmental parameter based on a preset environmental parameter sampling rule to obtain a plurality of sampling values ​​of the environmental parameter, and the plurality of sampling values ​​constitute a sampling set of the environmental parameter; wherein after the sampling values ​​are arranged in order from small to large, the difference between two adjacent sampling values ​​is consistent; Performing a Cartesian product operation on the sample sets corresponding to the environmental parameters to obtain a plurality of environmental parameter setting information; For each of the environmental parameter setting information, a test environment of the virtual model is constructed based on the environmental parameter setting information, and the electronic control firmware to be tested is tested based on the operating parameter exploration agent and the component fault exploration agent in the test environment to obtain an initial test result corresponding to the environmental parameter setting information; The initial test results are integrated to obtain the test result.

[0034] The step of testing the electronic control firmware to be tested based on the operating parameter exploration agent and the component fault exploration agent in the test environment to obtain an initial test result includes the following steps: A first abnormal information set and a second abnormal information set corresponding to the test environment are constructed in a preset experience replay pool; when the operating parameter exploration agent detects an abnormal state of the electronic control firmware to be tested, the parameter operating information corresponding to the abnormal state is stored in the first abnormal information set; and when the component fault exploration agent detects an abnormal state of the electronic control firmware to be tested, the component fault strategy corresponding to the abnormal state is stored in the second abnormal information set; the first abnormal information set and the second abnormal information set constitute the initial test result. For example, when the operating parameter exploration agent controls the operation of the virtual model with parameter operating information that meets the standards, the operating parameter exploration agent detects that the electronic control firmware to be tested issues an alarm message or controls the virtual model to stop running, and stores this parameter operating information that meets the standards in the first abnormal information set; when the operating parameter exploration agent controls the operation of the virtual model with parameter operating information that does not meet the standards, the operating parameter exploration agent detects that the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, and stores this parameter operating information that does not meet the standards in the first abnormal information set; when the component fault exploration agent inputs a component fault strategy, the component fault exploration agent detects that the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, and stores this component fault strategy in the second abnormal information set.

[0035] It can be understood that when the target hardware device exists in the first exception information set or the second exception information set during operation, the electronic control firmware to be tested has defects and cannot work normally. The defects of the electronic control firmware to be tested can be improved through the first exception information set and the second exception information set.

[0036] It can be understood that step S4 obtains a sampling set of the environmental parameters by sampling within the environmental parameter threshold of the environmental parameter based on a preset environmental parameter sampling rule for each of the environmental parameters, and performs a Cartesian product operation on the sampling set corresponding to each of the environmental parameters to obtain multiple environmental parameter setting information, and for each of the environmental parameter setting information, builds a test environment for the virtual model based on the environmental parameter setting information, and tests the electronic control firmware to be tested in the test environment based on the operating parameter exploration agent and the component fault exploration agent to obtain an initial test result corresponding to the environmental parameter setting information, thereby achieving a comprehensive exploration of the test environment within the effective range of the working environment parameter information, which helps to achieve a comprehensive detection of the electronic control firmware to be tested and improve the accuracy of the detection results.

[0037] The method provided in this embodiment realizes the automatic detection of the electronic control firmware, and improves the comprehensiveness and accuracy of the detection results.

[0038] See also Figure 2 , Figure 2 The schematic block diagram of the structure of the electric control firmware adaptive test system 100 based on deep reinforcement learning provided in the embodiment of the present application is as follows: Figure 2 As shown, the electric control firmware adaptive testing system 100 based on deep reinforcement learning provided in the embodiment of the present application includes: The first construction module 110 is used to construct a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device.

[0039] The acquisition module 120 is used to acquire the working environment parameter information, operating parameter threshold information and component information of the target hardware device.

[0040] The second construction module 130 is used to learn the operating parameter threshold information based on preset learning rules to construct an operating parameter exploration agent, and to learn the component information based on preset learning rules to construct a component fault exploration agent.

[0041] The testing module 140 is used to test the electronic control firmware to be tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

[0042] It should be noted that technical personnel in the relevant technical field can clearly understand that, for the convenience and conciseness of description, the specific working process of the system and each module described above can refer to the corresponding process in the aforementioned embodiment of the electric control firmware adaptive testing method based on deep reinforcement learning, and will not be repeated here.

[0043] The electric control firmware adaptive testing system 100 based on deep reinforcement learning provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 3 The system is run on the terminal device 200 shown.

[0044] See also Figure 3 , Figure 3 The present invention provides a schematic block diagram of the structure of a terminal device 200 according to an embodiment of the present application. The terminal device 200 includes a processor 201 and a memory 202. The processor 201 and the memory 202 are connected via a device bus 203, wherein the memory 202 may include a non-volatile storage medium and an internal memory.

[0045] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 201, the processor 201 can execute any of the above-mentioned electric control firmware adaptive testing methods based on deep reinforcement learning.

[0046] The processor 201 is used to provide computing and control capabilities to support the operation of the entire terminal device 200.

[0047] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 201, the processor 201 can execute any of the above-mentioned electric control firmware adaptive testing methods based on deep reinforcement learning.

[0048] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal device 200 involved in the scheme of the present application. The specific terminal device 200 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0049] It should be understood that the processor 201 may be a central processing unit (CPU), and the processor 201 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0050] In some embodiments, the processor 201 is used to run a computer program stored in the memory to implement the following steps: Constructing a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device; Obtaining working environment parameter information, operating parameter threshold information and component information of the target hardware device; The operating parameter threshold information is learned based on a preset learning rule to construct an operating parameter exploration agent, and the component information is learned based on a preset learning rule to construct a component fault exploration agent; The electronic control firmware to be tested is tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

[0051] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the terminal device 200 described above can refer to the corresponding process of the aforementioned electric control firmware adaptive testing method based on deep reinforcement learning, and will not be repeated here.

[0052] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement the electric control firmware adaptive testing method based on deep reinforcement learning as provided in the embodiment of the present application.

[0053] The computer-readable storage medium may be an internal storage unit of the terminal device 200 in the aforementioned embodiment, such as a hard disk or memory of the terminal device 200. The computer-readable storage medium may also be an external storage device of the terminal device 200, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the terminal device 200.

[0054] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An adaptive testing method for electric control firmware based on deep reinforcement learning, characterized in that: include: Constructing a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device; Obtaining working environment parameter information, operating parameter threshold information and component information of the target hardware device; The operating parameter threshold information is learned based on a preset learning rule to construct an operating parameter exploration agent, and the component information is learned based on a preset learning rule to construct a component fault exploration agent; The electronic control firmware to be tested is tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

2. The electric control firmware adaptive testing method based on deep reinforcement learning according to claim 1 is characterized in that: The step of constructing a virtual model of the target hardware device includes: Acquire hardware parameter information of the target hardware device, and construct a virtual framework corresponding to the target hardware device based on the hardware parameter information; Parsing the source code of the electronic control firmware to be tested to obtain a digital twin model of the electronic control firmware to be tested; the digital twin model includes multiple functional modules; The digital twin model is embedded in the virtual framework based on the embedding manner of the electronic control firmware to be tested in the target hardware device to obtain the virtual model.

3. The electric control firmware adaptive testing method based on deep reinforcement learning according to claim 1 is characterized in that: The step of learning the operating parameter threshold information based on a preset learning rule to construct an operating parameter exploration agent, and learning the component information based on a preset learning rule to construct a component fault exploration agent, includes: Generate first test case information based on the operation parameter threshold information; the first test case information includes multiple first test cases, each of which includes parameter operation information and first test result processing method information; Inputting the first test case information into a preset first deep network learning model, so that the first deep network learning model learns the first test case information to obtain the operating parameter exploration agent; Generate second test case information based on the component information; the second test case information includes multiple second test cases, each of which includes component failure strategy and second test result processing method information; The second test case information is input into a preset second neural network learning model so that the second neural network learning model learns the second test case information to obtain the component fault exploration agent.

4. The electric control firmware adaptive testing method based on deep reinforcement learning according to claim 3 is characterized in that: The operating parameter threshold information includes parameter operating threshold ranges of multiple parameters, and the generating of first test case information based on the operating parameter threshold information includes: For each of the parameters, the parameter operation threshold range of the parameter is expanded based on a preset parameter threshold expansion rule to obtain a parameter test threshold range of the parameter; the parameter operation threshold range is included in the parameter test threshold range, and the parameter operation threshold range and the parameter test threshold range do not have a common endpoint value; For each of the parameters, sampling is performed within the parameter test threshold range of the parameter based on a preset sampling rule to obtain multiple sampling values ​​of the parameter, and the multiple sampling values ​​constitute a test set corresponding to the parameter; wherein after the sampling values ​​are arranged in order from small to large, the difference between two adjacent sampling values ​​is consistent; Performing a Cartesian product operation on the test sets corresponding to the parameters to obtain multiple parameter operation information; For each of the parameter operation information, determine whether the parameter operation information meets the operation standard, and generate first test result processing method information corresponding to the parameter operation information based on the determination result; the first test result processing method information is: if it meets the operation standard, determine when the electronic control firmware to be tested issues an alarm message or controls the virtual model to stop running, store the parameter operation information in a preset experience replay pool; if it does not meet the operation standard, determine when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, store the parameter operation information in a preset experience replay pool; For each of the parameters, a group of fluctuation values ​​are generated for the parameter based on the parameter test threshold range of the parameter, a target parameter operation information that meets the operation standard is randomly selected from each of the parameter operation information, and a group of fluctuation values ​​of the parameter are used to replace the sampling value of the parameter in the target parameter operation information to obtain the fluctuation parameter operation information of the parameter; wherein, the first test result processing method information corresponding to the fluctuation parameter operation information is to control the operation of the virtual model with the fluctuation parameter operation information, and when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue to run, the parameter operation information is stored in a preset experience replay pool.

5. The electric control firmware adaptive testing method based on deep reinforcement learning according to claim 4 is characterized in that: The component information includes multiple components of the target hardware device and function information of each component, and generating the second test case information based on the component information includes: For each of the components, component failure strategy information of the component is generated based on the functional information of the component, and the component failure strategy information includes at least one component failure strategy; wherein, for each of the component failure strategies, the second test result processing method information corresponding to the component failure strategy is that when the component failure strategy exists, when the electronic control firmware to be tested does not issue an alarm message or controls the virtual model to continue running, the component failure strategy is stored in a preset experience replay pool.

6. The electric control firmware adaptive testing method based on deep reinforcement learning according to claim 1 is characterized in that: The working environment parameter information includes environment parameter thresholds of multiple environment parameters, and the electronic control firmware to be tested is tested based on the working environment parameter information, the operation parameter exploration agent, and the component fault exploration agent to obtain a test result, including: For each of the environmental parameters, sampling is performed within the environmental parameter threshold of the environmental parameter based on a preset environmental parameter sampling rule to obtain a plurality of sampling values ​​of the environmental parameter, and the plurality of sampling values ​​constitute a sampling set of the environmental parameter; wherein after the sampling values ​​are arranged in order from small to large, the difference between two adjacent sampling values ​​is consistent; Performing a Cartesian product operation on the sample sets corresponding to the environmental parameters to obtain a plurality of environmental parameter setting information; For each of the environmental parameter setting information, a test environment of the virtual model is constructed based on the environmental parameter setting information, and the electronic control firmware to be tested is tested based on the operating parameter exploration agent and the component fault exploration agent in the test environment to obtain an initial test result corresponding to the environmental parameter setting information; The initial test results are integrated to obtain the test result.

7. The electric control firmware adaptive testing method based on deep reinforcement learning according to claim 6 is characterized in that: The testing of the electronic control firmware to be tested based on the operating parameter exploration agent and the component fault exploration agent in the test environment to obtain an initial test result includes: A first abnormal information set and a second abnormal information set corresponding to the test environment are constructed in a preset experience replay pool; when the operating parameter exploration agent detects an abnormal state of the electronic control firmware to be tested, the parameter operating information corresponding to the abnormal state is stored in the first abnormal information set; and when the component fault exploration agent detects an abnormal state of the electronic control firmware to be tested, the component fault strategy corresponding to the abnormal state is stored in the second abnormal information set; the first abnormal information set and the second abnormal information set constitute the initial test result.

8. An electronic control firmware adaptive test system based on deep reinforcement learning, characterized in that: include: A first construction module is used to construct a virtual model of a target hardware device; the virtual model includes an electronic control firmware to be tested, and the electronic control firmware to be tested controls the operation of the target hardware device; An acquisition module, used to acquire working environment parameter information, operating parameter threshold information and component information of the target hardware device; A second construction module is used to learn the operating parameter threshold information based on a preset learning rule to construct an operating parameter exploration agent, and to learn the component information based on a preset learning rule to construct a component fault exploration agent; The test module is used to test the electronic control firmware to be tested based on the working environment parameter information, the operating parameter exploration agent and the component fault exploration agent to obtain a test result.

Images