A test method for a UDS diagnostic service and related device

Abstract

This invention discloses a testing method and related equipment for UDS diagnostic services. The method includes: acquiring a diagnostic session of the UDS protocol state machine to be tested; identifying the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session; constructing initial test cases based on the protocol specification data; mutating the initial test cases using a preset mutation algorithm to obtain mutated test cases; acquiring ECU feedback data for the mutated test cases; and determining the test case content corresponding to the running score of the mutated test cases under different evaluation indicators and the ECU feedback data based on the ECU feedback data using statistical analysis methods, which are counted as correlation features; optimizing the mutated test cases based on the running score to obtain optimized test cases, and acquiring ECU state data when executing the optimized test cases; and generating a test report based on the ECU state data to complete the testing of the UDS diagnostic service for the UDS protocol state machine to be tested.

Description

Technical Field

[0001] This invention relates to the field of service testing technology, and in particular to a testing method and related equipment for UDS diagnostic services. Background Technology

[0002] Currently, software is ubiquitous in major sectors such as industrial production, national defense, scientific research, transportation, communications, healthcare, and aerospace, and has also become deeply integrated into people's daily lives. Software quality not only affects people's quality of life but also the safe and stable development of the economy. Therefore, software quality testing is crucial. Software testing is an effective and feasible means of testing software quality. As software scales up and becomes increasingly complex, the likelihood of software problems also increases, further enhancing the challenges of software testing.

[0003] To achieve software testing goals, a large number of test cases need to be written. For a long time, test cases have required testers to manually write them, consuming a significant amount of time and effort, resulting in high costs. In the software development process, software testing accounts for more than 50% of the total project time, and despite the substantial time invested in testing, cost and time constraints often prevent thorough testing. Reducing the cost and improving the efficiency of software testing has become a crucial task. To shorten the cost and time cycle of traditional software testing, automated testing has been widely researched and applied. In automated software testing, automatically constructing test inputs for the system under test and generating high-quality test cases is key to ensuring the scientific validity and effectiveness of the test results.

[0004] In existing technologies, there are methods for randomly generating test cases. However, due to the differences between different protocols, randomly generated test cases often cannot be adapted to different protocols. That is, randomly generated test cases are often unusable in real-world scenarios, resulting in a waste of human and computing resources and low testing efficiency. There are also methods for using predefined test scripts for verification. However, predefined test scripts are relatively simple, and the testing process is subject to random errors, making the test results unreliable.

[0005] Furthermore, existing technologies include specification-based test case generation methods, search-based test case generation methods, and symbolic execution-based test case generation methods. However, specification-based generation methods rely on manually written formal specifications, which places high demands on the specification writers: they must possess domain knowledge of the program under test and master the domain-specific language for writing formal specifications. Moreover, relying on manual specification writing prevents complete automation. Search-based generation methods exhibit significant randomness in their generation results and require sophisticated search strategies. Symbolic execution-based generation methods are costly and cannot guarantee that the generated reference-type test inputs will conform to the calling conventions of the program under test.

[0006] In summary, existing test case generation methods still suffer from inaccuracies and inefficiencies. Summary of the Invention

[0007] The purpose of this invention is to propose a testing method and related equipment for UDS diagnostic services, aiming to solve the problems of inaccuracy and low efficiency in existing test case generation methods.

[0008] To achieve one, some, or all of the above objectives, or other objectives, in a first aspect, the present invention proposes a testing method for UDS diagnostic services, the method comprising: Obtain the diagnostic session of the UDS protocol state machine to be tested, and identify the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. The initial test cases are mutated using a preset mutation algorithm to obtain mutated test cases; Obtain ECU feedback data for the variant test case, and based on the ECU feedback data, determine the running score of the variant test case under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test case through statistical analysis. The test case content is used as an association feature. The running score includes a novelty score representing the novelty of the response, an anomaly probability score representing the probability of an abnormal response, a coverage score when responding to the variant test case, and a vulnerability potential score representing the vulnerability corresponding to the variant test case. The initial priority of the mutation test cases is calculated based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data. When the mutated test case has a target response path, the priority of the mutated test case is updated according to the first preset dynamic adjustment rule to obtain the target priority of the mutated test case. The mutated test cases are optimized based on the target priority and the associated features to obtain optimized test cases, and ECU status data is obtained when the optimized test cases are executed. A test report is generated based on the ECU status data to complete the testing of the UDS diagnostic service for the UDS protocol state machine under test.

[0009] Optionally, the step of constructing initial test cases for the UDS service category in the protocol specification data includes: Based on the UDS compliance conditions of the UDS service category in the aforementioned protocol specification data, construct the first test case; Based on the first test case, an initial test case is constructed under the following conditions: boundary value conditions for the parameter boundary values ​​and length boundary of the first test case; outlier conditions for the avoidance data of the first test case; state conditions for the state parameters of the first test case under different states; and order conditions for the test order of the test cases. The avoidance data includes illegal service identification information, out-of-range parameters, and format error parameters.

[0010] Optionally, the step of mutating the initial test cases using a preset mutation algorithm to obtain mutated test cases includes: By using a preset mutation algorithm, at least one of the service type, parameters, length, and format of the initial test case is updated to obtain mutated test cases.

[0011] Optionally, the step of determining the running scores of the variant test cases under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test cases through statistical analysis based on the ECU feedback data, and using the test case content as an association feature, includes: Obtain ECU feedback data for the variant test cases, and identify ECU feedback data that characterizes abnormal responses in the ECU feedback data. Use the ECU feedback data that characterizes abnormal responses as abnormal response data. The abnormal responses include at least negative responses, timeout responses, format error responses, data error responses, restart responses, and crash responses. By using statistical analysis, the operational scores of the variant test cases under different evaluation metrics and the correlation characteristics between the variant test cases and the ECU feedback data are determined based on the abnormal response data. The operational scores include a novelty score characterizing the novelty of the response, an anomaly probability score characterizing the probability of the abnormal response occurring, a coverage score when responding to the variant test cases, and a vulnerability potential score characterizing the vulnerability corresponding to the variant test cases.

[0012] Optionally, the step of optimizing the mutated test cases based on the running score and the correlation features to obtain optimized test cases includes: The initial priority of the mutation test cases is calculated based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data. According to the first preset dynamic adjustment rule, when the mutated test case has a target response path, the priority of the mutated test case is updated to obtain the target priority of the mutated test case. The variant test cases are optimized based on the target priority and the associated features to obtain optimized test cases.

[0013] Optionally, the step of optimizing the mutated test cases based on the target priority and the associated features to obtain optimized test cases includes: Obtain the anomaly type from the abnormal response data, and update the associated features of the mutated test cases according to the anomaly type and a second preset dynamic adjustment rule to obtain updated test cases; Based on the target priorities of the updated test cases and the mutated test cases, the mutated test cases are optimized to obtain optimized test cases.

[0014] Optionally, the step of obtaining ECU status data when executing the optimized test case includes: According to the preset concurrency strategy, the UDS protocol state machine under test is tested with the optimized test cases to obtain ECU state data when the optimized test cases are executed. The preset concurrency strategy includes multi-threaded testing, distributed testing and concurrent scenario testing.

[0015] On the other hand, this application provides a testing apparatus for UDS diagnostic services, the apparatus comprising: The test case construction module is used to obtain the diagnostic session of the UDS protocol state machine to be tested, and identify the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. The test case mutation module is used to mutate the initial test cases using a preset mutation algorithm to obtain mutated test cases. The analysis module is used to acquire ECU feedback data for the variant test cases, and based on the ECU feedback data, to determine the running score of the variant test cases under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test cases through statistical analysis methods, and to use the test case content as an association feature. The running score includes a novelty score that characterizes the novelty of the response, an anomaly probability score that characterizes the probability of an abnormal response, a coverage score when responding to the variant test cases, and a vulnerability potential score that characterizes the vulnerability corresponding to the variant test cases. The test case optimization module is used to calculate the initial priority of the mutated test case based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and preset weight data; when the mutated test case has a target response path, the priority of the mutated test case is updated according to a first preset dynamic adjustment rule to obtain the target priority of the mutated test case; the mutated test case is optimized according to the target priority and the associated features to obtain optimized test cases, and ECU status data is obtained when the optimized test cases are executed; The report generation module is used to generate a test report based on the ECU status data to complete the test of the UDS diagnostic service for the UDS protocol state machine under test.

[0016] Thirdly, the present invention provides an electronic device, characterized in that it includes: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the test method for UDS diagnostic services as described above are performed.

[0017] Fourthly, the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, performs the steps of the test method for UDS diagnostic services as described above.

[0018] The embodiments of the present invention have the following beneficial effects: A diagnostic session for the UDS protocol state machine under test is acquired, and the protocol specification data of the UDS protocol state machine under test is identified based on the diagnostic session. Initial test cases are then constructed based on the protocol specification data. By determining the protocol specification data of the UDS protocol state machine under test, initial test cases can be generated in a targeted manner. That is, when faced with the differences between different UDS protocol state machines, initial test cases are generated in a targeted manner, avoiding the generation of invalid test cases (test cases that are not suitable for the current UDS protocol state machine under test), reducing resource waste, and improving testing efficiency. The initial test cases are mutated using a preset mutation algorithm to obtain mutated test cases. More test cases for the UDS protocol state machine under test are generated through the preset mutation algorithm, thereby avoiding random errors caused by a small number of tests and making the test results reliable. The process involves acquiring ECU feedback data for the variant test cases, and using statistical analysis to determine the performance scores of the variant test cases under different evaluation metrics and the test case content corresponding to the ECU feedback data within the variant test cases. This test case content is then used as a correlation feature. Based on the performance scores and correlation features, the variant test cases are optimized to obtain optimized test cases, and ECU state data is acquired during the execution of these optimized test cases. By determining the performance scores of the variant test cases under different evaluation metrics and the correlation features between the variant test cases and the ECU feedback data within the variant test cases, the variant test cases can be optimized purposefully. This allows for faster testing, better aligns with human / test requirements, and better addresses testing needs in different scenarios. A test report is generated based on the ECU state data to complete the testing of the UDS diagnostic service for the UDS protocol state machine under test. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] in: Figure 1 This is a flowchart of a testing method for UDS diagnostic services provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a testing device for UDS diagnostic services provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 4This is a schematic diagram of the structure of a storage medium provided in an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0027] like Figure 1 As shown, the present invention provides a testing method for UDS diagnostic services, the method comprising: S101. Obtain the diagnostic session of the UDS protocol state machine to be tested, and identify the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. In one possible implementation, the step of constructing initial test cases for the UDS service category in the protocol specification data includes: Based on the UDS compliance conditions of the UDS service category in the aforementioned protocol specification data, construct the first test case; Based on the first test case, an initial test case is constructed under the following conditions: boundary value conditions for the parameter boundary values ​​and length boundary of the first test case; outlier conditions for the avoidance data of the first test case; state conditions for the state parameters of the first test case under different states; and order conditions for the test order of the test cases. The avoidance data includes illegal service identification information, out-of-range parameters, and format error parameters.

[0028] For example, the strategy for constructing initial test cases can be understood as including: standard compliance testing (UDS compliance conditions for the UDS service category in the protocol specification data), boundary value testing (boundary value conditions for the parameter boundary values ​​within the first test case and the length boundary of the first test case), outlier testing (outlier conditions for the avoidance data of the first test case), state-related testing (state conditions for the state parameters of the first test case in different states), and sequence testing (sequential conditions for the test order of the test cases). The steps for standard compliance testing are as follows: test cases that fully comply with the UDS specification and verify whether the ECU conforms to the standard; the steps for boundary value testing are: test parameter boundary values ​​(0x00, 0xFF) and test length boundaries (minimum / maximum); the steps for outlier testing are: test illegal service IDs, test out-of-range parameters, and test parameters with incorrect formats; the steps for state-related testing are: test under different session states; test sensitive services and state transition anomalies under unauthorized states; the steps for sequence testing are: service call order; retry after timeout; recovery after interruption.

[0029] For example, managing a UDS diagnostic session enables protocol state tracking. Specifically, managing a UDS diagnostic session includes: obtaining session state; managing state transitions; implementing secure access control; and monitoring the session to obtain the protocol specification data of the UDS protocol state machine under test.

[0030] For example, the UDS service categories obtained from the protocol specification data through UDS service classification are: Diagnostic Session Control (0x10); ECU Reset (0x11); Secure Access (0x27); Communication Control (0x28); Authentication (0x29); Tester Online (0x3E); Read Data (0x22); Read Memory (0x23); Read Scaling Data (0x24); Read DID (0x2A); Write Data (0x3D); Write Memory (0x3E); Input / Output Control (0x2F / 0x7F); Routine Control (0x31); Request Download (0x34); Request Data Transfer (0x36); Transfer Data (0x36); Request Exit Transfer (0x37); Request File Transfer (0x38); Diagnostic Fault Code Control (0x19).

[0031] In one possible implementation, before constructing the initial test cases, a historical test database is introduced. Test cases that have triggered abnormal responses in historical tests are clustered to extract high-value seed test cases as the basis for generating the initial test cases. Specific steps include: Historical test cases and their corresponding ECU feedback data are obtained from the historical test database; the historical test cases are clustered according to the exception type and trigger frequency, and each cluster center is extracted as a seed test case; the seed test cases are merged with the initial test cases constructed based on the protocol specification data to obtain an enhanced initial test case set.

[0032] In one possible implementation, when constructing the initial test cases, the state machine of the UDS protocol to be tested is graph-modeled, with nodes in the graph representing diagnostic session states and edges representing state transitions triggered by service requests. Test sequences covering all state transition paths are generated based on graph traversal algorithms (such as depth-first search and breadth-first search), so that the initial test cases cover each edge in the state machine graph at least once.

[0033] For example, when constructing initial test cases, taking the UDS protocol state machine of a certain vehicle ECU as an example: First, the default session, programming session, and extended session of the ECU are modeled as graph nodes, and the different sub-function requests of the diagnostic session control service are modeled as state transition edges between nodes; then, starting from the default session, a breadth-first search algorithm is used to traverse the graph, generating state transition paths in sequence: "default session persist, default switch to programming, programming switch to extended, extended switch to default, default switch to extended, programming switch to default"; finally, each path is converted into a corresponding UDS service request message, forming 6 initial test cases, so that the initial test cases cover each state transition edge in the state machine graph at least once, ensuring complete testing of the ECU session state switching logic.

[0034] S102. The initial test cases are mutated using a preset mutation algorithm to obtain mutated test cases; In one possible implementation, the step of mutating the initial test cases using a preset mutation algorithm to obtain mutated test cases includes: By using a preset mutation algorithm, at least one of the service type, parameters, length, and format of the initial test case is updated to obtain mutated test cases.

[0035] For example, service ID mutations include: traversing all possible service IDs (0x00-0xFF); testing for unimplemented services; parameter mutations include: bit flipping: randomly flipping data bits; byte replacement: replacing with a known pattern; boundary scan: incrementing / decrementing tests; random padding: randomly generating data; length mutations include: length overflow: excessively long data; insufficient length: missing data; length mismatch: incorrect length field; format mutations include: type error: string / number obfuscation; encoding error: non-standard encoding; checksum error: incorrect checksum.

[0036] In one possible implementation, the strategy selection of the mutation algorithm is modeled as a reinforcement learning problem: the running score of the current mutation test case (novelty score, anomaly probability score, coverage score, vulnerability potential score) is used as the reward signal, and the mutation operation type (service type mutation, parameter mutation, length mutation, format mutation) is used as the action space. The next round of mutation operation is dynamically selected through the policy network, so that the mutation direction continuously converges towards the high score region.

[0037] The preset mutation algorithm includes an adaptive mutation strategy based on reinforcement learning. The adaptive mutation strategy includes: obtaining the running score of the current mutation test case as a reward signal; updating the parameters of the policy network based on the reward signal; and selecting the next round of mutation operation type according to the updated policy network to further mutate the mutation test case.

[0038] In one possible implementation, based on single test case mutation, cross-service related mutation is introduced: identify service pairs with dependencies in the UDS protocol (such as the security access service 0x27 depending on the diagnostic session switching service 0x10), and synchronously modify the relevant parameters in multiple test cases with dependencies during mutation, constructing a multi-step linked mutation test case sequence to detect potential vulnerabilities at service interaction boundaries.

[0039] S103. Obtain ECU feedback data for the variant test case, and based on the ECU feedback data, determine the running score of the variant test case under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test case through statistical analysis. The test case content is used as an association feature. The running score includes a novelty score representing the novelty of the response, an anomaly probability score representing the probability of an abnormal response, a coverage score when responding to the variant test case, and a vulnerability potential score representing the vulnerability corresponding to the variant test case. In one possible implementation, the step of determining the running scores of the variant test cases under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test cases through statistical analysis based on the ECU feedback data, and using the test case content as an association feature, includes: Obtain ECU feedback data for the variant test cases, and identify ECU feedback data that characterizes abnormal responses in the ECU feedback data. Use the ECU feedback data that characterizes abnormal responses as abnormal response data. The abnormal responses include at least negative responses, timeout responses, format error responses, data error responses, restart responses, and crash responses. By using statistical analysis, the operational scores of the variant test cases under different evaluation metrics and the correlation characteristics between the variant test cases and the ECU feedback data are determined based on the abnormal response data. The operational scores include a novelty score characterizing the novelty of the response, an anomaly probability score characterizing the probability of the abnormal response occurring, a coverage score when responding to the variant test cases, and a vulnerability potential score characterizing the vulnerability corresponding to the variant test cases.

[0040] For example, coverage metrics include: Service coverage = number of services tested / total number of services; Parameter coverage = number of parameters tested / total number of parameters; State coverage = number of states reached / total number of states; Path coverage = number of paths tested / total number of paths.

[0041] For example, a negative response (0x7F) includes: request not supported (0x11); condition not met (0x22); request out of order (0x24); unauthorized secure access (0x33); invalid service (0x7F).

[0042] In one possible implementation, when acquiring ECU feedback data, in addition to identifying abnormal response content, response timing features (response delay, response delay jitter, response sequence interval) are extracted to construct a timing feature vector; abnormal responses at the timing level are identified through a timing anomaly detection model (such as an LSTM autoencoder or sliding window statistical method), and the timing anomaly responses are included in the anomaly response data to supplement the existing content-based anomaly response identification.

[0043] The abnormal response also includes a timing abnormal response, which is determined by the following steps: extracting the response delay sequence of the ECU feedback data; calculating the statistical characteristics of the response delay sequence, including the mean, standard deviation, and abnormal deviation; when the abnormal deviation exceeds a preset timing threshold, marking the corresponding ECU feedback data as a timing abnormal response.

[0044] S104. Calculate the initial priority of the mutation test case based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data; when the mutation test case has a target response path, update the priority of the mutation test case according to the first preset dynamic adjustment rule to obtain the target priority of the mutation test case; optimize the mutation test case according to the target priority and the associated features to obtain optimized test cases, and obtain ECU status data when executing the optimized test cases; In one possible implementation, the step of optimizing the variant test cases based on the running score and the correlation features to obtain optimized test cases includes: The initial priority of the mutation test cases is calculated based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data. According to the first preset dynamic adjustment rule, when the mutated test case has a target response path, the priority of the mutated test case is updated to obtain the target priority of the mutated test case. The variant test cases are optimized based on the target priority and the associated features to obtain optimized test cases.

[0045] For example, the initial priority = {response novelty weight × novelty score + anomaly probability weight × anomaly score + coverage weight × coverage score + vulnerability potential weight × potential score} For example, the first preset dynamic adjustment rule is to increase the priority (target priority) of the mutated test case after it is discovered when a new response path is found.

[0046] In one possible implementation, a dynamic weight adaptation mechanism is provided, which may include: automatically adjusting the weight ratio of novelty score, anomaly probability score, coverage score and vulnerability potential score according to the current testing phase (exploration phase, convergence phase, deep mining phase), for example, increasing the weight of coverage score in the exploration phase and increasing the weight of vulnerability potential score in the deep mining phase, so that the priority calculation is more in line with the goals of different testing phases.

[0047] In one possible implementation, the step of optimizing the mutated test cases based on the target priority and the associated features to obtain optimized test cases includes: Obtain the anomaly type from the abnormal response data, and update the associated features of the mutated test cases according to the anomaly type and a second preset dynamic adjustment rule to obtain updated test cases; Based on the target priorities of the updated test cases and the mutated test cases, the mutated test cases are optimized to obtain optimized test cases.

[0048] For example, the second preset dynamic adjustment rules include, but are not limited to: updating test cases by reducing the associated features of the timeout parameter range when a timeout exception occurs; updating test cases to achieve in-depth exploration of the area by changing the associated features of the exploration depth when a vulnerability is discovered; and updating test cases to quickly skip unresponsive intervals by changing the associated features of the exploration depth.

[0049] In one possible implementation, when executing optimized test cases, the same optimized test cases are sent simultaneously to multiple ECUs of different versions or from different manufacturers, and the response results of each ECU are compared. When different ECUs produce inconsistent responses to the same test case, the test case is marked as a consistency difference test case and the difference details are recorded and included in the test report to discover inconsistencies at the protocol implementation level.

[0050] S105. Generate a test report based on the ECU status data to complete the test of the UDS diagnostic service for the UDS protocol state machine under test.

[0051] In one possible implementation, the step of obtaining ECU state data during the execution of the optimized test case includes: According to the preset concurrency strategy, the UDS protocol state machine under test is tested with the optimized test cases to obtain ECU state data when the optimized test cases are executed. The preset concurrency strategy includes multi-threaded testing, distributed testing and concurrent scenario testing.

[0052] For example, multi-threaded testing includes, but is not limited to: each thread independently testing a path; sharing an exception detection module; and merging test results.

[0053] Distributed testing includes, but is not limited to: concurrent testing across multiple devices; load balancing; and centralized result collection.

[0054] Concurrency scenario testing includes, but is not limited to: sending multiple diagnostic requests simultaneously; testing race conditions; and testing concurrent processing capabilities.

[0055] In one possible implementation, when acquiring ECU status data during the execution of the optimized test case, abnormal behavior of the ECU is monitored in real time, and response time (normal / timeout), response frequency (normal / abnormal), error code distribution, and protocol errors for the optimized test case are monitored at the communication level.

[0056] At the system level, monitor the ECU status (running / restarting / crashing), CPU / memory usage (if available), DTC records (check for new DTCs), and functional anomalies (functional failures) for the optimized test cases.

[0057] At the security level, monitoring targets include authentication bypass (successful unauthorized access), privilege escalation (low-privilege access to high-privilege services), information leakage (returning sensitive data), and denial of service (ECU denial of service) for the optimized test cases.

[0058] When generating a test report based on the ECU status data, anomaly identification is performed. The anomaly identification includes: Time anomaly: response time < normal minimum value × 0.5; response time > normal maximum value × 10; Frequency anomaly: the same error response > 10 times / second; timeout rate > 50%; Status anomaly: ECU restart; ECU no response; sudden increase in DTC; Data anomaly: unexpected data returned; data format error; data length anomaly, etc.

[0059] When generating a test report based on the ECU status data, the test cases (complete messages of optimized test cases), ECU responses (raw data) to optimized test cases, response timestamps, session status, exception types, and reproduction steps are recorded.

[0060] In one possible implementation, real-time alarms based on the ECU status data are also included, wherein different alarm levels employ different response methods: no alarm message is sent when a new response is detected; a level 1 warning is sent when an abnormal response is detected; a level 2 warning is sent when a potential vulnerability is detected; a level 3 warning is sent when a crash / reboot is detected; and a level 4 warning is sent when a critical security vulnerability is detected.

[0061] In one possible implementation, discovered vulnerabilities are categorized and reported: Vulnerability categories include: buffer overflows, specifically stack-based overflows, heap-based overflows, and integer overflows; denial-of-service (DoS) vulnerabilities, specifically resource exhaustion, deadlock / livelock, and infinite loops; information disclosure vulnerabilities, specifically sensitive information exposure, debugging information disclosure, and memory reads; privilege escalation vulnerabilities, specifically authentication bypass, access control failure, and privileged operation abuse; and logic vulnerabilities, specifically race conditions, state confusion, and business logic errors.

[0062] In one possible implementation, the discovered vulnerabilities are scored, where the scoring dimensions include exploitability (0-10), impact scope (0-10), triggering difficulty (0-10), and remediation cost (0-10). The discovered vulnerabilities are scored using CVSS scoring, which includes: extracting the CVSS vector string; calculating the base score; calculating the impact score; and calculating the overall score to obtain the vulnerability score.

[0063] In one possible implementation, the test report structure includes: an execution overview, i.e., test time, number of test cases, and coverage statistics; a vulnerability list, i.e., vulnerability ID, vulnerability name, CVSS score, and vulnerability description; vulnerability details, i.e., vulnerability triggering conditions, vulnerability POC code, vulnerability reproduction steps, and vulnerability impact analysis; and remediation recommendations, i.e., remediation solutions, verification methods, and testing suggestions.

[0064] In one possible implementation, after completing a round of testing, the high-value test cases (i.e., test cases that trigger abnormal responses) and their associated features discovered in this test are stored in the test case knowledge base. In subsequent tests, the features in the knowledge base are transferred to the new UDS protocol state machine to be tested through transfer learning methods, so as to reduce the test case exploration time of the new target and realize the accumulation and reuse of test experience across targets.

[0065] Specifically, high-value test cases and their associated features stored in the test case knowledge base are structurally represented, and protocol layer features, test case behavior features, and feedback features are extracted to construct a unified standardized feature space. Secondly, a source domain prediction model is trained based on historical data in the knowledge base. This model takes standardized feature vectors as input and outputs anomaly trigger probability, anomaly type, and vulnerability potential score, thereby learning the vulnerability triggering patterns and anomaly response rules of historical UDS protocol state machines. Next, for a new UDS protocol state machine to be tested, its protocol specification information is extracted, a small number of basic probe test cases are generated, and corresponding ECU feedback data is obtained to construct a feature dataset for the new target.

[0066] Then, domain adaptation methods (such as maximum mean difference alignment or adversarial domain adaptation) are used to reduce the feature distribution difference between the source domain and the target domain, and a small number of samples from the new target are used to fine-tune the pre-trained model to achieve cross-domain transfer of model parameters and feature representations.

[0067] Finally, the mutated test cases of the new target are input into the migrated model to predict the abnormal risks and vulnerability potential of each test case. Based on this, the mutated test cases are prioritized and the preset mutation algorithm is guided to prioritize mutation operations on high-risk feature dimensions. This allows for the reuse of historical testing experience, reduction of invalid exploration, and rapid generation of high-value test cases adapted to the new UDS protocol state machine, realizing cross-target test knowledge transfer and experience reuse.

[0068] In one possible implementation, such as Figure 2 As shown, this application provides a testing apparatus for UDS diagnostic services, the apparatus comprising: The test case construction module 201 is used to obtain the diagnostic session of the UDS protocol state machine to be tested, and identify the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. The test case mutation module 202 is used to mutate the initial test cases using a preset mutation algorithm to obtain mutated test cases. The analysis module 203 is used to acquire ECU feedback data for the variant test case, and based on the ECU feedback data, to determine the running score of the variant test case under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test case through statistical analysis methods, and to use the test case content as an association feature. The running score includes a novelty score that characterizes the novelty of the response, an anomaly probability score that characterizes the probability of an abnormal response, a coverage score when responding to the variant test case, and a vulnerability potential score that characterizes the vulnerability corresponding to the variant test case. The test case optimization module 204 is used to calculate the initial priority of the mutated test case based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and preset weight data; when the mutated test case has a target response path, the priority of the mutated test case is updated according to the first preset dynamic adjustment rule to obtain the target priority of the mutated test case; the mutated test case is optimized according to the target priority and the associated features to obtain the optimized test case, and ECU status data when the optimized test case is executed is obtained; The report generation module 205 is used to generate a test report based on the ECU status data to complete the test of the UDS diagnostic service for the UDS protocol state machine under test.

[0069] In one possible implementation, such as Figure 3 As shown, this application provides an electronic device 300, including: a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it performs the following: acquiring a diagnostic session of the UDS protocol state machine to be tested, and identifying the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. The initial test cases are mutated using a preset mutation algorithm to obtain mutated test cases; Obtain ECU feedback data for the variant test case, and based on the ECU feedback data, determine the running score of the variant test case under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test case through statistical analysis. The test case content is used as an association feature. The running score includes a novelty score representing the novelty of the response, an anomaly probability score representing the probability of an abnormal response, a coverage score when responding to the variant test case, and a vulnerability potential score representing the vulnerability corresponding to the variant test case. The initial priority of the mutation test cases is calculated based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data. When the mutated test case has a target response path, the priority of the mutated test case is updated according to the first preset dynamic adjustment rule to obtain the target priority of the mutated test case. The mutated test cases are optimized based on the target priority and the associated features to obtain optimized test cases, and ECU status data is obtained when the optimized test cases are executed. A test report is generated based on the ECU status data to complete the test of the UDS diagnostic service for the UDS protocol state machine under test.

[0070] In one possible implementation, such as Figure 4 As shown, this application embodiment provides a computer-readable storage medium 400, on which a computer program 411 is stored. When the computer program 411 is executed by a processor, it implements: acquiring a diagnostic session of the UDS protocol state machine to be tested, and identifying the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. The initial test cases are mutated using a preset mutation algorithm to obtain mutated test cases; Obtain ECU feedback data for the variant test case, and based on the ECU feedback data, determine the running score of the variant test case under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test case through statistical analysis. The test case content is used as an association feature. The running score includes a novelty score representing the novelty of the response, an anomaly probability score representing the probability of an abnormal response, a coverage score when responding to the variant test case, and a vulnerability potential score representing the vulnerability corresponding to the variant test case. The initial priority of the mutation test cases is calculated based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data. When the mutated test case has a target response path, the priority of the mutated test case is updated according to the first preset dynamic adjustment rule to obtain the target priority of the mutated test case. The mutated test cases are optimized based on the target priority and the associated features to obtain optimized test cases, and ECU status data is obtained when the optimized test cases are executed. A test report is generated based on the ECU status data to complete the test of the UDS diagnostic service for the UDS protocol state machine under test.

[0071] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0072] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0073] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0074] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0075] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0076] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

[0077] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A testing method for UDS diagnostic services, characterized in that, The method includes: Obtain the diagnostic session of the UDS protocol state machine to be tested, and identify the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data; The initial test cases are mutated using a preset mutation algorithm to obtain mutated test cases; Obtain ECU feedback data for the variant test case, and based on the ECU feedback data, determine the running score of the variant test case under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test case through statistical analysis. The test case content is used as an association feature. The running score includes a novelty score representing the novelty of the response, an anomaly probability score representing the probability of an abnormal response, a coverage score when responding to the variant test case, and a vulnerability potential score representing the vulnerability corresponding to the variant test case. The initial priority of the mutation test cases is calculated based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and the preset weight data. When the mutated test case has a target response path, the priority of the mutated test case is updated according to the first preset dynamic adjustment rule to obtain the target priority of the mutated test case. The mutated test cases are optimized based on the target priority and the associated features to obtain optimized test cases, and ECU status data is obtained when the optimized test cases are executed. A test report is generated based on the ECU status data to complete the testing of the UDS diagnostic service for the UDS protocol state machine under test.

2. The testing method for UDS diagnostic services according to claim 1, characterized in that, The step of constructing initial test cases for the protocol specification data includes: Based on the UDS compliance conditions of the UDS service category in the aforementioned protocol specification data, construct the first test case; Based on the first test case, an initial test case is constructed under the following conditions: boundary value conditions for the parameter boundary values ​​and length boundary of the first test case; outlier conditions for the avoidance data of the first test case; state conditions for the state parameters of the first test case under different states; and order conditions for the test order of the test cases. The avoidance data includes illegal service identification information, out-of-range parameters, and format error parameters.

3. The testing method for UDS diagnostic services according to claim 2, characterized in that, The step of mutating the initial test cases using a preset mutation algorithm to obtain mutated test cases includes: By using a preset mutation algorithm, at least one of the service type, parameters, length, and format of the initial test case is updated to obtain mutated test cases.

4. The testing method for UDS diagnostic services according to claim 1, characterized in that, The step of determining the running scores of the variant test cases under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test cases through statistical analysis based on the ECU feedback data, and using the test case content as an association feature, includes: Obtain ECU feedback data for the variant test case, and identify ECU feedback data that characterizes abnormal response in the ECU feedback data. Use the ECU feedback data that characterizes abnormal response as abnormal response data. The abnormal response includes at least one of the following: negative response, timeout response, format error response, data error response, restart response, and crash response. By using statistical analysis, the running scores of the variant test cases under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test cases are determined based on the abnormal response data, and the test case content is used as an association feature.

5. The testing method for UDS diagnostic services according to claim 4, characterized in that, The step of optimizing the mutated test cases based on the target priority and the associated features to obtain optimized test cases includes: Obtain the anomaly type from the abnormal response data, and update the associated features of the mutated test cases according to the anomaly type and a second preset dynamic adjustment rule to obtain updated test cases; Based on the target priorities of the updated test cases and the mutated test cases, the mutated test cases are optimized to obtain optimized test cases.

6. The testing method for UDS diagnostic services according to claim 1, characterized in that, The step of obtaining ECU status data when executing the optimized test case includes: According to the preset concurrency strategy, the UDS protocol state machine under test is tested with the optimized test cases to obtain ECU state data when the optimized test cases are executed. The preset concurrency strategy includes multi-threaded testing, distributed testing and concurrent scenario testing.

7. A testing device for UDS diagnostic services, characterized in that, The device includes: The test case construction module is used to obtain the diagnostic session of the UDS protocol state machine to be tested, and identify the protocol specification data of the UDS protocol state machine to be tested based on the diagnostic session, so as to construct initial test cases for the protocol specification data. The test case mutation module is used to mutate the initial test cases using a preset mutation algorithm to obtain mutated test cases. The analysis module is used to acquire ECU feedback data for the variant test cases, and based on the ECU feedback data, to determine the running score of the variant test cases under different evaluation indicators and the test case content corresponding to the ECU feedback data within the variant test cases through statistical analysis methods, and to use the test case content as an association feature. The running score includes a novelty score that characterizes the novelty of the response, an anomaly probability score that characterizes the probability of an abnormal response, a coverage score when responding to the variant test cases, and a vulnerability potential score that characterizes the vulnerability corresponding to the variant test cases. The test case optimization module is used to calculate the initial priority of the mutated test case based on the novelty score, the anomaly probability score, the coverage score, the vulnerability potential score, and preset weight data; when the mutated test case has a target response path, the priority of the mutated test case is updated according to a first preset dynamic adjustment rule to obtain the target priority of the mutated test case; the mutated test case is optimized according to the target priority and the associated features to obtain optimized test cases, and ECU status data is obtained when the optimized test cases are executed; The report generation module is used to generate a test report based on the ECU status data to complete the test of the UDS diagnostic service for the UDS protocol state machine under test.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the test method for UDS diagnostic services as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the test method for the UDS diagnostic service as described in any one of claims 1 to 6.

Images