Test method and device and electronic equipment

Abstract

The invention relates to the technical field of Internet of Vehicles, in particular to a test method and device and electronic equipment. The method comprises the following steps: acquiring a communication protocol supported by a to-be-tested vehicle when establishing a connection with the to-be-tested vehicle; determining target test configuration based on the configuration information and the communication protocol of the to-be-tested vehicle; wherein the target test configuration comprises at least one test instruction, and one test instruction corresponds to one vehicle function; and performing a fuzzy test on the to-be-tested vehicle according to the target test configuration to generate a vulnerability detection result.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle networking technology, and in particular to a testing method, device and electronic equipment. Background Art

[0002] At present, with the advancement of technology, more and more vehicles are equipped with vehicle networking functions, so that users can control the vehicle to perform corresponding functions through electronic devices bound to the vehicle.

[0003] Therefore, how to detect vehicle network vulnerabilities to prevent vehicles from being attacked by the network has become an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a testing method, a device and an electronic device for solving the problem of how to detect network vulnerabilities of a vehicle to prevent the vehicle from being attacked by the network.

[0005] In a first aspect, the present application provides a testing method, comprising: when establishing a connection with a vehicle to be tested, obtaining a communication protocol supported by the vehicle to be tested; determining a target test configuration based on the configuration information and communication protocol of the vehicle to be tested; wherein the target test configuration includes at least one test instruction, and one test instruction corresponds to one vehicle function; performing a fuzzy test on the vehicle to be tested according to the target test configuration to generate a vulnerability detection result.

[0006] In some feasible examples, a target test configuration is determined based on the configuration information and communication protocol of the vehicle to be tested, including: determining at least one theoretical test configuration based on the configuration information and communication protocol of the vehicle to be tested; calculating an evaluation index for each theoretical test configuration; based on the evaluation index, determining an evaluation score for the theoretical test configuration corresponding to the evaluation index; and taking the theoretical test configuration corresponding to the maximum evaluation score as the target test configuration.

[0007] In some feasible examples, determining at least one theoretical test configuration based on the configuration information and communication protocol of the vehicle to be tested includes: querying the test configuration relationship based on the configuration information and communication protocol of the vehicle to be tested, and determining at least one theoretical test configuration.

[0008] In some implementable examples, the configuration information includes functional modules, and the evaluation indicators include test coverage; calculating the evaluation indicators of each theoretical test configuration includes: for each theoretical test configuration, obtaining a first total number of communication protocols tested by the theoretical test configuration and a second total number of functional modules tested; obtaining the test coverage based on the first total number, the second total number, a third total number of communication protocols supported by all vehicles stored in the database, and a fourth total number of functional modules of all vehicles stored in the database.

[0009] In some feasible examples, the evaluation indicators include test time, resource utilization, test security, and error detection capability, and the resource utilization includes computing resource utilization and network bandwidth utilization; the evaluation indicators of each theoretical test configuration are calculated, including: for each theoretical test configuration, obtaining theoretical information of each test instruction in the theoretical test configuration; wherein the theoretical information includes one or more of theoretical time consumption, theoretical computing resource occupancy, theoretical network bandwidth occupancy, theoretical security, and theoretical detection capability; and the evaluation indicators are obtained based on the theoretical information of all test instructions in the theoretical test configuration.

[0010] In some feasible examples, based on the evaluation indicator, the evaluation score of the theoretical test configuration corresponding to the evaluation indicator is determined, including: based on the evaluation indicator, querying the score relationship table to determine the theoretical score of each evaluation indicator; based on the sum of the product of the weight value corresponding to each evaluation indicator and the theoretical score of each evaluation indicator, determining the evaluation score of the theoretical test configuration corresponding to the evaluation indicator.

[0011] In some feasible examples, based on the evaluation indicator, the evaluation score of the theoretical test configuration corresponding to the evaluation indicator is determined, including: based on the evaluation indicator, querying the score relationship table to determine the theoretical score of each evaluation indicator; based on the sum of all theoretical scores, determining the evaluation score of the theoretical test configuration corresponding to the evaluation indicator.

[0012] In a second aspect, the present application provides a testing device, comprising: a processing module, used to control an acquisition module to obtain a communication protocol supported by the vehicle to be tested when establishing a connection with the vehicle to be tested; the processing module is also used to determine a target test configuration based on the configuration information of the vehicle to be tested and the communication protocol obtained by the acquisition module; wherein the target test configuration includes at least one test instruction, and one test instruction corresponds to one vehicle function; the processing module is also used to perform fuzzy testing on the vehicle to be tested according to the target test configuration to generate a vulnerability detection result.

[0013] In a third aspect, the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the above method when executing the computer program.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above method.

[0015] Compared with the prior art, the technical solution provided by the present invention has the following advantages:

[0016] The test method provided by the present disclosure obtains the communication protocol supported by the vehicle to be tested when the vulnerability of the vehicle to be tested needs to be tested by establishing a connection with the vehicle to be tested. After that, the target test configuration is determined based on the configuration information and communication protocol of the vehicle to be tested; the vehicle to be tested is fuzzy tested according to the target test configuration, so as to determine whether there are vulnerabilities in the vehicle function by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, and then generate vulnerability detection results based on the test results of all vehicle functions, solving the problem of how to detect the network vulnerabilities of the vehicle to prevent the vehicle from being attacked by the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 One of the flow charts of a testing method provided in the first embodiment is exemplarily shown in FIG.

[0020] Figure 2 FIG. 2 exemplarily shows a flow chart of a test method provided in the first embodiment;

[0021] Figure 3 FIG. 3 exemplarily shows a flow chart of a test method provided in the first embodiment;

[0022] Figure 4 FIG4 is a flow chart of a test method provided in the first embodiment of the present invention;

[0023] Figure 5 FIG. 5 is a flowchart of a test method provided in the first embodiment of the present invention;

[0024] Figure 6 6 is a flow chart of a test method provided in the first embodiment;

[0025] Figure 7 FIG. 7 is a flow chart showing a test method provided in the first embodiment of the present invention;

[0026] Figure 8 The structural diagram of the testing device provided in the second embodiment is exemplarily shown in FIG.

[0027] Fig. 9 A structural schematic diagram of an electronic device provided in the second embodiment is exemplarily shown in FIG. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0031] In some examples, Fuzzing testing in the embodiments of the present disclosure, also known as fuzz testing, is a software testing technology mainly used to discover vulnerabilities and defects in software.

[0032] In some examples, FlexRay in the disclosed embodiments is a high-speed, deterministic, fault-tolerant bus technology for automobiles.

[0033] Embodiment 1

[0034] Figure 1 A flow chart of a testing method is shown in FIG. 1 , and the execution subject of this example may be an electronic device, such as Figure 1 As shown, the method includes:

[0035] S11. When establishing a connection with the vehicle to be tested, obtaining a communication protocol supported by the vehicle to be tested.

[0036] In some examples, an application that can execute the test method provided by the embodiments of the present disclosure is installed on the electronic device, and the application includes a protocol parsing and adaptation module, an intelligent analysis and configuration module, a test function module and a data recording and analysis module; wherein the protocol parsing and adaptation module, the corresponding module identifier is such as M1, and its function is to parse the received communication protocol data and dynamically adapt the communication protocols of different vehicle models (such as the Controller Area Network (CAN) protocol and the Diagnostic over IP (DOIP) protocol based on IP); the intelligent analysis and configuration module, the corresponding module identifier is such as M2, and its function is to analyze the vehicle characteristics based on the model learning algorithm and generate the optimal test configuration; the test function module, the corresponding module identifier is such as M3, and its function is to realize the generation, replay, data modification and other functions of fuzzing commands and flexibly adjust the test strategy; the data recording and analysis module, the corresponding module identifier is such as M4, and its function is to record positive and negative feedback fuzzing commands in real time and generate a detailed safety test report.

[0037] In some examples, the protocol parsing and adaptation module (M1) receives the communication protocol data sent by the vehicle to be tested, identifies the communication protocol supported by the vehicle based on the communication protocol data (such as CAN protocol, DOIP protocol, etc.), and dynamically parses and adapts the communication protocols of different vehicle models to ensure the communication compatibility between the tool and the vehicle. The intelligent analysis and configuration module (M2) analyzes the vehicle communication characteristics through a deep learning model based on the parsing results of the M1 module, automatically generates the optimal fuzzing test configuration, reduces human intervention, and improves test efficiency. The test function module (M3) performs fuzzing tests on the communication protocol through the test configuration provided by M2, supports command replay and data modification, and deeply explores potential vulnerabilities in the protocol. The data recording and analysis module (M4) records the positive and negative feedback commands generated during the test, performs intelligent analysis, generates vulnerability reports, and provides data support for security reinforcement.

[0038] Some examples of how M1 works include:

[0039] M1 captures the data stream in the communication interface of the vehicle to be tested, uses the specific header features and data formats of different communication protocols, and combines the preset protocol parsing rules to classify the data and identify the communication protocol. The parsing process is as follows:

[0040] 1. Data capture: Establish a connection with the vehicle to be tested through a tool interface (such as an OBD-II interface or an Ethernet interface), and then capture the original data stream of the tool interface.

[0041] 2. Protocol header analysis: Identify the header features of the data packet and determine the communication protocol supported by the vehicle to be tested based on the pre-set byte sequence, length, and key field values. For example, preliminary identification is performed through the identifier (ID) of the CAN protocol frame and the specific message format of the DOIP protocol.

[0042] 3. Field parsing: further parse the fields in the data packet to verify the specific type and version of the protocol.

[0043] 4. Dynamic matching rules: Use the protocol database and regular matching method to dynamically adapt unknown protocols and be compatible with possible protocol variants.

[0044] In some examples, the communication protocol includes: one or more of: CAN protocol, DOIP protocol, LIN protocol (Local Interconnect Network), FlexRay, and Unified Diagnostic Services (UDS).

[0045] Characteristics of CAN protocol: Messages are based on identifiers (IDs), the frame format is fixed, and standard frames (11-bit IDs) and extended frames (29-bit IDs) are supported. Identification method: M1 determines whether the vehicle to be tested supports the CAN protocol by checking whether the data frame conforms to the header characteristics of the CAN protocol (start bit, control bit, CRC check bit).

[0046] Characteristics of LIN protocol: Based on master-slave architecture, short data frame length (e.g. 8 bytes), low-cost communication protocol. Identification method: M1 determines whether the vehicle to be tested supports LIN protocol by detecting the byte format of the synchronization field and frame header.

[0047] Characteristics of DOIP protocol: Based on Ethernet protocol, mainly used for diagnostic communication, containing specific DOIP protocol header fields (such as version, message length, etc.). Identification method: M1 determines whether the vehicle to be tested supports DOIP protocol by checking whether the Ethernet data frame contains DOIP protocol specific fields (such as fields with a starting byte of 0x02FD).

[0048] Features of FlexRay: High-bandwidth real-time communication protocol, data frames contain static and dynamic segments of fixed length. Identification method: M1 determines whether the vehicle to be tested supports FlexRay by detecting the specific flag field and fixed synchronization mode of the frame header.

[0049] Features of UDS: Used for vehicle diagnostic communication, based on ISO 14229 standard, usually transmitted through CAN protocol or DOIP protocol. Identification method: M1 determines whether the vehicle to be tested supports UDS by detecting whether the data frame contains a specific diagnostic service request flag (such as 0x10, 0x22).

[0050] In some examples, after a user starts an application on an electronic device that can execute the testing method provided by the embodiments of the present disclosure, the application initializes M1, M3, and M4.

[0051] Afterwards, M1 receives the vehicle communication protocol data through the OBD interface. M1 parses the communication protocol type and dynamically adapts the protocol characteristics to determine the communication protocol supported by the vehicle to be tested. Afterwards, M2 generates the target test configuration based on the configuration information of the vehicle to be tested and the communication protocol parsed by M1. M3 performs fuzz testing on the vehicle to be tested according to the target test configuration, including command generation, replay and data modification, and records the test feedback data. M4 records the test data in real time, analyzes the positive and negative feedback commands, and generates vulnerability detection results, including a detailed report of the test process, results and vulnerability analysis.

[0052] S12: Determine a target test configuration based on the configuration information and communication protocol of the vehicle to be tested, wherein the target test configuration includes at least one test instruction, and one test instruction corresponds to one vehicle function.

[0053] In some examples, M2 generates an optimal test configuration solution, namely, a target test configuration, based on the communication protocol parsed by M1 and in combination with a vehicle model database.

[0054] Among them, the vehicle model database is an important foundation for realizing intelligent testing and data analysis. Its content covers all kinds of information required by the vehicle during the test process to support the compatibility and accuracy of the test tools. The following is the main information content of the vehicle model database:

[0055] 1. Basic vehicle information

[0056] Vehicle model name (brand, model, year), VIN code (Vehicle Identification Number) rules and parsing methods, ECU (Electronic Control Unit) quantity and type, communication protocol information; among them, communication protocol information includes supported communication protocols (such as CAN protocol, LIN protocol, DOIP protocol, UDS, etc.), initialization parameters of each communication protocol (such as CAN protocol baud rate, DOIP protocol Ethernet IP address and port number, etc.), specific message format and field description of each communication protocol, as well as data frame ID range and key fields (such as diagnostic service flag).

[0057] 2. Functional characteristics information

[0058] The name and purpose of each ECU's functional module;

[0059] List of request and response messages for ECU communication (such as UDS diagnostic service);

[0060] Functional safety related features (such as fault tolerance mechanism, error handling method).

[0061] 3. Diagnosis and test information

[0062] Common fault codes (DTC, Diagnostic Trouble Codes) list and analysis methods;

[0063] Diagnostic service support (such as 0x10 session control, 0x22 data request, 0x31 control service, etc.).

[0064] 4. Common test configuration parameters (such as test timeout, response conditions, etc.).

[0065] 5. Vehicle configuration and topology information

[0066] In-vehicle communication network topology (such as CAN protocol bus structure, Ethernet connection method).

[0067] Gateway configuration for different subnets.

[0068] 6. Electrical configuration parameters (such as power, current and other test limits).

[0069] 7. Other dynamic information

[0070] Vehicle firmware version and upgrade records;

[0071] Known vulnerabilities and safety risks of vehicle models;

[0072] Historical data for specific tests for each vehicle model (e.g. test success rate, average response time).

[0073] In some examples, based on the configuration information and communication protocol of the vehicle to be tested, the test configuration relationship is queried to determine at least one theoretical test configuration.

[0074] Exemplarily, the test configuration relationship is shown in Table 1.

[0075] Table 1

[0076]

[0077] In this way, the test configuration relationship can be queried based on the configuration information and communication protocol of the vehicle to be tested, and at least one theoretical test configuration can be determined.

[0078] Since there may be multiple theoretical test configurations for the same vehicle function, it is necessary to calculate the evaluation indicators of multiple theoretical test configurations corresponding to the same vehicle function; based on the evaluation indicators, determine the evaluation scores of the theoretical test configurations corresponding to the evaluation indicators; and use the theoretical test configuration corresponding to the maximum evaluation score as the target test configuration corresponding to the vehicle function. For example, when the configuration information of the vehicle to be tested includes vehicle function 1 and vehicle function 2, and the communication protocol is communication protocol 1, it can be known from querying Table 1 that the theoretical test configurations corresponding to vehicle function 1 and communication protocol 1 include theoretical test configuration 1, theoretical test configuration 2, and theoretical test configuration 3. Afterwards, by calculating the evaluation score 1 of theoretical test configuration 1, the evaluation score 2 of theoretical test configuration 2, and the evaluation score 3 of theoretical test configuration 3, if the evaluation score 2 is the largest among the evaluation scores 1, 2, and 3, then the theoretical test configuration 2 is used as the target test configuration for vehicle function 1.

[0079] In some examples, the configuration information and communication protocol based on the vehicle to be tested can be input into the configuration model to determine the target test configuration. The training process of the configuration model includes:

[0080] Acquire first training sample data and a first marking result of the first training sample data; wherein the first training sample data includes configuration information of a historical vehicle and a communication protocol supported by the historical vehicle, and the first marking result includes a target test configuration of the historical vehicle.

[0081] The first training sample data is input into the first neural network model for learning, and a first prediction result of the first neural network model on the first training sample data is obtained.

[0082] Based on the first prediction result and the first labeling result, the network parameters of the first neural network model are adjusted until the first neural network model converges to obtain a configuration model.

[0083] S13. Perform fuzz testing on the vehicle to be tested according to the target test configuration to generate vulnerability detection results.

[0084] In some examples, pattern testing includes fuzzing testing.

[0085] In some examples, when fuzz testing is performed on a vehicle under test according to a target test configuration, one or more of replaying important commands and modifying data are supported to further explore vulnerabilities.

[0086] Among them, the implementation of replaying important commands and modifying data is as follows:

[0087] 1. How to replay important commands:

[0088] The replay command refers to resending the recorded communication data packets (such as CAN frames or DOIP messages) to the target vehicle to observe its response behavior, so as to detect whether the vehicle has unauthorized responses or vulnerabilities.

[0089] 1.1 Data packet recording and storage

[0090] Use a data acquisition module, such as a CAN adapter or Ethernet adapter, to capture communication data in a vehicle bus or network.

[0091] Data packets are stored in chronological order, including information such as data frame ID, data length, and data payload.

[0092] 1.2 Packet Replay

[0093] Send the data packets recorded in 1.1 to the vehicle's communication network, keep the original time interval or adjust the time parameters, so as to achieve single or multiple repeated sending of specific commands (such as Fuzzing test).

[0094] 1.3 Dynamic Replay

[0095] Dynamically adjust the order, frequency or trigger conditions of data packet replay according to test requirements. Capture real-time feedback from the target vehicle during replay and record abnormal responses.

[0096] 2. Data modification implementation method:

[0097] Data modification refers to changing certain fields of the recorded data packet (such as ID, data payload, etc.), generating a new data packet and sending it to the target vehicle to test the vehicle's ability to handle abnormal or boundary data.

[0098] 2.1 Extraction and parsing of data fields

[0099] Parse the raw data packets recorded in 1.1 into an easy-to-read format (such as JSON format) to facilitate manual or automatic modification.

[0100] 2.2 Data modification function

[0101] Supports modification of any field of the data packet, for example: changing the frame ID (such as changing the standard frame ID to the extended frame ID), modifying the data payload content (such as test limit values, random data, etc.), simulating communication anomalies (such as filling error check codes).

[0102] 2.3 Automation of data modification

[0103] Use scripts to generate a large number of test data packets, including boundary value tests, random value tests, protocol incompatibility tests, etc., and generate abnormal data packets through fuzzing testing.

[0104] 3. Implementation of replaying important commands and modifying data:

[0105] The replay of important commands and data modification can be combined to meet different testing requirements:

[0106] 3.1 Condition-based Intelligent Replay

[0107] According to the response fed back by the vehicle during the replay process, the content of the replayed data packet is dynamically adjusted (such as replaying after repairing a protocol error).

[0108] 3.2 Fuzz Testing and Dynamic Adjustment

[0109] Incorporating dynamic feedback into fuzz testing to achieve an automated cycle of replay and data modification.

[0110] In some examples, the technical tools and development interfaces required to perform important command replay and data modification include:

[0111] 4.1 Technical tools used

[0112] Hardware support: such as PEAK CAN adapter, LPC development board or Ethernet interface tool;

[0113] Protocol stack: Python's python-can library is used for CAN bus operations, or custom Socket-based Ethernet protocol implementation;

[0114] Database support: used to store and manage data packages and feedback during the test process.

[0115] 4.2 Open Interface

[0116] Provides API interfaces for replay and modification, supporting developers to perform customized operations at a higher level.

[0117] In some examples, M3 performs fuzzing tests on the test vehicle according to the target test configuration, and then the M4 module records all test feedback data, especially the positive and negative feedback fuzzing commands, and generates a vulnerability test report. The report will be displayed in the following way:

[0118] 1. Report cover

[0119] Report title (e.g. “Automobile Communication Protocol Vulnerability Testing Report”);

[0120] Test project name;

[0121] Testing time period;

[0122] Name of the testing organization or testing team;

[0123] Report writers and reviewers;

[0124] Version number and release date.

[0125] 2. Summary

[0126] A test overview that describes the purpose and scope of the test.

[0127] An overview of the testing approach, which is used to explain which testing techniques were used (such as black box testing, white box testing, fuzz testing, etc.).

[0128] The test result summary is used to summarize the overall test results, such as how many vulnerabilities were found and the distribution of the severity of the vulnerabilities.

[0129] 3. Test Scope

[0130] Target systems and devices, including vehicle models, modules (such as ECU, TCU, IVI, etc.), and protocols (such as CAN protocol, DOIP protocol, LIN protocol).

[0131] Test scope is used to describe the functional or regional scope of the test, for example: communication protocol vulnerability testing (such as CAN protocol injection, DOIP protocol unauthorized access), network security testing (such as TSP, APP interface), OTA security assessment.

[0132] The untested part is used to explain the contents outside the test scope and the reasons.

[0133] As can be seen from the above, the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerability of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, the target test configuration is determined based on the configuration information and communication protocol of the vehicle to be tested; the vehicle to be tested is fuzzy tested according to the target test configuration, so as to determine whether there is a vulnerability in the vehicle function by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, and then generate a vulnerability detection result based on the test results of all vehicle functions.

[0134] In some possible implementation examples, combined with Figure 1 ,like Figure 2 As shown, the above S12 can be specifically implemented through the following S120-S123.

[0135] S120: Determine at least one theoretical test configuration based on the configuration information and communication protocol of the vehicle to be tested.

[0136] S121. Calculate the evaluation index of each theoretical test configuration.

[0137] S122. Based on the evaluation indicator, determine the evaluation score of the theoretical test configuration corresponding to the evaluation indicator.

[0138] S123. Use the theoretical test configuration corresponding to the maximum evaluation score as the target test configuration.

[0139] From the above, it can be seen that the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerabilities of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, based on the configuration information and communication protocol of the vehicle to be tested, at least one theoretical test configuration is determined. The evaluation index of each theoretical test configuration is calculated. Based on the evaluation index, the evaluation score of the theoretical test configuration corresponding to the evaluation index is determined. The theoretical test configuration corresponding to the maximum evaluation score is used as the target test configuration; the vehicle to be tested is fuzzy tested according to the target test configuration, so that by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, it is possible to determine whether there are vulnerabilities in the vehicle function, and then generate vulnerability detection results based on the test results of all vehicle functions.

[0140] In some possible implementation examples, combined with Figure 2 ,like Figure 3 As shown, the above S120 can be specifically implemented through the following S1200.

[0141] S1200: Based on the configuration information and communication protocol of the vehicle to be tested, query the test configuration relationship and determine at least one theoretical test configuration.

[0142] From the above, it can be seen that the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerabilities of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, based on the configuration information and communication protocol of the vehicle to be tested, the test configuration relationship is queried to determine at least one theoretical test configuration. The evaluation index of each theoretical test configuration is calculated. Based on the evaluation index, the evaluation score of the theoretical test configuration corresponding to the evaluation index is determined. The theoretical test configuration corresponding to the maximum evaluation score is used as the target test configuration; the vehicle to be tested is fuzzy tested according to the target test configuration, so that by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, it is possible to determine whether there are vulnerabilities in the vehicle function, and then generate vulnerability detection results based on the test results of all vehicle functions.

[0143] In some feasible examples, the configuration information includes functional modules, and the evaluation index includes test coverage; Figure 2 ,like Figure 4 As shown, the above S121 can be specifically implemented through the following S1210-S1211.

[0144] S1210. For each theoretical test configuration, obtain a first total number of communication protocols tested by the theoretical test configuration and a second total number of functional modules tested.

[0145] In some examples, combined with the example given in S12 above, the communication protocol tested by the theoretical test configuration 1 is the communication protocol 1, so the first total is equal to 1. Similarly, the functional modules tested by the theoretical test configuration 1 include the vehicle function 1 and the vehicle function 2, so the second total is equal to 2.

[0146] In some examples, one functional module corresponds to one vehicle function.

[0147] S1211. Obtain test coverage based on the first total, the second total, the third total of the communication protocols supported by all vehicles stored in the database, and the fourth total of the functional modules of all vehicles stored in the database.

[0148] In some examples, the database of the electronic device stores at least one communication protocol supported by the vehicle and the functional modules of the vehicle. By summarizing the communication protocols supported by all vehicles in the database, the first actual total number corresponding to the protocol types of the communication protocols supported by all vehicles can be obtained, and then the first actual total number is used as the third total number. Similarly, by summarizing the functional modules supported by all vehicles in the database, the second actual total number corresponding to the module types of the functional modules supported by all vehicles can be obtained, and then the second actual total number is used as the fourth total number.

[0149] In some examples, the test coverage is equal to a ratio of a sum of the first total and the second total to a sum of the third total and the fourth total.

[0150] As can be seen from the above, the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerability of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, at least one theoretical test configuration is determined based on the configuration information and communication protocol of the vehicle to be tested. For each theoretical test configuration, the first total number of communication protocols tested by the theoretical test configuration and the second total number of functional modules tested are obtained. Based on the first total number, the second total number, the third total number of communication protocols supported by all vehicles stored in the database, and the fourth total number of functional modules of all vehicles stored in the database, the test coverage is obtained. Based on the evaluation index, the evaluation score of the theoretical test configuration corresponding to the evaluation index is determined. The theoretical test configuration corresponding to the maximum evaluation score is used as the target test configuration; the vehicle to be tested is fuzzy tested according to the target test configuration, so that by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, it is possible to determine whether there are vulnerabilities in the vehicle function, and then generate vulnerability detection results based on the test results of all vehicle functions.

[0151] In some feasible examples, the evaluation indicators include test time, resource utilization, test safety, and error detection capability. Resource utilization includes computing resource utilization and network bandwidth utilization. Figure 2 ,like Figure 5 As shown, the above S121 can be specifically implemented through the following S1212 and S1213.

[0152] S1212. For each theoretical test configuration, obtain theoretical information of each test instruction in the theoretical test configuration; wherein the theoretical information includes one or more of theoretical time consumption, theoretical computing resource occupancy, theoretical network bandwidth occupancy, theoretical security and theoretical detection capability.

[0153] In some examples, the theoretical time required to execute different test instructions is pre-configured in the memory of the electronic device. In this case, when the test indicator is the test time, the test time is obtained by obtaining the sum of the theoretical time required to execute all the test instructions in the theoretical test configuration.

[0154] In some examples, the theoretical computing resource occupancy rates for executing different test instructions are pre-configured in the memory of the electronic device. At this time, when the test indicator is the computing resource utilization rate, the maximum theoretical computing resource occupancy rate among the theoretical computing resource occupancy rates corresponding to all the test instructions in the theoretical test configuration is used as the computing resource utilization rate.

[0155] In some examples, the theoretical network bandwidth occupancy rates for executing different test instructions are pre-configured in the memory of the electronic device. In this case, when the test indicator is the network bandwidth usage rate, the maximum theoretical network bandwidth occupancy rate among the theoretical network bandwidth occupancy rates corresponding to all the test instructions in the theoretical test configuration is used as the network bandwidth occupancy rate.

[0156] In some examples, the theoretical safety corresponding to executing different test instructions is pre-configured in the memory of the electronic device; wherein the theoretical safety is used to indicate whether the test instruction will cause irreversible damage to the vehicle when executed by the vehicle, such as a command to shut down the vehicle computer, etc.; at this time, when the test indicator is test safety, by obtaining the theoretical safety corresponding to all the test instructions in the theoretical test configuration, if there are test instructions that will cause irreversible damage to the vehicle, the theoretical safety of the theoretical test configuration is determined to be unsafe; if there are no test instructions that will cause irreversible damage to the vehicle, and there are test instructions that will not cause irreversible damage to the vehicle, the theoretical safety of the theoretical test configuration is determined to be safe.

[0157] In some examples, theoretical detection capabilities corresponding to executing different test instructions are pre-configured in the memory of the electronic device; wherein theoretical safety is used to indicate that more potential problems (such as communication anomalies, protocol mismatch, etc.) can be discovered when the test instructions are executed by the vehicle; at this time, when the test indicator is error detection capability, by obtaining the theoretical detection capabilities corresponding to all the test instructions in the theoretical test configuration, if there are test instructions that can discover more potential problems, the theoretical detection capability is determined to be detectable; if there are no test instructions that can discover more potential problems, the theoretical detection capability is determined to be undetectable.

[0158] S1213. Obtain evaluation indicators based on theoretical information of all test instructions in the theoretical test configuration.

[0159] From the above, it can be seen that the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerability of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, based on the configuration information and communication protocol of the vehicle to be tested, at least one theoretical test configuration is determined. For each theoretical test configuration, the theoretical information of each test instruction in the theoretical test configuration is obtained; based on the theoretical information of all test instructions in the theoretical test configuration, an evaluation index is obtained. Based on the evaluation index, an evaluation score of the theoretical test configuration corresponding to the evaluation index is determined. The theoretical test configuration corresponding to the maximum evaluation score is used as the target test configuration; according to the target test configuration, a fuzzy test is performed on the vehicle to be tested, so that by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, it is possible to determine whether there are vulnerabilities in the vehicle function, and then generate vulnerability detection results based on the test results of all vehicle functions.

[0160] In some possible implementation examples, combined with Figure 2 ,like Figure 6 As shown, the above S122 can be specifically implemented through the following S1220 and S1221.

[0161] S1220. Based on the evaluation indicators, query the score relationship table to determine the theoretical score of each evaluation indicator.

[0162] In some examples, the score relationship table is shown in Table 2.

[0163] Table 2

[0164]

[0165] In this way, we can query Table 2 and determine the theoretical score based on the evaluation indicators of test coverage, test efficiency, test safety, resource utilization and error detection capability.

[0166] S1221. Determine the evaluation score of the theoretical test configuration corresponding to the evaluation indicator based on the sum of the products of the weight value corresponding to each evaluation indicator and the theoretical score of each evaluation indicator.

[0167] In some examples, different evaluation indicators have different corresponding weight values, such as the weight value corresponding to test coverage is w1, the weight value corresponding to test efficiency is w2, the weight value corresponding to test security is w3, the weight value corresponding to resource utilization is w4, and the weight value corresponding to error detection capability is w5. At this time, combined with the example given in S1220 above, if the theoretical score of test coverage is A, the theoretical score of test efficiency is D, the theoretical score of test security is 0, the theoretical score of computing resource utilization in resource utilization is I, the theoretical score of network bandwidth occupancy in resource utilization is L, and the theoretical score of error detection capability is 1, then the evaluation score is equal to A×w1+D×w2+0×w3+I×w4+L×w4+1×w5.

[0168] As can be seen from the above, the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerability of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, based on the configuration information and communication protocol of the vehicle to be tested, at least one theoretical test configuration is determined. The evaluation index of each theoretical test configuration is calculated. Based on the evaluation index, the score relationship table is queried to determine the theoretical score of each evaluation index. Based on the sum of the product of the weight value corresponding to each evaluation index and the theoretical score of each evaluation index, the evaluation score of the theoretical test configuration corresponding to the evaluation index is determined. The theoretical test configuration corresponding to the maximum evaluation score is used as the target test configuration; the fuzzy test is performed on the vehicle to be tested according to the target test configuration, so that by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, it is possible to determine whether there are vulnerabilities in the vehicle function, and then generate vulnerability detection results based on the test results of all vehicle functions.

[0169] In some possible implementation examples, combined with Figure 2 ,like Figure 7 As shown, the above S122 can be specifically implemented through the following S1220 and S1222.

[0170] S1220. Based on the evaluation indicators, query the score relationship table to determine the theoretical score of each evaluation indicator.

[0171] In some examples, in combination with the example given in S1220 above, the theoretical score of each evaluation indicator can be determined by querying Table 2.

[0172] S1222. Determine the evaluation score of the theoretical test configuration corresponding to the evaluation indicator based on the sum of all theoretical scores.

[0173] In some examples, combined with the example given in S1220 above, if the theoretical score of test coverage is A, the theoretical score of test efficiency is D, the theoretical score of test security is 0, the theoretical score of computing resource utilization in resource utilization is I, the theoretical score of network bandwidth occupancy in resource utilization is L, and the theoretical score of error detection capability is 1, then the evaluation score is equal to A+D+0+I+L+1.

[0174] From the above, it can be seen that the test method provided by the embodiment of the present disclosure, when it is necessary to test the vulnerabilities of the vehicle to be tested, obtains the communication protocol supported by the vehicle to be tested by establishing a connection with the vehicle to be tested. Afterwards, based on the configuration information and communication protocol of the vehicle to be tested, determine at least one theoretical test configuration. Calculate the evaluation index of each theoretical test configuration. Based on the evaluation index, query the score relationship table to determine the theoretical score of each evaluation index; based on the sum of all theoretical scores, determine the evaluation score of the theoretical test configuration corresponding to the evaluation index. Take the theoretical test configuration corresponding to the maximum evaluation score as the target test configuration; perform fuzzy testing on the vehicle to be tested according to the target test configuration, so as to determine whether there are vulnerabilities in the vehicle function by collecting the corresponding feedback information when the vehicle to be tested executes each test instruction, and then generate vulnerability detection results based on the test results of all vehicle functions.

[0175] Embodiment 2

[0176] Figure 8 The structural diagram of the testing device provided in the second embodiment of the present application is shown in FIG. Figure 8 As shown, the testing device includes: a processing module 80 and an acquisition module 81 .

[0177] The processing module 80 is used to control the acquisition module 81 to acquire the communication protocol supported by the vehicle to be tested when establishing a connection with the vehicle to be tested;

[0178] The processing module 80 is further used to determine a target test configuration based on the configuration information of the vehicle to be tested and the communication protocol obtained by the obtaining module 81; wherein the target test configuration includes at least one test instruction, and one test instruction corresponds to one vehicle function;

[0179] The processing module 80 is further used to perform fuzzy testing on the vehicle to be tested according to the target test configuration and generate vulnerability detection results.

[0180] In some implementable examples, the processing module 80 is specifically used to determine at least one theoretical test configuration based on the configuration information and communication protocol of the vehicle to be tested; the processing module 80 is specifically used to calculate the evaluation index of each theoretical test configuration; the processing module 80 is specifically used to determine the evaluation score of the theoretical test configuration corresponding to the evaluation index based on the evaluation index; the processing module 80 is specifically used to take the theoretical test configuration corresponding to the maximum evaluation score as the target test configuration.

[0181] In some implementable examples, the processing module 80 is specifically configured to query the test configuration relationship and determine at least one theoretical test configuration based on the configuration information and communication protocol of the vehicle to be tested.

[0182] In some implementable examples, the configuration information includes functional modules, and the evaluation indicators include test coverage; the processing module 80 is specifically used to obtain, for each theoretical test configuration, a first total number of communication protocols tested by the theoretical test configuration and a second total number of functional modules tested; the processing module 80 is specifically used to obtain the test coverage based on the first total number, the second total number, a third total number of communication protocols supported by all vehicles stored in the database, and a fourth total number of functional modules of all vehicles stored in the database.

[0183] In some implementable examples, the evaluation indicators include test time, resource utilization, test security, and error detection capability, and the resource utilization includes computing resource utilization and network bandwidth utilization; the processing module 80 is specifically used to obtain theoretical information of each test instruction in the theoretical test configuration for each theoretical test configuration; wherein the theoretical information includes one or more of theoretical time consumption, theoretical computing resource occupancy, theoretical network bandwidth occupancy, theoretical security, and theoretical detection capability; the processing module 80 is specifically used to obtain evaluation indicators based on the theoretical information of all test instructions in the theoretical test configuration.

[0184] In some implementable examples, the processing module 80 is specifically used to query the score relationship table based on the evaluation indicator to determine the theoretical score of each evaluation indicator; the processing module 80 is specifically used to determine the evaluation score of the theoretical test configuration corresponding to the evaluation indicator based on the sum of the product of the weight value corresponding to each evaluation indicator and the theoretical score of each evaluation indicator.

[0185] In some implementable examples, the processing module 80 is specifically used to query the score relationship table based on the evaluation indicator to determine the theoretical score of each evaluation indicator; the processing module 80 is specifically used to determine the evaluation score of the theoretical test configuration corresponding to the evaluation indicator based on the sum of all theoretical scores.

[0186] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.

[0187] Of course, the electronic device provided by the embodiment of the present invention includes but is not limited to the above modules, for example, the electronic device may also include a storage module 82. The storage module 82 may be used to store program codes of the electronic device, and may also be used to store data generated during operation of the electronic device, such as diagnostic data.

[0188] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Fig. 9 As shown, the electronic device may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .

[0189] Combine the following Fig. 9 A detailed introduction to the various components of electronic equipment:

[0190] The processor 51 is the control center of the electronic device, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0191] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Fig. 9 Also, as an embodiment, the electronic device may include multiple processors, such as Fig. 9 51 and 55 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0192] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0193] In a specific implementation, the memory 52 is used to store the data of the present invention and execute the software program of the present invention. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.

[0194] The communication interface 53 uses any transceiver-like device to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, the cloud, etc. The communication interface 53 may include an acquisition module to implement the acquisition function.

[0195] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0196] As an example, combining Figure 8The function implemented by the acquisition module 81 of the test device is similar to Fig. 9 The function of the communication interface 53 in the test device is the same as that of the processing module 80 in the test device. Fig. 9 The function of the processor 51 in the test device is the same as that of the storage module 82 in the test device. Fig. 9 The function of the memory 52 in is the same.

[0197] An embodiment of the present application also provides an electronic device, which may include the electronic device in any embodiment.

[0198] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method in any embodiment.

[0199] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A testing method, characterized in that: include: When establishing a connection with the vehicle to be tested, obtaining a communication protocol supported by the vehicle to be tested; Determine a target test configuration based on the configuration information of the vehicle to be tested and the communication protocol; wherein the target test configuration includes at least one test instruction, and one test instruction corresponds to one vehicle function; Perform fuzz testing on the vehicle to be tested according to the target test configuration to generate a vulnerability detection result.

2. The testing method according to claim 1, characterized in that: The determining of a target test configuration based on the configuration information of the vehicle to be tested and the communication protocol includes: Determining at least one theoretical test configuration based on the configuration information of the vehicle to be tested and the communication protocol; Calculating evaluation indicators for each of the theoretical test configurations; Based on the evaluation indicator, determining an evaluation score of the theoretical test configuration corresponding to the evaluation indicator; The theoretical test configuration corresponding to the maximum evaluation score is taken as the target test configuration.

3. The testing method according to claim 2, characterized in that: The determining at least one theoretical test configuration based on the configuration information of the vehicle to be tested and the communication protocol comprises: Based on the configuration information of the vehicle to be tested and the communication protocol, a test configuration relationship is queried to determine at least one theoretical test configuration.

4. The testing method according to claim 2, characterized in that: The configuration information includes functional modules, and the evaluation index includes test coverage; The calculating of the evaluation index of each theoretical test configuration comprises: For each of the theoretical test configurations, obtaining a first total number of communication protocols tested by the theoretical test configuration and a second total number of functional modules tested; The test coverage is obtained based on the first total, the second total, a third total of the communication protocols supported by all vehicles stored in the database, and a fourth total of the functional modules of all vehicles stored in the database.

5. The testing method according to claim 2, characterized in that: The evaluation indicators include test time, resource utilization, test security, and error detection capability. The resource utilization includes computing resource utilization and network bandwidth utilization. The calculating of the evaluation index of each theoretical test configuration comprises: For each of the theoretical test configurations, obtain theoretical information of each test instruction in the theoretical test configuration; wherein the theoretical information includes one or more of theoretical time consumption, theoretical computing resource occupancy, theoretical network bandwidth occupancy, theoretical security and theoretical detection capability; Based on the theoretical information of all the test instructions in the theoretical test configuration, an evaluation index is obtained.

6. The testing method according to claim 2, characterized in that: The step of determining, based on the evaluation indicator, an evaluation score of the theoretical test configuration corresponding to the evaluation indicator comprises: Based on the evaluation indicators, query the score relationship table to determine the theoretical score of each evaluation indicator; The evaluation score of the theoretical test configuration corresponding to the evaluation indicator is determined based on the sum of the products of the weight value corresponding to each evaluation indicator and the theoretical score of each evaluation indicator.

7. The testing method according to claim 2, characterized in that: The step of determining, based on the evaluation indicator, an evaluation score of the theoretical test configuration corresponding to the evaluation indicator comprises: Based on the evaluation indicators, query the score relationship table to determine the theoretical score of each evaluation indicator; Based on the sum of all the theoretical scores, an evaluation score of the theoretical test configuration corresponding to the evaluation indicator is determined.

8. A testing device, characterized in that: include: A processing module, configured to control an acquisition module to acquire a communication protocol supported by the vehicle to be tested when establishing a connection with the vehicle to be tested; The processing module is further used to determine a target test configuration based on the configuration information of the vehicle to be tested and the communication protocol acquired by the acquisition module; wherein the target test configuration includes at least one test instruction, and one test instruction corresponds to one vehicle function; The processing module is further used to perform fuzzy testing on the vehicle to be tested according to the target test configuration to generate a vulnerability detection result.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the test method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the testing method according to any one of claims 1 to 7.

Images