Vehicle-mounted communication middleware testing method and system and computer program product

Abstract

The invention provides a vehicle-mounted communication middleware test method and system and a computer program product, and relates to the technical field of vehicles, the method can perform data deformation on a first communication message generated in an initialization test to obtain a second communication message, and the second communication message is used for performing a fuzzy test to obtain a fuzzy test result; according to the method, the first communication message of the vehicle-mounted communication middleware is subjected to fuzz testing, and then the second communication message of the fuzz testing is subjected to data deformation to generate a new second communication message, so that the test sample is automatically generated, the automatically generated second communication message is subjected to data deformation, various functions of the vehicle-mounted communication middleware can be tested, and the test efficiency and the test coverage rate of the vehicle-mounted communication middleware are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a vehicle communication middleware testing method, system and computer program product. Background Art

[0002] With the continuous development of automotive electronic and electrical systems, vehicle networks are becoming increasingly complex, and the communication needs between vehicle systems are also increasing. As a key part of the Service-Oriented Architecture (SOA) in automobiles, vehicle communication middleware provides core functions such as communication, data sharing, and remote procedure calls for upper-layer applications, achieving seamless integration between vehicle systems and is an important infrastructure to support automotive intelligence.

[0003] In order to ensure the normal operation of the vehicle-mounted middleware, the vehicle-mounted middleware can be tested. The testing of vehicle-mounted middleware mainly relies on manually constructed test cases, which require a lot of human resources to construct. In addition, the coverage of manually constructed test cases is limited, the number of abnormal scenarios covered is small, and it is easy to produce test blind spots, resulting in poor test results. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a vehicle communication middleware testing method, system and computer program product.

[0005] In a first aspect, an embodiment of the present disclosure provides a vehicle communication middleware testing method, including:

[0006] In the process of obtaining the initialization test, a first communication message transmitted by the vehicle-mounted communication middleware to be tested is used, and the first communication message is stored in a sample queue as a test sample;

[0007] Extracting a target test sample from the sample queue, and performing data deformation on the target test sample to obtain a second communication message corresponding to the target test sample;

[0008] Performing a fuzzy test on the in-vehicle communication middleware using the second communication message, and determining test information corresponding to the fuzzy test;

[0009] Determine whether the test information satisfies the test termination condition, and if so, terminate the test; if not, store the second communication message as a test sample in the sample queue, and repeat the extraction step, the fuzzy test step, and the judgment step.

[0010] In an optional implementation manner, the number of the sample queues is at least two; different sample queues are matched with different priority levels;

[0011] The storing the second communication message as a test sample in the sample queue includes:

[0012] Determining a priority level of the second communication message based on the test information corresponding to the second communication message;

[0013] storing the second communication message in a sample queue matching the priority level;

[0014] The step of extracting a target test sample from the sample queue comprises:

[0015] Determining a target sample queue from the at least two sample queues based on the sample extraction frequencies corresponding to the priority levels;

[0016] A target test sample is extracted from the target sample queue.

[0017] In an optional implementation manner, the priority level of the second communication message is determined according to at least one of the following:

[0018] A first test code coverage indicated by the test information and a second test code coverage before the fuzz test;

[0019] The test information indicates whether an error message occurs;

[0020] The test information indicates first test code path information and second test code path information before the fuzz test.

[0021] In an optional implementation manner, the performing data deformation on the target test sample includes:

[0022] Randomly determine a first target byte for data deformation from the binary coded text of the target test sample, and perform a byte deformation operation on the first target byte to obtain the second communication message; the deformation operation includes at least one of byte flipping, adding, deleting and replacing; and / or,

[0023] Based on the semantic structure of the target test sample, a second target byte with target semantics is determined, and a semantic deformation operation is performed on the second target byte to obtain the second communication message.

[0024] In an optional implementation manner, performing a semantic deformation operation on the second target byte to obtain the second communication message includes:

[0025] Determining a target working scenario of the vehicle-mounted communication middleware to be tested;

[0026] Based on the target working scenario, a semantic deformation operation is performed on the second target byte to obtain a second communication message matching the target working scenario.

[0027] In an optional implementation manner, determining the test information corresponding to the fuzzy test includes:

[0028] Acquiring execution status information of a plurality of insertion positions in the fuzzy test; the insertion positions are set in the vehicle-mounted communication middleware and a system corresponding to the vehicle-mounted communication middleware;

[0029] Based on the execution status information, determining whether the vehicle-mounted communication middleware has abnormal behavior;

[0030] In the case where the vehicle-mounted communication middleware has abnormal behavior, determining attribute information corresponding to the abnormal behavior based on the execution state information;

[0031] Based on the execution status information and the attribute information corresponding to the abnormal behavior, test information corresponding to the fuzzy test is determined.

[0032] In an optional implementation manner, the method further includes:

[0033] Based on the attribute information corresponding to the abnormal behavior, the abnormal behavior is classified, and cause information and impact information of the abnormal behavior are generated.

[0034] In an optional implementation manner, the method further includes:

[0035] Building a system platform matching the vehicle communication middleware to be tested;

[0036] Obtain a test application sample, and build an operating environment corresponding to the test application sample on the system platform;

[0037] Run the test application sample in the running environment to perform an initialization test.

[0038] In a second aspect, an embodiment of the present disclosure further provides a vehicle-mounted communication middleware testing system, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the vehicle-mounted communication middleware testing system is running, the machine-readable instructions are executed by the processor to perform the steps of the vehicle-mounted communication middleware testing method described in the first aspect.

[0039] In a third aspect, an optional implementation of the present disclosure further provides a computer program product, including a computer program, which implements the steps of the method described in the first aspect when the computer program is executed.

[0040] For a description of the effects of the above-mentioned vehicle communication middleware testing system and computer program product, please refer to the description of the above-mentioned vehicle communication middleware testing method, which will not be repeated here.

[0041] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure.

[0042] The vehicle communication middleware testing method, system and computer program product provided by the embodiments of the present disclosure can perform data deformation on the first communication message generated in the initialization test to obtain a second communication message, use the second communication message to perform fuzzy testing, and then use the second communication message of the fuzzy test to perform data deformation to generate a new second communication message, thereby realizing automatic generation of test samples. The automatically generated second communication message can test various functions of the vehicle communication middleware after data deformation, thereby improving the test efficiency and test coverage of the vehicle communication middleware.

[0043] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0045] Figure 1 A flowchart of a vehicle communication middleware testing method provided by some embodiments of the present disclosure is shown;

[0046] Figure 2 A schematic diagram showing a step of obtaining a first communication message provided in some embodiments of the present disclosure;

[0047] Figure 3 A schematic diagram showing steps of generating a second communication message provided in some embodiments of the present disclosure is shown;

[0048] Figure 4 A schematic diagram of a vehicle communication middleware testing system provided by some embodiments of the present disclosure is shown;

[0049] Figure 5 A schematic diagram of another vehicle communication middleware testing system provided by some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown herein can be arranged and involved in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0051] Research has found that the testing of in-vehicle middleware mainly relies on manually constructed test cases. Constructing these test cases requires a lot of human resources. In addition, the coverage of manually constructed test cases is limited, the number of abnormal scenarios covered is small, and testing blind spots are easily generated, resulting in poor testing results.

[0052] Based on the above research, the present disclosure provides a vehicle-mounted communication middleware testing method, system and computer program product, which can perform data deformation on the first communication message generated in the initialization test to obtain a second communication message, use the second communication message to perform fuzzy testing, and then use the second communication message of the fuzzy test to perform data deformation to generate a new second communication message, thereby realizing automatic generation of test samples. The automatically generated second communication message can test various functions of the vehicle-mounted communication middleware after data deformation, thereby improving the test efficiency and test coverage of the vehicle-mounted communication middleware.

[0053] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0054] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0055] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the personal information involved in the present disclosure (including but not limited to attribute information, facial images, etc.) is obtained with the authorization of the user. Specifically, it can be through a pop-up window in the page, information push, etc., sending a prompt message requesting authorization to the user, and obtaining the above personal information after the user agrees to the authorization.

[0056] To facilitate understanding of this embodiment, a vehicle communication middleware testing method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the vehicle communication middleware testing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the vehicle communication middleware testing method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0057] See also Figure 1 FIG. 1 is a flow chart of a vehicle communication middleware testing method provided by an embodiment of the present disclosure, and the method includes:

[0058] S101. During the process of obtaining an initialization test, a first communication message transmitted by the vehicle-mounted communication middleware to be tested is used, and the first communication message is stored in a sample queue as a test sample.

[0059] In this step, the initialization test may refer to an initial communication test performed using a test application sample. The test application sample may include a test application client and a test application server, and the test application client and the test application server may communicate and transmit communication messages. When the test application client and the test application server communicate, the vehicle communication middleware may be used to transmit the communication messages.

[0060] In-vehicle middleware refers to the software layer that provides services and functions between the vehicle's hardware and application software. It enables applications to be developed more easily by abstracting the complexity of the underlying hardware. Middleware is responsible for managing communications within the vehicle, processing and distribution of data, and ensuring compatibility and efficiency between different components.

[0061] As part of the automotive service-oriented architecture (SOA), vehicle communication middleware can provide core functions such as communication, data sharing, and remote procedure calls for upper-level applications, achieving seamless integration between vehicle systems and serving as an important infrastructure to support automotive intelligence.

[0062] Service-oriented architecture (SOA) is a design pattern in which system components provide independent services to each other through the network. These services can be regarded as independent functional units that interact with each other through well-defined interfaces and contracts. In the context of in-vehicle communication middleware, SOA allows different parts of the vehicle system to flexibly communicate and interoperate with each other. This architecture supports the rapid integration of new services and the updating of existing services without affecting other parts of the system.

[0063] In the disclosed embodiment, the automobile may deploy a distributed system, a system consisting of multiple computer components connected via a network, which work together to complete a common set of tasks. In the context of in-vehicle middleware, a distributed system may refer to multiple electronic control units (ECUs) or domain controllers working together across the vehicle to share resources and process information. This architecture can improve the reliability and scalability of the system because it allows the system to partially perform fault isolation and load balancing.

[0064] The above test application examples can be deployed on different ECUs or domain controllers. For example, see Figure 2 As shown, it is a schematic diagram of the steps of obtaining the first communication message provided in an embodiment of the present disclosure. Figure 2 In the example, the test application client (sample_client) can run in the smart cabin domain, the test application server (sample_server) can run in the smart driving domain, and the test application server and the test application client can transmit data using the on-board communication middleware. By capturing the transmitted data, the first communication message can be obtained.

[0065] In vehicle systems, a domain controller refers to an electronic control unit (ECU) that manages a specific functional domain (such as powertrain, chassis, smart cockpit, smart driving, etc.). The domain controller is responsible for data processing, device control, and communication within the functional domain it governs. Through centralized management, the domain controller can optimize performance and security and ensure efficient data exchange and coordinated operations between various subsystems.

[0066] The above-mentioned test application samples can be developed based on the target communication protocol and vehicle communication middleware used by the vehicle. In order to fully test the vehicle communication middleware, the test application samples can cover the functions and boundary conditions of the vehicle communication middleware as much as possible and simulate various normal and abnormal communication behaviors.

[0067] During the running of the test application sample, a network packet capture tool can be used to capture the communication data of the test application sample in real time. The network packet capture tool can capture each request and response between sample programs and save it as specific structure data (message data).

[0068] In a possible implementation manner, the number of the first communication messages may be one or more.

[0069] The first communication message can be played back to the sample program through the network card, and the playback process can be called message playback. Message playback refers to resending previously recorded network communication or system operation data messages to the system during the test to observe the system's response and processing capabilities to old data. This technology is often used to evaluate the robustness and error handling capabilities of the system, and is particularly useful when simulating attack or fault recovery scenarios.

[0070] It is worth noting that new messages can also be played during the message playback process. During the playback of new messages, the system can respond according to the new messages, thereby obtaining new response data. In this way, the new messages can be played back to test the vehicle communication middleware.

[0071] In order to generate a new message, the embodiment of the present disclosure may use the first communication message as a test sample, store it in a sample queue, and use the test sample in the sample queue to generate a new message for testing.

[0072] The number of the sample queues may be at least two, and the priority levels matched by different sample queues may be different. The priority levels may be used to determine the frequency of extracting test samples from the corresponding sample queues.

[0073] For example, the number of the sample queues may be two, and the corresponding priority levels may include high and low. The frequency of extracting test samples from the sample queue with a high priority level may be higher than the frequency of extracting test samples from the sample queue with a low priority level. For example, after extracting multiple test samples from the sample queue with a high priority level continuously, one test sample may be extracted from the sample queue with a low priority level.

[0074] When storing the first communication message in the sample queue, the first communication message can be directly stored in the sample queue of the target priority level, such as a sample queue with a high priority level, or the sample queue to which the first communication message is assigned can be determined based on the attribute information corresponding to the first communication message, such as importance, test function type, etc.

[0075] In one possible implementation,

[0076] S102: extract a target test sample from the sample queue, and perform data deformation on the target test sample to obtain a second communication message corresponding to the target test sample.

[0077] After the first communication message is stored in the sample queue, the target test sample can be extracted from the sample queue. Specifically, the target sample queue from which the test sample should be extracted can be determined from at least two sample queues according to the sample extraction frequencies corresponding to each priority level, and then the target test sample can be extracted from the target sample queue.

[0078] The target test sample may be the test sample that is first dequeued according to the dequeuing order of the target sample queue.

[0079] The dequeue order of the sample queue may be a first-in-first-out queue. Generally, the test sample may be directly written at the end of the sample queue. If the test sample carries an expedited tag, the test sample may be written to the head or middle of the sample queue according to the expedited tag.

[0080] After obtaining the target test sample, data deformation may be performed on the target test sample to form a new test case, that is, a second communication message.

[0081] When data deformation is performed on the target test sample, the binary coded text of the target test sample may be directly randomly deformed, or semantic deformation may be performed based on the semantic structure of the target test sample.

[0082] Specifically, a first target byte for data deformation can be randomly determined from the binary coded text of the target test sample, and a byte deformation operation is performed on the first target byte to obtain the second communication message; the deformation operation includes at least one of byte flipping, adding, deleting and replacing; and / or,

[0083] Based on the semantic structure of the target test sample, a second target byte with target semantics is determined, and a semantic deformation operation is performed on the second target byte to obtain the second communication message.

[0084] The binary coded text may refer to the text of the target test sample in a binary language that can be understood by a computer, which may be a direct representation of the target test sample during data transmission.

[0085] When determining the first target byte in the binary-coded text, a segment of bytes can be randomly selected from the binary-coded text and used as the first target byte. The number of characters in the first target byte can be one or more. By performing byte deformation operations on it, a randomly deformed second communication message can be obtained.

[0086] Since the target test sample is written as binary-encoded text, the specific meaning of the first target byte of the excerpt cannot be known. This variation of the method can effectively generate new test cases and easily generate abnormal messages, thereby testing the ability of the vehicle communication middleware to deal with abnormal data.

[0087] When determining the semantic structure of the target test sample, the binary-coded text can be decompiled to obtain the target test sample of the computer programming language. After that, the target test sample of the computer programming language can be determined, and the meaning of each part of the text can be determined, so as to select the second target byte with a specific meaning, perform a semantic deformation operation on the second target byte, and obtain a second communication message.

[0088] In this step, the second target byte to be transformed and the content of the second target byte after the transformation can be determined based on the rules, templates, and states.

[0089] The rule-based data deformation method can be based on a deep understanding of the vehicle communication protocol. According to the field definition of the protocol, some key fields can be modified purposefully, such as setting the length field to an abnormal value, or modifying the identifier field to test the system's ability to handle unknown or non-compliant data.

[0090] Based on the template-driven data deformation method, defined templates can be used to guide the construction of data packets (communication messages). These templates are based on typical data packets in normal communication processes, but are selectively modified on this basis to generate potential abnormal situations.

[0091] The state-based data deformation method can take into account that the vehicle communication middleware may respond differently according to different states, and the mutation also needs to take into account the state of the middleware. For example, by simulating the scenario of packet loss or reordering, the middleware's ability to handle communication interruption and recovery can be tested.

[0092] Exemplarily, a target working scenario to be tested for the in-vehicle communication middleware may be determined, and based on the target working scenario, a semantic deformation operation may be performed on the second target byte to obtain a second communication message matching the target working scenario.

[0093] S103: Perform a fuzzy test on the in-vehicle communication middleware using the second communication message, and determine test information corresponding to the fuzzy test.

[0094] In this step, the second communication message may be replayed using a message replay mechanism, thereby performing a fuzzy test on the vehicle-mounted communication middleware.

[0095] Fuzz testing is an automated software testing method that triggers abnormal program behavior and discovers potential defects by inputting various randomly constructed invalid data (such as format errors, boundary values, etc.) into the target program.

[0096] Compared with traditional testing methods, fuzz testing has the following advantages: high degree of automation, which can generate massive test cases on a large scale and save manpower; and, the test cases cover a wide range, making it easier to find abnormal scenarios that are difficult to reach with traditional testing; at the same time, the problems found are usually more hidden, which helps to enhance the robustness of the software. By applying fuzz testing to vehicle communication middleware testing, the defect discovery rate can be greatly improved, and the software quality and reliability can be improved.

[0097] In the disclosed embodiment, the test case of the fuzzy test can be the above-mentioned second communication message. The second communication message is generated by deforming the data of the test sample, which can effectively save human resources. The second communication message obtained by data deformation has a wider coverage and is easier to improve the test effect.

[0098] During the fuzz test, the test information corresponding to the fuzz test can also be determined. During the fuzz test, the running status of the vehicle communication middleware can be obtained in real time, and the abnormal behavior reflected by the running status can be recorded, such as program crashes, memory leaks, or deadlocks. The above information can be used as test information.

[0099] Specifically, abnormal behaviors may include system crashes, functional failures, sharp performance degradation, unexpected output results, resource leaks (such as memory leaks), security vulnerability manifestations (such as buffer overflows), etc.

[0100] The above-mentioned operating status can be obtained by plugging the vehicle communication middleware and the system corresponding to the vehicle communication middleware.

[0101] Exemplarily, the execution status information of multiple insertion positions in the fuzzy test can be obtained; the insertion positions are set in the vehicle-mounted communication middleware and the system corresponding to the vehicle-mounted communication middleware; then, based on the execution status information, it is determined whether the vehicle-mounted communication middleware has abnormal behavior; thereafter, in the case that the vehicle-mounted communication middleware has abnormal behavior, based on the execution status information, the attribute information corresponding to the abnormal behavior is determined; finally, based on the execution status information and the attribute information corresponding to the abnormal behavior, the test information corresponding to the fuzzy test is determined.

[0102] The above-mentioned insertion location may refer to the target location in the code of the vehicle communication middleware and its corresponding system. The target location may be a key execution path (such as data processing flow, exception handling mechanism, resource management, etc.). At this target location, the corresponding execution status information may be obtained through the code implanted by the insertion.

[0103] In one implementation, for the abnormal behavior of system crash, a monitoring point may be set in the system, at which it may be monitored whether a process is exited or whether a core file is generated, thereby determining whether the system crashes.

[0104] When abnormal behavior is detected, a detailed abnormal log can be automatically recorded. The abnormal log can include attribute information corresponding to the abnormal behavior, such as the time, type, and severity of the abnormality. Through the execution status information, the detailed system status and runtime data corresponding to the abnormal behavior can be determined. In this way, the test information corresponding to the fuzz test can be determined.

[0105] In a possible implementation, the test information may also include the test code coverage after fuzz testing using the second communication message. In order to determine the test code coverage, a stub may be inserted into the code of the vehicle-mounted communication middleware to determine whether a corresponding section of code is executed, thereby determining the test code coverage. By inserting a stub, it is also possible to determine the code that has not been tested, thereby guiding the subsequent strategy for generating test cases and optimizing the test strategy. Test code coverage can be used as an important indicator for judging the test effect.

[0106] When calculating the test code coverage, the compiler's behavior can be modified so that the compiler inserts the required code at key locations during compilation. The above key locations can include control flow statements such as function entry, loops, and conditional judgments. In this way, it is possible to monitor and record which basic blocks are executed, thereby generating test code coverage data. For communication middleware that is integrated in the form of SDK by integration vendors, special tools can be used to insert additional stub code at runtime to detect the code execution process and calculate coverage data.

[0107] S104, determine whether the test information satisfies the test termination condition, if so, terminate the test, if not, store the second communication message as a test sample in the sample queue, and repeat the extraction step, the fuzzy test step and the judgment step.

[0108] After performing the fuzzy test using the second communication message and obtaining the test information corresponding to the fuzzy test, it can be determined whether the test information meets the test termination condition. If so, the test can be stopped; if not, the test can continue.

[0109] When the test information does not satisfy the test termination condition, the second communication message may be stored in the sample queue as a test sample, so that a new second communication message is generated by using the second communication message, and the new second communication message is used to perform fuzzy testing.

[0110] Exemplarily, when the test information does not meet the test termination condition, the second communication message currently being tested can be first stored in the sample queue as a test sample. Then, a new target test sample is extracted from the sample queue again, and the new target test sample is data deformed to obtain a new second communication message corresponding to the target test sample. Afterwards, a new round of fuzzy testing can be performed using the new second communication message to obtain new test information corresponding to the new round of fuzzy testing. Afterwards, it can be determined whether the new test information meets the test termination condition. When the new test information meets the test termination condition, the test is stopped; when the new test information does not meet the test termination condition, the new second communication message is stored in the sample queue as a test sample, and a new target test sample is extracted from the sample queue again, and subsequent steps are executed to start a new round of testing.

[0111] In this step, when the second communication message is stored as a test sample in the sample queue, if there are at least two sample queues, the priority level of the second communication message can be determined based on the test information corresponding to the second communication message, and then the second communication message is stored in the sample queue that matches the priority level, thereby controlling the frequency of generating new second communication messages using the second communication message, thereby improving the efficiency of the test.

[0112] In this step, the priority level of the second communication message may be judged, and its priority level may be determined according to the performance of the second communication message in the fuzzy test.

[0113] Generally, taking the example of high and low priority levels, the test samples of the high priority level may be samples that help improve the test effect, and the test samples of the low priority level may be samples that do not improve the test effect. Improving the test effect may refer to improving the test code coverage, adding new test code paths, or triggering errors during the test process. Although the test samples of the low priority level do not improve the test effect, after data deformation of the test samples of the low priority level, communication messages that can improve the test effect may still be generated.

[0114] Specifically, the priority level of the second communication message may be determined according to at least one of the following information:

[0115] A first test code coverage indicated by the test information and a second test code coverage before the fuzz test;

[0116] The test information indicates whether an error message occurs;

[0117] The test information indicates first test code path information and second test code path information before the fuzz test.

[0118] Exemplarily, if the coverage rate of the first test code is higher than that of the second test code, it can be determined that the priority level of the second communication message is high, otherwise it is low; if the error information indicates that an error has occurred, it can be determined that the priority level of the second communication message is high, otherwise it is low; if the first test code path information indicated by the test information generates new path information compared to the second test code path information before the fuzzy test, it can be determined that the priority level of the second communication message is high, otherwise it is low.

[0119] Alternatively, the priority level of the second communication message can also be determined based on multiple types of the above information at the same time. For example, if the first test code coverage is higher than the second test code coverage, and whether error information indicates that an error has occurred, or new path information is generated, then it can be determined that the priority level of the second communication message is high.

[0120] In a possible implementation manner, the above information may be used to calculate a priority score, and then the priority score may be used to determine the priority level of the second communication message.

[0121] Since samples with high priority levels are extracted more frequently, test samples that can improve test results can be used preferentially. At the same time, using a lower sample extraction frequency to extract test samples with low priority levels can ensure the comprehensiveness of the test and avoid missing potentially useful test samples.

[0122] See also Figure 3 As shown, it is a schematic diagram of the steps of generating a second communication message provided in an embodiment of the present disclosure. Figure 3 In the process of extracting test samples (i.e. seeds) from sample queues based on sample extraction of each sample queue, the seeds can be mutated (i.e. data deformation), and fuzz testing can be performed using the mutated seeds. After the fuzz testing, it can be determined whether the test code coverage has increased. If it has increased, the seeds are written to the high priority queue, and if not, they are written to the low priority queue. After that, seeds can be extracted for a new round of seed mutation according to the sample extraction frequency of the sample queues (e.g., the high priority queue reads a seed each time, the low priority queue reads a seed every n times, and the high priority queue is not read when the low priority queue is read).

[0123] The test termination condition may be determined based on the expected test effect. For example, the test termination condition may include at least one of the following: the test code coverage is greater than or equal to a preset coverage; the ratio between the test code path and the total test code path is greater than or equal to a preset ratio, etc.

[0124] In a possible implementation, after obtaining the test information, the abnormal behavior is classified based on the attribute information corresponding to the abnormal behavior, and the cause information and impact information of the abnormal behavior are generated, so as to further analyze the abnormal behavior and facilitate error location and repair of the vehicle communication middleware.

[0125] In the disclosed embodiment, the final result of the test may include triggered exceptions, test case execution records, test code coverage reports, etc., which will be output in a standardized format for analysis and follow-up actions by developers or testers.

[0126] In a possible implementation, the method can be run in a virtual environment such as QEMU, and can fully utilize multi-core CPUs or cluster environments for parallel execution to improve test efficiency. At the same time, each component of the framework can be flexibly replaced or expanded to adapt to different test requirements and target systems.

[0127] Exemplarily, a system platform matching the vehicle communication middleware to be tested can be constructed in a virtual environment; then, a test application sample is obtained, and an operating environment corresponding to the test application sample is constructed on the system platform; then, the test application sample can be run in the operating environment to perform an initialization test. Subsequently, the first communication message of the initialization test can be stored in a sample queue as a test sample, and the second communication message can be generated using the test sample for fuzzy testing.

[0128] See also Figure 4 , which is a schematic diagram of a vehicle communication middleware testing system provided by an embodiment of the present disclosure. Figure 4In the present invention, the vehicle communication middleware testing system may include a data generation module, a screening and mutation module, a test execution module and an abnormal behavior detection module. The data generation module can construct a test application sample program (such as a test application client sample_client and a test application server sample_server) based on the vehicle communication middleware, and can set sample_server as the target program. At several times of communication between sample_client and sample_server, the generated communication message is captured to obtain the initial seed (that is, the first communication message), and the initial seed is stored in the seed pool (that is, the sample queue); the test execution module can build the operating environment of the first communication message and the second communication message and the analysis environment of the test code coverage, and replay the test sample through the message playback function; the screening and mutation module can store the first communication message and the second communication message completed by the test in the seed pool of the corresponding priority level through seed screening, and filter out the used seeds from the seed pool according to the priority level corresponding to the seed pool, and mutate the seeds to generate test sample cases (that is, the second communication message), and provide them to the test execution module; the abnormal behavior detection module can obtain the test information generated during the fuzzy test, and analyze it when abnormal behaviors such as crashes and memory leaks occur, obtain possible bugs, and output the test results.

[0129] The vehicle communication middleware testing method provided by the embodiment of the present disclosure can perform data deformation on the first communication message generated in the initialization test to obtain a second communication message, use the second communication message to perform fuzzy testing, and then use the second communication message of the fuzzy test to perform data deformation to generate a new second communication message, thereby realizing automatic generation of test samples. The automatically generated second communication message can test various functions of the vehicle communication middleware after data deformation, thereby improving the test efficiency and test coverage of the vehicle communication middleware.

[0130] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0131] Based on the same inventive concept, an on-vehicle communication middleware testing device corresponding to the on-vehicle communication middleware testing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned on-vehicle communication middleware testing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0132] Based on the same technical concept, the embodiment of the present disclosure also provides another vehicle communication middleware testing system. Figure 5 As shown, it is a structural diagram of another vehicle communication middleware test system 500 provided by an embodiment of the present disclosure, including a processor 501 and a memory 502. Among them, the memory 502 is used to store execution instructions, including a memory 5021 and an external memory 5022; the memory 5021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 501, and the data exchanged with the external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the memory 5021. When the vehicle communication middleware test system 500 is running, the processor 501 communicates with the memory 502 through a bus, so that the processor 501 executes the following instructions:

[0133] In the process of obtaining the initialization test, a first communication message transmitted by the vehicle-mounted communication middleware to be tested is used, and the first communication message is stored in a sample queue as a test sample;

[0134] Extracting a target test sample from the sample queue, and performing data deformation on the target test sample to obtain a second communication message corresponding to the target test sample;

[0135] Performing a fuzzy test on the in-vehicle communication middleware using the second communication message, and determining test information corresponding to the fuzzy test;

[0136] Determine whether the test information satisfies the test termination condition, and if so, terminate the test; if not, store the second communication message as a test sample in the sample queue, and repeat the extraction step, the fuzzy test step, and the judgment step.

[0137] In an optional implementation, in the instruction executed by the processor 501, the number of the sample queues is at least two; different sample queues are matched with different priority levels;

[0138] The storing the second communication message as a test sample in the sample queue includes:

[0139] Determining a priority level of the second communication message based on the test information corresponding to the second communication message;

[0140] storing the second communication message in a sample queue matching the priority level;

[0141] The step of extracting a target test sample from the sample queue comprises:

[0142] Determining a target sample queue from the at least two sample queues based on the sample extraction frequencies corresponding to the priority levels;

[0143] A target test sample is extracted from the target sample queue.

[0144] In an optional implementation manner, in the instruction executed by the processor 501, the priority level of the second communication message is determined according to at least one of the following:

[0145] A first test code coverage indicated by the test information and a second test code coverage before the fuzz test;

[0146] The test information indicates whether an error message occurs;

[0147] The test information indicates first test code path information and second test code path information before the fuzz test.

[0148] In an optional implementation manner, in the instruction executed by the processor 501, the performing data deformation on the target test sample includes:

[0149] Randomly determine a first target byte for data deformation from the binary coded text of the target test sample, and perform a byte deformation operation on the first target byte to obtain the second communication message; the deformation operation includes at least one of byte flipping, adding, deleting and replacing; and / or,

[0150] Based on the semantic structure of the target test sample, a second target byte with target semantics is determined, and a semantic deformation operation is performed on the second target byte to obtain the second communication message.

[0151] In an optional implementation manner, in the instruction executed by the processor 501, performing a semantic deformation operation on the second target byte to obtain the second communication message includes:

[0152] Determining a target working scenario of the vehicle-mounted communication middleware to be tested;

[0153] Based on the target working scenario, a semantic deformation operation is performed on the second target byte to obtain a second communication message matching the target working scenario.

[0154] In an optional implementation manner, in the instruction executed by the processor 501, the determining the test information corresponding to the fuzzy test includes:

[0155] Acquiring execution status information of a plurality of insertion positions in the fuzzy test; the insertion positions are set in the vehicle-mounted communication middleware and a system corresponding to the vehicle-mounted communication middleware;

[0156] Based on the execution status information, determining whether the vehicle-mounted communication middleware has abnormal behavior;

[0157] In the case where the vehicle-mounted communication middleware has abnormal behavior, determining attribute information corresponding to the abnormal behavior based on the execution state information;

[0158] Based on the execution status information and the attribute information corresponding to the abnormal behavior, test information corresponding to the fuzzy test is determined.

[0159] In an optional implementation manner, the instructions executed by the processor 501 further include:

[0160] Based on the attribute information corresponding to the abnormal behavior, the abnormal behavior is classified, and cause information and impact information of the abnormal behavior are generated.

[0161] In an optional implementation manner, the instructions executed by the processor 501 further include:

[0162] Building a system platform matching the vehicle communication middleware to be tested;

[0163] Obtain a test application sample, and build an operating environment corresponding to the test application sample on the system platform;

[0164] Run the test application sample in the running environment to perform an initialization test.

[0165] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the vehicle communication middleware testing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0166] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the vehicle communication middleware testing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0167] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0168] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0169] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0171] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0172] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A vehicle communication middleware testing method, It is characterized in that include: In the process of obtaining the initialization test, a first communication message transmitted by the vehicle-mounted communication middleware to be tested is used, and the first communication message is stored in a sample queue as a test sample; Extracting a target test sample from the sample queue, and performing data deformation on the target test sample to obtain a second communication message corresponding to the target test sample; Performing a fuzzy test on the in-vehicle communication middleware using the second communication message, and determining test information corresponding to the fuzzy test; Determine whether the test information satisfies the test termination condition, and if so, terminate the test; if not, store the second communication message as a test sample in the sample queue, and repeat the extraction step, the fuzzy test step, and the judgment step.

2. The method according to claim 1, It is characterized in that The number of the sample queues is at least two; different sample queues are matched with different priority levels; The storing the second communication message as a test sample in the sample queue includes: Determining a priority level of the second communication message based on the test information corresponding to the second communication message; storing the second communication message in a sample queue matching the priority level; The step of extracting a target test sample from the sample queue comprises: Determining a target sample queue from the at least two sample queues based on the sample extraction frequencies corresponding to the priority levels; A target test sample is extracted from the target sample queue.

3. The method according to claim 2, It is characterized in that Determine the priority level of the second communication message according to at least one of the following: A first test code coverage indicated by the test information and a second test code coverage before the fuzz test; The test information indicates whether an error message occurs; The test information indicates first test code path information and second test code path information before the fuzz test.

4. The method according to claim 1, It is characterized in that The performing data deformation on the target test sample includes: Randomly determine a first target byte for data deformation from the binary coded text of the target test sample, and perform a byte deformation operation on the first target byte to obtain the second communication message; the deformation operation includes at least one of byte flipping, adding, deleting and replacing; and / or, Based on the semantic structure of the target test sample, a second target byte with target semantics is determined, and a semantic deformation operation is performed on the second target byte to obtain the second communication message.

5. The method according to claim 4, It is characterized in that The performing a semantic deformation operation on the second target byte to obtain the second communication message includes: Determining a target working scenario of the vehicle-mounted communication middleware to be tested; Based on the target working scenario, a semantic deformation operation is performed on the second target byte to obtain a second communication message matching the target working scenario.

6. The method according to claim 1, It is characterized in that The determining of the test information corresponding to the fuzzy test includes: Acquiring execution status information of a plurality of insertion positions in the fuzzy test; the insertion positions are set in the vehicle-mounted communication middleware and a system corresponding to the vehicle-mounted communication middleware; Based on the execution status information, determining whether the vehicle-mounted communication middleware has abnormal behavior; In the case where the vehicle-mounted communication middleware has abnormal behavior, determining attribute information corresponding to the abnormal behavior based on the execution state information; Based on the execution status information and the attribute information corresponding to the abnormal behavior, test information corresponding to the fuzzy test is determined.

7. The method according to claim 6, It is characterized in that The method further comprises: Based on the attribute information corresponding to the abnormal behavior, the abnormal behavior is classified, and cause information and impact information of the abnormal behavior are generated.

8. The method according to claim 1, It is characterized in that The method further comprises: Building a system platform matching the vehicle communication middleware to be tested; Obtaining a test application sample, and building a running environment corresponding to the test application sample on the system platform; Run the test application sample in the running environment to perform an initialization test.

9. A vehicle communication middleware testing system, It is characterized in that include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the vehicle-mounted communication middleware testing system is running, the machine-readable instructions are executed by the processor to perform the steps of the vehicle-mounted communication middleware testing method as described in any one of claims 1 to 8.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed, the steps of the vehicle communication middleware testing method as described in any one of claims 1 to 8 are implemented.

Images