A terminal communication protocol testing method, device, equipment and storage medium

Abstract

The application provides a terminal communication protocol test method, device, equipment and storage medium, comprising: selecting a test sample from a pre-constructed test sample set; using each test sample to test a plurality of measured terminals respectively, recording response information generated by each measured terminal based on its own communication protocol for each test sample; merging and processing the response information to determine the mapping relationship between each response information and the test sample in each measured terminal; comparing the differential information of the same response information mapped to different test samples in different measured terminals; comparing the differential information and the communication standard specification to determine the test result of the communication protocol of the measured terminal and the communication standard specification. In this way, by comparing the differential information of the same response information mapped to different test samples in multiple measured terminals, it can be automatically determined whether there is a problem with the communication protocol and standard specification of the multiple measured terminals, thereby improving the automation degree and efficiency of the test.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a testing method, apparatus, device, and storage medium for a terminal communication protocol. Background Technology

[0002] With the development of communication technology, mobile communication networks are being gradually promoted and popularized globally, and major mobile phone and device manufacturers are launching terminal products. Taking 5G mobile communication networks as an example, with the large-scale deployment of 5G terminals, their security has become an issue that cannot be ignored. On the one hand, 5G networks rely on complex technologies and protocols, providing attackers with more attack surfaces; on the other hand, 5G terminals access important and sensitive user data, posing higher security requirements.

[0003] To address this need, common existing approaches include: manually analyzing communication standards and specifications written in natural language based on human experience to infer security issues and manually verify them; and semi-automated methods, which involve manually analyzing communication standards and specifications, designing test cases that violate security rules, automatically injecting them into the normal protocol flow, and then manually observing whether the terminal correctly receives the data and whether easily observable abnormal behaviors such as disconnection occur after reception based on logs and other information, and analyzing the test results. The drawback of existing solutions is insufficient automation; both the testing process and the judgment of test results heavily rely on manual implementation, resulting in low testing efficiency. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a testing method, apparatus, device and storage medium for terminal communication protocols. Based on the differential testing method, by comparing the differential information of the same response information in multiple terminals under test mapped to different test samples, it is possible to automatically determine whether there are problems with the communication standard specifications and the communication protocols of multiple terminals under test, thereby improving the automation and efficiency of testing.

[0005] This application provides a method for testing a terminal communication protocol, the method comprising:

[0006] Select test cases from a pre-built set of test cases;

[0007] Each test case was used to test multiple terminals under test, and the response information generated by each terminal under test to each test case based on its own communication protocol was recorded.

[0008] The response information of each terminal under test to each test sample is merged and processed to determine the mapping relationship between each type of response information and the test sample in each terminal under test;

[0009] The mapping relationship between each type of response information in each tested terminal and the test sample is compared to determine the differential information that maps the same response information to different test samples in different tested terminals.

[0010] By comparing the differential information with the communication standard specifications, the test results of the communication protocol and the communication standard specifications of the terminal under test are determined.

[0011] This application embodiment also provides a testing device for terminal communication protocols, the testing device comprising:

[0012] The selection module is used to select test cases from a pre-built set of test cases;

[0013] The recording module is used to test multiple terminals under test using each test sample, and record the response information generated by each terminal under test to each test sample based on its own communication protocol.

[0014] The merging module is used to merge the response information of each terminal under test to each test sample, and determine the mapping relationship between each type of response information and the test sample in each terminal under test.

[0015] The comparison module is used to compare the mapping relationship between each type of response information in each tested terminal and the test sample, and to determine the differential information of the same response information mapped to different test samples in different tested terminals.

[0016] The determination module is used to compare the differential information with the communication standard specification to determine the test results of the communication protocol and the communication standard specification of the terminal under test.

[0017] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the test method described above are performed.

[0018] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the test method described above.

[0019] This application provides a terminal communication protocol testing method, apparatus, device, and storage medium. Based on the differential testing method, by comparing the differential information of the same response information in multiple tested terminals mapped to different test samples, it can automatically determine whether there are problems with the communication protocol and communication standard specifications of multiple tested terminals, thereby improving the automation and efficiency of testing.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a testing method for a terminal communication protocol provided in an embodiment of this application is shown;

[0023] Figure 2 A schematic diagram of a testing method for a terminal communication protocol provided in an embodiment of this application is shown;

[0024] Figure 3 A schematic diagram illustrating the abstract structure of a test sample provided in an embodiment of this application is shown;

[0025] Figure 4 A schematic diagram of a method for generating a test case set provided in an embodiment of this application is shown;

[0026] Figure 5 A schematic diagram of a multi-state traversal depth testing method provided in an embodiment of this application is shown;

[0027] Figure 6 A schematic diagram of the method for providing a simplified test sample set according to an embodiment of this application is shown;

[0028] Figure 7 A schematic diagram of the structure of a testing device for a terminal communication protocol provided in an embodiment of this application is shown;

[0029] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0031] Research has shown that with the development of communication technology, mobile communication networks are gradually being promoted and popularized globally, and major mobile phone and device manufacturers are launching terminal products (such as 4G and 5G terminal products). Taking 5G mobile communication networks as an example, with the large-scale deployment of 5G terminals, their security has become an issue that cannot be ignored. On the one hand, 5G networks rely on complex technologies and protocols, providing attackers with more attack surfaces; on the other hand, 5G terminals access important and sensitive user data, posing higher security requirements.

[0032] To address this need, common existing approaches include: manually analyzing communication standards and specifications written in natural language based on human experience to infer security issues and manually verify them; and semi-automated methods, which involve manually analyzing communication standards and specifications, designing test cases that violate security rules, automatically injecting them into the normal protocol flow, and then manually observing whether the terminal correctly receives the data and whether easily observable abnormal behaviors such as disconnection occur after reception based on logs and other information, and analyzing the test results. The drawbacks of existing solutions are insufficient automation; the testing process and the judgment of test results both heavily rely on manual implementation, resulting in insufficient testing and low testing efficiency.

[0033] Based on this, embodiments of this application provide a testing method, apparatus, device, and storage medium for terminal communication protocols. Based on the differential testing method, by comparing the differential information of the same response information in multiple tested terminals mapped to different test samples, it is possible to automatically determine whether there are problems with the communication protocols and standard specifications of multiple tested terminals, thereby improving the automation and efficiency of testing.

[0034] Please see Figure 1 and Figure 2 , Figure 1 The flowchart illustrates a testing method for a terminal communication protocol provided in an embodiment of this application. Figure 2This diagram illustrates a testing method for a terminal communication protocol provided in an embodiment of this application. It should be noted that the testing method provided in this application is applicable to multiple generations of communication protocols, including 4G, 5G, and future 6G. The following explanation will use a 5G communication protocol and 5G terminals as examples. Figure 1 and Figure 2 As shown in the embodiments of this application, the testing method includes:

[0035] S101. Select test samples from the pre-built test sample set.

[0036] In this step, a certain number of test seeds can be selected from the test sample set. The test seeds record the value method of the test sample. The test sample can be quickly restored and the test process can be reproduced through the test seeds. The test seeds are arranged in order for subsequent tests. The test seeds correspond one-to-one with the test samples. The test seeds have corresponding seed numbers, which can quickly index the corresponding test samples.

[0037] S102. Test multiple terminals under test using each test sample, and record the response information generated by each terminal under test for each test sample based on its own communication protocol.

[0038] It should be noted that the terminal under test in this application embodiment can be a closed-source 5G terminal, which formulates its own communication protocol code based on communication standard specification documents written in natural language, thereby implementing 5G functions. This application embodiment, based on the differential testing method, can automatically locate and discover potential security vulnerabilities in 5G terminals and communication standard specifications by considering the different results generated by different 5G terminals for the same test sample. Due to the closed-source nature of 5G terminals, different manufacturers or basebands may have ambiguities in their understanding of terminal functions implemented according to communication standard specifications. Therefore, 5G terminals from different manufacturers, with different baseband models, or different versions should be selected as the terminals under test as much as possible.

[0039] The testing objective of the terminal can be defined as follows: For a 5G terminal implementation I, let ∑ represent the set of input messages and Λ represent the set of output messages. There exists an input message sequence π. i =σ1σ2σ3…σ m , where σ j ∈∑, this implementation I is based on the input message sequence π i The generated output message sequence γ i =λ1λ2λ3…λ m , where λ j ∈Λ, the output message sequence γ is found. iInconsistencies with communication standard specifications indicate potential problems with the terminal communication protocol or communication standard specifications. Differential information can be defined as: for two different commercial 5G terminal implementations Ij and Ik, there exists an input message sequence π. i When it is used as input, the terminal implements the output message sequence γ generated by Ij and Ik respectively. j and γ k γ was found j ≠γ k .

[0040] In this step, the selected test seeds are used to test different terminals under test, and the response and interaction of different terminals under test to the test samples reproduced by the same test seed are observed and recorded. The response information generated by each terminal under test to each test sample based on its own communication protocol is also recorded.

[0041] S103. Merge the response information of each terminal under test to each test sample to determine the mapping relationship between each type of response information and the test sample in each terminal under test.

[0042] Since a terminal under test may generate the same response information when tested with multiple test samples, in order to improve testing efficiency, the responses are merged according to the same information. For example, if seed number 1 and seed number 2 both generate the same response information, they are merged to form a mapping relationship between "multiple seeds" and "one response information".

[0043] S104. Compare the mapping relationship between each type of response information and the test sample in each tested terminal to determine the differential information that maps the same response information to different test samples in different tested terminals.

[0044] The meaning of differential testing is to compare the different behaviors of two 5G terminals when testing with the same test sample. That is, based on the differences in state and function between different 5G terminals in the differential test, we can analyze the inconsistencies between the implementation of the terminal communication protocol and the communication standard specifications, as well as the differences in the implementation of different terminals due to the ambiguity of the communication standard specifications.

[0045] In this step, the mapping relationship between each type of response information and the test sample is compared between different tested terminals. If at least two tested terminals have different mapping relationships, that is, the same response information is mapped to different test samples, it indicates that they are inconsistent in the specific 5G communication protocol implementation. Accordingly, based on the test seed number corresponding to different mapping relationships, the corresponding test sample can be searched from the test sample set to determine its corresponding test message, message field and value, that is, to determine the differential information that has differences.

[0046] S105. Compare the differential information with the communication standard specification to determine the test results for the terminal under test and the communication standard specification.

[0047] In this step, the differential information is used to search and locate the issue within the communication standard specifications, considering whether there are any regulations or restrictions in the communication standard specifications, and whether the tested terminal has any implementations that violate these regulations or restrictions. Specifically, assuming that the comparison reveals inconsistencies in the implementations of tested terminals A and B, the judgment logic is as follows:

[0048] Scenario 1: Communication standards and specifications have made clear provisions for this issue. Terminal A follows the provisions of communication standards and specifications, while terminal B does not.

[0049] Scenario 2: Communication standards and specifications have made clear provisions for this issue, but neither terminal A nor B has complied with the provisions of communication standards and specifications.

[0050] For the first and second scenarios, communication standards and specifications clearly define the implementation of communication protocols for terminals. For example, when a terminal receives a certain message or a message containing a certain field value, it should make a positive response or should not make a negative response. In this case, at least one 5G terminal may be found to be non-compliant with the restrictive requirements of the communication standards and specifications, potentially indicating a security issue, thus confirming the test results for the terminal under test.

[0051] Scenario 3: The communication standard specification does not address this issue; Terminal A's implementation meets the security requirements, while Terminal B does not.

[0052] Scenario 4: Communication standards and specifications do not address this issue, and neither the implementations of terminals A and B meet the security requirements.

[0053] Scenario 5: The communication standard specification does not address this issue. The implementations of terminals A and B meet the security requirements, but there are better implementation methods for both.

[0054] For scenarios 3, 4, and 5, the communication standards and specifications do not define the test conditions. This is because the designers of the communication standards and specifications could not have considered all situations, resulting in omissions or undefined issues. This indicates that the design of the communication standards and specifications may have security problems. Furthermore, these problems may not directly pose a security threat, but the lack of regulations on these issues leads to differences in implementation by different manufacturers, which may cause other indirect functional problems. All of this demonstrates that the standards and specifications have problems in defining the test results for the terminal under test.

[0055] Furthermore, if it is determined that the terminal under test violates the provisions and restrictions of communication standards and specifications, then a design sample of those provisions and restrictions can be used to determine the scope of their impact. If it is determined that the communication standards and specifications have not considered the issue or that the issue is not clearly described, then relevant organizations can be contacted for improvement.

[0056] In this way, based on the differential testing method, by comparing the differential information of the same response information in multiple tested terminals mapped to different test samples, it is possible to automatically determine whether there are problems with the communication protocols and standard specifications of multiple tested terminals, thereby improving the automation and efficiency of testing.

[0057] On the other hand, the present application embodiments also found through research that the quality of test samples in the prior art depends on manually defined security rules; the number of test results is small, and only abnormal behaviors within a preset range can be found, but security problems that are not preset or are difficult to automatically judge after being preset cannot be detected, resulting in insufficient testing; the test standards are highly dependent on standard specifications written in natural language, and there may be ambiguities in understanding due to unclear semantic descriptions or undefined problems due to insufficient consideration of the situation during the standard specification formulation process, which are difficult to solve, thus affecting the quality of the generated test samples and thus affecting the test results.

[0058] The embodiments of this application can automatically generate a wide-area test case set that is independent of the natural language description of communication standards and specifications, and deeply test the complex and multi-state underlying interaction functions in the terminal communication protocol, thereby more accurately testing the terminal's communication protocol and achieving better testing results. The construction process of the test case set will be described in detail below with specific examples.

[0059] In a first possible implementation, test cases are generated using data structures from 3GPP standards and 5G open-source software. More specifically, the steps for constructing the test case set may include:

[0060] S201. Define the original abstract structure of the test samples based on the communication standard specification. The abstract structure includes: a set of uplink message types, a set of downlink message types, the composition structure of the message sequence, a set of mutable fields in each downlink message, an atomic data structure for each mutable field, and a set of values ​​corresponding to each atomic data structure.

[0061] First, consider the definition of messages in communication standards and specifications: taking 5G RRC and NAS control plane protocols as examples, 5G open-source software has implemented the frame structure in these protocols at the code level, converting the 3GPP standard specifications' definition of the protocol frame structure into data structures implemented in specific C / C++ programming languages. For example, the definition of a NAS protocol message is divided into two parts: the field organization method of the protocol frame and the meaning of the field values. In the field organization method part of the protocol frame, the 3GPP standard specifications first define the general composition structure of NAS layer protocol messages, including fields such as security header type, protocol message type, and message elements. Taking the "5G Registration Reject" protocol message as an example, the protocol message type corresponds to "5GRegistration Reject," and the message elements correspond to "Registration reject message identity," "5GMM cause," and "Rejected NSSAI," etc. In the field value section of the protocol frame, that is, the values ​​of the aforementioned message elements, are defined through corresponding data structures such as "Message type", "5GMM cause" and "Rejected NSSAI". The data structures are assigned values ​​by bit values ​​of corresponding length, and their data types include integer, boolean and string types.

[0062] Below, an abstract structure for test cases is designed to facilitate automated generation of test cases through message mutation in subsequent steps. Taking NAS protocol messages as an example, according to the different protocol message types defined in the communication standard specification, there are a total of 58 types of uplink and downlink messages, of which 3 types can be used as both downlink and uplink messages. For example, the set of downlink message types is denoted as S. D ={D1, D2, ... D 31 Let S be the set of uplink message types. U ={U1, U2, ... U 30}, message sequence L i The composition structure includes S D and S U The elements in the formula are combinations of D and U.

[0063] Since the test object is a 5G terminal, only downlink test message samples need to be generated when generating test cases. The message type field in the test message samples distinguishes different types of downlink messages. For each downlink message D, the fields other than the message type field are the mutable fields. The set of mutable fields for downlink message D is defined as S. E= {E1, E2, ...}, where E1 is "Extended protocol discriminator", E2 is "Security header type", etc. Since the data structure corresponding to the mutable field in a downlink message D, such as "Message type", may be repeated multiple times, this data structure needs to be uniformly processed when muting the message. Further analysis shows that these specific data structures are composed of more subdivided atomic data structures, such as integer types and string types. Therefore, we divide the data at the programming language level as the granularity and uniformly process the same atomic data structures. It should be noted that a mutable field can be decomposed into one or more fields of more subdivided atomic data structures.

[0064] At this granularity of atomic data structure, the set of values ​​for all mutable fields in all types of downlink messages D is S. V = {V1, V2, ...}, where elements include: constant integer elements, exponential elements, string elements, and enumeration type elements. Constant integers are linearly increasing integer terms, exponential elements are exponentially increasing integer terms, string elements are bit strings represented by strings, and enumeration type data represents the mapping relationship between the constructed enumeration data and constant terms.

[0065] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating an abstract structure of a test sample provided in an embodiment of this application. For example... Figure 3 As shown, examples of downlink message types may include: authentication request, authentication rejection, registration acceptance, and registration rejection; examples of uplink message types may include: authentication response, authentication failure, registration request, and registration completion; taking an authentication request as an example, examples of variable field sets may include: protocol descriptor (8 bits), security header type (4 bits), and authentication request identity (8 bits); selecting the interval between uplink and downlink messages can form a message sequence, and then the message sequence can form a message sequence set; for example, for an atomic data structure of linear growth type, the corresponding value set example is {-1, 1, 2, 3, 4, 5, 6, 7, 8, 9}.

[0066] Based on the structural abstract definition of the test message samples, this application embodiment designs a multi-type test case generation part within the protocol state. The main purpose is to generate as many test samples as possible, and these test samples should involve as many message elements, fields and values ​​as possible.

[0067] S202. Obtain the historical real message sequence; the historical real message sequence includes multiple uplink messages and multiple downlink messages.

[0068] In this embodiment, log information of the interaction between the 5G simulated network and the 5G terminal is extracted, and historical real uplink and downlink messages are extracted according to the time interaction order to obtain the historical real message sequence L0.

[0069] In theory, automated test sample generation can fully traverse all types of downlink messages, all message structures, and all field values. However, this presents problems such as an excessive number of message structures and an overly broad range of field values. Conversely, if only a few security-related fields and their values ​​are selected for testing, most test surfaces will be ignored. This application addresses both aspects by designing an automated test sample generation method that balances depth and breadth for the protocol. To avoid generating a large number of invalid message sequences using the full traversal method, this application uses the historical real message sequence L0 before testing as prior knowledge to generate more effective and reasonable test samples.

[0070] S203. Based on the original abstract structure, perform field mutations on each downlink message in the historical real message sequence to generate the test sample set.

[0071] For more details, please see Figure 4 , Figure 4 This is a schematic diagram illustrating a method for generating a test case set provided in an embodiment of this application.

[0072] The historical real message sequence is placed into a message sequence set. Message sequences are selected sequentially from the message sequence set; the historical real message sequence L0 is the first selected message sequence. Here, the message sequence set is initially empty, and new message sequences can be derived from the historical real message sequence L0 and stored in subsequent steps.

[0073] For each selected message sequence, the downlink messages in that message sequence are selected sequentially, such as D1 for the first time.

[0074] For each selected downlink message, select the mutable fields sequentially from that downlink message, such as E1 for the first time.

[0075] For each selected mutable field, based on the atomic data structure that makes up that mutable field, field values ​​are selected from the corresponding value set for field mutation; where, for fields with repeated atomic data types, the values ​​are the same each time under the same composition structure, such as the initial integer type being INT1.

[0076] Keep the values ​​of other fields that have not undergone field mutation unchanged, and combine them with the mutated fields to form test cases, and record the corresponding test seeds, until all test cases corresponding to the selected message sequence have been generated.

[0077] For example, D1 includes mutable fields E1, ..., E5; E1 and E5 in D1 can each be decomposed into fields of atomic data structures with two constant integer elements. Therefore, firstly, field mutation is achieved by iterating through the set of values ​​corresponding to the constant integers in the fields of these four atomic data structures. In any field mutation, a value is selected from the set of values, keeping the values ​​of these four fields the same, keeping the values ​​of the other atomic data structures decomposed from E1, ..., E5 unchanged, and keeping the values ​​of D2, ... unchanged, forming a test case, thus completing one field mutation. Afterwards, if other atomic data structure fields are decomposed in the downlink message of D1, a similar method is used to mutate them until the field mutation of D1 is completed, then D2... and so on, until the field mutation of L0 is completed. This will transform one instance of field mutation in S... V The process of selecting values ​​is denoted as the test seed for a test case. The test seed is an abstraction of the value selection process, and it is numbered and uniquely identifies a test. In this way, the test seed can be used in subsequent use to recreate the test case and reproduce the test.

[0078] It should be noted that in any given field mutation, fields with the same atomic data structure will have the same value. If subsequent tests reveal that such field mutations of atomic data structures are meaningful, they will elicit different responses from the terminal. In this case, different values ​​can be combined for the fields of this atomic data structure to generate more refined test cases, thus balancing test efficiency and test comprehensiveness.

[0079] The target terminal is tested based on the generated test cases, and a new message sequence is generated and placed into the message sequence set according to the test results of the target terminal.

[0080] Here, a 5G terminal is selected as the target terminal, and its communication protocol is tested according to the previously generated test samples. Based on the test results of the target terminal, i.e., the target terminal's response to the test samples, a new message sequence L1, ... L2 can be generated. n And put it into the message sequence set.

[0081] Then, the process of selecting the next message sequence from the message sequence set and performing field mutation is repeated to generate a wide-area test case set.

[0082] In this way, the other message sequences in the message sequence set in this application embodiment are all derived from historical real message sequences, avoiding the problem of a large number of invalid message sequences being generated due to the complexity of message types and values ​​and the random combination of massive amounts of data.

[0083] In specific implementation, the test number is updated by nested loops. Each time the seed number is updated, the data type number is incremented by one. When the data type number reaches a critical value, it is reset to 0 and the number of the mutable field E is incremented by one. When the number of the mutable field E reaches a critical value, it is reset to 0 and the number of the downlink message D is incremented by one. When the number of the mutable field D reaches a critical value, it is reset to 0 and the process returns to the first step to retrieve the next message sequence, and the above process is repeated.

[0084] On the other hand, the automatically generated test cases mentioned above are multi-type test cases within a single protocol state, meaning they do not take into account the terminal's response and interaction to the test cases. Furthermore, the terminal communication protocol contains complex and multi-state underlying interaction functions. The following section improves the coverage of the generated test cases by monitoring and guiding the terminal to perform multi-state transitions and traversals according to the communication protocol.

[0085] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating a multi-state traversal depth testing method provided in an embodiment of this application. Figure 5 As shown, this application embodiment constructs a method for in-depth testing of the underlying detailed functional implementation of complex multi-state interactions in terminal communication protocols through three parts: sequence state feedback, sequence state snapshot, and sequence structure variation extension. Specifically, we define the state information of a 5G terminal using the following six-tuple: (S, S0, Ψ, ∑, Λ, Ω), where S represents the terminal's state set, S0∈S represents the initial state, ∑ represents the input message set, Λ represents the output message set, the transformation relation Ψ: S×∑→S represents mapping from the current state combined with the input message to the next state, and the output relation Ω: S×∑→Λ represents mapping from the current state combined with the input message to the output message.

[0086] Firstly, regarding sequence state feedback: 5G terminal communication protocols contain complex states, increasing the difficulty of testing. However, the interactions between these states provide an informational foundation for in-depth testing and can improve testing efficiency. Therefore, this application's embodiments introduce a sequence state feedback component. Based on the interaction of protocol states, valid new message sequences are retained, while a large number of similar message sequences with repetitive effects are discarded, ensuring that the message sequences in each test are novel.

[0087] In practical implementation, the target terminal is tested based on the generated test samples. New message sequences are generated based on the test results of the target terminal and added to the message sequence set. This may also include:

[0088] Step 1: Select the target terminal and define the initial state S0 of the target terminal; reset the state of the target terminal to the initial state S0 at the end of each test.

[0089] Step 2: Select test samples from the test sample set; that is, select test samples one by one from the multiple test samples that have been generated.

[0090] Step 3: Test the target terminal in its initial state based on the selected test samples, and record the sequence of feedback messages generated by the target terminal after responding to the selected test samples based on the communication protocol.

[0091] Step 4: Search the message sequence set to see if there is a feedback message sequence corresponding to the selected test sample.

[0092] Step 5: If the feedback message sequence does not exist, add it to the message sequence set. Simultaneously, assign a number to the message sequence, retain the mapping relationship between the new message sequence number and the seed number, and store the seed's value method so that its field values ​​can be reused in subsequent tests to recreate the test; and retain this selected test case as a valid test case.

[0093] In this way, valid new message sequences can be discovered and preserved.

[0094] For example, in the historical message sequence L0 = {D1, U2, D2, U3}, D1 is mutated during the field mutation process to generate a new test case. Based on this test case, the target terminal is tested and it is found that the target terminal response changes from U2 to U3. Since there is no feedback message sequence {D1, U3} in the message sequence set, it is added to the message sequence set.

[0095] Secondly, although valid new message sequences can be discovered and preserved through sequence state feedback, during automated test case generation, the nested loop scheme will re-mutate all messages in the message sequence. These mutations may again destroy the validity of the new message sequence, or even mutate it into duplicate samples. To preserve the validity of new message sequences, this application embodiment adds the definition of location information and designs a sequence state snapshot method. In specific implementation, the step of constructing the test case set may further include:

[0096] Step 1: Expand the structure of the message sequence by adding gap position identifiers before and after each uplink and downlink message in the message sequence, and add a position number to each element in the expanded message sequence; for example, if the message sequence L is [*D1*U1*D2*U2*D3*U3*D4*U4*], * represents a gap position identifier. Define the gap position identifier * before D1 as number 0, D1 as number 1, and * after D1 as number 2, and so on.

[0097] Step 2: Select message sequences and their corresponding target location numbers from the message sequence set one by one.

[0098] Step 3: For each message sequence, select message elements numbered after the target position and perform field mutation.

[0099] For steps 2 and 3, the method for sequentially selecting downlink messages in the automated test case generation scheme S203 is optimized. It is modified to select the target position number and the subsequent downlink message or gap position* in each message sequence; the variation methods for the remaining fields remain unchanged. For example, if the position number of the triplet is 9, pointing to position D3, positions 9, 10, 12, 13, 14, and 16 can be selected, where 11 and 15 correspond to U3 and U4 respectively.

[0100] Step 4: Test the target terminal in its initial state based on the test sample generated after field mutation, and record the sequence of feedback messages generated by the target terminal after responding to the test sample generated after field mutation based on the communication protocol.

[0101] Step 5: If the message sequence set does not have a corresponding feedback message sequence, then based on the test seed number corresponding to the test sample, the feedback message sequence, and the position number of the last gap position identifier in the feedback message sequence, an extended message sequence with a triple structure is formed, and the extended message sequence is added to the message sequence set.

[0102] For example, the extended message sequence of the triple structure is represented as (Seed, [*D1*U1*D2*U2*D3*U3*D4*U4*], Position). Here, the Seed and Position for the historical real message sequence L0 are both defined as 0, i.e., the triple is (0, L0, 0).

[0103] For steps 4 and 5, the sequence state feedback part is optimized accordingly. Position information is added during feedback to reduce redundancy in repeated mutations. For example, if the terminal responds with message U5 when mutation testing is performed at position number 13 (D4), indicating that a new message sequence has been discovered, then the gap position identifier * after U5 (number 16) is added to the message sequence. At this time, the extended message sequence of the triple structure is (Seed, [*D1*U1*D2*U2*D3*U3*D4*U5*], 16).

[0104] Third, considering that the automatically generated test cases in S201 and S202 are multi-type test cases within a single protocol state, but some security issues require consideration of the interaction relationships between multiple states, a sequence structure variation extension method is proposed to modify the original single-granularity test case generation to multi-granularity. In specific implementation, the definition of protocol message types is first expanded. The aforementioned method, when determining whether message sequences are the same, i.e., searching the test case set for the existence of a feedback message sequence corresponding to the selected test case, only considers the type of protocol downlink messages.

[0105] Here, based on the types of protocol messages, we further consider some message fields. For example, when considering error message types, we treat the error message and its reason as a whole, with different reasons treated as different messages. These message fields are selected according to the rule that "message fields have a limited number of values, and different values ​​correspond to different meanings."

[0106] The steps for constructing the test case set may further include:

[0107] The location numbers are differentiated as follows: If the target location number points to a gap location identifier, a structural variation number is added to the triple structure of the feedback message sequence; the structural variation number is used to indicate the type of message element inserted at the position corresponding to the target location number. The message element type includes an empty message type that waits for a certain period of time and a downlink message type.

[0108] For empty message types that wait for a certain duration, the structure variation number is used to set a timer. It receives an integer field as the timer duration (waiting time), which is the feedback the terminal will produce after waiting for a certain period. This empty message type can be represented by the number 0. For downlink message types, the structure variation number is related to the downlink message type set S. D The elements in the middle correspond one-to-one, so that different types of downlink messages can be copied and added according to the structural variation number.

[0109] Furthermore, to eliminate test cases with repetitive effects and improve the testing efficiency of 5G terminals, this application embodiment also provides a method for simplifying the generated test case set. This method utilizes the simulated implementation of the protocol in open-source 5G terminal code to optimize and simplify the generated test case set. The open-source 5G terminal also implements 5G functions according to communication standard specifications and supports gray-box testing. A differential verification method is used to apply the information obtained from gray-box testing of the open-source 5G terminal to black-box testing of the closed-source 5G terminal, thereby improving the testing efficiency of the closed-source 5G terminal. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram illustrating a method for simplifying a test sample set provided in an embodiment of this application.

[0110] Step 1: Select an open-source terminal and extract the target source code of the target communication protocol from the open-source terminal.

[0111] In this step, considering that the source code of the open-source terminal contains multi-protocol functionality implementations, the source code of the target protocol is screened to make the testing more protocol-specific and improve testing efficiency. Taking the RRC protocol as an example, all code files related to this communication protocol are extracted to form the target source code file.

[0112] Step 2: Perform function-level instrumentation and statement-level instrumentation on the target source code.

[0113] Function-level instrumentation involves instrumenting all functions defined in the target protocol, adding instrumentation points at the beginning and end of these functions. When the software executes a function, the instrumentation point is triggered, and the corresponding function name is stored. Statement-level instrumentation, on the other hand, involves instrumenting statements within a selected function, adding an instrumentation point to each statement in the function's code snippet, monitoring the execution of the code snippet within the function, and storing the number of lines of statements executed by the corresponding function.

[0114] Step 3: Select a test case from the test case set, test the target communication protocol of the open source terminal, and record the information triggered by the test case in the target source code; the information includes the number of functions, the function name, and the number of lines of statements within the function.

[0115] Step 4: Compare the information triggered by each test case in the target source code. When a predetermined condition is met, add the test case to the simplified test case set. The predetermined condition includes at least one of the following: there are no other test cases with the same number of functions as the test case; there are no other test cases with the same number of functions and the same function names as the test case; there are no other test cases with the same number of functions and function names as the test case, and the same number of lines of statements within the functions.

[0116] Although building a simplified test case set takes some time, it is a one-time task. Once the test case set is built, simplified testing can be performed on multiple 5G terminals. Since all test cases in the simplified test case set have corresponding implementations in open-source 5G terminals, test cases with duplicate effects can be eliminated, improving the testing efficiency of commercial 5G terminals.

[0117] This application provides a testing method for a terminal communication protocol. First, by using differential testing to identify problems in closed-source 5G terminal communication protocol implementations or standard specifications based on behavioral differences without relying on source code, it can reduce manual analysis and comparison of test results, improve testing efficiency, and lower testing costs. Second, by abstracting and defining protocol test cases, it can generate test cases covering all functions of the protocol, without relying on predefined test cases or security models, and can discover unknown problems. In addition, considering the multi-state nature of commercial 5G terminal communication protocols, it uses state information to guide testing, improving testing efficiency. Finally, the testing method is reproducible, improving the efficiency of problem repair.

[0118] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a testing device for a terminal communication protocol provided in an embodiment of this application. Figure 7 As shown, the testing apparatus 600 includes:

[0119] Select module 610, used to select test samples from a pre-built set of test samples;

[0120] The recording module 620 is used to test multiple terminals under test using each test sample, and record the response information generated by each terminal under test to each test sample based on its own communication protocol.

[0121] The merging module 630 is used to merge the response information of each terminal under test to each test sample and determine the mapping relationship between each type of response information and the test sample in each terminal under test.

[0122] The comparison module 640 is used to compare the mapping relationship between each type of response information in each tested terminal and the test sample, and to determine the differential information of the same response information mapped to different test samples in different tested terminals.

[0123] The determination module 650 is used to compare the differential information and the communication standard specification to determine the test results of the communication protocol and the communication standard specification of the terminal under test.

[0124] Furthermore, the testing apparatus also includes a construction module; the construction module is used to construct the test sample set through the following steps:

[0125] The original abstract structure of the test sample is defined based on the communication standard specification; wherein, the abstract structure includes: a set of uplink message types, a set of downlink message types, a composition structure of message sequences, a set of mutable fields in each downlink message, an atomic data structure for each mutable field, and a set of values ​​corresponding to each atomic data structure;

[0126] Obtain historical real message sequences; the historical real message sequences include various uplink messages and various downlink messages;

[0127] Based on the original abstract structure, the downlink messages in the historical real message sequence are mutated one by one to generate the test sample set.

[0128] Furthermore, when the construction module performs field mutations on each downlink message in the historical real message sequence based on the original abstract structure to generate the test sample set, the construction module is used to:

[0129] Place the historical real message sequence into a message sequence set;

[0130] Message sequences are selected sequentially from the set of message sequences; the historical real message sequence is the first message sequence selected.

[0131] For each selected message sequence, the downlink messages in that message sequence are selected sequentially;

[0132] For each selected downlink message, select the mutable fields sequentially from that downlink message;

[0133] For each selected mutable field, based on the atomic data structure that makes up that mutable field, field values ​​are selected from the corresponding value set for field mutation;

[0134] Keep the values ​​of other fields that have not undergone field mutation unchanged, and combine them with the mutated fields to form test cases until all test cases corresponding to the selected message sequence have been generated.

[0135] The target terminal is tested based on the generated test cases, and a new message sequence is generated and placed into the message sequence set according to the test results of the target terminal.

[0136] Return to the set of message sequences and select the next message sequence, and continue to perform field mutations.

[0137] Furthermore, when the construction module tests the target terminal based on the generated test samples, and generates new message sequences based on the test results of the target terminal and adds them to the message sequence set, the construction module is also used to:

[0138] Select a target terminal and define the initial state of the target terminal;

[0139] Select test samples from the set of test samples;

[0140] The target terminal in its initial state is tested based on the selected test samples, and the sequence of feedback messages generated by the target terminal after responding to the selected test samples based on the communication protocol is recorded.

[0141] Search the set of message sequences to see if there is a feedback message sequence corresponding to the selected test sample;

[0142] If it does not exist, add the feedback message sequence to the message sequence set.

[0143] Furthermore, the building module is also used for:

[0144] The composition structure of the message sequence is expanded by adding gap position identifiers before and after each uplink and downlink message included in the message sequence, and adding a position number to each element in the expanded message sequence;

[0145] Select message sequences and their corresponding target location numbers one by one from the set of message sequences;

[0146] For each message sequence, select message elements numbered after the target location and perform field mutations;

[0147] The target terminal in its initial state is tested based on the test samples generated after field mutation, and the sequence of feedback messages generated by the target terminal after responding to the test samples generated after field mutation based on the communication protocol is recorded.

[0148] If the message sequence set does not have a corresponding feedback message sequence, then an extended message sequence with a triple structure is formed based on the test seed number corresponding to the test sample, the feedback message sequence, and the position number of the last gap position identifier in the feedback message sequence, and the extended message sequence is added to the message sequence set.

[0149] Furthermore, the building module is also used for:

[0150] If the target location number points to a gap location identifier, a structural variation number is added to the triple structure of the feedback message sequence; the structural variation number is used to indicate the type of message element inserted at the position corresponding to the target location number; the message element type includes an empty message type that waits for a certain period of time and a downlink message type.

[0151] Furthermore, the building module is also used for:

[0152] Select an open-source terminal and extract the target source code of the target communication protocol from the open-source terminal;

[0153] Perform function-level instrumentation and statement-level instrumentation on the target source code;

[0154] Test examples are selected from the test example set to test the target communication protocol of the open-source terminal, and the information triggered by the test example in the target source code is recorded; the information includes the number of functions, the function names, and the number of lines of statements within the functions;

[0155] Compare the information triggered by each test case in the target source code. When a predetermined condition is met, add the test case to a simplified test case set. The predetermined condition includes at least one of the following: there are no other test cases with the same number of functions as the test case; there are no other test cases with the same number of functions and the same function names as the test case; there are no other test cases with the same number of functions and the same number of lines of statements within the functions as the test case.

[0156] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 700 includes a processor 710, a memory 720, and a bus 730.

[0157] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 communicates with the memory 720 via the bus 730. When the machine-readable instructions are executed by the processor 710, they can perform the operations described above. Figure 1 The steps of the test method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0158] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the test method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0159] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0160] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0163] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for testing a terminal communication protocol, characterized in that, The testing method includes: Select test cases from a pre-built set of test cases; Each test case was used to test multiple terminals under test, and the response information generated by each terminal under test to each test case based on its own communication protocol was recorded. The response information of each terminal under test to each test sample is merged and processed to determine the mapping relationship between each type of response information and the test sample in each terminal under test; The mapping relationship between each type of response information in each tested terminal and the test sample is compared to determine the differential information that maps the same response information to different test samples in different tested terminals. By comparing the differential information with the communication standard specifications, the test results of the communication protocol and the communication standard specifications of the terminal under test are determined. The steps for constructing the test case set include: Select an open-source terminal and extract the target source code of the target communication protocol from the open-source terminal; Perform function-level instrumentation and statement-level instrumentation on the target source code; Select test cases from the generated test case set to test the target communication protocol of the open source terminal, and record the information triggered by the test case in the target source code; the information includes the number of functions, function names, and the number of lines of statements within the functions; Compare the information triggered by each test case in the target source code. When a predetermined condition is met, add the test case to a simplified test case set. The predetermined condition includes at least one of the following: there are no other test cases with the same number of functions as the test case; there are no other test cases with the same number of functions and the same function names as the test case; there are no other test cases with the same number of functions and the same number of lines of statements within the functions as the test case.

2. The method according to claim 1, characterized in that, The steps for constructing the test case set include: The original abstract structure of the test sample is defined based on the communication standard specification; wherein, the abstract structure includes: a set of uplink message types, a set of downlink message types, a composition structure of message sequences, a set of mutable fields in each downlink message, an atomic data structure for each mutable field, and a set of values ​​corresponding to each atomic data structure; Obtain historical real message sequences; the historical real message sequences include various uplink messages and various downlink messages; Based on the original abstract structure, the downlink messages in the historical real message sequence are mutated one by one to generate the test sample set.

3. The method according to claim 2, characterized in that, Based on the original abstract structure, each downlink message in the historical real message sequence is subjected to field mutation to generate the test sample set, including: Place the historical real message sequence into a message sequence set; Message sequences are selected sequentially from the set of message sequences; the historical real message sequence is the first message sequence selected. For each selected message sequence, the downlink messages in that message sequence are selected sequentially; For each selected downlink message, select the mutable fields sequentially from that downlink message; For each selected mutable field, based on the atomic data structure that makes up that mutable field, field values ​​are selected from the corresponding value set for field mutation; Keep the values ​​of other fields that have not undergone field mutation unchanged, and combine them with the mutated fields to form test cases until all test cases corresponding to the selected message sequence have been generated. The target terminal is tested based on the generated test cases, and a new message sequence is generated and placed into the message sequence set according to the test results of the target terminal. Return to the set of message sequences and select the next message sequence, and continue to perform field mutations.

4. The method according to claim 3, characterized in that, The step of testing the target terminal based on the generated test examples, generating new message sequences based on the test results of the target terminal and placing them into the message sequence set, further includes: Select a target terminal and define the initial state of the target terminal; Select test samples from the set of test samples; The target terminal in its initial state is tested based on the selected test samples, and the sequence of feedback messages generated by the target terminal after responding to the selected test samples based on the communication protocol is recorded. Search the set of message sequences to see if there is a feedback message sequence corresponding to the selected test sample; If it does not exist, add the feedback message sequence to the message sequence set.

5. The method according to claim 4, characterized in that, The methods for constructing the test case set also include: The composition structure of the message sequence is expanded by adding gap position identifiers before and after each uplink and downlink message included in the message sequence, and adding a position number to each element in the expanded message sequence; Select message sequences and their corresponding target location numbers one by one from the set of message sequences; For each message sequence, select message elements numbered after the target location and perform field mutations; The target terminal in its initial state is tested based on the test samples generated after field mutation, and the sequence of feedback messages generated by the target terminal after responding to the test samples generated after field mutation based on the communication protocol is recorded. If the message sequence set does not have a corresponding feedback message sequence, then an extended message sequence with a triple structure is formed based on the test seed number corresponding to the test sample, the feedback message sequence, and the position number of the last gap position identifier in the feedback message sequence, and the extended message sequence is added to the message sequence set.

6. The method according to claim 5, characterized in that, The methods for constructing the test case set also include: If the target location number points to a gap location identifier, a structural variation number is added to the triple structure of the feedback message sequence; the structural variation number is used to indicate the type of message element inserted at the position corresponding to the target location number; the message element type includes an empty message type that waits for a certain period of time and a downlink message type.

7. A testing device for a terminal communication protocol, characterized in that, The testing apparatus includes: The selection module is used to select test cases from a pre-built set of test cases; The recording module is used to test multiple terminals under test using each test sample, and record the response information generated by each terminal under test to each test sample based on its own communication protocol. The merging module is used to merge the response information of each terminal under test to each test sample, and determine the mapping relationship between each type of response information and the test sample in each terminal under test. The comparison module is used to compare the mapping relationship between each type of response information in each tested terminal and the test sample, and to determine the differential information of the same response information mapped to different test samples in different tested terminals. The determination module is used to compare the differential information with the communication standard specification to determine the test results of the communication protocol and the communication standard specification of the terminal under test; The testing apparatus further includes a construction module; the construction module is used for: Select an open-source terminal and extract the target source code of the target communication protocol from the open-source terminal; Perform function-level instrumentation and statement-level instrumentation on the target source code; Select test cases from the generated test case set to test the target communication protocol of the open source terminal, and record the information triggered by the test case in the target source code; the information includes the number of functions, function names, and the number of lines of statements within the functions; Compare the information triggered by each test case in the target source code. When a predetermined condition is met, add the test case to a simplified test case set. The predetermined condition includes at least one of the following: there are no other test cases with the same number of functions as the test case; there are no other test cases with the same number of functions and the same function names as the test case; there are no other test cases with the same number of functions and the same number of lines of statements within the functions as the test case.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of a test method for a terminal communication protocol as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a test method for a terminal communication protocol as described in any one of claims 1 to 6.

Images