Method and device for testing NAS protocol consistency of 5G terminal

Abstract

The invention provides a 5G terminal NAS protocol consistency test method and device, and the method comprises the steps: determining at least one target risk sentence in an NAS protocol based on test demand information; reasoning the target risk sentences by utilizing a prompt engineering language model to generate at least one test case corresponding to each target risk sentence; performing format constraint specification on each test case to generate a target test case corresponding to each test case; and based on the target test cases, testing the target 5G terminal equipment by using a test tool to obtain a test result corresponding to each target test case so as to determine whether the target 5G terminal equipment conforms to an NAS protocol in a test scene in the test demand information. Through the above method, the high coverage rate of the test case is ensured, the accuracy and efficiency of the 5G terminal NAS protocol conformance test are improved, and the comprehensiveness and flexibility of the conformance test are further improved.

Description

Technical Field

[0001] This application relates to the technical field of network communication protocol testing, and in particular, to a method and device for testing the conformance of a 5G terminal NAS protocol. Background Art

[0002] With the wide deployment of 5G technology, 5G terminal devices need to strictly follow the protocol specifications of the 3rd Generation Partnership Project (3GPP) to ensure the secure interconnection of the network and devices. Since the 3GPP protocol specifications, including the 5G System Non-Access Stratum (NAS) protocol standard, are described in natural language, it is easy to generate ambiguities in the process of understanding and implementing the specifications, resulting in possible deviations between the implementation of 5G terminal devices and the standard specifications. This makes protocol conformance testing a key link in verifying the security of terminal devices.

[0003] Currently, the method for testing the conformance of the 5G terminal NAS protocol mainly analyzes the protocol documents manually, extracts the specification descriptions that may cause inconsistencies between the implementation of 5G terminal devices and the specification constraints, and then determines test cases with limited coverage, reducing the efficiency and accuracy of the conformance testing of the 5G terminal NAS protocol; in addition, the above method is limited to the LTE system and fails to cover the protocol characteristics unique to 5G, restricting the comprehensiveness and flexibility of the conformance testing of the 5G terminal NAS protocol. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and device for testing the conformance of a 5G terminal NAS protocol. By defining the test case structure and core components for NAS protocol conformance testing, clarifying the test case design rules, and designing the writing standards for positive and negative test cases, the high coverage of test cases is ensured. Through the understanding ability and knowledge representation characteristics of the large language model for communication protocol texts, and using prompt engineering to guide the model to accurately understand the test requirements and protocol key points, the automatic conversion from the risk sentences determined from the NAS protocol to test cases by the prompt engineering language model is realized, improving the accuracy and efficiency of the conformance testing of the 5G terminal NAS protocol, and further improving the comprehensiveness and flexibility of the conformance testing.

[0005] An embodiment of this application provides a method for testing the conformance of a 5G terminal NAS protocol. The testing method includes:

[0006] In response to the obtained test requirement information for performing NAS protocol conformance testing on a target 5G terminal device, based on the test requirement information, determine at least one target risk sentence in a preset NAS protocol;

[0007] Input the target risk sentence into a preset prompt engineering language model, and use the prompt engineering language model to reason about the target risk sentence to generate at least one test case corresponding to each target risk sentence;

[0008] Use the prompt engineering language model to standardize the format of each test case to generate a target test case corresponding to each test case;

[0009] Based on the target test case, use a preset test tool to test the target 5G terminal device to obtain a test result corresponding to each target test case, so as to determine whether the target 5G terminal device complies with the NAS protocol in the test scenario of the test requirement information.

[0010] An embodiment of the present application further provides a test device for 5G terminal NAS protocol consistency. The test device includes:

[0011] A requirement processing module, configured to respond to test requirement information for NAS protocol consistency testing of a target 5G terminal device, and based on the test requirement information, determine at least one target risk sentence in a preset NAS protocol;

[0012] A use case generation module, configured to input the target risk sentence into a preset prompt engineering language model, and use the prompt engineering language model to reason about the target risk sentence to generate at least one test case corresponding to each target risk sentence;

[0013] A constraint specification module, configured to use the prompt engineering language model to standardize the format of each test case to generate a target test case corresponding to each test case;

[0014] A use case testing module, configured to based on the target test case, use a preset test tool to test the target 5G terminal device to obtain a test result corresponding to each target test case, so as to determine whether the target 5G terminal device complies with the NAS protocol in the test scenario of the test requirement information.

[0015] An embodiment of the present application further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the above-mentioned test method for 5G terminal NAS protocol consistency are executed.

[0016] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the above-mentioned 5G terminal NAS protocol conformance test method.

[0017] The 5G terminal NAS protocol conformance test method and device provided by the embodiments of the present application, the test method includes: in response to obtaining test requirement information for performing NAS protocol conformance test on a target 5G terminal device, based on the test requirement information, determining at least one target risk sentence in a preset NAS protocol; inputting the target risk sentence into a preset prompt engineering language model, and using the prompt engineering language model to reason about the target risk sentence to generate at least one test case corresponding to each target risk sentence; using the prompt engineering language model to standardize the format of each test case to generate a target test case corresponding to each test case; based on the target test case, using a preset test tool to test the target 5G terminal device to obtain a test result corresponding to each target test case, so as to determine whether the target 5G terminal device conforms to the NAS protocol in the test scenario in the test requirement information.

[0018] Compared with the prior art in which test cases with limited coverage are determined by manual analysis of protocol documents and test methods limited to the LTE system, by defining the test case structure and core components for NAS protocol conformance testing, clarifying the test case design rules, designing the writing standards for positive and negative test cases, ensuring high coverage of test cases, and leveraging the understanding ability and knowledge representation characteristics of the large language model for communication protocol texts, and using prompt engineering to guide the model to accurately understand test requirements and protocol key points, the automatic conversion from risk sentences determined from the NAS protocol to test cases is realized by the prompt engineering language model, improving the accuracy and efficiency of the 5G terminal NAS protocol conformance test, and further improving the comprehensiveness and flexibility of the conformance test.

[0019] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. Description of the Drawings

[0020] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can be obtained based on these drawings.

[0021] Figure 1Flowchart of a method for testing the NAS protocol consistency of a 5G terminal provided by an embodiment of the present application;

[0022] Figure 2 Architecture diagram of a 5G mobile communication system provided by an embodiment of the present application;

[0023] Figure 3 Structural schematic diagram of a testing device for the NAS protocol consistency of a 5G terminal provided by an embodiment of the present application;

[0024] Figure 4 Structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Usually, the components of the embodiments of the present application described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application that is required to be protected, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, every other embodiment obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0026] Through research, it is found that with the wide deployment of 5G technology, 5G terminal devices need to strictly follow the protocol specifications of the 3rd Generation Partnership Project (3GPP) to ensure the secure interconnection of the network and devices. Since the 3GPP protocol specifications, including the 5G system non-access stratum (NAS) protocol standards, are described in natural language, ambiguity is likely to occur during the understanding and implementation of the specifications, resulting in possible deviations between the implementation of 5G terminal devices and the standard specifications. This makes protocol consistency testing a key link in verifying the security of terminal devices.

[0027] When studying the 5G network protocol consistency testing, the testing experience of the LTE network provides an important reference. In the development process of the LTE system, security vulnerability problems occur frequently, mainly including problems such as memory corruption and protocol standard inconsistencies. Among them, the root cause of the protocol standard inconsistency problem is that the LTE specifications are written in natural language and have ambiguity. Secondly, the existing consistency test suites mainly focus on positive testing, that is, verifying whether the terminal device correctly processes valid information, while paying insufficient attention to the processing test of invalid or prohibited signaling (i.e., negative testing), resulting in many implementation vulnerabilities that cannot be effectively detected.

[0028] Currently, the method for conducting conformance testing on the 5G terminal NAS protocol mainly involves manually analyzing protocol documents, extracting specification descriptions that may lead to inconsistencies between the implementation of 5G terminal devices and specification constraints, and then determining test cases with limited coverage, which reduces the test efficiency and accuracy of the 5G terminal NAS protocol conformance; in addition, the above method is limited to the LTE system and fails to cover the protocol features unique to 5G, restricting the comprehensiveness and flexibility of the 5G terminal NAS protocol conformance testing.

[0029] Based on this, the embodiments of this application provide a method for testing the conformance of the 5G terminal NAS protocol. By defining the test case structure and core components for NAS protocol conformance testing, clarifying the test case design rules, and designing the writing standards for positive and negative test cases, it ensures a high coverage rate of test cases. Through the understanding ability and knowledge representation characteristics of the large language model for communication protocol texts, and using prompt engineering to guide the model to accurately understand the test requirements and protocol key points, it realizes the automatic conversion from the risk sentences determined from the NAS protocol by the prompt engineering language model to test cases, improving the accuracy and efficiency of the 5G terminal NAS protocol conformance testing, and further enhancing the comprehensiveness and flexibility of the conformance testing.

[0030] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for testing the conformance of the 5G terminal NAS protocol provided by the embodiments of this application. As Figure 1 shown in

[0031] S101. In response to the obtained test requirement information for conducting NAS protocol conformance testing on the target 5G terminal device, based on the test requirement information, determine at least one target risk sentence in the preset NAS protocol.

[0032] It should be noted that the control plane of the 5G network adopts a service-based network architecture, the network functions are disassembled into modular network services, the interfaces adopt service-based network interfaces, and the flexible customization of network functions is realized. The 5G system service-based architecture (SBA) mainly consists of three major parts: 5G terminal devices, radio access networks, and core networks.

[0033] Specifically, please refer to Figure 2 , Figure 2 which is an architecture diagram of a 5G mobile communication system provided by the embodiments of this application.

[0034] As Figure 2As shown, the 5G terminal device UE is directly connected to the Access and Mobility Management Function (AMF) in the core network through the N1 interface, and is connected to the base station through the Uu air interface. Signaling messages are transmitted between the base station and the core network through the N2 interface, and user data is transmitted through the N3 interface. Inside the core network, each functional network element realizes interconnection and interoperability through service-based interfaces and jointly completes network functions. This architecture design not only ensures the flexible deployment of network functions but also can efficiently process user services.

[0035] Among them, the 5G terminal device (i.e., the user equipment UE), which may include devices such as 5G smartphones or 5G cellular devices, is connected to the core network through the radio access network (gNB) and further connected to the data network (DN). The UE is directly connected to the core network through the N1 interface and establishes a wireless connection with the new-generation base station gNB through the air interface Uu.

[0036] As the core device of the radio access network, the gNB not only provides wireless signal coverage but also is responsible for important functions such as radio resource scheduling and mobility management. The gNB exchanges control signaling with the core network through the N2 interface and transmits user data through the N3 interface.

[0037] The 5G core network adopts a service-based architecture design and consists of multiple functional network elements working together. Among them, the Access and Mobility Management Function AMF is the single entry point for the UE to connect to the core network and is responsible for handling the access control, authentication, and mobility management of the UE; the Session Management Function (SMF) is responsible for handling the PDU session life cycle of the UE, including the establishment, modification, and release of the session; the Unified Data Management (UDM) is responsible for storing and managing user subscription data and providing data support for other network elements; the Authentication Server Function (AUSF) is responsible for user authentication and security key generation, allowing the AMF to authenticate the access service of the UE and the 5G core, and ensuring the security of network access; the User Plane Function (UPF) transmits IP data traffic between the UE and the external network.

[0038] Furthermore, in the 5G system, the NAS signaling interaction between the 5G terminal device and the Access and Mobility Management Function (AMF) module in the core network is the core mechanism for network access, security authentication, and service management. That is, NAS is the control plane communication protocol between the UE and the core network in the 5G network, and its security directly affects the reliable operation of the entire 5G system. There are various main NAS communication attack methods in the 5G environment, such as Fake Base Station attack (FBS), Man-in-the-Middle attack (MitM), and Signal Overshadowing attack, etc.

[0039] In the embodiment of this application, the 5G terminal NAS protocol conformance test is an important means to verify whether the 5G terminal device meets the requirements of the 3GPP NAS protocol specification. This test mainly verifies whether the protocol implementation of the terminal in aspects such as network access control, mobility management, and session management is consistent with the standard, ensuring that the 5G terminal can achieve reliable intercommunication with core network devices of different manufacturers.

[0040] Among them, the Non-Access Stratum (NAS) protocol undertakes the heavy responsibility of signaling interaction between the terminal and the core network in the 5G system, and its reliability directly affects the network service quality. The 3GPP TS24.501 specification details the 5G system NAS protocol standard. As the global mobile communication standard-setting organization, 3GPP (3rd Generation Partnership Project) has released technical specifications covering multiple levels such as radio access, core network, protocol stack, and security mechanisms to ensure the interoperability of devices and networks of different manufacturers globally.

[0041] In the embodiment of this application, aiming at the complexity and ambiguity of the NAS protocol, in-depth text analysis of the preset NAS protocol is carried out, the NAS protocol specification documents are systematically sorted out, key processes and risk points are identified, and by carefully studying the 3GPP TS24.501 specification, multiple risk sentences that are ambiguous or may lead to incorrect implementation of the terminal are extracted from the protocol specification.

[0042] In this step, when receiving the test requirement information for the NAS protocol conformance test of the target 5G terminal device input externally, determine the test scenario information, test dimension, test direction, test means, etc. in the test requirement information, and based on the above content, determine at least one target risk sentence from the multiple risk sentences extracted in the preset NAS protocol, so as to realize the NAS protocol conformance test of the target 5G terminal device by transforming the target risk sentence into test cases.

[0043] Exemplarily, the risk sentences may include "AUE shall be ready to respond to an IDENTITYREQUEST message at any time whilst in SGMM-CONNECTED mode", "Upon reception of a DL NAS TRANSPORT message, the UE shall stop the timer T3346 if running", and "If the security mode command cannot be accepted, the UE shall send a SECURITYMODE REJECT message", etc.

[0044] S102. Input the target risk sentence into a preset prompt engineering language model, and use the prompt engineering language model to reason about the target risk sentence to generate at least one test case corresponding to each target risk sentence.

[0045] In the embodiment of the present application, the prompt engineering language model may include a test case generation model based on large language models and prompt engineering.

[0046] Here, large language models (LLMs) are an important breakthrough in the field of artificial intelligence. They have shown significant advantages in natural language processing tasks. Through pre-training with massive data and deep neural network architectures, large language models not only master basic language knowledge but also possess powerful semantic understanding and logical reasoning abilities. In terms of semantic understanding, the model can accurately grasp context relationships, understand complex language expressions, and handle diverse language scenarios. In terms of reasoning ability, the model can perform multi-step logical analysis and solve complex tasks specific to professional fields.

[0047] In the embodiment of the present application, the large language model may include models such as the GPT model, Claude model, and LLaMA model. Exemplarily, when selecting the Claude model as the large language model in the embodiment of the present application, the model has excellent understanding ability of protocol specifications, can accurately grasp various technical details in the NAS protocol, can understand and strictly follow the preset test case writing rules, thereby ensuring that the generated test cases fully meet the requirements. In addition, the model can rely on its powerful reasoning ability. It can deeply understand the reasoning rules in the complex test case generation process and then generate test cases with high accuracy and good quality.

[0048] Furthermore, how large language models can effectively guide the model to perform specific tasks and ensure the output quality is an issue worthy of in-depth exploration. Prompt Engineering is a technical method to improve the model output effect by designing and optimizing input prompts, aiming to establish an effective bridge between human intentions and model understanding, and mainly solve problems such as the accuracy of task understanding, the controllability of output, and the precision of knowledge application in the model.

[0049] In this way, through prompt engineering, the language model can not only accurately grasp the technical details and specifications in the NAS protocol, but also understand and strictly follow the preset test case writing rules, so as to ensure that the generated test cases fully meet the requirements. The prompt engineering model can model the ability to solve complex problems and make the reasoning process more transparent and controllable. In the process of generating test cases, prompt engineering helps the large language model systematically analyze test requirements and design test scenarios according to logical steps, and then generate test cases with high accuracy and good quality.

[0050] In an implementation manner of the present application, in specific implementation, the steps of setting the prompt engineering language model in step S102 may include:

[0051] S102A. Input a preset prompt engineering information file into a preset large language model, so that the large language model performs knowledge loading based on the prompt engineering information file to obtain a first prompt engineering language model.

[0052] In the embodiments of the present application, the large language model has excellent capabilities in natural language understanding and complex task reasoning, and these capabilities are highly compatible with the requirements of the test case generation task. The model can understand the natural language description of risk sentences in the protocol document, analyze the rules and constraints therein, and generate test cases that meet the specifications. However, directly using the large language model to solve this complex professional task will have problems such as insufficient professionalism and lack of standardization, and it is necessary to combine prompt engineering to guide the behavior of the model through reasonable prompt strategies.

[0053] S102B. Set the expert role positioning in the first prompt engineering language model to guide the first prompt engineering language model to think about risk sentences in the role of a wireless communication protocol security expert, and obtain a second prompt engineering language model.

[0054] In the embodiments of the present application, the key to setting the expert role positioning lies in accurately defining the professional attributes of the role. Among them, the professional attributes of the role may include professional fields, technical directions, and job responsibilities. This multi-dimensional attribute definition enables the large language model to accurately understand the role it plays, so as to maintain a professional perspective when processing tasks.

[0055] In this step, by setting the identity of a wireless communication protocol security expert in the first prompt engineering language model, the first prompt engineering language model can clarify the professional field boundary and ability requirements, and then can process test tasks based on professional knowledge.

[0056] Here, when setting the expert role positioning, the definition of the professional field adopts a convergent method from large to small, that is, first determine the large technical field, and then limit the specific protocol scope, so as to avoid the problem of overly broad role definition and prevent the problem of lack of professional knowledge caused by excessive restrictions.

[0057] Furthermore, in order to give full play to the role of the expert role positioning, the prompt engineering language model adopts a scheme consistent with the expert work mode in the interaction design, providing a clear professional identity and work paradigm for the prompt engineering language model. The role positioning not only restricts the behavior mode of the model, but more importantly, establishes a professional thinking framework, enabling the model to always think and solve problems in the identity of a test expert, improving the professionalism and standardization of test case generation.

[0058] S102C. Inject the preset signaling value space table, parameter type value space table, parameter value space table, and process definition information table into the second prompt engineering language model respectively according to the preset table reference relationship, so that the second prompt engineering language model learns the professional knowledge in the tables and obtains the third prompt engineering language model.

[0059] In the embodiment of the present application, various types of knowledge in the NAS protocol are organized in a tabular manner, including signaling types, parameter specifications, process definitions, etc., and input into the second prompt engineering language model to facilitate the large language model to accurately understand and apply this professional knowledge.

[0060] Among them, the signaling value space table is used to help the large language model determine the downlink signaling that needs to be constructed, so that the model can select the appropriate downlink signaling in the signaling value space table according to different test scenarios.

[0061] The parameter type value space table adopts a hierarchical tabular scheme to organize parameter knowledge, specifically including local parameters, global parameters, and sentence dynamic parameters; among them, the local parameters clearly define the specific parameters corresponding to each signaling type, and the global parameter table defines the general parameters applicable to multiple signaling.

[0062] The parameter value space table clearly defines the value space of the parameters to ensure that the test cases generated by the model comply with the protocol specifications. The parameter value space table designs a detailed value space table for each parameter, clearly indicating the parameter value range, usage scenarios and legality requirements, so that the large language model can accurately select appropriate parameter values ​​and understand parameter constraints in different scenarios.

[0063] For example, the value space table of the security header type parameter not only defines the specific values, but also explains the usage scenarios of each value, providing an accurate reference for test case generation.

[0064] In this way, tabular knowledge is injected into the model, making the complex knowledge structure clear and intuitive. The large language model can accurately understand and extract the required information, and the table reference relationship establishes a complete link for the knowledge, so that the signaling type, parameters and value range form a whole. The tabular knowledge structure also improves the testing efficiency, enabling the model to quickly locate and apply the required knowledge, thereby improving the speed and quality of test case generation.

[0065] S102D. Decompose the process from analyzing risk sentences to generating test cases into steps to obtain step division information and reasoning process information, and inject the step division information and the reasoning process information into the third prompt engineering language model to guide the third prompt engineering language model to accurately generate test cases corresponding to risk sentences, and obtain the prompt engineering language model.

[0066] In this step, the thinking chain example-driven tool based on the thinking chain example-driven technology is used to decompose the process of analyzing risk sentences to generating test cases into steps, and obtain clear step division information and reasoning process information including the refinement of the single step reasoning process and the collaboration between each reasoning step. The step division information and the reasoning process information are injected into the third prompt engineering language model to guide the third prompt engineering language model to accurately generate test cases corresponding to risk sentences through the thinking chain-based example-driven technology, so as to obtain the prompt engineering language model.

[0067] It should be noted that the third prompt engineering language model may still have problems such as reasoning jumps and logical confusion when dealing with complex tasks. Especially when it is necessary to use multiple technologies in combination, the model often finds it difficult to accurately grasp the relationship between each link.

[0068] To address the above issues, the Thinking Chain Example-Driven Tool, which is based on the example-driven technology of Thinking Chain, guides the model to generate accurate test cases by providing a complete and detailed thinking process from risk sentence analysis to test case generation. The design of Thinking Chain is divided into two main parts: sentence understanding and test case generation.

[0069] In this way, the model can refer to the analysis ideas and generation logic in the examples, improve the quality and accuracy of the test cases, and encourage the large language model to explain its reasoning process. The main idea of ​​the thinking chain is to explain the reasoning process in the examples by showing the large language model a small number of examples. The large language model will also show the reasoning process when answering prompts and generating test cases. The examples not only cover the key links of sentence understanding and use case generation, but also show how to apply various rules and constraints.

[0070] In the embodiment of the present application, since the risk sentence can be understood from two dimensions, positive and reverse, for example, the positive sentence of a risk sentence means: when NAS security is not established, the UE can accept the listed a~g signaling; the reverse sentence means: except for the specified a~g signaling, the UE cannot accept the rest of the downlink signaling.

[0071] Based on the above understanding, the test cases include positive test cases and negative test cases, that is, the NAS protocol consistency test can be divided into two dimensions: positive test and negative test. This two-dimensional test strategy ensures the integrity of the test.

[0072] Among them, the positive test verifies whether the terminal device meets the positive constraints of the risk sentence, and the negative test verifies whether the terminal handles the reverse constraints of the risk sentence correctly.

[0073] Here, negative testing is very necessary. A simple positive test can only ensure that the terminal device meets the positive requirements expressed in the risk sentence, but it cannot rule out the possibility that the terminal device returns the same response in all cases (including abnormal scenarios). Therefore, relying solely on positive testing is not enough to ensure the reliability and security of the system.

[0074] For example, for a certain risk sentence, if only positive testing is performed, although it can be verified that the terminal device UE accepts the signaling in a~g when NAS security is not established, it can be reasonably suspected that the UE will accept any downlink signaling in this state (the signaling in a~g is only a subset of it). In order to eliminate this potential security risk, it is necessary to use negative testing to verify the terminal's rejection of non-a~g signaling.

[0075] Furthermore, when designing the test case structure, it is necessary to consider three aspects: the comprehensiveness of the test scenario information, the ease of the test system to automatically parse and execute the test cases, and the readability of the test cases. Among them, for the test scenario information, it is necessary to clarify in the test case: what state the test system needs to be placed in, what downlink signaling the network side needs to construct and send to the terminal device, and what the expected response of the terminal device is.

[0076] Based on the above considerations, the structure of the test case includes a unique identifier, uplink signaling, downlink signaling, operation type, parameter set, expected result, and the corresponding target risk sentence.

[0077] Among them, the unique identifier is the unique identifier of the test case, and the naming rule is "abbreviation of uplink signaling_abbreviation of downlink signaling_sequence number". For example, for the test case of uplink signaling Registration Request and downlink signaling IdentityRequest, its unique identifier id can be expressed as "regreq_idereq_001".

[0078] The uplink signaling (ue_ul_handle) is used to place the conformance test in the required state, trigger the test process by intercepting the uplink signaling, and guide the communication process to the test scenario in the test requirement information.

[0079] The downlink signaling (dl_reply) is the downlink signaling that the test needs to construct and send to the 5G terminal. This signaling directly affects the specific implementation of the test scenario and needs to strictly follow the signaling format defined in the NAS protocol specification.

[0080] The operation type (command_mode) is used to specify the operation type of the downlink signaling. The optional values are send and replay. Among them, send means sending once, and replay means sending twice (i.e., replaying).

[0081] The parameter set (dl_params) is used to define the parameter set in the constructed downlink signaling, including specific scenario parameters, specific signaling parameters, dynamic parameters specified by the risk sentence, etc.

[0082] The expected result (expected_result) is used to describe the expected response behavior of the terminal to analyze whether the test result conforms to the preset NAS protocol.

[0083] The corresponding target risk sentence indicates that the test case is associated with the corresponding risk sentence and is used for the traceability analysis of the test case.

[0084] In an implementation manner of the present application, when specifically implemented, the step of using the prompt engineering language model to reason about the target risk sentence in step S102 to generate at least one test case corresponding to each target risk sentence may include:

[0085] S1021. For each of the target risk sentences, the prompt engineering language model determines the security test status corresponding to the test case based on the first test status type in the target risk sentence; wherein, the security test status includes NAS security not established and NAS security established.

[0086] In the embodiments of the present application, the first test status type represents the specified situation of the security test status, and may include explicit specification, semi-implicit specification, and full implicit.

[0087] Among them, when the first test status type is explicit specification, that is, the target risk sentence directly indicates that the security test status is "NAS security not established" or "NAS security successfully established". At this time, the security test status corresponding to the test case can be directly determined according to the security test status specified in the risk sentence.

[0088] When the first test status type is semi-implicit specification, that is, although the target risk sentence does not directly indicate the security test status, it contains information from which the status can be deduced. For example, when the risk sentence mentions the "5GMM_CONNECTED state", it can be inferred that NAS security has been successfully established at this time.

[0089] When the first test status type is full implicit, there is no information in the target risk sentence that can be used to judge the test status. In this case, test cases need to be generated for the two security test statuses of "NAS security not established" and "NAS security established" respectively.

[0090] S1022. Based on the security test status, determine the uplink signaling corresponding to the test case.

[0091] In the embodiments of the present application, when the security test status is determined to be "NAS security not established", the uplink signaling corresponding to the test case should be set to "Registration Request"; when the security test status is determined to be "NAS security successfully established", the uplink signaling corresponding to the test case should be set to "Security Mode Complete".

[0092] Here, when the uplink signaling is "Registration Request", it indicates that the 5G mobile communication system has just initiated a registration request but has not established a NAS security connection; when the uplink signaling is "Security Mode Complete", it means that the 5G mobile communication system has completed the NAS security negotiation process and successfully established a NAS security connection.

[0093] S1023. Determine the downlink signaling corresponding to the positive test case and the negative test case respectively based on the second test status type and mobility management information in the target risk sentence.

[0094] In the embodiments of the present application, the mobility management information may include, but is not limited to, research registration, service request, configuration update, deregistration process, etc. That is, based on the mobility management information in the target risk sentence, multiple downlink instructions can be determined.

[0095] Exemplarily, multiple downlink instructions corresponding to the mobility management information in the target risk sentence are shown in the following table.

[0096]

[0097] Here, the second test status type indicates the specification situation of the downlink signaling, and may include explicit specification, semi-implicit specification, and full implicit.

[0098] In this step, when determining the downlink signaling corresponding to the positive test case, the value range of the downlink signaling is determined according to the positive sentence meaning and signaling process of the target risk sentence. In this way, the prompt engineering language model can traverse the value space of the downlink signaling to generate test cases.

[0099] When the second test status type is explicit specification, that is, the target risk sentence clearly specifies the value space of the downlink signaling to be constructed. At this time, the downlink signaling is determined by traversing the signaling value space specified by the target risk sentence.

[0100] When the second test status type is semi-implicit specification, that is, although the target risk sentence does not directly specify the value space of the downlink signaling, it indicates the expected response of the 5G terminal device to the downlink signaling. At this time, the model can combine the professional knowledge of the signaling process or the context of the target risk sentence to infer the downlink signaling corresponding to the expected response.

[0101] When the second test status type is full implicit, that is, the target risk sentence neither specifies the value space of the downlink signaling nor can the specific signaling be inferred. At this time, it is necessary to traverse multiple downlink instructions corresponding to the mobility management information to determine the downlink signaling.

[0102] Furthermore, when determining the downlink signaling corresponding to the negative test case, first, fully understand the positive meaning of the risk sentence, and then further deduce its reverse meaning. The key to the negative test case is to determine the value space of the downlink signaling through the reverse meaning of the risk sentence and signaling process knowledge.

[0103] Specifically, first, clarify the test focus of the target risk sentence; then, according to the test focus, negative test cases can be divided into the signaling layer and the parameter layer. Negative test cases at the signaling layer need to traverse the negative value space of the downlink signaling, and the value space of the downlink signaling for negative test cases at the parameter layer is the same as that for positive tests.

[0104] Among them, the test focus may include but is not limited to cases where the target risk sentence focuses on testing the signaling itself, the target risk sentence focuses on testing the parameters in the signaling, and the target risk sentence involves both signaling and parameter testing.

[0105] Exemplarily, when the target risk sentence focuses on integrity protection testing, integrity protection is mainly determined by signaling parameters (security header, MAC code). Therefore, in the test scenario of this target risk sentence, negative tests should focus on the parameter layer, and there is no need to consider the negative space at the signaling layer. When determining the downlink signaling, its positive value space (all downlink signaling) should be traversed, but negative tests should be carried out at the parameter layer, that is, to ensure that each downlink signaling uses incorrect integrity protection parameters.

[0106] Another example is when the target risk sentence involves both the signaling layer and the parameter layer in terms of test focus. At the signaling layer, the sentence states that when NAS security is not established, except for the signaling listed from a to g, other downlink signaling cannot be accepted by the UE. Therefore, in the process of converting this risk sentence into negative test cases, the value space of the downlink signaling should be all downlink signaling except for the signaling from a to g.

[0107] S1024. Based on the test scenario information in the test requirement information, the downlink signaling, and the parameter indication message in the target risk sentence, determine the target parameters corresponding to the downlink signaling in the security test state, and determine the value space corresponding to each target parameter, so as to determine the parameter sets corresponding to the positive test cases and the negative test cases respectively.

[0108] In the embodiments of the present application, the target parameter is the parameter that needs to be constructed in the downlink signaling. Different test scenarios are constructed by setting different parameter combinations. When determining the target parameter, first determine the parameter types to be tested, and then determine the value space of these parameter types; among them, the number of target parameters is greater than or equal to 0.

[0109] Exemplarily, for different security test states, the downlink signaling corresponding to the parameter type and the corresponding security risks are shown in the following table.

[0110]

[0111] It should be noted that the parameter compositions of NAS signaling in different security test states have both commonalities and differences. The commonality is that all NAS signaling contains the "security header type" parameter in any state. The difference is that when NAS security is not established, all signaling does not contain the "MAC" parameter, and after NAS security is established, all signaling will contain the "MAC" parameter.

[0112] Among them, the "security header type" parameter reflects the integrity protection and encryption status of NAS signaling, and the "MAC" parameter is used for integrity protection verification. When NAS security is not established, except for the "Security Mode Command" signaling, other signaling does not need to test the "security header type" parameter.

[0113] In addition, in the test after NAS security is successfully established, if you want to test the integrity protection verification function of the 5G terminal, you need to operate the "MAC" parameter to verify valid or invalid integrity protection. However, considering that some parameter combinations may interfere with the normal check of integrity protection, resulting in the 5G terminal device showing the wrong behavior of successfully passing the integrity protection check of the signaling when receiving signaling with abnormal integrity protection. Therefore, the test of this function must cover all possible risky parameter combinations.

[0114] Furthermore, when the downlink signaling is a specific signaling, for security considerations, the target parameters must include its corresponding risk parameters. In addition, the target risk sentence may also specify the corresponding test parameters, and this kind of sentence dynamic parameter also needs to be added to the test parameters. Therefore, when determining the target parameters corresponding to the downlink signaling in the security test state, it is determined from three aspects: the test scenario (whether the test scenario is a specific scenario), the unique parameters of the signaling (whether the downlink signaling has specific risk parameters), and the dynamic parameters of the risk sentence indication parameters (whether the target risk sentence specifies specific parameters).

[0115] In an implementation manner of the present application, in specific implementation, the step of determining the target parameters corresponding to the downlink signaling in the security test state based on the test scenario information in the test requirement information, the downlink signaling, and the parameter indication message in the target risk sentence in step S1024 may include:

[0116] S10241. Determine whether the downlink signaling is one of the specific signaling to obtain a judgment result.

[0117] In an embodiment of the present application, the specific signaling may include "Identity Request" and "Security Mode Command", and each specific signaling contains its respective specific risk parameter. When the downlink signaling is these two signals, for security considerations, the corresponding risk parameter must be included in the target parameter.

[0118] S10242. When the security test status is that NAS security has been established, based on the corresponding combination among the test scenario information in the test requirement information, the judgment result, and the parameter indication information in the target risk sentence, determine the target parameter corresponding to the downlink signaling.

[0119] In an embodiment of the present application, when the security test status is that NAS security has been established, an example of determining the target parameter corresponding to the downlink signaling is shown in the following table.

[0120]

[0121]

[0122] S10243. When the security test status is that NAS security has not been established, based on the corresponding combination between the judgment result and the parameter indication information in the target risk sentence, determine the target parameter corresponding to the downlink signaling.

[0123] In an embodiment of the present application, when the security test status is that NAS security has not been established, an example of determining the target parameter corresponding to the downlink signaling is shown in the following table.

[0124]

[0125]

[0126] Further, in step S1024, determine the value space corresponding to each target parameter to determine the parameter sets corresponding to the positive test cases and the negative test cases respectively.

[0127] In this step, in order to clarify the value space corresponding to the target parameters in the positive test cases and the negative test cases, first, determine the complete value space of each target parameter. Then, for each target parameter, if the value space of the target parameter in the positive test case is A, then the value space in the negative test case is the complement of A.

[0128] Among them, an example of the complete value space of each target parameter is shown in the following table.

[0129]

[0130]

[0131] Here, the value space of each target parameter may include legal values and illegal values, and the model can determine the corresponding value space according to the test scenario information.

[0132] Exemplarily, when the downlink signaling is "Security Mode Command", the legal value space of the target parameter "securityheader type" is "Integrity protected with new 5GNAS securitycontext", and the illegal value space is the remaining 4 value spaces; when the downlink signaling is not "Security ModeCommand", the legal value space of the target parameter "security header type" is "Integrity protectedand ciphered", and the illegal value space is the remaining 4 values.

[0133] As another example, the overall value space of the target parameter "MAC" has only 2 types. One is a valid MAC code, that is, correct integrity protection is performed on the signaling, and the other is an invalid MAC code, that is, incorrect integrity protection is performed on the signaling. Therefore, the legal value space of the target parameter "MAC" is the valid MAC, and the illegal value space is the invalid MAC.

[0134] S1025. Analyze and reason about the target risk sentence to determine the operation types and expected results corresponding to the positive test case and the negative test case respectively.

[0135] In this step, analyze and reason about the target risk sentence, and respectively determine the corresponding operation types and expected results for the positive test case and the negative test case.

[0136] Among them, the operation types may include send and replay.

[0137] Here, when determining the expected result, prompt the engineering language model to be able to generate a clear and accurate description of the expected result through a preset format template. Whether it is a positive test case or a negative test case, the expected result can clearly indicate the behavior that the target 5G terminal device should take.

[0138] S1026. Generate at least one positive test case and at least one negative test case corresponding to each target risk sentence based on the uplink signaling, the downlink signaling, the parameter set, the operation type, the expected result, and the target risk sentence.

[0139] In this step, based on the structure of the test cases, the prompt engineering language model can integrate the results of step-by-step reasoning including uplink signaling, downlink signaling, parameter sets, operation types, expected results, and target risk sentences into at least one positive test case and at least one negative test case, and generate a corresponding unique identifier for each test case.

[0140] In this way, through the multi-step reasoning of the prompt engineering language model based on step decomposition, the complex test case generation task becomes controllable and reliable. By clearly dividing steps and refining the reasoning process of individual steps, the prompt engineering language model can effectively handle various test scenarios, cooperate between individual reasoning steps, making the prompt engineering language model have good maintainability and scalability, and at the same time ensuring the quality of the generated test cases.

[0141] S103. Use the prompt engineering language model to standardize the format of each test case to generate a target test case corresponding to each test case.

[0142] It should be noted that the output format of test cases directly affects their usability and maintainability. When the prompt engineering language model generates test cases, problems such as inconsistent formats, missing fields, or incorrect field types may occur. These problems will seriously affect the actual application of test cases. Especially in an automated test environment, test cases with incorrect formats may cause the entire test process to fail to execute properly.

[0143] To address the above problems, the constraint specification module set in the prompt engineering language model is used to perform normalization constraints on the output test cases to ensure that the generated test cases meet the predetermined format requirements and specifications.

[0144] In an implementation manner of the present application, in specific implementation, step S103 may include:

[0145] S1031. The constraint specification module in the prompt engineering language model performs format constraint specification on each test case to generate a test case to be verified corresponding to each test case.

[0146] In the embodiments of the present application, the constraint specification module in the prompt engineering language model defines the format requirements of test cases, including constraint conditions such as field definitions, data types, and value ranges.

[0147] Among them, the constraint specification module may include a JSON Schema technology, which ensures the consistency of test case output, improves the usability of the generated test cases, and can use the JSON Schema technology in NAS protocol consistency testing to ensure that the generated test cases meet the predefined format requirements, facilitating subsequent processing and execution.

[0148] S1032. For each of the to-be-verified test cases, perform format integrity verification, data type verification, and value range verification in sequence, and obtain a format integrity verification result, a data type verification result, and a value range verification result respectively.

[0149] In the embodiment of the present application, each to-be-verified test case is sequentially verified at three levels, including format integrity verification, data type verification, and value range verification, to ensure that the generated test cases meet all constraint requirements.

[0150] Among them, format integrity verification focuses on the existence of required fields. In "JSON Schema", all fields are clearly specified as required through the "required" attribute, ensuring that each to-be-verified test case contains complete test information. Format integrity verification will check whether these fields all exist, and the absence of any field will determine that the format integrity verification result is verification failure.

[0151] Data type verification checks the type requirements for each field, specifically including: the unique identifier of the to-be-verified test case must be a string conforming to a specific pattern; the uplink signaling and operation type must be one of the predefined enumeration values; the downlink signaling must be a valid downlink message type; the parameter set must be an object containing legal parameter configurations; the expected result must be a string describing the expected behavior.

[0152] Value range verification checks whether the field values are within the allowed range to ensure the correctness of the generated test cases at the protocol specification level, specifically including: verifying whether the downlink signaling is within the value space of the downlink signaling; whether the target parameters in the parameter set meet the requirements of a specific signaling type; whether the parameter values of the target parameters are within the defined parameter value space.

[0153] S1033. When the format integrity verification result, the data type verification result, and the value range verification result are all verification passed, determine the to-be-verified test case as a target test case.

[0154] In this step, when the format integrity verification result, the data type verification result, and the value range verification result of each to-be-verified test case are all verification passed, determine the to-be-verified test case as a target test case.

[0155] When at least one of the format integrity verification result, the data type verification result, and the value range verification result of each to-be-verified test case is verification failed, remove the to-be-verified test case as a non-conforming test case.

[0156] In this way, the constraint specification module based on "JSON Schema" significantly improves the standardization and reliability of test cases through strict format definitions and multi-level verification mechanisms. The design of "Schema" fully considers the characteristics of test cases, and the verification mechanism ensures that these design requirements can be strictly enforced. The generated test cases not only have a unified structure, facilitating tool parsing, but also provide a good foundation for subsequent test execution and result analysis.

[0157] S104. Based on the target test cases, use a preset test tool to test the target 5G terminal device, and obtain the test results corresponding to each target test case to determine whether the target 5G terminal device complies with the NAS protocol in the test scenario of the test requirement information.

[0158] In the embodiments of the present application, from the perspective of conformance testing, two aspects need to be focused on in testing: First, the test should be able to verify whether the 5G network complies with the constraints of the NAS protocol specification; Second, the test needs to support a comprehensive conformance assessment of 5G terminal devices from different manufacturers.

[0159] In an implementation manner of the present application, in specific implementation, step S104 may include:

[0160] S1041. Use a preset test tool to test the target 5G terminal device according to each target test case, and obtain the test results corresponding to each target test case.

[0161] In the embodiments of the present application, for the 5G mobile communication system as Figure 2 shown, the test tool may include a 5G core network simulator and a 5G software radio suite.

[0162] Among them, the 5G core network simulator provides all 5G core network functions such as AMF, SMF, and UPF, and supports standard N1 / N2 / N3 interfaces. It can flexibly configure network functions and simulate various network scenarios, providing a core network environment for NAS protocol testing; The 5G software radio suite can cooperate with a general software radio peripheral (USRP) to implement complete radio access network functions and simulate the behavior of 5G base stations.

[0163] In an implementable manner of the present application, an Open5GS tool may be selected for the 5G core network simulator, and an SrsRAN_Project tool may be selected for the 5G software radio suite.

[0164] S1042. Respectively determine whether each test result is consistent with the expected result in the corresponding target test case.

[0165] In this step, by determining whether each of the test results is consistent with the expected result in the corresponding target test case, it is determined whether the target 5G terminal device complies with the NAS protocol in the test scenario of the test requirement information.

[0166] S1043. When each of the test results is consistent with the corresponding expected result, it is determined that the target 5G terminal device complies with the NAS protocol in the test scenario of the test requirement information.

[0167] S1044. When there is at least one test result that is inconsistent with the corresponding expected result, it is determined that the target 5G terminal device does not comply with the NAS protocol in the test scenario of the test requirement information.

[0168] The test method for 5G terminal NAS protocol consistency provided by the embodiments of the present application defines the test case structure and core components for NAS protocol consistency testing, clarifies the test case design rules, designs the writing standards for positive and negative test cases, ensures a high coverage rate of test cases, and uses the understanding ability and knowledge representation characteristics of the large language model for communication protocol texts, and uses prompt engineering to guide the model to accurately understand the test requirements and protocol key points, realizing the automatic conversion from the risk sentences determined from the NAS protocol by the prompt engineering language model to test cases, improving the accuracy and efficiency of 5G terminal NAS protocol consistency testing, and further improving the comprehensiveness and flexibility of consistency testing.

[0169] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of a test device for 5G terminal NAS protocol consistency provided by the embodiments of the present application. As Figure 3 shown in, the test device 300 includes:

[0170] A requirement processing module 310, configured to respond to the obtained test requirement information for performing NAS protocol consistency testing on a target 5G terminal device, and determine at least one target risk sentence in a preset NAS protocol based on the test requirement information;

[0171] A use case generation module 320, configured to input the target risk sentence into a preset prompt engineering language model, and use the prompt engineering language model to reason about the target risk sentence to generate at least one test case corresponding to each target risk sentence;

[0172] A constraint specification module 330, configured to use the prompt engineering language model to perform format constraint specification on each test case to generate a target test case corresponding to each test case;

[0173] The use case testing module 340 is used to test the target 5G terminal device based on the target test case by using a preset test tool, obtain the test results corresponding to each target test case, and determine whether the target 5G terminal device complies with the NAS protocol in the test scenario in the test requirement information.

[0174] Further, when the use case generation module 320 is used to set the prompt engineering language model, the use case generation module 320 is used to:

[0175] Input a preset prompt engineering information file into a preset large language model, so that the large language model performs knowledge loading based on the prompt engineering information file to obtain a first prompt engineering language model;

[0176] Set an expert role positioning in the first prompt engineering language model to guide the first prompt engineering language model to think about risk sentences in the role of a wireless communication protocol security expert, and obtain a second prompt engineering language model;

[0177] Inject a preset signaling value space table, a parameter type value space table, a parameter value space table, and a process definition information table into the second prompt engineering language model respectively according to a preset table reference relationship, so that the second prompt engineering language model learns the professional knowledge in the tables and obtains a third prompt engineering language model;

[0178] Decompose the process from analyzing risk sentences to generating test cases into step division information and reasoning process information, and inject the step division information and the reasoning process information into the third prompt engineering language model to guide the third prompt engineering language model to accurately generate test cases corresponding to risk sentences, and obtain a prompt engineering language model.

[0179] Further, when the use case generation module 320 is used to infer the target risk sentences by using the prompt engineering language model and generate at least one test case corresponding to each target risk sentence, the use case generation module 320 is used to:

[0180] For each target risk sentence, the prompt engineering language model determines the security test status corresponding to the test case based on the first test status type in the target risk sentence; wherein, the security test status includes NAS security not established and NAS security established;

[0181] Based on the security test status, determine the uplink signaling corresponding to the test case;

[0182] Determine the downlink signaling corresponding to the positive test case and the negative test case respectively based on the second test status type and mobility management information in the target risk sentence;

[0183] Based on the test scenario information in the test requirement information, the downlink signaling, and the parameter indication message in the target risk sentence, determine the target parameters corresponding to the downlink signaling in the security test state, and determine the value space corresponding to each target parameter, so as to determine the parameter sets corresponding to the positive test case and the negative test case respectively;

[0184] Analyze and reason the target risk sentence to determine the operation types and expected results corresponding to the positive test case and the negative test case respectively;

[0185] Based on the uplink signaling, the downlink signaling, the parameter set, the operation type, the expected result, and the target risk sentence, generate at least one positive test case and at least one negative test case corresponding to each target risk sentence.

[0186] Furthermore, when the use case generation module 320 is used to determine the target parameters corresponding to the downlink signaling in the security test state based on the test scenario information in the test requirement information, the downlink signaling, and the parameter indication message in the target risk sentence, the use case generation module 320 is used to:

[0187] Judge whether the downlink signaling is one of the specific signals to obtain a judgment result;

[0188] When the security test state is that NAS security has been established, determine the target parameters corresponding to the downlink signaling based on the corresponding combination among the test scenario information in the test requirement information, the judgment result, and the parameter indication information in the target risk sentence;

[0189] When the security test state is that NAS security has not been established, determine the target parameters corresponding to the downlink signaling based on the corresponding combination between the judgment result and the parameter indication information in the target risk sentence.

[0190] Furthermore, when the constraint specification module 330 is used to perform format constraint specification on each test case by using the prompt engineering language model to generate the target test case corresponding to each test case, the constraint specification module 330 is used to:

[0191] The constraint specification module in the prompt engineering language model performs format constraint specification on each test case to generate a test case to be verified corresponding to each test case;

[0192] For each of the to-be-verified test cases, format integrity verification, data type verification, and value range verification are sequentially performed to obtain a format integrity verification result, a data type verification result, and a value range verification result, respectively.

[0193] When the format integrity verification result, the data type verification result, and the value range verification result all pass the verification, the to-be-verified test case is determined as a target test case.

[0194] Further, when the use case testing module 340 is used to test the target 5G terminal device based on the target test case by using a preset test tool to obtain a test result corresponding to each target test case, so as to determine whether the target 5G terminal device conforms to the NAS protocol in the test scenario of the test requirement information, the use case testing module 340 is used for:

[0195] Using the preset test tool to test the target 5G terminal device according to each target test case to obtain a test result corresponding to each target test case;

[0196] Respectively determine whether each test result is consistent with the expected result in the corresponding target test case;

[0197] When each test result is consistent with the corresponding expected result, it is determined that the target 5G terminal device conforms to the NAS protocol in the test scenario of the test requirement information;

[0198] When there is at least one test result that is inconsistent with the corresponding expected result, it is determined that the target 5G terminal device does not conform to the NAS protocol in the test scenario of the test requirement information.

[0199] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 4 shown in

[0200] the electronic device 400 includes a processor 410, a memory 420, and a bus 430. Figure 1 The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 runs, the processor 410 communicates with the memory 420 through the bus 430. When the machine-readable instructions are executed by the processor 410, the steps of the 5G terminal NAS protocol consistency test method in the method embodiment as described above can be executed. The specific implementation manner can refer to the method embodiment and will not be elaborated here.

[0201] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it can execute the steps of the method for testing the NAS protocol consistency of a 5G terminal as described in the foregoing Figure 1 method embodiment shown. For the specific implementation manner, reference can be made to the method embodiment, which will not be elaborated herein.

[0202] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated herein.

[0203] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0204] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0205] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0206] When the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0207] Finally, it should be noted that: the above-mentioned embodiments are only specific implementation manners of the present application, used to illustrate the technical solutions of the present application, rather than limiting them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present application can still modify the technical solutions described in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application and should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A 5G terminal NAS protocol consistency testing method, characterized in that: The test method includes: In response to the acquired test requirement information for performing NAS protocol conformance test on the target 5G terminal device, at least one target risk sentence is determined in a preset NAS protocol based on the test requirement information; Inputting the target risk sentence into a preset prompt engineering language model, using the prompt engineering language model to infer the target risk sentence, and generating at least one test case corresponding to each target risk sentence; Using the prompt engineering language model to perform format constraint specification on each of the test cases, and generating a target test case corresponding to each of the test cases; Based on the target test case, the target 5G terminal device is tested using a preset test tool to obtain a test result corresponding to each target test case to determine whether the target 5G terminal device complies with the NAS protocol under the test scenario in the test requirement information.

2. The method according to claim 1, characterized in that The prompt engineering language model is set by following the steps below: Inputting a preset prompt engineering information file into a preset large language model, so that the large language model performs knowledge loading based on the prompt engineering information file to obtain a first prompt engineering language model; Setting an expert role positioning in the first prompt engineering language model to guide the first prompt engineering language model to think about risk sentences in the role of a wireless communication protocol security expert, thereby obtaining a second prompt engineering language model; Injecting the preset signaling value space table, parameter type value space table, parameter value space table and process definition information table into the second prompt engineering language model according to the preset table reference relationship, so that the second prompt engineering language model learns the professional knowledge in the table to obtain a third prompt engineering language model; The process from analyzing risk sentences to generating test cases is decomposed into steps to obtain step division information and reasoning process information, and the step division information and the reasoning process information are injected into the third prompt engineering language model to guide the third prompt engineering language model to accurately generate test cases corresponding to risk sentences, thereby obtaining a prompt engineering language model.

3. The method according to claim 1, characterized in that The test cases include positive test cases and negative test cases; the structure of the test cases includes a unique identifier, uplink signaling, downlink signaling, operation type, parameter set, expected result and corresponding target risk sentence.

4. The method according to claim 3, characterized in that The using the prompt engineering language model to reason the target risk sentences to generate at least one test case corresponding to each target risk sentence includes: The prompt engineering language model determines, for each of the target risk sentences, a security test state corresponding to the test case based on the first test state type in the target risk sentence; wherein the security test state includes NAS security not established and NAS security established; Based on the security test status, determining the uplink signaling corresponding to the test case; Determine downlink signaling corresponding to the positive test case and the negative test case respectively based on the second test state type and the mobility management information in the target risk sentence; Based on the test scenario information in the test requirement information, the downlink signaling and the parameter designation message in the target risk sentence, determine the target parameter corresponding to the downlink signaling in the safety test state, and determine the value space corresponding to each of the target parameters to determine the parameter sets corresponding to the positive test case and the negative test case respectively; Analyze and reason the target risk sentence to determine the operation type and expected result corresponding to the positive test case and the negative test case respectively; Based on the uplink signaling, the downlink signaling, the parameter set, the operation type, the expected result and the target risk sentence, at least one positive test case and at least one negative test case corresponding to each target risk sentence are generated.

5. The method according to claim 4, characterized in that The determining, based on the test scenario information in the test requirement information, the downlink signaling, and the parameter designation message in the target risk sentence, a target parameter corresponding to the downlink signaling in the safety test state comprises: Determine whether the downlink signaling is one of the specific signalings, and obtain a determination result; When the security test state is that NAS security is established, determining the target parameter corresponding to the downlink signaling based on the corresponding combination of the test scenario information in the test requirement information, the judgment result and the parameter specifying information in the target risk sentence; When the security test state is that NAS security is not established, the target parameter corresponding to the downlink signaling is determined based on the corresponding combination between the judgment result and the parameter designation information in the target risk sentence.

6. The method according to claim 1, characterized in that The step of using the prompt engineering language model to perform format constraint specification on each of the test cases to generate a target test case corresponding to each of the test cases includes: The constraint specification module in the prompt engineering language model performs format constraint specification on each of the test cases to generate a test case to be verified corresponding to each of the test cases; Perform format integrity verification, data type verification and value range verification on each of the test cases to be verified, and obtain format integrity verification results, data type verification results and value range verification results respectively; When the format integrity verification result, the data type verification result and the value range verification result are all verified to pass, the test case to be verified is determined as a target test case.

7. The method according to claim 1, characterized in that The testing of the target 5G terminal device based on the target test case using a preset test tool to obtain a test result corresponding to each of the target test cases to determine whether the target 5G terminal device complies with the NAS protocol under the test scenario in the test requirement information includes: Using a preset test tool to test the target 5G terminal device according to each of the target test cases, and obtaining a test result corresponding to each of the target test cases; Determine whether each of the test results is consistent with the expected result in the corresponding target test case; When each of the test results is consistent with the corresponding expected result, it is determined that the target 5G terminal device complies with the NAS protocol under the test scenario in the test requirement information; When at least one of the test results is inconsistent with the corresponding expected result, it is determined that the target 5G terminal device does not comply with the NAS protocol under the test scenario in the test requirement information.

8. A 5G terminal NAS protocol consistency test device, characterized in that: The testing device comprises: A requirement processing module, configured to respond to the acquired test requirement information for performing NAS protocol conformance test on the target 5G terminal device, and determine at least one target risk sentence in a preset NAS protocol based on the test requirement information; A test case generation module, used to input the target risk sentence into a preset prompt engineering language model, use the prompt engineering language model to infer the target risk sentence, and generate at least one test case corresponding to each target risk sentence; A constraint specification module, used to perform format constraint specification on each of the test cases using the prompt engineering language model, and generate a target test case corresponding to each of the test cases; A use case testing module is used to test the target 5G terminal device based on the target test case using a preset test tool to obtain a test result corresponding to each target test case to determine whether the target 5G terminal device complies with the NAS protocol under the test scenario in the test requirement information.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the steps of the 5G terminal NAS protocol consistency testing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the 5G terminal NAS protocol consistency testing method as described in any one of claims 1 to 7 are executed.

Images