5g terminal air interface security automation test system and test method

Abstract

The application relates to a 5G terminal air interface security automatic test system and a test method, wherein each target communication system simulation base station provides a network access environment for a target communication system terminal to be tested, the log similarity of the terminal corresponding to the normal log similarity range and the normal time delay range are obtained by carrying out similarity analysis on the logs from the normal simulation base station and the target communication system terminal to be tested and by counting the elapsed time duration through a log vectorization processing module, the test log similarity and the test time delay of the terminal corresponding to the tested simulation base station under a preset test case are compared with the normal log similarity range and the normal time delay range, and the security test of the air interface protocol stack of the target communication system terminal to be tested is realized; the design scheme of the application not only improves the test efficiency, reduces the labor cost investment, but also significantly enhances the security analysis and recognition precision of a large number of logs of multiple devices.

Description

Technical Field

[0001] This invention relates to an automated testing system and method for 5G terminal air interface security, belonging to the field of artificial intelligence detection technology. Background Technology

[0002] With the rapid development of mobile communication technology and the surge in the number of mobile terminals, ensuring the security of these terminals' wireless interface protocol stacks has become crucial. Attacks on mobile terminals via the air interface can lead to anything from minor issues like information leaks, service interruptions, and terminal restarts, to more serious problems requiring on-site power-off and restarts, or even firmware re-flashing to restore air interface services. Conducting security testing on the air interface of mobile terminals before their deployment, identifying and hardening vulnerabilities in the mobile terminal's air interface protocol stack, not only protects user privacy but also prevents business disruptions caused by malicious attacks or unauthorized use of mobile terminals.

[0003] Traditional 5G air interface security testing primarily relies on firmware analysis. This involves extracting firmware information from 5G terminals and using decompilation to identify firmware content and uncover potential vulnerabilities. However, this approach suffers from inaccurate identification, difficulty in verification, and a large workload for manual inspection. Existing technologies propose conducting 5G terminal air interface security testing using 5G simulated base stations combined with test scripts. By modifying the 5G base station to enable it to send altered air interface protocols, and by collecting and analyzing mobile terminal logs, it is possible to verify whether the mobile terminal has been affected by the test, thereby confirming the presence of relevant vulnerabilities.

[0004] Existing technologies have solved the problem of air interface testing and verification of 5G terminals to some extent, but still have the following shortcomings:

[0005] (1) The testing process requires manual operation of tools to send test cases, manual log collection, and manual identification of a large number of logs generated in the terminal. It requires analyzing each log one by one, which is inefficient. Furthermore, manual log analysis is prone to omissions and cannot carry out a large number of terminal tests.

[0006] (2) When the test involves a terminal air interface access downgrade (from 5G to 4G), it is not possible to effectively distinguish from the logs, which can easily lead to omissions and misjudgments.

[0007] (3) The test relies on terminal logs, which requires the terminal to provide access to the internal log interface or the core permissions of the terminal operating system. If the interface or permissions cannot be obtained, it will be impossible to confirm whether the terminal has been affected by the test. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide an automated testing system for 5G terminal air interface security, which simulates the network environment and introduces a log vectorization processing module to learn normal air interface behavior patterns and quantify abnormal characteristics. Through threshold judgment, it achieves efficient and accurate testing of mobile terminals.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention designs a 5G terminal air interface security automated testing system, including a target communication standard air interface security automated testing orchestration module, a target communication standard air interface security testing simulation base station module, a normal simulation target communication standard base station, a log collection and processing module, and a log vectorization processing module;

[0010] Among them, the normal simulation target communication standard base station is used to provide the target communication standard terminal under test with a normal network access environment under the target communication standard; the target communication standard air interface security test simulation base station module is used to provide the target communication standard terminal under test with a normal network access environment and a test network access environment under the target communication standard.

[0011] The target communication standard air interface security automated test orchestration module is used to control the target communication standard terminal under test to enable or disable the communication function. It controls the target communication standard air interface security test simulation base station module and the normal simulation target communication standard base station to enable or disable respectively. It is also used to send preset test cases to the target communication standard air interface security test simulation base station module. If the target communication standard terminal under test connects to the normal simulation target communication standard base station or the target communication standard air interface security test simulation base station module, the target communication standard air interface security automated test orchestration module obtains the timestamp of the target communication standard terminal connecting to the corresponding base station and sends it to the log collection and processing module.

[0012] The log collection and processing module is used to collect logs on the base station connected to the target communication system terminal based on the timestamp of the process of the target communication system terminal accessing the corresponding base station, and upload them to the log vectorization processing module.

[0013] The log vectorization processing module is used to analyze the received logs, complete the test of the process of the target communication system terminal accessing the base station, and send the test results to the target communication system air interface security automated test orchestration module.

[0014] As a preferred technical solution of the present invention: it also includes a terminal test controller, wherein the target communication standard air interface security automated test orchestration module interacts with the target communication standard terminal under test through the terminal test controller or AT command interface to control the target communication standard terminal under test to turn on or off the communication function.

[0015] The terminal test controller includes a control unit, an instruction sending unit connected to the control unit, and a robotic arm. The control unit receives control instructions from the target communication standard air interface security automated test orchestration module and generates corresponding hardware operation instructions. The hardware operation instructions control the target communication standard terminal under test to turn on or off its communication function via wired or wireless communication in the instruction sending unit, or the hardware operation instructions control the target communication standard terminal under test to turn on or off its communication function via physical contact between the robotic arm and the target communication standard terminal under test.

[0016] As a preferred technical solution of the present invention, it further includes a normal simulated low-generation communication standard base station, which is used to provide a normal network access environment under the low-generation communication standard to the target communication standard terminal under test. The low-generation communication standard is lower than the target communication standard. The target communication standard air interface security automated test orchestration module controls the normal simulated low-generation communication standard base station to be turned on or off. If the target communication standard terminal under test accesses the normal simulated low-generation communication standard base station, the target communication standard air interface security automated test orchestration module obtains the timestamp of the process of the target communication standard terminal accessing the normal simulated low-generation communication standard base station and sends it to the log collection and processing module.

[0017] As a preferred embodiment of the present invention, it further includes a terminal log collection module and a simulated communication core network. The terminal log collection module is used to collect logs with corresponding timestamps on the target communication system terminal based on the timestamps of the process of the target communication system terminal accessing the corresponding base station, and upload them to the log vectorization processing module. The simulated communication core network is used to simulate the communication access control logic under the target communication system and the communication access control logic under the low-generation communication system, and provides them to the normal simulated target communication system base station and the normal simulated low-generation communication system base station, respectively.

[0018] Corresponding to the above, the technical problem that this invention also needs to solve is to provide a testing method based on a 5G terminal air interface security automated testing system, which achieves efficient and accurate testing of mobile terminals by comparing the similarity between logs under different states of the mobile terminal.

[0019] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention designs a test method based on a 5G terminal air interface security automated test system, comprising the following steps:

[0020] Step A. The target communication standard air interface security automated test orchestration module controls the normal simulation target communication standard base station to be turned on and the target communication standard air interface security test simulation base station module to be turned off. Based on the logs of the process of controlling the target communication standard terminal under test to access the normal simulation target communication standard base station multiple times, the log vectorization processing module analyzes the logs to obtain the normal log similarity range corresponding to the target communication standard terminal under test, and sends it to the target communication standard air interface security automated test orchestration module, and then proceeds to Step B.

[0021] Step B. The target communication standard air interface security automated test orchestration module controls the target communication standard air interface security test simulation base station module to turn on and the normal simulation target communication standard base station to turn off. Based on the state before and after the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module, the logs of the process of the target communication standard terminal accessing the target communication standard air interface security test simulation base station module are analyzed by the log vectorization processing module to obtain the test log similarity corresponding to the target communication standard terminal under test, and then sent to the target communication standard air interface security automated test orchestration module, and then proceed to step C.

[0022] Step C. The target communication standard air interface security automated test orchestration module determines whether the similarity of the test logs corresponding to the target communication standard terminal under test is within the range of the corresponding normal log similarity. If yes, it means that the preset test cases have no impact on the target communication standard terminal under test; otherwise, it means that the preset test cases have an impact on the target communication standard terminal under test.

[0023] As a preferred technical solution of the present invention: step A includes the following steps A1 to A6;

[0024] Step A1. The target communication standard air interface security automated test orchestration module controls the normal simulation target communication standard base station to turn on, the target communication standard air interface security test simulation base station module to turn off, and initializes the iteration count. Proceed to step A2;

[0025] Step A2. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be enabled, obtains the start timestamp of the target communication standard terminal under test accessing the normal simulated target communication standard base station, and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The log collection and processing module obtains the logs on the normal simulated target communication standard base station corresponding to the start timestamp to the end timestamp range, forming the first... The first log of the next iteration is then sent to the log vectorization processing module, and then proceeds to step A3;

[0026] Step A3. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned off, and after a preset interval, controls the communication function of the target communication standard terminal under test to be turned on again. It obtains the start timestamp of the target communication standard terminal's access to the normal simulated target communication standard base station and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The log collection and processing module obtains the logs from the normal simulated target communication standard base station corresponding to the start timestamp to the end timestamp range, constituting the first... The second log of the next iteration is then sent to the log vectorization processing module, and then proceeds to step A4;

[0027] Step A4. The log vectorization processing module calculates and obtains the first... The similarity between the first log and the second log is calculated in the next iteration, and then proceeds to step A5;

[0028] Step A5. Judgment Is it equal to the preset maximum number of iterations? If yes, proceed to step A6; otherwise, proceed to step A6. The value is incremented by 1 and the process returns to step A2;

[0029] Step A6. The log vectorization processing module calculates and obtains... The average similarity and standard deviation of all similarities in the next iteration are calculated, and the fluctuation of the standard deviation of similarity around the average similarity is calculated to form the normal log similarity range corresponding to the target communication standard terminal under test. This range is then sent to the target communication standard air interface security automated test orchestration module, and then proceeds to step B.

[0030] As a preferred technical solution of the present invention: step B includes the following steps B1 to B4;

[0031] Step B1. The target communication standard air interface security automated test orchestration module controls the target communication standard air interface security test simulation base station module to turn on and the normal simulation target communication standard base station to turn off, and then proceeds to step B2;

[0032] Step B2. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be enabled, obtains the start timestamp of the target communication standard terminal under test accessing the target communication standard air interface security test simulation base station module, and the end timestamp of the first successful terminal access log, and sends it to the log collection and processing module. The log collection and processing module obtains the logs on the target communication standard air interface security test simulation base station module within the range of the corresponding start timestamp to end timestamp, which constitutes the first log to be analyzed, and then sends it to the log vectorization processing module, and then proceeds to step B3.

[0033] Step B3. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned off. Then, the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module. Then, the target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned on again. The module obtains the start timestamp of the target communication standard terminal under test accessing the target communication standard air interface security test simulation base station module and the end timestamp of the first successful terminal access log. The module sends these timestamps to the log collection and processing module. The log collection and processing module obtains the logs on the target communication standard air interface security test simulation base station module within the range of the start timestamp to the end timestamp, which constitutes the second log to be analyzed. The logs are then sent to the log vectorization processing module, and then the process proceeds to step B4.

[0034] Step B4. The log vectorization processing module calculates the similarity between the first log to be analyzed and the second log to be analyzed, forms the test log similarity, and sends it to the target communication standard air interface security automated test orchestration module, and then proceeds to step C.

[0035] As a preferred embodiment of the present invention: step A further includes obtaining the normal latency range corresponding to the target communication standard terminal under test, and sending it to the target communication standard air interface security automated test orchestration module; wherein, step A2 further includes obtaining the duration from the start timestamp to the end timestamp during the process of the target communication standard terminal under test accessing the normal simulated target communication standard base station, constituting the first... The first delay in the next iteration is sent to the log vectorization processing module; step A3 also includes obtaining the duration from the start timestamp to the end timestamp during the process of the tested target communication system terminal accessing the normal simulated target communication system base station, constituting the first... The second delay in the next iteration is sent to the log vectorization processing module; step A6 also includes the log vectorization processing module calculating and obtaining... The average delay and standard deviation of all delays in each iteration are calculated, and the standard deviation of delay fluctuates around the average delay as the center to form the normal delay range corresponding to the target communication standard terminal under test, and then sent to the target communication standard air interface security automated test orchestration module.

[0036] Step B further includes obtaining the test latency corresponding to the target communication standard terminal under test and sending it to the target communication standard air interface security automated test orchestration module; wherein, step B3 further includes obtaining the duration from the start timestamp to the end timestamp during the process of the target communication standard terminal under test accessing the target communication standard terminal after the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module, which constitutes the test latency corresponding to the target communication standard terminal under test, and sending it to the target communication standard air interface security automated test orchestration module;

[0037] Step C further includes the target communication standard air interface security automated test orchestration module determining whether the test delay corresponding to the target communication standard terminal under test is within the corresponding normal delay range. If it is, it means that the preset test cases have no effect on the target communication standard terminal under test; otherwise, it means that the preset test cases have an effect on the target communication standard terminal under test.

[0038] As a preferred embodiment of the present invention: during the execution of steps A1 to A6 in step A2, the first... Simultaneously with the first log in the next iteration, the terminal log collection module collects logs from the target communication type terminal under test, corresponding to the same start timestamp to end timestamp range as the first log, forming the second... The first terminal log in the next iteration is sent to the log vectorization processing module via the log collection and processing module; in step A3, the first terminal log is obtained. Simultaneously with the second log in the next iteration, the terminal log collection module collects logs from the target communication type terminal under test, corresponding to the second log's start and end timestamp range, forming the second... The second terminal log in the next iteration is sent to the log vectorization processing module via the log collection and processing module; in step A4, the log vectorization processing module calculates the... In the next iteration, the similarity between the first terminal log and the second terminal log is calculated; in step A6, the log vectorization processing module calculates the similarity between the two terminal logs. The average terminal similarity and standard deviation of all terminal log similarities in each iteration are calculated, and the standard deviation of terminal similarity fluctuates around the average terminal similarity as the center. This constitutes the normal terminal log similarity range corresponding to the target communication standard terminal under test, and is sent to the target communication standard air interface security automated test orchestration module.

[0039] During the execution of steps B1 to B4 in step B, in step B2, while obtaining the first log to be analyzed, the terminal log collection module collects logs from the target communication system terminal under test that have the same start timestamp to end timestamp range as the first log to be analyzed, forming the first terminal log to be analyzed, which is then sent to the log vectorization processing module via the log collection and processing module; in step B3, while obtaining the second log to be analyzed, the terminal log collection module collects logs from the target communication system terminal under test that have the same start timestamp to end timestamp range as the second log to be analyzed, forming the second terminal log to be analyzed, which is then sent to the log vectorization processing module via the log collection and processing module; in step B4, the log vectorization processing module calculates the similarity between the first terminal log to be analyzed and the second terminal log to be analyzed, forming the test terminal log similarity, and sends it to the target communication system air interface security automated test orchestration module;

[0040] Step C further includes the target communication standard air interface security automated test orchestration module determining whether the similarity of the test terminal logs corresponding to the target communication standard terminal under test is within the range of similarity of the corresponding normal terminal logs. If yes, it means that the preset test cases have no impact on the target communication standard terminal under test; otherwise, it means that the preset test cases have an impact on the target communication standard terminal under test.

[0041] As a preferred technical solution of the present invention: while executing steps A to C, the target communication standard air interface security automated test orchestration module controls the normal simulation of the low-generation communication standard base station to remain open; from the time the target communication standard terminal under test starts to access the target communication standard air interface security test simulation base station module or the normal simulation target communication standard base station, if the target communication standard air interface security automated test orchestration module exceeds the preset communication time and still does not find the end timestamp of the successful access log of the target communication standard terminal under test on the target communication standard air interface security test simulation base station module or the normal simulation target communication standard base station, it indicates that the preset test case has an impact on the target communication standard terminal under test, and the following steps are further analyzed;

[0042] Step a. The target communication standard air interface security automated test orchestration module determines whether the communication function of the target communication standard terminal under test is enabled. If yes, proceed to step b; otherwise, it means that the preset test case has triggered the failure of the communication function of the target communication standard terminal under test.

[0043] Step b. If the target communication standard air interface security automated test orchestration module finds the end timestamp of the successful access log of the target communication standard terminal under test on a normal simulated low-generation communication standard base station, it means that the preset test case triggers the communication connection degradation of the target communication standard terminal under test.

[0044] The 5G terminal air interface security automated testing system and method described in this invention, compared with the prior art, has the following technical advantages:

[0045] (1) This invention designs an automated testing system and method for 5G terminal air interface security. Each target communication standard simulation base station selects one to provide a network access environment to the target communication standard terminal under test. For the process of the terminal accessing the normal simulation base station, the log vectorization processing module performs similarity analysis and statistical analysis on the logs from the normal simulation base station and the target communication standard terminal under test, and obtains the normal log similarity range and normal delay range corresponding to the terminal. Then, for the test log similarity and test delay obtained by the test simulation base station under the preset test cases, it performs a comparison with the normal log similarity range and normal delay range, so as to realize the security test of the air interface protocol stack of the target communication standard terminal under test. The design scheme of this invention not only improves the testing efficiency and reduces the labor cost, but also significantly enhances the accuracy of security analysis and identification of a large number of logs from multiple devices.

[0046] (2) In the 5G terminal air interface security automated test system and test method designed in this invention, it can observe network access behavior under specific events in more detail, use the model to perform similarity threshold analysis on unstructured data, discover potential abnormal patterns or vulnerability identification, and use the log vectorization processing module to perform intelligent security analysis, which improves the detection accuracy. Moreover, the dynamic threshold generated by the standard deviation algorithm makes the judgment process more objective and data-supported, and the interactive learning benchmark of the target communication standard terminal on normal base station access improves the accuracy of the results.

[0047] (3) The 5G terminal air interface security automated testing system and testing method designed in this invention realizes the automation of terminal air interface security testing, reduces manual operation and manual log analysis, and improves testing efficiency; moreover, it can flexibly simulate different network switching scenarios and use models to process constantly changing log information, realizing flexible system configuration and adapting to different types of terminals and testing scenarios. Attached Figure Description

[0048] Figure 1 This is a structural block diagram of an embodiment of the 5G terminal air interface security automated testing system designed in this invention;

[0049] Figure 2 This is a flowchart of the test method based on the 5G terminal air interface security automated test system designed in this invention. Detailed Implementation

[0050] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0051] This invention designs an automated testing system for 5G terminal air interface security. In practical applications, such as... Figure 1 As shown, the design includes a target communication standard air interface security automated test orchestration module, a target communication standard air interface security test simulation base station module, a normal simulation target communication standard base station, a log collection and processing module, a log vectorization processing module (SentenceTransformer), a terminal test controller, a normal simulation low-generation communication standard base station, a terminal log collection module, and a simulated communication core network. In the actual implementation, the target communication standard uses 5G, and the low-generation communication standard uses 4G. That is, the system design includes a 5G air interface security automated test orchestration module, a 5G air interface security test simulation base station module, a normal simulation 5G base station, a log collection and processing module, a log vectorization processing module, a terminal test controller, a normal simulation 4G base station, and a simulated communication core network.

[0052] like Figure 1 As shown, in the system, the normal simulated target communication standard base station is used to provide the target communication standard terminal under test with a normal network access environment under the target communication standard; the target communication standard air interface security test simulation base station module is used to provide the target communication standard terminal under test with a normal network access environment and a test network access environment under the target communication standard; the normal simulated low-generation communication standard base station is used to provide the target communication standard terminal under test with a normal network access environment under the low-generation communication standard, and the low-generation communication standard is lower than the target communication standard.

[0053] In this embodiment, the normal simulated 5G base station and the normal simulated 4G base station provide real and standard-compliant 4G and 5G network access environments, respectively, for the system to learn normal behavior patterns and serve as backup access points for terminal testing. The hardware configuration design of the 5G air interface security test simulation base station module is slightly different from that of the normal simulated 5G base station. The 5G air interface security test simulation base station module is pre-configured with the ability to trigger specific security test events or inject interference interfaces, or the 5G air interface security test simulation base station module is a virtual base station entity that is specially controlled to simulate attack characteristics. Its main function is to perform critical security verification as an "abnormal" access point.

[0054] The simulated communication core network is used to simulate the communication access control logic under the target communication standard and the communication access control logic under the low-generation communication standard, and provides it to the normal simulated target communication standard base station and the normal simulated low-generation communication standard base station, respectively. In the implementation design, the simulated communication core network is the simulated 4 / 5G core network, which is used to simulate the behavior logic of the real operator core network (such as authentication, subscription processing, mobility management, etc.) so as to generate standardized response logs or interact with the real network for verification without relying on the actual base station. The simulated 4 / 5G core network receives logs and status information from the terminal test controller, the normal simulated 5G base station and the normal simulated 4G base station, and the 5G terminal under test through the interface.

[0055] The target communication standard air interface security automated test orchestration module is used to control the activation or deactivation of the communication function of the target communication standard terminal under test. It controls the activation or deactivation of the target communication standard air interface security test simulation base station module, the normal simulated target communication standard base station, and the normal simulated low-generation communication standard base station. It also sends preset test cases to the target communication standard air interface security test simulation base station module. In application, the activation or deactivation of the communication function of the target communication standard terminal under test includes specific actions such as switching the terminal's communication mode and deactivating or activating flight mode. The preset test cases include a series of standardized test cases and attack simulation scripts pre-stored in the target communication standard air interface security automated test orchestration module.

[0056] In the application, the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module. The target communication standard air interface security test simulation base station module then sends specific 5G air interface signaling to the target communication standard terminal under test to perform specific target communication standard security tests (such as authentication process, encryption integrity check, denial-of-service attack protection, etc.). Each test case has a clear ID.

[0057] In the further design of the system, the target communication standard air interface security automated test orchestration module interacts with the target communication standard terminal under test via the terminal test controller or AT command interface to control the target communication standard terminal under test to enable or disable its communication function. The terminal test controller includes a control unit, an instruction sending unit connected to the control unit, and a robotic arm. The control unit receives control instructions from the target communication standard air interface security automated test orchestration module and generates corresponding hardware operation instructions. The hardware operation instructions control the target communication standard terminal under test to enable or disable its communication function via wired or wireless communication methods (such as Wi-Fi, Bluetooth, infrared, etc.) in the instruction sending unit, or via physical contact between the robotic arm and the target communication standard terminal under test. By controlling the use of the robotic arm, the state of the target communication standard terminal under test can be forcibly changed (such as accessing a base station) or abnormal situations in specific scenarios can be simulated.

[0058] In application, if the target communication standard air interface security automated test orchestration module connects the tested target communication standard terminal to a normal simulated target communication standard base station, the target communication standard air interface security test simulation base station module, or a normal simulated low-generation communication standard base station, the target communication standard air interface security automated test orchestration module obtains the timestamp of the process of the tested target communication standard terminal connecting to the corresponding base station and sends it to the log collection and processing module.

[0059] The terminal log collection module is used to collect logs on the target communication system terminal with the corresponding timestamp based on the timestamp of the process of the target communication system terminal accessing the corresponding base station, and upload them to the log vectorization processing module; the log collection and processing module is used to collect logs on the base station accessed by the target communication system terminal with the corresponding timestamp based on the timestamp of the process of the target communication system terminal accessing the corresponding base station, and upload them to the log vectorization processing module.

[0060] In practical applications, the log collection and processing module mainly has the following functions: receiving and storing structured or semi-structured logs; performing basic log filtering and format conversion for processing by the statement converter model; and providing timestamps (e.g., using the NTP protocol to keep all components synchronized) and sending the timestamped logs to the log vectorization processing module.

[0061] The log vectorization processing module is used to analyze the received logs, complete the test of the process of the target communication system terminal accessing the base station, and send the test results to the target communication system air interface security automated test orchestration module. In practical applications, the main functions of the log vectorization processing module include: (1) analyzing the logs from the log collection and processing module, extracting key information (such as the re-establishment process of network access status before and after flight mode is turned on / off, and specific log message content), and calculating its vector representation (Embedding). Among them, special attention is paid to: the time delay of the target communication system terminal successfully re-accessing the corresponding base station after network switching; the log content changes of key interaction events (such as RRC connection establishment messages, registration process, etc.) between the target communication system terminal and the corresponding base station before and after the opening and closing of the communication function of the target communication system terminal; (2) for the flight mode operation sequence, calculating the similarity score of the two log segments before and after its execution. The similarity score can be obtained by embedding the vector distance of the model or by directly comparing specific keywords / sentences.

[0062] Based on the aforementioned designed automated testing system for 5G terminal air interface security, this invention further designs corresponding testing methods, such as... Figure 2 As shown, the following steps A to C are performed to test the communication standard terminal of the target device.

[0063] Step A. The target communication standard air interface security automated test orchestration module controls the normal simulation target communication standard base station to be turned on and the target communication standard air interface security test simulation base station module to be turned off. That is, in this embodiment, only the normal simulation 5G base station is controlled to be set to 5G SA mode. Based on the log of the process of the target communication standard terminal under test accessing the normal simulation target communication standard base station multiple times, the log vectorization processing module analyzes the log to obtain the normal log similarity range corresponding to the target communication standard terminal under test, as well as the normal latency range and normal terminal log similarity range corresponding to the target communication standard terminal under test. The normal log similarity range, normal latency range, and normal terminal log similarity range are sent to the target communication standard air interface security automated test orchestration module, and then proceed to step B.

[0064] In practical applications, step A above is specifically designed to be executed as follows: steps A1 to A6.

[0065] Step A1. The target communication standard air interface security automated test orchestration module controls the normal simulation target communication standard base station to be turned on and the target communication standard air interface security test simulation base station module to be turned off. That is, in this embodiment, only the normal simulation 5G base station is controlled to be set to 5G SA mode, and the iteration number is initialized. Proceed to step A2.

[0066] Step A2. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be enabled. That is, in this embodiment, the 5G terminal under test is set to 5G SA or 5G Only mode. The module obtains the start timestamp of the target communication standard terminal under test accessing the normal simulated target communication standard base station and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The log collection and processing module obtains the logs on the normal simulated target communication standard base station corresponding to the range from the start timestamp to the end timestamp, which constitute the first... The first log in the next iteration, and the duration from the start timestamp to the end timestamp, constitute the first... The first delay in the next iteration will further reduce the delay of the next iteration. In the next iteration, the first log and the first delay are sent together to the log vectorization processing module. Simultaneously, the terminal log collection module collects logs from the target communication terminal that correspond to the same start and end timestamp range as the first log, forming the second... The first terminal log in the next iteration is sent to the log vectorization processing module via the log collection and processing module, and then proceeds to step A3.

[0067] Step A3. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned off, and after a preset interval, controls the communication function of the target communication standard terminal under test to be turned on again. It obtains the start timestamp of the target communication standard terminal's access to the normal simulated target communication standard base station and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The log collection and processing module obtains the logs from the normal simulated target communication standard base station corresponding to the start timestamp to the end timestamp range, constituting the first... The second log in the next iteration, and the duration from the start timestamp to the end timestamp, constitute the second... The second delay in the next iteration will further reduce the second delay. In the next iteration, the second log and the second delay are sent together to the log vectorization processing module. Simultaneously, the terminal log collection module collects logs from the target communication terminal that correspond to the same start and end timestamp range as the second log, forming the second... The second terminal log in the next iteration is sent to the log vectorization processing module via the log collection and processing module, and then proceeds to step A4.

[0068] Step A4. The log vectorization processing module calculates and obtains the first... The similarity between the first and second logs in the iteration, and the log vectorization module calculates the similarity of the first log in the iteration. The similarity between the first terminal log and the second terminal log is calculated in the next iteration, and then the process proceeds to step A5.

[0069] Step A5. Judgment Is it equal to the preset maximum number of iterations? If yes, proceed to step A6; otherwise, proceed to step A6. The value is incremented by 1 and the process returns to step A2.

[0070] Step A6. The log vectorization processing module calculates and obtains... The average similarity and standard deviation of all similarities in each iteration, with the average similarity as the center, fluctuate around the standard deviation of similarity to form the normal log similarity range corresponding to the target communication system terminal being tested.

[0071] Log vectorization processing module calculates and obtains The average delay and standard deviation of all delays in each iteration are used to form the normal delay range corresponding to the communication standard terminal of the target device. The average delay is used as the center and the standard deviation of delay fluctuates above and below the average delay.

[0072] Log vectorization processing module calculates and obtains The average terminal similarity and standard deviation of all terminal log similarities in each iteration, and the fluctuation of the standard deviation of terminal similarity around the average terminal similarity, constitute the normal terminal log similarity range corresponding to the target communication system terminal being tested.

[0073] Then, the normal log similarity range, normal latency range, and normal terminal log similarity range corresponding to the target communication standard terminal under test are sent to the target communication standard air interface security automated test orchestration module, and then proceed to step B.

[0074] Step B. The target communication standard air interface security automated test orchestration module controls the target communication standard air interface security test simulation base station module to be turned on and the normal simulation target communication standard base station to be turned off. That is, in this embodiment, only the 5G air interface security test simulation base station module is controlled to be set to 5G SA mode. Based on the state before and after the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module, the logs of the target communication standard terminal under test accessing the target communication standard air interface security test simulation base station module are analyzed by the log vectorization processing module to obtain the test log similarity corresponding to the target communication standard terminal under test, and the test latency and test terminal log similarity corresponding to the target communication standard terminal under test. The test log similarity, test latency, and test terminal log similarity are sent to the target communication standard air interface security automated test orchestration module, and then proceed to step C.

[0075] In practical applications, step B above is specifically designed to be executed as follows: steps B1 to B4.

[0076] Step B1. The target communication standard air interface security automated test orchestration module controls the target communication standard air interface security test simulation base station module to turn on and the normal simulation target communication standard base station to turn off. That is, in this embodiment, only the 5G air interface security test simulation base station module is controlled to be set to 5G SA mode, and then proceed to step B2.

[0077] Step B2. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be enabled. In this embodiment, the target 5G terminal is set to 5G SA or 5G Only mode. The module obtains the start timestamp of the target communication standard terminal accessing the target communication standard air interface security test simulation base station module and the end timestamp of the first successful terminal access log. The module sends these timestamps to the log collection and processing module. The log collection and processing module obtains the logs on the target communication standard air interface security test simulation base station module within the range of the start timestamp to the end timestamp, forming the first log to be analyzed. The logs are then sent to the log vectorization processing module. At the same time, the terminal log collection module collects the logs on the target communication standard terminal within the same range of the start timestamp to the end timestamp as the first log to be analyzed, forming the first terminal log to be analyzed. The logs are then sent to the log vectorization processing module via the log collection and processing module, and then the process proceeds to step B3.

[0078] Step B3. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned off. Then, the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module. Then, the target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned on again. It obtains the start timestamp of the process of the target communication standard terminal under test accessing the target communication standard air interface security test simulation base station module, and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The processing module obtains the logs from the target communication standard air interface security test simulation base station module within the range of the start timestamp to the end timestamp, forming the second log to be analyzed, and obtains the duration from the start timestamp to the end timestamp, forming the test delay corresponding to the target communication standard terminal under test. The second log to be analyzed and the test delay are then sent to the log vectorization processing module. At the same time, the terminal log collection module collects the logs from the target communication standard terminal under test within the same range of the start timestamp to the end timestamp corresponding to the second log to be analyzed, forming the second terminal log to be analyzed. The log collection and processing module sends the logs to the log vectorization processing module, and then proceeds to step B4.

[0079] Step B4. The log vectorization processing module calculates the similarity between the first log to be analyzed and the second log to be analyzed, forming the test log similarity, and sends it to the target communication standard air interface security automated test orchestration module. At the same time, the log vectorization processing module calculates the similarity between the first terminal log to be analyzed and the second terminal log to be analyzed, forming the test terminal log similarity, and sends it to the target communication standard air interface security automated test orchestration module, and then proceeds to step C.

[0080] In practical applications, the acquisition of the start timestamp and the end timestamp of the first successful terminal access log during the process of the target communication system terminal accessing the target communication system air interface security test simulation base station module involved in steps A to B above should be specifically considered in the following Table 1.

[0081] Table 1

[0082]

[0083] As shown in Table 1 above, turning flight mode on and off corresponds to turning the communication function of the 5G terminal under test on and off. In actual application, based on the activation of the terminal's communication function, the log collection and processing module starts collecting logs from the corresponding base station until the terminal's communication function is turned off, obtaining the initial logs from the corresponding base station. Then, based on the timestamp of the terminal attempting to connect to the base station and the timestamp of the base station confirming the successful connection, i.e., the specific start and end timestamps involved in steps A to B, the initial logs are further obtained from the logs within the range of the start and end timestamps, forming the logs required for steps A and B, and then further processed.

[0084] Step C. The target communication standard air interface security automated test orchestration module performs the following three judgments:

[0085] Determine whether the similarity of the test logs corresponding to the target communication standard terminal under test is within the corresponding normal log similarity range;

[0086] Determine whether the test latency corresponding to the communication standard terminal of the target device is within the corresponding normal latency range;

[0087] Determine whether the similarity of the test terminal logs corresponding to the target communication standard terminal is within the range of similarity of the corresponding normal terminal logs;

[0088] If all three conditions are met, it means that the preset test cases have no effect on the target communication standard terminal under test; otherwise, it means that the preset test cases have an effect on the target communication standard terminal under test.

[0089] In practical applications, while executing steps A to C above, the target communication standard air interface security automated test orchestration module controls the normal simulation of the low-generation communication standard base station to remain open. From the moment the target communication standard terminal under test starts accessing the target communication standard air interface security test simulation base station module or the normal simulation target communication standard base station, if the target communication standard air interface security automated test orchestration module exceeds the preset communication time and still does not find the end timestamp of the successful access log of the target communication standard terminal under test on the target communication standard air interface security test simulation base station module or the normal simulation target communication standard base station, it indicates that the preset test case has an impact on the target communication standard terminal under test.

[0090] Regarding the impact of the preset test cases on the target communication standard terminal reflected by the above timeout, in practical applications, further analysis is carried out according to the following steps a to b.

[0091] Step a. The target communication standard air interface security automated test orchestration module determines whether the communication function of the target communication standard terminal under test is enabled. If yes, proceed to step b; otherwise, it means that the preset test case has triggered the failure of the communication function of the target communication standard terminal under test.

[0092] Step b. If the target communication standard air interface security automated test orchestration module finds the end timestamp of the successful access log of the target communication standard terminal under test on a normal simulated low-generation communication standard base station, it means that the preset test case triggers the communication connection degradation of the target communication standard terminal under test.

[0093] In application, the test method only tests a single target communication type terminal at a time. That is, during the access process of the target communication type terminal to the corresponding simulated base station, the base station considers accessing a single target communication type terminal.

[0094] The 5G terminal air interface security automated testing system and method designed in this invention can observe network access behavior under specific events in greater detail, use models to perform similarity threshold analysis on unstructured data to discover potential abnormal patterns or vulnerability identification, and use log vectorization processing module to perform intelligent security analysis, which improves detection accuracy. Moreover, the dynamic threshold generated by the standard deviation algorithm makes the judgment process more objective and data-supported, and the interactive learning benchmark based on the target communication standard terminal regarding normal base station access improves the accuracy of the results.

[0095] In practical applications, the solution designed in this invention automates terminal air interface security detection, reduces manual operation and manual log analysis, and improves detection efficiency. Moreover, it can flexibly simulate different network switching scenarios and use models to process constantly changing log information, enabling flexible system configuration that can adapt to different types of terminals and test scenarios.

[0096] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A 5G terminal air interface security automated testing system, used for testing target communication standard terminals under test, characterized in that: It includes a target communication standard air interface security automated test orchestration module, a target communication standard air interface security test simulation base station module, a normal simulation target communication standard base station, a log collection and processing module, and a log vectorization processing module; Among them, the normal simulation target communication standard base station is used to provide the target communication standard terminal under test with a normal network access environment under the target communication standard; the target communication standard air interface security test simulation base station module is used to provide the target communication standard terminal under test with a normal network access environment and a test network access environment under the target communication standard. The target communication standard air interface security automated test orchestration module is used to control the target communication standard terminal under test to enable or disable the communication function. It controls the target communication standard air interface security test simulation base station module and the normal simulation target communication standard base station to enable or disable respectively. It is also used to send preset test cases to the target communication standard air interface security test simulation base station module. If the target communication standard terminal under test connects to the normal simulation target communication standard base station or the target communication standard air interface security test simulation base station module, the target communication standard air interface security automated test orchestration module obtains the timestamp of the target communication standard terminal connecting to the corresponding base station and sends it to the log collection and processing module. The log collection and processing module is used to collect logs on the base station connected to the target communication system terminal based on the timestamp of the process of the target communication system terminal accessing the corresponding base station, and upload them to the log vectorization processing module. The log vectorization processing module is used to analyze the received logs, complete the test of the process of the target communication system terminal accessing the base station, and send the test results to the target communication system air interface security automated test orchestration module.

2. The 5G terminal air interface security automated testing system according to claim 1, characterized in that: It also includes a terminal test controller, wherein the target communication standard air interface security automated test orchestration module interacts with the target communication standard terminal under test via the terminal test controller or AT command interface to control the target communication standard terminal under test to enable or disable the communication function. The terminal test controller includes a control unit, an instruction sending unit connected to the control unit, and a robotic arm. The control unit receives control instructions from the target communication standard air interface security automated test orchestration module and generates corresponding hardware operation instructions. The hardware operation instructions control the target communication standard terminal under test to turn on or off its communication function via wired or wireless communication in the instruction sending unit, or the hardware operation instructions control the target communication standard terminal under test to turn on or off its communication function via physical contact between the robotic arm and the target communication standard terminal under test.

3. The 5G terminal air interface security automated testing system according to claim 1, characterized in that: It also includes a normal simulated low-generation communication standard base station, which is used to provide the target communication standard terminal under test with a normal network access environment under the low-generation communication standard. The low-generation communication standard is lower than the target communication standard. The target communication standard air interface security automated test orchestration module controls the normal simulation low-generation communication standard base station to be turned on or off. If the target communication standard terminal under test connects to the normal simulation low-generation communication standard base station, the target communication standard air interface security automated test orchestration module obtains the timestamp of the process of the target communication standard terminal connecting to the normal simulation low-generation communication standard base station and sends it to the log collection and processing module.

4. The 5G terminal air interface security automated testing system according to claim 3, characterized in that: It also includes a terminal log collection module and a simulated communication core network. The terminal log collection module is used to collect logs on the target communication system terminal with the corresponding timestamp based on the timestamp of the process of the target communication system terminal accessing the corresponding base station, and upload them to the log vectorization processing module. The simulated communication core network is used to simulate the communication access control logic under the target communication system and the communication access control logic under the low-generation communication system, and provides it to the normal simulated target communication system base station and the normal simulated low-generation communication system base station, respectively.

5. A test method based on the automated test system for 5G terminal air interface security as described in claim 3, characterized in that: Includes the following steps: Step A. The target communication standard air interface security automated test orchestration module controls the normal simulation target communication standard base station to be turned on and the target communication standard air interface security test simulation base station module to be turned off. Based on the logs of the process of controlling the target communication standard terminal under test to access the normal simulation target communication standard base station multiple times, the log vectorization processing module analyzes the logs to obtain the normal log similarity range corresponding to the target communication standard terminal under test, and sends it to the target communication standard air interface security automated test orchestration module, and then proceeds to Step B. Step B. The target communication standard air interface security automated test orchestration module controls the target communication standard air interface security test simulation base station module to turn on and the normal simulation target communication standard base station to turn off. Based on the state before and after the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module, the logs of the process of the target communication standard terminal accessing the target communication standard air interface security test simulation base station module are analyzed by the log vectorization processing module to obtain the test log similarity corresponding to the target communication standard terminal under test, and then sent to the target communication standard air interface security automated test orchestration module, and then proceed to step C. Step C. The target communication standard air interface security automated test orchestration module determines whether the similarity of the test logs corresponding to the target communication standard terminal under test is within the range of the corresponding normal log similarity. If yes, it means that the preset test cases have no impact on the target communication standard terminal under test; otherwise, it means that the preset test cases have an impact on the target communication standard terminal under test.

6. The test method of the 5G terminal air interface security automated test system according to claim 5, characterized in that: Step A includes the following steps A1 to A6; Step A1. The target communication standard air interface security automated test orchestration module controls the normal simulation target communication standard base station to turn on, the target communication standard air interface security test simulation base station module to turn off, and initializes the iteration count. Proceed to step A2; Step A2. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be enabled, obtains the start timestamp of the target communication standard terminal under test accessing the normal simulated target communication standard base station, and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The log collection and processing module obtains the logs on the normal simulated target communication standard base station corresponding to the start timestamp to the end timestamp range, forming the first... The first log of the next iteration is then sent to the log vectorization processing module, and then proceeds to step A3; Step A3. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned off, and after a preset interval, controls the communication function of the target communication standard terminal under test to be turned on again. It obtains the start timestamp of the target communication standard terminal's access to the normal simulated target communication standard base station and the end timestamp of the first successful terminal access log, and sends them to the log collection and processing module. The log collection and processing module obtains the logs from the normal simulated target communication standard base station corresponding to the start timestamp to the end timestamp range, constituting the first... The second log of the next iteration is then sent to the log vectorization processing module, and then proceeds to step A4; Step A4. The log vectorization processing module calculates and obtains the first... The similarity between the first log and the second log is calculated in the next iteration, and then proceeds to step A5; Step A5. Judgment Is it equal to the preset maximum number of iterations? If yes, proceed to step A6; otherwise, proceed to step A6. The value is incremented by 1 and the process returns to step A2; Step A6. The log vectorization processing module calculates and obtains... The average similarity and standard deviation of all similarities in the next iteration are calculated, and the fluctuation of the standard deviation of similarity around the average similarity is calculated to form the normal log similarity range corresponding to the target communication standard terminal under test. This range is then sent to the target communication standard air interface security automated test orchestration module, and then proceeds to step B.

7. The test method of the 5G terminal air interface security automated test system according to claim 6, characterized in that: Step B includes the following steps B1 to B4; Step B1. The target communication standard air interface security automated test orchestration module controls the target communication standard air interface security test simulation base station module to turn on and the normal simulation target communication standard base station to turn off, and then proceeds to step B2; Step B2. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be enabled, obtains the start timestamp of the target communication standard terminal under test accessing the target communication standard air interface security test simulation base station module, and the end timestamp of the first successful terminal access log, and sends it to the log collection and processing module. The log collection and processing module obtains the logs on the target communication standard air interface security test simulation base station module within the range of the corresponding start timestamp to end timestamp, which constitutes the first log to be analyzed, and then sends it to the log vectorization processing module, and then proceeds to step B3. Step B3. The target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned off. Then, the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module. Then, the target communication standard air interface security automated test orchestration module controls the communication function of the target communication standard terminal under test to be turned on again. The module obtains the start timestamp of the target communication standard terminal under test accessing the target communication standard air interface security test simulation base station module and the end timestamp of the first successful terminal access log. The module sends these timestamps to the log collection and processing module. The log collection and processing module obtains the logs on the target communication standard air interface security test simulation base station module within the range of the start timestamp to the end timestamp, which constitutes the second log to be analyzed. The logs are then sent to the log vectorization processing module, and then the process proceeds to step B4. Step B4. The log vectorization processing module calculates the similarity between the first log to be analyzed and the second log to be analyzed, forms the test log similarity, and sends it to the target communication standard air interface security automated test orchestration module, and then proceeds to step C.

8. The test method of the 5G terminal air interface security automated test system according to claim 7, characterized in that: Step A further includes obtaining the normal latency range corresponding to the target communication standard terminal under test and sending it to the target communication standard air interface security automated test orchestration module; wherein, step A2 further includes obtaining the duration from the start timestamp to the end timestamp during the process of the target communication standard terminal under test accessing the normal simulated target communication standard base station, constituting the first The first delay in the next iteration is sent to the log vectorization processing module; step A3 also includes obtaining the duration from the start timestamp to the end timestamp during the process of the tested target communication system terminal accessing the normal simulated target communication system base station, constituting the first... The second delay in the next iteration is sent to the log vectorization processing module; step A6 also includes the log vectorization processing module calculating and obtaining... The average delay and standard deviation of all delays in each iteration are calculated, and the standard deviation of delay fluctuates around the average delay as the center to form the normal delay range corresponding to the target communication standard terminal under test, and then sent to the target communication standard air interface security automated test orchestration module. Step B further includes obtaining the test latency corresponding to the target communication standard terminal under test and sending it to the target communication standard air interface security automated test orchestration module; wherein, step B3 further includes obtaining the duration from the start timestamp to the end timestamp during the process of the target communication standard terminal under test accessing the target communication standard terminal after the target communication standard air interface security automated test orchestration module sends preset test cases to the target communication standard air interface security test simulation base station module, which constitutes the test latency corresponding to the target communication standard terminal under test, and sending it to the target communication standard air interface security automated test orchestration module; Step C further includes the target communication standard air interface security automated test orchestration module determining whether the test delay corresponding to the target communication standard terminal under test is within the corresponding normal delay range. If it is, it means that the preset test cases have no effect on the target communication standard terminal under test; otherwise, it means that the preset test cases have an effect on the target communication standard terminal under test.

9. The test method of the 5G terminal air interface security automated test system according to claim 8, characterized in that: During the execution of steps A1 to A6 in step A2, the first step is obtained. Simultaneously with the first log in the next iteration, the terminal log collection module collects logs from the target communication type terminal under test, corresponding to the same start timestamp to end timestamp range as the first log, forming the second... The first terminal log in the next iteration is sent to the log vectorization processing module via the log collection and processing module. In step A3, the first... Simultaneously with the second log in the next iteration, the terminal log collection module collects logs from the target communication type terminal under test, corresponding to the second log's start and end timestamp range, forming the second... The second terminal log in the next iteration is sent to the log vectorization processing module via the log collection and processing module. In step A4, the log vectorization processing module calculates and obtains the first... Terminal log similarity between the first terminal log and the second terminal log in the next iteration; In step A6, the log vectorization processing module calculates and obtains... The average terminal similarity and standard deviation of all terminal log similarities in each iteration are calculated, and the standard deviation of terminal similarity fluctuates around the average terminal similarity as the center. This constitutes the normal terminal log similarity range corresponding to the target communication standard terminal under test, and is sent to the target communication standard air interface security automated test orchestration module. During the execution of steps B1 to B4 in step B, in step B2, while obtaining the first log to be analyzed, the terminal log collection module collects logs from the target communication terminal under test that have the same start timestamp to end timestamp range as the first log to be analyzed, forming the first terminal log to be analyzed, which is then sent to the log vectorization processing module via the log collection and processing module. In step B3, while obtaining the second log to be analyzed, the terminal log collection module collects logs from the target communication terminal under test that have the same start timestamp to end timestamp range as the second log to be analyzed, forming the second terminal log to be analyzed, which is then sent to the log vectorization processing module via the log collection and processing module. In step B4, the log vectorization processing module calculates the similarity between the logs of the first terminal to be analyzed and the logs of the second terminal to be analyzed, forms the test terminal log similarity, and sends it to the target communication standard air interface security automated test orchestration module. Step C further includes the target communication standard air interface security automated test orchestration module determining whether the similarity of the test terminal logs corresponding to the target communication standard terminal under test is within the range of similarity of the corresponding normal terminal logs. If yes, it means that the preset test cases have no impact on the target communication standard terminal under test; otherwise, it means that the preset test cases have an impact on the target communication standard terminal under test.

10. The test method of the 5G terminal air interface security automated test system according to claim 7, characterized in that: While executing steps A to C, the target communication standard air interface security automated test orchestration module controls the normal simulation of the low-generation communication standard base station to remain open. From the moment the target communication standard terminal under test begins to access the target communication standard air interface security test simulation base station module or the normal simulation target communication standard base station, if the target communication standard air interface security automated test orchestration module exceeds the preset communication time and still fails to find the end timestamp of the successful access log of the target communication standard terminal under test on the target communication standard air interface security test simulation base station module or the normal simulation target communication standard base station, it indicates that the preset test cases have an impact on the target communication standard terminal under test, and the following steps will be further analyzed. Step a. The target communication standard air interface security automated test orchestration module determines whether the communication function of the target communication standard terminal under test is enabled. If yes, proceed to step b; otherwise, it means that the preset test case has triggered the failure of the communication function of the target communication standard terminal under test. Step b. If the target communication standard air interface security automated test orchestration module finds the end timestamp of the successful access log of the target communication standard terminal under test on a normal simulated low-generation communication standard base station, it means that the preset test case triggers the communication connection degradation of the target communication standard terminal under test.

Images