Automatic gateway testing method and system

Abstract

The invention provides an automatic gateway testing method and system, and belongs to the technical field of network communication, and the method comprises the steps: in response to receiving a target testing template selected by a user and not receiving user-defined information input by the user, creating a target testing task based on the target testing template; in response to the received target test template selected by the user and user-defined information input by the user, updating the target test template based on the user-defined information, and creating a target test task based on the updated target test template; determining a resource allocation strategy of the target test task, and determining a target test resource based on the resource allocation strategy; generating a control instruction based on the target test task, and based on the control instruction, executing an automatic test of the gateway through the target test resource and obtaining test data; and determining a gateway test result based on the test data and the standard index threshold. According to the invention, gateway multi-protocol multi-device compatibility testing can be realized, and the testing efficiency and accuracy are improved.

Description

Technical Field

[0001] This application belongs to the field of network communication technology, and more specifically, relates to a gateway automated testing method and system. Background Technology

[0002] Gateways are core devices connecting heterogeneous networks in the fields of IoT, Industrial Internet, and smart homes. They primarily undertake critical tasks such as data forwarding, cross-protocol conversion, and terminal device access management. The functional integrity, compatibility, performance stability, and security of a gateway directly determine the operational efficiency of the entire network system and must be tested to verify compliance with industry and national standards. With the rapid popularization of 5G and IoT technologies, gateway application scenarios are becoming increasingly complex, requiring support for multiple mainstream protocols and various types of terminal devices. Multi-protocol and multi-device compatibility testing is challenging. Traditional gateway testing is designed for a single protocol or device type, requiring the construction of multiple test environments and repeated parameter configurations, resulting in a cumbersome process that struggles to cover full-scenario compatibility requirements. Furthermore, traditional gateway testing suffers from low efficiency in data collection and analysis and is prone to analytical errors. Summary of the Invention

[0003] The purpose of this application is to provide an automated testing method and system for gateways, so as to realize multi-protocol and multi-device compatibility testing of gateways and improve testing efficiency and accuracy.

[0004] A first aspect of this application provides a gateway automated testing method, including:

[0005] In response to receiving the target test template selected by the user but not receiving any custom information input by the user, a target test task is created based on the target test template. The target test task includes a task ID, a target test scenario, target test steps, and target test metrics. The target test template is a template in the standardized test template library, which includes multiple preset test templates. Each preset test template includes a preset test scenario, test steps, and test metrics.

[0006] In response to receiving the target test template selected by the user and the custom information input by the user, the target test template is updated based on the custom information, and a target test task is created based on the updated target test template; the custom information includes at least one of the custom test scenario, custom test steps, and custom test metrics; the custom information is used to update at least one of the preset test scenario, test steps, and test metrics in the target test template;

[0007] Determine the resource allocation strategy for the target test task, and determine the target test resources based on the resource allocation strategy;

[0008] If the target test resource is idle, control instructions are generated based on the target test task. Based on the control instructions, the automated test of the gateway is executed through the target test resource, and test data during the automated test process of the gateway is obtained.

[0009] The gateway test results are determined based on test data and standard indicator thresholds.

[0010] A second aspect of this application provides a gateway automated testing system, comprising:

[0011] The first task creation module is used to create a target test task based on the target test template when the user selects a target test template but no user input is received. The target test task includes a task ID, a target test scenario, target test steps, and target test metrics. The target test template is a template in the standardized test template library, which includes multiple preset test templates. Each preset test template includes a preset test scenario, test steps, and test metrics.

[0012] The second task creation module is used to respond to receiving the target test template selected by the user and the user's input of custom information, update the target test template based on the custom information, and create a target test task based on the updated target test template; the custom information includes at least one of custom test scenarios, custom test steps, and custom test metrics; the custom information is used to update at least one of the preset test scenarios, test steps, and test metrics in the target test template;

[0013] The test resource allocation module is used to determine the resource allocation strategy for the target test task and to determine the target test resources based on the resource allocation strategy.

[0014] The testing module is used to generate control commands based on the target test task if the target test resource is idle. Based on the control commands, it executes the gateway's automated test through the target test resource and obtains the test data during the gateway's automated test process.

[0015] The test result analysis module is used to determine the gateway test results based on test data and standard indicator thresholds.

[0016] When determining the resource allocation strategy for a target test task, the test resource allocation module specifically determines the task priority and resource requirements of the target test task; determines the resource allocation strategy based on the task priority and resource requirements; and the resource allocation strategy is the allocation strategy for multiple test resources.

[0017] An automated gateway testing system also includes: a task waiting module, used for:

[0018] If the target test resource is in use, then retrieve the current task waiting queue;

[0019] If the current task waiting queue is empty, add the target test task to the current task waiting queue until the target test resource becomes idle and the target test task is at the head of the task waiting queue, then remove the target test task from the task waiting queue.

[0020] If the current task waiting queue is not empty, calculate the priority score of the target test task; determine the target insertion position based on the priority score of the target test task and the priority scores of each waiting task in the current task waiting queue, add the target test task to the target insertion position in the current task waiting queue, until the target test resource becomes idle and the target test task is at the first position in the task waiting queue, then delete the target test task from the task waiting queue.

[0021] Control commands include gateway control commands, device adaptation control commands, network scenario simulation commands, and data acquisition control commands; target test resources include multi-protocol adaptation interfaces, scenario simulation processors, data acquisition channels, and data storage nodes; the automated testing process of the gateway includes multiple states, including gateway initialization, device access, scenario preparation, test execution, data acquisition startup, abnormal scenario triggering, and test termination. Each state has corresponding execution conditions and state switching rules; for each state, the state is executed when the corresponding execution conditions are met, and the state is switched when the corresponding state switching rules are met; when the test module executes the automated testing of the gateway based on control commands and obtains test data during the automated testing process of the gateway, it is specifically used for:

[0022] The gateway control commands are sent to the gateway under test through a multi-protocol adapter interface; the gateway control commands are used to instruct the gateway to initialize; the gateway under test is used to perform power-on, IP address configuration, protocol parameter configuration, access permission configuration, restart and factory reset according to the gateway control commands, and to provide feedback on the execution results;

[0023] Device adaptation control commands are sent to physical devices and virtual device simulation nodes through a multi-protocol adaptation interface; the device adaptation control commands are used to instruct device access; the physical devices are used to execute terminal access, sensor acquisition frequency configuration, and controller command cycle configuration based on the device adaptation control commands, and to return the execution results; the virtual device simulation nodes are used to execute device simulation start / stop, sensor acquisition frequency configuration, and controller command cycle configuration based on the device adaptation control commands, and to return the execution results.

[0024] The network scenario simulation command is sent to the scenario simulation processor through a multi-protocol adaptation interface; the network scenario simulation command is used to indicate scenario preparation; the scenario simulation processor is used to execute network parameter settings, high-concurrency access startup, network switching triggering and interference start / stop based on the network scenario simulation command, and to provide feedback on the execution results.

[0025] The data acquisition control command is sent to the data acquisition channel; the data acquisition channel is used to execute channel selection configuration, sampling frequency configuration and acquisition start / stop based on the data acquisition control command, and to provide feedback on the execution results;

[0026] Once all feedback results indicate completion, the test execution state is entered; data acquisition control commands are used to instruct data acquisition to begin.

[0027] Initial test data is acquired through the data acquisition channel during the automated testing process of the gateway. The initial test data is then preprocessed to obtain preprocessed test data. Preprocessing includes noise reduction, standardization, and data compression. The data storage node is used to partition and store the preprocessed test data according to the data type.

[0028] In response to network switching or interference start / stop triggered by the scenario simulation processor, it enters the abnormal scenario trigger state; in response to receiving abnormal scenario completion information from the scenario simulation processor, it enters the test termination state.

[0029] An automated gateway testing system further includes: a task status monitoring module, used to obtain the task execution status, resource usage, and test progress of the target test task; and to generate visual charts based on the task execution status, resource usage, and test progress.

[0030] Test data includes the test metric data corresponding to each of the target test metrics; standard metric thresholds include the standard metric thresholds corresponding to each of the target test metrics; the test result analysis module, when determining the gateway test results based on the test data and standard metric thresholds, is specifically used for:

[0031] For each target test metric, the metric test score is calculated based on the test metric data corresponding to that target test metric and the standard metric threshold.

[0032] The overall gateway test score is obtained by weighted summing of the test scores of all target test metrics.

[0033] If the overall gateway test score is greater than the preset gateway test score threshold, the gateway test result is determined to be passed.

[0034] If the overall gateway test score is not greater than the gateway test score threshold, the gateway test result is determined to be a failure.

[0035] An automated gateway testing system further includes: a test report generation module, which is used to, after determining that the gateway test result is unsuccessful, regard the target test indicator whose indicator test score is not greater than a preset indicator test score threshold as an abnormal indicator, and the test indicator data corresponding to the abnormal indicator is abnormal data.

[0036] Multi-dimensional collected data is obtained based on timestamps and associated with abnormal indicators and abnormal data;

[0037] Based on multi-dimensional data collection, the fault analysis results are determined through a pre-trained machine learning model; the fault analysis results include the fault type and the fault cause.

[0038] Optimization suggestions are generated based on the fault analysis results;

[0039] A structured test report is generated based on the task information, fault analysis results, and optimization suggestions of the target test task.

[0040] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described gateway automated testing method.

[0041] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described gateway automated testing method.

[0042] The beneficial effects of the gateway automated testing method and system provided in this application are as follows:

[0043] This application's embodiments combine a standardized test template library with customizable information, allowing users to flexibly create or adjust test tasks. This avoids the cumbersome process of repeatedly building test environments for different protocols and devices, reducing test preparation time and costs, and improving efficiency. The automated test execution and data analysis in this application's embodiments ensure real-time data collection, standardized processing, and efficient analysis, improving data processing efficiency and accuracy, and providing a guarantee for the rapid iteration and high-quality release of gateway products. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1A flowchart illustrating an embodiment of the gateway automated testing method provided in this application;

[0046] Figure 2 A flowchart illustrating an embodiment of the gateway automated testing platform provided in this application;

[0047] Figure 3 A flowchart illustrating a multi-protocol device adaptation module provided in an embodiment of this application;

[0048] Figure 4 A flowchart illustrating a complex network scenario simulation module provided in an embodiment of this application;

[0049] Figure 5 A flowchart illustrating the test data acquisition and synchronization module provided in one embodiment of this application;

[0050] Figure 6 A flowchart illustrating the state machine model of an automated test execution module provided in an embodiment of this application;

[0051] Figure 7 A flowchart illustrating the test result analysis and fault location module provided in one embodiment of this application;

[0052] Figure 8 This is a structural block diagram of a gateway automated testing system provided in an embodiment of this application;

[0053] Figure 9 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0056] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a gateway automated testing method provided in an embodiment of this application. This method can be executed by a gateway automated testing platform, which achieves full-process automation, standardization, and scenario-based application of gateway testing through the collaborative work of multiple modules. Figure 2 As shown, Figure 2This is a flowchart illustrating a gateway automated testing platform provided in one embodiment of this application. The platform includes seven core modules: a multi-protocol device adaptation module, a complex network scenario simulation module, a test data acquisition and synchronization module, a test task management module, an automated test execution module, a test result analysis and fault location module, and a user interaction and log management module. These modules are connected via a high-speed network bus (such as Ethernet or PCIe) to achieve data interaction and control signal transmission. Figure 2 The interface indicates the connection points of the gateway and external test equipment (such as network analyzers, oscilloscopes, and power meters), and presents the interaction logic between the platform and these external devices. The workflow of the gateway automated testing platform includes...

[0057] An embodiment of this application provides a gateway automated testing method that includes: initializing seven core modules, including multi-protocol device adaptation, complex network scenario simulation, test data acquisition and synchronization, test task management, automated test execution, test result analysis and fault location, and user interaction and log management, laying the functional foundation for the testing process. After module initialization, it determines whether external test equipment needs to be connected, and executes along two paths: if yes, it connects to external test equipment such as network analyzers, oscilloscopes, and power meters via a high-speed network bus to expand the dimensions of test data acquisition; if no, it relies solely on the already initialized internal modules to work together, simplifying the test environment configuration. Regardless of whether external devices are connected, subsequent data and signal transmission between modules is achieved through a high-speed network bus (Ethernet, PCIe) to realize data interaction and control signal transmission between modules, ensuring the real-time performance and stability of module collaboration. Then, it executes the test cases assigned by the test task management module, advancing the gateway automated testing according to the preset test scenarios and steps. It determines whether the test results are abnormal. If yes, it triggers the test result analysis and fault location module to locate the abnormal problem; if no, it directly enters the report generation stage. Whether the fault is located or the anomaly is determined directly, the process of generating a test report and storing logs is ultimately executed to complete the archiving of the results of the entire testing process.

[0058] S101: In response to receiving the target test template selected by the user and not receiving any custom information input by the user, create a target test task based on the target test template; the target test task includes a task ID, a target test scenario, a target test step, and a target test metric; the target test template is a template in the standardized test template library, which includes multiple preset test templates, each of which includes a preset test scenario, test step, and test metric.

[0059] In this embodiment, the target test template refers to a standardized test plan preset in the standardized test template library, containing specific test scenario steps and indicators, used to quickly create test tasks. The standardized test template library is a collection storing multiple preset test templates, covering gateway testing requirements in multiple scenarios. The target test task is a specific test execution unit created based on the template, containing a unique task ID, the target test scenario to be simulated, the target test steps in the execution flow, and the target test indicators as the judgment criteria. The task ID is a unique identifier for the test task, the target test scenario is the network or device environment that the test needs to reproduce, the target test steps are the ordered flow of test execution, and the target test indicators are the quantitative standards for measuring gateway performance.

[0060] The core objective of this embodiment is to address the cumbersome process and inconsistent parameter configurations inherent in traditional gateway testing task creation. This embodiment integrates mature test scenarios, steps, and metrics through pre-defined standardized test templates, avoiding redundant configurations. When no custom information is received, this embodiment directly reuses the template, ensuring standardized testing processes, reducing human error, and improving task creation efficiency. Simultaneously, the standardized template library adapts to various gateway testing needs, ensuring the comparability of tests performed by different testers, laying the foundation for subsequent data comparison and fault analysis, and meeting the application requirements for large-scale and standardized gateway testing.

[0061] S102: In response to receiving the target test template selected by the user and the custom information input by the user, update the target test template based on the custom information, and create a target test task based on the updated target test template; the custom information includes at least one of custom test scenarios, custom test steps and custom test metrics; the custom information is used to update at least one of the preset test scenarios, test steps and test metrics in the target test template.

[0062] In this embodiment, custom information refers to personalized content input by the user based on specific testing needs, used to update the target test template. Custom test scenarios are special scenarios added by the user or network or device environments set that are not preset in the template. Custom test steps are the ordered test execution process adjusted by the user. Custom test metrics are the quantitative judgment standards for measuring gateway performance modified by the user.

[0063] For example, the test task management module in the gateway automated testing platform includes a task creation and configuration unit, a task scheduling unit, and a task monitoring unit. Steps S101 and S102 correspond to the task creation and configuration unit. The task creation and configuration unit provides a standardized test template library. For example, the standardized test template library may include test templates for IoT gateway multi-protocol compatibility, industrial gateway high-concurrency access performance, home smart gateway network switching stability, and 5G edge gateway security. Each test template has preset test indicators (such as forwarding rate ≥100Mbps, protocol conversion latency ≤100ms), test scenarios (network latency and packet loss, high-concurrency access and interference, etc.), and test steps (gateway power-on, parameter configuration, scenario simulation, and data acquisition, etc.). Users can create gateway test tasks based on preset test templates, or modify indicator thresholds, add special scenarios, and / or adjust the step order by inputting custom parameters. The task creation and configuration unit supports batch task creation, such as compatibility testing of 10 IoT gateways of the same model, and assigns a unique task ID and task identification information to each task. The task identification information may include name, tester, and time.

[0064] S103: Determine the resource allocation strategy for the target test task, and determine the target test resources based on the resource allocation strategy.

[0065] In this embodiment, determining the resource allocation strategy for the target test task specifically includes: determining the task priority and resource requirements of the target test task; determining the resource allocation strategy based on the task priority and resource requirements; and the resource allocation strategy being the allocation strategy for multiple test resources.

[0066] In this embodiment, after determining the target test resources based on the resource allocation strategy, the method further includes:

[0067] If the target test resource is in use, then retrieve the current task waiting queue;

[0068] If the current task waiting queue is empty, add the target test task to the current task waiting queue until the target test resource becomes idle and the target test task is at the head of the task waiting queue, then remove the target test task from the task waiting queue.

[0069] If the current task waiting queue is not empty, calculate the priority score of the target test task; determine the target insertion position based on the priority score of the target test task and the priority scores of each waiting task in the current task waiting queue, add the target test task to the target insertion position in the current task waiting queue, until the target test resource becomes idle and the target test task is at the first position in the task waiting queue, then delete the target test task from the task waiting queue.

[0070] In this embodiment, the resource allocation strategy refers to a scheme for allocating multiple test resources based on the task priority and resource requirements of the target test task. The resource allocation strategy clarifies the allocation quantity, usage period, and compatibility of various resources, ensuring that resources match task requirements. Task priority refers to the level of urgency of the target test task; for example, mass production gateway testing has a higher priority than R&D prototype testing. Task priority directly determines the order in which tasks acquire target test resources. Resource requirements refer to the type and quantity of test resources required to execute the target test task, such as the number of physical ports for multi-protocol adaptation interfaces, the computing power of the scenario simulation processor, and the number of data acquisition channels. The current task waiting queue refers to an ordered queue used to store test tasks to be executed when the target test resources are occupied, ensuring the standardization of waiting task management. Priority score refers to a quantitative value calculated based on factors such as task priority, test duration, and resource usage, used to accurately determine the order of waiting tasks in the queue. Target insertion position refers to the specific storage position of the target test task in the queue, determined based on the priority scores of the target test task and each waiting task in the current task waiting queue, ensuring the queue's orderliness.

[0071] The core objective of this embodiment is to resolve the conflict problem of multiple target testing tasks competing for limited target testing resources, avoiding chaotic and wasteful resource allocation. By determining task priorities, this embodiment ensures that urgent tasks acquire resources first, meeting business timeliness requirements and preventing delays caused by non-urgent tasks occupying resources. Clearly defining resource requirements allows for precise matching of resource types and quantities, preventing resource overload or idleness. When resources are occupied, a current task waiting queue is introduced, and the target insertion position is determined based on priority scores, preventing disorderly task accumulation and ensuring the queue is arranged in an orderly manner according to priority. The overall logic balances resource utilization efficiency and task execution rationality, adapting to the needs of large-scale, multi-task parallel scenarios in gateway automated testing.

[0072] For example, step S103 corresponds to the task scheduling unit in the test task management module. The task scheduling unit uses a scheduling algorithm that combines priority and resource requirements, automatically allocating test resources based on task urgency and resource consumption needs. When resources are insufficient, tasks enter a waiting queue, sorted according to a first-come, first-served basis and priority weighting, and automatically start after resources are released. It also supports scheduled startup and breakpoint resumption, improving execution efficiency. For example, if a stability test is started outside of working hours, it can resume from the interrupted point after equipment failure or network interruption.

[0073] This embodiment enables precise allocation of target test resources, avoiding resource idleness or overload, and improving the utilization rate of core resources such as multi-protocol adaptation interfaces and scenario simulation processors. Through quantitative control of task priority and priority score, this embodiment ensures that urgent tasks such as mass production and factory testing receive priority access to resources, meeting business timeliness requirements and avoiding test delays caused by non-urgent tasks occupying resources. The design of the current task waiting queue and target insertion position ensures that waiting tasks are arranged in an orderly manner according to priority, avoiding resource contention and chaos caused by disorderly accumulation. The overall process of this embodiment reduces task waiting time, lowers the probability of resource conflicts, improves the overall execution efficiency of gateway automated testing, adapts to large-scale testing scenarios with multiple tasks running in parallel, and ensures a standardized and orderly testing process.

[0074] For example, after the target test task is created, the task scheduling unit of the test task management module first determines the task priority and resource requirements. If the task is a mass production and factory test of a batch of industrial gateways, the task priority is determined to be the highest level, higher than the gateway compatibility test in the R&D stage. Assuming that the mass production and factory test task of this batch of industrial gateways needs to complete the Modbus and OPC UA protocol compatibility test and the high-concurrency access test of 500 devices, the task scheduling unit can determine the resource requirements as follows: 2 multi-protocol adaptation interfaces (including RS485 and Ethernet ports), 1 scenario simulation processor (supporting 1Gbps bandwidth and 0-500ms latency configuration), and 4 data acquisition channels, which are used to collect network performance, device access, system status, and abnormal log data, respectively.

[0075] The task scheduling unit can determine the resource allocation strategy based on task priority (highest priority) and resource requirements: prioritize the use of currently idle multi-protocol adaptation interfaces, scene simulation processors, and data acquisition channels. If the target test resources are insufficient, they are placed in a waiting queue and sorted by priority. The task scheduling unit can query the target test resource status through the resource status monitoring interface. Assuming the scene simulation processor is executing a low-priority test task of the R&D prototype (assuming a priority score of 60) and is in a occupied state, the current task waiting queue is obtained. If the current task waiting queue is empty, the mass production test task is directly added to the queue, and the scene simulation processor status is monitored in real time. If the queue is not empty, such as if there is one R&D test task in the queue with a priority score of 60, the priority score of the mass production test task is calculated. For example, based on task priority (highest priority, 60%), test duration (2 hours, 20%), and resource usage (3 types of resources, 20%), the priority score is 95.

[0076] The task scheduling unit compares the priority score of the task with that of waiting tasks in the queue. If 95 points is higher than 60 points, the target insertion position is determined to be the first position in the queue, and the mass production test task is added to the first position of the queue. The task scheduling unit continuously monitors the status of the scenario simulation processor. When the original R&D test task is completed, the resources become idle, and the mass production test task is at the first position in the queue, it is removed from the queue. According to the resource allocation strategy, multi-protocol adaptation interfaces, scenario simulation processors, and data acquisition channels are allocated to trigger the execution of the target test task.

[0077] In this embodiment, the gateway automated testing method further includes: obtaining the task execution status, resource usage, and test progress of the target test task; and generating a visualization chart based on the task execution status, resource usage, and test progress.

[0078] In this embodiment, the task execution status refers to the current progress stage of the target test task, reflecting the overall progress of the task, such as pending execution, in progress, completed, failed, and paused. Resource usage refers to the real-time utilization ratio of the target test resources, such as network interface utilization, processor load, acquisition channel utilization, and remaining storage space, used to determine if resources are overloaded. Test progress refers to the ratio of completed content to the total task volume, such as completed steps / total steps, collected data / estimated data, and running time / estimated time, reflecting task execution efficiency. Visual charts refer to intuitive display charts generated based on the task execution status, resource usage, and test progress, such as bar charts, line charts, and progress bars, which can be presented through the user interaction module, facilitating testers to quickly grasp the test dynamics.

[0079] This embodiment generates visual charts that can meet the real-time monitoring needs of testers for task progress and resource usage, making it easier to detect and handle anomalies in a timely manner. It takes into account the automation, accuracy and ease of use of testing, and is suitable for all scenarios of gateway testing.

[0080] For example, the step of generating the visualization corresponds to the task monitoring unit in the test task management module. The task monitoring unit can collect task execution status, resource usage, and progress in real time, displaying this information as bar charts, line charts, or progress bars through the user interaction module. When a task encounters an anomaly, it can automatically trigger alarms via sound and light, pop-ups, SMS, or email, recording fault information such as time, type, location, and description. Users can manually pause, terminate, and restart tasks, and view detailed logs through the log management module.

[0081] S104: If the target test resource is idle, then control instructions are generated based on the target test task. Based on the control instructions, the automated test of the gateway is executed through the target test resource, and test data during the automated test process of the gateway is obtained.

[0082] In this embodiment, the control commands include gateway control commands, device adaptation control commands, network scenario simulation commands, and data acquisition control commands; the target test resources include multi-protocol adaptation interfaces, scenario simulation processors, data acquisition channels, and data storage nodes; the automated testing process of the gateway includes multiple states, including gateway initialization, device access, scenario preparation, test execution, data acquisition startup, abnormal scenario triggering, and test termination. Each state has corresponding execution conditions and state switching rules; for each state, the state is executed when the execution conditions corresponding to that state are met, and the state is switched when the state switching rules corresponding to that state are met.

[0083] Based on control commands, the automated testing of the gateway is executed through the target test resources, and test data during the automated testing process is obtained, including:

[0084] The gateway control commands are sent to the gateway under test through a multi-protocol adapter interface; the gateway control commands are used to instruct the gateway to initialize; the gateway under test is used to perform power-on, IP address configuration, protocol parameter configuration, access permission configuration, restart and factory reset according to the gateway control commands, and to provide feedback on the execution results;

[0085] Device adaptation control commands are sent to physical devices and virtual device simulation nodes through a multi-protocol adaptation interface; the device adaptation control commands are used to instruct device access; the physical devices are used to execute terminal access, sensor acquisition frequency configuration, and controller command cycle configuration based on the device adaptation control commands, and to return the execution results; the virtual device simulation nodes are used to execute device simulation start / stop, sensor acquisition frequency configuration, and controller command cycle configuration based on the device adaptation control commands, and to return the execution results.

[0086] The network scenario simulation command is sent to the scenario simulation processor through a multi-protocol adaptation interface; the network scenario simulation command is used to indicate scenario preparation; the scenario simulation processor is used to execute network parameter settings, high-concurrency access startup, network switching triggering and interference start / stop based on the network scenario simulation command, and to provide feedback on the execution results.

[0087] The data acquisition control command is sent to the data acquisition channel; the data acquisition channel is used to execute channel selection configuration, sampling frequency configuration and acquisition start / stop based on the data acquisition control command, and to provide feedback on the execution results;

[0088] Once all feedback results indicate completion, the test execution state is entered; data acquisition control commands are used to instruct data acquisition to begin.

[0089] Initial test data is acquired through the data acquisition channel during the automated testing process of the gateway. The initial test data is then preprocessed to obtain preprocessed test data. Preprocessing includes noise reduction, standardization, and data compression. The data storage node is used to partition and store the preprocessed test data according to the data type.

[0090] In response to network switching or interference start / stop triggered by the scenario simulation processor, it enters the abnormal scenario trigger state; in response to receiving abnormal scenario completion information from the scenario simulation processor, it enters the test termination state.

[0091] In this embodiment, initial test data refers to the unprocessed raw data directly acquired by the data acquisition channel during the gateway automated testing process, covering multi-dimensional information such as network performance, device access, and system status. Preprocessing refers to the processing operations performed on the initial test data, aiming to improve data quality and storage efficiency. Denoising refers to using filtering algorithms (such as sliding window filtering) to remove outliers in the initial test data (such as sudden high packet loss rates caused by momentary device failures), reducing data interference. Standardization refers to converting initial test data of different formats and units into a standardized form (such as standardizing latency units to milliseconds and rate units to bytes per second) to ensure data comparability. Data compression refers to using efficient compression algorithms (such as LZ4 and ZSTD) to reduce the storage footprint of the preprocessed data, with a compression ratio of no less than 10:1.

[0092] The "Abnormal Scenario Completion Information" refers to the status signal fed back to the automated test execution module by the scenario simulation processor after completing the execution of abnormal scenarios such as network switching triggers or interference start / stop, marking the end of the abnormal scenario. "Task Execution Status" refers to the current progress stage of the target test task, reflecting the overall progress of the task. "Resource Usage" refers to the real-time utilization ratio of the target test resources, used to determine if resources are overloaded. "Test Progress" refers to the ratio of completed content of the target test task to the total task volume, reflecting task execution efficiency. "Visual Charts" refers to intuitive charts generated based on task execution status, resource usage, and test progress, presented through the user interaction module, facilitating testers to quickly grasp the test dynamics.

[0093] For example, each step in this embodiment involves a multi-protocol device adaptation module, a complex network scenario simulation module, a test data acquisition and synchronization module, an automated test execution module, and a test result analysis and fault location module in the gateway automated testing platform.

[0094] like Figure 3 As shown, the multi-protocol device adaptation module includes a protocol adaptation unit, a device access unit, and a protocol compatibility verification unit. Figure 3Arrows are used to indicate the data flow between each unit. For example, the protocol data parsed by the protocol adaptation unit is transmitted to the device access unit, and the device status data of the device access unit is fed back to the protocol compatibility verification unit, thus clarifying the internal workflow of the module.

[0095] The protocol adaptation unit comprises an FPGA+embedded processor module and a software-defined protocol stack module. Specifically, the protocol adaptation unit uses an FPGA and embedded processor to design a modular protocol parsing circuit, supporting the parsing and conversion of various mainstream protocols such as IEEE 802.3 (Ethernet), IEEE 802.11a / b / g / n / ac / ax (Wi-Fi), LoRa, ZigBee, Bluetooth, Modbus, MQTT, CoAP, and OPC UA. Through the software-defined protocol stack, the protocol adaptation unit can dynamically load protocol drivers according to testing requirements (such as loading Modbus and OPC UA drivers for testing industrial gateways, and loading Wi-Fi and ZigBee drivers for testing home gateways), ensuring protocol interoperability between the platform, gateways, and terminal devices.

[0096] The device access unit comprises a physical interface module, a virtual device simulation module (Docker container), and a device status monitoring module. Specifically, the device access unit integrates analog and physical interfaces for various types of terminal devices, including simulated sensors (temperature, humidity, and light intensity, etc.), industrial controllers (PLC, DCS), and smart home appliances (lighting fixtures, air conditioners), as well as network interfaces such as Ethernet ports, Wi-Fi antenna interfaces, and LoRa RF interfaces. The device access unit supports both physical device access and virtual device simulation based on Docker container technology, such as simulating 1000 MQTT sensors or 50 Modbus industrial controllers. It also allows for flexible configuration of the types, quantities, and parameters of the accessed devices (such as sensor acquisition frequency and controller command transmission cycle). The device access unit has a built-in device status monitoring circuit for real-time monitoring of the device's online status and communication quality, ensuring stable connections.

[0097] The protocol compatibility verification unit includes a pre-set test case library module and a custom test case editing module. Specifically, the unit has a pre-set multi-protocol compatibility test case library, including protocol interaction test cases (such as gateway handshake with Wi-Fi devices, data read / write with Modbus devices) and protocol conversion test cases (such as Modbus to MQTT data correctness, ZigBee to Ethernet latency). The unit also supports custom test cases (such as setting specific protocol parameter combinations, simulating abnormal protocol interactions) to verify the gateway's parsing correctness, data conversion accuracy, and communication stability in a multi-protocol environment.

[0098] like Figure 4As shown, the complex network scenario simulation module includes a network parameter simulation unit, a multi-network switching simulation unit, and a device high-concurrency access simulation unit. Figure 4 The arrows in the diagram indicate the interaction between each unit and the gateway. For example, the network parameter simulation unit processes the gateway traffic and forwards it to the target network, while the multi-network switching simulation unit sends a switching command to the gateway, reflecting the implementation logic of the scenario simulation.

[0099] The network parameter simulation unit comprises a network environment simulation engine (Intel Xeon D / Broadcom StrataXGS chip) and a port mirroring module. Specifically, the network parameter simulation unit uses an Intel Xeon D series processor and a Broadcom StrataXGS series chip to build the network environment simulation engine, supporting dynamic configuration and real-time adjustment of bandwidth from 1Mbps to 10Gbps, latency from 0-1000ms, packet loss rate from 0-100%, jitter from 0-100ms, and bit error rate from 0-10^-3. The network parameter simulation unit uses port mirroring technology to introduce gateway network traffic into the simulation engine, processes it according to set parameters (such as rate limiting, delayed forwarding, random packet loss, and error injection), and then forwards it to the target network, simulating scenarios such as industrial congestion and weak home signals. The network parameter simulation unit supports timed and random changes to network parameters, such as randomly adjusting bandwidth and packet loss rate every 10 minutes to simulate dynamic network environments.

[0100] The multi-network handover simulation unit includes multiple network interface modules and a handover trigger control module. Specifically, it integrates multiple network interfaces such as Ethernet, 4G / 5G, Wi-Fi, and LoRa, and can simulate gateway handover between different networks, such as Wi-Fi to 4G or Ethernet to LoRa. The multi-network handover simulation unit supports custom handover trigger conditions (such as signal strength below a threshold or insufficient bandwidth), and monitors the gateway connection status, data transmission interruption time, and retransmission success rate in real time during the handover process, verifying handover capabilities and data continuity.

[0101] The high-concurrency device access simulation unit comprises a virtualization cluster module (KVM / Docker) and a business logic configuration module. Specifically, based on KVM and Docker virtualization technologies, the unit creates a large-scale virtual terminal device cluster, supporting concurrent access of 1,000-100,000 virtual devices, such as 10,000 IoT sensors or 500 industrial controllers. The virtual devices interact with the gateway according to preset business logic (such as periodically sending data, randomly initiating connections, and sending commands in batches). The number of concurrent devices can be dynamically adjusted, such as gradually increasing from 1,000 to 10,000, to test the gateway's access processing capacity, forwarding efficiency, and stability.

[0102] The network interference simulation unit comprises an electromagnetic interference generator module and a radio frequency (RF) interference module. Specifically, the unit interfaces with an electromagnetic interference generator adjustable from 0-60dBμV / m across the 1MHz-6GHz frequency band. The unit also supports RF interference modules for Wi-Fi, LoRa, and ZigBee bands, simulating scenarios such as electromagnetic interference from industrial motors and RF interference from home wireless devices. The unit allows for customization of interference signal frequency, intensity, and duration (e.g., generating strong Wi-Fi band interference for a specific time period) to test gateway signal reception sensitivity, data transmission reliability, and anti-interference capabilities.

[0103] like Figure 5 As shown, the test data acquisition and synchronization module includes a multi-channel data acquisition unit, a data preprocessing unit, and a distributed storage and synchronization unit. Figure 5 The system demonstrates the complete data flow path from acquisition and preprocessing to storage. For example, data acquired by the acquisition card is preprocessed and then transmitted to the distributed storage unit via a high-speed bus, clearly illustrating the data flow process.

[0104] The multi-channel data acquisition unit comprises a high-speed data acquisition card module (1GS / s sampling rate, 16-bit precision) and a high-precision timestamp module (≤10ns). Specifically, the multi-channel data acquisition unit can be designed with a high-speed data acquisition card boasting a maximum sampling rate of 1GS / s and 16-bit sampling precision. It interfaces with gateway network interfaces, device access interfaces, and external testing equipment such as network analyzers, oscilloscopes, and power meters. The multi-channel data acquisition unit can acquire network performance data (forwarding rate, protocol conversion latency, throughput, and packet loss rate, etc.), device access data (number of accesses, communication success rate, and offline rate, etc.), system status data (CPU utilization, memory usage, port status, and power supply voltage, etc.), and abnormal log data (protocol parsing errors, network connection failures, and device access anomalies, etc.). Each acquisition channel is configured with a high-precision timestamp module with an accuracy of ≤10ns to ensure a consistent data time base.

[0105] The data preprocessing unit comprises a data cleaning module (sliding window filtering), a data standardization module, and a data compression module (LZ4 / ZSTD). Specifically, the data preprocessing unit uses a sliding window filtering algorithm to clean the collected test data, including removing outliers and filtering noise, such as eliminating abnormal packet loss rates caused by momentary equipment failures; standardizing different data formats, such as unifying forwarding rates (Mbps) and data volume (GB) to bytes per second, and latency (ms) and μs to ms; and compressing the data using LZ4 and ZSTD compression algorithms (compression ratio ≥10:1) to reduce storage usage. The preprocessed data is transmitted to the storage unit via PCIe 4.0 and 10Gbps Ethernet to ensure real-time performance and reliability.

[0106] The distributed storage and synchronization unit comprises a Ceph / GlusterFS storage module, a multi-dimensional indexing module, and a data backup module (local redundancy + off-site disaster recovery). Specifically, the distributed storage and synchronization unit builds a distributed storage architecture based on Ceph and GlusterFS, supporting petabyte-level data storage. The unit partitions data according to data types such as network performance, device access, system status, and anomaly logs, and establishes multi-dimensional indexes such as test task ID, time, gateway model, and scenario type for easy and rapid querying. The distributed storage and synchronization unit enables real-time data synchronization and backup (local redundant storage + off-site disaster recovery, using RAID5 / 6 redundancy technology) to prevent data loss. The unit supports access control; for example, test engineers can only access their own task data, while administrators can access all data, ensuring data security.

[0107] like Figure 6 As shown, Figure 6 The core states of automated test execution are presented using status nodes and arrows: gateway initialization state, device access and scenario preparation state, test execution state, abnormal scenario triggering state, and test termination state. Each status node indicates the execution conditions, such as "gateway initialization state" requiring "gateway parameter configuration completed" and "device connection normal"; arrows indicate state switching rules, such as switching from "device access and scenario preparation state" to "test execution state" requiring "terminal device access ready" and "network scenario configuration completed". Abnormal switching paths are also indicated, such as switching from "test execution state" to "test termination state" when "device failure" occurs, clearly demonstrating the automated control logic of the test process.

[0108] Specifically, the automated test execution module includes a control command generation unit, a command issuance and execution unit, and a test process automation control unit. The control command generation unit generates standardized control commands based on the test task steps and parameters, including gateway control commands, device adaptation control commands, network scenario simulation commands, and data acquisition control commands. The command format conforms to industry standard protocols such as Modbus TCP, SNMP, and MQTT, or vendor-defined protocols, ensuring recognizability and executability.

[0109] The command issuance and execution unit issues commands to each module and gateway via high-speed communication interfaces such as Ethernet, RS485, or USB, employing CRC checksum and ACK confirmation mechanisms to ensure accurate command delivery and execution. If command issuance fails, it automatically retryes; if the retry fails, an alarm is triggered and the test is paused. The unit receives real-time feedback information from each module (such as successful command execution, complete parameter configuration, scene readiness, or data acquisition initiation) and transmits it to the task monitoring unit to update task status and progress.

[0110] The automated test process control unit can manage the test process using a state machine model. It decomposes the test into ordered states such as gateway initialization, device access and scenario preparation, test execution, abnormal scenario triggering, and test termination. Each state has clearly defined execution conditions and switching rules. For example, gateway initialization requires parameter configuration to be complete and device connection to be normal before entering the scenario preparation state. Through real-time monitoring of module feedback (such as "parameter configuration complete" from the scenario simulation module and "terminal access ready" from the device adaptation module), the unit automatically completes state switching and advances the process. The automated test process control unit supports complex process nesting, such as periodically triggering interference and high-concurrency scenarios in stability testing. It can also dynamically adjust parameters based on data; for example, when CPU utilization consistently exceeds 90%, it reduces the number of concurrent devices to prevent gateway crashes.

[0111] S105: Determine the gateway test results based on test data and standard indicator thresholds.

[0112] In this embodiment, the test data includes the test indicator data corresponding to each of the target test indicators; the standard indicator threshold includes the standard indicator threshold corresponding to each of the target test indicators.

[0113] The gateway test results are determined based on test data and standard indicator thresholds, specifically including:

[0114] For each target test metric, the metric test score is calculated based on the test metric data corresponding to that target test metric and the standard metric threshold.

[0115] The overall gateway test score is obtained by weighted summing of the test scores of all target test metrics.

[0116] If the overall gateway test score is greater than the preset gateway test score threshold, the gateway test result is determined to be passed.

[0117] If the overall gateway test score is not greater than the gateway test score threshold, the gateway test result is determined to be a failure.

[0118] In this embodiment, after determining that the gateway test result is unsuccessful, the following steps are also included:

[0119] Target test indicators whose test scores are not greater than the preset test score threshold are considered abnormal indicators, and the test indicator data corresponding to abnormal indicators are considered abnormal data.

[0120] Multi-dimensional collected data is obtained based on timestamps and associated with abnormal indicators and abnormal data;

[0121] Based on multi-dimensional data collection, the fault analysis results are determined through a pre-trained machine learning model; the fault analysis results include the fault type and the fault cause.

[0122] Optimization suggestions are generated based on the fault analysis results;

[0123] A structured test report is generated based on the task information, fault analysis results, and optimization suggestions of the target test task.

[0124] In this embodiment, the indicator test score refers to the quantitative score calculated for each target test indicator, combining its corresponding test indicator data with the standard indicator threshold. The indicator test score reflects the degree of compliance of the indicator. The gateway test comprehensive score refers to the total score obtained by summing the indicator test scores of all target test indicators according to preset weights, reflecting the overall test performance of the gateway. The preset gateway test score threshold refers to the pre-set comprehensive score threshold used to determine whether the gateway test passes. An abnormal indicator refers to a target test indicator whose indicator test score is not greater than the preset indicator test score threshold, i.e., a test indicator that has not met the standard. Abnormal data refers to the test indicator data corresponding to the abnormal indicator. Multi-dimensional collected data refers to multiple types of test data associated with abnormal indicators and abnormal data through timestamps, covering dimensions such as network performance, device status, or scenario parameters. The pre-trained machine learning model refers to a model that has been trained in advance using a large amount of gateway test data, used for fault analysis. Fault analysis results refer to the analysis conclusions output by the model, including the fault type and fault cause. Fault type refers to the fault category corresponding to the anomaly, such as protocol compatibility fault. Fault cause refers to the specific factors that cause the fault, such as incorrect protocol parameter configuration. Optimization suggestions refer to improvement plans proposed based on the causes of failures. A structured test report is a standardized report that integrates task information, failure analysis results, and optimization suggestions.

[0125] In this embodiment, after determining that the gateway test result is passed, the method further includes: generating a structured test report based on the task information of the target test task, the indicator test scores of all target test indicators, and the comprehensive gateway test score.

[0126] For example, the test result analysis and test report generation in this embodiment correspond to the test result analysis and fault location module in the gateway automated testing platform. Figure 7 As shown, the test result analysis and fault location module includes a multi-dimensional result judgment unit, an intelligent fault analysis unit, an optimization suggestion generation unit, and a visualization report generation unit. Figure 7The module's workflow is presented in a step-by-step manner, as follows: Data Input (obtaining test data from distributed storage units) → Multi-dimensional Result Judgment (single indicator verification + comprehensive scoring) → Intelligent Fault Analysis (data tracing → anomaly classification → root cause localization) → Optimization Suggestion Generation (combining fault and gateway characteristics) → Report Output (structured report + interactive dashboard). Each step is marked with key operations. For example, the "Multi-dimensional Result Judgment" step includes "calling the standard database," "threshold comparison," and "weighted score calculation"; the "Intelligent Fault Analysis" step is marked with "correlating multi-dimensional data," "machine learning classification," and "specific localization methods," clearly defining the complete process of result analysis and fault localization.

[0127] The multi-dimensional result judgment unit incorporates a gateway testing standard database, covering industry standards such as IEEE 802.3 and MQTT, national / local standards such as GB / T 17626.3, and enterprise-defined standards, such as home gateway Wi-Fi access ≥50 devices and 5G edge gateway forwarding rate ≥1Gbps. The multi-dimensional result judgment unit employs a single-indicator verification and comprehensive scoring mechanism. First, individual data points are compared with standard thresholds (e.g., protocol conversion latency ≤100ms meets the standard, access success rate ≥99.5% meets the standard). Then, a weighted algorithm is used to calculate a comprehensive score (e.g., functionality 35%, performance 30%, compatibility 20%, security 15%). A score ≥85 is considered "passing," otherwise "failing." Custom indicator weights are supported, such as increasing the industrial gateway compatibility weight to 30%.

[0128] The intelligent fault analysis unit combines test data with network communication principles, employing a three-tiered logic of data tracing, anomaly classification, and root cause localization. Data tracing uses a unified timestamp to correlate multi-dimensional data (e.g., correlation of latency, CPU usage, and access logs when packet loss rate suddenly increases) to filter anomaly correlation chains. The intelligent fault analysis unit classifies anomalies using a pre-set fault type library (protocol compatibility, network performance, device access, and security protection faults, etc.) and a machine learning classification model (trained based on historical data, with an accuracy ≥92%). Specific methods are used for different fault types; for example, protocol compatibility faults compare protocol logs with standard formats, network performance faults analyze traffic curves and resource usage, and device access faults repeatedly simulate access tests, generating a fault report containing anomaly descriptions, associated data screenshots, root cause evidence, and reproduction steps.

[0129] The optimization suggestion generation unit generates targeted suggestions based on fault analysis results and gateway technical characteristics. For example, for protocol compatibility faults, the suggestion is "update the Modbus driver to V2.3"; for network performance faults, the suggestion is "optimize the forwarding algorithm or upgrade the CPU"; and for device access faults, the suggestion is "expand the number of Wi-Fi channels and ZigBee nodes". The suggestions include specific implementation steps (driver update process, algorithm optimization code direction) and expected results, such as an access success rate ≥99.8% after driver update and a 30% reduction in latency after algorithm optimization.

[0130] The visualization report generation unit can automatically generate structured reports, including basic task information (such as ID, gateway model and version, time, and personnel), configuration details (such as indicator thresholds, scenario parameters, and device lists), result statistics (such as indicator compliance rate, overall score, and number of devices that passed / failed), anomaly analysis (trend charts, fault percentage pie charts, and data comparison tables), and fault and optimization suggestions (fault list, root causes, and solutions). The visualization report generation unit supports exporting to PDF, Excel, or HTML formats and automatically uploads to document management systems such as SharePoint and Confluence. It generates interactive data dashboards that support drag-and-drop, filtering, and zooming to view detailed data; for example, clicking on anomaly timestamps allows viewing logs and traffic data.

[0131] This embodiment uses quantitative calculations of indicator test scores and gateway test comprehensive scores to make the judgment of gateway test results more objective and accurate. This embodiment finally generates a unified test report. For cases where the test fails, it locates abnormal indicators and correlates them with multi-dimensional collected data. Combined with a pre-trained machine learning model to analyze the fault, it can quickly identify the fault type and cause, significantly improving fault diagnosis efficiency. The structured test report integrates task information, fault analysis results, and optimization suggestions, allowing testing and development personnel to intuitively obtain improvement directions, assisting in rapid gateway iteration and optimization, and effectively improving the collaborative efficiency of testing and development.

[0132] For example, taking the multi-protocol compatibility and high-concurrency access performance testing of industrial gateways as an example, the automated gateway testing in this embodiment includes the following steps:

[0133] (1) Test Preparation Phase. This embodiment configures the gateway and device access. Specifically, the industrial gateway to be tested (supporting Modbus, OPC UA, and EtherCAT protocols) is connected to the multi-protocol device adapter module via an Ethernet port. The platform automatically identifies the gateway type and loads the Modbus V2.3, OPC UAV 1.04, and EtherCAT V1.5 protocol drivers. Two physical PLC controllers and five physical temperature sensors are connected through the device access unit. At the same time, 100 Modbus sensors and 50 OPC UA industrial instruments are simulated based on Docker containers, and the device parameters are configured (such as the PLC controller sampling frequency of 1 time / second and the sensor data upload cycle of 500ms).

[0134] This embodiment describes the creation and parameter configuration of test tasks. Specifically, users can select the "Industrial Gateway Multi-Protocol Compatibility + High Concurrency Access Performance" combined test template in the test task management module, customize test indicators (Modbus device access success rate ≥99.8%, OPC UA protocol conversion latency ≤80ms, data forwarding rate ≥500Mbps when 150 devices access concurrently, CPU utilization ≤85%), test scenarios (normal network scenario + industrial electromagnetic interference scenario (interference frequency 500MHz, intensity 35dBμV / m)), and test duration (12-hour stability test), create a test task, and assign the ID "IndGateway-20250610-001".

[0135] (2) Test Execution Phase. Automated process initiated. Specifically, the automated test execution module generates control commands and completes the following in sequence: gateway initialization (configuring gateway IP address 192.168.1.100, protocol timeout 500ms), device access (starting physical and virtual devices, establishing communication connections between devices and the gateway), scenario preparation (configuring normal network parameters in the complex network scenario simulation module: bandwidth 1Gbps, latency 20ms, packet loss rate 0.1%), and data acquisition initiation (collecting protocol interaction logs, device access status, data forwarding rate, gateway CPU / memory usage, sampling frequency 1 time / second).

[0136] Dynamic scenario triggering. Specifically, in the 4th hour of the test, the platform automatically triggered an industrial electromagnetic interference scenario (interference intensity 35dBμV / m, lasting 30 minutes) to monitor the device communication stability and data forwarding reliability of the gateway in the interference environment in real time; in the 8th hour, the number of concurrent access devices was increased from 150 to 200 to test the gateway's high concurrency limit performance.

[0137] Anomaly Handling and Resumption of Testing. Specifically, during the 10th hour of testing, a node failure in the virtual device cluster caused 20 Modbus sensors to disconnect. The platform detected the offline status through the device status monitoring circuit, immediately triggered pop-up and email alerts, and simultaneously started a backup virtual device node to reconnect the offline devices, continuing the test from the point of interruption without having to re-execute the previous 10 hours of testing.

[0138] (3) Results Analysis and Feedback Phase. Result Judgment. Specifically, after the test, the test results analysis module calls the industrial gateway test standard database to compare the collected data: Modbus device access success rate 99.9% (meets the standard), average OPC UA protocol conversion latency 65ms (meets the standard), data forwarding rate of 520Mbps when 200 devices are connected concurrently (meets the standard), and peak CPU utilization rate of 82% (meets the standard). A comprehensive score is calculated using a weighted algorithm (functionality 35%, performance 30%, compatibility 20%, security 15%), resulting in a final score of 91 points, thus determining "test passed".

[0139] Fault Analysis. Specifically, the module traced the source of the problem through data analysis and found that during the fourth hour of electromagnetic interference, three ZigBee sensors experienced a brief communication interruption of 2-3 seconds. Corresponding to the gateway interface status data and interference signal data during the same period, a machine learning classification model categorized this as a "network performance failure." Using a specialized localization method (comparing the ZigBee interface signal strength and protocol interaction logs before and after the interference), the root cause was determined to be "insufficient anti-interference capability of the gateway ZigBee interface, causing temporary interface loss due to interference signals." A fault report was generated, including screenshots of the anomaly, related data tables, and steps to reproduce the fault.

[0140] Report Application and Optimization. Specifically, a structured test report is automatically generated, exported, and sent to the industrial gateway R&D team. Based on the optimization suggestions in the report (such as "adding a signal filtering circuit to the gateway's ZigBee interface and upgrading the interface chip's anti-interference level"), the R&D personnel complete the hardware design optimization. The iterated gateway is then subjected to the same tests again. Under industrial electromagnetic interference scenarios, the ZigBee device communication interruption rate is reduced to below 0.1%, meeting the requirements of industrial field applications.

[0141] Corresponding to the gateway automated testing method in the above embodiment, Figure 8 This is a structural block diagram of a gateway automated testing system provided according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. Reference Figure 8 The gateway automated testing system 20 includes: a first task creation module 21, a second task creation module 22, a test resource allocation module 23, a testing module 24, and a test result analysis module 25.

[0142] The first task creation module 21 is used to create a target test task based on the target test template in response to receiving the target test template selected by the user and not receiving the user's custom input. The target test task includes a task ID, a target test scenario, a target test step, and a target test metric. The target test template is a template in the standardized test template library, which includes multiple preset test templates. Each preset test template includes a preset test scenario, test step, and test metric.

[0143] The second task creation module 22 is used to respond to receiving the target test template selected by the user and the custom information input by the user, update the target test template based on the custom information, and create a target test task based on the updated target test template; the custom information includes at least one of custom test scenarios, custom test steps and custom test metrics; the custom information is used to update at least one of the preset test scenarios, test steps and test metrics in the target test template;

[0144] The test resource allocation module 23 is used to determine the resource allocation strategy for the target test task and to determine the target test resources based on the resource allocation strategy.

[0145] Test module 24 is used to generate control instructions based on the target test task if the target test resource is idle, and to execute the automated test of the gateway through the target test resource and obtain the test data during the automated test process of the gateway based on the control instructions.

[0146] The test result analysis module 25 is used to determine the gateway test results based on test data and standard indicator thresholds.

[0147] In one embodiment of this application, when determining the resource allocation strategy for the target test task, the test resource allocation module 23 is specifically used to determine the task priority and resource requirements of the target test task; determine the resource allocation strategy based on the task priority and resource requirements; and the resource allocation strategy is an allocation strategy for multiple test resources.

[0148] In one embodiment of this application, a gateway automated testing system further includes: a task waiting module, used for:

[0149] If the target test resource is in use, then retrieve the current task waiting queue;

[0150] If the current task waiting queue is empty, add the target test task to the current task waiting queue until the target test resource becomes idle and the target test task is at the head of the task waiting queue, then remove the target test task from the task waiting queue.

[0151] If the current task waiting queue is not empty, calculate the priority score of the target test task; determine the target insertion position based on the priority score of the target test task and the priority scores of each waiting task in the current task waiting queue, add the target test task to the target insertion position in the current task waiting queue, until the target test resource becomes idle and the target test task is at the first position in the task waiting queue, then delete the target test task from the task waiting queue.

[0152] In one embodiment of this application, the control instructions include gateway control instructions, device adaptation control instructions, network scenario simulation instructions, and data acquisition control instructions; the target test resources include a multi-protocol adaptation interface, a scenario simulation processor, a data acquisition channel, and a data storage node; the automated testing process of the gateway includes multiple states, including gateway initialization, device access, scenario preparation, test execution, data acquisition startup, abnormal scenario triggering, and test termination, each state corresponding to execution conditions and state switching rules; for each state, the state is executed when the execution conditions corresponding to that state are met, and the state is switched when the state switching rules corresponding to that state are met; when the test module 24 executes the automated testing of the gateway based on the control instructions and obtains test data during the automated testing process of the gateway through the target test resources, it is specifically used for:

[0153] The gateway control commands are sent to the gateway under test through a multi-protocol adapter interface; the gateway control commands are used to instruct the gateway to initialize; the gateway under test is used to perform power-on, IP address configuration, protocol parameter configuration, access permission configuration, restart and factory reset according to the gateway control commands, and to provide feedback on the execution results;

[0154] Device adaptation control commands are sent to physical devices and virtual device simulation nodes through a multi-protocol adaptation interface; the device adaptation control commands are used to instruct device access; the physical devices are used to execute terminal access, sensor acquisition frequency configuration, and controller command cycle configuration based on the device adaptation control commands, and to return the execution results; the virtual device simulation nodes are used to execute device simulation start / stop, sensor acquisition frequency configuration, and controller command cycle configuration based on the device adaptation control commands, and to return the execution results.

[0155] The network scenario simulation command is sent to the scenario simulation processor through a multi-protocol adaptation interface; the network scenario simulation command is used to indicate scenario preparation; the scenario simulation processor is used to execute network parameter settings, high-concurrency access startup, network switching triggering and interference start / stop based on the network scenario simulation command, and to provide feedback on the execution results.

[0156] The data acquisition control command is sent to the data acquisition channel; the data acquisition channel is used to execute channel selection configuration, sampling frequency configuration and acquisition start / stop based on the data acquisition control command, and to provide feedback on the execution results;

[0157] Once all feedback results indicate completion, the test execution state is entered; data acquisition control commands are used to instruct data acquisition to begin.

[0158] Initial test data is acquired through the data acquisition channel during the automated testing process of the gateway. The initial test data is then preprocessed to obtain preprocessed test data. Preprocessing includes noise reduction, standardization, and data compression. The data storage node is used to partition and store the preprocessed test data according to the data type.

[0159] In response to network switching or interference start / stop triggered by the scenario simulation processor, it enters the abnormal scenario trigger state; in response to receiving abnormal scenario completion information from the scenario simulation processor, it enters the test termination state.

[0160] In one embodiment of this application, a gateway automated testing system further includes: a task status monitoring module, used to obtain the task execution status, resource usage, and test progress of the target test task; and to generate a visual chart based on the task execution status, resource usage, and test progress.

[0161] In one embodiment of this application, the test data includes test indicator data corresponding to each of the target test indicators; the standard indicator thresholds include standard indicator thresholds corresponding to each of the target test indicators; and the test result analysis module 25, when determining the gateway test result based on the test data and the standard indicator thresholds, is specifically used for:

[0162] For each target test metric, the metric test score is calculated based on the test metric data corresponding to that target test metric and the standard metric threshold.

[0163] The overall gateway test score is obtained by weighted summing of the test scores of all target test metrics.

[0164] If the overall gateway test score is greater than the preset gateway test score threshold, the gateway test result is determined to be passed.

[0165] If the overall gateway test score is not greater than the gateway test score threshold, the gateway test result is determined to be a failure.

[0166] In one embodiment of this application, a gateway automated testing system further includes: a test report generation module, used to regard target test indicators whose indicator test scores are not greater than a preset indicator test score threshold as abnormal indicators, and the test indicator data corresponding to the abnormal indicators as abnormal data.

[0167] Multi-dimensional collected data is obtained based on timestamps and associated with abnormal indicators and abnormal data;

[0168] Based on multi-dimensional data collection, the fault analysis results are determined through a pre-trained machine learning model; the fault analysis results include the fault type and the fault cause.

[0169] Optimization suggestions are generated based on the fault analysis results;

[0170] A structured test report is generated based on the task information, fault analysis results, and optimization suggestions of the target test task.

[0171] See Figure 9 , Figure 9 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 9 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned system embodiments, for example... Figure 8 The functions of the first task creation module 21, the second task creation module 22, the test resource allocation module 23, the test module 24, and the test result analysis module 25 are shown.

[0172] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0173] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0174] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store information about test data.

[0175] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the gateway automated testing method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0176] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0177] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0179] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces or units, or they may be electrical, mechanical, or other forms of connection.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0182] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0183] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A gateway automated testing method, characterized in that, include: In response to receiving a target test template selected by the user and not receiving any custom information input by the user, a target test task is created based on the target test template; The target test task includes a task ID, a target test scenario, target test steps, and target test metrics; the target test template is a template in a standardized test template library, which includes multiple preset test templates, each of which includes a preset test scenario, test steps, and test metrics. In response to receiving the target test template selected by the user and the custom information input by the user, the target test template is updated based on the custom information, and a target test task is created based on the updated target test template; The custom information includes at least one of the following: custom test scenario, custom test steps, and custom test metrics; The custom information is used to update at least one of the preset test scenarios, test steps, and test metrics in the target test template; Determine the resource allocation strategy for the target test task, and determine the target test resources based on the resource allocation strategy; If the target test resource is idle, then control instructions are generated based on the target test task. Based on the control instructions, the automated test of the gateway is executed through the target test resource and test data during the automated test process of the gateway is obtained. The gateway test results are determined based on the test data and standard indicator thresholds.

2. The gateway automated testing method as described in claim 1, characterized in that, The determination of the resource allocation strategy for the target test task includes: Determine the task priority and resource requirements of the target test task; A resource allocation strategy is determined based on the task priority and the resource requirements; the resource allocation strategy is a strategy for allocating multiple test resources.

3. The gateway automated testing method as described in claim 1, characterized in that, After determining the target test resources based on the resource allocation strategy, the method further includes: If the target test resource is occupied, then obtain the current task waiting queue; If the current task waiting queue is empty, the target test task is added to the current task waiting queue until the target test resource becomes idle and the target test task is at the head of the task waiting queue, then the target test task is deleted from the task waiting queue. If the current task waiting queue is not empty, calculate the priority score of the target test task; determine the target insertion position based on the priority score of the target test task and the priority scores of each waiting task in the current task waiting queue, add the target test task to the target insertion position in the current task waiting queue, until the target test resource becomes idle and the target test task is at the first position in the task waiting queue, then delete the target test task from the task waiting queue.

4. The gateway automated testing method as described in claim 1, characterized in that, The control commands include gateway control commands, device adaptation control commands, network scenario simulation commands, and data acquisition control commands; the target test resources include multi-protocol adaptation interfaces, scenario simulation processors, data acquisition channels, and data storage nodes; the automated testing process of the gateway includes multiple states, namely gateway initialization, device access, scenario preparation, test execution, data acquisition startup, abnormal scenario triggering, and test termination. Each state has corresponding execution conditions and state switching rules; for each state, the state is executed when the execution conditions corresponding to that state are met, and the state is switched when the state switching rules corresponding to that state are met. The step of executing automated testing of the gateway through the target test resources based on the control command and obtaining test data during the automated testing process of the gateway includes: The gateway control command is sent to the gateway under test through the multi-protocol adaptation interface; the gateway control command is used to instruct the gateway to initialize; the gateway under test is used to perform power-on, IP address configuration, protocol parameter configuration, access permission configuration, restart and factory reset according to the gateway control command, and to provide feedback on the execution results; The device adaptation control command is sent to the physical device and the virtual device simulation node through the multi-protocol adaptation interface; the device adaptation control command is used to instruct the device to access; the physical device is used to perform terminal access, sensor acquisition frequency configuration and controller command cycle configuration based on the device adaptation control command, and to feed back the execution results; the virtual device simulation node is used to perform device simulation start and stop, sensor acquisition frequency configuration and controller command cycle configuration based on the device adaptation control command, and to feed back the execution results. The network scenario simulation command is sent to the scenario simulation processor through a multi-protocol adaptation interface; the network scenario simulation command is used to indicate scenario preparation; the scenario simulation processor is used to perform network parameter settings, high-concurrency access startup, network switching triggering and interference start / stop based on the network scenario simulation command, and to provide feedback on the execution results; The data acquisition control command is sent to the data acquisition channel; the data acquisition channel is used to execute channel selection configuration, sampling frequency configuration and acquisition start / stop based on the data acquisition control command, and to provide feedback on the execution results; Upon receiving all feedback results indicating completion, the system enters the test execution state; the data acquisition control command is used to instruct data acquisition to begin. Initial test data during the automated testing process of the gateway is acquired through the data acquisition channel. The initial test data is then preprocessed to obtain preprocessed test data. The preprocessing includes noise reduction, standardization, and data compression. The data storage node is used to partition and store the preprocessed test data according to its data type. In response to the scenario simulation processor executing network switching trigger or interference start / stop, it enters the abnormal scenario trigger state; in response to receiving the abnormal scenario completion information fed back by the scenario simulation processor, it enters the test termination state.

5. The gateway automated testing method as described in claim 1, characterized in that, Also includes: Obtain the task execution status, resource usage, and test progress of the target test task; Visual charts are generated based on the task execution status, resource usage, and test progress.

6. The gateway automated testing method as described in claim 1, characterized in that, The test data includes the test indicator data corresponding to each of the target test indicators; the standard indicator thresholds include the standard indicator thresholds corresponding to each of the target test indicators. The process of determining the gateway test result based on the test data and standard indicator thresholds includes: For each target test metric, the metric test score is calculated based on the test metric data corresponding to that target test metric and the standard metric threshold. The overall gateway test score is obtained by weighted summing of the test scores of all target test metrics. If the overall gateway test score is greater than the preset gateway test score threshold, the gateway test result is determined to be passed. If the overall gateway test score is not greater than the gateway test score threshold, then the gateway test result is determined to be a failure.

7. The gateway automated testing method as described in claim 6, characterized in that, After determining that the gateway test result is unsuccessful, the method further includes: Target test indicators whose test scores are not greater than the preset test score threshold are considered abnormal indicators, and the test indicator data corresponding to abnormal indicators are considered abnormal data. Multi-dimensional collected data associated with the abnormal indicators and the abnormal data are obtained based on timestamps; Based on the multi-dimensional collected data, the fault analysis results are determined through a pre-trained machine learning model; the fault analysis results include the fault type and the fault cause. Optimization suggestions are generated based on the fault analysis results; A structured test report is generated based on the task information of the target test task, the fault analysis results, and the optimization suggestions.

8. An automated gateway testing system, characterized in that, include: The first task creation module is used to create a target test task based on the target test template when it receives the target test template selected by the user but does not receive any custom information input by the user. The target test task includes a task ID, a target test scenario, a target test step, and a target test metric. The target test template is a template in a standardized test template library, which includes multiple preset test templates. Each preset test template includes a preset test scenario, test step, and test metric. The second task creation module is used to respond to receiving the target test template selected by the user and the custom information input by the user, update the target test template based on the custom information, and create a target test task based on the updated target test template; The custom information includes at least one of the following: custom test scenario, custom test steps, and custom test metrics; The custom information is used to update at least one of the preset test scenarios, test steps, and test metrics in the target test template; The test resource allocation module is used to determine the resource allocation strategy for the target test task and determine the target test resources based on the resource allocation strategy. The testing module is used to generate control instructions based on the target test task if the target test resource is idle, and to execute automated testing of the gateway through the target test resource and obtain test data during the automated testing process of the gateway based on the control instructions. The test result analysis module is used to determine the gateway test results based on the test data and standard indicator thresholds.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images