Method and device for testing efficiency of industrial control equipment based on LLM large model

Abstract

The invention relates to an efficiency test method and device for industrial control equipment based on an LLM large model. The method is applied to an efficiency test system of industrial control equipment. The system comprises an industrial control target range platform, a tool library, a pre-trained LLM large model and an analysis module. The method comprises the following steps: simulating a test environment of to-be-tested target industrial control equipment based on an industrial control target range platform, and collecting operation data of the target industrial control equipment; inputting the operation data, the test task requirements of the target industrial control equipment and a tool library into a pre-trained large model to obtain a test scheme; the test scheme comprises tools required for completing the test task and a test sequence and test parameters of each tool; according to the test scheme, the corresponding tools are used in sequence for performance test, and test data of the tools are sent to the analysis module; and analyzing the test data by using an analysis module, and generating an efficiency test report of the target industrial control equipment. According to the invention, the efficiency test of the industrial control equipment can be efficiently and accurately completed.

Description

Technical Field

[0001] The present invention relates to the technical field of performance testing, and in particular to a performance testing method and device for industrial control equipment based on an LLM large model. Background Art

[0002] In the field of industrial control, the performance and efficiency of various types of industrial control equipment are directly related to the stability, safety and efficiency of the entire industrial production system. Traditional industrial control equipment performance testing methods are relatively limited, relying mostly on manual experience and fixed test scripts, and are difficult to adapt to the increasingly complex and changeable industrial control environment and the intelligent evolution of equipment. Although the industrial control range platform can provide an environmental foundation for simulating industrial control scenarios, it lacks the ability to generate and optimize intelligent test strategies. Although the tool library contains a variety of test tools, it has shortcomings in the intelligent calling and collaborative operation of the tools. Therefore, the existing technology cannot accurately and comprehensively test the performance of industrial control equipment.

[0003] Based on this, there is an urgent need for a performance testing method and device for industrial control equipment based on the LLM large model to solve the above problems. Summary of the invention

[0004] The present invention provides a performance testing method and device for industrial control equipment based on the LLM large model, which can efficiently, accurately and comprehensively complete the performance testing of industrial control equipment. The technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides an efficiency testing method for industrial control equipment based on an LLM large model, which is applied to an efficiency testing system for industrial control equipment, wherein the system comprises an industrial control range platform, a tool library, a pre-trained LLM large model and an analysis module; the large model communicates with the industrial control range platform, the tool library and the analysis module respectively, and the tool library communicates with the analysis module; the method comprises:

[0006] Simulating the test environment of the target industrial control equipment to be tested based on the industrial control range platform, and collecting the operating data of the target industrial control equipment;

[0007] Input the operating data, the test task requirements of the target industrial control equipment and the tool library into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool;

[0008] According to the test plan, use corresponding tools in turn to perform performance tests, and send the test data of each tool to the analysis module;

[0009] The test data is analyzed by using the analysis module to generate a performance test report for the target industrial control equipment.

[0010] In a second aspect, an embodiment of the present invention further provides an LLM large model-based performance testing device for industrial control equipment, which is applied to a performance testing system for industrial control equipment. The system includes an industrial control range platform, a tool library, a pre-trained LLM large model, and an analysis module; the large model communicates with the industrial control range platform, the tool library, and the analysis module respectively, and the tool library communicates with the analysis module; the device includes:

[0011] A simulation unit, used to simulate the test environment of the target industrial control equipment to be tested based on the industrial control range platform, and collect the operation data of the target industrial control equipment;

[0012] An input unit, used to input the operating data, the test task requirements of the target industrial control equipment and the tool library into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool;

[0013] A testing unit, used to perform performance testing using corresponding tools in sequence according to the testing scheme, and send the test data of each tool to the analysis module;

[0014] A generating unit is used to analyze the test data using the analyzing module to generate a performance test report for the target industrial control equipment.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0017] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method described above when executed by a processor.

[0018] The embodiment of the present invention provides a method and device for performance testing of industrial control equipment based on the LLM large model. The method is applied to the performance testing system of industrial control equipment. When it is necessary to perform performance testing on the target industrial control equipment to be tested, the test environment of the target industrial control equipment can be simulated through the industrial control range platform, and the operation data of the target industrial control equipment can be collected. Then, the operation data, the test task requirements of the target industrial control equipment and the tool library are input into the pre-trained large model, and the large model can select the tools required to complete the test task and the test sequence and test parameters of each tool from the tool library, and then the corresponding tools can be used in turn for performance testing, and the test data of each tool can be sent to the analysis module. Finally, the test data is analyzed based on the analysis module to generate a performance test report for the target industrial control equipment. It can be seen that the present application can efficiently, accurately and comprehensively complete the performance testing of industrial control equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a performance testing method of industrial control equipment based on the LLM large model provided by one embodiment of the present invention;

[0021] Figure 2 It is a structural diagram of a performance testing device for industrial control equipment based on an LLM large model provided by an embodiment of the present invention;

[0022] Figure 3 is a hardware architecture diagram of a computer device provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of a performance testing system for industrial control equipment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] The inventor discovered in his work that the artificial intelligence large language model (LLM) has excellent semantic understanding, logical reasoning and knowledge integration capabilities. Based on this, the inventor proposed that it can be integrated into the test system of the industrial control range platform and tool library, and the performance test method of industrial control equipment based on the LLM large model can be used to break the bottleneck of existing testing technology and create a new intelligent situation for the performance test of industrial control equipment.

[0026] The specific implementation of the above concept is described below.

[0027] Please refer to Figure 1 The embodiment of the present invention provides a performance testing method for industrial control equipment based on the LLM large model, which is applied to the performance testing system of industrial control equipment, such as Figure 4 As shown, the system includes an industrial control range platform, a tool library, a pre-trained LLM large model and an analysis module; the large model communicates with the industrial control range platform, the tool library and the analysis module respectively, and the tool library communicates with the analysis module; the method includes:

[0028] Step 100, simulating the test environment of the target industrial control equipment to be tested based on the industrial control range platform, and collecting the operating data of the target industrial control equipment;

[0029] Step 102, inputting the operation data, the test task requirements of the target industrial control equipment and the tool library into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool;

[0030] Step 104, according to the test plan, use the corresponding tools in turn to perform performance tests, and send the test data of each tool to the analysis module;

[0031] Step 106: Analyze the test data using the analysis module to generate a performance test report for the target industrial control equipment.

[0032] In this embodiment, the method is applied to the performance test system of industrial control equipment. When it is necessary to perform a performance test on the target industrial control equipment to be tested, the test environment of the target industrial control equipment can be simulated through the industrial control range platform, and the operating data of the target industrial control equipment can be collected. Then, the operating data, the test task requirements of the target industrial control equipment and the tool library are input into the pre-trained large model. The large model can select the tools required to complete the test task and the test sequence and test parameters of each tool from the tool library, and then use the corresponding tools in turn to perform performance tests, and send the test data of each tool to the analysis module. Finally, the test data is analyzed based on the analysis module to generate a performance test report for the target industrial control equipment. It can be seen that the present application can efficiently, accurately and comprehensively complete the performance test of industrial control equipment.

[0033] It should be noted that the test system also includes a display module for displaying test results and test reports.

[0034] Described below Figure 1 How the various steps are performed.

[0035] First, with respect to step 100, the test environment of the target industrial control equipment to be tested is simulated based on the industrial control range platform, and the operating data of the target industrial control equipment is collected.

[0036] In this step, the industrial control range platform can build a realistic industrial control simulation environment, which can simulate various industrial production processes, network topologies, and possible external interference sources. In addition, the industrial control range platform is equipped with a wealth of industrial data acquisition nodes and interfaces, which can capture various key data of industrial control equipment during operation in real time, such as the operation data, operation status information, network traffic data, control instruction execution status, etc. of industrial control equipment, and provide controllable test scene construction and switching functions for equipment performance testing. In addition, after the industrial control scenario of the industrial control equipment is determined, clear test task objectives can be set, such as measuring the key business continuity indicators and system resource utilization changes of industrial control equipment under specific network attack modes. When the data acquisition function of the industrial control range platform is started, the initial operation data of the equipment can be fully collected, including the configuration information of the hardware equipment, the initial state parameters of the software system, the network connection status, etc., and these data can be transmitted to the LLM large model in real time.

[0037] Of course, the industrial control range platform can collect the operating data of industrial control equipment at any time. The type of operating data collected is determined according to the industrial scenario and user needs, and this application does not make specific limitations.

[0038] For step 102, the operation data, the test task requirements of the target industrial control equipment and the tool library are input into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool.

[0039] In addition, before inputting the operation data and test task requirements into the large model, the data can also be cleaned and formatted to remove noise data and irrelevant information, and the equipment data can be converted into a structured format suitable for model input, such as normalizing the numerical data collected by the sensor, and performing word segmentation and word vector encoding on the text data. In this way, the recognition accuracy and efficiency of the large model can be improved.

[0040] In addition, the large model uses a pre-trained language model based on the Transformer architecture. This model can be pre-trained and fine-tuned on a large amount of text data, technical documents, test cases, and related standards and specifications in the field of industrial control, so that the model can deeply understand the professional knowledge and task requirements of industrial control equipment performance testing.

[0041] The following first introduces the training process of the large model:

[0042] Step A1, constructing a test task sample set, which includes multiple known test tasks. The test environment of the industrial control equipment corresponding to each test task is built based on the industrial control range platform, and each test task is marked with the test objectives, performance indicators and required tool types;

[0043] Step A2: for each known test task, the operation data of the industrial control equipment corresponding to the test task is collected, and the operation data, the test task and the tool library are input into the big model; the big model performs semantic analysis on the test task and the operation data based on a preset algorithm to obtain the test target of the test task, and selects the type of tool required to execute the test task from the tool library based on the test target, and selects tools from each tool type, as well as the use order and test parameters of each tool, to generate a test plan;

[0044] Step A3: according to the test plan, the relevant tools in the tool library are activated in order, and the performance test of the corresponding industrial control equipment is performed; the test data of each tool is fed back to the analysis module; the analysis module determines the test effect evaluation index of the test plan based on the test data and the corresponding performance index;

[0045] Step A4, reward or punish the decision of the large model based on the test effect evaluation index, and adjust the decision strategy of the large model based on the reward or punishment; and so on, until the task test is performed according to the test plan output by the large model, and the obtained test effect evaluation index meets the corresponding performance index, and a trained large model is obtained.

[0046] In step A2, the preset algorithms include a semantic understanding algorithm, a knowledge fusion algorithm, and a strategy generation algorithm. The above three algorithms are respectively introduced in detail below.

[0047] (1) Semantic understanding algorithm: This algorithm uses a semantic analysis algorithm based on a neural network to perform semantic analysis on the input test task description and the operating data of the industrial control equipment, convert the text data into a vector representation, and identify key information (such as test objectives, equipment types, test scenarios, and performance indicators) by calculating the similarity and semantic relationship between vectors. This key information is then combined with the operating data of the industrial control equipment to construct a complete test task scenario description.

[0048] In some embodiments, the algorithm can use the word vector representation method of the Word2Vec or BERT model, combined with the attention mechanism, to focus on important words and phrases in the text and accurately extract semantic information.

[0049] For example, when the input test task description is: "In an environment with electromagnetic interference, evaluate the positioning accuracy and repeatability of an industrial robot during material handling", the semantic understanding algorithm can accurately identify "industrial robot" as the equipment type, "material handling" as the test scenario, that is, the working scenario, "positioning accuracy" and "repeatability" as key performance indicators, and "electromagnetic interference" as an environmental factor.

[0050] For example, if the received equipment data shows that the current port traffic of an industrial switch is abnormal, and the test task is to evaluate its stability in a high-load network environment, the LLM model will integrate this information and clearly define the test goal as exploring the performance of the switch under high traffic and possible fault points.

[0051] (2) Knowledge fusion algorithm: This algorithm combines the knowledge graph related to industrial control equipment with the neural network so that the large model can perform knowledge reasoning and decision optimization based on the entity relationships and attribute information in the knowledge graph and its own language understanding ability.

[0052] The algorithm uses a combination of knowledge graph technology and neural networks to construct the knowledge system in the field of industrial control, including equipment structure knowledge, process flow knowledge, testing standard knowledge, etc. into a knowledge graph.

[0053] For example, when testing a specific model of PLC (programmable logic controller), the knowledge graph can provide information such as the PLC's hardware architecture, supported communication protocols, common failure modes, etc. The LLM model can use this knowledge and the currently collected PLC operation data to decide whether to use specific communication protocol analysis tools and how to set test parameters in order to more efficiently detect potential communication failure risks.

[0054] (3) Strategy generation algorithm: This algorithm is based on the reinforcement learning algorithm framework. It determines the required tools according to the functional characteristics, scope of application, and matching degree of each tool with the current industrial control equipment and test environment. Based on the knowledge fusion algorithm, it combines the knowledge graph related to the current industrial control equipment to determine the order of use, parameter settings, and collaborative working mode of each tool to generate a test plan. In response to the execution of the test plan, the decision-making strategy is adjusted based on the test effect evaluation index fed back by the analysis module, and the optimal test strategy is gradually learned.

[0055] In some implementations, a Deep Q-Network (DQN) algorithm may be used. The tool screening and strategy formulation process is as follows:

[0056] According to the analyzed test task scenario, the strategy generation algorithm is used to select suitable test tools in the tool library. The functional characteristics, scope of application and matching degree of the tool with the current equipment and test environment are considered. At the same time, based on the knowledge fusion algorithm and combined with the industrial control knowledge graph, the use order, parameter setting and collaborative working mode of the test tools are determined to formulate a detailed test strategy.

[0057] For example, for the test of the above-mentioned industrial switch, the LLM model may select a network traffic generation tool to simulate high-load traffic, select a protocol analysis tool to monitor the packet processing of the switch port, and determine to use the traffic generation tool to gradually increase the traffic to a predetermined threshold. At the same time, use the protocol analysis tool to capture and analyze the data packets in real time to observe the response status of the switch, such as whether there is packet loss, increased latency, etc.

[0058] For step A3 and step A4, the strategy output and execution monitoring process is as follows:

[0059] Output the formulated test strategy to the test execution system, including the startup instructions of the test tools in the tool library, parameter setting values, etc. During the test process, continuously monitor the test execution and receive feedback data from the test tools, such as test progress, intermediate results, etc. According to the feedback information, dynamically adjust the test strategy until the obtained test effect evaluation indicators meet the corresponding performance indicators, ensuring that the test process can proceed smoothly according to the expected goals.

[0060] For example, if during the test it is found that the switch behaves abnormally under a certain traffic pattern, but the current test strategy does not cover in-depth analysis of the abnormal situation, LLM can adjust the strategy based on real-time feedback and add detailed data collection and analysis steps for the abnormal traffic pattern, such as extending the data capture time and adjusting the filtering conditions of the analysis tool, in order to gain a more comprehensive understanding of the equipment performance issues.

[0061] After obtaining the trained large model using the above steps, for the specific target industrial control equipment to be tested, you only need to input the collected operating data, the test task requirements of the target industrial control equipment and the tool library into the large model to obtain the test plan.

[0062] In addition, the tool library includes both hardware-level tools and software-level tools;

[0063] Hardware-level tools include at least electrical parameter test instruments and signal analyzers;

[0064] Software-level tools include at least vulnerability scanning software, protocol analysis tools, and industrial control system performance monitoring software.

[0065] In this embodiment, the above-mentioned tools can detect and evaluate industrial control equipment from different dimensions, such as hardware reliability, software security, network communication stability, etc.

[0066] Regarding step 104 and step 106:

[0067] According to the test plan output by the big model, the performance test is carried out, and each tool will feed back the test progress, intermediate results and final results to the analysis module. After receiving the test data sent by each tool, the analysis module will analyze the quantitative results of various performance indicators of the industrial control equipment, the comprehensive performance evaluation results, potential risk points and targeted improvement suggestions based on the difference between the test data and the performance indicators of the corresponding test tasks, and generate a performance test report based on the analysis results. Users can carry out follow-up work based on this test report.

[0068] The following is an actual industrial control equipment performance test scenario to illustrate the specific testing process of this application method:

[0069] First, the industrial control equipment to be tested, such as a complex industrial automation control system, is properly placed in the specific industrial production environment simulated by the industrial control range platform, and test scenarios such as simulating malicious network intrusions are set through the scenario configuration interface of the range platform.

[0070] Then, various data acquisition devices of the industrial control range platform are quickly started to widely collect the initial hardware status of the control system (such as the temperature and voltage of each controller), software running status (such as the number of processes, memory usage, etc.) and initial network connection status (such as the list of connected devices, initial network traffic, etc.), and transmit these data to the trained LLM large model in real time.

[0071] Then, the LLM model intelligently selects network vulnerability detection tools, network traffic simulation attack tools, and business data integrity monitoring software from the tool library based on the received data and preset test tasks (such as evaluating the business continuity and data integrity of the control system when it suffers a specific type of network attack), and develops a test plan that first performs vulnerability scanning and assessment, then launches a simulated network attack and continuously monitors system performance and data integrity during the attack.

[0072] Then, the relevant tools in the tool library are started in sequence according to the plan. During the test, each tool transmits data such as vulnerability scanning results, network traffic changes during the attack, and error rate of business data to the analysis module in real time.

[0073] Finally, the analysis module accurately evaluates the performance of the industrial control equipment in simulated network attack scenarios based on the analysis logic provided by the LLM large model, for example by comparing the change curves of the key performance indicators of the system before and after the attack, analyzing the types and distribution of business data errors, etc., and finally generates a detailed test report that includes the performance degradation of the system when attacked, details of potential security vulnerabilities, and improvement suggestions for improving the system's anti-attack capabilities, providing a solid data basis and technical guidance for the safe and stable operation and optimization and upgrading of the industrial control equipment in the actual industrial environment.

[0074] like Figure 2 , Figure 3 As shown, the embodiment of the present invention provides a performance test device for industrial control equipment based on the LLM large model. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 2 As shown in FIG. 1 , a hardware architecture diagram of a computing device where a performance test device for industrial control equipment based on an LLM large model is located is provided in an embodiment of the present invention. Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown in the figure, the computing device in which the device is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 3 As shown, as a device in a logical sense, the CPU of the computing device in which it is located reads the corresponding computer program in the non-volatile memory into the internal memory and runs it.

[0075] Please refer to Figure 3 The embodiment of the present invention provides an LLM large model-based performance test device for industrial control equipment, which is applied to the performance test system of industrial control equipment. The system includes an industrial control range platform, a tool library, a pre-trained LLM large model and an analysis module; the large model communicates with the industrial control range platform, the tool library and the analysis module respectively, and the tool library communicates with the analysis module; the device includes:

[0076] The simulation unit 300 is used to simulate the test environment of the target industrial control equipment to be tested based on the industrial control range platform, and collect the operation data of the target industrial control equipment;

[0077] The input unit 302 is used to input the operation data, the test task requirements of the target industrial control equipment and the tool library into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool;

[0078] The testing unit 304 is used to perform performance testing using corresponding tools in sequence according to the testing plan, and send the test data of each tool to the analysis module;

[0079] The generating unit 306 is used to analyze the test data using the analyzing module to generate a performance test report for the target industrial control equipment.

[0080] In some embodiments, the LLM large model is trained in the following manner:

[0081] Construct a test task sample set, which includes multiple known test tasks. The test environment of industrial control equipment corresponding to each test task is built based on the industrial control range platform, and each test task is marked with test objectives, performance indicators and required tool types;

[0082] For each known test task, the operation data of the industrial control equipment corresponding to the test task is collected, and the operation data, the test task and the tool library are input into the big model; the big model performs semantic analysis on the test task and the operation data based on a preset algorithm to obtain the test target of the test task, and selects the type of tool required to execute the test task from the tool library based on the test target, and selects tools from each tool type, as well as the use order and test parameters of each tool, to generate a test plan;

[0083] According to the test plan, the relevant tools in the tool library are activated in order, and the performance test of the corresponding industrial control equipment is performed; the test data of each tool is fed back to the analysis module; the analysis module determines the test effect evaluation index of the test plan based on the test data and the corresponding performance indicators;

[0084] Based on the test effect evaluation index, the decision of the large model is rewarded or punished, and the decision strategy of the large model is adjusted based on the reward or punishment; and so on, until the task test is carried out according to the test plan output by the large model, and the obtained test effect evaluation index meets the corresponding performance index, and a trained large model is obtained.

[0085] In some implementations, the preset algorithms include a semantic understanding algorithm, a knowledge fusion algorithm, and a strategy generation algorithm;

[0086] The semantic understanding algorithm performs semantic analysis on the input test task description and the operation data of the industrial control equipment based on the neural network, converts the text data into vector representation, and identifies key information by calculating the similarity and semantic relationship between vectors; the key information includes the test target, equipment type, test scenario and performance index;

[0087] The knowledge fusion algorithm combines the knowledge graph related to industrial control equipment with the neural network, so that the large model can perform knowledge reasoning and decision optimization based on the entity relationship and attribute information in the knowledge graph and its own language understanding ability;

[0088] The strategy generation algorithm is based on the reinforcement learning algorithm framework. It determines the required tools according to the functional characteristics, scope of application and matching degree of each tool with the current industrial control equipment and test environment. Based on the knowledge fusion algorithm and combined with the knowledge graph related to the current industrial control equipment, it determines the usage order, parameter settings and collaborative working mode of each tool to generate a test plan. In response to the execution of the test plan, the decision-making strategy is adjusted based on the test effect evaluation index feedback from the analysis module, and the optimal test strategy is gradually learned.

[0089] In some embodiments, the large model uses a pre-trained language model based on the Transformer architecture.

[0090] In some implementations, the operating data of the industrial control equipment includes configuration information of the hardware device, initial state parameters of the software system, and network connection status.

[0091] In some embodiments, the tool library includes hardware-level tools and software-level tools;

[0092] Hardware-level tools include at least electrical parameter test instruments and signal analyzers;

[0093] Software-level tools include at least vulnerability scanning software, protocol analysis tools, and industrial control system performance monitoring software.

[0094] In some implementations, the generating unit 306 is configured to perform the following operations:

[0095] Based on the difference between the test data fed back by the test tool and the performance indicators of the corresponding test tasks, the analysis module is used to analyze the quantitative results of various performance indicators of industrial control equipment, comprehensive performance evaluation results, potential risk points and targeted improvement suggestions, and the analysis results are used to generate a performance test report.

[0096] It should be noted that the performance testing device for industrial control equipment based on the LLM large model provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the performance testing device for industrial control equipment based on the LLM large model provided in the above embodiment and the performance testing method embodiment of industrial control equipment based on the LLM large model belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0097] The embodiment of the present application also provides a computer device, please refer to Figure 3The computer device includes a processor and a memory, in which at least one instruction, at least one program, code set or instruction set is stored, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the performance testing method of industrial control equipment based on the LLM large model provided by the above-mentioned method embodiments.

[0098] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the performance testing method of industrial control equipment based on the LLM large model provided by the above-mentioned method embodiments.

[0099] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the performance testing method for industrial control equipment based on the LLM large model as described in any of the above embodiments.

[0100] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0101] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0102] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0103] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A performance testing method for industrial control equipment based on LLM large model, characterized in that: A performance test system for industrial control equipment, the system comprising an industrial control range platform, a tool library, a pre-trained LLM large model and an analysis module; the large model communicates with the industrial control range platform, the tool library and the analysis module respectively, and the tool library communicates with the analysis module; the method comprises: Simulating the test environment of the target industrial control equipment to be tested based on the industrial control range platform, and collecting the operating data of the target industrial control equipment; Input the operating data, the test task requirements of the target industrial control equipment and the tool library into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool; According to the test plan, use corresponding tools in turn to perform performance tests, and send the test data of each tool to the analysis module; The test data is analyzed by using the analysis module to generate a performance test report for the target industrial control equipment.

2. The method according to claim 1, characterized in that The LLM large model is trained in the following way: Construct a test task sample set, wherein the sample set includes a plurality of known test tasks, the test environment of the industrial control equipment corresponding to each of the test tasks is built based on the industrial control range platform, and each of the test tasks is annotated with the test objectives, performance indicators and required tool types; For each known test task, the operation data of the industrial control equipment corresponding to the test task is collected, and the operation data, the test task and the tool library are input into the large model; the large model performs semantic analysis on the test task and the operation data based on a preset algorithm to obtain the test target of the test task, and selects the type of tool required to execute the test task from the tool library based on the test target, and selects tools from each tool type, as well as the use order and test parameters of each tool, to generate a test plan; According to the test plan, the relevant tools in the tool library are activated in order, and the performance test of the corresponding industrial control equipment is performed; the test data of each tool is fed back to the analysis module; the analysis module determines the test effect evaluation index of the test plan based on the test data and the corresponding performance index; Based on the test effect evaluation index, the decision of the large model is rewarded or punished, and the decision strategy of the large model is adjusted based on the reward or punishment; and so on, until the task test is performed according to the test plan output by the large model, and the obtained test effect evaluation index meets the corresponding performance index, thus obtaining a trained large model.

3. The method according to claim 2, characterized in that The preset algorithms include semantic understanding algorithm, knowledge fusion algorithm and strategy generation algorithm; The semantic understanding algorithm performs semantic analysis on the input test task description and the operation data of the industrial control equipment based on the neural network, converts the text data into vector representation, and identifies key information by calculating the similarity and semantic relationship between the vectors; the key information includes the test target, equipment type, test scenario and performance index; The knowledge fusion algorithm combines the knowledge graph related to industrial control equipment with the neural network, so that the large model can perform knowledge reasoning and decision optimization based on the entity relationship and attribute information in the knowledge graph and combined with its own language understanding ability; The strategy generation algorithm is based on the reinforcement learning algorithm framework, and determines the required tools according to the functional characteristics, applicable scope and matching degree of each tool with the current industrial control equipment and test environment. Based on the knowledge fusion algorithm, combined with the knowledge graph related to the current industrial control equipment, the use order, parameter settings and collaborative working mode of each tool are determined to generate a test plan. In response to executing the test plan, the decision strategy is adjusted based on the test effect evaluation index fed back by the analysis module, and the optimal test strategy is gradually learned.

4. The method according to claim 1, characterized in that: The large model uses a pre-trained language model based on the Transformer architecture.

5. The method according to claim 1, characterized in that The operation data of the industrial control equipment includes configuration information of hardware devices, initial state parameters of the software system and network connection status.

6. The method according to claim 1, characterized in that The tool library includes tools at the hardware level and tools at the software level; The hardware-level tools include at least an electrical parameter tester and a signal analyzer; The software-level tools include at least vulnerability scanning software, protocol analysis tools, and industrial control system performance monitoring software.

7. The method according to claim 1, characterized in that The analyzing the test data by using the analysis module to generate a performance test report for the target industrial control equipment includes: Based on the difference between the test data fed back by the test tool and the performance indicators of the corresponding test tasks, the analysis module is used to analyze the quantitative results of various performance indicators of the industrial control equipment, the comprehensive performance evaluation results, the potential risk points and targeted improvement suggestions, and the analysis results are used to generate a performance test report.

8. A performance testing device for industrial control equipment based on the LLM large model, characterized in that: A performance test system for industrial control equipment, the system comprising an industrial control range platform, a tool library, a pre-trained LLM large model and an analysis module; the large model communicates with the industrial control range platform, the tool library and the analysis module respectively, and the tool library communicates with the analysis module; the device comprises: A simulation unit, used to simulate the test environment of the target industrial control equipment to be tested based on the industrial control range platform, and collect the operation data of the target industrial control equipment; An input unit, used to input the operating data, the test task requirements of the target industrial control equipment and the tool library into the pre-trained large model to obtain a test plan; the test plan includes the tools required to complete the test task and the test sequence and test parameters of each tool; A testing unit, used to perform performance testing using corresponding tools in sequence according to the testing scheme, and send the test data of each tool to the analysis module; A generating unit is used to analyze the test data using the analyzing module to generate a performance test report for the target industrial control equipment.

9. A computing device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images