Information communication test instrument and method and storage medium

Abstract

The embodiment of the invention provides an information communication test instrument and method and a storage medium. The method comprises the following steps: establishing an integrated system comprising an assistant module, an MCP server module, a local knowledge base module, a test module and a report generation module, establishing a communication bridge with the assistant module through the MCP server module, and providing domain knowledge support for the assistant module by utilizing the local knowledge base module, and then receiving a user natural language task through the assistant module and generating a test case, executing a test corresponding to the test case through the test module and collecting a test result, and finally analyzing the test result through the report generation module and generating a test report. The method is used for achieving the effect of improving the test efficiency and the test accuracy.

Description

Technical Field

[0001] This application relates to the field of intelligent testing equipment technology, and in particular to an information communication testing instrument, method and storage medium. Background Technology

[0002] With the rapid development of emerging technologies such as intelligent computing networks, the complexity of network protocols in network communication equipment is increasing, and the testing requirements for network communication equipment are also constantly being updated.

[0003] Existing network testing methods typically rely on pre-defined rule templates and manually written automated scripts. For example, test steps are defined manually in the script to control the target instrument to perform protocol compatibility verification or performance testing.

[0004] However, existing technologies struggle to dynamically respond to real-time test data. When encountering non-standard protocols or abnormal events, manual intervention is required, reducing testing efficiency and failing to meet testing needs in complex scenarios. Summary of the Invention

[0005] This application provides information communication testing instruments, methods, and storage media to improve testing efficiency and enhance adaptability to complex scenarios.

[0006] In a first aspect, embodiments of this application provide an information communication testing instrument, including:

[0007] The assistant module is used to receive natural language tasks from users and generate test cases based on the natural language tasks.

[0008] The MCP server module communicates with the assistant module and is used to provide model context protocol services to the assistant module.

[0009] The local knowledge base module stores network protocol documents, test cases, and instrument operation information to provide domain knowledge support to the assistant module;

[0010] The testing module, connected to the MCP server module, is used to control the testing instruments to execute testing tasks according to the test instructions corresponding to the test cases, and to collect test results.

[0011] The report generation module is used to analyze the test results and generate test reports.

[0012] Optionally, the assistant module includes:

[0013] The intent recognition unit is used to recognize the intent of the user's natural language input task and obtain the test intent;

[0014] The dialogue management unit is used to maintain contextual dialogue with the user;

[0015] The test case generation unit is used to generate test cases based on the test intent.

[0016] Optionally, the test case generation unit includes:

[0017] The task parsing subunit is used to break down a natural language task into test steps;

[0018] The test case combination subunit is used to select matching test cases from the test cases stored in the local knowledge base module according to the test steps, and to obtain test cases by combining the matching test cases.

[0019] Optionally, the MCP server module includes:

[0020] The tool interface unit is used to provide standardized tool call interfaces, enabling the large language model in the assistant module to call the functions of the test module;

[0021] The protocol adapter unit supports multiple test instrument control protocols, enabling communication between the MCP server module and various test instruments.

[0022] Optionally, the local knowledge base module includes:

[0023] The vector database unit is used for vectorized storage and semantic retrieval of network protocol documents, test cases, and instrument operation information;

[0024] The RAG enhancement unit is used to perform knowledge retrieval based on the vector database unit when the assistant module generates test cases, and then send the retrieved knowledge to the assistant module.

[0025] Optionally, the test module includes:

[0026] The test task orchestration unit is used to orchestrate the test process based on the test instructions generated by the assistant module.

[0027] The execution monitoring unit is used to monitor the status of the test module when it executes test tasks;

[0028] The results collection unit is used to collect the test results generated by the test instruments during the test process.

[0029] Optionally, the report generation module includes:

[0030] The data analysis unit is used to perform multi-dimensional analysis on the test results collected by the results collection unit and obtain analytical conclusions.

[0031] The report synthesis unit is used to generate test reports based on the analysis conclusions. The test reports include problem descriptions and analysis conclusions.

[0032] Optionally, it also includes a cloud testing large model module for storing supervised fine-tuned domain-specific large language models for processing natural language tasks when invoked by the assistant module;

[0033] The prompt word management module stores prompt word templates corresponding to multiple test scenarios. When processing tasks, the assistant module calls the prompt word template corresponding to the current test scenario to guide the large language model.

[0034] Secondly, embodiments of this application provide an information communication testing method, including:

[0035] It receives natural language tasks input by users and generates test cases based on these tasks. When generating test cases, it provides model context protocol services through the MCP server module and provides domain knowledge support by storing network protocol documents, test cases, and instrument operation information in the local knowledge base module.

[0036] Control the testing instruments to execute testing tasks according to the test instructions corresponding to the test cases, and collect the test results;

[0037] Analyze the test results and generate a test report.

[0038] Optionally, maintain a contextual dialogue with the user to perform intent recognition on the natural language task inputted by the user and obtain the test intent;

[0039] Test cases are generated based on the test intent.

[0040] Optionally, the natural language task can be broken down into test steps;

[0041] According to the testing steps, matching test cases are selected from the test cases stored in the local knowledge base module. By combining the matching test cases, test cases are obtained.

[0042] Optionally, by providing standardized tool call interfaces, the large language model in the assistant module can call the functions of the test module;

[0043] By supporting multiple test instrument control protocols, the MCP server module enables communication between various test instruments.

[0044] Optionally, the network protocol documents, test cases, and instrument operation information are stored in a vectorized manner and semantically retrieved; when the assistant module generates test cases, knowledge retrieval is performed based on the vector database unit, and the retrieved knowledge is sent to the assistant module.

[0045] Optionally, the test process can be arranged according to the test instructions generated by the assistant module;

[0046] The status of the monitoring test module when it executes test tasks;

[0047] The test results generated by the testing instruments during the testing process are collected.

[0048] Optionally, the collected test results can be analyzed in multiple dimensions to obtain analytical conclusions;

[0049] A test report is generated based on the analysis results. The test report includes a problem description and analysis results.

[0050] Optionally, a domain-specific large language model, after supervised fine-tuning, can be stored to process natural language tasks when invoked by the assistant module.

[0051] The system stores prompt word templates corresponding to multiple test scenarios. When processing tasks, the assistant module calls the prompt word template corresponding to the current test scenario to guide the large language model.

[0052] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the second aspect and / or various possible implementations of the second aspect.

[0053] The information communication testing instruments, methods, and storage media provided in this application construct an integrated system including an assistant module, an MCP server module, a local knowledge base module, a testing module, and a report generation module. The MCP server module establishes a communication bridge with the assistant module, and the local knowledge base module provides domain knowledge support to the assistant module. The assistant module then receives user natural language tasks and generates test cases. The testing module executes the tests corresponding to these test cases and collects the test results. Finally, the report generation module analyzes the test results and generates a test report. This process achieves closed-loop control from understanding requirements to report generation, effectively solving the technical problems of traditional testing methods, such as reliance on manually written scripts, low testing efficiency, and poor adaptability to complex scenarios. This results in improved testing efficiency and accuracy. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] Figure 1 A schematic diagram of the structure of the information and communication test instrument provided in this application;

[0056] Figure 2 Flowchart of the information communication testing method provided in this application Figure 1 ;

[0057] Figure 3 Flowchart of the information communication testing method provided in this application Figure 2 ;

[0058] Figure 4 Flowchart of the information communication testing method provided in this application Figure 3 ;

[0059] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0060] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0062] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation portals for users to choose to authorize or refuse.

[0063] The information and communication testing instruments involved in this application are mainly used in the automated testing scenarios of network equipment (such as 5G base stations, SDN switches, NFV gateways, etc.) in emerging fields of rapid iteration of communication technology (such as intelligent computing networks, edge computing, industrial internet, etc.).

[0064] In traditional network testing, testers need to manually write corresponding test scripts based on current requirements and execute tasks such as protocol compatibility verification, performance stress testing (e.g., RFC2544 standard testing), and fault simulation. For example, testers need to manually write SCPI / TCL scripts according to RFC standards or vendor documents, defining test steps one by one (e.g., sending specific protocol messages, setting parameters). However, with the rapid increase in network protocol complexity and the continuous updating of testing requirements (e.g., the need to adjust test parameters according to real-time network conditions), current requirements are highly scenario-based (e.g., simulating network behavior under gigabit-level concurrent connections). In the above-mentioned traditional network testing methods, manually writing scripts takes a long time, easily leads to poor reusability of test cases, prolongs the testing cycle, and is difficult to cope with the testing requirements of complex scenarios such as multiple protocol combinations.

[0065] Existing technologies also execute tests through preset flowcharts or state machine control. However, this method relies on preset rule templates, making it difficult to dynamically adjust test parameters (e.g., throughput rate, concurrent connections) based on the device's real-time operating status (e.g., CPU load, memory usage). For example, when abnormal data occurs during testing (e.g., sudden changes in packet loss rate, latency jitter), it is difficult to adjust the test strategy in real time, reducing dynamic adaptability and potentially leading to deviations between test results and actual scenarios, thus lowering the accuracy and reliability of the test results. Furthermore, when using existing testing methods, if non-standard protocols or abnormal events (e.g., unknown error codes) are encountered, relevant testers need to consult documentation or rely on experience to diagnose the fault. This method not only requires testers to have a deep understanding of the relevant protocols and instrument operating logic but also has a high probability of misjudgment and can easily prolong fault location time. After testing, existing technologies output raw data (e.g., throughput, latency histograms), which require further secondary analysis by testers, ultimately presenting the final analysis results in numerical form. This method is difficult to accurately and quickly locate faults when problems exist. Furthermore, since many manufacturers' test instruments use independent control interfaces, existing technologies still require the development of dedicated drivers for each instrument, which reduces test scalability, increases test costs, and reduces test efficiency.

[0066] Based on the above scenarios, it is clear that existing technologies are insufficient to meet the needs of intelligent testing of communication equipment in complex scenarios, resulting in low testing efficiency and accuracy, as well as technical issues related to insufficient adaptability.

[0067] The information communication testing instrument provided in this application receives natural language tasks input by the user through an assistant module and generates test cases based on the natural language tasks. It communicates with the assistant module through an MCP (Model Context Protocol) server module to provide model context protocol services to the assistant module. A local knowledge base module stores network protocol documents, test cases, and instrument operation information to provide domain knowledge support to the assistant module. A testing module connects to the MCP server module to control the testing instrument to execute test tasks according to the test instructions corresponding to the test cases and collect test results. A report generation module analyzes the test results and generates test reports. This achieves closed-loop control throughout the entire process from understanding requirements to report generation, effectively solving the technical problems of traditional testing methods, such as reliance on manually written scripts, low testing efficiency, and poor adaptability to complex scenarios, thereby improving testing efficiency and accuracy.

[0068] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0069] Figure 1 A schematic diagram of the structure of the information and communication test instrument provided in this application is shown below. Figure 1 As shown, the information and communication testing instrument includes an assistant module, an MCP server module, a local knowledge base module, a testing module, and a report generation module. The assistant module receives natural language tasks input by the user and generates test cases based on these tasks. The MCP server module communicates with the assistant module and provides model context protocol services. The local knowledge base module stores network protocol documents, test cases, and instrument operation information to provide domain knowledge support to the assistant module. The testing module, connected to the MCP server module, controls the testing instrument to execute testing tasks according to the test instructions corresponding to the test cases and collects the test results. The report generation module analyzes the test results and generates a test report.

[0070] For example, the assistant module obtains the natural language task input by the user (e.g., "Please perform throughput testing on device A"), and performs semantic understanding and task planning processing on the data corresponding to the natural language task, ultimately obtaining a structured test case (e.g., a test script conforming to the target protocol). This test case is then passed to the test module, enabling the test module to control the test instrument to execute the specific test task corresponding to the natural language task.

[0071] In one possible embodiment, the user-input natural language task "Test the throughput of router X" is obtained through an assistant module. This assistant module then processes the natural language task to generate data corresponding to an executable test case (e.g., a test script including parameters such as packet transmission rate, frame length, and test duration). The data corresponding to the test case is then sent to the test module. Upon receiving the test case data, the test module controls the testing instrument to execute the corresponding throughput test.

[0072] This embodiment constructs an integrated test instrumentation system architecture consisting of an assistant module, an MCP server module, a local knowledge base module, a testing module, and a report generation module, realizing intelligent testing throughout the entire process from natural language task input to test report generation. This modular integration combines decision-making, protocol services, knowledge support, test execution, and result analysis, improving the automation level of information and communication testing, reducing reliance on manually written test scripts and parsed test results, and ultimately enhancing the efficiency of information and communication testing.

[0073] Optionally, after generating test cases, the assistant module generates corresponding test instructions based on the generated test cases (e.g., start executing test case A).

[0074] Optionally, the MCP server module includes a tool interface unit and a protocol adaptation unit. The tool interface unit provides a standardized tool call interface, enabling the large language model in the assistant module to call the functions of the test module. The protocol adaptation unit supports multiple test instrument control protocols, enabling communication between the MCP server module and various test instruments.

[0075] For example, a model context protocol service refers to a communication framework that allows large language models to securely and structurally invoke external tools (such as test modules). A standardized tool invocation interface refers to a predefined list of functions or APIs that can be understood and used by large language models.

[0076] In one possible embodiment, the assistant module obtains the internally generated test instructions and converts them into standardized call requests recognizable by the test module via the tool interface unit of the MCP server module. Then, the protocol adaptation unit obtains the standardized call request and converts it into a command string conforming to the communication protocol supported by the target test instrument. Finally, the obtained command string is sent to the target test instrument to control it to start the test.

[0077] This embodiment standardizes the calling method between the large language model and the test module through the tool interface unit and protocol adaptation unit in the MCP server module, enhances the communication stability between the assistant module and the test instrument, improves the communication adaptability with various test instruments, solves the compatibility problem between different communication protocols and different test instruments, and enhances the scalability and applicability of the information communication test instrument.

[0078] Optionally, the local knowledge base module includes a vector database unit and a RAG enhancement unit. The vector database unit is used for vectorized storage and semantic retrieval of network protocol documents, test cases, and instrument operation information; the RAG enhancement unit is used to perform knowledge retrieval based on the vector database unit when the assistant module generates test cases, and then sends the retrieved knowledge to the assistant module.

[0079] For example, vectorized storage refers to the technique of converting text-formatted knowledge documents (such as network protocol documents, test cases, and arbitrary documents in instrument operation information) into numerical vectors through an embedding model. Semantic retrieval refers to the technique of querying the document fragment with the highest semantic similarity to any given statement by calculating the similarity between different numerical vectors. The RAG enhancement unit is a component that performs retrieval enhancement generation techniques, and it performs real-time retrieval during the test case generation process.

[0080] In one possible embodiment, when the assistant module receives the user's input natural language task "Perform RFC 6349 testing" and begins generating test cases, the RAG enhancement unit retrieves the content of the natural language task and sends it as a query request to the vector database unit for semantic retrieval processing. Then, the vector database unit returns document fragments related to the "RFC 6349" standard (e.g., test methods, parameter settings), which are then used as retrieved knowledge and sent to the assistant module. The assistant module uses this knowledge to generate test cases conforming to the RFC 6349 standard.

[0081] This embodiment combines vector databases with retrieval-enhanced generation technology to enable the local knowledge base to perform semantic retrieval and proactive knowledge push, thereby improving the accuracy and efficiency of the knowledge provided to the assistant module, effectively reducing errors that may occur in large models in professional testing fields, and improving the accuracy and professionalism of generated test cases and decisions.

[0082] Optionally, the assistant module includes an intent recognition unit, a dialogue management unit, and a test case generation unit. The intent recognition unit is used to recognize the intent of the user's natural language input task to obtain the test intent; the dialogue management unit is used to maintain the contextual dialogue with the user; and the test case generation unit is used to generate test cases based on the test intent.

[0083] For example, a test intent refers to a specific test objective abstracted from the natural language task input by the user, such as "perform performance testing" or "diagnose connection failures." The dialogue management unit stores the multi-turn dialogue history between the user and the assistant module to ensure that the assistant module can understand the context of previous dialogues when processing subsequent natural language tasks input by the user, thereby handling complex test requirements that require multiple interactions to clarify.

[0084] In one possible embodiment, the user first inputs "Perform a performance test on device C". The intent recognition unit processes this statement to obtain a preliminary test intent (e.g., "Perform a performance test"). Because this intent is not specific enough, the test case generation unit cannot immediately generate a complete test case. At this point, the assistant module can ask the user, "Please specify whether you need to test throughput or latency?" The user subsequently inputs "Test throughput", which is then sent to the dialogue management unit. The dialogue management unit associates this input with the previously stored dialogue history (i.e., "Perform a performance test on device C") to form a complete contextual dialogue. Afterward, the intent recognition unit performs intent recognition processing again based on the complete contextual dialogue to obtain a precise test intent (e.g., "Perform a throughput test on device C"). This test intent is used to generate a throughput test case.

[0085] This embodiment clarifies the internal processing flow from understanding user intent to generating test cases by constructing an assistant module that includes intent recognition, dialogue management, and test case generation functions. It also improves the accuracy of converting user-input natural language tasks into executable test logic, which helps to improve the accuracy of test results and the reliability of task execution.

[0086] Optionally, the test case generation unit includes a task parsing subunit and a test case combination subunit. The task parsing subunit is used to decompose the natural language task into test steps; the test case combination subunit is used to select matching test cases from the test cases stored in the local knowledge base module according to the test steps, and to obtain test cases by combining the matching test cases.

[0087] For example, a test step refers to an atomic operation obtained after decomposing a macro-level task, such as "configure traffic model", "set test duration", and "start test". A matching test case refers to a basic test case script that corresponds semantically and / or functionally to the test step and is pre-stored in the local knowledge base module.

[0088] In one possible embodiment, the task parsing subunit obtains the test intent (e.g., "perform throughput and latency tests") from the intent recognition unit, and decomposes the test intent to obtain two test steps: "execute throughput test" and "execute latency test". Then, the test case combination subunit obtains these test steps and retrieves test cases (e.g., two preset standardized test scripts) matching "throughput test" and "latency test" respectively from the vector database unit of the local knowledge base module. The test case combination subunit combines these two test scripts to obtain a single test case that can simultaneously measure throughput and latency, which is then used to generate the final test instructions.

[0089] This embodiment decomposes the test intent into multiple test steps through task parsing, selects pre-stored content related to each test step from the local knowledge base module, and combines the pre-stored content to obtain the final test case. This realizes the automated combination and generation of test cases, avoids writing them from scratch, improves testing efficiency, and enhances the professionalism and reusability of test cases.

[0090] Optionally, the information and communication testing instrument also includes a cloud-based large-scale model module and a prompt word management module. The cloud-based large-scale model module stores supervised fine-tuning of a professional domain-specific large language model for processing natural language tasks when invoked by the assistant module. The prompt word management module stores prompt word templates corresponding to multiple test scenarios. When processing tasks, the assistant module invokes the prompt word template corresponding to the current test scenario to guide the large language model.

[0091] For example, a supervised fine-tuned domain-specific large language model refers to a specialized model obtained by additionally training a general large language model using specialized question-and-answer data from the communication testing domain. The prompt word template is a predefined text framework, including but not limited to instructions, context, and formatting requirements specific to the testing scenario, used to guide the large language model in generating professional and standardized responses.

[0092] In one possible embodiment, when the intent recognition unit of the assistant module needs to process a user-input natural language task, the intent recognition unit first obtains a prompt word template corresponding to the current scenario (e.g., a protocol conformance test scenario) from the prompt word management module. Based on the prompt word template and the user's original input natural language task, a structured prompt is generated. Subsequently, the assistant module calls the professional domain large language model in the cloud testing large model module and inputs the structured prompt into the professional domain large language model for inference processing. The professional domain large language model outputs a more accurate and professional intermediate result (e.g., identifying the user's intent as "perform RFC 2544 conformance test"), so that the test case generation unit of the assistant module can generate the final test cases based on the intermediate result.

[0093] This embodiment optimizes the assistant module's performance in handling specialized tasks by introducing a dedicated cloud-based large-scale testing model module and a prompt word management module, addressing both model capabilities and interactive guidance. By using a domain-fine-tuned professional large-scale language model and scenario-specific prompt words, the rationality of test instruction planning, as well as the accuracy and standardization of output results, are improved.

[0094] Optionally, the testing module includes a test task orchestration unit, an execution monitoring unit, and a result collection unit. The test task orchestration unit is used to orchestrate the test process based on the test instructions generated by the assistant module; the execution monitoring unit is used to monitor the status of the testing module during test task execution; and the result collection unit is used to collect the test results generated by the testing instruments during the testing process.

[0095] This embodiment realizes automated scheduling and real-time status monitoring of the test process, improves the orderliness and reliability of test task execution, and enhances the integrity of capturing key data during the test process, providing a foundation for subsequent analysis.

[0096] Optionally, the report generation module includes a data analysis unit and a report synthesis unit. The data analysis unit performs multi-dimensional analysis on the test results collected by the results collection unit to obtain analytical conclusions; the report synthesis unit generates a test report based on the analytical conclusions, the test report including a problem description and analytical conclusions.

[0097] For example, test results are raw data generated by the test instrument and collected by the result collection unit, such as throughput values, latency statistics, packet loss rate, etc. Analysis conclusions are judgmental statements derived from interpreting the raw test results, such as whether the device performance is substandard or whether packet loss occurs at a frame length of 1518 bytes.

[0098] In one possible embodiment, the test task orchestration unit of the test module obtains test cases (e.g., a complete performance test script) generated by the assistant module, and orchestrates specific test flows based on these test cases to obtain test instructions. The test instrument executes the test flow based on the test instructions and generates raw test results. The result collection unit obtains these raw test results and sends them to the report generation module. The data analysis unit of the report generation module performs multi-dimensional statistical analysis on the raw test results (e.g., calculates the average and maximum values, and compares them with corresponding preset thresholds) to obtain analysis conclusions (e.g., average latency exceeds the threshold). The report synthesis unit obtains these analysis conclusions, organizes and formats them, and generates a final test report containing a problem description (e.g., high latency was observed during the test) and analysis conclusions.

[0099] This embodiment enables the generated report to not only present data, but also directly point out problems and provide analytical conclusions, thereby improving the readability of test results and the reliability of decision support.

[0100] In one possible embodiment, Figure 2 Flowchart of the information communication testing method provided in this application Figure 1 ,like Figure 2 As shown, steps 1-6 constitute the data entry process before information communication testing. Specifically, this includes: unstructured processing of local documents to obtain text data; data slicing of the text data to obtain text blocks; and vectorization of the text blocks to obtain corresponding vectorized knowledge documents, which are then stored in the vector database. Steps 7-10 constitute the problem relevance retrieval steps during information communication testing. Specifically, this includes: obtaining the user's input natural language task (i.e., ...) through the assistant module. Figure 2 The "question" in the document is obtained through the RAG augmentation unit, which performs vectorization encoding on the natural language task to obtain the "question vector". The RAG augmentation unit then sends the "question vector" to a vector database, where it performs similarity matching on stored vectorized knowledge documents to retrieve paragraphs relevant to the natural language task (i.e., ...). Figure 2 (The "Related Paragraph Context" in the text). Figure 2 Steps 11-14 are the output steps of the large model, specifically including: combining the retrieved relevant paragraphs with the original natural language task through the RAG enhancement unit to obtain a prompt word that includes domain context information (i.e., Figure 2 The prompt word ("problem + context") is then output to the assistant module. The assistant module then retrieves this prompt word and performs inference processing based on its internal large language model, ultimately generating test cases (i.e.,...). Figure 2 The answer in the text is output.

[0101] In one possible embodiment, Figure 3 Flowchart of the information communication testing method provided in this application Figure 2 ,like Figure 3 As shown, the prompt word management module stores prompt word templates. These templates include explicit "instructions" and optional "output indicators" to define the target task and target output format. When processing a user's natural language task, the assistant module retrieves a prompt word template matching the current test scenario. Then, the assistant module processes the user's natural language task input (i.e.,...) Figure 3 The "data" in the code is combined with the prompt word template to obtain a complete instruction prompt word. The assistant module calls the professional domain-specific large language model stored in the cloud testing large model module, and inputs the complete instruction prompt word into this large language model (LLM). The large language model then uses its internally supervised and fine-tuned professional knowledge to infer and process the instruction prompt word, generating and outputting text that meets the instruction requirements.

[0102] In one possible embodiment, Figure 4 Flowchart of the information communication testing method provided in this application Figure 3 ,like Figure 4 As shown, supervised fine-tuning is first selected as the model tuning method. A supervised fine-tuning dataset is constructed based on fundamental knowledge of data communication protocols and the network tester's help documentation, and the necessary hyperparameters for training are configured. The open-source large language model is trained using the supervised fine-tuning dataset and the configured hyperparameters. During training, mixed training data can be added to enhance the model's generalization ability. After training, the model is deployed, and its performance is scored to determine if it meets the preset value. If the score does not meet the preset value, the training data is adjusted (and / or optimized, tuning method changed), and training is re-run. If the preset value is met, model tuning is complete, and the final model is deployed in the cloud-based large model module for the assistant module to use.

[0103] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0104] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0105] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0106] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0107] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0108] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0109] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0110] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0111] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0112] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0113] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0114] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0116] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0118] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An information communication testing instrument, characterized in that, include: The assistant module is used to receive natural language tasks input by the user and generate test cases based on the natural language tasks. The MCP server module communicates with the assistant module and is used to provide model context protocol services to the assistant module. The local knowledge base module is used to store network protocol documents, test cases, and instrument operation information to provide domain knowledge support to the assistant module; The testing module, connected to the MCP server module, is used to control the testing instruments to execute testing tasks according to the test instructions corresponding to the test cases, and to collect test results. The report generation module is used to analyze the test results and generate a test report.

2. The instrument according to claim 1, characterized in that, The assistant module includes: The intent recognition unit is used to recognize the intent of the user's natural language input task and obtain the test intent; The dialogue management unit is used to maintain contextual dialogue with the user; The test case generation unit is used to generate test cases based on the test intent.

3. The instrument according to claim 2, characterized in that, The test case generation unit includes: The task parsing subunit is used to decompose the natural language task into test steps; The test case combination subunit is used to select matching test cases from the test cases stored in the local knowledge base module according to the test steps, and to obtain test cases by combining the matching test cases.

4. The instrument according to claim 1, characterized in that, The MCP server module includes: The tool interface unit is used to provide a standardized tool call interface, enabling the large language model in the assistant module to call the functions of the test module; The protocol adaptation unit is used to support multiple test instrument control protocols, so that the MCP server module can communicate with multiple test instruments.

5. The instrument according to claim 1, characterized in that, The local knowledge base module includes: The vector database unit is used for vectorized storage and semantic retrieval of the network protocol documents, test cases, and instrument operation information; The RAG enhancement unit is used to perform knowledge retrieval based on the vector database unit when the assistant module generates test cases, and send the retrieved knowledge to the assistant module.

6. The instrument according to claim 1, characterized in that, The test module includes: The test task orchestration unit is used to orchestrate the test process according to the test instructions generated by the assistant module. An execution monitoring unit is used to monitor the status of the test module when it executes test tasks; The results collection unit is used to collect the test results generated by the test instruments during the test process.

7. The instrument according to claim 1, characterized in that, The report generation module includes: The data analysis unit is used to perform multi-dimensional analysis on the test results collected by the results collection unit and obtain analytical conclusions. The report synthesis unit is used to generate a test report based on the analysis conclusions, the test report including a problem description and analysis conclusions.

8. The instrument according to claim 1, characterized in that, Also includes: The cloud testing large model module is used to store supervised fine-tuning professional domain large language models for processing the natural language task when called by the assistant module; The prompt word management module is used to store prompt word templates corresponding to multiple test scenarios. When processing tasks, the assistant module calls the prompt word template corresponding to the current test scenario to guide the large language model.

9. An information communication testing method, characterized in that, include: The system receives natural language tasks input by the user and generates test cases based on these tasks. When generating test cases, the system provides model context protocol services through the MCP server module and provides domain knowledge support by storing network protocol documents, test cases, and instrument operation information in the local knowledge base module. The test instrument is controlled to execute the test task according to the test instructions corresponding to the test case, and the test results are collected. The test results are analyzed and a test report is generated.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in claim 9.

Images