A communication protocol test evaluation method and device based on multi-agent cooperation

Abstract

The application provides a communication protocol test evaluation method and device based on multi-agent cooperation, relates to the technical field of communication protocol testing, and comprises the following steps: inputting a communication protocol test text into a document preprocessing unit to obtain a text block; inputting the text block into a semantic retrieval unit to obtain protocol core data; inputting the protocol core data into a test point design unit to obtain an initial test point set; inputting the initial test point set into a test point optimization unit to obtain a final test point set; inputting the final test point set into a test case generation unit to obtain an initial test case set; inputting the initial test case set into a test case sorting unit to obtain a test mind map, sorting the initial test case set according to the test mind map, and obtaining a final test case list; performing communication protocol testing according to the final test case list to obtain a communication protocol test result; and inputting the communication protocol test result into an expert evaluation model to obtain a communication protocol test evaluation result.

Description

Technical Field

[0001] This application relates to the field of communication protocol testing technology, and in particular to a communication protocol testing and evaluation method and apparatus based on multi-agent collaboration. Background Technology

[0002] As the core rules for data interaction between devices in a communication system, the functional integrity, performance stability, compatibility, and fault tolerance of communication protocols directly determine the overall operational quality of the communication system. Therefore, the testing and evaluation of communication protocols is a crucial aspect of the communication technology field. With the rapid development of communication technology, the types of communication protocols are constantly increasing, and their functions are becoming more complex, placing higher demands on the comprehensiveness, accuracy, and efficiency of testing and evaluation.

[0003] However, existing communication protocol testing and evaluation methods require separate design of test procedures, test points, and test cases for different types of communication protocols, making it difficult to quickly adapt to diverse protocol testing needs and lacking universality and scalability. Furthermore, test evaluation results often rely on manual judgment, making it difficult to generate quantitative evaluation conclusions. These shortcomings lead to inaccurate evaluations in communication protocol testing, reducing the reliability of communication protocols in communication systems and making communication systems prone to problems such as communication interruptions and data loss after deployment. Summary of the Invention

[0004] To address the aforementioned problems, in a first aspect, the present invention provides a method for testing and evaluating communication protocols based on multi-agent collaboration, comprising: A test requirement analysis intelligent agent is constructed, which includes a document preprocessing unit and a semantic retrieval unit. The communication protocol test text is obtained, and the communication protocol test text is input into the document preprocessing unit for preprocessing to obtain text blocks. The text blocks are input into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain the core protocol data. A test point generation agent is constructed, which includes a test point design unit and a test point optimization unit. The core protocol data is input into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set. The initial test point set is then input into the test point optimization unit for classification, deduplication, and missing point supplementation to obtain the final test point set. A test case generation agent is constructed, which includes a test case generation unit and a test case sorting unit. The final set of test points is input into the test case generation unit to transform test cases and obtain an initial set of test cases. The initial set of test cases is input into the test case sorting unit to perform correlation analysis between test cases and obtain a test mind map. The initial set of test cases is sorted according to the test mind map to obtain the final list of test cases. Based on the final test case list, conduct communication protocol tests to obtain communication protocol test results; construct an expert evaluation model, input the communication protocol test results into the expert evaluation model for test result grading evaluation, and obtain communication protocol test evaluation results.

[0005] Optionally, the step of preprocessing the communication protocol test text input document preprocessing unit to obtain text blocks includes: The communication protocol test text is input into the document preprocessing unit, and text denoising, term tagging, invalid information filtering, and sentence segmentation are performed sequentially to obtain semantic blocks. Obtain the hierarchical logic corresponding to the communication protocol test text, and perform structured division of semantic blocks according to the hierarchical logic to obtain text blocks identified by the hierarchical logic.

[0006] Optionally, the step of inputting text blocks into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain core protocol data includes: The text block is input into the semantic retrieval unit for feature extraction to obtain the semantic features corresponding to the text block; Obtain a knowledge graph of communication protocols, build a basic index library based on the knowledge graph of communication protocols, associate and map semantic features with the basic index library, and generate knowledge index data; The protocol database is obtained, and vector similarity is performed in the protocol database based on the knowledge index data. The retrieved data with a vector similarity greater than a first preset threshold with the knowledge index data is filtered out. The retrieved data is deduplicated to obtain the core protocol data, which includes protocol key field definition data, functional constraint data, interaction process specification data, and exception handling requirement data.

[0007] Optionally, the step of inputting the core protocol data into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set includes: Input the core protocol data into the test point design unit to identify the protocol type and obtain the communication protocol type; Obtain the test dimensions corresponding to the communication protocol type. The test dimensions include functional implementation dimension, performance indicator dimension, compatibility dimension and exception tolerance dimension. Call the test point design template corresponding to the test dimension. Based on the protocol's key field definition data, functional constraint data, interaction process specification data, and exception handling requirements data in the core protocol data, test points are constructed on the basis of the test point design template to obtain an initial set of test points.

[0008] Optionally, the step of inputting the initial test point set into the test point optimization unit for classification, deduplication, and missing point supplementation to obtain the final test point set includes: Obtain the test dimensions corresponding to the initial test point set. The test point optimization unit classifies the initial test point set according to the test dimensions to obtain test point groups. The test point groups include functional test group, performance test group, compatibility test group, and exception test group. The similarity matching deduplication algorithm is used to calculate the text similarity between test points in the same test point group in the initial test point set, and remove duplicate test points with text similarity greater than the second preset threshold to obtain the first test point set. Obtain the functional constraint data from the core protocol data, perform missing detection on the test points in the first test point set based on the functional constraint data, and obtain the missing detection points; reconstruct the missing detection points to obtain the final test point set.

[0009] Optionally, the step of inputting the final test point set into the test case generation unit for test case conversion to obtain the initial test case set includes: The test case generation unit sets the mapping relationship between test points and test cases, and calls the test case templates for each test point in the final test point set according to the mapping relationship; Obtain the core protocol data corresponding to each test point in the final test point set. Based on the protocol key field definition data, functional constraint data, interaction process specification data, and exception handling requirement data in the core protocol data, construct test cases on the basis of the test case template to obtain the initial test case set.

[0010] Optionally, the step of inputting the initial test case set into the test case sorting unit for correlation analysis between test cases to obtain a test mind map, and sorting the initial test case set according to the test mind map to obtain the final test case list includes: The core attributes of each test case in the initial test case set are extracted by the test case sorting unit. The core attributes include test objective, test object and prerequisite dependency conditions. Based on the core attributes, perform correlation analysis on each test case to obtain correlation analysis results. The correlation analysis results include sequential execution correlation, same object test correlation, and abnormal dependency correlation. By using test cases as nodes and correlation analysis results as connections between nodes, a test mind map is constructed. Based on the test mind map, the test cases in the initial test case set are divided into basic function test cases, derived function test cases, and exception test cases. The basic function test cases are prioritized, and then the derived function test cases and exception test cases are sorted in turn according to the correlation analysis results to generate the final test case list.

[0011] Secondly, the present invention provides a communication protocol testing and evaluation device based on multi-agent cooperation, used to implement the aforementioned communication protocol testing and evaluation method based on multi-agent cooperation, the device comprising: The protocol core data acquisition module is used to construct a test requirement analysis intelligent agent, which includes a document preprocessing unit and a semantic retrieval unit. It acquires the communication protocol test text, inputs the communication protocol test text into the document preprocessing unit for preprocessing to obtain text blocks, and inputs the text blocks into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain the protocol core data. The final test point set acquisition module is used to construct a test point generation agent, which includes a test point design unit and a test point optimization unit. The core protocol data is input into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set. The initial test point set is input into the test point optimization unit for classification, deduplication, and missing data supplementation to obtain the final test point set. The final test case list acquisition module is used to construct a test case generation agent, which includes a test case generation unit and a test case sorting unit. The final test point set is input into the test case generation unit to transform test cases and obtain an initial test case set. The initial test case set is input into the test case sorting unit to perform correlation analysis between test cases and obtain a test mind map. The initial test case set is then sorted according to the test mind map to obtain the final test case list. The evaluation result acquisition module is used to perform communication protocol testing based on the final test case list and obtain the communication protocol test results; it also constructs an expert evaluation model, inputs the communication protocol test results into the expert evaluation model for graded evaluation of the test results, and obtains the communication protocol test evaluation results.

[0012] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned multi-agent cooperative communication protocol testing and evaluation method.

[0013] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned multi-agent cooperative communication protocol testing and evaluation method.

[0014] The present invention has the following beneficial effects: 1. The test requirement analysis agent automatically completes text preprocessing and core data extraction, providing accurate data basis for the subsequent construction of test points; the test point generation agent can extract core protocol features, automatically match corresponding test dimensions and templates, and the test case generation agent dynamically fills templates based on core protocol data, eliminating the need to redesign test processes for different protocols and quickly generating adaptable test plans, greatly improving the versatility and scalability of test cases; finally, the test results are substituted into the expert evaluation model to calculate scores and output a quantitative evaluation report. The evaluation process is not affected by subjective factors, greatly improving the accuracy of communication protocol test evaluation.

[0015] 2. The test requirements analysis employs a dual design of preprocessing and semantic retrieval for the intelligent agent, ensuring the accuracy and completeness of core data extraction. The document preprocessing unit eliminates invalid information and clarifies text hierarchical logic through noise reduction, terminology annotation, and structured partitioning, avoiding the omission of key information. The semantic retrieval unit constructs an index based on the communication protocol knowledge graph, combining vector similarity retrieval and deduplication algorithms to accurately select core data such as protocol key field definitions, functional constraints, interaction process specifications, and exception handling requirements. It also associates text hierarchical identifiers with entity IDs, providing accurate basis for the construction of subsequent test points and test cases, solving the problems of error-prone and incomplete traditional manual extraction.

[0016] 3. The test point generation agent adapts to protocol types based on four test dimensions. After constructing the initial test points, it undergoes triple optimization through classification, similarity deduplication, and missing point supplementation. This eliminates duplicate test points to reduce redundancy and supplements missing test points based on functional constraint data, ensuring that the test points cover all dimensions of the protocol requirements without omissions or redundancy. The test case generation agent generates standardized test cases by mapping test points to test cases, avoiding the problems of non-standard manual writing. At the same time, it constructs a test mind map through correlation analysis, sorting test cases according to the logic of basic functions - derived functions - anomaly tests, ensuring that the execution of test cases conforms to the protocol's operating logic and improving test execution efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a structural diagram of the device according to an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0020] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0021] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0022] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0023] Reference Figure 1 This invention provides a method for testing and evaluating communication protocols based on multi-agent cooperation, comprising: The S100 constructs a test requirement analysis agent, which includes a document preprocessing unit and a semantic retrieval unit. It acquires the communication protocol test text, inputs the communication protocol test text into the document preprocessing unit for preprocessing, and obtains text blocks. The text blocks are then input into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain the core protocol data.

[0024] In some embodiments, the preprocessing of the communication protocol test text input document preprocessing unit to obtain text blocks includes: The communication protocol test text is input into the document preprocessing unit, and text denoising, term tagging, invalid information filtering, and sentence segmentation are performed sequentially to obtain semantic blocks. Obtain the hierarchical logic corresponding to the communication protocol test text, and perform structured division of semantic blocks according to the hierarchical logic to obtain text blocks identified by the hierarchical logic.

[0025] In some embodiments, the text preprocessing unit includes a text denoising module, a terminology annotation module, an invalid information filtering module, a sentence segmentation and block segmentation module, and a hierarchical logical structure partitioning module.

[0026] The text denoising module uses regular expression matching to remove special characters, format marks, and meaningless placeholders from the communication protocol test text; it filters out garbled characters through encoding verification and unifies the text encoding format to UTF-8.

[0027] The terminology annotation processing module calls the terminology library of the communication protocol domain and uses the BERT named entity recognition model to identify terms such as protocol field names and functional module names in the text, and adds a type label to each term.

[0028] The invalid information filtering module uses keyword blacklists and semantic similarity algorithms to filter and delete invalid information such as descriptive text and cited literature excerpts that are irrelevant to the protocol test.

[0029] The sentence segmentation and block processing module uses periods, semicolons, etc. as delimiters to segment sentences, and then combines semantically close sentences into semantic blocks based on terminology relevance.

[0030] The hierarchical logical structured partitioning module extracts hierarchical logic by parsing text title levels or logical connectors; according to the hierarchical structure of protocol outline - functional module - specific requirements, it adds hierarchical identifiers to each semantic block, ultimately forming a structured text block.

[0031] In some embodiments, the step of inputting text blocks into a semantic retrieval unit for knowledge index construction and semantic retrieval to obtain core protocol data includes: The text block is input into the semantic retrieval unit for feature extraction to obtain the semantic features corresponding to the text block; Obtain a knowledge graph of communication protocols, build a basic index library based on the knowledge graph of communication protocols, associate and map semantic features with the basic index library, and generate knowledge index data; The protocol database is obtained, and vector similarity is performed in the protocol database based on the knowledge index data. The retrieved data with a vector similarity greater than a first preset threshold with the knowledge index data is filtered out. The retrieved data is deduplicated to obtain the core protocol data, which includes protocol key field definition data, functional constraint data, interaction process specification data, and exception handling requirement data.

[0032] In some embodiments, the semantic retrieval unit includes a feature extraction module, a knowledge index construction module, a vector similarity retrieval module, and a retrieval result deduplication module.

[0033] The feature extraction module adopts the RoBERTa domain-adapted model. The pre-trained data incorporates communication protocol corpora such as TCP / IP and MQTT to perform semantic encoding on the input structured text blocks. Weighted word embedding is performed on the protocol terms already labeled in the text blocks to enhance the weight of domain keywords and extract word-level features. A global semantic vector of the text block is generated to capture the contextual logical relationship and extract sentence-level features. The word-level features and sentence-level features are fused to generate semantic features with the structure of text block ID-term weight vector-global semantic vector, which are then associated with the original text block hierarchical identifier.

[0034] The system acquires a pre-built knowledge graph of communication protocols, containing core entities, relationships, and attributes. A knowledge index building module constructs a basic index library based on the knowledge graph, including entity indexes (mapping entity IDs to entity names), attribute indexes (associating entity IDs with corresponding attributes), and relationship indexes (storing relationships between entities). Through a dual mechanism of semantic similarity matching and term-entity alignment, the semantic features of text blocks are associated with the basic index library. First, the semantic similarity between the global semantic vector in the semantic features and the entity attribute descriptions in the knowledge graph is calculated, matching the corresponding entity IDs. Then, the labeled terms in the text blocks are directly aligned with the entity indexes. Finally, knowledge index data is generated, with the structure: text block hierarchical identifier - associated entity ID set - feature similarity score - index type.

[0035] The protocol database stores structured protocol resources, partitioned by protocol type and core data category, containing structured records of four types of core data.

[0036] The vector similarity retrieval module uses the cosine similarity algorithm to calculate vector similarity. It uses the global semantic vector in the knowledge index data as the retrieval index, traversing the corresponding partitions of the protocol database. The formula for calculating vector similarity is: in, The semantic vector of the knowledge index data. For retrieving data from the protocol database, semantic vectors For vector similarity, Let i be the i-th dimension component of the semantic vector of the knowledge index data. Let n be the i-th dimension component of the semantic vector used to retrieve data from the protocol database, where n is the dimension of the semantic vector.

[0037] Set a first preset threshold for similarity, filter out search data with similarity higher than the first preset threshold, and retain metadata such as data ID, core content, and similarity score.

[0038] The deduplication module for search results employs a key field hashing algorithm, using four key dimensions of the protocol's core data as hash keys: key field definition data (field name + value range); functional constraint data (constraint condition + applicable scenario); interaction process specification data (process node + trigger condition); and exception handling requirement data (exception type + handling strategy). For search data with duplicate hash values, the record with the highest similarity score is retained. The deduplicated search data is then integrated and categorized according to key field definition, functional constraint, interaction process specification, and exception handling requirement to generate structured protocol core data. This data is then linked to entity IDs and text block hierarchy identifiers in the knowledge index data, facilitating the construction of subsequent test points.

[0039] The S200 constructs a test point generation agent, which includes a test point design unit and a test point optimization unit. The core protocol data is input into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set. The initial test point set is then input into the test point optimization unit for classification, deduplication, and missing point supplementation to obtain the final test point set.

[0040] In some embodiments, the step of inputting core protocol data into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set includes: Input the core protocol data into the test point design unit to identify the protocol type and obtain the communication protocol type; Obtain the test dimensions corresponding to the communication protocol type. The test dimensions include functional implementation dimension, performance indicator dimension, compatibility dimension and exception tolerance dimension. Call the test point design template corresponding to the test dimension. Based on the protocol's key field definition data, functional constraint data, interaction process specification data, and exception handling requirements data in the core protocol data, test points are constructed on the basis of the test point design template to obtain an initial set of test points.

[0041] In some embodiments, the test point design unit includes a protocol type identification module, a test dimension and template calling module, and a test point construction module.

[0042] The protocol type identification module extracts feature items from the core protocol data, including: key protocol fields such as SYN, ACK, and Topic fields; interaction process features such as the three-way handshake process and publish-subscribe process; functional constraint keywords such as connection-oriented and reliable transmission; and calls a protocol feature library that stores the exclusive features of mainstream protocols such as TCP / IP, MQTT, and HTTP. A feature matching algorithm is used to compare the extracted feature items with the protocol features in the library. The one with the highest matching degree is the identified communication protocol type. For example, if the SYN or ACK field is matched with the three-way handshake process, it is determined to be the TCP protocol.

[0043] The test dimension and template calling module associates preset test dimension configurations with the identified communication protocol type. The four test dimensions are the general foundation, and the dimension weights can be fine-tuned for different protocols. For example, the performance indicator dimension weight can be increased for real-time communication protocols. The template library is stored in categories according to protocol type and test dimension, such as TCP protocol - function implementation dimension templates, MQTT protocol - compatibility dimension templates. Each template contains standardized test point items. For example, the function dimension template contains items for field validity verification and process integrity verification. The corresponding protocol's four test dimension templates can be directly called.

[0044] The test point construction module constructs test points based on the protocol's key field definition data, including field value range verification and field format validity checks; it also constructs test points based on interaction flow specification data, including flow node sequence verification and key step trigger verification; based on performance requirements in functional constraint data, such as transmission latency ≤100ms, it constructs latency testing - extreme scenarios and throughput testing - concurrent connection count test points; referencing the adaptation range in functional constraint data, such as support for IPv4 / IPv6, it constructs protocol version compatibility testing and cross-platform adaptation verification test points; based on anomaly handling requirements data, such as network interruption reconnection mechanisms, it constructs anomaly scenario triggering - reconnection effectiveness and erroneous parameter input - fault tolerance handling test points; and finally, it summarizes all test points across all dimensions, removes obvious logical conflicts, and forms an initial test point set.

[0045] In some embodiments, the step of inputting the initial test point set into the test point optimization unit for classification, deduplication, and missing point supplementation to obtain the final test point set includes: Obtain the test dimensions corresponding to the initial test point set. The test point optimization unit classifies the initial test point set according to the test dimensions to obtain test point groups. The test point groups include functional test group, performance test group, compatibility test group, and exception test group. The similarity matching deduplication algorithm is used to calculate the text similarity between test points in the same test point group in the initial test point set, and remove duplicate test points with text similarity greater than the second preset threshold to obtain the first test point set. Obtain the functional constraint data from the core protocol data, perform missing detection on the test points in the first test point set based on the functional constraint data, and obtain the missing detection points; reconstruct the missing detection points to obtain the final test point set.

[0046] In some embodiments, the test point optimization unit includes a test point classification module, a test point deduplication module, and a test point missing supplementation module.

[0047] The test point classification module extracts the core keywords for each test point in the initial test point set, such as field validation, latency testing, cross-platform adaptation, and abnormal reconnection. It pre-defines feature word libraries for four test dimensions: functional (field, process, legality); performance (latency, throughput, concurrency); compatibility (cross-platform, version adaptation); and anomaly (incorrect input, interruption, fault tolerance). The test points are then grouped into functional, performance, compatibility, and anomaly test categories based on the matching degree between keywords and feature word libraries, generating test point sets with group identifiers.

[0048] The test point deduplication module uses TF-IDF + cosine similarity as the similarity matching deduplication algorithm to extract the text of all test points within a certain test point group, thus constructing a corpus. ,in Indicates the first Test point text, , This represents the total number of test points in the same group. (For the corpus) All test point texts are segmented into words, stop words are filtered out, and a unique set of terms is obtained. ,in Indicates the first A unique term, , This represents the size of the global vocabulary.

[0049] Calculate the term frequency in a single test point text. Indicates a term Text at a single test point The frequency of occurrence in the word is calculated using the normalized term frequency formula: in, Normalized word frequency.

[0050] Calculate the inverse document frequency of a term in the same group of corpora. Indicates a term The entire corpus The scarcity of text in the test points is considered; the higher the scarcity, the stronger the discriminative power of the test point text. The smoothed IDF calculation formula is used as follows: in, This indicates that corpus C contains terms. The number of test point texts.

[0051] Calculate the TF-IDF weight of each term in the text of a single test point. The product of normalized term frequency and inverse document frequency reflects the frequency of a particular term. For the first Test point text Semantic importance: With global vocabulary Size As a dimension of the feature vector, according to the global vocabulary Using a fixed order, the TF-IDF weights of each term are used as components of the corresponding dimension of the vector to generate the first term. Test point text TF-IDF feature vectors : The text of the two test points was calculated. , eigenvectors , Next, text similarity is calculated for deduplication. The formula for text similarity is as follows: in, for and Text similarity, For feature vectors The i-th dimension component, For feature vectors The i-th dimension component.

[0052] Set a second preset threshold, and iterate through and calculate the text similarity between any two test points in the same group. If the text similarity is greater than the second preset threshold, it is determined to be a duplicate test point. The test point with more complete description and more comprehensive core data of the associated protocol is retained, and redundant items are removed to form the first test point set.

[0053] The test point missing supplementation module extracts key items from the functional constraint data in the core protocol data, such as support for IPv4 / IPv6 dual protocols, transmission latency ≤100ms, and automatic reconnection after network interruption, and constructs a list of constraint items. It then compares each constraint item with the first test point set and the constraint item list. If a constraint item lacks a corresponding test point (e.g., no related compatibility test point for IPv6 support), it is marked as a missing detection point. Based on the constraint items corresponding to the missing detection points, and referring to the format specifications of test points in the same group, test points are supplemented and constructed. For example, for IPv6 support, IPv6 protocol version adaptation verification test points are constructed. These are then integrated to form the final test point set.

[0054] The S300 constructs a test case generation agent, which includes a test case generation unit and a test case sorting unit. The final set of test points is input into the test case generation unit to transform test cases and obtain an initial set of test cases. The initial set of test cases is then input into the test case sorting unit to perform correlation analysis between test cases and obtain a test mind map. The initial set of test cases is sorted according to the test mind map to obtain the final list of test cases.

[0055] In some embodiments, the step of inputting the final test point set into the test case generation unit for test case conversion to obtain an initial test case set includes: The test case generation unit sets the mapping relationship between test points and test cases, and calls the test case templates for each test point in the final test point set according to the mapping relationship; Obtain the core protocol data corresponding to each test point in the final test point set. Based on the protocol key field definition data, functional constraint data, interaction process specification data, and exception handling requirement data in the core protocol data, construct test cases on the basis of the test case template to obtain the initial test case set.

[0056] In some embodiments, the test case generation unit includes a mapping relationship configuration module, a test case template management module, and a core data association and filling module.

[0057] The mapping relationship configuration module sets standardized mapping rules according to test dimension and test point type. For example, for functional dimension field value verification test points, the corresponding mapping field validity test case template is mapped; for performance dimension latency test points, the corresponding mapping performance index quantification test case template is mapped. The mapping relationship is stored in the rule base and associated with the protocol type identifier.

[0058] The test case template management module matches the corresponding test case template from the template library based on the test dimension, type, and protocol type of the final test point. The template contains fixed fields such as test purpose, input parameters, test steps, expected results, and preconditions. For example, for the version adaptation test point of the MQTT protocol compatibility dimension, the MQTT protocol-compatibility test-version adaptation template is called.

[0059] The core data association and filling module extracts association information from the core protocol data for each test point. For functional test points, it extracts key field definitions and interaction process specifications; for performance test points, it extracts performance indicators from functional constraints; and for exception test points, it extracts exception handling requirements.

[0060] Following a template with a fixed field structure, the extracted core data is transformed into specific test case content. Combined with test point descriptions, such as verifying the validity of address field values, valid and invalid values ​​for key fields and test environment parameters are filled in. Operations are broken down according to the interaction flow specification, such as 1. Input valid field values; 2. Trigger protocol interaction; 3. Record response results. Judgment criteria are clearly defined based on functional constraints and exception handling requirements, such as a normal protocol response after valid value input and an error message returned after invalid value input. All completed test cases are summarized and verified to form an initial test case set.

[0061] In some embodiments, the step of inputting the initial test case set into the test case sorting unit for correlation analysis between test cases to obtain a test mind map, and sorting the initial test case set according to the test mind map to obtain the final test case list includes: The core attributes of each test case in the initial test case set are extracted by the test case sorting unit. The core attributes include test objective, test object and prerequisite dependency conditions. Based on the core attributes, perform correlation analysis on each test case to obtain correlation analysis results. The correlation analysis results include sequential execution correlation, same object test correlation, and abnormal dependency correlation. By using test cases as nodes and correlation analysis results as connections between nodes, a test mind map is constructed. Based on the test mind map, the test cases in the initial test case set are divided into basic function test cases, derived function test cases, and exception test cases. The basic function test cases are prioritized, and then the derived function test cases and exception test cases are sorted in turn according to the correlation analysis results to generate the final test case list.

[0062] In some embodiments, the test case sorting unit includes a core attribute extraction module, a test case association analysis module, a test mind map construction module, and a test case classification and sorting module.

[0063] The core attribute extraction module extracts core attributes from the standardized fields of the initial test cases. Test objectives: Extract core actions and objects from the test objective field, such as verifying the legality of the address field value; Test objects: Identify protocol modules and key fields, such as the data interaction module and the address field; Prerequisite dependencies: Read the associated test case IDs or function points from the prerequisite fields, such as executing test case UC001 to verify connection establishment first, forming a structured attribute list of test case ID-test objective-test object-prerequisite dependencies.

[0064] Use case association analysis is performed through the use case association analysis module. Sequential execution association: If the pre-dependency of use case A includes the ID of use case B, or if the test object of use case A requires the execution result of use case B, then A and B are determined to be sequentially executed. Same object test association: Use cases with the same test object are selected and determined to be same object association. Abnormal dependency association: If the pre-dependency of an abnormal test case includes basic or derived function use cases, then it is determined to be an abnormal dependency association.

[0065] The test mind map building module uses individual test cases as nodes, with each node labeled with its test case ID and test type. It uses the association analysis results as the connections between nodes, labeling the association types. The mind map is built according to the hierarchy of basic test cases, derived test cases, and abnormal test cases. Within the same level, test cases are clustered according to the degree of association, and test cases of the same object are arranged adjacently to form a visual association network.

[0066] Test cases are categorized using the test case classification and sorting module. Basic function test cases are those with no prerequisites and that ensure the core operation of the protocol, such as field validity verification and basic connection establishment. Derivative function test cases are those that depend on basic test cases and extend the protocol's functionality, such as high-concurrency data transmission. Anomaly test cases simulate fault scenarios and depend on basic or derived test cases, such as network interruption and reconnection.

[0067] Prioritize basic function test cases, sorting them by the importance of the test objects, with core modules taking precedence; then arrange derived function test cases according to their execution order, grouping test cases for the same object together; finally, arrange exception test cases according to their exception dependency order, with exception test cases that depend on the same basic test case adjacent to each other; finally, generate a final test case list containing test case ID, sorting number, test type, and associated test case ID.

[0068] The S400 performs communication protocol testing based on the final test case list to obtain communication protocol test results; it then constructs an expert evaluation model, inputs the communication protocol test results into the expert evaluation model for graded evaluation of the test results, and obtains communication protocol test evaluation results.

[0069] In some embodiments, hardware and software conforming to communication protocol requirements are deployed, and network parameters are configured to match test case requirements. Test cases are invoked through test execution tools according to the final test case list, simulating protocol interaction scenarios such as normal data transmission, abnormal interruption, and cross-platform connection, automatically executing operations such as input parameter injection and process triggering. The core results of each test case are recorded, including: 1. Execution status; 2. Actual output data, such as response time, transmission success rate, and error messages; 3. Related metrics, such as latency values ​​in the performance dimension and adaptation results in the compatibility dimension, generating structured test results with test case ID, test type, execution status, actual result, and related metrics.

[0070] The core configuration of the expert evaluation model includes: 1. Evaluation index system: corresponding to four dimensions: function implementation, performance index, compatibility, and fault tolerance. Each dimension has specific sub-indicators, such as latency, throughput, and concurrency in the performance dimension; 2. Index weights: set by domain experts, such as 0.4 for function implementation, 0.3 for performance index, 0.15 for compatibility, and 0.15 for fault tolerance; 3. Scoring rules: scores are quantified based on test results, such as 100 points for all functional test cases passing and 20 points deducted for a single failure; full marks are awarded for meeting performance index standards, and points are deducted according to the deviation ratio for failures; 4. Grading thresholds: four levels of standards are set: excellent (≥90 points), good (80-89 points), qualified (60-79 points), and unqualified (<60 points).

[0071] The structured test results are substituted into the expert evaluation model, the total score is calculated according to the indicator weights, and the protocol test level is determined by combining the grading thresholds; the output includes the total evaluation score, the scores of each dimension, the level conclusion, and the details of the deductions for the communication protocol test evaluation.

[0072] Reference Figure 2 This invention provides a communication protocol testing and evaluation device 20 based on multi-agent cooperation, used to implement a communication protocol testing and evaluation method based on multi-agent cooperation. The device includes: The protocol core data acquisition module 21 is used to construct a test requirement analysis intelligent agent, which includes a document preprocessing unit and a semantic retrieval unit; it acquires the communication protocol test text, inputs the communication protocol test text into the document preprocessing unit for preprocessing to obtain text blocks; and inputs the text blocks into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain the protocol core data. The final test point set acquisition module 22 is used to construct a test point generation agent, which includes a test point design unit and a test point optimization unit. The core protocol data is input into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set. The initial test point set is input into the test point optimization unit for classification, deduplication, and missing data supplementation to obtain the final test point set. The final test case list acquisition module 23 is used to construct a test case generation agent, which includes a test case generation unit and a test case sorting unit. The final test point set is input into the test case generation unit to transform the test cases and obtain an initial test case set. The initial test case set is input into the test case sorting unit to perform correlation analysis between test cases and obtain a test mind map. The initial test case set is then sorted according to the test mind map to obtain the final test case list. The evaluation result acquisition module 24 is used to perform communication protocol testing based on the final test case list and obtain the communication protocol test results; it also constructs an expert evaluation model, inputs the communication protocol test results into the expert evaluation model for graded evaluation of the test results, and obtains the communication protocol test evaluation results.

[0073] This application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements a multi-agent cooperative communication protocol testing and evaluation method based on any of the above schemes.

[0074] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0075] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0076] This application also provides a computer-readable medium storing a computer program thereon, which, when executed by a processor, implements the multi-agent cooperative communication protocol testing and evaluation method described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0077] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0078] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for testing and evaluating communication protocols based on multi-agent cooperation, characterized in that, include: Construct a test requirements analysis intelligent agent, which includes a document preprocessing unit and a semantic retrieval unit; The communication protocol test text is obtained, and then preprocessed by the document preprocessing unit to obtain text blocks. The text blocks are then input into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain the core protocol data. A test point generation agent is constructed, which includes a test point design unit and a test point optimization unit. Input the core protocol data into the test point design unit to perform protocol type adaptation and test point construction, and obtain the initial test point set; The initial set of test points is input into the test point optimization unit for classification, deduplication, and missing point supplementation to obtain the final set of test points. A test case generation agent is constructed, which includes a test case generation unit and a test case sorting unit. The final set of test points is input into the test case generation unit to transform test cases and obtain the initial set of test cases; the initial set of test cases is input into the test case sorting unit to perform correlation analysis between test cases and obtain a test mind map; the initial set of test cases is sorted according to the test mind map to obtain the final list of test cases. Perform communication protocol testing based on the final test case list and obtain the communication protocol test results; An expert evaluation model is constructed, and the test results of the communication protocol are input into the expert evaluation model to conduct a graded evaluation of the test results and obtain the test evaluation results of the communication protocol.

2. The method for testing and evaluating communication protocols based on multi-agent collaboration according to claim 1, characterized in that, The step of preprocessing the communication protocol test text input document to obtain text blocks includes: The communication protocol test text is input into the document preprocessing unit, and text denoising, term tagging, invalid information filtering, and sentence segmentation are performed sequentially to obtain semantic blocks. Obtain the hierarchical logic corresponding to the communication protocol test text, and perform structured division of semantic blocks according to the hierarchical logic to obtain text blocks identified by the hierarchical logic.

3. The method for testing and evaluating communication protocols based on multi-agent collaboration according to claim 1, characterized in that, The process of inputting text blocks into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain core protocol data includes: The text block is input into the semantic retrieval unit for feature extraction to obtain the semantic features corresponding to the text block; Obtain a knowledge graph of communication protocols, build a basic index library based on the knowledge graph of communication protocols, associate and map semantic features with the basic index library, and generate knowledge index data; The protocol database is obtained, and vector similarity is performed in the protocol database based on the knowledge index data. The retrieved data with a vector similarity greater than a first preset threshold with the knowledge index data is filtered out. The retrieved data is deduplicated to obtain the core protocol data, which includes protocol key field definition data, functional constraint data, interaction process specification data, and exception handling requirement data.

4. The method for testing and evaluating communication protocols based on multi-agent collaboration according to claim 1, characterized in that, The process involves inputting the core protocol data into the test point design unit for protocol type adaptation and test point construction, resulting in an initial test point set, including: Input the core protocol data into the test point design unit to identify the protocol type and obtain the communication protocol type; Obtain the test dimensions corresponding to the communication protocol type. The test dimensions include functional implementation dimension, performance indicator dimension, compatibility dimension and exception tolerance dimension. Call the test point design template corresponding to the test dimension. Based on the protocol's key field definition data, functional constraint data, interaction process specification data, and exception handling requirements data in the core protocol data, test points are constructed on the basis of the test point design template to obtain an initial set of test points.

5. The method for testing and evaluating communication protocols based on multi-agent collaboration according to claim 1, characterized in that, The process of inputting the initial test point set into the test point optimization unit for classification, deduplication, and missing point supplementation to obtain the final test point set includes: Obtain the test dimensions corresponding to the initial test point set. The test point optimization unit classifies the initial test point set according to the test dimensions to obtain test point groups. The test point groups include functional test group, performance test group, compatibility test group, and exception test group. The similarity matching deduplication algorithm is used to calculate the text similarity between test points in the same test point group in the initial test point set, and remove duplicate test points with text similarity greater than the second preset threshold to obtain the first test point set. Obtain the functional constraint data from the core protocol data, perform missing detection on the test points in the first test point set based on the functional constraint data, and obtain the missing detection points; reconstruct the missing detection points to obtain the final test point set.

6. The method for testing and evaluating communication protocols based on multi-agent cooperation according to claim 1, characterized in that, The step of inputting the final test point set into the test case generation unit for test case transformation to obtain the initial test case set includes: The test case generation unit sets the mapping relationship between test points and test cases, and calls the test case templates for each test point in the final test point set according to the mapping relationship; Obtain the core protocol data corresponding to each test point in the final test point set. Based on the protocol key field definition data, functional constraint data, interaction process specification data, and exception handling requirement data in the core protocol data, construct test cases on the basis of the test case template to obtain the initial test case set.

7. The method for testing and evaluating communication protocols based on multi-agent cooperation according to claim 1, characterized in that, The initial test case set is input into the test case sorting unit for correlation analysis between test cases to obtain a test mind map. The initial test case set is then sorted according to the test mind map to obtain the final test case list, including: The core attributes of each test case in the initial test case set are extracted by the test case sorting unit. The core attributes include test objective, test object and prerequisite dependency conditions. Based on the core attributes, perform correlation analysis on each test case to obtain correlation analysis results. The correlation analysis results include sequential execution correlation, same object test correlation, and abnormal dependency correlation. By using test cases as nodes and correlation analysis results as connections between nodes, a test mind map is constructed. Based on the test mind map, the test cases in the initial test case set are divided into basic function test cases, derived function test cases, and exception test cases. The basic function test cases are prioritized, and then the derived function test cases and exception test cases are sorted in turn according to the correlation analysis results to generate the final test case list.

8. A communication protocol testing and evaluation apparatus based on multi-agent cooperation, used to implement the communication protocol testing and evaluation method based on multi-agent cooperation as described in any one of claims 1 to 7, characterized in that, The device includes: The protocol core data acquisition module is used to construct a test requirement analysis intelligent agent, which includes a document preprocessing unit and a semantic retrieval unit. It acquires the communication protocol test text, inputs the communication protocol test text into the document preprocessing unit for preprocessing to obtain text blocks, and inputs the text blocks into the semantic retrieval unit for knowledge index construction and semantic retrieval to obtain the protocol core data. The final test point set acquisition module is used to construct a test point generation agent, which includes a test point design unit and a test point optimization unit. The core protocol data is input into the test point design unit for protocol type adaptation and test point construction to obtain an initial test point set. The initial test point set is input into the test point optimization unit for classification, deduplication, and missing data supplementation to obtain the final test point set. The final test case list acquisition module is used to construct a test case generation agent, which includes a test case generation unit and a test case sorting unit. The final test point set is input into the test case generation unit to transform test cases and obtain an initial test case set. The initial test case set is input into the test case sorting unit to perform correlation analysis between test cases and obtain a test mind map. The initial test case set is then sorted according to the test mind map to obtain the final test case list. The evaluation result acquisition module is used to perform communication protocol testing based on the final test case list and obtain the communication protocol test results; it also constructs an expert evaluation model, inputs the communication protocol test results into the expert evaluation model for graded evaluation of the test results, and obtains the communication protocol test evaluation results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the communication protocol testing and evaluation method based on multi-agent cooperation as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the communication protocol testing and evaluation method based on multi-agent cooperation as described in any one of claims 1 to 7.

Images