An integrated equipment testing and diagnosis method based on intelligent mining of interface models

Abstract

The present invention discloses an integrated equipment testing and diagnosis method based on intelligent mining of interface models, which is used to construct test cases for equipment and carry out fault detection and diagnosis. The method inputs are equipment construction knowledge graphs describing domain concepts, natural language documents describing the working mechanisms between equipment components, equipment interface format documents describing message formats, and equipment historical communication message records. First, based on a large language model and sequence mining, an interface model with automata as the basic representation means is constructed. Then, positive and negative test cases are generated based on the interface model, and detection and diagnosis of timing anomalies are carried out based on the interface model. Compared with existing test and diagnosis methods, the present invention greatly reduces the manual workload required in the process of constructing a system interface model, and can improve the efficiency of equipment construction test and diagnosis.

Description

Technical Field

[0001] The present invention belongs to the field of computer software, and in particular relates to an integrated equipment testing and diagnosis method based on intelligent mining of interface models. Background Art

[0002] In complex systems such as industrial equipment, the code is voluminous and difficult to obtain, and testing mainly adopts black box methods. Automated black box testing often uses model-based techniques. These techniques require first building a mathematical model for the system under test, and then generating test cases based on the mathematical model, or analyzing the adequacy of the test.

[0003] How to construct a model is a difficult point in the above model-based testing. In the past, models were mainly constructed manually, which relied heavily on professionals and was costly. Mining models from historical operation records is an idea, but existing methods require the provision of collected sessions in advance, which is a difficult point for large-scale multi-task parallel distributed equipment systems. In addition, for a new software, or software that has evolved from a historical version, a complete and comprehensive historical operation record may not exist, and new ways must be found to obtain the model. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide an integrated equipment test and diagnosis method based on intelligent mining of interface models. The equipment interface model can be automatically constructed to support test and diagnosis, avoiding the reliance of existing methods on model construction experts, and reducing the cost of designing and implementing test and diagnosis solutions.

[0005] Technical solution: The present invention provides an integrated equipment testing and diagnosis method based on intelligent mining of interface models, which is used to mine the system interface model of the equipment from natural language documents and historical communication message records, and then construct test cases for testing, and carry out automatic fault detection and diagnosis, including the following steps:

[0006] Step 1: Input the equipment construction knowledge graph describing the domain concept, the equipment natural language document describing the working mechanism between equipment components, and the equipment interface format document describing the message format from the computer client, and use the text source event automaton mining technology based on the large language model to obtain the event automaton A that expresses the working mechanism between some equipment components. nl ;

[0007] Step 2: Input the equipment historical communication message records from the computer client, and use the message record source event automaton mining technology based on association extraction to obtain the event automaton A that expresses the working mechanism between some equipment components. tr ;

[0008] Step 3: Use the classic automaton merging algorithm to merge the event automaton A in step 1nl and the event automaton A of step 2 tr Perform integrated synthesis so that the combined automaton accepts A nl or A tr The corresponding language is used to obtain a system interface model that can fully describe the working mechanism between equipment components. The system interface model is expressed by a finite automaton. The initial state is the start of the session, the migration corresponds to the interface message between components, and the state transition path corresponds to the legal message communication sequence between interfaces.

[0009] Step 4: Based on the system interface model obtained in step 3, construct positive and negative example message sequences for the selected equipment component boundary as test cases for the equipment, and test the equipment through the test cases;

[0010] Step 5: Based on the system interface model obtained in step 3, generate an automaton-based equipment fault detector to detect message timing anomalies, and locate fault-related components in combination with the equipment structure knowledge graph to perform automatic fault detection and diagnosis on the equipment.

[0011] Furthermore, step 1 specifically includes the following steps:

[0012] Step 1.1, conversation extraction, the goal is to extract paragraphs expressing the workflow between interfaces from the natural language document describing the working mechanism between equipment components;

[0013] Step 1.2, knowledge injection: The goal is to expand the workflow description paragraph obtained in step 1.1, annotate the external knowledge unknown to the current large language model, and then perform reasoning;

[0014] Step 1.3, prompt tuning: The goal is to fine-tune the large language model so that it can better output the sequence information between interface messages. The prompts used for prompt training are the workflow description paragraphs and questions after the knowledge injection in step 1.2. The answers to the prompts are mathematical descriptions of the workflow process.

[0015] Step 1.4, event timing extraction: The workflow description of the equipment to be analyzed obtained from step 1.1 is put into step 1.2 for knowledge injection, and then constructed into prompt questions according to the method of step 1.3, and input into the fine-tuned large language model to obtain the interface message timing relationship in a single session contained in each workflow description paragraph;

[0016] Step 1.5, event automaton construction: construct an automaton that can describe the overall timing process of the equipment interface from a set of event timing relationships about the interface message; the specific process is to first construct a message acceptance tree PTA from the known message sequence, the tree node is the state, each transition corresponds to an interface message, each branch path of the tree represents a different session, and the sequence of all interface messages on the path is the interface message sequence in the session; then simplify the automaton, and the simplification is achieved by merging the states on the initial automaton; if two states are reached through the same message or the subsequent states of the two states are very close, they can be considered to be merged; the merging rules are defined in a heuristic way; after the merge, the state is expressed by the same state node, and the related transitions will also be redirected; after the merge operation, an automaton A that can describe the event timing relationship between the interface messages described in the equipment document is obtained nl .

[0017] Furthermore, the specific process of step 1.1 is as follows:

[0018] (1.1.1) Given a set of paragraphs describing project background, domain concepts, system workflow, etc. obtained from historical project design documents, a set of prompt and answer combinations are constructed according to the following template and input into the large language model for model fine-tuning training;

[0019] Prompt template: "Please determine whether the following content belongs to the workflow. Answer yes or no without further explanation:

[0020] <paragraph>”

[0021] Answer template: "Yes" / "No"

[0022] (1.1.2) Divide the given document of information to be extracted into paragraphs, and input each paragraph into the large language model according to the prompt style in step (1.1.1). If the answer given by the model is "yes", it is retained as a paragraph expressing the workflow between interfaces, and other paragraphs are discarded.

[0023] Furthermore, the specific process of step 1.2 is as follows:

[0024] (1.2.1) Obtain the main functional interface of each equipment component from the equipment construction knowledge graph, and then extract the specific message corresponding to the functional interface from the equipment interface format document that describes the message format. The equipment construction knowledge graph contains the entity information of each component in the target equipment and the interface entity corresponding to the component. There is an excitation and response relationship between the component entity and the interface entity. This knowledge graph is manually revised on the basis of the general knowledge graph in the field, reflecting the interface of the equipment at the conceptual level;

[0025] The present invention utilizes the interface names, component entity names, entity relationships and attribute information in the equipment construction knowledge graph to extract specific messages related to each interface in the equipment interface format document. These messages are the mapping of the interface at the implementation level. Specifically, the equipment interface format document contains a series of field format definitions of interface messages and natural language annotations about the interface, and the document is provided in standard formats such as XML and IDL. From the identifier names and annotations such as fields in the interface format document, an interface-related noun description and a relationship description between the overall name of the interface message and its field member names are extracted. These descriptions and relationships will be fuzzy matched with the interface entities and related relationships and attributes in the equipment construction knowledge graph, and interface messages with high similarity will be used as message attributes of the interface entities in the equipment construction knowledge graph.

[0026] (1.2.2) After extracting the interface entities in the equipment construction knowledge graph and their mapping at the specific message level, the paragraphs expressing the workflow between interfaces are expanded according to the following example format, and external knowledge that is not available in the large language model is injected into them:

[0027] "First, the client [service requester: sends a request message, receives a confirmation message] sends a request message [TCP segment: with flags SYN and ACK, with sequence number Seq, with confirmation number Ack] to the server [service provider: sends a confirmation message, receives a request message], where: the flag SYN [TCP segment flag SYN] is set to 1, indicating a request to establish a connection; the sequence number Seq [TCP segment sequence number Seq] is initialized to x";

[0028] The augmented content is in the area marked by square brackets []. The augmentation process first uses the entity recognition algorithm to find the relevant entities in the equipment construction knowledge graph that appear in a given paragraph, and then labels the entities with the relevant concepts, relationships, and attribute information in the above graph to facilitate reasoning by the large language model.

[0029] Furthermore, the specific process of step 1.3 is as follows:

[0030] Hint: "Assuming the format<a,x[value],b> It means that a sends x to b, where value represents all the attribute values ​​of x. For the following given text T, use the above format to express the order relationship in the text T, and only answer the formatted content: First, the client [service requester: sends a request message, receives a confirmation message] sends a request message [TCP segment: with flags SYN and ACK, with sequence number Seq, with confirmation number Ack] to the server [service provider: sends a confirmation message, receives a request message], where: the flag SYN [TCP segment flag SYN] is set to 1, indicating a request to establish a connection; the sequence number Seq [TCP segment sequence number Seq] is initialized to x; then the server [service provider] receives the request message sent by the client [service requester]. After receiving the TCP segment, it returns a confirmation message, in which: the flag bits SYN and ACK are set to 1, indicating that the sequence number Seq of the client is valid, the server can normally receive the data sent by the client, and agrees to create a new connection; the sequence number Seq is set to y; the confirmation number Ack is set to x+1, indicating that the sequence number Seq of the client is received, and the value is increased by 1 as the value of its own confirmation number Ack.

[0031] Answer: <Client, request message [TCP segment: SYN=1, Seq=x], server>

[0032] <Server, confirmation message [TCP segment: SYN=1, ACK=1, Seq=y, Ack=x+1], client>

[0033] Combining automatically constructed prompt questions and manually given answers to the questions and inputting them into a large language model for training can enable the model to acquire the ability to infer the interface message sequence from the workflow description.

[0034] Furthermore, the specific process of step 2 is as follows:

[0035] Step 2.1, vertical session extraction: The goal is to extract all interface message history records into several sessions according to the time clues. The functional services of the equipment are provided repeatedly in a cycle. One cycle is a session, which represents a usage process of the system. There are intervals between different sessions. The session extraction algorithm first scans the entire message history timeline, and then clusters the historical messages according to time. Those that can be clustered in one class are considered one session.

[0036] Step 2.2, horizontal session extraction: Multiple tasks will be run simultaneously in the equipment, and the interface messages of different tasks will not affect each other; separate the interface messages of different tasks so that one session focuses on one task; based on the session obtained in step 2.1, analyze the interface messages appearing in a session, if the probability of two messages appearing in a session is lower than the threshold, separate them; if a session S contains two groups of messages S t1 and S t2 , the probability of them appearing in a session is low, so S is separated into S t1 and S t2 The corresponding two sessions;

[0037] Step 2.3: Based on the interface message sequence in the session, construct its event automaton A tr The specific process is to construct a message acceptance tree PTA from the message sequence of each session. tr , the tree node is a state, each migration corresponds to an interface message, each branch path of the tree represents a different session, and the sequence of all interface messages on the path is the interface message sequence in the session; then the automaton is simplified, and the simplification is achieved by merging the states on the initial automaton; if two states are reached through the same message or the subsequent states of the two states are very close, they can be considered to be merged; the merging rules are defined in a heuristic way; after the merger, the state is expressed by the same state node, and the related migration will also be redirected; after the merging operation, an automaton A that can describe the temporal relationship between the events of the interface messages described in the equipment history message record is obtained tr Furthermore, the specific process of step 4 is as follows:

[0038] Step 4.1, positive example construction: First, define the boundary of the system to be tested. Complex equipment is composed of several components. The messages on the interfaces of the components included in the test boundary are generated from within the defined range and are not given by the test case. The test case is driven by the stimulus interface message outside the boundary. Next, perform path generation on the automaton on the equipment interface model. Each path corresponds to a test and is composed of a sequence of interface messages. Then, use the defined test boundary to filter the interface message sequence, remove the internal messages, and obtain the positive example test case.

[0039] Step 4.2, counter-example construction: Mutate the positive test case to obtain the counter-example test case. The mutation methods are injecting interface messages into the session, deleting interface messages, and grafting test message sequences. The obtained results are verified on the system interface model, and the message sequences that conform to the interface model are removed to obtain the counter-example test case that expresses the abnormal working scenario.

[0040] Step 4.3, filling in message data values: Based on the equipment interface format document, use random generation and other means to fill in the message data, further specify the message to make the test case executable.

[0041] Furthermore, the specific process of step 5 is as follows:

[0042] Step 5.1, interface message session identification: To ensure that a fault detected on a session does not affect the fault detection in other subsequent sessions and that interface messages from other sessions do not cause misjudgment of timing anomalies on this session, fault detection is performed with the session as the scope; if the time interval between two interface messages exceeds an upper limit range, they are considered to belong to different sessions; if two interface messages never belong to the same session in the system interface model, they are considered to come from different sessions and do not interfere with each other;

[0043] Step 5.2, fault detection based on the system interface model: In the same session, fault detection is performed based on the system interface model. The detection process inputs the interface message into the automaton of the model, and the model runs according to the transition corresponding to the interface message. If the transition does not exist, or the transition jumps to a state marked as abnormal, it is considered that a fault is found;

[0044] Step 5.3, fault location: After the fault is detected, the equipment construction knowledge graph is used to locate the components related to the fault; in the equipment construction knowledge graph, the component entity has an excitation and response relationship to the interface entity, and the interface entity has a communication message attribute; based on this association, the component that sends the message and the expected responder entity of the message are inferred from the fault message, and the associated components of the front interface message in the current session can be located to form fault detection and diagnosis.

[0045] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0046] (1) The present invention relies on a large language model and proposes a method for mining test models from natural language design documents of equipment, and combining them with models mined from historical operation records to support equipment testing and diagnosis. This method can construct a system model of the equipment in a highly automated manner from resources such as equipment documents, historical operation records, and message format descriptions, and use the model to automatically generate test cases to improve testing efficiency. At the same time, a fault diagnostic device is constructed based on the model to help detect and troubleshoot equipment quality problems. Compared with technologies that require manual construction of test models, the present invention provides a method with better usability.

[0047] (2) The present invention has a high degree of automation and can automatically construct an equipment interface model to support test diagnosis, thereby avoiding the reliance of existing methods on model construction experts, reducing the cost of test diagnosis scheme design and implementation, and improving method availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is the overall flow chart of the test and diagnosis method of the present invention;

[0049] Figure 2 The flowchart of the text event automaton mining based on the large language model of the present invention;

[0050] Figure 3 An example of a knowledge graph for equipment of the present invention;

[0051] Figure 4 A schematic diagram for constructing the event automaton of the present invention;

[0052] Figure 5 This is a flow chart of the message record event automaton mining based on association extraction of the present invention;

[0053] Figure 6 This is a schematic diagram of message sequence session extraction of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0055] like Figure 1 The specific steps of the scheme are as follows:

[0056] Step 1: Input the equipment construction knowledge graph describing the domain concept, the natural language document describing the working mechanism between equipment components, and the equipment interface format document describing the message format (such as .IDL format file) from the computer client, and use the text source event automaton mining technology based on the large language model to obtain the event automaton A that expresses the working mechanism between some equipment components. nl .

[0057] The specific process is as follows Figure 2 As shown, the input format is as follows.

[0058] Equipment structure knowledge graph (ZKG): reflects the concepts, entities, and relationships related to equipment composition and operation, and describes the working principle of complex integrated equipment from a structural perspective. An example is Figure 3 As shown in Figure 2. A graph consists of nodes and edges. Nodes represent concepts, entities, or attribute values; edges represent the properties of entities or the relationships between entities.

[0059] A concept reflects the type of a group of entities, including concepts corresponding to equipment composition and concepts corresponding to entity communication. The former are signal processors, screen display terminals, clock devices, etc. in complex equipment, and the latter correspond to different types of communication instructions. An entity is an instance of a concept, such as a specific processor X, screen display Y, startup command, etc. Attributes include feature descriptions of the equipment, specific protocol types corresponding to communication commands, and message names at the implementation level. The following types of relationships may exist between entities:

[0060] Composition relationship: Entity A is composed of entity B, etc., then A and B form a composition relationship;

[0061] Operation dependency relationship: Component A relies on the platform provided by component B to run, forming an operation dependency relationship;

[0062] · Initiation relationship: Component A can stimulate command M, forming a command-initiation relationship;

[0063] Response relationship: Component A can respond to command M, forming a command-response relationship;

[0064] Natural language documents of equipment working mechanism: from sources such as equipment design instructions, in the form of txt, doc, etc., provided in the form of continuous paragraphs, each paragraph introduces relevant background knowledge, basic principles, or describes the working mechanism of cooperation between equipment components through message communication. For example, a paragraph may have the following format: "After Z receives instruction A, it sends instruction B to W from the background, and W activates activity XXX. After success, it feeds back message C to Z. Z then issues instruction D. After W successfully performs activity YYY, it issues command E to Z to end the task process.

[0065] Equipment interface format document: IDL, XML and other standardized format files, including descriptions of message field composition, as well as annotations on messages and their fields. This document is provided by the equipment implementation process, and its essence is the same as the structure definition in C language.

[0066] Step 1.1: Conversation extraction. The goal is to extract paragraphs that express the workflow between interfaces from the natural language document that describes the working mechanism between equipment components. The specific process is as follows:

[0067] (a) Given a set of paragraphs describing project background, domain concepts, system workflow, etc. obtained from historical project design documents, a set of prompt and answer combinations are constructed according to the following two templates and input into the large language model for model fine-tuning training.

[0068] Prompt template: "Please determine whether the following content belongs to the workflow. Answer yes or no without further explanation:

[0069] <paragraph>”

[0070] Answer template: "Yes" / "No"

[0071] Workflow paragraphs describe the system workflow, while non-workflow paragraphs include concept and background introduction paragraphs. Typical examples of workflow description paragraphs are:

[0072] The workflow is: "First, the client sends a SYN packet to the server and waits for the server to confirm, where: the flag is SYN, indicating a request to establish a connection; the sequence number is Seq=x (x is generally a random number); then the client enters the SYN-SENT phase."

[0073] Not a workflow: "The handshake protocol is an important means to ensure that communications are secure, reliable, and in line with the expectations of both parties. Different handshake protocols are suitable for different communication scenarios to meet various security and communication requirements."

[0074] (b) Segment the given document to be extracted into paragraphs. Input each paragraph into the large language model according to the prompt style in step (a). If the answer given by the model is "yes", it is retained as a paragraph expressing the workflow between interfaces, and other paragraphs are discarded.

[0075] Step 1.2: Knowledge injection. The goal is to expand the workflow description paragraph obtained in step 1.1 and annotate the external knowledge that may be unknown to the current large language model to facilitate its reasoning. The specific steps are as follows:

[0076] (a) Obtain the main functional interfaces of each equipment component from the equipment construction knowledge graph, and then extract the specific messages corresponding to these functional interfaces from the equipment interface format document that describes the message format. The equipment construction knowledge graph of the present invention contains the entity information of each component in the target equipment and the interface entity corresponding to the component. There are relationships such as excitation and response between the component entity and the interface entity. This knowledge graph is obtained by manual revision based on the general knowledge graph of the field, reflecting the interface of the equipment at the conceptual level.

[0077] The present invention utilizes the interface names, component entity names, entity relationships and attribute information in the equipment construction knowledge graph to extract specific messages related to each interface in the equipment interface format document. These messages are the mapping of the interface at the implementation level. Specifically, the equipment interface format document contains a series of field format definitions of interface messages and natural language annotations about the interface, and the document is provided in standard formats such as XML and IDL. From the identifier names and annotations such as fields in the interface format document, an interface-related noun description and a relationship description between the overall name of the interface message and its field member names are extracted. These descriptions and relationships will be fuzzy matched with the interface entities and related relationships and attributes in the equipment construction knowledge graph, and interface messages with high similarity will be used as message attributes of the interface entities in the knowledge graph.

[0078] (b) After extracting the interface entities in the equipment construction knowledge graph and their mapping at the specific message level, the present invention amplifies the paragraphs expressing the workflow between interfaces according to the following example format, and injects external knowledge that the large language model does not have.

[0079] "First, the client [service requester: sends a request message, receives a confirmation message] sends a request message [TCP segment: with flags SYN and ACK, with sequence number Seq, with confirmation number Ack] to the server [service provider: sends a confirmation message, receives a request message], where: the flag SYN [TCP segment flag SYN] is set to 1, indicating a request to establish a connection; the sequence number Seq [TCP segment sequence number Seq] is initialized to x. ..."

[0080] The augmented content is in the area marked by the square brackets []. The augmentation process first uses the entity recognition algorithm to find the relevant entities in the equipment construction knowledge graph that appear in a given paragraph, and then labels the entities with the relevant concepts, relationships, and attribute information in the above graph to facilitate reasoning by the large language model. Entity recognition can be achieved by direct retrieval in the text paragraph or by applying the existing natural language entity recognition algorithm. After the entity is identified, a fuzzy entity search is performed in the equipment construction knowledge graph to match the workflow-related entities and their attributes and relationships in the text paragraph.

[0081] Step 1.3, prompt training (prompt tuning), the goal is to fine-tune the large language model so that it can better output the sequence information between interface messages. The prompts used for prompt training are the workflow description paragraphs and questions after the knowledge injection in step 1.2, and the answers to the prompts are mathematical descriptions of the workflow process. The format of prompts and answers is given in the form of examples below. The bold part of the prompt is the fixed question text, and the other parts are dynamic content; in the answer, a pair of angle brackets is used to represent an event, and multiple lines represent event sequences.

[0082] Hint: "Assuming the format<a,x[value],b> It means that a sends x to b, where value represents all the attribute values ​​of x. For the following given text T, please use the above format to express the order relationship in the text T, and only answer the formatted content: First, the client [service requester: sends a request message, receives a confirmation message] sends a request message [TCP segment: with flags SYN and ACK, sequence number Seq, and confirmation number Ack] to the server [service provider: sends a confirmation message, receives a request message], where: the flag SYN [TCP segment flag SYN] is set to 1, indicating a request to establish a connection; the sequence number Seq [TCP segment sequence number Seq] is initialized to x. Then, after the server [service provider] receives the request message [TCP segment] sent by the client [service requester], it returns a confirmation message [TCP segment], in which: the flag bits SYN [TCP segment flag bit SYN] and ACK [TCP segment flag bit ACK] are set to 1, indicating that the client [service requester]'s sequence number Seq [TCP segment sequence number Seq] is valid, and the server [service provider] can normally receive the data sent by the client [service requester] and agrees to create a new connection; the sequence number Seq [TCP segment sequence number Seq] is set to y; the confirmation number Ack [TCP segment confirmation number Ack] is set to x+1, indicating that the client [service requester]'s sequence number Seq [TCP segment sequence number Seq] is received, and its value is added by 1 as the value of its own confirmation number Ack [TCP segment confirmation number Ack]. "

[0083] Answer: <Client, request message [TCP segment: SYN=1, Seq=x], server>

[0084] <Server, confirmation message [TCP segment: SYN=1, ACK=1, Seq=y, Ack=x+1], client>

[0085] Combining automatically constructed prompt questions and manually given answers to the questions and inputting them into a large language model for training can enable the model to acquire the ability to infer the interface message sequence from the workflow description.

[0086] Step 1.4, event timing extraction. In this step, the workflow description of the equipment to be analyzed obtained from step 1.1 is put into step 1.2 for knowledge injection, and then constructed into prompt questions according to the method of step 1.3, and input into the fine-tuned large language model to obtain the interface message timing relationship in a single session contained in each workflow description paragraph.

[0087] Step 1.5: Event automaton construction: In this step, an automaton capable of describing the overall timing process of the equipment interface is constructed from a set of event timing relations about the interface message.

[0088] The specific process is to construct a message acceptance tree PTA from the known message sequence. Figure 4 As shown in the upper part, the tree nodes are states, and each transition corresponds to an interface message. Each branch path of the tree represents a different session, and the sequence of all interface messages on the path is the interface message sequence within the session. The message acceptance tree is essentially an initial version of the automaton that can accept all target message sequences.

[0089] The next step is to simplify and optimize the automaton. Simplification is achieved by merging states on the initial automaton. If two states are reached through the same message, or the subsequent states of the two states are very close, then they can be considered for merging. For example, in Figure 4 In the upper message acceptance tree, states 2 and 7, 5 and 6, 4 and 8, etc. can be merged. Detailed merging rules can be defined in a heuristic way. By properly applying domain knowledge, better merging effects can be achieved. After merging, the merged states are expressed by the same state node, and related transitions will also be redirected. For example, Figure 4 The lower automaton is a merging effect of the upper message receiving tree. After a series of merging operations, a relatively sophisticated automaton can be obtained that can describe the temporal relationship of events between interface messages described in the equipment document.

[0090] Step 2: Input the equipment historical communication message records from the computer client, and use the message record source event automaton mining technology based on association extraction to obtain the event automaton A that expresses the working mechanism between some equipment components. tr .

[0091] The input historical communication message record is a communication message sequence sorted by time, and each communication message includes message content, timestamp, communication source and destination, etc. The message field definition can be found according to the interface format document. The historical record is obtained by monitoring the data on the configuration and measurement module and the historical version of the related module.

[0092] Step 2.1, vertical session extraction (Session), such as Figure 6 As shown in (a) in the figure, the goal is to extract all interface message history records into several sessions according to the time clue. The functional services of the equipment are often provided repeatedly, and each cycle is a session, which represents a usage process of the system. There are often intervals between different sessions. The session extraction algorithm first scans the entire message history record timeline, and then clusters the historical messages by time. Those that can be clustered in one class are counted as one session. The specific clustering method is to extract the occurrence time of each interface message and obtain the time points t1, t2, …, t n ,Conversation clustering is to cluster these time points. The clustering algorithms that can be used include density-based methods such as DBSCAN and mean shift clustering.

[0093] Step 2.2, horizontal session extraction. Multiple tasks are often run simultaneously in the equipment, and the interface messages of different tasks do not affect each other. This step separates the interface messages of different tasks so that one session focuses on one task, such as Figure 6 As shown in (b) in .

[0094] The specific approach is to use association rule mining to analyze which interface messages often appear in a session based on the session obtained in step 2.1. If the probability of two messages appearing in a session is lower than the threshold, it means that they may only appear in the same session occasionally and should be separated. Suppose that after executing step 2.1, a total of H t sessions, of which H A∪B A session contains one of the interface messages A or B, H A∩B A session contains both interface messages A and B. If A∩B / H A∪B If it is greater than a certain threshold λ, A and B are considered to be messages belonging to the same session, otherwise they are considered to belong to different sessions, and only because of accidental parallel task flows that they appear at similar times.

[0095] If there are two groups of messages S in a session S t1 and S t2 , the two do not belong to the same session after analysis, that is, the probability of them appearing in the same session in the historical message records is low, so S is separated into S t1 and S t2 The two corresponding sessions.

[0096] Step 2.3: Based on the interface message sequence in the session, construct its event automaton, which includes two sub-steps: message acceptance tree construction and automaton reduction. The specific approach is the same as step 1.5. First, construct a message acceptance tree PTA from the message sequence of each session. tr , the tree node is a state, each migration corresponds to an interface message, each branch path of the tree represents a different session, and the sequence of all interface messages on the path is the interface message sequence in the session; then the automaton is simplified, and the simplification is achieved by merging the states on the initial automaton; if two states are reached through the same message or the subsequent states of the two states are very close, they can be considered to be merged; the merging rules are defined in a heuristic way; after the merger, the state is expressed by the same state node, and the related migration will also be redirected; after the merging operation, an automaton A that can describe the temporal relationship between the events of the interface messages described in the equipment history message record is obtained tr .

[0097] Step 3: Use the classic automaton merging algorithm to merge the event automata A in steps 1 and 2. nl and A tr Merge so that the merged automaton accepts A nl or A tr Corresponding language is used to obtain a system interface model that can more comprehensively describe the working mechanism between equipment components. The model is expressed by finite automata. The initial state is the start of the session, the migration corresponds to the interface message between components, and the state transition path corresponds to the legal message communication sequence between interfaces.

[0098] The specific merging method is, first, use A nl and A tr The Cartesian product of the states in the merged automaton is taken as the state of the merged automaton, and the initial state S of the merged automaton is start Yes A nl and A tr The corresponding initial state S nl and S tr The Cartesian product of nl ,S tr >. Then, from S start Start targeting A nl and A tr All interface packets in the merged state are migrated. a ,S b >, facing an interface message p, if A nl Medium S a Able to accept p and move to state S via transition c , A nl Medium S b Able to accept p and move to state S via transition d , then migration occurs a ,S b >→ c ,S d ​​​​>. If one of the automata cannot accept p, the corresponding part of the automaton in the target state of the migration goes to a non-accepting final state S t ; If neither automaton accepts p, then both the left and right parts of the target state jump to the non-accepting final state. In the state of the merged automaton, as long as one of the corresponding components of the automaton is in an accepting state, the state is considered to be an accepting state.

[0099] Step 4: Based on the system interface model obtained in step 3, construct positive and negative example message sequences for the selected equipment component boundaries as test cases for the equipment.

[0100] Step 4.1: Positive example construction.

[0101] First, define the boundary of the system to be tested. Complex equipment is composed of a series of components. The messages on the interfaces of the components included in the test boundary are generated from within the delimited range and are not given by the test case. The test case is mainly driven by the stimulus interface message outside the boundary. For example, if the equipment to be tested is composed of four components A, B, C, and D, if the test scope is defined as C and D, then the communication messages between C and D are completed by themselves, and the stimulus and response of the test case mainly simulate the interface messages between C and D from the outside and components A and B.

[0102] Next, the path generation on the automaton is performed on the equipment interface model. Each path corresponds to a test case and is composed of a sequence of interface messages. The path generation on the automaton is achieved through depth-first traversal. Each traversal selects a non-repeated path and non-repeated traversal nodes until the final state of the automaton is reached. In this way, a complete interface message communication sequence between a set of components can be obtained.

[0103] Then, the interface message sequence is filtered using the defined test boundary, internal messages are removed, and positive test cases are obtained. For example, if the message sequence covers messages related to the four components A, B, C, and D, and only C and D are divided into the current test scope, the messages between C and D are removed from the complete message sequence. These messages are generated by the module under test itself without external stimulation.

[0104] Step 4.2, counter-example construction. The present invention mutates the positive test case to obtain the counter-example test case. The main mutation means are injecting interface messages into the session, deleting interface messages, and grafting test message sequences.

[0105] Assume that the message in the current test case T corresponds to session S1, and the set of all possible messages of session S1 is P1. The present invention selects any message from P1 and injects it into any position of the current test case T to form a candidate counter-example case. The present invention will also delete an interface message from any position of the test case T to form another candidate counter-example case. At the same time, the present invention selects test cases T1 and T2 from the same session message set, respectively intercepts the front part of T1 and the back part of T2, and combines and grafts the two to obtain candidate counter-example cases.

[0106] The candidate counter-example cases obtained above will be run on the system interface model, and the interface messages in the case drive the operation of the migration corresponding to the interface messages in the current state of the model. If the system interface model always runs in a valid state, it means that the candidate test case is not a counter-example and needs to be removed. All test cases that jump out of the system interface model through migration or reach an invalid state are regarded as the final counter-example test cases.

[0107] Step 4.3, filling in message data values. The test case obtained in steps 4.1 and 4.2 only contains the sequence of interface messages, and lacks detailed field data of the message. This step will fill in the message data based on the equipment interface format document, using random generation and other means to further concretize the message, so that the test case is executable. Specifically, in interface formats such as XML and IDL, the types of message fields are specified, such as integers, enumerations, floating-point numbers, etc. In specifications such as XML Schema, the value constraints of the fields can also be limited. According to the type and constraints, a random algorithm can be used to generate legal values ​​that meet the constraints for the fields to simulate the message content that may appear in real scenarios. After filling in the message data, the message sequence and data constitute a relatively complete test case.

[0108] Step 5: Based on the system interface model obtained in step 3, generate an automaton-based equipment fault detector to detect message timing anomalies, and locate the fault-related components in combination with the equipment structure knowledge graph.

[0109] Step 5.1: Interface message session identification: To ensure that a fault detected in one session does not affect fault detection in other subsequent sessions and that interface messages from other sessions do not cause misjudgment of timing anomalies in this session, the present invention performs fault detection within the scope of the session.

[0110] For a message sequence p1,…,p n-1 ,p n , if p1 to p2 until p n-2 to p n-1 , the time interval between two messages is less than a preset threshold τ, then they are considered to belong to the same session and fault detection is performed as a whole. n-1 and p n The time interval between them is greater than τ, then it is considered that p n A new session has started. At this point, p n It is detected as the start of a new session and is not affected by the change from p1 to p n-1 The impact of messages between them.

[0111] For a message sequence a; b; c; d; e, if the two groups of interface messages a, c, e and b, d never belong to the same session in the system interface model, their continuous appearance is considered to be caused by parallel task flows that do not interfere with each other. The two sequences a; c; e and b; d will be detected by two independent detector instances respectively, and their results will not affect each other.

[0112] Step 5.2, fault detection based on the system interface model. In the same session, the present invention performs fault detection based on the system interface model. The detection process inputs interface messages one by one on the model's automaton, so that the model runs according to the transitions corresponding to the interface messages. If the current state of the model is S, its m transitions t1,…,t k ,…,t m Interface packets p1,…,p k ,…,p m As a label, enter the new message p that appears in the current system k , will change t k To run the interface model, the state jumps to the transition t k During the transition, if a message p k If the corresponding transition does not exist, or the transition jumps to a state marked as abnormal, it is considered that a fault is found and a fault alarm is given.

[0113] Step 5.3, fault location. After detecting a fault, the present invention uses the equipment construction knowledge graph to locate the components related to the fault. In the equipment construction knowledge graph, the component entity has an excitation and response relationship to the interface entity, and the interface entity has a communication message attribute. Based on this association, the component that sent the message and the expected responder of the message can be inferred from the fault message, and the associated components can also be located for the front interface message in the current session. This information forms a basic fault diagnosis, which can help users understand the fault.

[0114] For example, if in the interface message p f A fault is detected on the equipment structure knowledge graph. There is a functional interface entity I in the equipment structure knowledge graph, and its message attribute is p f , and there is an equipment component entity C, and there is an incentive relationship between C and I. Then it is diagnosed that component C has a fault at functional interface I. Tracing back all message sequences p1 in the complete session that arrives at the interface message p; ..., p f-1 ; pf, similar analysis is performed on each interface message, and the cause of the fault can be diagnosed as from C1:I1; C2:I2 to C f-1 :I f-1 ; A complete component interaction sequence of C:I, which will significantly help in fault analysis and troubleshooting.

[0115] In summary, the present invention performs integrated equipment test diagnosis based on intelligent mining of interface models through five steps. Through intelligent mining of interface models, the manual modeling overhead in the test diagnosis process is greatly reduced, and the overall efficiency of the test diagnosis is improved.

[0116] The present invention has carried out experiments on five typical equipment in the field of aircraft and ships that are integrated through data buses. The results show that the proposed method can reduce the modeling overhead of the manual interface model by more than 50%, achieving its expected value.

[0117] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.< / paragraph> < / paragraph>

Claims

1. An integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that It is used to mine the system interface model from the equipment natural language documents and historical communication message records, and then construct test cases for testing, and carry out automatic fault detection and diagnosis, including the following steps: Step 1: Input the equipment construction knowledge graph describing the domain concept, the equipment natural language document describing the working mechanism between equipment components, and the equipment interface format document describing the message format from the computer client, and use the text source event automaton mining technology based on the large language model to obtain the event automaton A that expresses the working mechanism between some equipment components. nl ; Step 2: Input the equipment historical communication message records from the computer client, and use the message record source event automaton mining technology based on association extraction to obtain the event automaton A that expresses the working mechanism between some equipment components. tr ; Step 3: Use the classic automaton merging algorithm to merge the event automaton A in step 1 nl and the event automaton A of step 2 tr Perform integrated synthesis so that the combined automaton accepts A nl or A tr The corresponding language is used to obtain a system interface model that can fully describe the working mechanism between equipment components. The system interface model is expressed by a finite automaton. The initial state is the start of the session, the migration corresponds to the interface message between components, and the state transition path corresponds to the legal message communication sequence between interfaces. Step 4: Based on the system interface model obtained in step 3, construct positive and negative example message sequences for the selected equipment component boundary as test cases for the equipment, and test the equipment through the test cases; Step 4: The specific process is as follows: Step 4.1, positive example construction: First, define the boundary of the system to be tested. Complex equipment is composed of several components. The messages on the interfaces of the components included in the test boundary are generated from within the defined range and are not given by the test case. The test case is driven by the stimulus interface message outside the boundary. Next, perform path generation on the automaton on the equipment interface model. Each path corresponds to a test and is composed of a sequence of interface messages. Then, use the defined test boundary to filter the interface message sequence, remove the internal messages, and obtain the positive test case. Step 4.2, counter-example construction: mutate the positive test case to obtain the counter-example test case. The mutation methods include injecting interface messages in the session, deleting interface messages, and grafting test message sequences. The obtained results are verified on the system interface model, and the message sequences that conform to the interface model are removed to obtain counter-example test cases that express abnormal working scenarios; Step 4.3, filling in message data values: According to the equipment interface format document, use random generation and other means to fill in message data, further specify the message to make the test case executable; Step 5: Based on the system interface model obtained in step 3, generate an automaton-based equipment fault detector to detect message timing anomalies, and locate fault-related components in combination with the equipment structure knowledge graph to perform automatic fault detection and diagnosis on the equipment.

2. According to claim 1, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that Step 1 specifically includes the following steps: Step 1.1, conversation extraction, the goal is to extract paragraphs expressing the workflow between interfaces from the natural language document describing the working mechanism between equipment components; Step 1.2, knowledge injection: The goal is to expand the workflow description paragraph obtained in step 1.1, annotate the external knowledge unknown to the current large language model, and then perform reasoning; Step 1.3, prompt tuning: The goal is to fine-tune the large language model so that it can better output the sequence information between interface messages. The prompts used for prompt training are the workflow description paragraphs and questions after the knowledge injection in step 1.

2. The answers to the prompts are mathematical descriptions of the workflow process. Step 1.4, event timing extraction: The workflow description of the equipment to be analyzed obtained from step 1.1 is put into step 1.2 for knowledge injection, and then constructed into prompt questions according to the method of step 1.3, and input into the fine-tuned large language model to obtain the interface message timing relationship in a single session contained in each workflow description paragraph; Step 1.5, event automaton construction: construct an automaton that can describe the overall timing process of the equipment interface from a set of event timing relationships about the interface message; the specific process is to first construct a message acceptance tree PTA from the known message sequence, where the tree node is the state, each transition corresponds to an interface message, each branch path of the tree represents a different session, and the sequence of all interface messages on the path is the interface message sequence in the session; then simplify the automaton, and the simplification is achieved by merging the states on the initial automaton; if two states arrive through the same message or the subsequent states of the two states are very close, they can be considered to be merged; the merging rules are defined in a heuristic way; after merging, the states are expressed by the same state node, and the related transitions will also be redirected; After the merging operation, an automaton A is obtained that can describe the temporal relationship between the events in the interface messages described in the equipment document. nl .

3. According to claim 2, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that Step 1.1 The specific process is as follows: (1.1.1) Given a set of paragraphs describing project background, domain concepts, system workflow, etc. obtained from historical project design documents, a set of prompt and answer combinations are constructed according to the following template and input into the large language model for model fine-tuning training; Prompt template: "Please determine whether the following content belongs to the workflow. Answer yes or no without further explanation: <paragraph> ”< / paragraph> Answer template: "Yes" / "No" (1.1.2) Divide the given document of information to be extracted into paragraphs, and input each paragraph into the large language model according to the prompt style in step (1.1.1). If the answer given by the model is "yes", it is retained as a paragraph expressing the workflow between interfaces, and other paragraphs are discarded.

4. According to claim 2, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that Step 1.2 The specific process is as follows: (1.2.1) Obtain the main functional interface of each equipment component from the equipment construction knowledge graph, and then extract the specific message corresponding to the functional interface from the equipment interface format document that describes the message format. The equipment construction knowledge graph contains the entity information of each component in the target equipment and the interface entity corresponding to the component. There is an excitation and response relationship between the component entity and the interface entity. This knowledge graph is manually revised on the basis of the general knowledge graph in the field, reflecting the interface of the equipment at the conceptual level; (1.2.2) After extracting the interface entities in the equipment construction knowledge graph and their mapping at the specific message level, the paragraphs expressing the workflow between interfaces are expanded according to the following example format, and external knowledge that is not available in the large language model is injected into them: "First, the client [service requester: sends a request message, receives a confirmation message] sends a request message [TCP segment: with flags SYN and ACK, with sequence number Seq, with confirmation number Ack] to the server [service provider: sends a confirmation message, receives a request message], where: the flag SYN [TCP segment flag SYN] is set to 1, indicating a request to establish a connection; the sequence number Seq [TCP segment sequence number Seq] is initialized to x"; The augmented content is in the area marked by square brackets []. The augmentation process first uses the entity recognition algorithm to find the relevant entities in the equipment construction knowledge graph that appear in a given paragraph, and then labels the entities with the relevant concepts, relationships, and attribute information in the above graph to facilitate reasoning by the large language model.

5. According to claim 2, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that Step 1.3 The specific process is as follows: Hint: "Assuming the format<a,x[value],b> It means that a sends x to b, where value represents all the attribute values ​​of x. For the following given text T, use the above format to express the order relationship in the text T, and only answer the formatted content: First, the client [service requester: sends a request message, receives a confirmation message] sends a request message [TCP segment: with flags SYN and ACK, with sequence number Seq, with confirmation number Ack] to the server [service provider: sends a confirmation message, receives a request message], where: the flag SYN [TCP segment flag SYN] is set to 1, indicating a request to establish a connection; the sequence number Seq [TCP segment sequence number Seq] is initialized to x; then the server [service provider] receives the request message sent by the client [service requester]. After receiving the TCP segment, it returns a confirmation message, in which: the flag bits SYN and ACK are set to 1, indicating that the sequence number Seq of the client is valid, the server can normally receive the data sent by the client, and agrees to create a new connection; the sequence number Seq is set to y; the confirmation number Ack is set to x+1, indicating that the sequence number Seq of the client is received, and the value is increased by 1 as the value of its own confirmation number Ack. Answer: <Client, request message [TCP segment: SYN=1, Seq=x], server> <Server, confirmation message [TCP segment: SYN=1, ACK=1, Seq=y, Ack=x+1], client> Combining automatically constructed prompt questions and manually given answers to the questions and inputting them into a large language model for training can enable the model to acquire the ability to infer the interface message sequence from the workflow description.

6. According to claim 1, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that Step 2: The specific process is as follows: Step 2.1, vertical session extraction: The goal is to extract all interface message history records into several sessions according to the time clues. The functional services of the equipment are provided repeatedly in a cycle. One cycle is a session, which represents a usage process of the system. There are intervals between different sessions. The session extraction algorithm first scans the entire message history timeline, and then clusters the historical messages according to time. Those that can be clustered in one class are considered one session. Step 2.2, horizontal session extraction: Multiple tasks will be run simultaneously in the equipment, and the interface messages of different tasks will not affect each other; separate the interface messages of different tasks so that one session focuses on one task; Based on the sessions obtained in step 2.1, analyze the interface messages that appear in a session. If the probability of two messages appearing in a session is lower than the threshold, separate them. If a session S contains two groups of messages S t1 and S t2 , the probability of them appearing in a session is low, so S is separated into S t1 and S t2 The corresponding two sessions; Step 2.3: Based on the interface message sequence in the session, construct its event automaton A tr The specific process is as follows: First, construct a message acceptance tree PTA from the message sequence of each session tr , the tree node is the state, each migration corresponds to an interface message, each branch path of the tree represents a different session, and the sequence of all interface messages on the path is the interface message sequence in the session; then the automaton is simplified, and the simplification is achieved by merging the states on the initial automaton; if two states are reached through the same message or the subsequent states of the two states are very close, they are merged; the merging rules are defined in a heuristic way; after the merger, the state is expressed by the same state node, and the related migration will also be redirected; after the merging operation, an automaton A that can describe the temporal relationship between the events of the interface messages described in the equipment history message record is obtained tr .

7. According to claim 1, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that The specific process of step 3 is as follows: First, use A nl and A tr The Cartesian product of the states in the merged automaton is taken as the state of the merged automaton, and the initial state S of the merged automaton is start Yes A nl and A tr The corresponding initial state S nl and S tr The Cartesian product of nl ,S tr > Then from S start Start targeting A nl and A tr All interface packets in the merged state are constructed and migrated. a ,S b >, facing an interface message p, if A nl Medium S a Able to accept p and move to state S via transition c , A nl Medium S b Able to accept p and move to state S via transition d , then migration occurs a ,S b >→ c ,S d >; If one of the automata cannot accept p, the corresponding part of the automaton in the target state of the migration goes to a non-accepting final state S t ; If neither of the two automata accepts p, then both the left and right parts of the target state jump to the non-accepting final state; in the state of the merged automaton, as long as one of the corresponding components of the automaton is in an accepting state, the state is considered to be an accepting state.​​​​ 8. According to claim 1, an integrated equipment testing and diagnosis method based on intelligent mining of interface models, It is characterized in that Step 5: The specific process is as follows: Step 5.1, interface message session identification: perform fault detection based on the session range; if the time interval between two interface messages exceeds an upper limit range, they are considered to belong to different sessions; if the two interface messages never belong to the same session in the system interface model, they are considered to come from different sessions and do not interfere with each other; Step 5.2, fault detection based on the system interface model: In the same session, fault detection is performed based on the system interface model. The detection process inputs the interface message into the automaton of the model, and the model runs according to the transition corresponding to the interface message. If the transition does not exist, or the transition jumps to a state marked as abnormal, it is considered that a fault is found; Step 5.3, fault location: After the fault is detected, the equipment construction knowledge graph is used to locate the fault-related components; In the equipment construction knowledge graph, the component entity has an excitation and response relationship to the interface entity, and the interface entity has a communication message attribute; Based on this association, the component that sent the message and the expected responder entity of the message can be inferred from the fault message, and the associated component can be located for the front interface message in the current session to form fault detection and diagnosis.

Images