Equipment characteristic knowledge association completion method and system
By introducing memory modules and fusion modules in the large model of the Transformer, the problem of lack of correlation and accuracy in the completion of equipment characteristics knowledge is solved, and a more accurate knowledge complement effect is achieved.
Patent Information
- Application Number
- CN202411362577.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-24
AI Technical Summary
In the complete knowledge of equipment characteristics in the prior art, the output results are not highly correlated with the missing content, the inference results are inaccurate, and the deep intention cannot be accurately captured.
By improving the Transformer large model based on the multi-head self-attention mechanism, a memory module and a fusion module are introduced. The memory module is used to update the memory matrix, and the fusion module is used to fuse the outputs of the Transformer module and the memory module to form a knowledge completion model.
By storing long-term historical knowledge and integrating local self-attention, the accuracy and relevance of the completion information are improved, and more accurate knowledge completion of equipment characteristics is achieved.
Smart Images
Figure CN119358649A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for associating and completing equipment characteristic knowledge. Background Art
[0002] Modern equipment is changing with each passing day, and equipment characteristic data plays a vital role in the success or failure of mission execution. However, due to the current situation of incomplete information, it is difficult to obtain complete characteristic information for some important equipment, which seriously affects the mission deployment and execution capabilities.
[0003] At present, the technology for completing equipment characteristics mainly relies on traditional machine learning, which makes it difficult to use long-term historical information and make comprehensive inferences to provide effective equipment characteristic knowledge completion. The emergence of large models provides a feasible way to integrate historical equipment characteristic text information and perform completion reasoning.
[0004] However, the large model is based on the knowledge base retrieval enhancement method, which only mechanically writes the historical equipment characteristics text into the vector database and obtains the results through retrieval and matching. This processing method does not process the information, resulting in a low level of information processing, and cannot accurately capture the deep intention, resulting in the output results not being highly relevant to the missing content and inaccurate reasoning results. Summary of the invention
[0005] In view of the above analysis, an embodiment of the present invention aims to provide a method and system for associative completion of equipment characteristic knowledge, so as to solve the problem that the existing inferred equipment characteristic knowledge is inaccurate.
[0006] On the one hand, an embodiment of the present invention provides a method for associating and completing equipment characteristic knowledge, comprising the following steps:
[0007] Collect equipment knowledge text data to build an equipment characteristic data set; select shielded parts of equipment characteristic data from the equipment characteristic data set and splice them with preset prompt words as training samples, and put them into the training set;
[0008] The Transformer large model based on the multi-head self-attention mechanism is improved by introducing a memory module and a fusion module; the memory module is used to update the memory matrix based on the processing results of the multi-head self-attention mechanism, and the fusion module is used to fuse the outputs of the Transformer module and the memory module; the improved large model is trained using the training set to obtain a knowledge completion model;
[0009] The complete equipment characteristic knowledge in the equipment characteristic data set is transferred into the knowledge completion model, and the memory matrix of the memory module is updated to obtain the learned knowledge completion model;
[0010] The equipment characteristic knowledge to be completed and the prompt words are passed into the learned knowledge completion model to obtain the completed equipment characteristic knowledge.
[0011] Based on the further improvement of the above method, the Transformer large model based on the multi-head self-attention mechanism includes multiple stacked Transformer modules, a fusion module is added to the original output branch of one or more Transformer modules, and an output branch is added to connect the memory module. The Transformer module outputs the processing result of the multi-head self-attention mechanism to the memory module; the output of the memory module is connected to the fusion module, and the output of the fusion module is passed to the next Transformer module.
[0012] Based on the further improvement of the above method, when using the knowledge completion model, the data identifier is used to control the update of the memory matrix in the memory module, including: when the data identifier is set to true, the memory module updates the memory matrix based on the processing results of the multi-head self-attention mechanism; when the data identifier is set to false, the memory module does not update the memory matrix.
[0013] Based on the further improvement of the above method, the memory module updates the memory matrix through the following formula:
[0014] Mi=f(UMi-1+W(K) T V)
[0015] Among them, M i-1 and M i Respectively represent the memory matrices of the i-1th time step and the i-th time step, U represents the weight matrix for updating the memory matrix, W represents the weight matrix input to the memory module, (K) T V represents the input of the memory module at the i-th time step, K represents the key matrix in the multi-head self-attention mechanism, V represents the value matrix in the multi-head self-attention mechanism, and f(·) represents the activation function.
[0016] Based on the further improvement of the above method, the memory matrix is initialized with random numbers in the range of (0,1), and the activation function adopts the Softmax activation function or the ReLU activation function.
[0017] Based on the further improvement of the above method, the fusion module fuses the outputs of the Transformer module and the memory module through the following formula:
[0018] A=sigmoid(γ)⊙(QM)+(1-sigmoid(γ))⊙A t
[0019] Among them, Q represents the query matrix in the multi-head self-attention mechanism, M represents the memory matrix in the memory module, QM represents the output of the memory module, γ represents the fusion coefficient, ⊙ represents the Hadamard product, and A tRepresents the output of the Transformer module.
[0020] Based on the further improvement of the above method, the improved big model is trained with the equipment characteristic data set to obtain the knowledge completion model, including: using the public data set to train the big model before improvement, and solidifying the parameters of the big model after training; introducing the memory module and the fusion module based on the trained big model before improvement to obtain the improved big model, initializing the memory matrix in the memory module, training again with the training set, updating the parameters of the memory module and the fusion module, and obtaining the knowledge completion model after the training.
[0021] Based on the further improvement of the above method, when the complete equipment characteristic knowledge in the equipment characteristic data set is passed into the knowledge completion model, the data flag is set to true and the memory matrix of the memory module is updated; when the equipment characteristic knowledge and questions to be completed are passed into the knowledge completion model, the data flag is set to false to obtain the output text.
[0022] On the other hand, an embodiment of the present invention provides an equipment characteristic knowledge association completion system, including:
[0023] A sample construction module is used to collect equipment knowledge text data to construct an equipment characteristic data set, select a masked part of the equipment characteristic data from the equipment characteristic data set, and then splice it with the preset prompt words as a training sample and put it into the training set;
[0024] The model building module is used to improve the Transformer large model based on the multi-head self-attention mechanism and introduce the memory module and the fusion module. The memory module is used to update the memory matrix based on the processing results of the multi-head self-attention mechanism, and the fusion module is used to fuse the outputs of the Transformer module and the memory module. The improved large model is trained using the training set to obtain the knowledge completion model.
[0025] A knowledge storage module, used to transfer the complete equipment characteristic knowledge in the equipment characteristic data set into the knowledge completion model, update the memory matrix of the memory module, and obtain the learned knowledge completion model;
[0026] The knowledge completion module is used to pass the equipment characteristic knowledge to be completed and the prompt words into the learned knowledge completion model to obtain the completed equipment characteristic knowledge.
[0027] Based on the further improvement of the above system, in the model building module, the Transformer large model based on the multi-head self-attention mechanism includes multiple stacked Transformer modules, a fusion module is added to the original output branch of one or more Transformer modules, and an output branch is added to connect the memory module. The Transformer module outputs the processing result of the multi-head self-attention mechanism to the memory module; the output of the memory module is connected to the fusion module, and the output of the fusion module is passed to the next Transformer module.
[0028] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0029] 1. Long-term historical knowledge is stored through the memory matrix. By integrating the local self-attention and the associated historical knowledge obtained by retrieving the memory matrix, the extracted features are more accurate and the accuracy of the completed information is improved.
[0030] 2. When using the knowledge completion model, two data processing processes are provided. One is that the knowledge processing and storage process can be executed to continuously learn the historically complete equipment characteristic knowledge, thereby realizing the continuous updating of historical knowledge; the other is that the knowledge reasoning and completion process can be executed to complete the knowledge learned by local self-attention by retrieving the learned long-term historical knowledge to obtain related historical knowledge, thereby accurately capturing the deep intention and improving the relevance and accuracy of the completed knowledge.
[0031] 3. The memory module uses a fixed-size memory matrix and updates it iteratively, which avoids occupying a large amount of storage space, optimizes the model's video memory usage, and improves model performance.
[0032] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;
[0034] Figure 1 This is a flow chart of a method for associating and completing equipment characteristic knowledge in Embodiment 1 of the present invention;
[0035] Figure 2 Schematic diagram of improvement of a Transformer module in Embodiment 1 of the present invention;
[0036] Figure 3 This is a schematic diagram of the structure of an equipment characteristic knowledge associative completion system in Example 2 of the present invention. DETAILED DESCRIPTION
[0037] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.
[0038] Example 1
[0039] A specific embodiment of the present invention discloses a method for associating and completing equipment characteristic knowledge, such as Figure 1 As shown, the following steps are included:
[0040] S1. Collect equipment knowledge text data to build an equipment characteristic data set; select shielded parts of equipment characteristic data from the equipment characteristic data set and splice them with preset prompt words as training samples, and put them into the training set;
[0041] It should be noted that the equipment knowledge texts collected from home and abroad are used to extract equipment characteristic data from the equipment knowledge text data, which may include: country, equipment product model, various performance indicators and general descriptions.
[0042] For example, a piece of equipment characteristic data is: an ABW model aircraft of country T, with a speed of 56, an attitude accuracy of 7.024283473972967e-06, an attitude angular velocity of 30, a maximum range of 2784, a minimum range of 2110, a maximum altitude of 7829, a minimum altitude of 4078, a position accuracy of 7.622469929455012e-06, a landing inclination angle of 10, a pitch frame angle of 2, and a heading frame angle of 73.
[0043] In the equipment characteristic data set, a certain proportion of domestic equipment characteristic data is selected, and after blocking part of the content, it is spliced with the preset prompt words as training samples and put into the training set. The test set is constructed according to the same method, and foreign equipment characteristic data is selected to construct the verification set. These are common usages and will not be explained separately.
[0044] It should be noted that the preset prompt words are used to clearly specify the output direction and content focus of the improved large model, prompting the large model to generate more accurate and comprehensive equipment characteristic knowledge. For equipment characteristic knowledge completion, the use of professional prompt words can enable the large model to better understand the task requirements and provide more professional output.
[0045] Exemplarily, the prompt is: As an equipment characteristic knowledge expert, please supplement and improve the equipment characteristic knowledge parameters.
[0046] S2. Improve the Transformer large model based on the multi-head self-attention mechanism, and introduce a memory module and a fusion module; the memory module is used to update the memory matrix based on the processing results of the multi-head self-attention mechanism, and the fusion module is used to fuse the outputs of the Transformer module and the memory module; use the equipment characteristic data set to train the improved large model to obtain the knowledge completion model.
[0047] It should be noted that the Transformer large model based on the multi-head self-attention mechanism is usually composed of multiple Transformer modules stacked together. The complete Transformer module includes an encoder and a decoder, but in the application scenarios of text generation or language modeling, only the decoder part of the Transformer may be used. For example, the GPT series model uses a large number of Transformer decoder layers. Each Transformer decoder layer includes a multi-head self-attention mechanism, which uses a left-to-right masked self-attention mechanism to predict the next word or token in an autoregressive manner. In the application scenarios of feature extraction or encoding, only the encoder part of the Transformer may be used, and the encoder is used to process the input data and generate a fixed-length output vector, which is then used for downstream tasks such as text classification or named entity recognition.
[0048] This embodiment improves the large model by introducing a memory module and a fusion module, mainly by adding a processing process to the multi-head attention mechanism in the Transformer module. Therefore, it is not limited to whether each Transformer module uses the complete encoder-decoder result or only the decoder; nor is it limited to the number of improved Transformer modules and their positions in the large model.
[0049] Taking the improvement of a Transformer module as an example, the improved model structure is as follows Figure 2 As shown in the figure: a fusion module is added to the original output branch of the Transformer module, and an output branch is added to connect the memory module. The Transformer module outputs the processing results of the multi-head self-attention mechanism to the memory module; the memory module includes a memory matrix, which continuously learns new knowledge with time steps and iteratively updates the stored learned knowledge information; the output of the memory module is connected to the fusion module, and the fusion module fuses the outputs of the Transformer module and the memory module and passes them to the next module.
[0050] Specifically, there are three weight matrices in the Transformer model: query weight matrix W Q , key weight matrix W K Sum value weight matrix W V, these weight matrices are the parameters learned during model training. The input text is converted into word embedding vectors by the word embedding layer. The multi-head self-attention mechanism multiplies the embedding vectors with the three weight matrices to obtain the query matrix Q, key matrix K and value matrix V corresponding to each head.
[0051] Furthermore, for each self-attention mechanism, the query matrix is multiplied by the transpose of the key matrix to obtain an attention score matrix, which is used to represent the attention intensity of each element in the input to other elements. In order to obtain a stable gradient, the score is scaled by dividing by the square root of the key vector dimension. Finally, the softmax function is applied to the scaled score to convert the score into a probability distribution to obtain the attention weight. The value matrix is weighted and summed using the attention weight to obtain the output of each attention mechanism. The formula is as follows:
[0052]
[0053] Among them, A t,j Represents the output of the j-th head self-attention mechanism.
[0054] The outputs of the multi-head self-attention mechanism are concatenated and mapped through a linear layer to integrate the information learned by different heads to obtain the final output A of the multi-head self-attention mechanism. t , the formula is as follows:
[0055] A t =Concat(A1,A2,…,A h )W O (2)
[0056] Among them, W O is a trainable weight matrix used to map the concatenated output back to the original dimension.
[0057] The output of the multi-head self-attention mechanism is connected to the input residual and then normalized to obtain the output of the Transformer model.
[0058] It should be noted that the processing results of the multi-head self-attention mechanism in the Transformer module include the query matrix Q, the key matrix K and the value matrix V. These three matrices are passed to the memory module. The memory matrix in the memory module is initialized with a random number in the range of (0,1). The memory matrix is updated based on the processing results of the multi-head self-attention mechanism using the following formula:
[0059] Mi=f(UMi-1+W(K) T V) (3)
[0060] Among them, M i-1 and Mi Represent the memory matrices of the i-1th time step and the i-th time step respectively, U represents the weight matrix for updating the memory matrix, W represents the weight matrix input to the memory module, (K) T V represents the input of the memory module at the i-th time step, K represents the key matrix in the multi-head self-attention mechanism, V represents the value matrix in the multi-head self-attention mechanism, and f(·) represents the activation function, which uses the Softmax activation function or the ReLU activation function.
[0061] It can be seen from the above formula that the memory matrix continuously learns new knowledge, iteratively updates based on the data of the previous time step, and compresses and stores the learned knowledge, which does not occupy a large amount of storage space and improves the model performance.
[0062] It should be noted that the memory module uses the query matrix in the multi-head self-attention mechanism to retrieve the updated memory matrix, and passes the retrieved memory association knowledge as output to the fusion module. The fusion module fuses the outputs of the Transformer module and the memory memory module through the following formula:
[0063] A=sigmoid(γ)⊙(QM)+(1-sigmoid(γ))⊙A t (4)
[0064] Among them, Q represents the query matrix in the multi-head self-attention mechanism, M represents the memory matrix output by the memory module, QM represents the memory-related knowledge retrieved from the memory matrix, γ represents the fusion coefficient, ⊙ represents the Hadamard product, and A t Represents the output of the Transformer module; the value of sigmoid(γ) determines the proportion of the retrieved memory-associated knowledge and the local attention output of the original Transformer in the final fusion result. As a trainable scalar value, γ enables the improved model to achieve a learnable trade-off between long-term knowledge and local knowledge, thereby improving the ability to complete associated knowledge.
[0065] The improved big model is trained with the equipment characteristic data set to obtain the knowledge completion model, including: using public data sets, such as the global equipment fact database and the Mozi platform data set, to train the big model before improvement, and solidifying the parameters of the big model after training; introducing the memory module and the fusion module based on the trained big model before improvement to obtain the improved big model, initializing the memory matrix in the memory module, training again with the training set, updating the parameters of the memory module and the fusion module, and obtaining the knowledge completion model after the training.
[0066] The training samples in the training set are passed to the improved large model for training, and the next word of the input text is predicted using autoregression. The final output is compared with the real text, and the U and W parameters in the memory module and the γ parameter in the fusion module are updated. After the training is completed, a knowledge completion module with both knowledge storage and knowledge reasoning capabilities is obtained.
[0067] S3. The complete equipment characteristic knowledge in the equipment characteristic data set is transferred into the knowledge completion model, the memory matrix of the memory module is updated, and the learned knowledge completion model is obtained.
[0068] It should be noted that when using the knowledge completion model, data identifiers are used to control the updating of the memory matrix in the memory module to execute different data processing processes.
[0069] Specifically, when the data flag is set to true and a memory module is connected to the Transformer module, the knowledge processing and storage process is executed at this time. It mainly uses the processing results of the memory module based on the multi-head self-attention mechanism to update the memory matrix, continuously learn the equipment characteristic knowledge, and does not pay attention to the output of the large model; when the data flag is set to false, the knowledge reasoning and completion process is executed at this time. The memory matrix in the memory module is no longer updated. Instead, the query matrix is used to directly retrieve the memory knowledge that has been learned in the memory matrix, and the retrieved memory association information is passed to the fusion module. The fusion module combines it with the knowledge learned by the multi-head self-attention mechanism, and finally outputs the accurate content obtained by reasoning to complete the equipment characteristic knowledge.
[0070] Therefore, this step is to set the data flag to true, pass the complete equipment characteristic knowledge in the equipment characteristic data set into the knowledge completion model, use the memory matrix to store the knowledge, and continuously update it as long-term historical knowledge to obtain the learned knowledge completion model, which is convenient for associative retrieval in reasoning and inferring the latest and most relevant knowledge.
[0071] S4. The equipment characteristic knowledge to be completed and the prompt words are passed into the learned knowledge completion model to obtain the completed equipment characteristic knowledge.
[0072] It should be noted that this step is to set the data flag to false, concatenate the equipment characteristic knowledge to be completed with the prompt word and pass it into the learned knowledge completion model, execute the knowledge reasoning completion process, and obtain the completed equipment characteristic knowledge.
[0073] For example, the speed and range data of country B's new equipment CBW are missing. The existing characteristic knowledge of country B's new equipment CBW and the prompt words that need to be completed for speed and range are concatenated and passed into the knowledge completion model to obtain the completed speed and range data from the output text.
[0074] Compared with the prior art, the present embodiment provides an equipment characteristic knowledge associative completion method, which stores long-term historical knowledge through a memory matrix, and makes the extracted features more accurate by fusing local self-attention with the associated historical knowledge obtained by retrieving the memory matrix, thereby improving the accuracy of the completed information. Two data processing processes are provided when using the knowledge completion model, namely, a knowledge processing storage process can be executed to continuously learn historically complete equipment characteristic knowledge, thereby realizing the continuous updating of historical knowledge; and a knowledge reasoning completion process can be executed to retrieve the learned long-term historical knowledge to obtain associated historical knowledge to complete the knowledge learned by local self-attention, accurately capture deep intentions, and improve the relevance and accuracy of the completed knowledge. The memory module uses a fixed-size memory matrix and is updated in an iterative manner, thereby avoiding a large amount of storage space occupation, optimizing the model memory occupation, and improving model performance.
[0075] Example 2
[0076] Another embodiment of the present invention discloses an equipment characteristic knowledge associative completion system, thereby realizing an equipment characteristic knowledge associative completion method in embodiment 1. The specific implementation of each module refers to the corresponding description in embodiment 1. Figure 3 As shown, the system includes:
[0077] The sample construction module 101 is used to collect equipment knowledge text data, construct an equipment characteristic data set, select a masked part of the equipment characteristic data from the equipment characteristic data set, and then splice it with a preset prompt word as a training sample and put it into the training set;
[0078] The model building module 102 is used to improve the Transformer large model based on the multi-head self-attention mechanism, and introduce a memory module and a fusion module; the memory module is used to update the memory matrix based on the processing result of the multi-head self-attention mechanism, and the fusion module is used to fuse the output of the Transformer module and the memory module; the improved large model is trained using the training set to obtain a knowledge completion model;
[0079] The knowledge storage module 103 is used to transfer the complete equipment characteristic knowledge in the equipment characteristic data set into the knowledge completion model, update the memory matrix of the memory module, and obtain the learned knowledge completion model;
[0080] The knowledge completion module 104 is used to pass the equipment characteristic knowledge to be completed and the prompt words into the learned knowledge completion model to obtain the completed equipment characteristic knowledge.
[0081] Furthermore, in the model building module, the Transformer large model based on the multi-head self-attention mechanism includes multiple stacked Transformer modules, a fusion module is added to the original output branch of one or more Transformer modules, and an output branch is added to connect the memory module. The Transformer module outputs the processing result of the multi-head self-attention mechanism to the memory module; the output of the memory module is connected to the fusion module, and the output of the fusion module is passed to the next Transformer module.
[0082] Since the equipment characteristic knowledge associative completion system of this embodiment and the equipment characteristic knowledge associative completion method described above can be mutually referenced, it is a repeated description here, so it will not be repeated here. Since the principles of this system embodiment and the above method embodiment are the same, this system embodiment also has the corresponding technical effects of the above method embodiment.
[0083] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0084] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.
Claims
1. A method for associating and completing equipment characteristic knowledge, characterized in that: The following steps are involved: Collect equipment knowledge text data to build an equipment characteristic data set; select shielded parts of equipment characteristic data from the equipment characteristic data set and splice them with preset prompt words as training samples, and put them into the training set; The Transformer large model based on the multi-head self-attention mechanism is improved, and a memory module and a fusion module are introduced; the memory module is used to update the memory matrix based on the processing result of the multi-head self-attention mechanism, and the fusion module is used to fuse the output of the Transformer module and the memory module; the improved large model is trained using the training set to obtain a knowledge completion model; The complete equipment characteristic knowledge in the equipment characteristic data set is transferred into the knowledge completion model, and the memory matrix of the memory module is updated to obtain the learned knowledge completion model; The equipment characteristic knowledge to be completed and the prompt words are passed into the learned knowledge completion model to obtain the completed equipment characteristic knowledge.
2. The equipment characteristic knowledge associative completion method according to claim 1 is characterized in that: The Transformer large model based on the multi-head self-attention mechanism includes multiple stacked Transformer modules, a fusion module is added to the original output branches of one or more Transformer modules, and an output branch is added to connect the memory module. The Transformer module outputs the processing result of the multi-head self-attention mechanism to the memory module; the output of the memory module is connected to the fusion module, and the output of the fusion module is transmitted to the next Transformer module.
3. The equipment characteristic knowledge associative completion method according to claim 2 is characterized in that: When using the knowledge completion model, the data identifier is used to control the update of the memory matrix in the memory module, including: when the data identifier is set to true, the memory module updates the memory matrix based on the processing results of the multi-head self-attention mechanism; when the data identifier is set to false, the memory module does not update the memory matrix.
4. The equipment characteristic knowledge associative completion method according to claim 1 or 3, characterized in that: The memory module updates the memory matrix by the following formula: Among them, M i-1 and M i Represent the memory matrices of the i-1th time step and the i-th time step respectively, U represents the weight matrix for updating the memory matrix, W represents the weight matrix input to the memory module, (K) T V represents the input of the memory module at the i-th time step, K represents the key matrix in the multi-head self-attention mechanism, V represents the value matrix in the multi-head self-attention mechanism, and f(·) represents the activation function.
5. The equipment characteristic knowledge associative completion method according to claim 4 is characterized in that: The memory matrix is initialized using random numbers in the range of (0, 1), and the activation function uses a Softmax activation function or a ReLU activation function.
6. The equipment characteristic knowledge associative completion method according to claim 1 or 3, characterized in that: The fusion module fuses the outputs of the Transformer module and the memory module through the following formula: A=sigmoid(γ)⊙(QM)+(1-sigmoid(γ))⊙A t Among them, Q represents the query matrix in the multi-head self-attention mechanism, M represents the memory matrix in the memory module, QM represents the output of the memory module, γ represents the fusion coefficient, ⊙ represents the Hadamard product, and A t Represents the output of the Transformer module.
7. The equipment characteristic knowledge associative completion method according to claim 1 is characterized in that: The improved large model trained by the equipment characteristic data set is used to obtain a knowledge completion model, including: using a public data set to train the large model before improvement, and solidifying the parameters of the large model after training; introducing a memory module and a fusion module based on the trained large model before improvement to obtain an improved large model, initializing a memory matrix in the memory module, training again using the training set, updating the parameters of the memory module and the fusion module, and obtaining the knowledge completion model after the training.
8. The method for associating and completing equipment characteristic knowledge according to claim 3, characterized in that: When the complete equipment characteristic knowledge in the equipment characteristic data set is passed into the knowledge completion model, the data identifier is set to true and the memory matrix of the memory module is updated; when the equipment characteristic knowledge and questions to be completed are passed into the knowledge completion model, the data identifier is set to false to obtain the output text.
9. An equipment characteristic knowledge association and completion system, characterized in that: include: A sample construction module is used to collect equipment knowledge text data to construct an equipment characteristic data set, select a masked part of the equipment characteristic data from the equipment characteristic data set, and then splice it with the preset prompt words as a training sample and put it into the training set; A model building module is used to improve the Transformer large model based on the multi-head self-attention mechanism, and introduce a memory module and a fusion module; the memory module is used to update the memory matrix based on the processing result of the multi-head self-attention mechanism, and the fusion module is used to fuse the outputs of the Transformer module and the memory module; the improved large model is trained using the training set to obtain a knowledge completion model; A knowledge storage module, used to transfer the complete equipment characteristic knowledge in the equipment characteristic data set into the knowledge completion model, update the memory matrix of the memory module, and obtain the learned knowledge completion model; The knowledge completion module is used to pass the equipment characteristic knowledge to be completed and the prompt words into the learned knowledge completion model to obtain the completed equipment characteristic knowledge.
10. The equipment characteristic knowledge association and completion system according to claim 9, characterized in that: In the model construction module, the Transformer large model based on the multi-head self-attention mechanism includes multiple stacked Transformer modules, a fusion module is added to the original output branch of one or more Transformer modules, and an output branch is added to connect the memory module. The Transformer module outputs the processing result of the multi-head self-attention mechanism to the memory module; the output of the memory module is connected to the fusion module, and the output of the fusion module is transmitted to the next Transformer module.