Knowledge intelligent classification method

Abstract

This paper discloses an intelligent knowledge classification method. It retrieves relevant process knowledge graphs from external knowledge sources, treats the conceptual information within them as a type of knowledge, and integrates it into a deep neural network. To assess the importance of knowledge, an attention mechanism is introduced, and a knowledge-driven data classification method applicable to process knowledge graphs is proposed. This method utilizes the attention of concepts to text and concepts to concept sets to obtain concept weights from two perspectives. Knowledge is then classified using this conceptual information. Unlike traditional methods, the algorithm of this paper has the inherent ability to make decisions based on external knowledge sources and focuses more on important knowledge.

Description

Technical Field

[0001] The present invention relates to the field of knowledge data classification, and in particular to a knowledge intelligent classification algorithm. Background Art

[0002] In today's era of information explosion, we face an immense amount of data, especially in the field of process engineering. This data contains a wealth of process knowledge, including but not limited to material properties, process flows, equipment parameters, and quality control. As the demand for knowledge increases, we need to extract and learn process knowledge from this massive data, eliminating redundant and invalid information. Once this knowledge is acquired and learned, effectively preserving and managing it becomes a critical issue.

[0003] One effective knowledge management method is to classify knowledge. This classification process requires specific requirements and standards. Through comparison, acquired knowledge is divided into different knowledge systems based on similarities, differences, and correlations. For example, process knowledge can be classified according to different attributes such as process type, material type, and equipment type to form a hierarchical process knowledge system. This allows learned process knowledge to be systematically stored in a knowledge base and retrieved at any time when needed.

[0004] This is just the first step. Only by successfully completing this first step can the subsequent process knowledge edifice be firmly established. Building upon this foundation, the research direction of this invention is to leverage modern artificial intelligence technologies to more efficiently and intelligently classify and manage process knowledge. We will employ artificial intelligence methods to classify process knowledge, such as machine learning to automatically categorize process knowledge or natural language processing to extract and classify knowledge from unstructured process text. These technologies will enable better utilization and sharing of process knowledge, laying a solid foundation for subsequent process innovation and optimization.

[0005] Existing knowledge classification algorithms have two major problems: (1) High sensitivity to data: For some basic algorithms, decision trees, KNN algorithms, SVM, AdaBoost, Naive Bayes, Logistic regression and other algorithms are easy to understand and implement, but the classification accuracy is not high, and the sample data requirements are high. If the data is missing or the data is unbalanced, the results will be greatly affected. Therefore, neural networks are widely used, but some simple algorithms such as NLP natural language processing and convolutional neural networks (CNN) cannot meet the classification capabilities of big data at this stage. At the same time, processing text data and image data separately will also increase the computing cost. Therefore, this patent design changes this data sensitivity and uses a deep learning network that combines CNN and attention mechanism. Both image data and text data can be classified through our algorithm model, which will reduce the input data requirements of the classification algorithm and improve the efficiency of knowledge classification. (2) No prior knowledge is applied: At this stage, there is more and more knowledge data, and the correlation between knowledge is also getting higher and higher. How to associate this part of knowledge and how to associate it has become a challenge today. When we receive knowledge data, we unconsciously draw on background knowledge such as spatial relationships, causal relationships, scientific facts, and social customs. Once we have this relevant knowledge base, it helps us categorize the current knowledge more clearly. To address this issue, we propose using external knowledge sources to help us acquire knowledge concepts. Specifically, when we acquire a piece of knowledge, we use external knowledge sources to find a set of knowledge concepts. By performing attention calculations on the concept set and the original data, and on each concept, on the concept set, the model learns information about the original data, each concept, and the concept set. Through this concept set, the algorithm can learn prior knowledge related to the original data. We expect the model to leverage this external knowledge to make correct judgments and provide explicit, interpretable evidence that can be subsequently utilized. This not only helps us classify knowledge more accurately and efficiently, but also facilitates subsequent analysis and exploration. Summary of the Invention

[0006] To address the above problems, the present invention retrieves relevant process knowledge graphs from external knowledge sources and integrates the conceptual information therein into a deep neural network as a kind of knowledge. At the same time, in order to evaluate the importance of knowledge, an attention mechanism is introduced, and a knowledge-driven data classification method that can be used for process knowledge graphs is proposed.

[0007] In order to achieve the above object, the present invention provides a knowledge intelligent classification method, which includes:

[0008] S1: Obtain various types of structured and unstructured documents containing process knowledge graphs and extract words from the documents;

[0009] S2: Use the continuous bag-of-words model to map all words in the text into a high-dimensional vector space, and obtain the dimension of the target word as D. Specifically, the following steps are included:

[0010] S201: Randomly initialize word vectors;

[0011] S202: Mapping context words in the text to word vectors;

[0012] S203: averaging the context word vectors to obtain a context vector;

[0013] S204: Input the context vector into the softmax layer to calculate the probability of each word in the vocabulary. The calculation formula of the softmax layer is:

[0014]

[0015] Where x i is the i-th context vector;

[0016] S205: Update the word vector using the cross entropy loss function. The calculation formula is:

[0017]

[0018] Where y i is the label corresponding to the i-th word, is the label corresponding to the predicted i-th word;

[0019] S206: Repeat steps S202-S205 multiple times until the model converges or reaches a preset number of iterations;

[0020] S207: Output the word vector of the target word, with a dimension of D;

[0021] S3: Acquire various concepts of the knowledge base;

[0022] S4: Input the knowledge base concept into S2 and obtain a concept vector with a dimension of D;

[0023] S5: Use the convolutional neural network (CNN) model to map all words in the text into a high-dimensional vector space and obtain character vectors with a dimension of D.

[0024] S6: directly connect the word vector to the character vector to form a 2D vector as the total word vector;

[0025] S7: directly connect the concept vector to the character vector to form a vector with a dimension of 2D as the total concept vector;

[0026] S8: Calculate the text representation of the total word vector, which can be expressed as (x1, x2, ... x n ) is a 2D word vector sequence, where n is the length of the total word vector. Specifically, the following steps are included:

[0027] S801: The total word vector x t (t=1,…,n) is input into the LSTM model to obtain a hidden state h t , the number of hidden units is u, and the LSTM calculation formula is as follows:

[0028] h t =LSTM(x t ,h t-1 )=o t tanh(C t )

[0029] in:

[0030]

[0031] o t represents the output of the output gate at time t, W o represents the output gate weight, b o represents the output gate bias, f t Represents the output of the forget gate at time t, W f represents the weight of the forget gate, b f Represents the forget gate bias, i t represents the output of the input gate at time t, W i represents the weight of the forget gate, b i represents the forget gate bias, C t Represents the cell layer state at time t in LSTM, represents the intermediate state of the cell layer at time t in LSTM, W C represents the cell weight, b C represents the forget gate bias;

[0032] Finally, we get the matrix of all hidden states

[0033]

[0034] S802: Calculate the matrix The attention mechanism finally gets the output matrix Select the largest one in this matrix as the text representation The calculation formula of the attention mechanism is as follows:

[0035]

[0036] S9: Calculate the attention weight α of the concept to the text relevance layer i , given a concept set C of size m, which consists of concept vectors, denoted as (c1,c2,…,c m ), where c i is the i-th concept vector;

[0037]

[0038] Where, α i For the i-th concept c i The weight of attention to text q; tanh is the hyperbolic tangent transformation function, softmax is used to normalize the attention weight of each concept, W1 is the weight matrix, is the weight vector, b1 is the offset;

[0039] S10: Calculate the attention weight β of the concept to concept set relevance layer i ;

[0040]

[0041] Where, β i For the i-th concept c i The attention weight of the entire concept set, W2 is the weight matrix, is the weight vector, b2 is the offset;

[0042] S11: Calculate the final attention weight a of the concept for the text i ;

[0043] a i =softmax(γα i +(1-γ)β i )

[0044] Where a i represents the final attention weight of the i-th concept for the text; γ∈{0,1}, which is a “soft switch” for adjusting the weights of the two, γ=σ(w T [α; β]+b), w and b are the parameters to be learned, and σ is the sigmoid function;

[0045] S12: Calculate the weighted sum of the concept vectors to obtain the semantic vector p representing the concept;

[0046]

[0047] Where c i represents the i-th concept, and m represents a total of m concepts;

[0048] S13: Concatenate the text representation q and the concept representation p to obtain input;

[0049] input=concat(p,q)

[0050] S14: Map the key feature representation from the high-dimensional latent space vector to the low-dimensional label space through a fully connected operation on the input. The calculation formula is as follows:

[0051] output=softmax(f(w3·input+b3))

[0052] Where w3 is the weight vector, b3 is the offset, and finally the text is normalized and classified by the softmax classifier to obtain the corresponding label result of the text;

[0053] S15: Compare the label results of each category with the actual label based on the loss function The gradient of each parameter is calculated, and then the network parameters are iteratively updated by the gradient descent method to minimize the loss function.

[0054] Preferably, the unstructured file type includes at least one of the following: Word, Excel, PPT and PDF file types.

[0055] In a preferred manner, the performance of the model is measured using the evaluation index accuracy in the test set. The formula for the evaluation index accuracy is:

[0056] P=TP / (TP+FP)

[0057] Where P is the accuracy, also known as precision, which represents the proportion of samples predicted to be positive that are actually positive; TP is the true positive example, which represents the number of samples predicted by the model to be positive and are actually positive; FP is the false positive example, which represents the number of samples predicted by the model to be positive but are actually negative.

[0058] Preferably, the knowledge base includes at least one of the following: YAGO, Probase and CNProbase.

[0059] The beneficial effects of the present invention are as follows: by integrating conceptual information in an external data source, the present invention not only enhances the representation of data, but also improves the accuracy and efficiency of knowledge acquisition; by introducing two attention mechanisms, the importance of knowledge is measured from multiple aspects, and a suitable weight is adaptively assigned to each knowledge of different texts, which can more accurately understand and parse text data; finally, according to the output class label probability of the original data and its related concepts, the knowledge is classified more accurately and efficiently; not only that, the present invention also has the inherent ability to make decisions based on observation, which enables a deeper understanding of the data and more attention to important knowledge; therefore, the method of the present invention not only improves the accuracy of knowledge classification, but also improves the efficiency of knowledge processing, and is of great value for practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is the logical relationship diagram of the text presentation layer of the present invention;

[0061] Figure 2 This is the algorithm model diagram of the present invention. DETAILED DESCRIPTION Specific embodiment:

[0063] S1: Get data

[0064] We need to acquire various structured and unstructured files containing the knowledge of the process knowledge graph. This includes structured data, Word, Excel, PPT, and PDF files. These files contain various aspects of the process knowledge graph, such as process flow, equipment information, operating parameters, and raw material characteristics.

[0065] By collecting this data, we can better understand the inherent structure and relationships of the process knowledge graph. These data sources may contain a large amount of text information, which contains rich process knowledge, such as process descriptions, process parameter explanations, and equipment usage guides. To fully utilize this knowledge, we need to extract text data from these files.

[0066] When extracting text data from files, it's important to consider the characteristics and parsing methods of different file formats. For example, to extract text from Word, Excel, and PPT files, you can use corresponding file parsing libraries, such as Python's python-docx, openpyxl, and python-pptx. For PDF files, you can use PDF parsing libraries like PyPDF2 or pdfminer.six to extract text content. During the text data extraction process, you need to address potential formatting and encoding issues to ensure that the extracted text data is high-quality and readable.

[0067] Corresponding parsing library source:

[0068] 1. python-docx: This is a Python library for working with Microsoft Word / .docx files. It allows you to easily read, query, and modify the contents of Word files. The official GitHub repository for this project is: https: / / github.com / python-openxml / python-docx.

[0069] 2. openpyxl: This is a Python library for reading and writing Microsoft Excel / .xlsx files. It allows you to manipulate data within Excel worksheets, rows, and cells. The official GitHub repository for this project is: https: / / github.com / chronossc / openpyxl.

[0070] 3. python-pptx: This is a Python library for working with Microsoft PowerPoint / .pptx files. It provides functionality for creating, modifying, and parsing PPT files. The official GitHub repository for this project is: https: / / github.com / scanny / python-pptx.

[0071] 4. PyPDF2: This is a Python library for working with PDF files. It allows you to extract text from PDF files, merge and split PDF files, encrypt and decrypt PDF files, and more. The official GitHub repository for this project is: https: / / github.com / mstamy2 / PyPDF2.

[0072] 5.pdfminer.six: This is a Python library for processing PDF files, providing functionality for extracting text, images, and other content from PDF files. It is a Python 3 version of the Python 2-based PDFMiner library. The project's official GitHub repository is: https: / / github.com / pdfminer / pdfminer.six .

[0073] In this study, we used all the text data contained in Chapter 4 of the PDF file "Basic Terminology of Mechanical Manufacturing Processes," part of the National Standard of the People's Republic of China (GBT 4863-2008), covering typical surface finishing terms. These data are divided into the following eight categories.

[0074] The following is a summary of the names of the 8 categories:

[0075] 1. Hole processing: including drilling, reaming, countersinking, turning, boring, milling, pulling, pushing, inserting, grinding, honing, lapping, scraping, squeezing, rolling, punching, laser drilling, ultrasonic drilling, electric spark drilling, and electron beam drilling. A total of 21 types.

[0076] 2. External cylindrical machining: including external cylindrical turning, external cylindrical grinding, external cylindrical lapping, external cylindrical polishing, and external cylindrical rolling. A total of 6 types.

[0077] 3. Surface processing: including milling, planing, milling, grinding, honing, spot facing, scraping, pulling, lapping, grinding, and polishing. A total of 10 types.

[0078] 4. Grooving: including grooving, milling, planing, slotting, drawing, pushing, boring, grinding, lapping, rolling, and scraping. A total of 11 types.

[0079] 5. Thread processing: including thread turning, thread combing, thread milling, thread whirling, thread rolling, thread rubbing, thread pulling, thread tapping, thread inserting, thread grinding, thread honing, and thread lapping. A total of 12 types.

[0080] 6. Tooth surface processing: including gear milling, gear hobbing, gear shaping, gear planing, gear shaving, gear honing, gear lapping, gear grinding, gear broaching, gear rolling, gear extrusion, gear punching, and gear casting. A total of 13 types.

[0081] 7. Surface processing: including turning, milling, polishing, planing, grinding, and electro-machining. There are 6 types in total.

[0082] 8. Others: including knurling, chamfering, filleting, center hole drilling, center hole grinding, center hole lapping, center hole extrusion, and segment cutting. A total of 8 types.

[0083] Each piece of data includes Chinese name, English name, and text description.

[0084] The following is an example of a piece of data:

[0085] Chinese name: drilling; English name: drilling: drilling from the solid

[0086] Text description: A method of drilling holes in solid materials using a drill.

[0087] When performing specific text classification tasks, text descriptions are selected as research objects.

[0088] There are 87 data items in total. Each data item can be divided into multiple words or multiple characters.

[0089] Characters: the method of using a drill bit to make a hole in a solid material.

[0090] Words: drill, drill bit, entity, material, processing hole, method.

[0091] S2: Get character vector

[0092] Because text consists of many characters, the primary task of this layer is to map each character in the text into a high-dimensional vector space, providing a numerical representation for text analysis and modeling. Convolutional neural networks (CNNs) are used to capture local information between characters, thereby obtaining character-level representations for each word. For more information on CNNs, refer to the book Deep Learning by Ian Goodfellow, Joshua Bengio, and Aaron Courville, published by Posts and Telecommunications Press. 7.9.1 Concepts

[0093] In this process, we first embed each character into a low-dimensional vector using the fully connected function input = w0x + b0, where w0 is the weight matrix and b0 is the bias. The input x is the acquired character data; the output input is the low-dimensional vector of the character.

[0094] The low-dimensional character vectors are input to the convolutional layer Conv1d, which captures potential local features and patterns across multiple characters. The output of the convolutional layer then enters the MaxPool layer, which performs a full-width max pooling operation to map each word into a high-dimensional vector space. The resulting character vector has a dimension of D.

[0095] Character embedding=MaxPool(Conv1d(input))

[0096] S3: Get word vectors

[0097] Each word in each text data obtained in step S1 is processed through method A to obtain the word vector of each word, and the word vector dimension is D.

[0098] Method A:

[0099] The method is called word2vector model, and is based on the paper Efficient Estimation of Word Representations in Vector Space. Specifically, it uses the Continuous Bag of Words (CBOW) model, and the steps are S301-S307:

[0100] S301: Randomly initialize word vectors, i.e., weight matrices;

[0101] S302: Mapping the input context word to a word vector, i.e., searching for a weight matrix;

[0102] S303: averaging the context word vectors to obtain a context vector;

[0103] S304: Pass the context vector into the softmax layer to calculate the probability of each word in the vocabulary;

[0104] S305: Update the word vector by maximizing the target word probability, i.e., the cross entropy loss function;

[0105] S306: Repeat steps 2-5 multiple times until the model converges or reaches a preset number of iterations;

[0106] S307: Output the target word including the word vector and concept vector. In this step, the word vector is obtained.

[0107] S4: Acquiring the knowledge base

[0108] We searched the three knowledge bases, YAGO, Probase and CNProbase, to obtain the conceptual information related to the process knowledge graph as the knowledge base.

[0109] 1. YAGO, Yet Another Great Ontology: YAGO is a large-scale knowledge base containing a large number of entities and semantic relations. See the paper YAGO: A Core of Semantic Knowledge.

[0110] 2. Probase: Probase is a knowledge base developed by Microsoft Research Asia, built on a concept hierarchy. See the paper Probase: A Probabilistic Taxonomy for Text Understanding.

[0111] 3. CN-Probase: CN-Probase is a large-scale knowledge base for Chinese. It is built on Probase and is compatible with Probase's data structure and interface. See the paper CN-Probase: A Large-Scale Chinese Taxonomy Knowledge Base.

[0112] S5: Get concept vector

[0113] We use method A to obtain the concept vector of each concept in the knowledge base obtained in step S4, and the dimension of the concept vector is D.

[0114] Method A:

[0115] The method is called word2vector model, and is based on the paper Efficient Estimation of Word Representations in Vector Space. Specifically, it uses the Continuous Bag of Words (CBOW) model, and the steps are S501-S507:

[0116] S501: Randomly initialize word vectors, i.e., weight matrices;

[0117] S502: Map the input context word to the word vector, i.e., look up the weight matrix;

[0118] S503: averaging the context word vectors to obtain a context vector;

[0119] S504: Pass the context vector into the softmax layer to calculate the probability of each word in the vocabulary;

[0120] S505: Update the word vector by maximizing the target word probability, i.e., the cross entropy loss function;

[0121] S506: Repeat steps 2-5 multiple times until the model converges or reaches a preset number of iterations;

[0122] S507: Output the target word including the word vector and concept vector. In this step, the concept vector is obtained.

[0123] S6: Get the total word vector

[0124] Then, we concatenate the word vector obtained by S3 and the character vector obtained by S2 in the dimension D direction into a 2D vector as the total word vector.

[0125] S7: Get the total concept vector

[0126] Similarly, we concatenate the concept vector obtained by S55 and the character vector obtained by S2 in the dimension D direction into a 2D vector as the total concept vector.

[0127] S8: Text presentation layer

[0128] The total length of the word vector obtained by S6 is n, and the text is represented as (x1, x2, ..., x n ) is a 2D word vector sequence.

[0129] This layer is divided into two steps S801 and S802:

[0130] S801: Input the total word vector obtained in S6 into the long short-term memory network LSTM model, refer to the concept of 10.10.1 in the book Deeplearning, and obtain the hidden state matrix. t (t=1,…,n) is input into the LSTM model to obtain a hidden state h t , the number of hidden units is u,

[0131] h t =LSTM(x t ,h t-1 )=o t tanh(C t )

[0132] Combine all hidden states to get the final matrix of all hidden states

[0133]

[0134] S802: Calculate the attention mechanism for all hidden state matrices obtained in S801, refer to the paper Attention is all you need.

[0135]

[0136] Finally, the output matrix Finally, the largest one is selected in this matrix as the text representation, and the text representation is

[0137] S9: Concept and text relevance layer

[0138] The total length of the concept vector obtained by S7 is m, and the concept is represented as (c1, c2, ..., c m ), where c i is the i-th concept vector, and q is the encoded text representation obtained by S8.

[0139]

[0140] α i Represents the i-th concept c i The weight of attention to text q is larger, indicating that the concept is more relevant to the semantics of the text. Tanh is the hyperbolic tangent transformation function, and softmax is used to normalize the attention weight of each concept. W1 is the weight matrix, is the weight vector and b1 is the offset.

[0141] S10: Concept and concept set relevance layer

[0142] This layer calculates the importance of each concept in the concept set. The effect of this module is similar to that of feature selection. It is a "soft" feature selection that assigns larger weights to important concepts and smaller, near-zero weights to common concepts.

[0143]

[0144] β i For the i-th concept c i The attention weight of the entire concept set, W2 is the weight matrix, is the weight vector, b2 is the offset;

[0145] S11: Conceptual Representation Layer

[0146] The α obtained by S9 i and β obtained by S10 i To combine:

[0147] a i =soft max(γα i +(1-γ)β i )

[0148] a i Represents the final attention weight of the i-th concept for the text; γ∈{0,1}, which is a "soft switch" for adjusting the weights of the two, which participates in the training of the neural network and is automatically adjusted. γ=σ(w T [α; β]+b), w and b are the parameters to be learned, σ is the sigmoid function; α is all α i , β is all β i .

[0149] Calculate the weighted sum of the concept vectors to obtain the semantic vector p representing the concept;

[0150]

[0151] c i Represents the i-th concept, and m represents a total of m concepts.

[0152] S12: Fully connected input layer

[0153] Concatenate the text representation q and the concept representation p to obtain input;

[0154] input=concat(p,q)

[0155] S13: Fully connected output layer

[0156] The input is input through a fully connected operation to map the key feature representation from the high-dimensional latent space vector to the low-dimensional label space. The calculation formula is as follows:

[0157] output=softmax(f(w3·input+b3))

[0158] In the formula, w3 is the weight vector, b3 is the offset, and finally the normalized text classification is performed through the softmax classifier to obtain the corresponding label result of the text.

[0159] S14: Training Model

[0160] In this phase, we use a labeled dataset for training. We need to compare the labels of each category obtained by the S13 output layer with the actual labels, and then perform gradient updates through the backpropagation algorithm, referring to the concept 6.5.8 in the Deeplearning book, to optimize the model parameters.

[0161] Specifically, we typically use a loss function such as cross entropy to calculate the difference between the model's predicted value and the actual value, as discussed in Section 5.5 of the Deep Learning book. We then use an optimizer such as Adam to update the model's weights and biases, as discussed in the paper Adam: A Method for Stochastic Optimization.

[0162] During the training process, we also need to perform model validation to prevent overfitting. Refer to the concept of 5.2 in the book Deeplearning. We can evaluate the performance of the model on the validation set and then perform early stopping as needed. Refer to the concept of 10.2.3 in the book Deep learning.

[0163] S15: Test Model

[0164] After model training is complete, we need to evaluate it on the test set to determine the final performance of the model. During this process, we should ensure that the data in the test set has never been used in the training phase to ensure the fairness and impartiality of the evaluation results. The performance of the model is measured using the accuracy evaluation metric in the test set. The formula for the accuracy evaluation metric is:

[0165] P=TP / (TP+FP)

[0166] Where P is the accuracy, also known as precision, which represents the proportion of samples predicted to be positive that are actually positive; TP is the true positive example, which represents the number of samples predicted by the model to be positive and are actually positive; FP is the false positive example, which represents the number of samples predicted by the model to be positive but are actually negative.

[0167] S16: Application Model

[0168] Finally, we apply the trained model to actual tasks. For example, we can use the model to automatically extract process knowledge, identify concepts, link concepts, and reason about process knowledge.

[0169] When applying the model, we can also fine-tune it as needed. Refer to Concept 11.5.3 in the Deep Learning book. Specifically, we can use a small amount of labeled data for additional training to optimize model performance for a specific task or dataset. Overall, this is a complete process from data acquisition to model application, involving many specific steps and methods. Each step contains rich information about the process knowledge graph, which is key to understanding and optimizing the process knowledge graph.

[0170] Each piece of data includes Chinese name, English name, and text description.

[0171] When performing specific text classification tasks, text descriptions are selected as research objects.

[0172] This experiment is carried out through the method of steps S1-S13. Then, the model is trained, tested and applied through steps S14-S16.

[0173] In this study, the dataset was divided into training set, test set, and validation set in a ratio of 8:1:1, and all data were shuffled.

[0174] By counting the length of hidden danger text, the sentence length of each sample is set to 30 characters. Sentences with a length of less than 30 characters are used. <0> The Adam optimizer is used as the model optimizer, and the learning rate determines the step size of the weight iteration.

[0175] After calculation by the model, the accuracy of the eight categories is as follows:

[0176] 1. Hole processing: accuracy rate 92%

[0177] 2. External cylindrical machining: accuracy rate 90%

[0178] 3. Plane processing: accuracy rate 91%

[0179] 4. Groove processing: accuracy rate 93%

[0180] 5.Thread processing: accuracy rate 88%

[0181] 6. Tooth surface processing: accuracy rate 89%

[0182] 7. Forming surface processing: accuracy rate 94%

[0183] 8. Other: 91% accuracy

[0184] 9. Overall average accuracy: 91%

[0185] The key point of this invention is that we fully utilize the prior knowledge in external data sources and incorporate it as explicit features into the deep learning network, thereby greatly improving the accuracy and efficiency of the knowledge classification algorithm of the process knowledge graph. This deep learning network, combined with prior knowledge, can more deeply understand and analyze the data, while more accurately identifying and extracting important features of the knowledge in the process knowledge graph. In addition, our method breaks through the limitations of traditional methods that rely solely on internal data information. By introducing external knowledge sources, the expressiveness of the data is significantly improved.

[0186] The key point to be protected by the present invention is that we not only enrich the data information with the help of external data sources, but also innovatively introduce two attention modules on this basis. These two attention modules can comprehensively evaluate and measure the importance of knowledge from different angles and levels, thereby adaptively assigning appropriate weights to each piece of knowledge. This efficient and intelligent knowledge classification capability enables our method to maintain excellent performance and effects when processing large amounts of complex data. This knowledge classification method that combines external knowledge sources and attention modules is the core innovation of the present invention and provides a solid foundation for the knowledge classification technology of our process knowledge graph.

[0187] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A knowledge intelligent classification method, characterized in that: It includes: S1: Obtain various types of structured and unstructured documents containing process knowledge graphs and extract words from the documents; S2: Use the continuous bag-of-words model to map all words in the text into a high-dimensional vector space, and obtain the dimension of the target word as D. Specifically, the following steps are included: S201: Randomly initialize word vectors; S202: Mapping context words in the text to word vectors; S203: averaging the context word vectors to obtain a context vector; S204: Input the context vector into the softmax layer to calculate the probability of each word in the vocabulary. The calculation formula of the softmax layer is: Where x i is the i-th context vector; S205: Update the word vector using the cross entropy loss function. The calculation formula is: Where y i is the label corresponding to the i-th word, is the label corresponding to the predicted i-th word; S206: Repeat steps S202-S205 multiple times until the model converges or reaches a preset number of iterations; S207: Output the word vector of the target word, with a dimension of D; S3: Acquire various concepts of the knowledge base; S4: Input the knowledge base concept into S2 and obtain a concept vector with a dimension of D; S5: Use the convolutional neural network (CNN) model to map all words in the text into a high-dimensional vector space and obtain character vectors with a dimension of D. S6: directly connect the word vector to the character vector to form a 2D vector as the total word vector; S7: directly connect the concept vector to the character vector to form a vector with a dimension of 2D as the total concept vector; S8: Calculate the text representation of the total word vector, which can be expressed as (x1, x2, ... x n ) is a 2D word vector sequence, where n is the length of the total word vector. Specifically, the following steps are included: S801: The total word vector x t (t=1,…,n) is input into the LSTM model to obtain a hidden state h t , the number of hidden units is u, and the LSTM calculation formula is as follows: h t =LSTM(x t ,h t-1 )=o t ·tanh(C t ) in: o t represents the output of the output gate at time t, W o represents the output gate weight, b o represents the output gate bias, f t Represents the output of the forget gate at time t, W f represents the weight of the forget gate, b f represents the forget gate bias, i t represents the output of the input gate at time t, W i represents the weight of the forget gate, b i represents the forget gate bias, C t Represents the cell layer state at time t in LSTM, represents the intermediate state of the cell layer at time t in LSTM, W C represents the cell weight, b C represents the forget gate bias; Finally, we get the matrix of all hidden states S802: Calculate the matrix The attention mechanism finally gets the output matrix Select the largest one in this matrix as the text representation The calculation formula of the attention mechanism is as follows: S9: Calculate the attention weight α of the concept to the text relevance layer i , given a concept set C of size m, which consists of concept vectors, denoted as (c1,c2,…,c m ), where c i is the i-th concept vector; Where, α i For the i-th concept c i The weight of attention to text q; tanh is the hyperbolic tangent transformation function, softmax is used to normalize the attention weight of each concept, W1 is the weight matrix, is the weight vector, b1 is the offset; S10: Calculate the attention weight β of the concept to concept set relevance layer i ; Where, β i For the i-th concept c i The attention weight of the entire concept set, W2 is the weight matrix, is the weight vector, b2 is the offset; S11: Calculate the final attention weight a of the concept for the text i ; a i =soft max(ca i +(1-c)b i ) Where a i represents the final attention weight of the i-th concept for the text; γ∈{0,1}, which is a "soft switch" for adjusting the weights of the two, γ=σ(w T [α; β]+b), w and b are the parameters to be learned, and σ is the sigmoid function; S12: Calculate the weighted sum of the concept vectors to obtain the semantic vector p representing the concept; Where c i represents the i-th concept, and m represents a total of m concepts; S13: Concatenate the text representation q and the concept representation p to obtain input; input=concat(p,q) S14: Map the key feature representation from the high-dimensional latent space vector to the low-dimensional label space through a fully connected operation on the input. The calculation formula is as follows: output=softmax(f(w3·input+b3)) Where w3 is the weight vector, b3 is the offset, and finally the text is normalized and classified by the softmax classifier to obtain the corresponding label result of the text; S15: Compare the label results of each category with the actual label based on the loss function The gradient of each parameter is calculated, and then the network parameters are iteratively updated by the gradient descent method to minimize the loss function.

2. The method for intelligent knowledge classification according to claim 1, characterized in that: The unstructured file type includes at least one of the following: Word, Excel, PPT and PDF file types.

3. The method for intelligent knowledge classification according to claim 1, characterized in that: The evaluation indicator accuracy is used in the test set to measure the performance of the model. The formula for the evaluation indicator accuracy is: P=TP / (TP+FP) Where P is the accuracy, also known as precision, which represents the proportion of samples predicted to be positive that are actually positive; TP is the true positive example, which represents the number of samples predicted by the model to be positive and are actually positive; FP is the false positive example, which represents the number of samples predicted by the model to be positive but are actually negative.

4. The method for intelligent knowledge classification according to claim 1, characterized in that: The knowledge base includes at least one of the following: YAGO, Probase and CNProbase.

Images