An entity extraction method for constructing knowledge graph for mechanical fault diagnosis

Abstract

The present invention discloses an entity extraction method for mechanical fault diagnosis knowledge graph construction, including step S1: constructing a mechanical fault diagnosis corpus database and dividing it into a training set and a test set; step S2: constructing a mechanical fault diagnosis knowledge graph named entity recognition model EBBC-DAM-PATM that integrates BERT-BiLSTM-CRF, data enhancement and adversarial training, and training and testing it to obtain the trained EBBC-DAM-PATM model; step S3: using the trained EBBC-DAM-PATM model to perform named entity recognition; step S4: using the neo4j graph database to construct the extracted entity characters into a knowledge graph. The data enhancement module described in the present invention can be completed adaptively according to the amount of data, reducing the problem of sample imbalance and alleviating the overfitting of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of mechanical fault diagnosis, and in particular relates to an entity extraction method for constructing a mechanical fault diagnosis knowledge graph. Background Art

[0002] In order to achieve safe operation requirements such as status analysis, fault prediction, risk assessment, and maintenance optimization of mechanical equipment, it is often necessary to collect various aspects of equipment information, integrate and screen it, and finally extract key information to analyze the equipment.

[0003] Knowledge graph is a technical method that uses a graph model to describe knowledge and model the relationship between all things in the world. The construction of knowledge graph involves many technologies such as knowledge modeling, entity extraction, relationship extraction, knowledge fusion, graph storage, and relationship reasoning. Entity extraction is one of the key technologies. Whether the required entities can be accurately extracted will directly affect the quality and quantity of data in subsequent knowledge modeling. At the same time, the entity extraction process of the knowledge graph is actually a process of structuring unstructured data, which is difficult to achieve through traditional algorithms. With the introduction of the Transformer architecture, more and more large models based on its structure have achieved better results in the field of natural language processing, especially the BERT model with a bidirectional structure designed on its basis, which can achieve good performance improvement in the field of entity extraction. However, there are still some problems with the current BERT-based methods, such as the following papers: David S, Carrell, Robert E, Schoen, Daniel A, Leffler, et al. Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings. Journal of the American Medical Informatics Association: JAMIA. 2017, and Qi Pengnian et al. A review of Chinese named entity recognition based on deep learning. Small and Microcomputer Systems. 2023;44(09):1857-68. They are based on English extraction, and the algorithm needs to be improved for Chinese named entity recognition. Faced with relatively broad and abstract tasks, the application effect in actual professional fields needs to be improved.

[0004] At the same time, the current algorithms mainly use some relatively comprehensive evaluation indicators as standards, but in actual operations, different application scenarios often have different requirements, and also need to evaluate and optimize the models from different angles. Summary of the invention

[0005] In view of the above shortcomings, the present invention provides an entity extraction method for constructing a knowledge graph for mechanical fault diagnosis.

[0006] In order to solve the above problems, the present invention adopts the following technical solutions:

[0007] An entity extraction method for constructing a knowledge graph for mechanical fault diagnosis comprises the following steps:

[0008] Step S1: construct a mechanical fault diagnosis corpus database and divide it into a training set and a test set;

[0009] Step S2: construct a mechanical fault diagnosis knowledge graph named entity recognition model EBBC-DAM-PATM that integrates the BERT-BiLSTM-CRF model, the data enhancement module, and the adversarial training module, and perform training and testing on it to obtain the trained EBBC-DAM-PATM model;

[0010] Step S3: Use the trained EBBC-DAM-PATM model to perform named entity recognition;

[0011] Step S4: Use the neo4j graph database to construct the extracted entity characters into a knowledge graph.

[0012] Further optimization, step S1 includes the following steps:

[0013] Step S1.1: extracting text information from operation logs, maintenance records, and related operation manuals and teaching books during the actual operation of the mechanical equipment;

[0014] Step S1.2: Preliminarily screen the extracted text information to obtain a corpus database;

[0015] Step S1.3: Label each text information in the corpus database manually to generate a corresponding label sequence, where each letter in the label sequence corresponds to a word in the text sequence;

[0016] Step S1.4: Divide the text information of the labeled sequence into a training set and a test set according to a ratio of 7:3.

[0017] Further optimization, in step S2, the data enhancement module processes the input text information as follows:

[0018] Step S2.1: Setting the adjustable threshold H , calculate the importance score of each word based on its position in the text , compare and adjustable thresholdH ,right The word is deleted, and the importance score calculation formula is as follows:

[0019] (1)

[0020] in, is the position of the current word in the text, is the position of the entity character closest to the current character, and is an adjustable parameter; tanh() is the hyperbolic tangent function.

[0021] Step S2.2: Calculate the threshold , as a parameter to control the data enhancement strategy in text information, the specific formula is as follows:

[0022] (2)

[0023] in, is the number of entity characters that need to be extracted. When the number of entity characters to be extracted increases, the complexity of the extraction task will also increase accordingly, and the amount of data required will also be greater. , , are coefficients respectively.

[0024] Step S2.3: Based on the number of data rows and threshold in the text information Calculating Probability , and then randomly replace the entity characters according to the probability. The calculation formula is as follows:

[0025] (3)

[0026] Where l is the number of data rows in the text information, is the threshold value.

[0027] Step S2.4: Combine the text information processed by step S2.1 and the text information processed by step S2.3, and generate the final enhanced text data by alternating and interspersing sentences. The alternating and interspersing specifically means: the first sentence in the text information processed by step S2.1, the first sentence in the text information processed by step S2.3, the second sentence in the text information processed by step S2.1, the second sentence in the text information processed by step S2.3, etc., interspersed and combined in this way.

[0028] Further optimization, step S2.3, based on the probability The corresponding entity words in the training set are replaced with random Chinese characters. Specifically, first, the corresponding entity words in each sentence are found based on the previous label sequence; second, a random number is generated in the range of (0,1) ,when , then the character will not be operated, otherwise it will be overwritten with a random Chinese character.

[0029] Further optimization, in step S2, the mechanical fault diagnosis knowledge graph named entity recognition model EBBC-DAM-PATM adds a data enhancement module and a degraded PGD adversarial training module on the basis of the BERT-BiLSTM-CRF model; the BERT-BiLSTM-CRF model is composed of a BERT layer, a BiLSTM layer and a CRF layer in series, and the text is processed as follows:

[0030] BERT is a bidirectional language network based on the Transformer encoder structure. BERT's tokenizer first converts the text sequence into a corresponding ID sequence for input, and then the model converts it into a word vector sequence of [1, n, 768] to extract the semantic information. n is the length of the text sequence.

[0031] The BiLSTM layer is a network that combines two long short-term memory networks in opposite directions to capture both forward and backward information in a sequence. It outputs a [1, n, 768] word vector sequence to further extract contextual relationship information, where n is the length of the text sequence.

[0032] The CRF layer is a probabilistic network commonly used in structured prediction tasks. It calculates the conditional probability of each word corresponding to an entity or a non-entity based on the input word vector sequence, and determines whether each word in the sentence is an entity word based on the probability. Each sentence forms a corresponding label sequence as output.

[0033] Further optimization, the degenerate PGD adversarial training module reduces the number of iterations on the basis of traditional PGD adversarial training, greatly reducing the computational cost. The specific steps of adversarial training are:

[0034] First, add a perturbation to the output of the embedding layer of the BERT layer r , embedding layer output and perturbation r They are all matrices of [1,n,768], and the result is obtained by adding the corresponding elements;

[0035] Secondly, the projected gradient descent algorithm is first used in the inner loop to perturb r Will be carried out Update iterations, the formula is as follows:

[0036] (4)

[0037] in, represents the projection operator, Indicates the step size, set to 0.4, represents the gradient; It represents the optimal solution at the kth iteration, which is projected onto the optimal solution obtained by iterating one step. The k+1th optimal solution is generated on the feasible set represented by , when actually updating the disturbance, the feasible set is a spherical space with a length of 1.

[0038] in, t is the current iteration round, R is the total iteration round, represents rounding down, M is the number of iterations of the original PGD for perturbation;

[0039] In the first round of training, the perturbation will be iterated M times, and the number of iterations of the perturbation will gradually decrease in each subsequent round of training until the last round of training when the perturbation is only iterated once.

[0040] The updating process of the inner loop perturbation r is as follows:

[0041] 1) Before entering the inner loop, the parameters of each layer of the model will be backed up;

[0042] 2) After entering the loop, the model will be trained and the model parameters will be updated;

[0043] 3) Obtain the gradient of the BERT embedding layer through gradient backpropagation, and update the perturbation r using the projected gradient descent method;

[0044] 4) Repeat steps 2) and 3) After that, the inner loop is jumped out and the model parameters are restored to the state backed up in the first step, keeping only The perturbation r after the update.

[0045] Then, the loop is broken out and the updated perturbation r is added to the BERT embedding layer for another model training. The model parameters are updated and retained. After the last training is completed, the perturbation r will re-enter the next training, and this will be repeated R times to finally achieve the effect of adversarial training. In each round of training, the perturbation will be iteratively updated multiple times, and the number of updates will decay with the training rounds until the last round of training when only a single update iteration is performed on the perturbation.

[0046] Compared with the original PGD adversarial training, the number of iterative updates of the degraded PGD adversarial training is less than half of the original PGD adversarial training. The specific reasons are as follows: The number of iterative updates of the original PGD adversarial training is as shown in formula (5), and the additional computational cost of the degraded PGD adversarial training is obtained. Less than half of the additional computational cost of the original PGD adversarial training, as shown in formula (6), M :

[0047] (6)

[0048] Since the computational cost of each iterative update is not much different, and the additional computational cost is emphasized here, the degraded PGD adversarial training module described in this application can save more than 50% of the additional computational cost generated by adversarial training compared to the original PGD adversarial training.

[0049] Further optimization, in step S4, the extracted entity words are constructed into a knowledge graph using the neo4j graph database, the extracted entity words are used as nodes in the knowledge graph, and the nodes are connected with corresponding relationships to form a network structure, namely the knowledge graph. The network structure of the knowledge graph enables it to combine all information related to mechanical equipment, such as structural relationships, fault causes, fault phenomena, solutions, location information, etc., in a graph in the form of nodes and relationships, and can quickly find a specific node and related nodes according to the query statement to help troubleshoot and diagnose mechanical faults.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. Compared with the original PGD adversarial training, the degraded PGD adversarial training module described in the present invention can save more than 50% of the additional computing expenses generated by adversarial training, while still achieving good results.

[0052] 2. The data enhancement module described in the present invention can adaptively perform data enhancement according to the amount of data, thereby alleviating the problem of sample imbalance and alleviating overfitting of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of the entity extraction method for constructing a knowledge graph for mechanical fault diagnosis according to the present invention;

[0054] Figure 2 It is a schematic diagram of the adversarial training in the present invention;

[0055] Figure 3 It is a knowledge graph rendering. DETAILED DESCRIPTION

[0056] The specific implementation of the present invention will be described in detail below with reference to the accompanying drawings.

[0057] like Figure 1 As shown, an entity extraction method for constructing a knowledge graph for mechanical fault diagnosis includes the following steps:

[0058] Step S1: construct a mechanical fault diagnosis corpus database and divide it into a training set and a test set.

[0059] Taking the extraction of mechanical failure information as an example, text information related to mechanical failure is extracted from the operation logs, maintenance records, and related operation manuals and teaching books of mechanical equipment, and a corpus is obtained after preliminary screening. Each text information in the corpus database is manually labeled to generate a corresponding label sequence.

[0060] In this embodiment, the text information is "It was found that the brake failed due to the corrosion of the brake spring.", and the corresponding label sequence is "OOO B-FLT I-FLT I-FLT I-FLT I-FLT OO B-EQP I-EQP I-EQP O OO". Each letter in the label sequence corresponds to a word in the text sequence, O corresponds to a non-entity word, "B-FLT" and "I-FLT" represent the extracted fault entity words, "B-EQP" and "I-EQP" represent the extracted equipment or component entity words, "B-XXX" represents the first word at the beginning of the entity word, and "I-XXX" represents the remaining words of the entity word, which respectively correspond to "brake spring corrosion" and "brake" in the text information. After labeling, it is divided into training set and test set according to 7:3 for subsequent model training and testing.

[0061] Step S2: construct a mechanical fault diagnosis knowledge graph named entity recognition model EBBC-DAM-PATM that integrates the BERT-BiLSTM-CRF model, the data enhancement module, and the adversarial training module, and perform training and testing on it to obtain the trained EBBC-DAM-PATM model, such as Figure 2 shown.

[0062] Among them, the data enhancement module will process the input text data in four steps:

[0063] The first step is to reduce the sample imbalance problem caused by the low proportion of entity words. The importance score of each word is calculated based on its position in the text information. , compare and adjustable threshold H, for The word is deleted, and in this embodiment, H is set to 0.27. The specific formula is as follows:

[0064] (1)

[0065] in, is the position of the current word in the text, is the position of the entity word closest to the current word; and It is an adjustable parameter and is set to 1 in this embodiment.

[0066] The second step is to calculate the threshold , as a parameter to control the data enhancement strategy, the specific formula is as follows:

[0067] (2)

[0068] in, The number of entity types to be extracted is generally in the range of 1-5. When the number of entity types to be extracted increases, the complexity of the extraction task will also increase accordingly, and the amount of data required will also increase. , , are coefficients, which are set to 2000, 4000 and 10000 respectively in this embodiment.

[0069] The third step is to calculate the amount of data and threshold Generate a probability formula, and then randomly replace entity words based on the probability. The probability calculation formula is as follows:

[0070] (3)

[0071] is the number of rows of current input data, is the threshold value.

[0072] Based on probability The corresponding entity words in the training set are replaced with random Chinese characters. Specifically, first, the corresponding entity words in each sentence are found based on the previous label sequence; second, a random number is generated in the range of (0,1) ,when , then the character will not be operated, otherwise it will be overwritten with a random Chinese character.

[0073] In the fourth step, the text data processed in the first step and the text data processed in the third step are combined, and the final enhanced text data is generated by alternately interspersing sentences. The alternate interspersing specifically means: the first sentence in the text information processed in the first step, the first sentence in the text information processed in the third step, the second sentence in the text information processed in the first step, the second sentence in the text information processed in the third step, etc., interspersed and combined in this way.

[0074] The BERT-BiLSTM-CRF model is composed of a BERT layer, a BiLSTM layer, and a CRF layer connected in series.

[0075] BERT is a bidirectional language network based on the Transformer encoder structure. The BERT tokenizer first converts the text sequence into a corresponding ID sequence for input, and then the model converts it into a word vector sequence of [1, n, 768] to extract the semantic information, where n is the length of the text sequence.

[0076] The BiLSTM layer is a network that combines two long short-term memory networks in opposite directions to capture both forward and backward information in a sequence. It outputs a [1,n,768] word vector sequence to further extract contextual relationship information;

[0077] The CRF layer (conditional random field) is a probabilistic network commonly used in structured prediction tasks. It calculates the conditional probability of each word corresponding to an entity or non-entity based on the input word vector sequence, and determines whether each word in the sentence is an entity word based on the probability. Each sentence forms a corresponding label sequence as output.

[0078] The degenerate PGD adversarial training module reduces the number of iterations based on the traditional PGD adversarial training and significantly reduces the computational cost. Specifically:

[0079] First, add a perturbation r to the embedding layer output of the BERT layer. Both the embedding layer output and the perturbation r are matrices of [1, n, 768], and the result is obtained by adding the corresponding elements.

[0080] Secondly, the perturbation r is first performed in the inner loop using the projected gradient descent method PGD update iterations, where the formula for the projected gradient descent method is as follows:

[0081] (4)

[0082] in, represents the projection operator, Indicates the step size, set to 0.4, represents the gradient; It represents the optimal solution at the kth iteration, which is projected onto the optimal solution obtained by iterating one step. The k+1th optimal solution is generated on the feasible set represented by , when actually updating the disturbance, the feasible set is a spherical space with a length of 1.

[0083] Update times The formula is as follows:

[0084] (5)

[0085] in, is the current training round, R is the total training round, M is the number of iterations of the original PGD on the perturbation, here R is set to 6, M is set to 3, the perturbation will be iterated M times in the first training, and the perturbation will only be updated once in the last training. The whole process will gradually degenerate PGD into FGSM.

[0086] The update process of the inner loop perturbation r:

[0087] 1) Before entering the inner loop, the parameters of each layer of the model will be backed up;

[0088] 2) After entering the loop, the model will be trained and the model parameters will be updated;

[0089] 3) Obtain the gradient of the BERT embedding layer through gradient backpropagation, and update the perturbation r using the projected gradient descent method;

[0090] 4) Repeat steps 2) and 3) After that, the inner loop is jumped out and the model parameters are restored to the state backed up in the first step, keeping only The disturbance r after the update;

[0091] Then, the loop is jumped out and the updated perturbation r is added to the BERT embedding layer for another model training, and the model parameters are updated and retained. After the last training is completed, the perturbation r will re-enter the next training, and this will be repeated 6 times to finally achieve the effect of adversarial training.

[0092] The complete process of adversarial training is as follows:

[0093] Step 1: Initialization

[0094] Input the processed ID sequence into the BERT-BiLSTM-CRF model and add initial perturbations to the BERT embedding layer ;

[0095] Step 2: Initial Training

[0096] Conduct a model training;

[0097] Step 3: External loop

[0098] Step 3.1: Model parameter backup

[0099] Back up the parameters of all layers of the current model;

[0100] Step 3.2: Inner loop

[0101] Step 3.2.1: Calculate the gradient

[0102] Through gradient backpropagation, the gradient direction of the BERT embedding layer after the last training is obtained;

[0103] Step 3.2.2: Update the perturbation

[0104] Use the projection descent method to update the perturbation r in the direction of the positive gradient once;

[0105] Step 3.2.3: Inner loop control

[0106] Determine whether the internal loop disturbance r is fully updated Second-rate;

[0107] If yes, then jump out of the inner loop and jump to step 3.3;

[0108] If not, continue the inner loop and execute step 3.2.4;

[0109] Step 3.2.4: Inner loop training

[0110] Add the iterated new perturbation r to the output of the BERT embedding layer, perform a training, and jump to step 3.2.1;

[0111] Step 3.3: Restore model parameters

[0112] Restore the parameters of each layer of the model to the state backed up in step 3.1, and only keep the rotation r updated by the internal loop;

[0113] Step 3.4: External loop training

[0114] Perform a model training, update the model parameters and keep them;

[0115] Step 3.5: External loop control

[0116] Determine whether the model training times meet 6 times;

[0117] If yes, jump out of the outer loop and jump to step 4;

[0118] If not, continue with the outer loop and jump to step 3.1;

[0119] Step 4: Complete the training

[0120] Output the trained model.

[0121] Step S3: Use the trained EBBC-DAM-PATM model to perform named entity recognition; take the sentence "It was found that the brake failed due to the corrosion of the brake spring." as an example, input this text into the trained EBBC-DAM-PATM model, and the model will output a corresponding label "OOO B-FLT I-FLT I-FLT I-FLT I-FLT OO B-EQP I-EQP I-EQP OOO". According to the label and the input text, the fault entity word "brake spring corrosion" and the equipment or component entity word "brake" in this sentence can be obtained.

[0122] Step S4, using the neo4j graph database to construct the extracted entities into a knowledge graph, using the extracted entity words as nodes in the knowledge graph, such as "brake spring rust" and "brake", and connecting the nodes with corresponding relationships to form triples, such as: "brake spring rust—fault location—brake", connecting a large number of triples and saving them in the database, and finally forming a network structure, i.e., a knowledge graph. The network structure of the knowledge graph enables it to combine all information related to mechanical equipment, such as structural relationships, fault causes, fault phenomena, solutions, location information, etc., in a graph in the form of nodes and relationships, and can quickly find a specific node and related nodes based on the query statement, to help troubleshoot and diagnose mechanical faults, with specific effects such as Figure 3 shown.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. An entity extraction method for constructing a knowledge graph for mechanical fault diagnosis, characterized in that: The steps include: Step S1: construct a mechanical fault diagnosis corpus database and divide it into a training set and a test set; including the following steps: Step S1.1: extracting text information from operation logs, maintenance records, and related operation manuals and teaching books during the actual operation of the mechanical equipment; Step S1.2: Preliminarily screen the extracted text information to obtain a corpus database; Step S1.3: Label each text information in the corpus database manually to generate a corresponding label sequence, where each letter in the label sequence corresponds to a word in the text sequence; Step S1.4: Divide the labeled text into a training set and a test set according to a ratio of 7:3; Step S2: construct a mechanical fault diagnosis knowledge graph named entity recognition model EBBC-DAM-PATM that integrates the BERT-BiLSTM-CRF model, the data enhancement module, and the adversarial training module, and perform training and testing on it to obtain the trained EBBC-DAM-PATM model; The data enhancement module processes the input text information as follows: Step S2.1: Setting the adjustable threshold H , calculate the importance score of each word based on its position in the text f ( s ), compare f ( s ) and adjustable threshold H ,right f ( s ) is less than H The word is deleted, and the importance score calculation formula is as follows: (1) in, is the position of the current word in the text, is the position of the entity character closest to the current character, and is an adjustable parameter; tanh() is the hyperbolic tangent function; Step S2.2: Calculate the threshold , as a parameter to control the data enhancement strategy in text information, the specific calculation formula is as follows: （2） in, C The number of entity characters to be extracted; a , b , d are the coefficients respectively; Step S2.3: Based on the number of data rows in the text information and the threshold Calculating Probability , and then randomly replace the entity characters according to the probability. The calculation formula is as follows: （3） Where l is the number of data rows in the text information, is the threshold value; Step S2.4: combining the text information processed in step S2.1 with the text information processed in step S2.3, and generating the final enhanced text data by alternating sentences; Step S3: Use the trained EBBC-DAM-PATM model to perform named entity recognition; Step S4: Use the neo4j graph database to construct the extracted entity characters into a knowledge graph.

2. The entity extraction method for mechanical fault diagnosis knowledge graph construction according to claim 1 is characterized in that: The step S2.3, based on probability The corresponding entity words in the training set are replaced with random Chinese characters. Specifically, first, the corresponding entity words in each sentence are found based on the previous label sequence; second, a random number is generated in the range of (0,1) ,when , then the character will not be operated, otherwise it will be overwritten with a random Chinese character.

3. The entity extraction method for constructing a knowledge graph for mechanical fault diagnosis according to claim 2 is characterized in that: In step S2, the mechanical fault diagnosis knowledge graph named entity recognition model EBBC-DAM-PATM is based on the BERT-BiLSTM-CRF model, with the addition of a data enhancement module and a degraded PGD adversarial training module; the BERT-BiLSTM-CRF model is composed of a BERT layer, a BiLSTM layer and a CRF layer in series, and the text information is processed as follows: First, the BERT tokenizer converts the text sequence into a corresponding ID sequence for input; the BiLSTM layer outputs a [1, n, 768] word vector sequence to further extract contextual relationship information, where n is the length of the text sequence; The CRF layer calculates the conditional probability of each word corresponding to an entity or non-entity based on the input word vector sequence, determines whether each word in the sentence is an entity word based on the conditional probability, and forms a corresponding label sequence for each sentence as output.

4. The entity extraction method for constructing a knowledge graph for mechanical fault diagnosis according to claim 3 is characterized in that: The specific steps of adversarial training using the degraded PGD adversarial training module are: First, add a perturbation to the output of the embedding layer of the BERT layer r , embedding layer output and perturbation r They are all matrices of [1,n,768], and the result is obtained by adding the corresponding elements; Secondly, the projected gradient descent algorithm is used in the inner loop to perturb r Will be carried out Update iterations, the formula is as follows: ； in, is the projection operator, is the step size, set to 0.4, x represents the model input, For the k The perturbation after iterations is For the k +1 perturbation after iteration, represents the loss function of the degraded PGD adversarial training module, Indicates that the degraded PGD adversarial training module is The gradient of the loss function under perturbation; （4） in, is the projection operator, is the step size, set to 0.4, is the gradient; For the k The optimal solution of the iteration, For the k +1 optimal solution for iteration; Update iterations The formula is as follows: （5） in, t is the current iteration round, R is the total iteration round, represents rounding down, M is the number of iterations of the original PGD for perturbation; In the first round of training, the perturbation is iterated M times, and the number of iterations of the perturbation in each subsequent round of training gradually decreases until the last round of training when the perturbation is only iterated once; Jump out of the iteration loop and add the updated perturbation r to the BERT embedding layer for another model training, update the model parameters and retain them; after the last training is completed, the perturbation r will re-enter the next training, and train for a total of R rounds to finally achieve the effect of adversarial training.

5. The entity extraction method for mechanical fault diagnosis knowledge graph construction according to claim 4 is characterized in that: In step S4, the extracted entity characters are constructed into a knowledge graph using the neo4j graph database, the extracted entity characters are used as nodes in the knowledge graph, and the nodes are connected by corresponding relationships to form a network structure, namely, a knowledge graph.

Images