Fault text entity recognition method and device based on deep learning
The engine fault log is preprocessed and feature extraction through deep learning methods, which solves the impact of text length on feature extraction, improves the accuracy of named entity recognition, and realizes accurate identification of entities in engine fault log.
Patent Information
- Application Number
- CN202411194145.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, the naming entity recognition method fails to effectively consider the impact of different lengths of text on feature extraction, resulting in low accuracy.
Using the fault text entity recognition method based on deep learning, the engine fault log is obtained for preprocessing, including regularization, deduplication, word segmentation and filtering, the vector representation is used for BERT model, and the CNN-BIGRU-Softmax feature extraction model is used for identification. , combined with the TF-IDF algorithm to assign weights and adjust text length to ensure the accuracy of feature extraction.
Effectively eliminates the impact of text length on feature extraction, improves the accuracy of naming entity recognition, and ensures accurate extraction of entities in the engine failure log.
Smart Images

Figure CN119067107A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for fault text entity recognition based on deep learning, a computer-readable storage medium, and a data processing system. Background Art
[0002] The engine fault record table is a log of maintenance personnel's report records. In addition to basic structured data such as fault time, fault location, and maintenance order number, it also contains unstructured data such as processing results. The log includes a large amount of text information, and due to different text formats, the language length of the text varies, generally ranging from 20 to 300 words, all of which belong to the category of short text. Since the log is recorded manually, there is a lot of unnecessary language cost due to personal style, and there are problems such as semantic redundancy and cumbersomeness caused by different recording methods of different staff members.
[0003] Therefore, in the prior art, the order record log is processed by a machine learning model or other methods to fully explore the information contained in the engine failure short text and eliminate the influence of semantic redundancy and complexity. Entity extraction is usually achieved through the bag-of-words model, BiGUR model and BiLSTM model, among which the bag-of-words model focuses on the relationship between words and does not focus on the entire sentence. The BiGUR model and BiLSTM model focus on the word order when extracting features, and do not consider the position relationship of the text.
[0004] In summary, in the prior art, when extracting text features through models, the interference of different text language lengths in logs on feature extraction is not considered, resulting in deviations in feature extraction. Summary of the invention
[0005] The main purpose of the present application is to provide a method, device, computer-readable storage medium and data processing system for fault text entity recognition based on deep learning, so as to at least solve the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy.
[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a fault text entity recognition method based on deep learning is provided, including: obtaining an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure; preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; processing the first text data based on an entity BIO annotation method to obtain second text data, and inputting the second text data into a BERT model to convert it into a vector representation to obtain multiple first target vectors; performing a splicing or cropping operation based on the first target vector so that the vector length of the first target vector is within a preset range to obtain a second target vector; inputting the second target vector into a CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, wherein the third text data is a fault text entity corresponding to the target entity in the engine fault log.
[0007] Optionally, the engine fault log is preprocessed to obtain first text data, including: regularizing the engine fault log to remove character data and label data in the engine fault log to obtain third text data; deduplicating the third text data to obtain fourth text data; segmenting the fourth text data based on Jieba word segmentation technology, and deleting preset words therein to obtain fifth text data, wherein the preset words include one or more of stop words, modal particles and conjunctions; adding corresponding weight parameters to the fifth text data through the TF-IDF algorithm, and deleting the fifth text data whose weight parameters are less than a first threshold value to obtain the first text data.
[0008] Optionally, adding a corresponding weight parameter to the fifth text data by using a TF-IDF algorithm includes:
[0009] The weight parameter is calculated based on the fifth text data by using the first target formula: Among them, TF-IDF is the weight parameter, is the frequency value of the i-th word in document j, n i,j is the number of occurrences of the i-th word in document j, ∑ m n m,j is the total number of occurrences of all words in document j, is the inverse word frequency of the i-th word in the document j in the fifth text data, where |D| is the total number of documents, |{j:q i ∈d j}| contains the word q i The total number of documents, For word qi The corresponding preset weight, wherein the preset weight is based on the second objective formula Calculated, where For word q i The frequency of occurrence, For word q i The corresponding preset rating.
[0010] Optionally, the first text data is processed based on the entity BIO annotation method to obtain the second text data, including: traversing the entities in the first text data, adding a first preset identifier to the starting position of the entity, adding a second preset identifier to the end position of the entity, and adding third identification information to the part of the entity except the starting position and the end position to obtain the second text data.
[0011] Optionally, a splicing or cropping operation is performed based on the first target vector so that the vector length of the first target vector is within a preset range to obtain a second target vector, including: when the vector length of the first target vector is greater than or equal to a first threshold and less than a second threshold, respectively calculating cosine similarity for each of the first target vectors to obtain a first similarity; when the first similarity is greater than a fifth threshold, splicing each of the first target vectors with a first preset weight until the vector length of the spliced first target vector is greater than or equal to a third threshold, and determining the first target vector as the second target vector; when the vector length of the first target vector is greater than the second threshold and less than or equal to the third threshold, respectively calculating cosine similarity for each of the first target vectors to obtain a second similarity; when the second similarity is greater than a sixth threshold, splicing each of the first target vectors with a second preset weight until the vector length of the spliced first target vector is greater than or equal to the third threshold, and determining the first target vector as the second target vector; when the first target vector is greater than a fourth threshold, cropping each of the first target vectors with the fourth threshold to obtain the second target vector.
[0012] Optionally, before determining the first target vector as the second target vector, the method further includes: padding the first target vector with preset characters until the vector length of the first target vector is equal to the fourth threshold.
[0013] Optionally, the CNN-BIGRU-Softmax feature extraction model includes a CNN layer, a BIGRU layer and a Softmax layer, and the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, including: convolving the second target vector through the CNN layer to obtain a third target vector; performing a pooling operation on the third target vector, and performing feature recognition on the pooled third target vector through the BIGRU layer to extract timing features to obtain a fourth target vector; calculating according to the fourth target vector through the Softmax layer to obtain the target probability corresponding to each of the fourth target vectors, and determining the entity corresponding to the fourth target vector corresponding to the maximum value of the target probability as the third text data.
[0014] According to another aspect of the present application, a deep learning-based fault text entity recognition device is provided, the device comprising: a first acquisition unit, used to acquire an engine fault log, the engine fault log at least comprising a fault location, a fault cause, a fault phenomenon and a treatment measure; a first processing unit, used to pre-process the engine fault log to obtain a plurality of first text data, the pre-processing at least comprising regularization, deduplication operation, word segmentation operation and screening operation; a second processing unit, used to process the first text data based on an entity BIO annotation method to obtain second text data, and input the second text data into a BERT model to convert it into a vector representation to obtain a first target vector; a third processing unit, used to perform a splicing or cropping operation based on the first target vector so that the vector length of the first target vector is within a preset range to obtain a second target vector; an input unit, used to input the second target vector into a CNN-BIGRU-Softmax feature extraction model for identification to obtain third text data, the third text data being the fault text entity corresponding to the target entity in the engine fault log.
[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.
[0016] According to another aspect of the present application, a data processing system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods described.
[0017] Applying the technical solution of the present application, in the above-mentioned fault text entity recognition method based on deep learning, first, an engine fault log is obtained, and the above-mentioned engine fault log at least includes the fault location, fault cause, fault phenomenon and treatment measures; then, the above-mentioned engine fault log is preprocessed to obtain first text data, and the above-mentioned preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; then, the above-mentioned first text data is processed based on the entity BIO annotation method to obtain second text data, and the above-mentioned second text data is input into the BERT model to convert into a vector representation to obtain multiple first target vectors; then, a splicing or cropping operation is performed based on the above-mentioned first target vector so that the vector length of the above-mentioned first target vector is within a preset range to obtain a second target vector; finally, the above-mentioned second target vector is input into the CNN-BIGRU-Softmax feature extraction model for identification to obtain third text data, and the above-mentioned third text data is the fault text entity corresponding to the target entity in the above-mentioned engine fault log. This application first removes meaningless words based on screening, and assigns weights to words in the text based on TF-IDF and deletes low-weight words to shorten the text length and reduce the impact of text length on feature extraction. On this basis, the text is spliced or trimmed based on the text length to unify the text length without changing the feature points, eliminating the impact of text length on feature extraction, and then the classification model is used for feature extraction and the target entity is matched. This method solves the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A hardware structure block diagram of a mobile terminal according to a method for fault text entity recognition based on deep learning provided in an embodiment of the present application is shown;
[0019] Figure 2 A schematic diagram of a process of a fault text entity recognition method based on deep learning provided according to an embodiment of the present application is shown;
[0020] Figure 3 A flowchart of a specific method for fault text entity recognition based on deep learning provided according to an embodiment of the present application is shown;
[0021] Figure 4 A schematic flow chart of a method for preprocessing an original fault text according to the present application is shown;
[0022] Figure 5 A schematic diagram of a process of a vector length unification method according to the present application is shown;
[0023] Figure 6A structural block diagram of a fault text entity recognition device based on deep learning provided according to an embodiment of the present application is shown.
[0024] The above drawings include the following reference numerals:
[0025] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION
[0026] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:
[0030] Entity BIO annotation method: a method for annotating entities in text. In this method, each identified entity is divided into three categories: B (Begin), I (Inside) and O (Outside). Among them, B indicates that the entity is the beginning of an entity, I indicates that the entity is the middle part of an entity, and O indicates that the word is not an entity. For example, in a sentence, if we want to annotate the entity "apple", the annotation sequence of the sentence may be "B-fruitI-fruit OO". This sequence means that in the sentence, the beginning of the "apple" entity is "B-fruit", the middle part of the entity is "I-fruit", and other words are not entities.
[0031] CNN-BIGRU-Softmax feature extraction model: The composite neural network structure adopted in this application includes a CNN layer, i.e. a neural network layer, a BiGRU layer, i.e. a bidirectional gated recurrent network layer, and a Softmax layer. Softmax is an activation function used to convert neural network results into probabilities.
[0032] CNN (Convolutional Neural Network): Convolutional neural network, usually consists of five parts: input layer, convolution layer, pooling layer, fully connected layer, and output layer. CNN can be used to extract the spatial features of text vector data.
[0033] BiGRU (Bidirectional Gate Recurrent Unit): Bidirectional gated recurrent network. BiGRU is based on GRU. The neurons of GRU are composed of a memory unit and two gate units (reset gate and update gate) to complete the extraction of sequence features. BiGRU combines two GRUs in opposite directions to realize the feature extraction of forward and reverse text vector sequences.
[0034] Softmax: Converts the neural network results into corresponding probabilities. The larger the probability, the greater the possibility that the sample belongs to the label.
[0035] As introduced in the background technology, in the prior art, when text feature extraction is performed through a model, the interference of different text language lengths in the log on feature extraction is not considered, resulting in deviations in feature extraction. In order to solve the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy, the embodiments of the present application provide a fault text entity recognition method, device, computer-readable storage medium and data processing system based on deep learning.
[0036] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0037] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for fault text entity recognition based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0038] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0039] In this embodiment, a fault text entity recognition method based on deep learning that runs on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0040] Figure 2 is a flow chart of a method for fault text entity recognition based on deep learning according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:
[0041] Step S201, obtaining an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure;
[0042] In the specific implementation, Figure 3 As shown, first, the original fault text is obtained, and the original fault text is preprocessed to remove unnecessary words to narrow the length difference between long and short texts. The processed text is represented by a BERT vector, and the converted vector is classified based on length. Based on the classification result, the vector is concatenated or trimmed and filled until the vector length is unified, and then input into the model for feature extraction to determine the target entity.
[0043] Specifically, the maintenance personnel's report record log (original fault text) is obtained to obtain the above-mentioned engine fault log, which at least needs to include various entity records such as fault location, fault phenomenon, fault cause, handling measures and maintenance time.
[0044] It is easy to understand that, in addition to the above entities, the above engine fault log also contains unstructured data describing various processing results, as well as connecting words of the maintenance personnel's personal recording habits.
[0045] Step S202, preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation;
[0046] Specifically, since the above-mentioned engine fault log contains a large number of digital or alphabetic languages such as engine numbers, time and date, and there is a lot of repetition, the present application is configured to preprocess the above-mentioned engine fault log before analysis, and streamline the above-mentioned engine fault log through methods such as regularization, deduplication operation, word segmentation operation and screening operation, shorten the record text therein, and obtain the above-mentioned first text data, so as to reduce the text length difference between texts to a certain extent, and reduce the impact of text length on subsequent feature extraction operations.
[0047] Step S203, processing the first text data based on the entity BIO annotation method to obtain second text data, and inputting the second text data into the BERT model to convert it into a vector representation to obtain a plurality of first target vectors;
[0048] Specifically, Figure 4 As shown, all entities included in the above-mentioned first text data are annotated based on the above-mentioned entity BIO annotation to obtain the above-mentioned second text data, so as to facilitate the subsequent model to identify the entity type, and then convert the above-mentioned second text data into a model language, that is, convert the above-mentioned second text data into a vector representation through the BERT model to obtain the above-mentioned first target vector.
[0049] Step S204, performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector;
[0050] Specifically, in order to eliminate the influence of text length on feature extraction during entity recognition, the present application is configured to unify the text length of the preprocessed text, that is, the above-mentioned first target vector is spliced or trimmed based on the above-mentioned preset range, and the text length is standardized to obtain the above-mentioned second target vector.
[0051] Step S205: input the second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is the fault text entity corresponding to the target entity in the engine fault log.
[0052] Specifically, the second target vector is divided into a training set and a test set, and a feature extraction model, namely, a CNN-BIGRU-Softmax feature extraction model, is trained based on the training set and the test set. The effect of the feature extraction model is evaluated based on the test set. Then, when the loss function is less than a preset value, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain the third text data and complete the fault entity recognition.
[0053] Through this embodiment, first, an engine fault log is obtained, and the above-mentioned engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure; then, the above-mentioned engine fault log is preprocessed to obtain first text data, and the above-mentioned preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; then, the above-mentioned first text data is processed based on the entity BIO annotation method to obtain second text data, and the above-mentioned second text data is input into the BERT model to be converted into a vector representation to obtain multiple first target vectors; then, a splicing or cropping operation is performed based on the above-mentioned first target vector so that the vector length of the above-mentioned first target vector is within a preset range to obtain a second target vector; finally, the above-mentioned second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, and the above-mentioned third text data is the fault text entity corresponding to the target entity in the above-mentioned engine fault log. This application first removes meaningless words based on screening, and assigns weights to words in the text based on TF-IDF and deletes low-weight words to shorten the text length and reduce the impact of text length on feature extraction. On this basis, the text is spliced or trimmed based on the text length to unify the text length without changing the feature points, eliminating the impact of text length on feature extraction, and then the classification model is used for feature extraction and the target entity is matched. This method solves the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy.
[0054] In order to pre-process the above engine fault log, in an optional implementation, as Figure 4 As shown, the above step S202 includes:
[0055] Step S2021, regularizing the engine fault log to remove character data and label data in the engine fault log to obtain third text data;
[0056] First, the present application is configured to perform preliminary regularization on the above-mentioned engine fault log, eliminating the necessary costs of the above-mentioned engine number, time, date and other digital and letter types, that is, the above-mentioned character data and the above-mentioned label data, and only retains the text language to obtain the above-mentioned third text data.
[0057] Step S2022, performing a deduplication operation on the third text data to obtain fourth text data;
[0058] Then, a deduplication operation is performed on the third text data to obtain the fourth text data.
[0059] It is understandable that the deduplication operation is used to reduce the complexity of the model and prevent overfitting. Removing duplicate words can also reduce the gap between texts of different lengths and reduce the impact of text length.
[0060] Step S2023, segmenting the fourth text data based on Jieba word segmentation technology, and deleting preset words therein to obtain fifth text data, wherein the preset words include one or more of stop words, modal particles and conjunctions;
[0061] Specifically, the fourth text data is segmented by Jieba segmentation technology, and then the segmentation results are screened, including but not limited to removing stop words, deleting non-content words such as modal particles and conjunctions, building a text vocabulary, and obtaining the fifth text data.
[0062] Step S2024: adding corresponding weight parameters to the fifth text data through the TF-IDF algorithm, and deleting the fifth text data whose weight parameters are less than the first threshold, to obtain the first text data.
[0063] Specifically, the application sets up an expert based on engine fault troubleshooting to build an engine fault feature word library, assign an initial score to each feature word, and then assign a weight parameter to the fifth text data based on the initial score based on the YF-IDF algorithm, and delete the feature words with lower weights. Then, the processed text data is returned to the original word order.
[0064] It can be understood that the above operation of assigning weights and deleting low-weight words removes non-important text in the above fifth text data, and once again reduces the length of some longer text before analysis, thereby reducing the vector dimension during data analysis.
[0065] In the specific implementation, Figure 4 As shown, this application annotates the screened BIOs and performs vectorization to complete the data preparation process.
[0066] In order to weight the feature words by using the TF-IDF algorithm, in an optional implementation, the above step S2024 further includes:
[0067] Step S20241, calculating the weight parameter based on the fifth text data by using the first target formula:
[0068]
[0069] Among them, TF-IDF is the above weight parameter, is the frequency value of the i-th word in document j, n i,j is the number of occurrences of the i-th word in document j, ∑ m nm,j is the total number of occurrences of all words in document j, is the inverse word frequency of the i-th word in document j in the fifth text data, where |D| is the total number of documents, |{j:q i ∈d j}| contains the word q i The total number of documents, For word q i The corresponding preset weights, wherein the preset weights are based on the second objective formula Calculated, where For word q i The frequency of occurrence, For word q i The corresponding preset rating.
[0070] Specifically, in the actual calculation process, the above weight parameters are calculated according to the above formula.
[0071] In order to facilitate the BERT model to convert the first text data into a vector representation, in an optional implementation, the step S203 includes:
[0072] Step S2031, traverse the entities in the above-mentioned first text data, add a first preset identifier to the starting position of the above-mentioned entity, add a second preset identifier to the ending position of the above-mentioned entity, and add third identifier information to the part of the above-mentioned entity except the above-mentioned starting position and the above-mentioned ending position to obtain the above-mentioned second text data.
[0073] Specifically, the identified entities are annotated according to the BIO annotation specification, that is, each identified entity is divided into three parts: B (Beginning), i.e., the first preset identifier, I (Inside), and O (Outside). B represents the beginning of the entity, I represents the middle part of the entity, i.e., the second preset identifier, and O represents the part outside the entity, i.e., the third preset identifier, to obtain the second text data.
[0074] It is understandable that the present application implements the above operations through a pre-trained annotation model.
[0075] In order to unify the text length of each of the above second text data, in an optional implementation, as Figure 5 As shown, the above step S204 includes:
[0076] Step S2041, when the vector length of the first target vector is greater than or equal to the first threshold and less than the second threshold, respectively calculate the cosine similarity of each of the first target vectors to obtain a first similarity, and when the first similarity is greater than a fifth threshold, splice each of the first target vectors with a first preset weight until the vector length of the spliced first target vector is greater than or equal to a third threshold, and determine the first target vector as the second target vector;
[0077] Specifically, in order to further solve the problem of a large span in the length of short text data and reduce its impact on feature recognition results, the present application sets limits of 20, 100, 200 and 300, that is, the first threshold, the second threshold, the third threshold and the fourth threshold to divide the first target vector. When the vector length of the first target vector is greater than or equal to 20 and less than 100, the cosine similarity is calculated within the text of this length to obtain the first similarity. When the first similarity is greater than α, the first target vector is spliced with the first preset weight. When the first similarity is not satisfied, the original vector is retained.
[0078] Step S2042, when the vector length of the first target vector is greater than the second threshold and less than or equal to the third threshold, respectively calculate the cosine similarity of each of the first target vectors to obtain a second similarity, and when the second similarity is greater than a sixth threshold, concatenate each of the first target vectors with a second preset weight until the vector length of the concatenated first target vector is greater than or equal to the third threshold, and determine the first target vector as the second target vector;
[0079] Specifically, when the vector length of the first target vector is greater than or equal to 100 and less than 200, the cosine similarity is calculated within the text of this length to obtain the second similarity. When the second similarity is greater than β, the first target vector is spliced with the second preset weight. When the second similarity is not satisfied, the original vector is retained.
[0080] Step S2043: When the first target vector is greater than the fourth threshold, each of the first target vectors is clipped by the fourth threshold to obtain the second target vector.
[0081] Specifically, when the vector length of the first target vector is greater than or equal to 300, the text of this length is cropped.
[0082] In order to ensure that the length of the corrected vector is uniform, in an optional implementation manner, before determining the third target vector as the second target vector, the following steps are included:
[0083] Step S301, padding the third target vector with preset characters until the vector length of the third target vector is equal to the fourth threshold.
[0084] Specifically, when the vector length of the concatenated vector or the original vector is greater than or equal to 200 and less than 300, the vector length between 200 and 300 is increased to 300 by adding 0.
[0085] It can be understood that the above embodiment ensures that the vector length is uniform as much as possible, reducing the impact of large differences in text length on subsequent feature extraction.
[0086] In order to identify the target entity, in an optional implementation, the above step S205 includes:
[0087] Step S2051, convolving the second target vector through the CNN layer to obtain a third target vector;
[0088] Specifically, the above-mentioned CNN-BIGRU-Softmax feature extraction model includes a CNN layer, a BIGRU layer and a Softmax layer. CNN realizes the expression of the target text entity vector in a higher dimension through convolution operation to obtain the above-mentioned third target vector.
[0089] Step S2052, performing a pooling operation on the third target vector, and performing feature recognition on the third target vector after pooling through a BIGRU layer to extract time series features to obtain a fourth target vector;
[0090] Specifically, the third target vector is reduced in dimension by a pooling operation, and then input into BIGRU to identify the temporal features of the target entity text context through a bidirectional gated recurrent network to obtain the fourth target vector.
[0091] Step S2053, calculating according to the fourth target vector through the Softmax layer to obtain the target probability corresponding to each of the fourth target vectors, and determining the entity corresponding to the fourth target vector corresponding to the maximum value of the target probability as the third text data.
[0092] Specifically, the probability of converting the BIGRU output feature into a label is calculated through the Softmax activation function, and the entity with the maximum output probability is the target entity, that is, the third text data mentioned above is obtained.
[0093] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0094] The embodiment of the present application also provides a fault text entity recognition device based on deep learning. It should be noted that the fault text entity recognition device based on deep learning in the embodiment of the present application can be used to execute the fault text entity recognition method based on deep learning provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0095] The following introduces the deep learning-based fault text entity recognition device provided in an embodiment of the present application.
[0096] Figure 6 It is a structural block diagram of a fault text entity recognition device based on deep learning according to an embodiment of the present application.
[0097] like Figure 6 As shown, the device comprises:
[0098] A first acquisition unit 10 is used to acquire an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure;
[0099] Specifically, the maintenance personnel's report record log is obtained to obtain the above-mentioned engine fault log, which at least needs to include various entity records such as fault location, fault phenomenon, fault cause, handling measures and maintenance time.
[0100] It is easy to understand that, in addition to the above entities, the above engine fault log also contains unstructured data describing various processing results, as well as connecting words of the maintenance personnel's personal recording habits.
[0101] A first processing unit 20 is used to pre-process the engine fault log to obtain first text data, wherein the pre-processing at least includes regularization, deduplication operation, word segmentation operation and screening operation;
[0102] Specifically, since the above-mentioned engine fault log contains a large number of digital or alphabetic languages such as engine numbers, time and date, and there is a lot of repetition, the present application is configured to preprocess the above-mentioned engine fault log before analysis, and streamline the above-mentioned engine fault log through methods such as regularization, deduplication operation, word segmentation operation and screening operation, shorten the record text therein, and obtain the above-mentioned first text data, so as to reduce the text length difference between texts to a certain extent, and reduce the impact of text length on subsequent feature extraction operations.
[0103] The second processing unit 30 is used to process the first text data based on the entity BIO annotation method to obtain second text data, and input the second text data into the BERT model to convert it into a vector representation to obtain multiple first target vectors;
[0104] Specifically, Figure 4 As shown, all entities included in the above-mentioned first text data are annotated based on the above-mentioned entity BIO annotation to obtain the above-mentioned second text data, so as to facilitate the subsequent model to identify the entity type, and then convert the above-mentioned second text data into a model language, that is, convert the above-mentioned second text data into a vector representation through the BERT model to obtain the above-mentioned first target vector.
[0105] A third processing unit 40 is used to perform a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector;
[0106] Specifically, in order to eliminate the influence of text length on feature extraction during entity recognition, the present application is configured to unify the text length of the preprocessed text, that is, the above-mentioned first target vector is spliced or trimmed based on the above-mentioned preset range, and the text length is standardized to obtain the above-mentioned second target vector.
[0107] The input unit 50 is used to input the above-mentioned second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the above-mentioned third text data is the fault text entity corresponding to the target entity in the above-mentioned engine fault log.
[0108] Specifically, the second target vector is divided into a training set and a test set, a feature extraction model is trained based on the training set and the test set, that is, a CNN-BIGRU-Softmax feature extraction model, and the effect of the feature extraction model is evaluated based on the test set. Then, when the loss function is less than a preset value, Figure 3 As shown, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain the third text data and complete the fault entity recognition.
[0109] Through this embodiment, the first acquisition unit acquires the engine fault log, and the above-mentioned engine fault log at least includes the fault location, fault cause, fault phenomenon and treatment measures; the first processing unit preprocesses the above-mentioned engine fault log to obtain first text data, and the above-mentioned preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; the second processing unit processes the above-mentioned first text data based on the entity BIO annotation method to obtain second text data, and inputs the above-mentioned second text data into the BERT model to convert it into a vector representation to obtain multiple first target vectors; the third processing unit performs splicing or cropping operations based on the above-mentioned first target vectors, so that the vector length of the above-mentioned first target vector is within a preset range, and obtains the second target vector; the input unit inputs the above-mentioned second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, and the above-mentioned third text data is the fault text entity corresponding to the target entity in the above-mentioned engine fault log. This application first removes meaningless words based on screening, and assigns weights to words in the text based on TF-IDF and deletes low-weight words to shorten the text length and reduce the impact of text length on feature extraction. On this basis, the text is spliced or trimmed based on the text length to unify the text length without changing the feature points, eliminating the impact of text length on feature extraction, and then the classification model is used for feature extraction and the target entity is matched. This method solves the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy.
[0110] In order to pre-process the above engine fault log, in an optional implementation manner, as Figure 4 As shown, the first processing unit includes:
[0111] A first processing module is used to regularize the engine fault log to remove character data and label data in the engine fault log to obtain third text data;
[0112] First, the present application is configured to perform preliminary regularization on the above-mentioned engine fault log, eliminating the necessary costs of the above-mentioned engine number, time, date and other digital and letter types, that is, the above-mentioned character data and the above-mentioned label data, and only retains the text language to obtain the above-mentioned third text data.
[0113] A second processing module, configured to perform a deduplication operation on the third text data to obtain fourth text data;
[0114] Then, a deduplication operation is performed on the third text data to obtain the fourth text data.
[0115] It is understandable that the deduplication operation is used to reduce the complexity of the model and prevent overfitting. Removing duplicate words can also reduce the gap between texts of different lengths and reduce the impact of text length.
[0116] A third processing module is used to segment the fourth text data based on Jieba segmentation technology, and delete preset words therein to obtain fifth text data, wherein the preset words include one or more of stop words, modal particles and conjunctions;
[0117] Specifically, the fourth text data is segmented by Jieba segmentation technology, and then the segmentation results are screened, including but not limited to removing stop words, deleting non-content words such as modal particles and conjunctions, building a text vocabulary, and obtaining the fifth text data.
[0118] The fourth processing module is used to add a corresponding weight parameter to the fifth text data through the TF-IDF algorithm, and delete the fifth text data whose weight parameter is less than the first threshold, so as to obtain the first text data.
[0119] Specifically, the application sets up an expert based on engine fault troubleshooting to build an engine fault feature word library, assign an initial score to each feature word, and then assign a weight parameter to the fifth text data based on the initial score based on the YF-IDF algorithm, and delete the feature words with lower weights. Then, the processed text data is returned to the original word order.
[0120] It can be understood that the above operation of assigning weights and deleting low-weight words removes non-important text in the above fifth text data, and once again reduces the length of some longer text before analysis, thereby reducing the vector dimension during data analysis.
[0121] In the specific implementation, Figure 4 As shown, this application annotates the screened BIOs and performs vectorization to complete the data preparation process.
[0122] In order to weight the feature words by using the TF-IDF algorithm, in an optional implementation, the first processing module further includes:
[0123] The calculation submodule is used to calculate the weight parameter based on the fifth text data by using the first target formula:
[0124]
[0125] Among them, TF-IDF is the above weight parameter, is the frequency value of the i-th word in document j, n i,j is the number of occurrences of the i-th word in document j, ∑ m nm,j is the total number of occurrences of all words in document j, is the inverse word frequency of the i-th word in document j in the fifth text data, where |D| is the total number of documents, |{j:q i ∈d j}| contains the word q i The total number of documents, For word q i The corresponding preset weights, wherein the preset weights are based on the second objective formula Calculated, where For word q i The frequency of occurrence, For word q i The corresponding preset rating.
[0126] Specifically, in the actual calculation process, the above weight parameters are calculated according to the above formula.
[0127] In order to facilitate the BERT model to convert the first text data into a vector representation, in an optional implementation, the second processing unit includes:
[0128] The fifth processing module is used to traverse the entities in the above-mentioned first text data, add a first preset identifier to the starting position of the above-mentioned entity, add a second preset identifier to the ending position of the above-mentioned entity, and add third identifier information to the part of the above-mentioned entity except the above-mentioned starting position and the above-mentioned ending position to obtain the above-mentioned second text data.
[0129] Specifically, the identified entities are annotated according to the BIO annotation specification, that is, each identified entity is divided into three parts: B (Beginning), i.e., the first preset identifier, I (Inside), and O (Outside). B represents the beginning of the entity, I represents the middle part of the entity, i.e., the second preset identifier, and O represents the part outside the entity, i.e., the third preset identifier, to obtain the second text data.
[0130] It is understandable that the present application implements the above operations through a pre-trained annotation model.
[0131] In order to unify the text length of each of the above second text data, in an optional implementation, as Figure 5 As shown, the third processing unit includes:
[0132] a first determination module, configured to, when the vector length of the first target vector is greater than or equal to a first threshold and less than a second threshold, respectively calculate the cosine similarity for each of the first target vectors to obtain a first similarity, and when the first similarity is greater than a fifth threshold, splice each of the first target vectors with a first preset weight until the vector length of the spliced first target vector is greater than or equal to a third threshold, and determine the first target vector as the second target vector;
[0133] Specifically, in order to further solve the problem of a large span in the length of short text data and reduce its impact on feature recognition results, the present application sets limits of 20, 100, 200 and 300, that is, the first threshold, the second threshold, the third threshold and the fourth threshold to divide the first target vector. When the vector length of the first target vector is greater than or equal to 20 and less than 100, the cosine similarity is calculated within the text of this length to obtain the first similarity. When the first similarity is greater than α, the first target vector is spliced with the first preset weight. When the first similarity is not satisfied, the original vector is retained.
[0134] A second determination module is used for, when the vector length of the first target vector is greater than the second threshold and less than or equal to the third threshold, respectively calculating the cosine similarity for each of the first target vectors to obtain a second similarity, and when the second similarity is greater than a sixth threshold, splicing each of the first target vectors with a second preset weight until the vector length of the spliced first target vector is greater than or equal to the third threshold, and determining the first target vector as the second target vector;
[0135] Specifically, when the vector length of the first target vector is greater than or equal to 100 and less than 200, the cosine similarity is calculated within the text of this length to obtain the second similarity. When the second similarity is greater than β, the first target vector is spliced with the second preset weight. When the second similarity is not satisfied, the original vector is retained.
[0136] The clipping module is used for clipping each of the first target vectors by the fourth threshold value to obtain the second target vector when the first target vector is greater than the fourth threshold value.
[0137] Specifically, when the vector length of the first target vector is greater than or equal to 300, the text of this length is cropped.
[0138] In order to ensure that the length of the corrected vector is uniform, in an optional implementation manner, the apparatus further includes:
[0139] The fourth processing unit is used to fill the third target vector with a preset character before determining the third target vector as the second target vector, until the vector length of the third target vector is equal to the fourth threshold.
[0140] Specifically, when the vector length of the concatenated vector or the original vector is greater than or equal to 200 and less than 300, the vector length between 200 and 300 is increased to 300 by adding 0.
[0141] It can be understood that the above embodiment ensures that the vector length is uniform as much as possible, reducing the impact of large differences in text length on subsequent feature extraction.
[0142] In order to identify the target entity, in an optional implementation, the input unit includes:
[0143] A first input module, configured to convolve the second target vector through the CNN layer to obtain a third target vector;
[0144] Specifically, the above-mentioned CNN-BIGRU-Softmax feature extraction model includes a CNN layer, a BIGRU layer and a Softmax layer. CNN realizes the expression of the target text entity vector in a higher dimension through convolution operation to obtain the above-mentioned third target vector.
[0145] The second input module is used to perform a pooling operation on the third target vector, and perform feature recognition on the third target vector after pooling through a BIGRU layer to extract time series features to obtain a fourth target vector;
[0146] Specifically, the third target vector is reduced in dimension by a pooling operation, and then input into BIGRU to identify the temporal features of the target entity text context through a bidirectional gated recurrent network to obtain the fourth target vector.
[0147] The third input module is used to calculate according to the fourth target vector through the Softmax layer to obtain the target probability corresponding to each of the fourth target vectors, and determine the entity corresponding to the fourth target vector corresponding to the maximum value of the target probability as the third text data.
[0148] Specifically, the probability of converting the BIGRU output feature into a label is calculated through the Softmax activation function, and the entity with the maximum output probability is the target entity, that is, the third text data mentioned above is obtained.
[0149] The above-mentioned fault text entity recognition device based on deep learning includes a processor and a memory. The above-mentioned first acquisition unit, first processing unit, second processing unit, third processing unit and input unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in the form of any combination.
[0150] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the influence of text length on entity extraction can be reduced by adjusting kernel parameters.
[0151] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0152] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the fault text entity recognition method based on deep learning.
[0153] Specifically, the fault text entity recognition method based on deep learning includes:
[0154] Step S201, obtaining an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure;
[0155] Specifically, Figure 3 As shown, the maintenance personnel report record log is obtained to obtain the above-mentioned engine fault log, which at least needs to include various entity records such as fault location, fault phenomenon, fault cause, treatment measures and maintenance time.
[0156] It is easy to understand that, in addition to the above entities, the above engine fault log also contains unstructured data describing various processing results, as well as connecting words of the maintenance personnel's personal recording habits.
[0157] Step S202, preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation;
[0158] Specifically, Figure 3As shown, since the above-mentioned engine fault log contains a large number of digital or alphabetic languages such as engine numbers, time and date, and there are many repetitive words, the present application is configured to pre-process the above-mentioned engine fault log before analysis, and streamline the above-mentioned engine fault log through methods such as regularization, deduplication operation, word segmentation operation and screening operation, shorten the record text therein, and obtain the above-mentioned first text data, so as to reduce the text length difference between texts to a certain extent, and reduce the impact of text length on subsequent feature extraction operations.
[0159] Step S203, processing the first text data based on the entity BIO annotation method to obtain second text data, and inputting the second text data into the BERT model to convert it into a vector representation to obtain a plurality of first target vectors;
[0160] Specifically, Figure 4 As shown, all entities included in the above-mentioned first text data are annotated based on the above-mentioned entity BIO annotation to obtain the above-mentioned second text data, so as to facilitate the subsequent model to identify the entity type, and then convert the above-mentioned second text data into a model language, that is, convert the above-mentioned second text data into a vector representation through the BERT model to obtain the above-mentioned first target vector.
[0161] Step S204, performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector;
[0162] Specifically, Figure 3 As shown, in order to eliminate the influence of text length on feature extraction during entity recognition, the present application is configured to unify the text length of the preprocessed text, that is, the above-mentioned first target vector is spliced or trimmed based on the above-mentioned preset range, and the text length is standardized to obtain the above-mentioned second target vector.
[0163] Step S205: input the second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is the fault text entity corresponding to the target entity in the engine fault log.
[0164] Specifically, the second target vector is divided into a training set and a test set, a feature extraction model is trained based on the training set and the test set, that is, a CNN-BIGRU-Softmax feature extraction model, and the effect of the feature extraction model is evaluated based on the test set. Then, when the loss function is less than a preset value, Figure 3 As shown, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain the third text data and complete the fault entity recognition.
[0165] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the deep learning-based fault text entity recognition method when running.
[0166] Specifically, the fault text entity recognition method based on deep learning includes:
[0167] Step S201, obtaining an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure;
[0168] Specifically, Figure 3 As shown, the maintenance personnel report record log is obtained to obtain the above-mentioned engine fault log, which at least needs to include various entity records such as fault location, fault phenomenon, fault cause, treatment measures and maintenance time.
[0169] It is easy to understand that, in addition to the above entities, the above engine fault log also contains unstructured data describing various processing results, as well as connecting words of the maintenance personnel's personal recording habits.
[0170] Step S202, preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation;
[0171] Specifically, Figure 3 As shown, since the above-mentioned engine fault log contains a large number of digital or alphabetic languages such as engine numbers, time and date, and there are many repetitive words, the present application is configured to pre-process the above-mentioned engine fault log before analysis, and streamline the above-mentioned engine fault log through methods such as regularization, deduplication operation, word segmentation operation and screening operation, shorten the record text therein, and obtain the above-mentioned first text data, so as to reduce the text length difference between texts to a certain extent, and reduce the impact of text length on subsequent feature extraction operations.
[0172] Step S203, processing the first text data based on the entity BIO annotation method to obtain second text data, and inputting the second text data into the BERT model to convert it into a vector representation to obtain a plurality of first target vectors;
[0173] Specifically, Figure 4 As shown, all entities included in the above-mentioned first text data are annotated based on the above-mentioned entity BIO annotation to obtain the above-mentioned second text data, so as to facilitate the subsequent model to identify the entity type, and then convert the above-mentioned second text data into a model language, that is, convert the above-mentioned second text data into a vector representation through the BERT model to obtain the above-mentioned first target vector.
[0174] Step S204, performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector;
[0175] Specifically, Figure 3 As shown, in order to eliminate the influence of text length on feature extraction during entity recognition, the present application is configured to unify the text length of the preprocessed text, that is, the above-mentioned first target vector is spliced or trimmed based on the above-mentioned preset range, and the text length is standardized to obtain the above-mentioned second target vector.
[0176] Step S205: input the second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is the fault text entity corresponding to the target entity in the engine fault log.
[0177] Specifically, the second target vector is divided into a training set and a test set, a feature extraction model is trained based on the training set and the test set, that is, a CNN-BIGRU-Softmax feature extraction model, and the effect of the feature extraction model is evaluated based on the test set. Then, when the loss function is less than a preset value, Figure 3 As shown, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain the third text data and complete the fault entity recognition.
[0178] An embodiment of the present invention provides a data processing system, wherein the data processing system includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are implemented:
[0179] Step S201, obtaining an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure;
[0180] Specifically, Figure 3 As shown, the maintenance personnel report record log is obtained to obtain the above-mentioned engine fault log, which at least needs to include various entity records such as fault location, fault phenomenon, fault cause, treatment measures and maintenance time.
[0181] It is easy to understand that, in addition to the above entities, the above engine fault log also contains unstructured data describing various processing results, as well as connecting words of the maintenance personnel's personal recording habits.
[0182] Step S202, preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation;
[0183] Specifically, Figure 3 As shown, since the above-mentioned engine fault log contains a large number of digital or alphabetic languages such as engine numbers, time and date, and there are many repetitive words, the present application is configured to pre-process the above-mentioned engine fault log before analysis, and streamline the above-mentioned engine fault log through methods such as regularization, deduplication operation, word segmentation operation and screening operation, shorten the record text therein, and obtain the above-mentioned first text data, so as to reduce the text length difference between texts to a certain extent, and reduce the impact of text length on subsequent feature extraction operations.
[0184] Step S203, processing the first text data based on the entity BIO annotation method to obtain second text data, and inputting the second text data into the BERT model to convert it into a vector representation to obtain a plurality of first target vectors;
[0185] Specifically, Figure 4 As shown, all entities included in the above-mentioned first text data are annotated based on the above-mentioned entity BIO annotation to obtain the above-mentioned second text data, so as to facilitate the subsequent model to identify the entity type, and then convert the above-mentioned second text data into a model language, that is, convert the above-mentioned second text data into a vector representation through the BERT model to obtain the above-mentioned first target vector.
[0186] Step S204, performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector;
[0187] Specifically, Figure 3 As shown, in order to eliminate the influence of text length on feature extraction during entity recognition, the present application is configured to unify the text length of the preprocessed text, that is, the above-mentioned first target vector is spliced or trimmed based on the above-mentioned preset range, and the text length is standardized to obtain the above-mentioned second target vector.
[0188] Step S205: input the second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is the fault text entity corresponding to the target entity in the engine fault log.
[0189] Specifically, the second target vector is divided into a training set and a test set, a feature extraction model is trained based on the training set and the test set, that is, a CNN-BIGRU-Softmax feature extraction model, and the effect of the feature extraction model is evaluated based on the test set. Then, when the loss function is less than a preset value, Figure 3As shown, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain the third text data and complete the fault entity recognition.
[0190] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:
[0191] Step S201, obtaining an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure;
[0192] Specifically, Figure 3 As shown, the maintenance personnel report record log is obtained to obtain the above-mentioned engine fault log, which at least needs to include various entity records such as fault location, fault phenomenon, fault cause, treatment measures and maintenance time.
[0193] It is easy to understand that, in addition to the above entities, the above engine fault log also contains unstructured data describing various processing results, as well as connecting words of the maintenance personnel's personal recording habits.
[0194] Step S202, preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation;
[0195] Specifically, Figure 3 As shown, since the above-mentioned engine fault log contains a large number of digital or alphabetic languages such as engine numbers, time and date, and there are many repetitive words, the present application is configured to pre-process the above-mentioned engine fault log before analysis, and streamline the above-mentioned engine fault log through methods such as regularization, deduplication operation, word segmentation operation and screening operation, shorten the record text therein, and obtain the above-mentioned first text data, so as to reduce the text length difference between texts to a certain extent, and reduce the impact of text length on subsequent feature extraction operations.
[0196] Step S203, processing the first text data based on the entity BIO annotation method to obtain second text data, and inputting the second text data into the BERT model to convert it into a vector representation to obtain a plurality of first target vectors;
[0197] Specifically, Figure 4 As shown, all entities included in the above-mentioned first text data are annotated based on the above-mentioned entity BIO annotation to obtain the above-mentioned second text data, so as to facilitate the subsequent model to identify the entity type, and then convert the above-mentioned second text data into a model language, that is, convert the above-mentioned second text data into a vector representation through the BERT model to obtain the above-mentioned first target vector.
[0198] Step S204, performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector;
[0199] Specifically, Figure 3 As shown, in order to eliminate the influence of text length on feature extraction during entity recognition, the present application is configured to unify the text length of the preprocessed text, that is, the above-mentioned first target vector is spliced or trimmed based on the above-mentioned preset range, and the text length is standardized to obtain the above-mentioned second target vector.
[0200] Step S205: input the second target vector into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is the fault text entity corresponding to the target entity in the engine fault log.
[0201] Specifically, the second target vector is divided into a training set and a test set, a feature extraction model is trained based on the training set and the test set, that is, a CNN-BIGRU-Softmax feature extraction model, and the effect of the feature extraction model is evaluated based on the test set. Then, when the loss function is less than a preset value, Figure 3 As shown, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain the third text data and complete the fault entity recognition.
[0202] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0203] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0204] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0207] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0208] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0209] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0210] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0211] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:
[0212] 1) The deep learning-based fault text entity recognition method of the present application first obtains an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure; then, the engine fault log is preprocessed to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; then, the first text data is processed based on the entity BIO annotation method to obtain second text data, and the second text data is input into the BERT model to be converted into a vector representation to obtain multiple first target vectors; then, a splicing or cropping operation is performed based on the first target vector so that the vector length of the first target vector is within a preset range to obtain a second target vector; finally, the second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, wherein the third text data is the fault text entity corresponding to the target entity in the engine fault log. This application first removes meaningless words based on screening, and assigns weights to words in the text based on TF-IDF and deletes low-weight words to shorten the text length and reduce the impact of text length on feature extraction. On this basis, the text is spliced or trimmed based on the text length to unify the text length without changing the feature points, eliminating the impact of text length on feature extraction, and then the classification model is used for feature extraction and the target entity is matched. This method solves the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy.
[0213] 2) The deep learning-based fault text entity recognition device of the present application comprises: a first acquisition unit acquires an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure; a first processing unit pre-processes the engine fault log to obtain first text data, wherein the pre-processing at least includes regularization, deduplication operation, word segmentation operation and screening operation; a second processing unit processes the first text data based on an entity BIO annotation method to obtain second text data, and inputs the second text data into a BERT model to convert it into a vector representation to obtain a plurality of first target vectors; a third processing unit performs a splicing or cropping operation based on the first target vector so that the vector length of the first target vector is within a preset range to obtain a second target vector; an input unit inputs the second target vector into a CNN-BIGRU-Softmax feature extraction model for identification to obtain third text data, wherein the third text data is a fault text entity corresponding to the target entity in the engine fault log. This application first removes meaningless words based on screening, and assigns weights to words in the text based on TF-IDF and deletes low-weight words to shorten the text length and reduce the impact of text length on feature extraction. On this basis, the text is spliced or trimmed based on the text length to unify the text length without changing the feature points, eliminating the impact of text length on feature extraction, and then the classification model is used for feature extraction and the target entity is matched. This method solves the problem that the named entity recognition method in the prior art does not consider the impact of different text lengths on feature extraction, resulting in low accuracy.
[0214] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A fault text entity recognition method based on deep learning, characterized in that: include: Obtaining an engine fault log, wherein the engine fault log includes at least a fault location, a fault cause, a fault phenomenon, and a treatment measure; Preprocessing the engine fault log to obtain first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; Processing the first text data based on the entity BIO annotation method to obtain second text data, and inputting the second text data into the BERT model to convert it into a vector representation to obtain multiple first target vectors; Performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector; The second target vector is input into the CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is the fault text entity corresponding to the target entity in the engine fault log.
2. The method according to claim 1, characterized in that Preprocessing the engine fault log to obtain first text data includes: Regularizing the engine fault log to remove character data and label data in the engine fault log to obtain third text data; Performing a deduplication operation on the third text data to obtain fourth text data; Segmenting the fourth text data based on Jieba word segmentation technology, and deleting preset words therein to obtain fifth text data, wherein the preset words include one or more of stop words, modal particles, and conjunctions; The first text data is obtained by adding a corresponding weight parameter to the fifth text data through a TF-IDF algorithm and deleting the fifth text data whose weight parameter is less than a first threshold.
3. The method according to claim 2, characterized in that Adding corresponding weight parameters to the fifth text data by using the TF-IDF algorithm includes: The weight parameter is calculated based on the fifth text data by using the first target formula: Among them, TF-IDF is the weight parameter, is the frequency value of the i-th word in document j, n i,j is the number of occurrences of the i-th word in document j, ∑ m n m,j is the total number of occurrences of all words in document j, is the inverse word frequency of the i-th word in the document j in the fifth text data, where D| is the total number of documents, |{j:q i ∈d j }| contains the word q i The total number of documents, v qi For word q i The corresponding preset weight, wherein the preset weight is based on the second objective formula Calculated, where For word q i Frequency of occurrence, t qi For word q i The corresponding preset rating.
4. The method according to claim 1, characterized in that: The first text data is processed based on the entity BIO annotation method to obtain second text data, including: The entities in the first text data are traversed, a first preset identifier is added to the starting position of the entity, a second preset identifier is added to the ending position of the entity, and a third identifier information is added to the part of the entity except the starting position and the ending position to obtain the second text data.
5. The method according to claim 1, characterized in that Performing a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range to obtain a second target vector, including: When the vector length of the first target vector is greater than or equal to a first threshold and less than a second threshold, respectively calculate the cosine similarity for each of the first target vectors to obtain a first similarity, and when the first similarity is greater than a fifth threshold, splice each of the first target vectors with a first preset weight until the vector length of the spliced first target vector is greater than or equal to a third threshold, and determine the first target vector as the second target vector; When the vector length of the first target vector is greater than the second threshold and less than or equal to the third threshold, respectively calculate the cosine similarity for each of the first target vectors to obtain a second similarity, and when the second similarity is greater than a sixth threshold, splice each of the first target vectors with a second preset weight until the vector length of the spliced first target vector is greater than or equal to the third threshold, and determine the first target vector as the second target vector; When the first target vector is greater than a fourth threshold, each of the first target vectors is clipped by the fourth threshold to obtain the second target vector.
6. The method according to claim 5, characterized in that Before determining the first target vector as the second target vector, the method further includes: The first target vector is padded with preset characters until the vector length of the first target vector is equal to the fourth threshold.
7. The method according to claim 1, characterized in that The CNN-BIGRU-Softmax feature extraction model includes a CNN layer, a BIGRU layer and a Softmax layer. The second target vector is input into the CNN-BI GRU-Softmax feature extraction model for recognition to obtain third text data, including: Convolving the second target vector through the CNN layer to obtain a third target vector; Performing a pooling operation on the third target vector, and performing feature recognition on the pooled third target vector through a BIGRU layer to extract time series features to obtain a fourth target vector; The Softmax layer performs calculation according to the fourth target vector to obtain the target probability corresponding to each of the fourth target vectors, and determines the entity corresponding to the fourth target vector corresponding to the maximum value of the target probability as the third text data.
8. A fault text entity recognition device based on deep learning, characterized in that: The device comprises: A first acquisition unit, used to acquire an engine fault log, wherein the engine fault log at least includes a fault location, a fault cause, a fault phenomenon and a treatment measure; A first processing unit, configured to preprocess the engine fault log to obtain a plurality of first text data, wherein the preprocessing at least includes regularization, deduplication operation, word segmentation operation and screening operation; A second processing unit is used to process the first text data based on the entity BIO annotation method to obtain second text data, and input the second text data into a BERT model to convert it into a vector representation to obtain a first target vector; A third processing unit is used to perform a splicing or clipping operation based on the first target vector so that the vector length of the first target vector is within a preset range, thereby obtaining a second target vector; An input unit is used to input the second target vector into a CNN-BIGRU-Softmax feature extraction model for recognition to obtain third text data, where the third text data is a fault text entity corresponding to the target entity in the engine fault log.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.
10. A data processing system, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of claims 1 to 7.
