A terminology processing method based on deep learning

Abstract

The present invention discloses a term processing method based on deep learning, which belongs to the technical field of natural language processing. The method comprises the following steps: constructing a BERT+BiLSTM+CRF model to extract terms, utilizing a feature extraction algorithm in combination with a multi-layer perceptron MLP model to perform difficulty-layered annotations on terms, and performing style adjustment through a natural language processing model T5. The method solves the technical problem of realizing efficient term recognition, difficulty assessment, and automatic annotation generation through deep learning technology. The present invention not only improves the accuracy of term recognition, but also significantly improves processing efficiency and user experience through automated difficulty assessment and annotation generation. The method can identify terms with different meanings in different contexts, thereby improving recognition accuracy and reducing labor costs. The method can provide personalized annotations for different users, thereby improving user experience, ensuring that the language style of the annotations is consistent with the original text, and enhancing the coherence of information transmission.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a terminology processing method based on deep learning. Background Art

[0002] In the field of natural language processing, term extraction and annotation generation technology has made some progress, mainly based on traditional rules and statistical methods, such as TF-IDF, LDA, etc. However, these methods have the following disadvantages: 1) The ability to recognize and understand complex terms is insufficient, and context information cannot be effectively captured; 2) It is difficult to deal with the ambiguity of polysemy and synonyms; 3) The difficulty assessment of terms often relies on manual experience and lacks automation and systematization. With the development of deep learning technology, neural network-based methods have gradually replaced traditional technologies, but in practical applications, they still face the challenges of high-dimensional feature processing and context modeling. Summary of the invention

[0003] The purpose of the present invention is to provide a terminology processing method based on deep learning, which solves the technical problem of achieving efficient terminology recognition, difficulty assessment and automatic annotation generation through deep learning technology.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A terminology processing method based on deep learning includes the following steps:

[0006] Step 1: The document collection module crawls or downloads the corpus in the technical field from public resources through the Internet to establish a basic data set; the document collection module adds a data upload interface module to the client for users to upload technical documents;

[0007] Step 2: The term extraction module obtains the basic data set and technical documents, builds the BERT+BiLSTM+CRF model, uses the basic data set as the training set to train the constructed BERT+BiLSTM+CRF model, uses the cross entropy loss function to calculate the error between the predicted label and the actual label, and uses the Adam optimization algorithm to optimize the BERT+BiLSTM+CRF model;

[0008] Get text data from technical documents, use the optimized BERT+BiLSTM+CRF model to process context information, identify and extract a list of terms, output the data as labeled data containing each term, and send it to the difficulty assessment module;

[0009] Step 3: The difficulty assessment module receives the term annotation data, uses the feature extraction algorithm combined with the multi-layer perceptron MLP model to stratify the difficulty of the terms, generates output data of the terms and their difficulty labels, and sends them to the term annotation generation module;

[0010] Step 4: The term annotation generation module retrieves annotations of corresponding difficulty from the database according to the terms and their difficulty labels, outputs a complete data package containing the terms and their corresponding annotations, and sends it to the annotation style preservation module;

[0011] Step 5: The annotation style preservation module receives the complete data packet of the term and its corresponding annotation, and performs style adjustment through the natural language processing model T5 to make the narrative style of the annotation language consistent with the narrative style of the original text. Finally, it outputs the formatted term annotation data and sends the result to the user end for display.

[0012] Preferably, when executing step 1, the document collection module specifically performs the following steps:

[0013] Step 1-1: The document collection module collects public text data through Internet crawling or from open source technology corpora. The public data includes term names and term annotations.

[0014] Step 1-2: Preprocess the collected text data to generate a basic data set;

[0015] Step 1-3: Use the basic dataset as an input data for the term extraction module;

[0016] Step 1-4: Create a data upload interface on the user side, and the user uploads the technical documents on the user side;

[0017] Step 1-5: Use the technical documents as another input data for the term extraction module.

[0018] Preferably, when executing step 2, the term extraction module specifically performs the following steps:

[0019] Step 2-1: Obtain basic data sets and technical documentation;

[0020] Step 2-2: Build a deep learning sequence labeling model based on BERT+BiLSTM+CRF, that is, the BERT+BiLSTM+CRF model. Use the text in the basic data set as the training set, input it into the BERT encoding layer, and convert the input sequence after word segmentation into a contextual semantic vector. The embedding vector of each word will be adjusted based on the context information to form an embedding representation suitable for the sequence labeling task:

[0021] ;

[0022] represents the context embedding vector of the i-th word; x i represents the i-th word in the input sequence; θ is the model parameter of BERT;

[0023] Step 2-3: The BiLSTM layer captures the embedding vector generated by the BERT encoding layer, processes it into a feature-enhanced hidden layer representation, and generates a bidirectional LSTM hidden state representation for each word:

[0024] ;

[0025] in, Represents the hidden state of the i-th word after bidirectional LSTM processing;

[0026] Step 2-4: The CRF layer predicts the label of each word by maximizing the conditional probability of the sequence and generates the annotation results for each word:

[0027] ;

[0028] Among them, P(y|X) represents the label sequence probability given the input sequence X; y i Represents the label of word i, y i−1 Indicates the label of the previous word; score(y i ,y i−1 ,X) is the joint score of the current label and the previous label; Y represents the set Y of all possible label sequences; y' represents all possible label combinations of the input sequence X;

[0029] Step 2-5: Use the cross entropy loss function to calculate the error between the predicted label and the actual label, and use the Adam optimization algorithm to optimize the parameters of the BERT+BiLSTM+CRF model:

[0030] The cross entropy loss function is used to calculate the error between the predicted label and the actual label, as shown in the following formula:

[0031] ;

[0032] in, represents the cross entropy loss, which is used to adjust the parameters of the BERT+BiLSTM+CRF model; i and Represent the actual label and the predicted label respectively;

[0033] The Adam optimization algorithm adjusts the model parameters to minimize the loss function, as shown in the following formula:

[0034] ;

[0035] Among them, m t is the first-order momentum at the tth iteration, i.e., the exponentially weighted average of the gradient; m t−1 is the first-order momentum at the t−1th iteration; β1 is the decay rate hyperparameter of the first-order momentum, with a default value of 0.9; g tis the current gradient, that is, the gradient value at the tth iteration; v t is the second-order momentum at the tth iteration; v t−1 is the second-order momentum at the t−1th iteration; β2 is the decay rate hyperparameter of the second-order momentum, with a default value of 0.999; is the square of the gradient, used to estimate the variance of the gradient;

[0036] Step 2-6: Use the optimized BERT+BiLSTM+CRF model and the methods from step 2-2 to step 2-4 to process the contextual information of the technical document, identify and extract the term list, and output the labeled data containing each term.

[0037] Preferably, when executing step 3, the specific steps are as follows:

[0038] Step 3-1: Perform feature analysis on term length L, frequency F, and complexity C to generate difficulty vector V d :

[0039] V d = [L,F,C];

[0040] Step 3-2: Use the multi-layer perceptron MLP model to score the term difficulty, that is, the category probability p. The specific formula is as follows:

[0041] ;

[0042] Among them, p is the output category probability, W i and W h is the weight matrix, b i and b h is the bias, σ represents the activation function;

[0043] Step 3-3: According to the three levels of easy, medium and difficult, the terms are divided into different difficulty levels according to the category probability p, and the term level labels are output;

[0044] Step 3-4: According to the term level label, select the level of detail of the explanation, that is, the length of the explanation.

[0045] Preferably, when executing step 4, annotations are preset in the database for different levels of difficulty.

[0046] The deep learning-based term processing method described in the present invention solves the technical problem of achieving efficient term recognition, difficulty assessment and automatic annotation generation through deep learning technology. The present invention adopts deep learning models such as BERT, BiLSTM and CRF, which not only improves the accuracy of term recognition, but also significantly improves processing efficiency and user experience through automatic difficulty assessment and annotation generation. By utilizing the context understanding ability of BERT, terms with different meanings in different contexts can be identified, thereby improving the accuracy of recognition. The deep learning model is used to automatically complete term extraction and annotation generation, thereby reducing dependence on manual intervention and reducing labor costs. A multi-layer perceptron (MLP) is introduced to perform difficulty assessment on terms, thereby providing personalized annotations for different users and improving user experience. By using text generation models such as T5, it is ensured that the language style of the annotation is consistent with the original text, thereby enhancing the coherence of information transmission. By using the CRF layer and optimization algorithms (such as Adam), efficient parameter updates and model training are achieved, thereby improving the overall processing speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the main flow chart of the present invention;

[0048] Figure 2 is a flow chart of step 1 of the present invention;

[0049] Figure 3 is a flow chart of step 2 of the present invention;

[0050] Figure 4 This is a flow chart of step 3 of the present invention. DETAILED DESCRIPTION

[0051] like Figure 1-Figure 4 A term processing method based on deep learning is shown, comprising the following steps:

[0052] Step 1: The document collection module crawls or downloads the corpus in the technical field from public resources through the Internet to establish a basic data set; the document collection module adds a data upload interface module to the client for users to upload technical documents;

[0053] When executing step 1, the document collection module specifically performs the following steps:

[0054] Step 1-1: The document collection module collects public text data through Internet crawling or from open source technology corpora. The public data includes term names and term annotations.

[0055] In this embodiment, an existing annotated dataset is obtained from a publicly available technical document dataset, such as Wikipedia, academic databases, or industry-specific terminology libraries (such as medicine, computer science, etc.). These documents usually contain annotations and explanations of terms.

[0056] Step 1-2: Preprocess the collected text data to generate a basic data set;

[0057] The document that the user inputs and expects to annotate is uploaded by establishing a file upload interface on the user side. The user can directly drag or select the file to upload. This interface can support multiple file formats (such as PDF, Word, TXT), and provide options to indicate the type of uploaded document, the expected annotation method, etc.

[0058] In this embodiment, a backend server is deployed. The document collection module, term extraction module, difficulty assessment module, term annotation generation module and annotation style maintenance module are all established in the backend server. A special file receiving interface is designed in the backend service to receive files through the API. REST API or GraphQL can be used to receive requests and transmit documents, and the requirements entered by users are recorded at the same time.

[0059] The user side can use HTML and JavaScript (such as React or Vue) to create a file upload component that provides the function of dragging or selecting files; use JavaScript to perform format and size verification before uploading files to ensure that the files uploaded by users meet the requirements; package the files and additional annotation requirements (such as annotation depth, term type) into an HTTP request and send it to the backend API via the POST method.

[0060] The backend server can use frameworks such as Python's Flask or Django, Node.js's Express, etc. to process HTTP requests and configure the path of the file upload API; the backend stores the file in the server's file system or cloud storage (such as AWS S3), and records the user's annotation requirements (such as annotation format, term type) in the database; checks the file format and parses the content into a unified format (such as text format) to facilitate subsequent term extraction and processing.

[0061] Step 1-3: Use the basic dataset as an input data for the term extraction module to build a training set and train the BERT+BiLSTM+CRF model;

[0062] Step 1-4: Create a data upload interface on the user side, and the user uploads the technical documents on the user side;

[0063] Step 1-5: Use the technical document as another input data of the term extraction module to store the text file that the user expects to annotate.

[0064] Step 2: The term extraction module obtains the basic data set and technical documents, builds the BERT+BiLSTM+CRF model, uses the basic data set as the training set to train the constructed BERT+BiLSTM+CRF model, uses the cross entropy loss function to calculate the error between the predicted label and the actual label, and uses the Adam optimization algorithm to optimize the BERT+BiLSTM+CRF model;

[0065] Get text data from technical documents, use the optimized BERT+BiLSTM+CRF model to process context information, identify and extract a list of terms, output the data as labeled data containing each term, and send it to the difficulty assessment module;

[0066] When executing step 2, the term extraction module specifically performs the following steps:

[0067] Step 2-1: Obtain basic data sets and technical documentation;

[0068] Step 2-2: Build a deep learning sequence labeling model based on BERT+BiLSTM+CRF, that is, the BERT+BiLSTM+CRF model. Use the text in the basic data set as the training set, input it into the BERT encoding layer, and convert the input sequence after word segmentation into a contextual semantic vector. The embedding vector of each word will be adjusted based on the context information to form an embedding representation suitable for the sequence labeling task:

[0069] ;

[0070] represents the context embedding vector of the i-th word; x i represents the i-th word in the input sequence; θ is the model parameter of BERT;

[0071] The BERT encoding layer converts the input sequence after word segmentation into a contextual semantic vector. The embedding vector of each word is adjusted based on the context information to form an embedding representation suitable for sequence labeling tasks.

[0072] Step 2-3: The BiLSTM layer captures the embedding vector generated by the BERT encoding layer, processes it into a feature-enhanced hidden layer representation, and generates a bidirectional LSTM hidden state representation for each word:

[0073] ;

[0074] in, Represents the hidden state of the i-th word after bidirectional LSTM processing;

[0075] The BiLSTM layer further captures sequence information and considers the bidirectional context before and after each word, which can deepen the sequential information in the vector extracted by BERT. BiLSTM processes the output sequence of BERT into a hidden layer representation with feature enhancement.

[0076] Step 2-4: The CRF layer predicts the label of each word by maximizing the conditional probability of the sequence and generates the annotation results for each word:

[0077] ;

[0078] Among them, P(y|X) represents the label sequence probability given the input sequence X; y i Represents the label of word i, y i−1 Indicates the label of the previous word; score(y i ,y i−1 ,X) is the joint score of the current label and the previous label; Y represents the set Y of all possible label sequences; y' represents all possible label combinations of the input sequence X;

[0079] The CRF layer predicts the label of each word by maximizing the conditional probability of the sequence. CRF can use the context sequence information to adjust the labeling result of each word and is suitable for the recognition of continuous terms.

[0080] Step 2-5: Use the cross entropy loss function to calculate the error between the predicted label and the actual label, and use the Adam optimization algorithm to optimize the parameters of the BERT+BiLSTM+CRF model:

[0081] The cross entropy loss function is used to calculate the error between the predicted label and the actual label, as shown in the following formula:

[0082] ;

[0083] in, represents the cross entropy loss, which is used to adjust the parameters of the BERT+BiLSTM+CRF model; i and Represent the actual label and the predicted label respectively;

[0084] The Adam optimization algorithm adjusts the model parameters to minimize the loss function, as shown in the following formula:

[0085] ;

[0086] Among them, m t is the first-order momentum at the tth iteration, i.e., the exponentially weighted average of the gradient; m t−1 is the first-order momentum at the t−1th iteration; β1 is the decay rate hyperparameter of the first-order momentum, with a default value of 0.9; gt is the current gradient, that is, the gradient value at the tth iteration; v t is the second-order momentum at the tth iteration; v t−1 is the second-order momentum at the t−1th iteration; β2 is the decay rate hyperparameter of the second-order momentum, with a default value of 0.999; is the square of the gradient, used to estimate the variance of the gradient;

[0087] Adam (Adaptive Moment Estimation) is an adaptive learning rate optimization algorithm that uses first-order and second-order momentum estimates to adjust the learning rate to improve the convergence speed and stability of gradient descent. Its main advantage is that it dynamically adjusts the learning rate and has good adaptability to noise and sparse gradients.

[0088] Step 2-6: Use the optimized BERT+BiLSTM+CRF model and the methods from step 2-2 to step 2-4 to process the contextual information of the technical document, identify and extract the term list, and output the labeled data containing each term.

[0089] Step 3: The difficulty assessment module receives the term annotation data, uses the feature extraction algorithm combined with the multi-layer perceptron MLP model to stratify the difficulty of the terms, generates output data of the terms and their difficulty labels, and sends them to the term annotation generation module;

[0090] When executing step 3, the specific steps are as follows:

[0091] Step 3-1: Perform feature analysis on term length L, frequency F, and complexity C to generate difficulty vector V d :

[0092] V d = [L,F,C];

[0093] Step 3-2: Use the multi-layer perceptron MLP model to score the term difficulty, that is, the category probability p. The specific formula is as follows:

[0094] ;

[0095] Among them, p is the output category probability, W i and W h is the weight matrix, b i and b h is the bias, σ represents the activation function;

[0096] Step 3-3: According to the three levels of easy, medium and difficult, the terms are divided into different difficulty levels according to the category probability p, and the term level labels are output;

[0097] Step 3-4: According to the term level label, select the level of detail of the explanation, that is, the length of the explanation.

[0098] Step 4: The term annotation generation module retrieves annotations of corresponding difficulty from the database according to the terms and their difficulty labels, outputs a complete data package containing the terms and their corresponding annotations, and sends it to the annotation style preservation module to preset annotations for different difficulties in the database.

[0099] In this embodiment, the database design is specifically as follows: three explanations of different difficulty levels (detailed, general, and brief) are prepared for each term, and each explanation is a record field in the database.

[0100] After the hierarchical model evaluates the difficulty of a term, an explanation of the corresponding difficulty is extracted from the database based on the label, for example:

[0101] High difficulty: call for detailed explanation;

[0102] Medium difficulty: Calling ordinary explanation;

[0103] Low difficulty: call for brief explanation.

[0104] This can effectively improve the response speed of the system and provide explanations that match the difficulty level of the user's reading experience. The advantage is that it avoids repeated content generation and ensures consistency in explanations, and also facilitates subsequent explanation management and maintenance.

[0105] Step 5: The annotation style preservation module receives the complete data packet of the term and its corresponding annotation, and performs style adjustment through the natural language processing model T5 to make the narrative style of the annotation language consistent with the narrative style of the original text. Finally, it outputs the formatted term annotation data and sends the result to the user end for display.

[0106] T5 (Text-To-Text Transfer Transformer) is a natural language processing (NLP) model proposed by Google in 2020. It is a pre-trained model based on the Transformer architecture. Its core concept is to convert all NLP tasks into "text-to-text" problems, so whether it is translation, classification, summarization or question-answering, the input and output are unified in the form of text.

[0107] Google has released the T5 code and pre-trained model weights on its open source platform, which can be accessed and used through Hugging Face's Transformers library. Google's open source version contains multiple pre-trained models, including models of different sizes (such as T5-Small, T5-Base, T5-Large, T5-3B, T5-11B), to meet different computing resources and application requirements.

[0108] Text-to-Text unified framework: T5 simplifies the task form by converting all tasks into text-to-text problems. In this framework, the model converts input text into output text, regardless of the task type, whether it is translation, summarization, or question answering.

[0109] Pre-training objective (Span Corruption): T5 uses a pre-training objective called "span corruption", which is similar to filling in the blanks. The model will randomly cover some parts of the text and require the model to predict the content of these covered parts.

[0110] Model size: There are multiple versions of the T5 model, and different versions differ in parameter quantity and computing power, including T5-Small, T5-Base, T5-Large, T5-3B and T5-11B. As the number of parameters increases from small to large, the effect and computing requirements also gradually increase.

[0111] Large-scale dataset (C4) training: T5 uses a large text dataset called C4 (Colossal Clean Crawled Corpus), which contains high-quality text cleaned in the network, greatly improving the model's ability to handle real language tasks.

[0112] In this embodiment, when T5 is used to adjust the style, the following steps are specifically included:

[0113] Step 5-1: Organize the received terms and their corresponding annotations into text that conforms to the T5 input format. For example, the format can be "annotation task: term, original text: original sentence" to ensure that the model can understand the type of task to be performed.

[0114] Step 5-2: When using T5 for style preservation, you can consider introducing sample data with a style similar to the original text during training. In this way, the model will tend to imitate the language style of the original text when generating annotations, including word choice, syntax, and tone.

[0115] Step 5-3: Fine-tune T5 in a specific domain using domain-related corpus. Through continuous training and verification, the model's ability to understand specific terms and contexts can be improved, making the generated annotations more in line with user needs.

[0116] Step 5-4: Use the fine-tuned T5 model for inference to generate term annotations that are consistent with the original style. The model will output adjusted annotation text based on the input terms and context information.

[0117] Step 5-5: The generated annotations may undergo post-processing steps, such as formatting or filtering, to ensure that the output results meet the display requirements of the user end, including character limits, language fluency, etc.

[0118] Step 5-6: Incorporate user feedback into the model optimization process, and further adjust and optimize T5's generation strategy based on users' evaluation of the generated annotations to continuously improve annotation quality and user satisfaction.

[0119] Step 5-7: Design an API interface so that other modules in the backend server or user terminals can easily call the term annotations generated by T5 to achieve seamless data transmission and real-time feedback.

[0120] The deep learning-based term processing method described in the present invention solves the technical problem of achieving efficient term recognition, difficulty assessment and automatic annotation generation through deep learning technology. The present invention adopts deep learning models such as BERT, BiLSTM and CRF, which not only improves the accuracy of term recognition, but also significantly improves processing efficiency and user experience through automatic difficulty assessment and annotation generation. By utilizing the context understanding ability of BERT, terms with different meanings in different contexts can be identified, thereby improving the accuracy of recognition. The deep learning model is used to automatically complete term extraction and annotation generation, thereby reducing dependence on manual intervention and reducing labor costs. A multi-layer perceptron (MLP) is introduced to perform difficulty assessment on terms, thereby providing personalized annotations for different users and improving user experience. By using text generation models such as T5, it is ensured that the language style of the annotation is consistent with the original text, thereby enhancing the coherence of information transmission. By using the CRF layer and optimization algorithms (such as Adam), efficient parameter updates and model training are achieved, thereby improving the overall processing speed and accuracy.

Claims

1. A terminology processing method based on deep learning, characterized in that: The steps include: Step 1: The document collection module crawls or downloads the corpus in the technical field from public resources through the Internet to establish a basic data set; the document collection module adds a data upload interface module to the client for users to upload technical documents; Step 2: The term extraction module obtains the basic data set and technical documents, builds the BERT+BiLSTM+CRF model, uses the basic data set as the training set to train the constructed BERT+BiLSTM+CRF model, uses the cross entropy loss function to calculate the error between the predicted label and the actual label, and uses the Adam optimization algorithm to optimize the BERT+BiLSTM+CRF model; Get text data from technical documents, use the optimized BERT+BiLSTM+CRF model to process context information, identify and extract a list of terms, output the data as labeled data containing each term, and send it to the difficulty assessment module; Step 3: The difficulty assessment module receives the term annotation data, uses the feature extraction algorithm combined with the multi-layer perceptron MLP model to stratify the difficulty of the terms, generates output data of the terms and their difficulty labels, and sends them to the term annotation generation module; Step 4: The term annotation generation module retrieves annotations of corresponding difficulty from the database according to the terms and their difficulty labels, outputs a complete data package containing the terms and their corresponding annotations, and sends it to the annotation style preservation module; Step 5: The annotation style preservation module receives the complete data packet of the term and its corresponding annotation, and performs style adjustment through the natural language processing model T5 to make the narrative style of the annotation language consistent with the narrative style of the original text. Finally, it outputs the formatted term annotation data and sends the result to the user end for display.

2. A terminology processing method based on deep learning as claimed in claim 1, characterized in that: When executing step 1, the document collection module specifically performs the following steps: Step 1-1: The document collection module collects public text data by crawling the Internet or from open source technology corpora. The public data includes term names and term annotations. Step 1-2: Preprocess the collected text data to generate a basic data set; Step 1-3: Use the basic dataset as an input data for the term extraction module; Step 1-4: Create a data upload interface on the user side, and the user uploads the technical documents on the user side; Step 1-5: Use the technical documents as another input data for the term extraction module.

3. The terminology processing method based on deep learning according to claim 1, characterized in that: When executing step 2, the term extraction module specifically performs the following steps: Step 2-1: Obtain basic data sets and technical documentation; Step 2-2: Build a deep learning sequence labeling model based on BERT+BiLSTM+CRF, that is, the BERT+BiLSTM+CRF model. Use the text in the basic data set as the training set, input it into the BERT encoding layer, and convert the input sequence after word segmentation into a contextual semantic vector. The embedding vector of each word will be adjusted based on the context information to form an embedding representation suitable for the sequence labeling task: ； Indicates i The context embedding vector of each word; x i The first i words; θ are the model parameters of BERT; Step 2-3: The BiLSTM layer captures the embedding vector generated by the BERT encoding layer, processes it into a feature-enhanced hidden layer representation, and generates a bidirectional LSTM hidden state representation for each word: ； in, Represents the hidden state of the i-th word after being processed by the bidirectional LSTM; Step 2-4: The CRF layer predicts the label of each word by maximizing the conditional probability of the sequence and generates the annotation results for each word: ； in, P(y|X) Represents a given input sequence X The label sequence probability of y i Expressive words i Tags, Indicates the label of the previous word; is the joint score of the current label and the previous label; Y Represents the set of all possible label sequences Y ; y＇ Represents the input sequence X All possible label combinations of Step 2-5: Use the cross entropy loss function to calculate the error between the predicted label and the actual label, and use the Adam optimization algorithm to optimize the parameters of the BERT+BiLSTM+CRF model: The cross entropy loss function is used to calculate the error between the predicted label and the actual label, as shown in the following formula: ； in, Represents the cross entropy loss, which is used to adjust the parameters of the BERT+BiLSTM+CRF model; y i and Represent the actual label and the predicted label respectively; The Adam optimization algorithm adjusts the model parameters to minimize the loss function, as shown in the following formula: ； in, m t It is t The first-order momentum at the iteration, which is the exponentially weighted average of the gradient; It is The first-order momentum of the iteration; β 1 is the decay rate hyperparameter of the first-order momentum, the default value is 0.9; g t is the current gradient, that is, the gradient value at the tth iteration; v t It is t The second-order momentum of the iteration; It is The second-order momentum of the iteration; β 2 is the decay rate hyperparameter of the second-order momentum, the default value is 0.999; is the square of the gradient, used to estimate the variance of the gradient; Step 2-6: Use the optimized BERT+BiLSTM+CRF model and the methods from step 2-2 to step 2-4 to process the contextual information of the technical document, identify and extract the term list, and output the labeled data containing each term.

4. The terminology processing method based on deep learning according to claim 1, characterized in that: When executing step 3, the specific steps are as follows: Step 3-1: Term length L ,frequency F , Complexity C Perform feature analysis and generate difficulty vector V d : V d = [ L, F, C ]； Step 3-2: Use the multi-layer perceptron MLP model to score the term difficulty, i.e., the category probability p , the specific formula is as follows: ； in, p is the output category probability, W i and W h is the weight matrix, b i and b h is the bias, σ represents the activation function; Step 3-3: Set the three levels of easy, medium and difficult, and select the correct answer based on the category probability. p , classify the terms into different difficulty levels and output the term level labels; Step 3-4: According to the term level label, select the level of detail of the explanation, that is, the length of the explanation.

5. The terminology processing method based on deep learning according to claim 1, characterized in that: When executing step 4, annotations are preset in the database for different levels of difficulty.

Images