Model training method and device, equipment and storage medium

Abstract

The application discloses a model training method and device, equipment and a storage medium. The method comprises the following steps: obtaining text information of a material field literature and a corresponding entity type label sequence; performing word segmentation and word embedding processing on the text information by using a language representation model to obtain word vectors corresponding to a plurality of words, wherein the language representation model comprises a pre-training model for the material field; performing entity type identification on the plurality of words based on the word vectors to obtain entity type identification results corresponding to the plurality of words; and training an entity identification model according to the entity type identification results and the entity type label sequence to obtain a trained entity identification model for the material field text information, wherein the entity identification model comprises the language representation model. According to the scheme provided in the embodiments of the application, the entity type of the material field literature text information can be effectively identified, and then useful information can be extracted from the text information.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous research and development of new materials, there are more and more references in the field of materials. How to quickly extract useful information from the massive amount of references is one of the keys to improving the work efficiency of R&D personnel.

[0003] Currently, existing natural language recognition models trained on ordinary text information are not suitable for entity type recognition of text information in materials science literature. This can easily lead to the trained models being unable to effectively identify the entity types of text information in materials science literature, thus failing to extract useful information. Summary of the Invention

[0004] This application provides a model training method, apparatus, device, and storage medium, which enables the trained model to effectively identify entity types in textual information in materials science literature, thereby helping to extract useful information from the textual information.

[0005] Firstly, this application provides a model training method, which includes: acquiring textual information of materials science literature and its corresponding entity type label sequence, wherein the entity type label sequence includes entity type labels corresponding to multiple words in the textual information; performing word segmentation and word embedding processing on the textual information using a language representation model to obtain word vectors corresponding to multiple words, wherein the language representation model includes a pre-trained model for the materials science field; performing entity type recognition on the multiple words based on the word vectors to obtain entity type recognition results corresponding to the multiple words; and training the entity recognition model according to the entity type recognition results and the entity type label sequence to obtain a trained entity recognition model for materials science textual information, wherein the entity recognition model includes the language representation model.

[0006] In this way, by using textual information from materials science literature as training samples and employing a pre-trained language representation model specific to materials science, the textual information is segmented and embedded to obtain word vectors corresponding to multiple words. Then, based on the entity type identification results from these word vectors and the actual entity type label sequence of the textual information, the entity recognition model is trained, resulting in an entity recognition model specifically for materials science textual information. Because the model is trained using textual information from materials science literature, and the word embedding is performed using a pre-trained language representation model specific to materials science, the trained entity recognition model is more targeted towards materials science literature. This allows the trained entity recognition model to effectively identify entity types in the textual information of materials science literature, thereby helping to extract useful information from the textual information.

[0007] In some embodiments, obtaining the text information of materials science literature and its corresponding entity type label sequence includes: obtaining the text information of materials science literature and extracting word feature information of multiple words from the text information; performing clustering processing on the multiple word feature information to determine multiple entity types and at least one corresponding word feature information; adding entity type labels to the multiple words according to the entity type corresponding to each word feature information in the multiple word feature information to obtain an entity type label sequence corresponding to the text information.

[0008] In this way, by using clustering to determine the entity type of the words included in the text information, and then adding corresponding entity type labels to each word, the automatic identification of entity types and the automatic addition of labels can be achieved. This can solve the problems of inaccuracy and low efficiency caused by manual identification and manual labeling, and effectively improve the accuracy and processing efficiency of entity type identification and labeling.

[0009] In some embodiments, the step of clustering the plurality of word feature information to determine a plurality of entity types and at least one word feature information corresponding to them includes: clustering the plurality of word feature information to obtain a plurality of clustering results, wherein the number of clusters contained in different clustering results is different; determining the silhouette coefficients corresponding to the plurality of clustering results respectively; determining a plurality of entity types and at least one word feature information corresponding to each entity type based on the clustering result with the highest silhouette coefficient.

[0010] In this way, by evaluating multiple clustering results, the final result can be determined based on the clustering result with the highest silhouette coefficient, along with at least one word feature information corresponding to each entity type. This makes the final entity type more accurate, thereby further improving the accuracy of entity type identification and labeling processes.

[0011] In some embodiments, the entity recognition model further includes a semantic recognition model; the step of performing entity type recognition on the plurality of words based on the word vectors to obtain entity type recognition results corresponding to the plurality of words respectively includes: inputting the word vectors into the semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network; performing semantic recognition on the text information using the semantic recognition model, and outputting semantic recognition results corresponding to the plurality of words respectively; and determining entity type recognition results corresponding to the plurality of words based on the semantic recognition results.

[0012] Thus, by using a semantic recognition model built on a bidirectional long short-term memory network to perform semantic recognition on text information, the accuracy of semantic recognition results can be improved.

[0013] In some embodiments, the entity recognition model further includes an entity type prediction model; determining the entity type recognition result corresponding to the plurality of words based on the semantic recognition result includes: inputting the semantic recognition result into the entity type prediction model, wherein the entity type prediction model includes a model constructed based on a conditional random field; using the entity type prediction model to predict the entity type in the semantic recognition result, and outputting the entity type recognition result corresponding to the plurality of words.

[0014] In this way, by using an entity type prediction model based on conditional random fields to predict the entity type in the semantic recognition results, the probability of unreasonable recognition results in the entity type recognition results can be reduced.

[0015] In some embodiments, training the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for material domain text information includes: determining a standard semantic recognition result based on the entity type label sequence; determining a first loss value based on the semantic recognition result and the standard semantic recognition result; adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model; determining a second loss value based on the entity type recognition result and the entity type label sequence; and adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model.

[0016] In this way, by adjusting the parameters of the language representation model and the entity type prediction model respectively, personalized training of different models can be achieved, thereby improving the model training effect.

[0017] In some embodiments, adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model includes: adjusting the parameters of the language representation model according to a first learning rate based on the first loss value to obtain a trained language representation model; adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model includes: adjusting the parameters of the entity type prediction model according to a second learning rate based on the second loss value to obtain a trained entity type prediction model; wherein the first learning rate is less than the second learning rate.

[0018] In this way, by setting different learning rates for different models, that is, using a smaller first learning rate to adjust the parameters of the language representation model built based on the pre-trained model, and using a larger second learning rate to adjust the parameters of the entity type prediction model, the entity type prediction model can be fully trained while the language representation model converges.

[0019] In some embodiments, after adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model, the method further includes: adding a target perturbation value to the word vectors corresponding to the plurality of words respectively to obtain perturbed word vectors; re-identifying the entity types of the plurality of words based on the perturbed word vectors to obtain perturbed entity type identification results; determining a third loss value based on the perturbed entity type identification results and the entity type label sequence; calculating a second backpropagation gradient based on the third loss value; and updating the parameters of the entity type prediction model based on the second backpropagation gradient.

[0020] In this way, by adopting adversarial training, adding target perturbation values ​​to the parameters of the language representation model, changing the loss value of the entity type recognition result output by the entity type prediction model, and then updating the parameters of the entity type prediction model based on the changed loss value, the robustness and generalization ability of the entity type prediction model can be improved.

[0021] In some embodiments, before adding a target perturbation value to the word vectors corresponding to the plurality of words to obtain perturbated word vectors, the method further includes: calculating a first backpropagation gradient based on the second loss value; and determining the target perturbation value based on the first backpropagation gradient.

[0022] In this way, the target perturbation value can be dynamically determined by using the first backpropagation gradient calculated using the second loss value, thereby improving the effectiveness of adversarial training.

[0023] In some embodiments, determining the target perturbation value based on the first backpropagation gradient includes: calculating the Euclidean norm based on the first backpropagation gradient; and determining the ratio of the first backpropagation gradient to the Euclidean norm as the target perturbation value.

[0024] In this way, by calculating the Euclidean norm and using it to determine the target perturbation value, the effectiveness of adversarial training can be further improved.

[0025] In some embodiments, the entity type includes at least one of the following: material type, application type, research method type, research numerical type, material sample description type, experimental condition type, material property type, and experimental result type.

[0026] In some embodiments, after training the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for text information in the materials field, the method further includes: acquiring text information of the target materials field document to be identified, obtaining target text information; inputting the target text information into the entity recognition model; using the entity recognition model to perform entity type recognition on the target text information, and outputting a target entity type label sequence corresponding to the target text information.

[0027] Thus, by using the entity recognition model trained by the model training method provided in this application embodiment to identify the entity type of text information in the materials science literature, a more accurate entity type label sequence can be output, thereby effectively identifying the entity type of text information in the materials science literature, which in turn helps to extract useful information from the text information. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic flowchart of a model training method provided in one embodiment of this application;

[0030] Figure 2 This is a schematic diagram of entity annotation provided in one embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the structure of an entity recognition model provided in one embodiment of this application;

[0032] Figure 4This is a schematic diagram of the structure of a model training device provided in one embodiment of this application;

[0033] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0036] Currently, commonly used natural language recognition (NLP) models are trained using ordinary text information as samples and general model structures and training methods. However, ordinary text information primarily consists of everyday language, while academic text information typically contains a large number of technical terms, which are significantly different from everyday language. This is especially true for text information in the materials science field, which contains a large number of materials-specific technical terms. Therefore, commonly used NLP models are not suitable for entity type identification in materials science literature. This can easily lead to models trained using existing methods failing to effectively identify entity types in materials science literature text information, thus failing to extract useful information.

[0037] To address the aforementioned technical problems, embodiments of this application provide a model training method, apparatus, device, and storage medium. The model training method provided in this application uses textual information from materials science literature as training samples. It utilizes a pre-trained language representation model specific to the materials science domain to perform word segmentation and word embedding processing on the textual information, obtaining word vectors corresponding to multiple words. Then, based on the entity type recognition results identified by these word vectors and the actual entity type label sequence of the textual information, an entity recognition model is trained to obtain an entity recognition model for materials science textual information. Thus, because the model training uses textual information from materials science literature, and the word embedding uses a pre-trained language representation model specific to the materials science domain, the trained entity recognition model is more targeted towards materials science literature. This allows the trained entity recognition model to effectively identify entity types in the textual information of materials science literature, thereby facilitating the extraction of useful information from the textual information.

[0038] The model training method provided in this application embodiment will be described in detail below. This model training method can be applied to application scenarios involving entity type recognition of textual information in materials science literature. This model training method can be executed by an electronic device with computing capabilities. The electronic device may include terminal devices such as mobile phones and computers, or devices such as servers.

[0039] Figure 1 This is a schematic flowchart of a model training method provided in one embodiment of this application. Figure 1 As shown, the model training method may specifically include the following steps:

[0040] S110. Obtain the text information of literature in the field of materials and its corresponding entity type label sequence, wherein the entity type label sequence includes entity type labels corresponding to multiple words in the text information;

[0041] S120. Use a language representation model to segment and embed the text information to obtain word vectors corresponding to multiple words. The language representation model includes a pre-trained model for the material domain.

[0042] S130. Based on word vectors, perform entity type recognition on multiple words respectively to obtain entity type recognition results corresponding to multiple words respectively;

[0043] S140. Based on the entity type recognition results and entity type label sequence, train the entity recognition model to obtain the trained entity recognition model for text information in the material domain. The entity recognition model includes a language representation model.

[0044] Therefore, by using textual information from materials science literature as training samples and employing a pre-trained language representation model specific to materials science, the textual information is segmented and embedded to obtain word vectors corresponding to multiple words. Then, based on the entity type identification results from these word vectors and the actual entity type label sequence of the textual information, the entity recognition model is trained, resulting in an entity recognition model specifically for materials science textual information. Because the model is trained using textual information from materials science literature, and the word embedding is performed using a pre-trained language representation model specific to materials science, the trained entity recognition model is more targeted towards materials science literature. This allows the trained entity recognition model to effectively identify entity types in the textual information of materials science literature, thereby facilitating the extraction of useful information from the textual information.

[0045] The specific implementation methods for each of the above steps are described below.

[0046] In some embodiments, in S110, the materials science literature may include, for example, literature in the field of battery materials. The text information may include, for example, textual information such as characters and symbols. The text information may include multiple words, and the multiple words may include at least one of characters and terms.

[0047] In this embodiment, an entity refers to a word or phrase that has actual meaning in the text, and the specific task of entity recognition is to identify entities from natural language text. An entity consists of one or more words, and the specific entity types required can be defined in advance according to actual needs.

[0048] For example, when constructing training samples for a model in the materials science field, textual information from materials science literature can be obtained and entity annotation performed. This involves adding corresponding entity type labels to multiple words contained in the text information, resulting in an entity type label sequence corresponding to the text information. This text information and entity type label sequence are then used as training samples to train the entity recognition model. The entity type label sequence can include multiple entity type labels, with each entity type label corresponding to a word in the text information. The order of the multiple entity type labels can be the same as the order of the corresponding words in the text information.

[0049] For entity annotation, such as Figure 2As shown, this embodiment can use the "BIO" format for annotation: B - "Entity Type", I - "Entity Type", O. Here, B stands for "Begin", representing the first word or phrase of an entity; I stands for "Inside", representing all parts of an entity except the first word or phrase; and O stands for "Outside", representing non-entity words or phrases. An entity consisting of a single word or phrase is composed of only one type of B - "Entity Type" tag, while an entity consisting of multiple words or phrases contains both B - "Entity Type" and I - "Entity Type" entity type tags.

[0050] Based on this, in this embodiment, one entity type can correspond to two entity type labels. For example, five entity type labels can be set for two entity types x and y: Bx, Ix, By, Iy, and O.

[0051] In this embodiment, multiple entity types can be pre-defined, and entity type identification can be performed within the scope of these multiple entity types. In some implementations, the entity type may include at least one of the following: material type, application type, research method type, research numerical type, material sample description type, experimental condition type, material property type, and experimental result type. By setting the above types, key information corresponding to the above types can be extracted from materials science literature in a relatively comprehensive and accurate manner.

[0052] In some implementations, in S120, the entity recognition model for textual information in the materials domain to be trained in this embodiment may specifically include a language representation model.

[0053] Language representation models can be models capable of extracting word vectors from natural language information. Specifically, they can include pre-trained models for the materials science domain, such as the MatBERT model. MatBERT is a pre-trained model specifically for the materials science domain. Compared to the ordinary BERT (Bidirectional Encoder Representations from Transformers) model, the MatBERT model has been trained on a large amount of materials science literature and possesses excellent word embedding representation capabilities in this domain. Therefore, the MatBERT model can be used for word segmentation and word embedding processing.

[0054] For example, text information can be input into the MatBERT model, where the text information is segmented and word embedded to obtain word vectors corresponding to multiple words in the text information.

[0055] In some implementations, in S130, after obtaining multiple word vectors, entity type recognition can be performed on multiple words contained in the text information based on the multiple word vectors.

[0056] In this embodiment, the entity recognition model may include not only a language representation model but also other models. These other models can then be used to identify the entity types of the multiple words, yielding entity type recognition results. These results may include the entity type label corresponding to each word within its respective entity type.

[0057] In some implementations, in S140, after obtaining the entity type recognition result corresponding to each word in the text information, the entity recognition model can be trained based on the entity type recognition result of each word and the entity type label corresponding to it in the pre-set entity type label sequence, thereby obtaining the trained entity recognition model.

[0058] For example, the corresponding loss value can be calculated based on the entity type recognition result of each word and the entity type label corresponding to that word, and the parameters of the model included in the entity recognition model can be adjusted using the loss value until the model converges, thus completing the training process of the entity recognition model.

[0059] Based on this, when adding entity type labels to multiple words in the text information used as training samples, in order to improve the accuracy and processing efficiency of entity type recognition and label addition, in some implementations, the above-mentioned S110 may specifically include:

[0060] Acquire textual information from literature in the field of materials science, and extract word feature information of multiple words from the textual information;

[0061] Clustering is performed on multiple word feature information to determine multiple entity types and their corresponding at least one word feature information;

[0062] Based on the entity type corresponding to each word feature information in multiple word feature information, entity type labels are added to multiple words respectively, resulting in a sequence of entity type labels corresponding to the text information.

[0063] Here, word feature information can be, for example, word vectors. Clustering methods can include, for example, k-means clustering.

[0064] For example, after obtaining some materials science literature as training samples, word segmentation and word2vec word embedding processing can be performed on the text information in the materials science literature to extract word feature information of multiple words contained in the text information, resulting in multiple word vectors. K-means clustering is then performed on these multiple word vectors to obtain multiple clusters. Each cluster can correspond to an entity type; that is, words corresponding to word vectors in the same cluster can belong to the same entity type. This yields multiple entity types corresponding to multiple clusters, as well as the word feature information contained under each entity type. Based on this, entity type labels can be added to the words corresponding to each word feature information according to the entity type to which each word feature information belongs. This results in an entity type label sequence composed of entity type labels of multiple words, that is, the entity type label sequence corresponding to the text information containing these multiple words.

[0065] In this way, by using clustering to determine the entity type of the words included in the text information, and then adding corresponding entity type labels to each word, the automatic identification of entity types and the automatic addition of labels can be achieved. This can solve the problems of inaccuracy and low efficiency caused by manual identification and manual labeling, and effectively improve the accuracy and processing efficiency of entity type identification and labeling.

[0066] Based on this, in order to further improve the accuracy of entity type identification and label addition processes, in some implementations, the above-mentioned clustering of multiple word feature information to determine multiple entity types and their corresponding at least one word feature information may specifically include:

[0067] Clustering of multiple word feature information yields various clustering results, with different clustering results containing different numbers of clusters.

[0068] Determine the silhouette coefficients corresponding to the various clustering results;

[0069] Multiple entity types are determined based on the clustering results with the highest silhouette coefficients, along with at least one word feature information corresponding to each entity type.

[0070] For example, multiple word feature information can be clustered to obtain multiple clusters. Based on these multiple clusters, entity types can be further divided. Specifically, the division can be performed multiple times according to different numbers of entity types. Clusters belonging to the same entity type are merged, resulting in multiple clustering results. The number of clusters in different clustering results can vary. The silhouette coefficient of each clustering result is calculated to evaluate the clustering effect. A higher silhouette coefficient indicates a better classification effect. The silhouette coefficients for different numbers of entity types are observed. The clustering result with the highest silhouette coefficient can be taken as the final clustering result. Based on this clustering result, a corresponding entity type is assigned to each cluster, and corresponding entity type labels are added to the words corresponding to the word feature information in each cluster.

[0071] Furthermore, when setting corresponding entity types for each cluster, the first preset number of words in the cluster can be observed and identified to summarize type characteristics and define entity types and their corresponding ranges. Additionally, it is necessary to note whether a certain type of entity is common in materials science literature but not reflected in the cluster. If this entity type is important, it can be considered for addition as one of the entity types set in this application embodiment.

[0072] After determining the entity types and their corresponding word ranges within the textual information of materials science literature, labels can be added to the words in this text for use in training the entity recognition model. Furthermore, it is crucial to continuously optimize the quality of the training samples; that is, to dynamically update the types of entity types and the labeling of words in the textual information.

[0073] In this way, by evaluating multiple clustering results, the final result can be determined based on the clustering result with the highest silhouette coefficient, along with at least one word feature information corresponding to each entity type. This makes the final entity type more accurate, thereby further improving the accuracy of entity type identification and labeling processes.

[0074] In addition, in some implementations, the entity recognition model may also include a semantic recognition model. Based on this, the above-mentioned S130 may specifically include:

[0075] Word vectors are input into a semantic recognition model, which includes a model built on a bidirectional long short-term memory network.

[0076] The semantic recognition model is used to perform semantic recognition on text information, and the semantic recognition results corresponding to multiple words are output.

[0077] Based on the semantic recognition results, the entity type recognition results corresponding to multiple words are determined.

[0078] Here, the semantic recognition model can be any model capable of semantic recognition. Specifically, it can include models built on bidirectional long short-term memory networks, such as BiLSTM (Bidirectional Long Short-Term Memory) layers. BiLSTM is a bidirectional LSTM (Long Short-Term Memory) layer that can process the input word vector sequence in both forward and backward directions, concatenate the processing results for each word, and combine the concatenated vectors into a matrix as the output.

[0079] In addition, the semantic recognition model may also include a fully connected layer. For example, this fully connected layer can be used to further process the output of the BiLSTM layer, thereby obtaining the semantic recognition result for each word. This semantic recognition result may include the probability value of the word being classified as a label for each entity type.

[0080] Based on this, the semantic recognition result of a word can be used as the entity type recognition result of that word, or the semantic recognition result of a word can be adjusted according to certain constraints, and the adjusted result can be used as the entity type recognition result of that word.

[0081] Thus, by using a semantic recognition model built on a bidirectional long short-term memory network to perform semantic recognition on text information, the accuracy of semantic recognition results can be improved.

[0082] Furthermore, to improve the accuracy of entity type recognition results, in some implementations, the entity recognition model may also include an entity type prediction model. Based on this, the aforementioned entity type recognition results, which determine the entity type corresponding to multiple words based on semantic recognition results, may specifically include:

[0083] The semantic recognition results are input into the entity type prediction model, which includes a model based on conditional random fields.

[0084] The entity type prediction model is used to predict the entity type in the semantic recognition results, and the entity type recognition results corresponding to multiple words are output.

[0085] Here, the entity type prediction model can be a model that can predict entity types based on constraints. Its main function is to constrain the entity type recognition result corresponding to each word in order to reduce the occurrence of unreasonable entity type recognition results. For example, unreasonable situations may include situations where the entity type recognition result corresponding to a word is the O label, and the entity type recognition result of the word following that word is the I-"entity type" label.

[0086] Based on this, the entity type prediction model can specifically include a model built on Conditional Random Fields (CRF), such as a CRF layer. CRF solves a path optimization problem based on dynamic programming, which requires two matrices: an emission matrix and a transition matrix. The emission matrix is ​​the input to the CRF; for example, it can be the matrix composed of the semantic recognition results output from the fully connected layer. The transition matrix is ​​a parameter matrix within the CRF that needs to be updated and optimized. During CRF training, the corresponding loss value can be backpropagated to continuously update the transition matrix until convergence. The trained transition matrix contains the entity type recognition rules learned by the CRF; for example, in the entity type recognition results, there is a high probability that an "I" "entity type" label will not appear after an "O" label.

[0087] For example, the semantic recognition results corresponding to multiple words can be input into the CRF layer. The CRF layer adjusts the semantic recognition results so that the semantic recognition results corresponding to multiple words conform to the currently learned entity type arrangement rules, and then outputs the adjusted entity type recognition results, which form an entity type label sequence. This entity type label sequence is the entity type label sequence predicted by the model.

[0088] In this way, by using an entity type prediction model based on conditional random fields to predict the entity type in the semantic recognition results, the probability of unreasonable recognition results in the entity type recognition results can be reduced.

[0089] Based on this, in some specific examples, the structure of the entity recognition model can be as follows: Figure 3 As shown, the model includes a MatBERT model 31, a BiLSTM layer 32, a fully connected layer 33, and a CRF layer 34. For example, text information containing n words is input into the MatBERT model 31, which performs word segmentation and word embedding to obtain n word vectors. These n word vectors are then input into the BiLSTM layer 32, which performs forward and backward processing on them, combining the contextual information of the words in the text for semantic recognition. The results are concatenated into a matrix and input into the fully connected layer 33, which performs classification and outputs the semantic recognition result for each word. The semantic recognition result may include the probability value of the word under each entity type label. The semantic recognition results for each word are then input into the CRF layer 34, which adjusts the semantic recognition results and finally outputs the entity type label corresponding to each word, resulting in a predicted entity type label sequence.

[0090] Based on this, in order to improve the model training effect, the language representation model and entity type prediction model included in the entity recognition model can be trained separately. In some implementations, the above-mentioned S140 may specifically include:

[0091] Determine the standard semantic recognition result based on the entity type label sequence;

[0092] The first loss value is determined based on the semantic recognition result and the standard semantic recognition result;

[0093] The parameters of the language representation model are adjusted based on the first loss value to obtain the trained language representation model.

[0094] The second loss value is determined based on the entity type recognition result and the entity type label sequence;

[0095] The parameters of the entity type prediction model are adjusted based on the second loss value to obtain the trained entity type prediction model.

[0096] Here, since the semantic recognition result can include the probability that a word is classified into each entity type label, the pre-annotated entity type label sequence of the text information can be transformed into the same form of standard semantic recognition result. That is, the standard semantic recognition result also includes the probability that a word is classified into each entity type label, wherein the probability of a word under its corresponding annotated entity type label is higher than the probability under other unannotated entity type labels.

[0097] For example, for a language representation model, the corresponding loss value can be calculated using the cross-entropy formula.

[0098] For example, the first loss value Loss1 can be calculated according to the following formula (1).

[0099]

[0100] Among them, t ij y represents the predicted probability value of the i-th word under the j-th entity type label. ij This represents the standard probability value, or true value, corresponding to the i-th word under the j-th entity type label. n represents the number of words in the text information, and k represents the number of pre-set entity type labels.

[0101] Using this first loss value, the parameters of the language representation model can be adjusted until the model converges, thus obtaining the trained language representation model.

[0102] Furthermore, for the entire entity recognition model, the language representation model has the characteristic of pre-training, so its initial first loss value is very low. The training process of the entity recognition model is mainly the process of minimizing the second loss value corresponding to the entity type prediction model. The second loss value corresponding to the entity type prediction model can be calculated using the forward loss calculation formula built into the entity type prediction model.

[0103] For example, for an entity type prediction model, the second loss value Loss2 can be calculated using the following formula (2).

[0104]

[0105] in, The path score is the actual path score corresponding to the entity type label sequence. Let m be the path scores for the m paths, where m = k n Here, one path corresponds to one type of tag sequence. For example, taking two entity types, x and y, as an example, if there is a sentence containing 4 words, then the tag sequence corresponding to that sentence has 5. 4 There are 5 possibilities. 4 There are several paths. Furthermore, the path score for each path can be determined based on the parameters in the transition matrix and the emission matrix involved in that path.

[0106] Using this second loss value, the parameters of the entity type prediction model can be adjusted until the model converges, thus obtaining the trained entity type prediction model.

[0107] In this way, by adjusting the parameters of the language representation model and the entity type prediction model respectively, personalized training of different models can be achieved, thereby improving the model training effect.

[0108] Therefore, to further improve the model training effect, different learning rates can be used to train different models. In some implementations, the step of adjusting the parameters of the language representation model based on the first loss value to obtain the trained language representation model may specifically include:

[0109] Based on the first loss value, the parameters of the language representation model are adjusted according to the first learning rate to obtain the trained language representation model.

[0110] In some implementations, the step of adjusting the parameters of the entity type prediction model based on the second loss value to obtain the trained entity type prediction model may specifically include:

[0111] Based on the second loss value, the parameters of the entity type prediction model are adjusted according to the second learning rate to obtain the trained entity type prediction model. The first learning rate is less than the second learning rate.

[0112] For example, the language representation model, as a pre-trained model, already has its initial parameters updated to a relatively optimal level. Therefore, only fine-tuning is needed to complete the downstream task. In other words, the initial first loss value of the language representation model is already very low, and an optimizer that reduces the loss value can converge the first loss value of the language representation model to the optimal level with a small learning rate and a small number of sample traversals. However, if the parameters of the entity type prediction model are trained with this learning rate and number of sample traversals, when the language representation model converges to the optimal level, the second loss value of the entity type prediction model is still very large, indicating that the built-in transition matrix of the entity type prediction model has not been fully learned. Conversely, if a larger learning rate is set, the entity type prediction model can optimize the second loss value to the optimal level, but the language representation model, based on this larger learning rate, may fail to converge its first loss value. Therefore, in this embodiment, different learning rates are set for the language representation model and the entity type prediction model to train them separately and optimize their loss values ​​accordingly.

[0113] In this way, by setting different learning rates for different models, that is, using a smaller first learning rate to adjust the parameters of the language representation model built based on the pre-trained model, and using a larger second learning rate to adjust the parameters of the entity type prediction model, the entity type prediction model can be fully trained while the language representation model converges.

[0114] Furthermore, to improve the robustness and generalization ability of the model, this application embodiment also introduces adversarial training, such as FGM (Fast Gradient Method) adversarial training. Based on this, in some embodiments, after the step of adjusting the parameters of the entity type prediction model based on the second loss value to obtain the trained entity type prediction model, the model training method provided in this application embodiment may further include:

[0115] Add the target perturbation value to the word vectors corresponding to multiple words to obtain perturbed word vectors;

[0116] Based on the perturbed word vectors, entity type identification is performed on multiple words separately to obtain perturbed entity type identification results;

[0117] Based on the perturbed entity type identification results and entity type label sequence, a third loss value is determined;

[0118] The second backpropagation gradient is calculated based on the third loss value;

[0119] The parameters of the entity type prediction model are updated based on the second backpropagation gradient.

[0120] Here, the target perturbation value can be a randomly set perturbation value or a perturbation value determined based on the second loss value. The basic principle of adversarial training is to add additional perturbation values ​​to the input features as an attack method to increase the model's loss value. The model is then optimized based on the increased loss value to improve its robustness and generalization ability.

[0121] For example, a target perturbation value r can be added to the embedding matrix consisting of word vectors corresponding to multiple words output by the language representation model. Then, based on the word vectors after adding the target perturbation value r, entity type identification is performed again for each of the multiple words. The entity type identification process can refer to the aforementioned related implementation methods, which will not be repeated here. After obtaining the perturbed entity type identification result, the forward loss value of the entity type prediction model, i.e., the third loss value, can be determined again by combining the entity type label sequence and using the above formula (2). The backpropagation gradient, i.e., the second backpropagation gradient, is calculated using the third loss value. The values ​​in the embedding matrix are restored to their values ​​before adding the target perturbation value r, and then the parameters of the entity type prediction model are updated using the second backpropagation gradient.

[0122] In this way, by adopting adversarial training, adding target perturbation values ​​to word vectors, changing the loss value of the entity type recognition result output by the entity type prediction model, and then updating the parameters of the entity type prediction model based on the changed loss value, the robustness and generalization ability of the entity type prediction model can be improved.

[0123] Based on this, in order to improve the adversarial training effect, in some embodiments, before the step of adding target perturbation values ​​to the word vectors corresponding to multiple words to obtain perturbated word vectors, the model training method provided in this application embodiment may further include:

[0124] The first backpropagation gradient is calculated based on the second loss value;

[0125] The target perturbation value is determined based on the first backpropagation gradient.

[0126] Here, the first backpropagation gradient can be the backpropagation gradient corresponding to the forward loss value of the entity type prediction model during normal model training.

[0127] For example, the forward loss value of the prediction model based on the entity type, i.e. the second loss value, can be used to calculate the gradient of backpropagation to the embedding matrix, i.e. the first backpropagation gradient, and the target perturbation value can be calculated based on the first backpropagation gradient.

[0128] In this way, the target perturbation value can be dynamically determined by using the first backpropagation gradient calculated using the second loss value, thereby improving the effectiveness of adversarial training.

[0129] Therefore, in order to further improve the adversarial training effect, in some implementations, the step of determining the target perturbation value based on the first backpropagation gradient may specifically include:

[0130] Calculate the Euclidean norm based on the first backpropagation gradient;

[0131] The ratio of the first backpropagation gradient to the Euclidean norm is determined as the target perturbation value.

[0132] Here, the Euclidean norm is also known as the L2 norm.

[0133] For example, the L2 norm can be calculated based on the first backpropagation gradient in the entity type prediction model that propagates back to the embedding matrix, and the target perturbation value can be obtained by dividing the first backpropagation gradient by the L2 norm.

[0134] In this way, by calculating the Euclidean norm and using it to determine the target perturbation value, the effectiveness of adversarial training can be further improved.

[0135] Furthermore, by training an entity recognition model for textual information in the materials field according to the model training methods provided in the above embodiments of this application, the entity recognition model can also be used. Based on this, in some embodiments, after S140 above, the following may be included:

[0136] Obtain the textual information of the literature in the target material field to be identified, and obtain the target textual information;

[0137] Input the target text information into the entity recognition model;

[0138] The entity recognition model is used to identify the entity type of the target text information and outputs a sequence of target entity type labels corresponding to the target text information.

[0139] Here, the target materials literature can be the materials literature that needs to be identified.

[0140] For example, after training the entity recognition model, the text information corresponding to any document in the material field that needs to be recognized can be input into the entity recognition model. Then, the various models contained in the entity recognition model can be used to identify the entity type of the target text information. Finally, a relatively accurate target entity type label sequence can be output, thereby helping users to quickly extract useful information from the document in the material field.

[0141] Thus, by using the entity recognition model trained by the model training method provided in this application embodiment to identify the entity type of text information in the materials science literature, a more accurate entity type label sequence can be output, thereby effectively identifying the entity type of text information in the materials science literature, which in turn helps to extract useful information from the text information.

[0142] It should be noted that the application scenarios described in the above embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0143] Based on the same inventive concept, this application also provides a model training device. (Specifically combined with...) Figure 4 Please provide a detailed explanation.

[0144] Figure 4 This is a schematic diagram of the structure of a model training device provided in one embodiment of this application.

[0145] like Figure 4 As shown, the model training device 400 may include:

[0146] The information acquisition module 401 is used to acquire text information of literature in the field of materials and its corresponding entity type tag sequence, wherein the entity type tag sequence includes entity type tags corresponding to multiple words in the text information;

[0147] The word segmentation and embedding module 402 is used to perform word segmentation and word embedding on the text information using a language representation model to obtain word vectors corresponding to multiple words respectively. The language representation model includes a pre-trained model for the material domain.

[0148] The type recognition module 403 is used to perform entity type recognition on the plurality of words based on the word vectors, and obtain entity type recognition results corresponding to the plurality of words respectively;

[0149] The model training module 404 is used to train the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for text information in the material domain, wherein the entity recognition model includes the language representation model.

[0150] The model training device 400 described above will be explained in detail below:

[0151] In some embodiments, the information acquisition module 401 includes:

[0152] The feature extraction submodule is used to acquire textual information from literature in the field of materials science and extract word feature information of multiple words from the textual information.

[0153] The clustering processing submodule is used to perform clustering processing on the multiple word feature information to determine multiple entity types and their corresponding at least one word feature information;

[0154] The tag adding submodule is used to add entity type tags to the multiple words according to the entity type corresponding to each word feature information in the multiple word feature information, so as to obtain the entity type tag sequence corresponding to the text information.

[0155] In some embodiments, the clustering processing submodule may specifically include:

[0156] The first processing unit is used to perform clustering processing on the multiple word feature information to obtain multiple clustering results, wherein the number of clusters contained in different clustering results is different;

[0157] The first determining unit is used to determine the silhouette coefficients corresponding to the various clustering results respectively;

[0158] The second determining unit is used to determine multiple entity types and at least one word feature information corresponding to each entity type based on the clustering result with the highest silhouette coefficient.

[0159] In some embodiments, the type identification module 403 may specifically include:

[0160] The first input submodule is used to input the word vectors into the semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network;

[0161] The first recognition submodule is used to perform semantic recognition on the text information using the semantic recognition model, and output the semantic recognition results corresponding to the multiple words respectively;

[0162] The first determining submodule is used to determine the entity type recognition result corresponding to the plurality of words based on the semantic recognition result.

[0163] In some embodiments, the entity recognition model further includes an entity type prediction model; the result determination submodule may specifically include:

[0164] The first input unit is used to input the semantic recognition result into the entity type prediction model, wherein the entity type prediction model includes a model built based on a conditional random field.

[0165] The first prediction unit is used to predict the entity type in the semantic recognition result using the entity type prediction model, and output the entity type recognition result corresponding to the multiple words respectively.

[0166] In some embodiments, the model training module 404 may specifically include:

[0167] The second determining submodule is used to determine the standard semantic recognition result based on the entity type label sequence;

[0168] The third determining submodule is used to determine a first loss value based on the semantic recognition result and the standard semantic recognition result;

[0169] The first adjustment submodule is used to adjust the parameters of the language representation model based on the first loss value to obtain a trained language representation model.

[0170] The fourth determining submodule is used to determine the second loss value based on the entity type identification result and the entity type label sequence;

[0171] The second adjustment submodule is used to adjust the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model.

[0172] In some embodiments, the first adjustment submodule is specifically used for:

[0173] Based on the first loss value, the parameters of the language representation model are adjusted according to the first learning rate to obtain the trained language representation model.

[0174] The second adjustment submodule is specifically used for:

[0175] Based on the second loss value, the parameters of the entity type prediction model are adjusted according to the second learning rate to obtain the trained entity type prediction model.

[0176] Wherein, the first learning rate is less than the second learning rate.

[0177] In some embodiments, the model training module 404 may further include:

[0178] The first addition submodule is used to add target perturbation values ​​to the word vectors corresponding to the multiple words after adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model, thereby obtaining perturbed word vectors.

[0179] The second recognition submodule is used to re-identify the entity types of the multiple words based on the perturbed word vectors, and obtain the perturbed entity type recognition results.

[0180] The fifth determining submodule is used to determine a third loss value based on the perturbed entity type identification result and the entity type label sequence;

[0181] The first calculation submodule is used to calculate the second backpropagation gradient based on the third loss value;

[0182] The parameter update submodule is used to update the parameters of the entity type prediction model based on the second backpropagation gradient.

[0183] In some embodiments, the model training module 404 may further include:

[0184] The second calculation submodule is used to add the target perturbation value to the word vectors corresponding to the multiple words respectively, and calculate the first backpropagation gradient based on the second loss value before obtaining the perturbated word vectors;

[0185] The sixth determining submodule is used to determine the target perturbation value based on the first backpropagation gradient.

[0186] In some embodiments, the sixth determining submodule may specifically include:

[0187] The first calculation unit is used to calculate the Euclidean norm based on the first backpropagation gradient.

[0188] The third determining unit is used to determine the ratio of the first backpropagation gradient to the Euclidean norm as the target perturbation value.

[0189] In some embodiments, the entity type includes at least one of the following: material type, application type, research method type, research numerical type, material sample description type, experimental condition type, material property type, and experimental result type.

[0190] In some embodiments, the model training device 400 may further include:

[0191] The text acquisition module is used to train the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain the trained entity recognition model for text information in the material field, and then acquire the text information of the target material field document to be identified, thereby obtaining the target text information.

[0192] A text input module is used to input the target text information into the entity recognition model;

[0193] The text recognition module is used to perform entity type recognition on the target text information using the entity recognition model, and output a target entity type label sequence corresponding to the target text information.

[0194] Therefore, by using textual information from materials science literature as training samples and employing a pre-trained language representation model specific to materials science, the textual information is segmented and embedded to obtain word vectors corresponding to multiple words. Then, based on the entity type identification results from these word vectors and the actual entity type label sequence of the textual information, the entity recognition model is trained, resulting in an entity recognition model specifically for materials science textual information. Because the model is trained using textual information from materials science literature, and the word embedding is performed using a pre-trained language representation model specific to materials science, the trained entity recognition model is more targeted towards materials science literature. This allows the trained entity recognition model to effectively identify entity types in the textual information of materials science literature, thereby facilitating the extraction of useful information from the textual information.

[0195] Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application.

[0196] Electronic device 500 may include processor 501 and memory 502 storing computer program instructions.

[0197] Specifically, the processor 501 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0198] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0199] In certain embodiments, the memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Thus, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0200] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any of the model training methods in the above embodiments.

[0201] In some examples, the electronic device 500 may also include a communication interface 503 and a bus 510. For example, Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 510 and complete communication with each other.

[0202] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0203] Bus 510 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not as a limitation, bus 510 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0204] For example, the electronic device 500 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0205] The electronic device 500 can execute the model training method in the embodiments of this application, thereby achieving the combination Figures 1 to 4 The model training method and apparatus described.

[0206] Furthermore, in conjunction with the model training methods in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the model training methods in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0207] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0208] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0209] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0210] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0211] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A model training method, characterized in that, include: Obtain textual information of literature in the field of materials and its corresponding entity type tag sequence, wherein the entity type tag sequence includes entity type tags corresponding to multiple words in the text information; The text information is segmented and embedded using a language representation model to obtain word vectors corresponding to multiple words. The language representation model includes a pre-trained model for the material domain. Based on the word vectors, entity type recognition is performed on the multiple words respectively to obtain entity type recognition results corresponding to the multiple words respectively; Based on the entity type recognition results and the entity type label sequence, the entity recognition model is trained to obtain a trained entity recognition model for text information in the material domain, wherein the entity recognition model includes the language representation model.

2. The method according to claim 1, characterized in that, The acquisition of textual information and corresponding entity type tag sequences from materials science literature includes: Obtain textual information from literature in the field of materials science, and extract word feature information of multiple words from the textual information; Clustering is performed on the multiple word feature information to determine multiple entity types and their corresponding at least one word feature information; Based on the entity type corresponding to each word feature information in the plurality of word feature information, entity type labels are added to the plurality of words respectively to obtain an entity type label sequence corresponding to the text information.

3. The method according to claim 2, characterized in that, The step of clustering the multiple word feature information to determine multiple entity types and their corresponding at least one word feature information includes: Clustering is performed on the multiple word feature information to obtain various clustering results, in which different clustering results contain different numbers of clusters; Determine the silhouette coefficients corresponding to the various clustering results; Based on the clustering result with the highest silhouette coefficient, multiple entity types are determined, along with at least one word feature information corresponding to each entity type.

4. The method according to claim 1, characterized in that, The entity recognition model further includes a semantic recognition model; the step of performing entity type recognition on the multiple words based on the word vectors to obtain entity type recognition results corresponding to the multiple words includes: The word vectors are input into the semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network; The semantic recognition model is used to perform semantic recognition on the text information, and the semantic recognition results corresponding to the multiple words are output. Based on the semantic recognition results, the entity type recognition results corresponding to the multiple words are determined respectively.

5. The method according to claim 4, characterized in that, The entity recognition model further includes an entity type prediction model; the step of determining the entity type recognition results corresponding to the plurality of words based on the semantic recognition results includes: The semantic recognition result is input into the entity type prediction model, wherein the entity type prediction model includes a model built based on a conditional random field; The entity type prediction model is used to predict the entity type in the semantic recognition result, and the entity type recognition result corresponding to the multiple words is output.

6. The method according to claim 5, characterized in that, The step of training the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for text information in the materials domain includes: Determine the standard semantic recognition result based on the entity type label sequence; Based on the semantic recognition result and the standard semantic recognition result, a first loss value is determined; The parameters of the language representation model are adjusted based on the first loss value to obtain a trained language representation model. Based on the entity type identification result and the entity type label sequence, a second loss value is determined; The parameters of the entity type prediction model are adjusted based on the second loss value to obtain a trained entity type prediction model.

7. The method according to claim 6, characterized in that, The step of adjusting the parameters of the language representation model based on the first loss value to obtain the trained language representation model includes: Based on the first loss value, the parameters of the language representation model are adjusted according to the first learning rate to obtain the trained language representation model. The step of adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model includes: Based on the second loss value, the parameters of the entity type prediction model are adjusted according to the second learning rate to obtain the trained entity type prediction model. Wherein, the first learning rate is less than the second learning rate.

8. The method according to claim 6, characterized in that, After adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model, the method further includes: Add the target perturbation value to the word vectors corresponding to the multiple words to obtain the perturbated word vectors; Based on the perturbed word vectors, entity type identification is performed on the multiple words again to obtain the perturbed entity type identification results; Based on the perturbed entity type identification result and the entity type label sequence, a third loss value is determined; Calculate the second backpropagation gradient based on the third loss value; The parameters of the entity type prediction model are updated based on the second backpropagation gradient.

9. The method according to claim 8, characterized in that, Before adding the target perturbation value to the word vectors corresponding to the multiple words to obtain the perturbated word vectors, the method further includes: The first backpropagation gradient is calculated based on the second loss value; The target perturbation value is determined based on the first backpropagation gradient.

10. The method according to claim 9, characterized in that, Determining the target perturbation value based on the first backpropagation gradient includes: Calculate the Euclidean norm based on the first backpropagation gradient; The ratio of the first backpropagation gradient to the Euclidean norm is determined as the target perturbation value.

11. The method according to any one of claims 1-10, characterized in that, The entity type includes at least one of the following: material type, application type, research method type, research numerical type, material sample description type, experimental condition type, material property type, and experimental result type.

12. The method according to any one of claims 1-10, characterized in that, After training the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for text information in the materials domain, the method further includes: Obtain the textual information of the literature in the target material field to be identified, and obtain the target textual information; The target text information is input into the entity recognition model; The entity recognition model is used to identify the entity type of the target text information, and the target entity type label sequence corresponding to the target text information is output.

13. A model training device, characterized in that, include: The information acquisition module is used to acquire text information of literature in the field of materials and its corresponding entity type tag sequence, wherein the entity type tag sequence includes entity type tags corresponding to multiple words in the text information; The word segmentation and embedding module is used to segment and embed the text information using a language representation model to obtain word vectors corresponding to multiple words. The language representation model includes a pre-trained model for the material domain. The type recognition module is used to perform entity type recognition on the multiple words based on the word vectors, and obtain entity type recognition results corresponding to the multiple words respectively; The model training module is used to train the entity recognition model based on the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for text information in the material domain, wherein the entity recognition model includes the language representation model.

14. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the steps of the model training method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the steps of the model training method as described in any one of claims 1 to 12.

Images