Model training method and device, equipment and storage medium

Abstract

The invention discloses a model training method and device, equipment and a storage medium. The method comprises the steps that text information of documents in the material field and an entity type tag sequence corresponding to the text information are acquired; the text information is subjected to word segmentation and word embedding processing through a language representation model, word vectors corresponding to the multiple words are obtained, and the language representation model comprises a pre-training model for the material field; performing entity type identification on the plurality of words based on the word vectors to obtain entity type identification results corresponding to the plurality of words; according to the entity type recognition result and the entity type label sequence, an entity recognition model is trained, the trained entity recognition model for the material field text information is obtained, and the entity recognition model comprises a language representation model. According to the scheme provided by the embodiment of the invention, the entity type of the text information in the literature in the material field can be effectively identified, so that useful information can be extracted from the text information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, device, equipment and storage medium. Background Art

[0002] With the continuous research and development of new materials, there are more and more references in the field of materials. How to quickly extract useful information from massive references is one of the keys to improving the work efficiency of R&D personnel.

[0003] At present, the existing natural language recognition models trained for general text information are not suitable for entity type recognition of text information in materials field literature. This may easily lead to the trained model being unable to effectively recognize the entity type of text information in materials field literature, and thus unable to extract useful information. Summary of the invention

[0004] The present application provides a model training method, apparatus, device and storage medium, which can enable the trained model to effectively identify the entity types of text information in materials field documents, thereby helping to extract useful information from the text information.

[0005] In a first aspect, the present application provides a model training method, which includes: obtaining text information of literature in the field of materials and its corresponding entity type label sequence, wherein the entity type label sequence includes entity type labels corresponding to multiple words in the text information; using a language representation model to perform word segmentation and word embedding processing on the text information to obtain word vectors corresponding to multiple words, wherein the language representation model includes a pre-trained model for the field of materials; based on the word vectors, performing entity type recognition on the multiple words to obtain entity type recognition results corresponding to the multiple words; training an entity recognition model based on the entity type recognition results and the entity type label sequence to obtain a trained entity recognition model for text information in the field of materials, wherein the entity recognition model includes the language representation model.

[0006] In this way, by using the text information in the literature in the field of materials as training samples, and using the pre-trained language representation model for the field of materials, the text information is segmented and word embedded to obtain word vectors corresponding to multiple words, and then the entity recognition model is trained according to the entity type recognition results recognized by the word vector and the real entity type label sequence of the text information to obtain an entity recognition model for the text information in the field of materials. In this way, since the text information in the literature in the field of materials is used for model training, and the pre-trained language representation model for the field of materials is used for word embedding, the trained entity recognition model is more targeted to the literature in the field of materials, so that the trained entity recognition model can effectively identify the entity type of the text information in the literature in the field of materials, and thus help to extract useful information from the text information.

[0007] In some embodiments, the method of obtaining text information of documents in the field of materials and their corresponding entity type label sequence includes: obtaining text information of documents in the field of materials, and extracting word feature information of multiple words from the text information; performing clustering processing on the multiple word feature information to determine multiple entity types and at least one word feature information corresponding to them; and adding entity type labels to the multiple words respectively according to the entity type corresponding to each word feature information in the multiple word feature information to obtain an entity type label sequence corresponding to the text information.

[0008] In this way, by using a clustering method to determine the entity types of words included in the text information, and then adding corresponding entity type labels to each word, automatic identification of entity types and automatic adding of labels can be achieved, thereby solving the problems of inaccuracy and low efficiency caused by manual identification and manual adding of labels, and effectively improving the accuracy and processing efficiency of the entity type identification and label adding process.

[0009] In some embodiments, clustering the multiple word feature information to determine multiple entity types and at least one word feature information corresponding to them includes: clustering the multiple word feature information to obtain multiple clustering results, wherein different clustering results contain different numbers of clusters; determining silhouette coefficients corresponding to the multiple clustering results; and determining multiple entity types and at least one word feature information corresponding to each entity type based on the clustering result with the highest silhouette coefficient.

[0010] In this way, by evaluating multiple clustering results, multiple entity types determined according to the clustering result with the highest silhouette coefficient and at least one word feature information corresponding to each entity type are taken as the final result, the final determined entity type can be made more accurate, thereby further improving the accuracy of the entity type recognition and label adding process.

[0011] In some embodiments, the entity recognition model also includes a semantic recognition model; the entity type recognition is performed on the multiple words based on the word vector to obtain entity type recognition results corresponding to the multiple words, including: inputting the word vector into the semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network; using the semantic recognition model to perform semantic recognition on the text information, and outputting semantic recognition results corresponding to the multiple words; and determining entity type recognition results corresponding to the multiple words based on the semantic recognition results.

[0012] In this way, by using the semantic recognition model constructed based on the bidirectional long short-term memory network to perform semantic recognition on text information, the accuracy of the semantic recognition results can be improved.

[0013] In some embodiments, the entity recognition model also includes an entity type prediction model; determining the entity type recognition results corresponding to the multiple words respectively based on the semantic recognition results includes: inputting the semantic recognition results into the entity type prediction model, wherein the entity type prediction model includes a model constructed based on conditional random fields; using the entity type prediction model to predict the entity type in the semantic recognition result, and outputting the entity type recognition results corresponding to the multiple words respectively.

[0014] In this way, by using the entity type prediction model constructed based on the conditional random field to predict the entity type in the semantic recognition results, the probability of unreasonable recognition results in the entity type recognition results can be reduced.

[0015] In some embodiments, the entity recognition model is trained according to the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for material field text information, including: determining a standard semantic recognition result according to the entity type label sequence; determining a first loss value according to the semantic recognition result and the standard semantic recognition result; adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model; determining a second loss value according to the entity type recognition result and the entity type label sequence; adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model.

[0016] In this way, by adjusting the parameters of the language representation model and the entity type prediction model respectively, personalized training of different models can be achieved, thereby improving the model training effect.

[0017] In some embodiments, the adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model includes: based on the first loss value, adjusting the parameters of the language representation model according to a first learning rate to obtain a trained language representation model; the adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model includes: based on the second loss value, adjusting the parameters of the entity type prediction model according to a second learning rate to obtain a trained entity type prediction model; wherein the first learning rate is less than the second learning rate.

[0018] In this way, by setting different learning rates for different models, that is, using a smaller first learning rate to adjust the parameters of the language representation model built based on the pre-trained model, and using a larger second learning rate to adjust the parameters of the entity type prediction model, the entity type prediction model can be fully trained while the language representation model converges.

[0019] In some embodiments, after adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model, the method further includes: adding a target perturbation value to the word vectors corresponding to the multiple words respectively to obtain a perturbed word vector; re-performing entity type recognition on the multiple words respectively based on the perturbed word vector to obtain a perturbed entity type recognition result; determining a third loss value based on the perturbed entity type recognition result and the entity type label sequence; calculating a second back propagation gradient based on the third loss value; and updating the parameters of the entity type prediction model based on the second back propagation gradient.

[0020] In this way, by adopting the adversarial training method, adding the target perturbation value to the parameters of the language representation model, changing the loss value of the entity type recognition result output by the entity type prediction model, and then updating the parameters of the entity type prediction model based on the changed loss value, the robustness and generalization ability of the entity type prediction model can be improved.

[0021] In some embodiments, before adding the target disturbance value to the word vectors corresponding to the multiple words respectively to obtain the disturbed word vectors, the method also includes: calculating a first back propagation gradient based on the second loss value; and determining the target disturbance value according to the first back propagation gradient.

[0022] In this way, the target perturbation value is dynamically determined by using the first back-propagation gradient calculated by the second loss value, thereby improving the adversarial training effect.

[0023] In some embodiments, determining the target disturbance value according to the first back-propagation gradient includes: calculating the Euclidean norm according to the first back-propagation gradient; and determining the ratio of the first back-propagation gradient to the Euclidean norm as the target disturbance value.

[0024] In this way, by calculating the Euclidean norm and using the Euclidean norm to determine the target perturbation value, the adversarial training effect can be further improved.

[0025] In some embodiments, the entity type includes at least one of a material type, an application type, a research method type, a research value type, a material sample description type, an experimental condition type, a material property type, and an experimental result type.

[0026] In some embodiments, after the entity recognition model is trained according to the entity type recognition result and the entity type label sequence to obtain the trained entity recognition model for material field text information, the method further includes: acquiring text information of the target material field document to be identified to obtain target text information; inputting the target text information into the entity recognition model; using the entity recognition model to perform entity type recognition on the target text information, and outputting a target entity type label sequence corresponding to the target text information.

[0027] In this way, by using the entity recognition model trained by the model training method provided in the embodiment of the present application to perform entity type recognition on the text information of the materials field literature, a more accurate entity type label sequence can be output, thereby effectively identifying the entity type of the text information in the materials field literature, and further helping to extract useful information from the text information. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 It is a flowchart of a model training method provided by an embodiment of the present application;

[0030] Figure 2 It is a schematic diagram of entity annotation provided by an embodiment of the present application;

[0031] Figure 3 It is a schematic diagram of the structure of an entity recognition model provided by an embodiment of the present application;

[0032] Figure 4It is a structural schematic diagram of a model training device provided by an embodiment of the present application;

[0033] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0036] At present, the commonly used natural language recognition models are trained by using ordinary text information as samples and general model structures and training methods. Since ordinary text information is basically daily language, and academic text information usually contains a large number of technical terms, which are obviously different from daily language, especially for text information in the field of materials, which contains a large number of technical terms unique to the field of materials, the commonly used natural language recognition models are not suitable for entity type recognition of text information in materials literature. This easily leads to the inability of the model trained by the existing method to effectively identify the entity type of text information in materials literature, and thus to extract useful information.

[0037] In order to solve the above technical problems, the embodiments of the present application provide a model training method, device, equipment and storage medium. Among them, the model training method provided by the embodiments of the present application, by using the text information in the material field document as a training sample, and using the pre-trained language representation model for the material field, the text information is segmented and word embedded to obtain word vectors corresponding to multiple words, and then the entity type recognition result identified by the word vector and the real entity type label sequence of the text information are trained to obtain an entity recognition model for the material field text information. In this way, since the text information in the material field document is used for model training, and the pre-trained language representation model for the material field is used for word embedding, the trained entity recognition model is more targeted to the material field document, so that the trained entity recognition model can effectively identify the entity type of the text information in the material field document, and then help to extract useful information from the text information.

[0038] The model training method provided in the embodiment of the present application is introduced in detail below. The model training method can be applied to the application scenario of entity type recognition of text information in the literature in the field of materials, and the model training method can be executed by an electronic device with computing power. Among them, the electronic device can include terminal devices such as mobile phones and computers, and can also include devices such as servers.

[0039] Figure 1 FIG. 1 is a flow chart of a model training method provided by an embodiment of the present application. Figure 1 As shown, the model training method may specifically include the following steps:

[0040] S110, obtaining text information of a document in the field of materials and a corresponding entity type label sequence, wherein the entity type label sequence includes entity type labels corresponding to a plurality of words in the text information;

[0041] S120, using a language representation model to perform word segmentation and word embedding processing on the text information to obtain word vectors corresponding to the plurality of words, wherein the language representation model includes a pre-trained model for the material field;

[0042] S130, performing entity type recognition on the multiple words based on the word vectors, and obtaining entity type recognition results corresponding to the multiple words respectively;

[0043] S140. Train an entity recognition model according to the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for material field text information, wherein the entity recognition model includes a language representation model.

[0044] Thus, by using the text information in the literature in the field of materials as training samples, and using the pre-trained language representation model for the field of materials, the text information is segmented and word embedded to obtain word vectors corresponding to multiple words, and then the entity recognition model is trained according to the entity type recognition results identified by the word vector and the real entity type label sequence of the text information, and the entity recognition model for the text information in the field of materials is obtained. In this way, since the text information in the literature in the field of materials is used for model training, and the pre-trained language representation model for the field of materials is used for word embedding, the trained entity recognition model is more targeted to the literature in the field of materials, so that the trained entity recognition model can effectively identify the entity type of the text information in the literature in the field of materials, and thus help to extract useful information from the text information.

[0045] The specific implementation methods of the above steps are introduced below.

[0046] In some embodiments, in S110, the material field document may include, for example, battery material field document. The text information may include, for example, text information such as words and symbols. The text information may include multiple words, and the multiple words may include at least one of a character and a word.

[0047] In the embodiment of the present application, an entity refers to a word or phrase with practical meaning in a text, and the specific task of entity recognition is to identify entities from a natural language text. An entity consists of one or more words, and the specific entity type required can be defined in advance according to actual needs.

[0048] For example, when constructing a model training sample for the field of materials, the text information of the literature in the field of materials can be obtained and entity annotation can be performed, that is, corresponding entity type labels are added to the multiple words contained in the text information to obtain an entity type label sequence corresponding to the text information, and the text information and the entity type label sequence are used as model training samples to train the entity recognition model. Among them, the entity type label sequence may include multiple entity type labels, one entity type label corresponds to one word in the text information, and the arrangement order of the multiple entity type labels can be the same as the arrangement order of the corresponding multiple words in the text information.

[0049] For entity annotations, such as Figure 2As shown, in this embodiment, the "BIO" format can be used for annotation: B-"entity type", I-"entity type", O. Among them, B stands for "Begin", which is the first word or word of an entity; I stands for "Inside", which means all parts of an entity except the first word or word; O stands for "Outside", which means a non-entity word or word. The entity of a single word or word is composed of only one B-"entity type" label, while the entity composed of multiple words or words contains two entity type labels, B-"entity type" and I-"entity type".

[0050] Based on this, in this embodiment, one entity type may correspond to two entity type labels. For example, five entity type labels, Bx, Ix, By, Iy, and O, may be set corresponding to two entity types x and y.

[0051] In the embodiment of the present application, multiple entity types can be pre-set, and entity type identification can be performed within the scope of the multiple entity types. In some embodiments, the entity type may include at least one of a material type, an application type, a research method type, a research value type, a material sample description type, an experimental condition type, a material property type, and an experimental result type. By setting the above types, key information corresponding to the above types can be extracted from the literature in the field of materials in a relatively comprehensive and accurate manner.

[0052] In some implementations, in S120, the entity recognition model for material field text information to be trained in the embodiments of the present application may specifically include a language representation model.

[0053] The language representation model can be a model that can extract word vectors from natural language information, and can specifically include a pre-trained model for the material field, such as the MatBERT model. Among them, MatBERT is a pre-trained model for the material field. Compared with the ordinary BERT (Bidirectional Encoder Representations from Transformers) model, the MatBERT model has been trained with a large number of literature texts in the material field and has a good word embedding representation capability in the material field. Therefore, the MatBERT model can be used here for word segmentation and word embedding processing.

[0054] Exemplarily, text information can be input into the MatBERT model, and the text information is segmented and embedded in MatBERT to obtain word vectors corresponding to multiple words in the text information.

[0055] In some implementations, in S130, after a plurality of word vectors are obtained, entity type recognition may be performed on a plurality of words included in the text information, respectively, based on the plurality of word vectors.

[0056] In the embodiment of the present application, the entity recognition model may include other models in addition to the language representation model, and then the other models in the entity recognition model may be used to perform entity type recognition on the multiple words to obtain entity type recognition results. The entity type recognition results may include the entity type labels corresponding to the words under the entity type to which they belong.

[0057] In some embodiments, in S140, after obtaining the entity type recognition result corresponding to each word in the text information, the entity recognition model can be trained according to the entity type recognition result of each word and its corresponding entity type label in a pre-set entity type label sequence, and then the trained entity recognition model can be obtained.

[0058] Exemplarily, the corresponding loss value can be calculated based on the entity type recognition result of each word and the entity type label corresponding to the word, and the loss value can be used to adjust the parameters of the model included in the entity recognition model until the model converges, thereby completing the training process of the entity recognition model.

[0059] Based on this, when adding entity type labels to multiple words in text information as training samples, in order to improve the accuracy and processing efficiency of entity type recognition and label adding process, in some embodiments, the above S110 may specifically include:

[0060] Obtain text information of literature in the material field, and extract word feature information of multiple words from the text information;

[0061] Performing clustering processing on the plurality of word feature information to determine a plurality of entity types and at least one word feature information corresponding thereto;

[0062] According to the entity type corresponding to each word feature information in the multiple word feature information, entity type labels are added to the multiple words respectively to obtain an entity type label sequence corresponding to the text information.

[0063] Here, the word feature information may be, for example, a word vector. The clustering processing method may include, for example, k-means clustering.

[0064] Exemplarily, after obtaining some documents in the field of materials as training samples, the text information in the documents in the field of materials can be segmented and word2vec word embedding processed to extract the word feature information of multiple words contained in the text information and obtain multiple word vectors. The multiple word vectors are clustered by k-means to obtain multiple clusters, and one cluster can correspond to an entity type, that is, the words corresponding to the word vectors in the same cluster can belong to the same entity type, and then multiple entity types corresponding to multiple clusters and the word feature information contained in each entity type can be obtained. Based on this, according to the entity type to which each word feature information belongs, an entity type label can be added to the word corresponding to each word feature information, so that an entity type label sequence composed of the entity type labels of multiple words can be obtained, that is, an entity type label sequence corresponding to the text information containing the multiple words.

[0065] In this way, by using a clustering method to determine the entity types of words included in the text information, and then adding corresponding entity type labels to each word, automatic identification of entity types and automatic adding of labels can be achieved, thereby solving the problems of inaccuracy and low efficiency caused by manual identification and manual adding of labels, and effectively improving the accuracy and processing efficiency of the entity type identification and label adding process.

[0066] Based on this, in order to further improve the accuracy of entity type identification and label adding process, in some embodiments, the above clustering process of multiple word feature information to determine multiple entity types and their corresponding at least one word feature information may specifically include:

[0067] Clustering is performed on multiple word feature information to obtain multiple clustering results, wherein different clustering results contain different numbers of clusters;

[0068] Determine the silhouette coefficients corresponding to various clustering results;

[0069] A plurality of entity types and at least one word feature information corresponding to each entity type are determined according to the clustering result with the highest silhouette coefficient.

[0070] Exemplarily, multiple word feature information can be clustered to obtain multiple clusters, and entity types can be divided again based on the multiple clusters. Specifically, multiple divisions can be performed according to different numbers of entity types, and clusters belonging to the same entity type can be merged to obtain multiple clustering results. The number of clusters contained in different clustering results can be different. The silhouette coefficient of each clustering result in the multiple clustering results is calculated to evaluate the clustering effect. Among them, the higher the silhouette coefficient, the better the classification effect. Observe the silhouette coefficient when the number of entity types is different, and then the clustering result with the highest silhouette coefficient can be used as the final clustering result, and set the corresponding entity type for each cluster based on the clustering result, and add the corresponding entity type label to the word corresponding to the word feature information in each cluster.

[0071] In addition, when setting the corresponding entity type for each cluster, the first preset number of words in the cluster can be observed and identified, the type characteristics can be summarized, and the entity type and its corresponding range can be defined. In addition, it is also necessary to pay attention to whether a certain type of entity is common in the literature in the material field but is not reflected in the cluster. If the entity type is more important, you can consider adding this entity type as one of the entity types set in the embodiment of this application.

[0072] After determining the entity type and the word range in the text information in the materials field literature, the words in the text information can be labeled for use in training the entity recognition model. In addition, it is also necessary to pay attention to the continuous optimization of the quality of training samples, that is, to continuously and dynamically update the types of entity types and the labeling of words in the text information.

[0073] In this way, by evaluating multiple clustering results, multiple entity types determined according to the clustering result with the highest silhouette coefficient and at least one word feature information corresponding to each entity type are taken as the final result, the final determined entity type can be made more accurate, thereby further improving the accuracy of the entity type recognition and label adding process.

[0074] In addition, in some implementations, the entity recognition model may also include a semantic recognition model. Based on this, the above S130 may specifically include:

[0075] Inputting the word vector into a semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network;

[0076] Using the semantic recognition model to perform semantic recognition on text information, and outputting semantic recognition results corresponding to multiple words respectively;

[0077] Entity type recognition results respectively corresponding to the multiple words are determined based on the semantic recognition results.

[0078] Here, the semantic recognition model may be a model capable of semantic recognition, and may specifically include a model built based on a bidirectional long short-term memory network. For example, the semantic recognition model may include a BiLSTM (Bidirectional LongShort-Term Memory) layer. Among them, BiLSTM is a bidirectional LSTM (Long Short-Term Memory) layer, which can perform forward and reverse processing on the input word vector sequence, and splice the processing results corresponding to each word, and the spliced ​​vectors are combined into a matrix as output.

[0079] In addition, the semantic recognition model may also include a fully connected layer. Exemplarily, the fully connected layer may be used to further process the output result of the BiLSTM layer, thereby obtaining a semantic recognition result corresponding to each word. The semantic recognition result may include a probability value of the word being classified as each entity type label.

[0080] Based on this, the semantic recognition result of a word can be used as the entity type recognition result of the word, or the semantic recognition result of the word can be adjusted according to certain restrictions, and the adjusted result can be used as the entity type recognition result of the word.

[0081] In this way, by using the semantic recognition model constructed based on the bidirectional long short-term memory network to perform semantic recognition on text information, the accuracy of the semantic recognition results can be improved.

[0082] In addition, in order to improve the accuracy of entity type recognition results, in some embodiments, the entity recognition model may also include an entity type prediction model. Based on this, the above-mentioned entity type recognition results corresponding to multiple words respectively determined based on the semantic recognition results may specifically include:

[0083] Inputting the semantic recognition result into an entity type prediction model, wherein the entity type prediction model includes a model constructed based on a conditional random field;

[0084] The entity type prediction model is used to predict the entity type in the semantic recognition result, and the entity type recognition results corresponding to multiple words are output.

[0085] Here, the entity type prediction model can be a model that can predict the entity type according to constraints. Its main function is to constrain the entity type recognition results corresponding to each word to reduce the occurrence of unreasonable entity type recognition results. Unreasonable situations may include, for example, the case where the entity type recognition result corresponding to a word is an O label, and the entity type recognition result of the word after the word is an I-"entity type" label.

[0086] Based on this, the entity type prediction model can specifically include a model built based on conditional random fields. For example, the entity type prediction model can include a CRF (Conditional Random Fields) layer. CRF solves a path optimization problem based on dynamic programming, and solving the path optimization problem requires relying on two matrices, one is the emission matrix and the other is the transfer matrix. Among them, the emission matrix is ​​the input of CRF. For example, the matrix composed of the semantic recognition results output by the fully connected layer can be used as the emission matrix. The transfer matrix is ​​a parameter matrix inside the CRF that needs to be updated and optimized. During the training process of CRF, the loss value corresponding to the CRF can be back-propagated, and the transfer matrix is ​​continuously updated until it converges. The trained transfer matrix contains the entity type recognition rules learned by CRF. For example, there is a high probability that the I-"entity type" label will not appear after the O label in the entity type recognition result.

[0087] Exemplarily, the semantic recognition results corresponding to multiple words can be input into the CRF layer, and the semantic recognition results can be adjusted through the CRF layer so that the semantic recognition results corresponding to the multiple words conform to the currently learned entity type arrangement rules, and then the adjusted entity type recognition results are output to form an entity type label sequence, which is the entity type label sequence predicted by the model.

[0088] In this way, by using the entity type prediction model constructed based on the conditional random field to predict the entity type in the semantic recognition results, the probability of unreasonable recognition results in the entity type recognition results can be reduced.

[0089] Based on this, in some specific examples, the structure of the entity recognition model can be as follows Figure 3 As shown, it includes a MatBERT model 31, a BiLSTM layer 32, a fully connected layer 33 and a CRF layer 34. Exemplarily, text information including n words is input into the MatBERT model 31, and the text information is segmented and embedded using the MatBERT model 31 to obtain n word vectors. The n word vectors are then input into the BiLSTM layer 32, and the BiLSTM layer 32 is used to perform forward and reverse processing on the n word vectors, and semantic recognition is performed in combination with the contextual information of the words in the text information, and the results are spliced ​​into a matrix and input into the fully connected layer 33, and the fully connected layer 33 is used for classification, and the semantic recognition result corresponding to each word is output, and the semantic recognition result may include the probability value corresponding to the word under each entity type label. The semantic recognition results corresponding to the multiple words are respectively input into the CRF layer 34, and the CRF layer is used to adjust the semantic recognition results, and finally the entity type label corresponding to each word is output to obtain a predicted entity type label sequence.

[0090] Based on this, in order to improve the model training effect, the language representation model and the entity type prediction model included in the entity recognition model may be trained separately. In some implementations, the above S140 may specifically include:

[0091] Determine a standard semantic recognition result according to the entity type label sequence;

[0092] Determining a first loss value according to the semantic recognition result and the standard semantic recognition result;

[0093] Adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model;

[0094] Determine a second loss value according to the entity type recognition result and the entity type label sequence;

[0095] The parameters of the entity type prediction model are adjusted based on the second loss value to obtain a trained entity type prediction model.

[0096] Here, since the semantic recognition result may include the probability of a word being classified as each entity type label, the entity type label sequence pre-annotated with the text information may be converted into a standard semantic recognition result of the same form, that is, the standard semantic recognition result also includes the probability of a word being classified as each entity type label, wherein the probability corresponding to a word under its corresponding annotated entity type label is higher than the probability corresponding to other unannotated entity type labels.

[0097] Exemplarily, for the language representation model, the cross entropy formula can be used to calculate the corresponding loss value.

[0098] For example, the first loss value Loss1 may be calculated according to the following formula (1).

[0099]

[0100] Among them, t ij is the probability value corresponding to the predicted i-th word under the j-th entity type label, that is, the predicted value. ij is the standard probability value corresponding to the i-th word under the j-th entity type label, that is, the true value. n represents the number of words contained in the text information, and k represents the number of pre-set entity type labels.

[0101] The first loss value can be used to adjust the parameters of the language representation model until the model converges, thereby obtaining a trained language representation model.

[0102] In addition, for the entire entity recognition model, the language representation model has the characteristics of pre-training, so its corresponding initial first loss value is very low, and the training process of the entity recognition model is also mainly the process of minimizing the second loss value corresponding to the entity type prediction model. The second loss value corresponding to the entity type prediction model can be calculated using the forward loss value calculation formula provided by the entity type prediction model.

[0103] Exemplarily, for the entity type prediction model, the second loss value Loss2 can be calculated using the following formula (2).

[0104]

[0105] in, is the path score of the true path corresponding to the entity type label sequence, is the path score corresponding to m paths, m = k n Here, a path corresponds to a label sequence. For example, taking the two entity types x and y as an example, if there is a sentence containing 4 words, then the label sequence corresponding to the sentence is 5. 4 There are 5 possibilities. 4 In addition, the path score of each path can be determined according to the parameters in the transfer matrix and the parameters in the emission matrix involved in the path.

[0106] The second loss value can be used to adjust the parameters of the entity type prediction model until the model converges, thereby obtaining a trained entity type prediction model.

[0107] In this way, by adjusting the parameters of the language representation model and the entity type prediction model respectively, personalized training of different models can be achieved, thereby improving the model training effect.

[0108] Based on this, in order to further improve the model training effect, different models can also be trained using different learning rates. In some embodiments, the step of adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model may specifically include:

[0109] Based on the first loss value, the parameters of the language representation model are adjusted according to the first learning rate to obtain a trained language representation model.

[0110] In some implementations, the step of adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model may specifically include:

[0111] Based on the second loss value, the parameters of the entity type prediction model are adjusted according to the second learning rate to obtain a trained entity type prediction model, wherein the first learning rate is less than the second learning rate.

[0112] Exemplarily, for the language representation model, as a pre-trained model, the initial parameters of the model itself have been updated to a relatively optimal level, so only fine-tuning the model is required to complete the downstream task. In other words, the initial first loss value of the language representation model is already very low, and the optimizer that reduces the loss value can make the first loss value of the language representation model converge to the best with a smaller learning rate and a smaller number of sample traversals, but if the parameters of the entity type prediction model are trained with this learning rate and sample traversals, when the language representation model converges to the best, the second loss value of the entity type prediction model is still large, and the built-in transfer matrix of the entity type prediction model has not been fully learned. If the learning rate is set to a larger value, the second loss value can be optimized to the best for the entity type prediction model, but the language representation model may not be able to converge its first loss value based on this larger learning rate. Therefore, the embodiment of the present application sets different learning rates for the language representation model and the entity type prediction model to train them separately to optimize the loss values ​​respectively.

[0113] In this way, by setting different learning rates for different models, that is, using a smaller first learning rate to adjust the parameters of the language representation model built based on the pre-trained model, and using a larger second learning rate to adjust the parameters of the entity type prediction model, the entity type prediction model can be fully trained while the language representation model converges.

[0114] In addition, in order to improve the robustness and generalization ability of the model, the embodiment of the present application also introduces adversarial training, such as FGM (Fast Gradient Method) adversarial training. Based on this, in some embodiments, after the above step of adjusting the parameters of the entity type prediction model based on the second loss value to obtain the trained entity type prediction model, the model training method provided in the embodiment of the present application may also include:

[0115] Adding target perturbation values ​​to word vectors corresponding to the multiple words respectively to obtain perturbed word vectors;

[0116] Re-performing entity type recognition on multiple words based on the perturbed word vectors to obtain perturbed entity type recognition results;

[0117] Determining a third loss value according to the perturbed entity type recognition result and the entity type label sequence;

[0118] Calculate a second back propagation gradient based on the third loss value;

[0119] The parameters of the entity type prediction model are updated according to the second back-propagation gradient.

[0120] Here, the target perturbation value can be a randomly set perturbation value or a perturbation value determined according to the second loss value. The basic principle of adversarial training is to add additional perturbation values ​​to the input features as an attack method to increase the loss value of the model. After that, the model is optimized according to the increased loss value to improve the robustness and generalization ability of the model.

[0121] Exemplarily, the target perturbation value r can be added to the embedding matrix composed of word vectors corresponding to multiple words output by the language representation model, and then the entity type recognition of the multiple words is performed again based on the word vectors after adding the target perturbation value r, wherein the entity type recognition process can refer to the aforementioned related implementation methods and will not be repeated here. After obtaining the perturbed entity type recognition result, the forward loss value of the entity type prediction model, that is, the third loss value, can be determined again by combining the entity type label sequence using the above formula (2). The back propagation gradient, that is, the second back propagation gradient, is calculated using the third loss value. The value in the embedding matrix is ​​restored to the value before adding the target perturbation value r, and then the parameters of the entity type prediction model are updated using the second back propagation gradient.

[0122] In this way, by adopting the adversarial training method, adding the target perturbation value to the word vector, changing the loss value of the entity type recognition result output by the entity type prediction model, and then updating the parameters of the entity type prediction model based on the changed loss value, the robustness and generalization ability of the entity type prediction model can be improved.

[0123] Based on this, in order to improve the adversarial training effect, in some embodiments, before the step of adding the target perturbation value to the word vectors corresponding to the multiple words to obtain the perturbed word vectors, the model training method provided in the embodiment of the present application may further include:

[0124] Calculate a first back-propagation gradient based on the second loss value;

[0125] The target perturbation value is determined based on the first back-propagation gradient.

[0126] Here, the first back-propagation gradient may be the back-propagation gradient corresponding to the forward loss value of the entity type prediction model when the model is trained normally.

[0127] Exemplarily, the gradient back-propagated to the embedding matrix, i.e., the first back-propagation gradient, can be calculated based on the forward loss value of the entity type prediction model, i.e., the second loss value, and the target perturbation value can be calculated based on the first back-propagation gradient.

[0128] In this way, the target perturbation value is dynamically determined by using the first back-propagation gradient calculated by the second loss value, thereby improving the adversarial training effect.

[0129] Based on this, in order to further improve the adversarial training effect, in some embodiments, the step of determining the target perturbation value according to the first back propagation gradient may specifically include:

[0130] Calculate the Euclidean norm based on the first back-propagated gradient;

[0131] The ratio of the first back-propagation gradient to the Euclidean norm is determined as the target perturbation value.

[0132] Here, the Euclidean norm is also the L2 norm.

[0133] Exemplarily, the L2 norm may be calculated based on the first back-propagation gradient back-propagated to the embedding matrix in the entity type prediction model, and the target disturbance value may be obtained by dividing the first back-propagation gradient by the L2 norm.

[0134] In this way, by calculating the Euclidean norm and using the Euclidean norm to determine the target perturbation value, the adversarial training effect can be further improved.

[0135] In addition, according to the model training method provided in the above embodiments of the present application, an entity recognition model for text information in the material field is trained, and the entity recognition model can also be used. Based on this, in some embodiments, after the above S140, the following may also be included:

[0136] Acquire text information of documents in the target material field to be identified, and obtain target text information;

[0137] Input the target text information into the entity recognition model;

[0138] The entity recognition model is used to identify the entity type of the target text information, and a target entity type label sequence corresponding to the target text information is output.

[0139] Here, the target material field document may be a material field document that needs to be identified.

[0140] Exemplarily, after the entity recognition model is trained, the text information corresponding to any material field document that needs to be identified can be input into the entity recognition model, and the various models contained in the entity recognition model can be used to perform entity type recognition on the target text information. Finally, a relatively accurate target entity type label sequence can be output, thereby assisting users to quickly extract useful information from the material field document.

[0141] In this way, by using the entity recognition model trained by the model training method provided in the embodiment of the present application to perform entity type recognition on the text information of the materials field literature, a more accurate entity type label sequence can be output, thereby effectively identifying the entity type of the text information in the materials field literature, and further helping to extract useful information from the text information.

[0142] It should be noted that the application scenarios described in the above-mentioned embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0143] Based on the same inventive concept, the present application also provides a model training device. Figure 4 Provide detailed explanation.

[0144] Figure 4 It is a structural diagram of a model training device provided in one embodiment of the present application.

[0145] like Figure 4 As shown, the model training device 400 may include:

[0146] An information acquisition module 401 is used to acquire text information of a document in the field of materials and a corresponding entity type label sequence, wherein the entity type label sequence includes entity type labels corresponding to a plurality of words in the text information;

[0147] A word segmentation and embedding module 402, configured to perform word segmentation and word embedding processing on the text information using a language representation model to obtain word vectors corresponding to a plurality of words, wherein the language representation model includes a pre-trained model for a material field;

[0148] A type identification module 403 is used to perform entity type identification on the multiple words based on the word vectors to obtain entity type identification results corresponding to the multiple words respectively;

[0149] The model training module 404 is used to train the entity recognition model according to the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for material field text information, wherein the entity recognition model includes the language representation model.

[0150] The model training device 400 is described in detail below, as shown below:

[0151] In some embodiments, the information acquisition module 401 includes:

[0152] A feature extraction submodule is used to obtain text information of documents in the field of materials and extract word feature information of multiple words from the text information;

[0153] A clustering processing submodule, used to perform clustering processing on the plurality of word feature information to determine a plurality of entity types and at least one word feature information corresponding thereto;

[0154] The label adding submodule is used to add entity type labels to the multiple words respectively according to the entity type corresponding to each word feature information in the multiple word feature information, so as to obtain an entity type label sequence corresponding to the text information.

[0155] In some embodiments, the cluster processing submodule may specifically include:

[0156] A first processing unit is used to perform clustering processing on the plurality of word feature information to obtain a plurality of clustering results, wherein different clustering results contain different numbers of clusters;

[0157] A first determining unit, used to determine silhouette coefficients corresponding to the plurality of clustering results respectively;

[0158] The second determining unit is used to determine a plurality of entity types and at least one word feature information corresponding to each entity type according to the clustering result with the highest silhouette coefficient.

[0159] In some embodiments, the type identification module 403 may specifically include:

[0160] A first input submodule, used to input the word vector into the semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network;

[0161] A first recognition submodule, configured to perform semantic recognition on the text information using the semantic recognition model, and output semantic recognition results corresponding to the plurality of words respectively;

[0162] The first determination submodule is used to determine entity type recognition results corresponding to the multiple words respectively based on the semantic recognition results.

[0163] In some embodiments, the entity recognition model also includes an entity type prediction model; the result determination submodule may specifically include:

[0164] A first input unit, used to input the semantic recognition result into the entity type prediction model, wherein the entity type prediction model includes a model constructed based on a conditional random field;

[0165] The first prediction unit is used to predict the entity type in the semantic recognition result by using the entity type prediction model, and output entity type recognition results corresponding to the multiple words respectively.

[0166] In some embodiments, the model training module 404 may specifically include:

[0167] A second determination submodule is used to determine a standard semantic recognition result according to the entity type label sequence;

[0168] A third determination submodule, used to determine a first loss value according to the semantic recognition result and the standard semantic recognition result;

[0169] a first adjustment submodule, configured to adjust parameters of the language representation model based on the first loss value to obtain a trained language representation model;

[0170] A fourth determination submodule, configured to determine a second loss value according to the entity type recognition result and the entity type label sequence;

[0171] The second adjustment submodule is used to adjust the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model.

[0172] In some embodiments, the first adjustment submodule is specifically configured to:

[0173] Based on the first loss value, adjusting the parameters of the language representation model according to a first learning rate to obtain a trained language representation model;

[0174] The second adjustment submodule is specifically used for:

[0175] Based on the second loss value, adjusting the parameters of the entity type prediction model according to a second learning rate to obtain a trained entity type prediction model;

[0176] The first learning rate is smaller than the second learning rate.

[0177] In some embodiments, the model training module 404 may further include:

[0178] A first adding submodule is used for adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model, and then adding a target disturbance value to the word vectors corresponding to the plurality of words to obtain a disturbed word vector;

[0179] A second recognition submodule, configured to re-recognize entity types of the plurality of words based on the perturbed word vectors to obtain perturbed entity type recognition results;

[0180] a fifth determination submodule, configured to determine a third loss value according to the perturbed entity type recognition result and the entity type label sequence;

[0181] A first calculation submodule, used for calculating a second back propagation gradient based on the third loss value;

[0182] A parameter updating submodule is used to update the parameters of the entity type prediction model according to the second back-propagation gradient.

[0183] In some embodiments, the model training module 404 may further include:

[0184] A second calculation submodule is used to add a target disturbance value to the word vectors corresponding to the multiple words respectively, and before obtaining the disturbed word vectors, calculate a first back propagation gradient based on the second loss value;

[0185] A sixth determination submodule is used to determine the target disturbance value according to the first back-propagation gradient.

[0186] In some embodiments, the sixth determining submodule may specifically include:

[0187] A first computing unit, configured to compute a Euclidean norm according to the first back-propagation gradient;

[0188] The third determining unit is used to determine the ratio of the first back-propagation gradient to the Euclidean norm as the target disturbance value.

[0189] In some embodiments, the entity type includes at least one of a material type, an application type, a research method type, a research value type, a material sample description type, an experimental condition type, a material property type, and an experimental result type.

[0190] In some embodiments, the model training device 400 may further include:

[0191] A text acquisition module is used to train an entity recognition model according to the entity type recognition result and the entity type label sequence to obtain the trained entity recognition model for material field text information, and then obtain the text information of the target material field document to be recognized to obtain the target text information;

[0192] A text input module, used for inputting the target text information into the entity recognition model;

[0193] The text recognition module is used to use the entity recognition model to perform entity type recognition on the target text information, and output a target entity type label sequence corresponding to the target text information.

[0194] Thus, by using the text information in the literature in the field of materials as training samples, and using the pre-trained language representation model for the field of materials, the text information is segmented and word embedded to obtain word vectors corresponding to multiple words, and then the entity recognition model is trained according to the entity type recognition results identified by the word vector and the real entity type label sequence of the text information, and the entity recognition model for the text information in the field of materials is obtained. In this way, since the text information in the literature in the field of materials is used for model training, and the pre-trained language representation model for the field of materials is used for word embedding, the trained entity recognition model is more targeted to the literature in the field of materials, so that the trained entity recognition model can effectively identify the entity type of the text information in the literature in the field of materials, and thus help to extract useful information from the text information.

[0195] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application.

[0196] The electronic device 500 may include a processor 501 and a memory 502 storing computer program instructions.

[0197] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0198] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0199] In certain embodiments, the memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0200] The processor 501 implements any one of the model training methods in the above embodiments by reading and executing computer program instructions stored in the memory 502.

[0201] In some examples, the electronic device 500 may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0202] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0203] Bus 510 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus 510 may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0204] Exemplarily, the electronic device 500 may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA).

[0205] The electronic device 500 can execute the model training method in the embodiment of the present application, thereby realizing the combination Figures 1 to 4 Described model training method and device.

[0206] In addition, in combination with the model training method in the above embodiment, the embodiment of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the model training methods in the above embodiment is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0207] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0208] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0209] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0210] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0211] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A model training method, characterized in that: include: Acquire text information of a document in the field of materials and a corresponding entity type label sequence thereof, wherein the entity type label sequence includes entity type labels corresponding to a plurality of words in the text information respectively; Performing word segmentation and word embedding processing on the text information using a language representation model to obtain word vectors corresponding to a plurality of words, wherein the language representation model includes a pre-trained model for a material field; Based on the word vectors, entity type recognition is performed on the multiple words respectively to obtain entity type recognition results corresponding to the multiple words respectively; According to the entity type recognition result and the entity type label sequence, an entity recognition model is trained to obtain a trained entity recognition model for material field text information, wherein the entity recognition model includes the language representation model.

2. The method according to claim 1, characterized in that The step of acquiring text information of documents in the material field and their corresponding entity type label sequences includes: Acquire text information of documents in the field of materials, and extract word feature information of a plurality of words from the text information; Performing clustering processing on the plurality of word feature information to determine a plurality of entity types and at least one word feature information corresponding thereto; According to the entity type corresponding to each word feature information in the plurality of word feature information, entity type labels are respectively added to the plurality of words to obtain an entity type label sequence corresponding to the text information.

3. The method according to claim 2, characterized in that The clustering process of the plurality of word feature information to determine a plurality of entity types and at least one word feature information corresponding thereto includes: Performing clustering processing on the plurality of word feature information to obtain a plurality of clustering results, wherein different clustering results contain different numbers of clusters; Determine the silhouette coefficients corresponding to the plurality of clustering results respectively; A plurality of entity types and at least one word feature information corresponding to each entity type are determined according to the clustering result with the highest silhouette coefficient.

4. The method according to claim 1, characterized in that: The entity recognition model also includes a semantic recognition model; the entity type recognition is performed on the multiple words based on the word vector to obtain entity type recognition results corresponding to the multiple words, including: Inputting the word vector into the semantic recognition model, wherein the semantic recognition model includes a model constructed based on a bidirectional long short-term memory network; Performing semantic recognition on the text information using the semantic recognition model, and outputting semantic recognition results corresponding to the multiple words respectively; Entity type recognition results corresponding to the multiple words are determined based on the semantic recognition results.

5. The method according to claim 4, characterized in that The entity recognition model also includes an entity type prediction model; the determining of entity type recognition results corresponding to the multiple words based on the semantic recognition results includes: Inputting the semantic recognition result into the entity type prediction model, wherein the entity type prediction model includes a model constructed based on a conditional random field; The entity type prediction model is used to predict the entity type in the semantic recognition result, and the entity type recognition results corresponding to the multiple words are output.

6. The method according to claim 5, characterized in that The step of training the entity recognition model according to the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for text information in the material field includes: Determine a standard semantic recognition result according to the entity type label sequence; Determining a first loss value according to the semantic recognition result and the standard semantic recognition result; Adjusting parameters of the language representation model based on the first loss value to obtain a trained language representation model; Determining a second loss value according to the entity type recognition result and the entity type label sequence; The parameters of the entity type prediction model are adjusted based on the second loss value to obtain a trained entity type prediction model.

7. The method according to claim 6, characterized in that The adjusting the parameters of the language representation model based on the first loss value to obtain a trained language representation model includes: Based on the first loss value, adjusting the parameters of the language representation model according to a first learning rate to obtain a trained language representation model; The adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model includes: Based on the second loss value, adjusting the parameters of the entity type prediction model according to a second learning rate to obtain a trained entity type prediction model; The first learning rate is smaller than the second learning rate.

8. The method according to claim 6, characterized in that After adjusting the parameters of the entity type prediction model based on the second loss value to obtain a trained entity type prediction model, the method further includes: Adding target disturbance values ​​to the word vectors corresponding to the multiple words respectively to obtain disturbed word vectors; Re-performing entity type recognition on the multiple words based on the disturbed word vectors to obtain disturbed entity type recognition results; Determining a third loss value according to the disturbed entity type recognition result and the entity type label sequence; Calculate a second back propagation gradient based on the third loss value; The parameters of the entity type prediction model are updated according to the second back-propagation gradient.

9. The method according to claim 8, characterized in that Before adding target disturbance values ​​to the word vectors corresponding to the plurality of words to obtain the disturbed word vectors, the method further includes: Calculate a first back propagation gradient based on the second loss value; The target disturbance value is determined according to the first back-propagation gradient.

10. The method according to claim 9, characterized in that The determining the target disturbance value according to the first back propagation gradient includes: Calculate the Euclidean norm based on the first back-propagated gradient; The ratio of the first back-propagation gradient to the Euclidean norm is determined as the target disturbance value.

11. The method according to any one of claims 1 to 10, characterized in that: The entity type includes at least one of a material type, an application type, a research method type, a research value type, a material sample description type, an experimental condition type, a material property type, and an experimental result type.

12. The method according to any one of claims 1 to 11, characterized in that: After training the entity recognition model according to the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for material field text information, the method further includes: Acquire text information of documents in the target material field to be identified, and obtain target text information; Inputting the target text information into the entity recognition model; The entity recognition model is used to perform entity type recognition on the target text information, and a target entity type label sequence corresponding to the target text information is output.

13. A model training device, characterized in that: include: An information acquisition module, used to acquire text information of documents in the field of materials and a corresponding entity type label sequence, wherein the entity type label sequence includes entity type labels corresponding to a plurality of words in the text information; A word segmentation and embedding module, used to perform word segmentation and word embedding processing on the text information using a language representation model to obtain word vectors corresponding to a plurality of words, wherein the language representation model includes a pre-trained model for the material field; A type recognition module, used to perform entity type recognition on the multiple words based on the word vectors, and obtain entity type recognition results corresponding to the multiple words respectively; The model training module is used to train the entity recognition model according to the entity type recognition result and the entity type label sequence to obtain a trained entity recognition model for material field text information, wherein the entity recognition model includes the language representation model.

14. An electronic device, characterized in that: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the steps of the model training method as described in any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the model training method according to any one of claims 1 to 12.

Images