Traditional Chinese medicine ancient book text data anaphora resolution method, device and medium

Abstract

The invention discloses a traditional Chinese medicine ancient book text data anaphora resolution method, device and medium, and the method comprises the following steps: constructing a traditional Chinese medicine ancient book corpus based on the anaphora relationship of traditional Chinese medicine ancient books pre-marked by a marking rule; pre-training a BERT model by taking the traditional Chinese medicine ancient book corpus as a training set; performing reference identification on the traditional Chinese medicine ancient book text data through the pre-trained BERT model; performing context analysis and anaphora relation identification on the traditional Chinese medicine ancient book text data through a BiLSTM model; and decoding the anaphora relationship through a CRF model to obtain a final anaphora resolution result. The method can efficiently and accurately analyze the anaphora relationship in the traditional Chinese medicine ancient books.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and medium for deconstructing text data of ancient Chinese medical books. Background Art

[0002] In natural language processing, coreference resolution is the process of identifying the referents of pronouns and noun phrases in a text. There are a large number of coreference phenomena in ancient Chinese medical texts, and these coreference relationships are crucial for understanding the content of the text and extracting effective information. Traditional coreference resolution methods have limitations when processing ancient Chinese texts, especially when processing ancient Chinese medical texts with complex structures and rich cultural connotations.

[0003] However, when dealing with ancient Chinese texts such as ancient Chinese medical texts with complex language structures and rich cultural connotations, traditional reference resolution techniques show obvious limitations. The large number of metaphorical and culturally specific reference phenomena commonly seen in ancient Chinese medical texts make these reference relationships an obstacle to the understanding of text content and information extraction. References in ancient Chinese medical texts not only involve specific things such as names of people, names of medicines, and symptoms, but also often include contextual associations across multiple paragraphs or different chapters. Traditional reference resolution techniques are difficult to cope with these challenges.

[0004] With the advancement of the digitization process of ancient Chinese medical books, how to efficiently and accurately parse the reference relationship has become an important issue in realizing the intelligent processing of ancient Chinese medical texts. Therefore, there is an urgent need for a reference resolution technology that can combine the unique language structure of ancient Chinese medical books to enhance its application value and promotion prospects in natural language processing. Summary of the invention

[0005] Based on the above deficiencies in the prior art, the present invention provides a method, electronic device, and medium for resolving reference relationships in ancient Chinese medical text data, which can efficiently and accurately parse reference relationships in ancient Chinese medical texts.

[0006] In order to solve the above technical problems, the present invention discloses a method for decomposing ancient Chinese medical text data, the method comprising:

[0007] Based on the reference relationship of the ancient Chinese medical books pre-annotated by the annotation rules, a corpus of ancient Chinese medical books is constructed, and the BERT model is pre-trained with the corpus of ancient Chinese medical books as a training set; the reference recognition of the ancient Chinese medical books text data is performed by the pre-trained BERT model;

[0008] Use the BiLSTM model to perform context analysis and referential relationship recognition on ancient Chinese medical text data;

[0009] The reference relationship is decoded through the CRF model to obtain the final reference resolution result.

[0010] In some embodiments, it further comprises:

[0011] According to the complexity of the reference resolution of the ancient Chinese medical text data, the BERT model, BiLSTM model and CRF model are adjusted and optimized.

[0012] In some embodiments, according to the complexity of the reference resolution of the ancient Chinese medical text data, the BERT model, the BiLSTM model and the CRF model are adjusted and optimized, including:

[0013] Adjust the BERT model according to the complexity of the reference resolution of the ancient Chinese medical text data, the size of the data set, and the model performance;

[0014] The hyperparameters of the BiLSTM model and CRF model were gradually adjusted by the control variable method;

[0015] The optimal parameter combination of the BiLSTM model and the CRF model is determined through cross-validation and hyperparameter search.

[0016] In some embodiments, the hyperparameters of the BiLSTM model and the CRF model are gradually adjusted by a control variable method, including:

[0017] Determine initial hyperparameters, and train the BiLSTM model and the CRF model using the initial hyperparameters; the hyperparameters include the number of hidden layer units, the Dropout rate, and the learning rate;

[0018] Each of the hyperparameters is gradually adjusted by the control variable method, and the verification performance of the BiLSTM model and the CRF model after the hyperparameters are adjusted is verified;

[0019] Adjust the hyperparameters based on validation performance;

[0020] The optimal parameter combination of the BiLSTM model and the CRF model is determined through cross-validation and hyperparameter search, including:

[0021] Divide the data set of ancient Chinese medical text data into multiple subsets, and determine the cross-validation training set and validation set;

[0022] Each of the hyperparameters is set with a set of candidate values, and all hyperparameter combinations are traversed to perform grid search;

[0023] Perform cross-validation for each hyperparameter combination and record the validation performance of each cross-validation;

[0024] The best hyperparameter combination is determined based on the validation performance.

[0025] In some implementations, pre-marking the reference relationship of ancient Chinese medical book data based on the marking rules includes:

[0026] Determine the referent, the pronoun, and the referential relationship, establish entity labels and referential relationship labels, and annotate the referential relationship according to the annotation guidelines;

[0027] Conducting consistency evaluation on the marked reference relationship, and when the consistency evaluation fails to meet the consistency evaluation criteria, re-marking the reference relationship, and updating the reference type and / or marking rules of the reference relationship in the marking guide;

[0028] Until the consistency evaluation meets the consistency evaluation criteria, the labeling of the reference relationship is completed.

[0029] In some embodiments, the BERT model performs reference recognition on ancient Chinese medical text data, including:

[0030] Perform feature extraction on the text sequence through the BERT model to generate a first word vector; the text sequence is composed of ancient Chinese medical text data;

[0031] The first word vector is labeled with reference to the pronoun and the referent.

[0032] In some embodiments, the BiLSTM model is used to perform context analysis and referential relationship recognition on ancient Chinese medical text data, including:

[0033] Taking the first word vector as input, analyzing the context information in the text sequence from forward and backward through the BiLSTM model, outputting a second word vector containing the context relationship, and identifying the potential reference relationship between the reference and the pronoun;

[0034] The reference relationship of the reference is marked according to the marking rule and the potential reference relationship.

[0035] In some implementations, the reference relationship is decoded by a CRF model to obtain a final reference resolution result, including:

[0036] The second word vector is used as input, and decoding is performed based on probability maximization through the CRF model to output a reference resolution result; the reference resolution result includes the reference relationship between the reference and the pronoun.

[0037] In a second aspect, a computer device is disclosed, characterized in that it includes: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of a method for decomposing ancient Chinese medical text data as described in any one of the above items.

[0038] In a third aspect, a computer storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, a method for resolving references to text data of ancient Chinese medical books as described in any one of the above items is implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention provides a method, device and medium for reference resolution of ancient Chinese medical book text data. By integrating BERT, BiLSTM and CRF models, combined with targeted pre-training of ancient Chinese medical book corpus and phased recognition and decoding steps, a large amount of ancient Chinese medical book text data can be effectively processed, reference relationships can be quickly identified and resolved, the accuracy of reference resolution can be improved, the adaptability of the model to ancient Chinese medical book texts and the automatic information extraction capability can be enhanced, and the efficiency and accuracy of ancient Chinese medical book text data processing can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of a method for resolving references to text data in ancient Chinese medical books provided by the present invention;

[0042] Figure 2 A schematic diagram showing the definition and example of labels in a method for decomposing ancient Chinese medical text data provided by the present invention;

[0043] Figure 3 The present invention provides a schematic diagram of an interface for marking entity labels and reference labels in a method for reference resolution of ancient Chinese medical text data provided by the present invention. DETAILED DESCRIPTION

[0044] For better understanding and implementation, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products or apparatus.

[0046] The embodiment of the present invention discloses a method for resolving references in text data of ancient Chinese medical books, which can efficiently and accurately analyze reference relationships in ancient Chinese medical books.

[0047] like Figure 1 As shown, the method includes:

[0048] Step S1: construct a corpus of ancient Chinese medical books based on the reference relationships of ancient Chinese medical books pre-annotated according to annotation rules.

[0049] To perform coreference resolution on traditional Chinese medical ancient book text data more efficiently, a corpus of traditional Chinese medical ancient books is constructed. First, collect recognized professional traditional Chinese medical ancient book resources, such as "Huangdi Neijing", "Shanghan Lun", "Compendium of Materia Medica", etc., to ensure the integrity and applicability of the text content in the corpus of traditional Chinese medical ancient books. Secondly, perform digital processing on the above-mentioned traditional Chinese medical ancient book resources, including but not limited to sentence cleaning, sentence or paragraph segmentation, to improve the accuracy of annotation and ensure that the text content can be used for natural language processing.

[0050] Specifically, pre-annotate the coreference relationships of traditional Chinese medical ancient book data based on annotation rules, including:

[0051] Step S11: Determine the referent, the pronoun, and the coreference relationship, establish entity labels and coreference relationship labels, and annotate the coreference relationship according to the annotation guidelines;

[0052] Step S12: Conduct a consistency evaluation on the annotated coreference relationships. When the consistency evaluation does not meet the consistency evaluation criteria, re-annotate the coreference relationships and update the coreference type and / or annotation rules of this coreference relationship in the above-mentioned annotation guidelines; until the consistency evaluation meets the consistency evaluation criteria, complete the annotation of this coreference relationship.

[0053] Based on the requirements of coreference resolution, determine the referent, the pronoun, and the coreference relationship, and establish a label system including entity labels and coreference relationship labels. Based on the requirements of natural language processing, as Figure 2 shown in the definitions and examples of the labels provided by this application. There are differences in the referential expressions of people and things in traditional Chinese medical ancient book resources. Generally, the referents are divided into two types: people and things. Considering that in modern Chinese, the pronoun does not necessarily refer to a non-human antecedent. Therefore, focus on the pronouns of people. Since in traditional Chinese medical ancient book resources, the demonstrative pronouns used to refer to people can also be used as personal pronouns, such as "zhi" and "qi". Therefore, personal pronouns are defined as pronouns that refer to people, and thing pronouns are defined as pronouns that refer to things.

[0054] At the same time, considering that there is a real connection between the interrogative pronoun and its referent content, define the interrogative pronoun and its answer as a special form of coreference and annotate it. The reason for such setting and definition is that in ancient traditional Chinese medical literature, the author already knew the answer to the interrogative sentence. The use of the interrogative sentence is mainly to introduce the content of the discussion and express an emphatic tone. Therefore, for each interrogative sentence, the answer is defined as what the interrogative pronoun refers to.

[0055] The annotation guidelines are formulated based on the above-mentioned referent, pronoun, and coreference relationship with reference to relevant corpora in combination with the characteristics of traditional Chinese medical ancient book resources, ensuring that annotators have the same understanding of the annotation rules. As Figure 3As shown, it is a schematic diagram of the interface for the annotator to refer to the annotation guide to annotate the entity labels and reference labels of ancient Chinese medical books resources.

[0056] A sample was randomly selected for preliminary annotation. The IAA value between annotators was calculated after each round of annotation. If the consistency was lower than the set threshold, the annotation guidelines were discussed and adjusted. In this application, Cohen's kappa was used to detect the consistency of two or more annotations. The calculation formula is as follows:

[0057]

[0058] Where P0 is the observed agreement between the two annotators, and P e is the probability of agreement between each annotator when they randomly select a category for each annotation. Annotations are considered sufficiently consistent when the κ value exceeds 0.85. The IAA value of the annotation can be determined by pairwise calculation based on the output of the three annotators and taking the average.

[0059] When the consistency evaluation does not meet the consistency evaluation criteria, the reference relationship is re-marked, and the reference type and / or marking rules of the reference relationship in the marking guide are updated; until the consistency evaluation meets the consistency evaluation criteria, the marking of the reference relationship is completed.

[0060] The annotation relationships obtained through the above steps introduce annotation guidelines and expert prior knowledge to build an accurate and highly consistent corpus of ancient Chinese medical books, providing basic data support for subsequent natural language processing and reference resolution tasks.

[0061] Step S2: pre-training the BERT model using the ancient Chinese medical book corpus as a training set; and performing reference recognition on the ancient Chinese medical book text data using the pre-trained BERT model.

[0062] In this application, a large-scale corpus of ancient Chinese medical books is used to pre-train the model to ensure the accuracy of entity recognition. Before the start of pre-training, the appropriate BERT model version is selected according to the task relevance, model size and characteristics of the training set to meet the needs of different computing resources. For example, a BERT model fine-tuned with Chinese medical text similar to the training task can be selected; if the required amount of calculation is large, the BERT-large model can be selected. Correspondingly, if the amount of training set data is small, a smaller model can be used or fine-tuned to prevent overfitting.

[0063] In some embodiments, pre-training includes Mask Language Modeling (MLM) and Next Sentence Prediction (NSP). The Mask Language Model randomly masks some words in the text, allowing the BERT model to predict these words in the context and enhance the understanding of the context; NSP helps the model learn the relationship between sentences, so as to adapt to more complex text relationship understanding tasks. The training set uses the corpus of ancient Chinese medicine books, so that the BERT model can capture specific semantic features, such as personal pronouns, thing pronouns, etc., and perform feature extraction on this basis, converting each word or phrase into a vector representation, that is, the first word vector.

[0064] After obtaining the first word vector, the model is able to identify important word types and structural relationships in the context by performing referential annotation on noun phrases and pronouns, thus building a foundation for semantic understanding for the model.

[0065] Referent labeling is usually based on large-scale data sets and is performed in an unsupervised or semi-supervised manner for pre-training tasks. In some embodiments, the labeled data can be used for fine-tuning so that the model learns to recognize referents, such as specific names of people or entities.

[0066] The BERT model is pre-trained using the ancient Chinese medical text corpus to enable it to understand and capture the semantic features of the ancient Chinese medical text. After the pre-training is completed, the BERT model is used to perform reference recognition on the ancient Chinese medical text data that needs to be resolved.

[0067] In some embodiments, the BERT model is used to perform reference identification on the ancient Chinese medical text data that needs to be reference resolved, including:

[0068] Perform feature extraction on the text sequence through the BERT model to generate a first word vector; the text sequence is composed of ancient Chinese medical text data;

[0069] The first word vector is labeled with reference to the pronoun and the referent.

[0070] The BRET model is:

[0071] Output = BERT(X)

[0072] Where X is the token representation of the input text sequence. For example, the text sequence Word embedding maps words to a d-dimensional real number vector space, that is, each word is converted into a vector with d numerical elements, where each element is a real number. By mapping, the semantic similarity, part of speech, or some grammatical information of vocabulary is captured, so that the model can better understand and generate semantically reasonable results when processing language tasks.

[0074] The BERT model receives the input text sequence, and after its multi-layer Transformer encoding, it outputs the first word vector, which contains the word structure information. Based on the first word vector generated by BERT, preliminary referential annotation is performed, that is, marking which words or phrases belong to the category of pronouns or referents. Exemplarily, the vector is annotated by adding labels, such as adding specific category labels to pronouns, noun phrases, etc., so that the first word vector carries referential information.

[0075] Step S3, using the BiLSTM model to perform context analysis and referential relationship recognition on the ancient Chinese medical text data;

[0076] The Bidirectional Long Short-Term Memory (BiLSTM) network is mainly used to process the contextual relationship in the text sequence. It is improved on the basis of LSTM and can capture the dependency relationship in the context more comprehensively by considering the forward and backward information at each time step.

[0077] The BiLSTM model is shown below:

[0078] h t =LSTM(h t-1 ,x t )

[0079] h t ′ =LSTM(h t ′ +1 ,x t )

[0080]

[0081] h t ′ : The LSTM hidden state at time t, which represents the state at that moment in the sequence processing.

[0082] h t ′ : The reverse LSTM hidden state at time t (processing the sequence from right to left).

[0083] The bidirectional LSTM representation combines the forward and backward hidden states to form a global contextual understanding of the word.

[0084] The BiLSTM model adds a reverse LSTM layer on the basis of the traditional LSTM model. The BiLSTM model consists of two LSTM models, one running along the forward direction of the text sequence, that is, from front to back, and the other running along the reverse direction of the text sequence, that is, from back to front. This bidirectional structure captures the front and back information of each time step, allowing the BiLSTM model to see a more comprehensive context when predicting. For each time step, BiLSTM outputs the results of the forward and backward LSTM layers, and the two are usually concatenated or summed to obtain the final output. This output can capture richer sequence relationships and is suitable for complex tasks in natural language processing. In this application, the use of the BiLSTM model can better understand the dependencies between words, especially based on the vectors provided by pre-trained models such as BERT, BiLSTM can supplement the temporal information of the sequence, thereby improving the understanding of the referent and the accuracy of annotation.

[0085] Specifically, step S3 includes:

[0086] Step S31: using the first word vector as input, analyzing the context information in the text sequence from forward and backward through the BiLSTM model, outputting a second word vector containing the context relationship, and identifying the potential reference relationship between the reference and the pronoun;

[0087] Step S32: label the reference relationship of the reference according to the labeling rule and the potential reference relationship.

[0088] The first word vector that has been labeled is used as input and processed by the first word vector of the BiLSTM model, while analyzing the contextual information in the text sequence from the forward and backward directions. In this process, BiLSTM will learn and output the second word vector containing the contextual relationship. These second word vectors not only represent the semantics of each word, but also contain the relationship between the word and its context. Using the second word vector, the potential reference relationship between the reference and the pronoun is identified. This process involves the analysis of the association between pronouns and noun phrases, with the aim of finding out which pronouns may refer to which specific nouns, and clarifying under what conditions the reference and pronoun can be regarded as a reference relationship.

[0089] According to the pre-defined annotation rules, the text is annotated to clearly indicate which pronouns refer to which noun phrases, forming the final reference relationship annotation results, providing a basis for subsequent reference resolution.

[0090] Step S4: Decode the reference relationship through the CRF model to obtain the final reference resolution result.

[0091] A conditional random field (CRF) model is constructed, and the CRF layer is superimposed on the output of the BiLSTM model. The CRF model can capture the dependencies between sequence labels and generate more coherent annotations by calculating the joint probability of adjacent labels. The CRF model determines the reference relationship between words based on the second word vector output by BiLSTM in order to decode the most likely label path. The label path includes the relationship markers between reference and pronoun, thereby marking the reference relationship of each word.

[0092] Step S4 specifically includes:

[0093] The second word vector is used as input, and decoding is performed based on probability maximization through the CRF model to output a reference resolution result; the reference resolution result includes the reference relationship between the reference and the pronoun.

[0094] During the decoding process, the CRF model learns the probability distribution in the sequence based on the second word vector generated by the BiLSTM model and the annotated reference and referent relationship labels. According to the context of each word and the possibility of the label, the optimal label sequence is decoded to more accurately annotate the relationship between pronouns and referents, and obtain the final referent resolution result. The referent resolution result includes the referent relationship between referents and pronouns, and determines which pronouns point to which referents.

[0095] Step S5: adjust and optimize the BERT model, BiLSTM model and CRF model according to the complexity of the reference resolution of the ancient Chinese medical text data.

[0096] In the process of performing the code resolution, the model parameters and optimization algorithm can be adjusted according to the complexity of the code resolution of the ancient Chinese medical text data to improve the accuracy and efficiency of the code resolution.

[0097] Specifically, step S5 includes:

[0098] Step S51, adjusting the BERT model according to the complexity of the reference resolution of the ancient Chinese medical text data, the size of the data set and the model performance;

[0099] Step S52: gradually adjust the hyperparameters of the BiLSTM model and the CRF model by using the control variable method;

[0100] Step S53: Determine the optimal parameter combination of the BiLSTM model and the CRF model through cross-validation and hyperparameter search.

[0101] The BERT model can be adjusted according to the complexity of the coreference resolution task, the size of the dataset, and the performance of the model, such as adjusting the version and structure of the BERT model. First, during the pre-training process, select a BERT pre-trained model that is similar to the target task domain. For example, if the task involves TCM text analysis, a BERT model fine-tuned on TCM domain data can be used to better understand domain-specific terminology and language structures. The BERT model has different sizes, such as BERT-base and BERT-large. BERT-large has more parameters and is usually better in performance, but it also has a higher computational overhead and is suitable for use under high-performance computing resources and sufficient time conditions. If resources are limited, choose a smaller model such as BERT-base to maintain a balance between performance and efficiency. Moreover, if the dataset is small, choosing a smaller model can reduce the risk of overfitting. At the same time, it is also possible to fine-tune on a smaller model so that the model adapts to the characteristics of specific data without adjusting too many parameters.

[0102] In this application, the BERT model is used to perform reference resolution of the text data of ancient Chinese medical books. It is usually necessary to fine-tune the layers of the BERT model to obtain richer vector representations in complex contextual understanding. On the one hand, the number of layers that need to be frozen can be determined according to the size of the data set of the text data that needs to be referenced and resolved. If the data set is small, freezing some layers can effectively reduce the risk of overfitting. If the data set is large, all layers can be fine-tuned so that the BERT model can fully learn the specific features of the data set and obtain better performance.

[0103] On the other hand, observe the model performance during the initial training. If the model is obviously overfitted, that is, the training set performs well but the validation set performs poorly, you can try freezing some layers to reduce the model's excessive dependence on the training data. If overfitting does not occur, you can gradually unfreeze more layers until you find the best frozen layer setting to balance model performance and training efficiency.

[0104] The hyperparameters of the BiLSTM model and CRF model are gradually adjusted by the control variable method, including:

[0105] Determine initial hyperparameters, and train the BiLSTM model and the CRF model using the initial hyperparameters; the hyperparameters include the number of hidden layer units, the Dropout rate, and the learning rate;

[0106] Each of the hyperparameters is gradually adjusted by the control variable method, and the verification performance of the BiLSTM model and the CRF model after the hyperparameters are adjusted is verified;

[0107] The hyperparameters are tuned based on the validation performance.

[0108] Generally, the initial hyperparameters are set to 64 hidden layer units, 0.2 dropout rate, and 0.001 learning rate. The BiLSTM model and CRF model are trained with the training set and the initial hyperparameters. The validation set is set to evaluate the validation performance of the currently recorded BiLSTM model and CRF model. The validation performance includes loss, accuracy, and F1 score. The loss and accuracy on the training set and validation set are observed in real time. If the training set performs better than the validation set, for example, the training loss is lower and the validation loss is higher, overfitting may occur; if both losses are high, underfitting may occur.

[0109] Each of the hyperparameters is gradually adjusted by the control variable method, and the verification performance of the BiLSTM model and the CRF model after the hyperparameters are adjusted is verified. While keeping other hyperparameters unchanged, the number of hidden layer units is gradually increased or decreased, for example, from 64 to 128 or decreased to 32; when overfitting occurs, the Dropout rate is increased, such as from 0.2 to 0.5, or when underfitting occurs, the Dropout rate is reduced; if the model converges too slowly or falls into a local minimum, the learning rate can be reduced, for example, from 0.001 to 0.0001. By keeping other hyperparameters unchanged and only adjusting a single hyperparameter, the verification performance of the model is verified to ensure that the model can achieve the best performance on both the training and verification sets, so that the model can learn enough features on the training set and obtain good generalization ability on the verification set, which can solve specific problems encountered by the model in different training stages and improve the stability and accuracy of the model.

[0110] The optimal parameter combination of the BiLSTM model and the CRF model is determined through cross-validation and hyperparameter search, including:

[0111] Divide the data set of ancient Chinese medical text data into multiple subsets, and determine the cross-validation training set and validation set;

[0112] Each of the hyperparameters is set with a set of candidate values, and all hyperparameter combinations are traversed to perform grid search;

[0113] Perform cross-validation for each hyperparameter combination and record the validation performance of each cross-validation;

[0114] The best hyperparameter combination is determined based on the validation performance.

[0115] Specifically, the dataset of ancient Chinese medical text data was divided into 6 subsets and 6-fold cross validation was performed. Each time, one of the subsets was used as the validation set, and the rest were used as the training set, and repeated 6 times to ensure the stability of the model's performance under different data distributions. A set of candidate values ​​was set for each hyperparameter, and all combinations were traversed. In the 6-fold cross validation, the validation performance of each set of hyperparameter combinations, such as the average F1 score and accuracy, was recorded, the average validation performance of each set of hyperparameter combinations was calculated, and the combination with the best performance in the cross validation was determined. The hyperparameter combination with the best validation effect was taken as the best hyperparameter combination.

[0116] Furthermore, the Adam optimization algorithm is combined to adjust and optimize the BERT model, BiLSTM model, and CRF model. Specifically, the Adam optimization algorithm includes:

[0117] m t =β1m t-1 +(1-β1)g t

[0118]

[0119] Among them, m t is the first-order momentum (moving average of the gradient) at time t, used to smooth the gradient update; v t is the second-order momentum at time t (moving average of the square of the gradient), which is used to adjust the update step size of the gradient; β1 and β2 are hyperparameters that control the decay rate of the first-order and second-order momentum; g t is the gradient at time t; α is the learning rate, which controls the step size of parameter update; θ t are model parameters; ∈ is a small value to prevent the denominator from being zero, which is used for numerical stability.

[0120] When performing context analysis on the first word vector generated by BERT on the ancient Chinese medical texts corpus, the BiLSTM model uses the Adam optimization algorithm to update hyperparameters, such as adjusting the learning rate based on the convergence speed and final performance, to better capture the relationship between the referent and the referent. In the CRF model that decodes the output of the BiLSTM, the Adam optimization algorithm can be used to optimize the CRF hyperparameters to accurately predict the referential relationship and ensure the coherence and accuracy of the final referential resolution results.

[0121] In some embodiments, the difference between the model prediction result and the true label is also measured by a cross entropy loss function. The cross entropy loss function is:

[0122]

[0123] Where y is the true label, is the model prediction label and N is the number of samples.

[0124] In the BiLSTM model, the cross entropy loss is used to calculate the error between the output reference relationship prediction and the true label, so as to better capture the correct reference relationship and gradually optimize the recognition of reference and pronouns. In the CRF model, the cross entropy loss is used to measure the difference between the predicted reference label sequence and the true label sequence. By minimizing the cross entropy loss, the parameters can be adjusted so that the decoded sequence is more consistent with the true reference relationship label.

[0125] The present invention provides a method for reference resolution of ancient Chinese medical book text data. By integrating BERT, BiLSTM and CRF models, combined with targeted pre-training of an ancient Chinese medical book corpus and phased recognition and decoding steps, a large amount of ancient Chinese medical book text data can be effectively processed, reference relationships can be quickly identified and resolved, the accuracy of reference resolution can be improved, the adaptability of the model to ancient Chinese medical book texts and the ability of automatic information extraction can be enhanced, and the efficiency and accuracy of ancient Chinese medical book text data processing can be effectively improved.

[0126] Based on the same inventive concept, the present invention also provides a computer device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above-mentioned method for decomposing ancient Chinese medical text data.

[0127] The processing method of the computer device can refer to the description of the above method and will not be repeated here.

[0128] An embodiment of the present application also provides a non-transitory machine-readable storage medium, on which an executable program is stored. When the executable program is executed by a microprocessor, the processor executes the method provided in the above embodiment.

[0129] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the described method.

[0130] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the described method.

[0131] The embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without paying creative labor.

[0132] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage or any other computer-readable medium that can be used to carry or store data.

[0133] Finally, it should be noted that the embodiments disclosed in the present invention are only preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for deconstructing ancient Chinese medical text data, characterized in that: The method comprises: Construct a corpus of ancient Chinese medical books based on the reference relationship of ancient Chinese medical books pre-annotated by annotation rules; Using the ancient Chinese medical books corpus as a training set, pre-training the BERT model; performing reference recognition on the ancient Chinese medical books text data through the pre-trained BERT model; Use the BiLSTM model to perform context analysis and referential relationship recognition on ancient Chinese medical text data; The reference relationship is decoded through the CRF model to obtain the final reference resolution result.

2. A method for decomposing ancient Chinese medical text data according to claim 1, characterized in that: Also includes: According to the complexity of the reference resolution of the ancient Chinese medical text data, the BERT model, BiLSTM model and CRF model are adjusted and optimized.

3. A method for deconstructing ancient Chinese medical text data according to claim 2, characterized in that: According to the complexity of the reference resolution of the ancient Chinese medical text data, the BERT model, BiLSTM model and CRF model are adjusted and optimized, including: Adjust the structure of the BERT model according to the complexity of the reference resolution of the ancient Chinese medical text data, the data set, and the model performance; The hyperparameters of the BiLSTM model and CRF model were gradually adjusted by the control variable method; The optimal parameter combination of the BiLSTM model and the CRF model is determined through cross-validation and hyperparameter search.

4. A method for resolving references to ancient Chinese medical text data according to claim 3, characterized in that: The hyperparameters of the BiLSTM model and CRF model are gradually adjusted by the control variable method, including: Determine initial hyperparameters, and train the BiLSTM model and the CRF model using the initial hyperparameters; the hyperparameters include the number of hidden layer units, the Dropout rate, and the learning rate; Each of the hyperparameters is gradually adjusted by the control variable method, and the verification performance of the BiLSTM model and the CRF model after the hyperparameters are adjusted is verified; Adjust the hyperparameters based on validation performance; The optimal parameter combination of the BiLSTM model and the CRF model is determined through cross-validation and hyperparameter search, including: Divide the data set of ancient Chinese medical text data into multiple subsets, and determine the cross-validation training set and validation set; Each of the hyperparameters is set with a set of candidate values, and all hyperparameter combinations are traversed to perform grid search; Perform cross-validation for each hyperparameter combination and record the validation performance of each cross-validation; The best hyperparameter combination is determined based on the validation performance.

5. A method for resolving references to ancient Chinese medical text data according to claim 2, characterized in that: Pre-annotate the reference relationship of ancient Chinese medical books data based on annotation rules, including: Determine the referent, the pronoun, and the referential relationship, establish entity labels and referential relationship labels, and annotate the referential relationship according to the annotation guidelines; Conducting consistency evaluation on the marked reference relationship, and when the consistency evaluation fails to meet the consistency evaluation criteria, re-marking the reference relationship, and updating the reference type and / or marking rules of the reference relationship in the marking guide; Until the consistency evaluation meets the consistency evaluation criteria, the labeling of the reference relationship is completed.

6. A method for resolving references to ancient Chinese medical text data according to claim 3, characterized in that: The BERT model is used to identify the reference of ancient Chinese medical text data, including: Perform feature extraction on the text sequence through the BERT model to generate a first word vector; the text sequence is composed of ancient Chinese medical text data; The first word vector is labeled with reference to the pronoun and the referent.

7. A method for resolving references to ancient Chinese medical text data according to claim 6, characterized in that: The BiLSTM model is used to perform context analysis and referential relationship recognition on ancient Chinese medical text data, including: Taking the first word vector as input, analyzing the context information in the text sequence from forward and backward through the BiLSTM model, outputting a second word vector containing the context relationship, and identifying the potential reference relationship between the reference and the pronoun; The reference relationship of the reference is marked according to the marking rule and the potential reference relationship.

8. A method for resolving references to ancient Chinese medical text data according to claim 7, characterized in that: The reference relationship is decoded through the CRF model to obtain the final reference resolution result, including: The second word vector is used as input, and decoding is performed based on probability maximization through the CRF model to output a reference resolution result; the reference resolution result includes the reference relationship between the reference and the pronoun.

9. A computer device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of a method for decomposing ancient Chinese medical text data as claimed in any one of claims 1 to 8.

10. A computer storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of a method for decomposing ancient Chinese medical text data references as claimed in any one of claims 1 to 8 are implemented.

Images