Method for translating traditional Chinese medicine multi-source data based on deep learning

Abstract

The invention relates to the technical field of natural language processing. The method for translating the multi-source data of the traditional Chinese medicine based on deep learning comprises the following steps: acquiring related data of the traditional Chinese medicine to obtain an initial data set of the traditional Chinese medicine; formatting the traditional Chinese medicine initial data set to obtain a formatted data set; performing term annotation and cultural background annotation on the formatted data set to obtain an annotated data set; performing data cleaning on the labeled data set to obtain a preprocessed data set; performing data enhancement on the preprocessed data set; constructing an initial translation model, and training the initial translation model to obtain a pre-trained translation model; verifying the pre-training translation model, finely adjusting the pre-training translation model according to a verification result, testing the finely-adjusted pre-training translation model, and evaluating a test result; and according to an evaluation result, optimizing the pre-trained translation model after fine tuning to obtain a translation optimization model. The problems that an existing machine translation tool is not accurate enough in translation result and poor in cultural adaptability are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for translating multi-source data of traditional Chinese medicine based on deep learning. Background Art

[0002] As a treasure of Chinese culture, Chinese medicine classics not only carry profound medical knowledge, but also contain rich cultural connotations. With the acceleration of globalization and the gradual increase in the influence of Chinese medicine in the international community, how to achieve accurate and culturally adaptable translation has become a key issue in promoting the global dissemination of Chinese medicine culture. However, traditional translation methods face many challenges when dealing with Chinese medicine classics. First, the complexity and uniqueness of Chinese medicine terminology make it difficult for many terms to find expressions corresponding to Western medical concepts, which often results in inaccurate translation or deviation from the original meaning. Secondly, the rich cultural background information contained in these classics is often ignored or cannot be fully conveyed in the traditional translation process, which weakens the integrity of the translated text and may lead to misunderstandings of Chinese medicine culture among foreign readers.

[0003] The translation of TCM classics faces great challenges both in terms of language and culture. The complexity and uniqueness of TCM terminology make its translation much more complicated than in other fields. Many TCM terms not only involve complex medical concepts, but also incorporate the philosophical thoughts and historical background of Chinese culture. These terms are often difficult to find direct corresponding expressions in the target language. For example, concepts such as "qi", "yin and yang", "five elements", and "dampness" in TCM do not have accurate equivalents in Western medicine, which brings great difficulties to translation.

[0004] In addition, TCM classics are usually recorded in classical Chinese rather than modern Chinese, which further increases the difficulty of translation. Translators must first have a deep understanding of classical Chinese before they can accurately translate it into English or other languages. This means that the translator's classical Chinese skills and comprehension ability directly affect the accuracy of the translation, which is one of the fundamental reasons why many TCM classics are inaccurately translated. At the same time, the language style of TCM classics is usually rhythmic and literary, and traditional translation methods find it difficult to accurately convey their content while retaining the charm of the original text. This complexity makes it difficult for existing translation methods to fully convey the cultural background and connotation when dealing with TCM texts.

[0005] Existing translation methods can be roughly divided into two categories: manual translation and machine translation. Although manual translation has high accuracy, it faces problems such as high time cost, low efficiency, and strong translator subjectivity. In addition, due to differences in translators' background knowledge and language skills, the quality of translation by different translators is also uneven. As an efficient solution, machine translation has made significant progress in recent years, especially when dealing with general texts. However, when machine translation is applied to complex professional fields such as traditional Chinese medicine classics, its limitations quickly become apparent. Existing machine translation tools often lack a deep understanding of traditional Chinese medicine terminology and classical Chinese, and are prone to terminology translation errors or misunderstandings of cultural background. These problems have seriously affected the quality of translation, especially in terms of cultural transmission, where existing machine translation methods are almost incapable of handling the task. Summary of the invention

[0006] The purpose of the present invention is to provide a method for translating multi-source data of traditional Chinese medicine based on deep learning, aiming to solve the problem that existing machine translation tools lack a deep understanding of traditional Chinese medicine terminology and classical Chinese, are prone to terminology translation errors or misunderstandings of cultural background, resulting in inaccurate translation results and poor cultural adaptability.

[0007] The present invention is achieved through the following technical solutions: A method for translating multi-source data of traditional Chinese medicine based on deep learning, comprising the steps of: Obtain TCM-related data through several data sources to obtain the initial TCM data set; Formatting the initial data set of traditional Chinese medicine to obtain a formatted data set; Perform terminology annotation and cultural background annotation on the formatted data set to obtain an annotated data set; Clean the labeled data set, establish a data set index according to the data source, type and semantic characteristics, store the cleaned labeled data set according to the data set index, and obtain the preprocessed data set; Perform data augmentation on the preprocessed data set, and dynamically divide the augmented preprocessed data set into training set, validation set, and test set; Based on deep learning, an initial translation model is constructed, and the initial translation model is trained using a training set to obtain a pre-trained translation model; Verify the pre-trained translation model through the validation set, fine-tune the pre-trained translation model based on the verification results, test the fine-tuned pre-trained translation model through the test set, and evaluate the test results; According to the evaluation results, the fine-tuned pre-trained translation model is optimized to obtain a translation optimization model.

[0008] Optionally, the specific process of obtaining TCM-related data through several data sources to obtain the TCM initial data set is: The texts of Chinese medicine classics, modern Chinese medicine literature and corresponding Chinese medicine translation texts are obtained from several data sources; among them, several data sources include Chinese medicine digital libraries, academic journals, collections of Chinese medicine academic papers, Chinese medicine research institutions and expert translations.

[0009] Optionally, the specific process of formatting the initial data set of traditional Chinese medicine to obtain the formatted data set is: Convert the text data of the initial dataset of traditional Chinese medicine into a standard format to obtain a standardized text dataset; Remove noise characters, extra spaces and garbled characters from the standardized text dataset, and correct punctuation marks to obtain a preliminary cleaned dataset; Perform word segmentation on the preliminary cleaned data set, segment it into sentences, extract the structural hierarchical information of the text, and align the source language and target language of the multilingual text to obtain the text sorting data set; Labels are added to the text sorting dataset, and the labeled text sorting dataset is saved in a standardized file format to obtain a formatted dataset.

[0010] Optionally, the specific process of performing terminology annotation and cultural background annotation on the formatted data set to obtain the annotated data set is: Based on the formatted data set, a TCM terminology database is constructed to record the definition, context examples and corresponding translation versions of each TCM term, and the contextual relevance of each TCM term is captured through a lexical co-occurrence matrix; Based on the formatted data set, we sorted out cultural background information covering historical allusions, philosophical thoughts and customs, built a cultural background database, and quantified the importance of each cultural element in the cultural background database through a weighted scoring mechanism; Based on the importance and corresponding attributes of each cultural element, each TCM term is annotated in layers, and each TCM term is aligned with the corresponding cultural element to construct a cultural background dictionary of TCM terms and obtain an annotated dataset.

[0011] Optionally, the specific process of performing data cleaning on the annotated data set, establishing a data set index according to the data source, type and semantic characteristics, and storing the cleaned annotated data set according to the data set index to obtain the preprocessed data set is as follows: A weighted deduplication mechanism is used to deduplicate the labeled data set, assigning corresponding weights to different types of duplicate content, and deduplicating based on the weights of the duplicate content to obtain a deduplicated data set; Correct grammatical and spelling errors in the deduplicated data set, and standardize the data format of the deduplicated data set to obtain a cleaned data set; Establish a data set index based on data source, type and semantic characteristics, and use the data set index to index and annotate the cleaned data set to obtain an indexed data set; The index dataset is stored in a structured file format, and the preprocessed dataset is obtained through segment numbering and version control mechanism.

[0012] Optionally, the expression of weighted deduplication is as shown in the following formula (5):

[0013] in, Indicates the comprehensive cleaning score for a single data item during data cleaning; Indicates The weight of the class error; Indicates Class error in data item frequency or severity of occurrence; is the total number of error types that need to be handled.

[0014] Optionally, the specific process of performing data enhancement on the preprocessed data set and dynamically dividing the enhanced preprocessed data set into a training set, a validation set, and a test set is: Replace TCM terms with their corresponding synonyms in the preprocessed dataset; randomly delete or insert non-key terms to simulate missing or redundant information in real contexts; use semantic transformation to generate diverse sentence structures; combine the term annotations and cultural background annotations of the preprocessed dataset to expand the terms and cultural background in the specified context; According to the data volume of the preprocessed data set and the requirements of the translation task, the ratio of the training set, validation set, and test set is set; a stratified sampling strategy is adopted to ensure that the divided training set, validation set, and test set are balanced in terms of TCM terminology and cultural background information; oversampling or undersampling technology is used to adjust the sample distribution; the divided training set, validation set, and test set are labeled and classified for storage to distinguish between original data and enhanced data, and a version control mechanism is established to record changes in each data division and enhancement.

[0015] Optionally, the specific process of constructing an initial translation model based on deep learning, training the initial translation model through a training set, and obtaining a pre-trained translation model is as follows: Select a deep learning model architecture that meets the needs of TCM translation tasks; Initialize the parameters of the selected deep learning model architecture and load pre-trained weights for general domains to provide the model with basic language understanding and generation capabilities; According to the characteristics of TCM translation, the training task is defined, and the loss function is configured according to the training task to obtain the initial translation model; Format the data in the training set into the input format required by the initial translation model, and use the distributed training framework to distribute the training tasks to several computing nodes; During the training process, the hyperparameters of the initial translation model are dynamically adjusted to balance the training speed and performance of the initial translation model. A monitoring cycle is preset to periodically monitor the loss changes during training, save model weights and training logs, and stop training when the performance of the initial translation model reaches the preset standard to obtain a pre-trained translation model.

[0016] Optionally, the specific process of verifying the pre-trained translation model through the verification set, fine-tuning the pre-trained translation model according to the verification result, testing the fine-tuned pre-trained translation model through the test set, and evaluating the test result is: Input the data in the validation set into the pre-trained translation model to obtain the translation results of the validation set; Based on the professional standards of TCM translation, the translation results of the validation set were manually evaluated. The evaluation content included the accuracy of terminology translation, the completeness of cultural background transmission and the fluency of the text. The types and frequencies of errors in the translation results were also recorded. Based on manual evaluation results, the translation defects of pre-trained translation models in traditional Chinese medicine translation tasks are analyzed; Based on the analyzed translation defects, the pre-trained translation model is fine-tuned. During the fine-tuning process, the cross-validation method is used to dynamically adjust the ratio of the training set and the validation set. After fine-tuning is completed, the data in the test set is input into the fine-tuned pre-trained translation model to obtain the translation results of the test set; Based on the translation quality evaluation indicators and combined with the professional standards of TCM translation, the translation results of the test set were quantitatively evaluated. The accuracy of term translation, the score of cultural background transmission and the fluency score of the text were calculated to form a comprehensive score. This score is used to comprehensively evaluate the performance of the fine-tuned pre-trained translation model in the TCM translation task and obtain the evaluation results.

[0017] Optionally, the specific process of optimizing the fine-tuned pre-trained translation model according to the evaluation result to obtain the translation optimization model is: Based on the translation defects, error types, and error frequencies recorded in the evaluation results, identify the deficiencies of the pre-trained translation model in TCM terminology translation, cultural background transmission, and text fluency; According to the identified defects, the preprocessed data set is optimized in a targeted manner, and the optimized preprocessed data set is re-enhanced and divided to obtain the optimized training set, optimized verification set, and optimized test set; Based on the optimized preprocessed data set, train the fine-tuned pre-trained translation model, and adjust the hyperparameters and learning strategy of the fine-tuned pre-trained translation model; During the training process, we continuously monitor the loss changes of the fine-tuned pre-trained translation model, optimize the performance indicators on the validation set, and optimize the comprehensive score on the test set to ensure that the model is gradually improved during iterative optimization. When the comprehensive score of the model on the optimized test set reaches the preset optimization standard and shows stable performance in different evaluation dimensions, training is stopped and the translation optimization model is obtained.

[0018] The technical solution of the present invention has at least the following advantages and beneficial effects: Improve translation accuracy: Through multi-source data fusion, we can collect a wide range of TCM-related data, including ancient literature, modern medical research, expert interpretation, etc., so as to build a comprehensive and rich initial TCM data set, which provides a solid foundation for subsequent translation work and helps to improve translation accuracy; at the same time, formatting, terminology annotation and cultural background annotation of the data set can further ensure the accuracy of terminology use and the integrity of cultural background information during the translation process.

[0019] Enhance cultural adaptability: Through in-depth understanding of TCM terminology and classical Chinese, combined with cultural background annotations, the essence of TCM culture can be fully retained in translation, enabling foreign readers to more accurately understand the connotation and philosophical thoughts of TCM culture, thereby enhancing the adaptability and influence of TCM culture in the international community.

[0020] Improve translation efficiency: Combining deep learning and multi-source data fusion methods, it is possible to automatically learn and optimize translation models, thereby significantly improving translation efficiency while ensuring translation quality, which is of great significance for promoting the global dissemination of Chinese medicine culture.

[0021] Promoting the internationalization of Chinese medicine culture: Through the translation optimization method of the present invention, a translation text of Chinese medicine classics that is both accurate and rich in cultural connotations can be generated, which helps to break down language and cultural barriers and promote the dissemination and exchange of Chinese medicine culture in the international community; at the same time, it also provides strong support for the development of Chinese medicine in the international medical field.

[0022] Promoting the in-depth development of TCM research: Accurate translation texts can not only help international scholars better understand and study TCM culture, but also provide important references for the modernization research and international development of TCM. Through the translation optimization method of the present invention, the in-depth development of TCM research can be promoted, contributing to the modernization and internationalization of TCM. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for translating multi-source data of traditional Chinese medicine based on deep learning according to Example 1 of the present invention. DETAILED DESCRIPTION

[0024] The following is a specific implementation method in conjunction with the drawings.

[0025] Example 1 Reference Figure 1 , a method for translating multi-source data of traditional Chinese medicine based on deep learning, comprising the steps of: Step 1: Obtain TCM-related data through several data sources to obtain an initial TCM data set. In this embodiment, the specific process is as follows: The texts of Chinese medicine classics, modern Chinese medicine literature and corresponding Chinese medicine translation texts are obtained from several data sources; among them, several data sources include Chinese medicine digital libraries, academic journals, collections of Chinese medicine academic papers, Chinese medicine research institutions and expert translations.

[0026] In this embodiment, the texts of Chinese medicine classics and their high-quality translation versions are obtained through several channels. These Chinese medicine classic texts cover multiple fields of Chinese medicine, including classic works such as "Huangdi Neijing" and "Compendium of Materia Medica", as well as modern Chinese medicine literature. Specific sources include: Public Chinese medicine digital libraries: A large number of digitized Chinese medicine classic texts are collected from online libraries such as China National Knowledge Infrastructure (CNKI) and the Chinese Medical Association. Academic journals and databases: By searching medical journals and databases, modern translations of several Chinese medicine literature are obtained, providing basic materials for constructing translation comparisons. Proceedings of Chinese medicine academic conferences: The focus is on collecting international academic conference papers in the field of Chinese medicine to ensure that the data set contains the latest translation research results. Translations of cooperative institutions and experts: Through cooperation with Chinese medicine research institutions and translation experts, high-quality translation versions that have been strictly reviewed are obtained to ensure the professionalism and reliability of the data.

[0027] Step 2: Format the initial data set of traditional Chinese medicine to obtain a formatted data set.

[0028] In this embodiment, the specific process is: Convert the text data of the initial TCM data set into a standard format to obtain a standardized text data set; unify the collected TCM data into a standardized format, such as JSON, CSV or XML.

[0029] Remove noise characters, extra spaces and garbled characters from the standardized text dataset, and correct punctuation marks to obtain a preliminary cleaned dataset; Perform word segmentation on the preliminary cleaned data set, segment it into sentences, extract the structural hierarchical information of the text, and align the source language and target language of the multilingual text to obtain the text sorting data set; Add labels to the text collection data set and save the labeled text collection data set in a standardized file format to obtain a formatted data set. Through field consistency check and script verification, ensure the format compatibility of data from different sources, and provide standardized data input for subsequent annotation and model training.

[0030] Step 3: Annotate the formatted data set with terms and cultural background to obtain an annotated data set.

[0031] In this embodiment, the specific process is: Based on the formatted data set, a TCM terminology database is constructed to record the definition, context examples and corresponding translation versions of each TCM term. The co-occurrence matrix is ​​used to capture the contextual relevance of each TCM term, so as to achieve accurate translation. The expression of the co-occurrence matrix is ​​shown in the following formula (1):

[0032] in, Words to express and The co-occurrence frequency in the entire text, represents the indicator function, Indicates the window size.

[0033] Based on the formatted data set, we sorted out cultural background information covering historical allusions, philosophical thoughts and customs, built a cultural background database, and quantified the importance of each cultural element in the cultural background database through a weighted scoring mechanism; Based on the importance and corresponding attributes of each cultural element, each TCM term is annotated in layers, and each TCM term is aligned with the corresponding cultural element to construct a cultural background dictionary of TCM terms and obtain an annotated dataset.

[0034] In this embodiment, the following sentences related to traditional Chinese medicine are taken as examples to construct a vocabulary co-occurrence matrix: Sentence 1: Qi deficiency leads to blood deficiency, and blood deficiency leads to pulse deficiency.

[0035] Sentence 2: Deficiency of qi and blood leads to weakness, and imbalance between yin and yang causes diseases.

[0036] Sentence 3: Deficiency of qi generates wind, and stagnation of qi leads to blood stasis.

[0037] Sentence 4: Stagnation of qi causes blood stasis, and deficiency of qi leads to cold.

[0038] Select the following key terms to construct a co-occurrence matrix: Qi Deficiency; Blood Deficiency; Qi Stagnation; Blood Stasis; Yin-Yang; Imbalance.

[0039] Statistically analyze the co-occurrence relationships of the above terms in the text. Assume the window size is 1, that is, the maximum interval between two terms is one Chinese character. A co-occurrence matrix as shown in Table 1 below can be generated : Table 1 Example Table of Lexical Co-occurrence Matrix

[0040] "1" indicates that two terms appear adjacent to each other in the text with the set window size, and "0" indicates that two terms do not appear adjacent to each other in the text with the set window size. As shown in Table 1, "Qi Deficiency" and "Blood Deficiency" appear adjacent to each other in the first sentence with the set window size, so in the lexical co-occurrence matrix the value at the corresponding position is 1.

[0041] Use this co-occurrence matrix to construct a list of related terms for each term, which is used by the model to call when processing translations. For example: Qi Deficiency, related terms: Blood Deficiency, Qi Stagnation.

[0042] Qi Stagnation, related terms: Blood Stasis, Qi Deficiency.

[0043] These relationships can be stored in a term database to help the translation model more accurately select relevant terms during translation and maintain context consistency.

[0044] Automatic labeling tool: Use NLP tools to automatically label terms, and combine word embedding technology (WordEmbedding) to convert terms into high-dimensional vector representations through the following formula (2). Formula (2) is as follows:

[0045] in, It is vocabulary The vector representation of is the weight matrix, is the one-hot encoding vector of the vocabulary. In this way, the translation model can process terms in different contexts through similarities in the vector space and ensure the accuracy of annotations through manual review, which significantly improves the efficiency of term processing.

[0046] In order to help the translation model better understand and convey the cultural background of traditional Chinese medicine, a cultural background database was created, covering information such as historical allusions, philosophical thoughts, customs and habits. In order to ensure that the importance of each cultural element in the translation is accurately conveyed, a weighted scoring mechanism is used to quantify these elements. Through the following formula (3), we calculated a comprehensive score for each cultural element:

[0047] in, Representing cultural elements The overall rating of Indicates The weight of background information, Representing cultural elements No. The frequency or importance of background information, Represents the total amount of cultural background information.

[0048] For example, when translating TCM texts involving the concepts of “Yin Yang and the Five Elements”, we gave a weighted score to the cultural background annotations of “Yin Yang”; we analyzed the frequency of occurrence of the cultural element “Yin Yang” in different TCM classics, its importance in philosophy, and its impact on treatment principles, and came up with the following scores: Since "Yin Yang" is mentioned many times in Huangdi Neijing, its historical allusions are of high importance. ; Since "Yin Yang" is one of the core concepts of ancient Chinese philosophy, its importance in theoretical construction is very high. ; Since "Yin Yang" is used to guide treatment principles in clinical applications, but its direct impact is relatively small, the weight of the treatment principle of the cultural element "Yin Yang" is set. .

[0049] Through calculation, the comprehensive score of the cultural element "Yin Yang" is obtained as shown in the following formula (4):

[0050] By formulating a unified cultural element annotation standard, we can ensure the consistency of cultural element information transmission in various data sets.

[0051] Step 4: Clean the labeled data set, establish a data set index according to the data source, type and semantic characteristics, store the cleaned labeled data set according to the data set index, and obtain the preprocessed data set.

[0052] In this embodiment, the specific process is: A weighted deduplication mechanism is used to deduplicate the labeled data set, assigning corresponding weights to different types of duplicate content, and deduplicating based on the weights of the duplicate content to obtain a deduplicated data set; Correct grammatical and spelling errors in the deduplicated data set, and standardize the data format of the deduplicated data set to obtain a cleaned data set; Establish a data set index based on data source, type and semantic characteristics, and use the data set index to index and annotate the cleaned data set to obtain an indexed data set; The index dataset is stored in a structured file format, and the preprocessed dataset is obtained through segment numbering and version control mechanism.

[0053] In this embodiment, the expression of weighted deduplication is shown in the following formula (5):

[0054] in, Indicates the comprehensive cleaning score for a single data item during data cleaning; Indicates The weight of the class error; Indicates Class error in data item frequency or severity of occurrence; is the total number of error types that need to be handled.

[0055] When cleaning the data items about "Yin Yang and the Five Elements", two major types of errors were identified, including spelling errors and context inconsistency errors. Different weights can be given to these two types of errors: In TCM terminology, spelling errors may lead to serious terminological misunderstandings, so spelling errors have a higher weight. ; Since the application of the concept of "Yin Yang and Five Elements" in different contexts varies greatly, the weight of context inconsistency error is higher. .

[0056] During the data cleaning process, calculate the comprehensive cleaning score for each data item For example, the comprehensive cleaning score of the data item "Yin Yang and Five Elements" is shown in the following formula (6):

[0057] According to the calculated comprehensive cleaning scores, data items with higher comprehensive scores are cleaned first to ensure that the cleaned dataset meets higher standards in terms of term accuracy and context consistency.

[0058] In this embodiment, all data is stored in a structured manner in a JSON / CSV file for easy subsequent processing and retrieval. Each translation pair will be recorded as a data entry, containing the following key fields: Source Text: The original text of traditional Chinese medicine classics; Translation: the corresponding translated text; Term Annotations: annotated TCM terms and their explanations, including definitions, contextual examples, and translations; Cultural Contexts: annotation of cultural background information, including historical allusions, philosophical thoughts, customs and habits, etc., and assigning different priorities through a weighted scoring mechanism; Error and Cleaning Info: Contains information about detected spelling errors, context inconsistencies, and other issues, and their corrections during the data cleaning process; Overall Score: A comprehensive score calculated through a weighted scoring mechanism, used to measure the quality and processing priority of data items; Translator Info: records the translator’s background information, translation year, and other relevant information.

[0059] A hybrid database structure is adopted, combining relational database and NoSQL database to meet the storage and retrieval needs of different types of data.

[0060] The relational database management system (RDBMS) includes: a terms table, a translation pair table, and a cultural context table. The terms table is used to store Chinese medicine terms and related information, supporting fast retrieval and precise matching. The translation pair table is used to store the comparison information between the original text and the translated text, and is associated with the terms table and the cultural context table. The cultural context table is used to store information related to the cultural background and its ratings, supporting cross-entry association queries.

[0061] The NoSQL database includes: Text Repository, Scoring Repository and Cleansing Log Repository; the Text Repository is used to store the original texts and translated texts of traditional Chinese medicine classics in JSON format, supporting flexible query and processing; the Scoring Repository is used to store the scoring information of each data item, including terminology scoring, cultural background scoring, data cleaning scoring, etc., for subsequent analysis and optimization; the Cleaning Log Repository is used to store the process records of data cleaning, including error types, correction measures and results, for easy backtracking and optimization.

[0062] In order to ensure the efficiency and scalability of data storage, a distributed storage system can be used to store data on multiple server nodes to achieve load balancing and high availability.

[0063] Data can be partitioned vertically based on domain attributes, such as medical terminology, cultural background, etc. Each partition is stored independently and supports parallel processing. Data can be partitioned horizontally based on time attributes, such as translation year, data creation time, to facilitate the management of historical data and new data.

[0064] Multidimensional indexes are built for terminology and cultural background tables to support fast retrieval based on terms, ratings, contexts, etc. Full-text search indexes are built for text libraries to support fuzzy queries and context matching, making it easier to handle complex text translation tasks.

[0065] A version control system is established for each data item and translation pair to record detailed information of each update and support version rollback and comparative analysis. A snapshot mechanism is established for the entire database to regularly save snapshots of the entire database to cope with the risk of data corruption or loss.

[0066] In order to ensure the security and integrity of data, a comprehensive data security and backup solution is designed, including: Data Encryption: Data is encrypted during storage and transmission to prevent unauthorized access and data leakage; Employ role-based access control (RBAC) to ensure that only authorized users can access and modify specific data; Data Backup: Adopt a scheduled backup strategy, daily incremental backup, weekly full backup, and store backup data in an off-site server to cope with catastrophic events; Use distributed file systems (such as HDFS / IPFS) to store backup data to ensure reliable backup and recovery of large-scale data.

[0067] Design open API interfaces for subsequent data processing and analysis tasks to facilitate integration with model training, testing, and evaluation systems, including: Data query API: supports complex queries based on multiple dimensions such as terminology, cultural background, and ratings, and returns structured query results for further analysis and processing; Data import / export API: supports batch import and export of large-scale data, ensures the efficiency and consistency of data flow, and supports multiple data formats (such as CSV, JSON, XML); Model interface: supports seamless integration with NLP models. The model can directly call the cleaned data through the API for training or translation tasks to ensure efficient use of data.

[0068] Step 5: Perform data enhancement on the preprocessed data set, and dynamically divide the enhanced preprocessed data set into training set, validation set, and test set.

[0069] In this embodiment, the specific process is: Replace TCM terms with their corresponding synonyms in the preprocessed dataset; randomly delete or insert non-key terms to simulate missing or redundant information in real contexts; use semantic transformation to generate diverse sentence structures; combine the term annotations and cultural background annotations of the preprocessed dataset to expand the terms and cultural background in the specified context; According to the data volume of the preprocessed data set and the requirements of the translation task, the ratio of the training set, validation set, and test set is set; a stratified sampling strategy is adopted to ensure that the divided training set, validation set, and test set are balanced in terms of TCM terminology and cultural background information; oversampling or undersampling technology is used to adjust the sample distribution; the divided training set, validation set, and test set are labeled and classified for storage to distinguish between original data and enhanced data, and a version control mechanism is established to record changes in each data division and enhancement.

[0070] In this example, the dataset size has reached hundreds of thousands of translation pairs, covering various types of TCM texts, such as classic texts, modern literature, clinical records, etc. The size of the dataset and model complexity The relationship between is shown in formula (7):

[0071] Formula (7) indicates that the size of the data set determines the effect of model training to a certain extent; Indicates the effect of model training; It is proportional to the size of the data set. The larger the model complexity, the The more data a model has, the better the training effect. Especially when dealing with complex TCM contexts, this extensive corpus can provide rich scenarios and contexts for deep learning of the model.

[0072] The preprocessed dataset is dynamically divided into training set, validation set and test set to evaluate the performance of the model. The division ratio is set as: 70% for training, 15% for validation, and 15% for testing. For the evaluation of model performance, the cross entropy loss function is used, as shown in the following formula (8):

[0073] in represents the true label, represents the predicted probability; Formula (8) is used to evaluate the accuracy of model prediction and help adjust the model parameters during the training process.

[0074] Depending on the complexity of the task, the division ratio of the preprocessed dataset can be adjusted in some experiments to ensure the generalization ability of the model. In future multilingual expansions, the dataset size may be further expanded, taking into account the multilingual factor. , the scale of the expanded dataset can be expressed by the following formula (9):

[0075] in, Indicates the size of the expanded dataset; Indicates the scale of the data set; based on this data set, further introduce data augmentation technology to enhance "If qi is weak, the blood will be weak; if qi is weak, the pulse will be weak." By replacing "weak blood" with "little blood" and "weak pulse" with "thin pulse" through synonym replacement, the synonymous expressions of terms are expanded; random insertion is used to add "Chinese medicine believes" to the sentence to increase field-related background information; random deletion is used to delete "weak qi", simulating information missing scenarios, so that the model can maintain semantic understanding under incomplete input; through semantic transformation (inversion), "if qi is weak, the blood will be weak" is changed to "weak blood is due to qi deficiency", and the word order is adjusted to generate sentences with diverse structures. Sentences are diversified through data augmentation technology to generate semantically equivalent but structurally different variants to improve the generalization and robustness of the model. For example: each data augmentation technology is regarded as an input data set. A transformation operation. Suppose the input data set is Include Sentences, namely ; Let the synonym replacement function be ,in, Represents the input sentence, represents a synonym dictionary. The synonym replacement function replaces the specified words in the sentence with the corresponding synonyms. The sentence expression after synonym replacement is shown in the following formula (10):

[0076] in, Indicates the sentence after the specified word is replaced by the corresponding synonym; Let the random insertion function be ,in, Represents a list of words to be inserted; through the random insertion function, randomly select words to insert into the sentence In the sentence expression after inserting random words, the sentence expression is shown in formula (11):

[0077] in, Represents the sentence after the random words are inserted; Let the random deletion function be ,in, Represents the deletion rate; through the random deletion function, the probability of deletion Randomly delete sentences The sentence expression after deleting the words in is shown in formula (12):

[0078] in, Deletion probability Randomly delete the sentences after the words; Let the back-translation function be ,in, Represents the translation API; through the back-translation function, the sentence After translating into another language and then translating back to the original language, the back-translated sentence expression is shown in formula (13):

[0079] in, represents the back-translated sentence; Let the semantic transformation function be , Represents semantic transformation rules; adjusts sentences through semantic transformation functions The structure of the adjusted sentence is shown in formula (14):

[0080] in, represents the sentence adjusted by the semantic transformation function; For a given sentence and a list of enhancement techniques, as well as the corresponding parameters, the enhanced sentence expression is shown in Equation (15):

[0081] in, represents a list of enhancement techniques, the list of enhancement techniques includes synonym replacement, random insertion, random deletion, back translation and semantic transformation; Indicates the parameters corresponding to the synonym replacement function, random insertion function, random deletion function, back translation function and semantic transformation function; represents a composite enhancement function consisting of a synonym replacement function, a random insertion function, a random deletion function, a back translation function, and a semantic transformation function; Represents the corresponding enhanced sentence obtained after the given sentence is processed by the composite enhancement function.

[0082] Step 6: Based on deep learning, build an initial translation model, train the initial translation model through the training set, and obtain a pre-trained translation model.

[0083] In this embodiment, the specific process is: Select a deep learning model architecture that meets the needs of TCM translation tasks; initialize the parameters of the selected deep learning model architecture, and load pre-trained weights in general fields to provide the model with basic language understanding and generation capabilities; define training tasks based on the characteristics of TCM translation, and configure loss functions based on the training tasks to obtain an initial translation model; format the data in the training set into the input format required by the initial translation model, and use a distributed training framework to distribute the training tasks to several computing nodes; during the training process, dynamically adjust the hyperparameters of the initial translation model to balance the training speed and performance of the initial translation model; preset a monitoring cycle, periodically monitor the loss changes during the training process, save the model weights and training logs, and stop training when the performance of the initial translation model reaches the preset standard to obtain a pre-trained translation model.

[0084] In this embodiment, ChatGPT can be used as the basic model, and its general pre-trained weights trained on large-scale corpus can be loaded. ChatGPT has powerful language generation and understanding capabilities, providing a solid foundation for the domain adaptation of traditional Chinese medicine translation tasks. During the training process, clear training tasks are defined, such as optimizing the difference between the model-generated translation and the target translation through the cross-entropy loss function, and using the term annotation information to hierarchically learn the characteristics of traditional Chinese medicine terminology and cultural background, thereby improving the model's adaptability to the domain language.

[0085] In order to ensure the stability and efficiency of training, a distributed training framework (such as Horovod or DeepSpeed) was adopted, and hyperparameters were set according to the training requirements, including learning rate (such as 2e-5), batch size (such as 32), and number of training rounds. The distributed training technology significantly improved the computing efficiency, ensuring that the model can complete high-quality training under limited computing resources. At the same time, in order to monitor the performance of the model in real time, a verification mechanism was introduced during the training process to regularly evaluate the model's performance in terminology consistency, cultural background transmission, and translation fluency, and save weights and training logs to support interruption recovery and phased optimization. For example, when the model's translation quality of the term "Qi deficiency" was monitored during training, the learning rate and data annotation coverage were further adjusted to achieve an improvement from the translation result of "Qi deficiency" "Qi deficiency" to a more accurate "Qi deficiency (vital energy)", significantly enhancing the accuracy of the translation and the transmission of cultural connotations. Ultimately, this pre-training process laid a solid foundation for the subsequent fine-tuning and practical application of the model.

[0086] Step 7: Verify the pre-trained translation model through the validation set, fine-tune the pre-trained translation model according to the verification results, test the fine-tuned pre-trained translation model through the test set, and evaluate the test results.

[0087] In this embodiment, the specific process is: The data in the validation set is input into the pre-trained translation model to obtain the translation results of the validation set; based on the professional standards of TCM translation, the translation results of the validation set are manually evaluated, and the evaluation content includes the accuracy of term translation, the completeness of cultural background transmission and the fluency of the text, and the error types and error frequencies in the translation results are recorded; according to the manual evaluation results, the translation defects of the pre-trained translation model in the TCM translation task are analyzed; according to the analyzed translation defects, the pre-trained translation model is fine-tuned, and during the fine-tuning process, the cross-validation method is used to dynamically adjust the ratio of the training set and the validation set; after the fine-tuning is completed, the data in the test set is input into the fine-tuned pre-trained translation model to obtain the translation results of the test set; based on the translation quality evaluation indicators and combined with the professional standards of TCM translation, the translation results of the test set are quantitatively evaluated, and the accuracy of term translation, the score of cultural background transmission and the score of text fluency are calculated to form a comprehensive score, which is used to comprehensively evaluate the performance of the fine-tuned pre-trained translation model in the TCM translation task and obtain the evaluation results.

[0088] In this example, the fine-tuned model can accurately translate complex terms, such as optimizing the general translation of "Qi deficiency" from "Qideficiency" to "Qi deficiency (vital energy)", which retains the cultural connotation. By adding term annotations and cultural markers to the input, the model can generate an explanatory translation when translating "Yin and Yang imbalance", and the translation result is: "Imbalance of Yin and Yang, a core concept in Chinese medicine." The fine-tuned model improved the BLEU and ROUGE evaluation indicators by 15% and 12% respectively, significantly enhancing the accuracy and fluency of the translation.

[0089] Step 8: Based on the evaluation results, optimize the fine-tuned pre-trained translation model to obtain a translation optimization model.

[0090] In this embodiment, the specific process is: According to the translation defects, error types and error frequencies recorded in the evaluation results, the defects of the pre-trained translation model in the translation of traditional Chinese medicine terms, cultural background transmission and text fluency are identified; the pre-processed dataset is optimized in a targeted manner for the identified defects, and the optimized pre-processed dataset is re-enhanced and data divided to obtain the optimized training set, optimized validation set and optimized test set; based on the optimized pre-processed dataset, the fine-tuned pre-trained translation model is trained, and the hyperparameters and learning strategies of the fine-tuned pre-trained translation model are adjusted; during the training process, the loss changes of the fine-tuned pre-trained translation model, the performance indicators on the optimized validation set and the comprehensive score on the optimized test set are continuously monitored to ensure that the model is gradually improved during iterative optimization; when the comprehensive score of the model on the optimized test set reaches the preset optimization standard and shows stable performance in different evaluation dimensions, the training is stopped to obtain the translation optimization model.

[0091] Example 2 Based on Example 1, in this embodiment, in the fine-tuning process, ChatGPT is used as the basic model, and the domain adaptation of the model is achieved by loading pre-trained weights and importing annotated traditional Chinese medicine data sets. First, the data is divided into training sets, validation sets, and test sets to ensure the scientific nature of fine-tuning. Next, the fine-tuning goals are clarified to optimize the consistency of term translation, the accurate transmission of cultural background, and the fluency of the text. Subsequently, the data is formatted into sentence pairs and combined with data enhancement techniques such as synonym replacement and context expansion to further enrich the training samples. During the fine-tuning training process, a distributed training framework is used to optimize computing efficiency, and model performance is improved by adjusting the learning rate, batch size, and generation strategy. After the training is completed, indicators such as BLEU and ROUGE are used in combination with manual verification to comprehensively evaluate the term translation, text fluency, and cultural background transmission effects of the model. Use Hugging Face or OpenAI's model loading tool to load the ChatGPT model and its pre-trained parameters.

[0092] In this embodiment, the pre-trained translation model is fine-tuned by a model fine-tuning training algorithm. Assume that the parameters of the model are , the training set is , the validation set is , the loss function is .

[0093] Load parameters from the pre-trained ChatGPT model For each training sample , define the loss function Predict for the model With the true label The overall loss function is The expression of is shown in formula (16):

[0094] Use gradient descent or its variants (such as the Adam optimizer) to update the parameters The optimization process can be expressed by the following formula (17):

[0095] in, Indicates that at time step The updated model parameters are: Indicates that at time step Parameters of the model when represents the learning rate, Represents the loss function with respect to the parameter gradient.

[0096] After each epoch, use the validation dataset To evaluate the performance of the model, and adjust the learning rate or other hyperparameters as needed. When the predetermined number of training rounds is reached or the validation performance no longer improves, stop training.

[0097] The performance evaluation algorithm is used to quantify the translation quality of the model. The performance evaluation algorithm mainly quantifies the translation quality of the model by calculating the BLEU and ROUGE scores. The BLEU (Bilingual Evaluation Understudy) score is a weighted sum of the geometric mean of the n-gram precision, while taking into account the brevity penalty of the translation. Suppose the reference translation is , the candidate translation is , then the BLEU score It can be expressed by the following formula (18):

[0098] in, represents the short penalty factor, Indicates the precision of n-gram, Represents the weight of n-gram. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score is a variety of indicators based on recall rate, such as ROUGE-N, ROUGE-L, etc. Then ROUGE-N score It can be expressed by the following formula (19):

[0099] in, Indicates the number of times the n-gram appears in the text; Indicates candidate translations and reference translation collections For the test dataset , calculate the BLEU and ROUGE scores for each sample, and then take the average as the final evaluation result of the model.

[0100] After fine-tuning the model, the model's performance in the translation task is tested through a combination of automated evaluation and manual verification. The test focuses on translation accuracy, text fluency, and terminology comprehension. The specific test steps include: Automatic evaluation indicators such as BLEU and ROUGE are used to quantify the degree of match between the generated translation and the target translation in terms of n-gram and fragment overlap, and evaluate the translation quality of the model; for TCM terminology, a term detection algorithm is designed to automatically locate the terms in the generated text, and compare them with the standard translation version to calculate the term consistency and accuracy; the Perplexity indicator is introduced to evaluate the language fluency of the generated text, and manual verification is combined to perform supplementary checks on the translation of key terms and the transmission of cultural background. The test results show that the model performs well in term translation and text fluency, but the translation effect of some complex terms and long texts still has room for improvement. For example, the consistency of the term "Qi deficiency" reached 80%, but it failed to fully cover common contexts. The test provides a clear direction for subsequent optimization.

[0101] Optimize the model based on the evaluation results, and further adjust the data set or translation process to achieve self-improvement. Based on the test results, optimize the model, data set, and translation process to further improve translation performance. The specific steps include: Analyze low-quality samples and identify the root causes of inaccurate term translation, such as the inconsistent translation of the term "qi deficiency" in a specific context; expand the term-related samples in the data set to address the inconsistent translation problem, and use data enhancement technology to generate more variant sentences, such as adding context expansion and synonym replacement; adjust the fine-tuning parameters of the model, reduce the learning rate (such as from 2e-5 to 1e-5), increase the number of training rounds, and introduce a loss function for term consistency weights to optimize the model's adaptability to specific tasks; introduce a term correction module into the translation process, combined with automated rules and manual review, to improve the accuracy of the translation output. The test results of the optimized model show that the consistency of the term "qi deficiency" increased from 80% to 92%, the BLEU score increased from 45 to 50, and the ROUGE score increased from 50 to 55, with a significant improvement in overall performance. This cyclic optimization process has laid a more solid foundation for the practical application of the model.

[0102] Through the above steps and strategies, the ChatGPT model will be able to better adapt to the translation task of traditional Chinese medicine classics, ensuring that the translation process not only maintains the accuracy of the language, but also effectively conveys the cultural connotations. This fine-tuning method will enable the model to show stronger professionalism and cultural sensitivity in the field of traditional Chinese medicine translation.

[0103] In this embodiment, relevant model translation tests are performed, and the terms "qi", "yin and yang", "phlegm and dampness", and "excessive liver fire" are respectively analyzed as examples.

[0104] The term "chi", initial translation: Input: Qi deficiency and physical weakness Initial model output: Qi deficiency, weak body Problem: Although "Qi" is correctly retained as pinyin, the subsequent "weak body" does not accurately convey the TCM concept of "weakness". This translation is relatively straightforward and lacks cultural background explanation.

[0105] Translation after fine-tuning: Fine-tuning data: added sentences containing “qi” and annotated the cultural background of “qi”-related translations, such as “qi deficiency” as “a state of deficiency in vital energy, often leading to physical weakness.” Model output after fine-tuning: Qi deficiency leading to physical weakness. Improvement: The fine-tuned translation more accurately conveys the meaning of "Qi deficiency", retains the cultural connotation of "Qi", and explains the symptom of "physical weakness".

[0106] The term "Yin Yang", initial translation: Input: Imbalance of Yin and Yang Initial model output: Imbalance of Yin and Yang. Problem: Although the translation is correct, it does not reflect the importance of Yin and Yang as a philosophical idea in Chinese medicine, nor does it explain the specific meaning of "disharmony".

[0107] Translation after fine-tuning: Fine-tuning the data: We added Yin-Yang related sentences and cultural background annotations to the dataset, especially the specific manifestations of Yin-Yang imbalance in different literature and translation strategies.

[0108] Model output after fine-tuning: Imbalance of Yin and Yang, a core concept in Chinese medicine representing the disruption of the body's dynamic equilibrium. Improvement: After minor adjustments, explanatory content was added to help the target readers understand the status and role of "yin and yang imbalance" in traditional Chinese medicine theory.

[0109] The term "phlegm dampness", initial translation: Input: Phlegm and dampness Initial Model Output: Excessive Phlegm-Dampness Problem: The translation is accurate but too brief, failing to reflect the pathological mechanism and severity of "phlegm and dampness" in traditional Chinese medicine.

[0110] Translation after fine-tuning: Fine-tuning data: The dataset is supplemented with the pathological background of "phlegm-dampness" and its manifestations in different diseases, and provides a variety of translation options.

[0111] Model output after fine-tuning: Excessive Phlegm-Dampness, a pathological condition characterized by the accumulation of pathological fluids in the body leading to various health issues. Improvement: The output has been fine-tuned to explain the pathological characteristics of "phlegm and dampness", making the translation more detailed and with TCM background knowledge.

[0112] The term "Liver Fire", initial translation: Input: Insomnia caused by excessive liver fire Initial model output: Excess Liver Fire causes insomnia Problem: The translation is direct, but it doesn’t convey how “liver fire” is connected to emotions and mental states in TCM.

[0113] Translation after fine-tuning: Fine-tuning data: The dataset supplements the relationship between "liver fire" and emotions in traditional Chinese medicine, as well as the symptoms it may cause.

[0114] Model output after fine-tuning: Excess Liver Fire, often associated with emotional stress and irritability, leading to insomnia. Improvement: Added explanations on the emotion of "strong liver fire" to make the translation more complete.

[0115] Experimental results analysis Through these use cases, the fine-tuned ChatGPT model showed significant improvements in translating TCM terms, specifically: Cultural context preservation: By adding term explanations and cultural context information to the dataset, the fine-tuned model is able to better convey the deeper meaning of TCM terms.

[0116] Improved translation accuracy: Compared with the initial model, the fine-tuned model is more accurate in handling polysemous words and specialized terms.

[0117] BLEU and ROUGE scores: In the post-translation evaluation, the BLEU score of the model increased by about 15%, and the ROUGE score increased by 12%, indicating that the fluency and accuracy of the translation have been improved.

[0118] The translation of TCM classics is not only a language conversion, but also a cultural transmission. By building a self-built dataset and fine-tuning the ChatGPT model, the ability to handle TCM terminology and cultural connotations has been significantly improved. The self-built dataset provides professional training materials for the model, enabling it to better understand and translate terminology while retaining cultural background information. The fine-tuned ChatGPT performs well in contextual adaptability and cultural transmission, and the generated translation text is more consistent with the original meaning.

Claims

1. A method for translating multi-source data of traditional Chinese medicine based on deep learning, characterized in that: Includes steps: Obtain TCM-related data through several data sources to obtain the initial TCM data set; Formatting the initial data set of traditional Chinese medicine to obtain a formatted data set; Perform terminology annotation and cultural background annotation on the formatted data set to obtain an annotated data set; Clean the labeled data set, establish a data set index according to the data source, type and semantic characteristics, store the cleaned labeled data set according to the data set index, and obtain the preprocessed data set; Perform data augmentation on the preprocessed data set, and dynamically divide the augmented preprocessed data set into training set, validation set, and test set; Based on deep learning, an initial translation model is constructed, and the initial translation model is trained using a training set to obtain a pre-trained translation model. Verify the pre-trained translation model through the validation set, fine-tune the pre-trained translation model based on the verification results, test the fine-tuned pre-trained translation model through the test set, and evaluate the test results; According to the evaluation results, the fine-tuned pre-trained translation model is optimized to obtain a translation optimization model.

2. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of obtaining TCM-related data through several data sources to obtain the initial TCM data set is as follows: The texts of Chinese medicine classics, modern Chinese medicine literature and corresponding Chinese medicine translation texts are obtained from several data sources; among them, several data sources include Chinese medicine digital libraries, academic journals, collections of Chinese medicine academic papers, Chinese medicine research institutions and expert translations.

3. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of formatting the initial data set of traditional Chinese medicine to obtain the formatted data set is as follows: Convert the text data of the initial dataset of traditional Chinese medicine into a standard format to obtain a standardized text dataset; Remove noise characters, extra spaces and garbled characters from the standardized text dataset, and correct punctuation marks to obtain a preliminary cleaned dataset; Perform word segmentation on the preliminary cleaned data set, segment it into sentences, extract the structural hierarchical information of the text, and align the source language and target language of the multilingual text to obtain the text sorting data set; Labels are added to the text sorting dataset, and the labeled text sorting dataset is saved in a standardized file format to obtain a formatted dataset.

4. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of performing terminology annotation and cultural background annotation on the formatted data set to obtain the annotated data set is as follows: Based on the formatted data set, a TCM terminology database is constructed to record the definition, context examples and corresponding translation versions of each TCM term, and the contextual relevance of each TCM term is captured through a lexical co-occurrence matrix; Based on the formatted data set, we sorted out cultural background information covering historical allusions, philosophical thoughts and customs, built a cultural background database, and quantified the importance of each cultural element in the cultural background database through a weighted scoring mechanism; Based on the importance and corresponding attributes of each cultural element, each TCM term is annotated in layers, and each TCM term is aligned with the corresponding cultural element to construct a cultural background dictionary of TCM terms and obtain an annotated dataset.

5. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of performing data cleaning on the annotated data set, establishing a data set index according to the data source, type and semantic characteristics, and storing the cleaned annotated data set according to the data set index to obtain the preprocessed data set is as follows: A weighted deduplication mechanism is used to deduplicate the labeled data set, assigning corresponding weights to different types of duplicate content, and deduplicating based on the weights of the duplicate content to obtain a deduplicated data set; Correct grammatical and spelling errors in the deduplicated data set, and standardize the data format of the deduplicated data set to obtain a cleaned data set; Establish a data set index based on data source, type and semantic characteristics, and use the data set index to index and annotate the cleaned data set to obtain an indexed data set; The index dataset is stored in a structured file format, and the preprocessed dataset is obtained through segment numbering and version control mechanism.

6. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 5, characterized in that: The expression of weighted deduplication is shown in the following formula (5): in, Indicates the comprehensive cleaning score for a single data item during data cleaning; Indicates The weight of the class error; Indicates Class error in data item frequency or severity of occurrence; is the total number of error types that need to be handled.

7. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of performing data enhancement on the preprocessed data set and dynamically dividing the enhanced preprocessed data set into a training set, a validation set, and a test set is as follows: Replace TCM terms with their corresponding synonyms in the preprocessed data set; randomly delete or insert non-key terms to simulate missing or redundant information in the real context; Use semantic transformation to generate diverse sentence structures; Combine the terminology annotation and cultural background annotation of the preprocessed dataset to expand the terminology and cultural background in the specified context; Set the ratio of training set, validation set, and test set according to the amount of preprocessed data and the requirements of the translation task; A stratified sampling strategy was adopted to ensure that the training set, validation set, and test set were balanced in terms of TCM terminology and cultural background information. Oversampling or undersampling techniques were used to adjust the sample distribution. The training set, validation set, and test set were labeled and classified for storage to distinguish between original data and enhanced data. A version control mechanism was established to record changes in each data division and enhancement.

8. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of constructing an initial translation model based on deep learning, training the initial translation model through a training set, and obtaining a pre-trained translation model is as follows: Select a deep learning model architecture that meets the needs of TCM translation tasks; Initialize the parameters of the selected deep learning model architecture and load pre-trained weights for general domains to provide the model with basic language understanding and generation capabilities; According to the characteristics of TCM translation, the training task is defined, and the loss function is configured according to the training task to obtain the initial translation model; Format the data in the training set into the input format required by the initial translation model, and use the distributed training framework to distribute the training tasks to several computing nodes; During the training process, the hyperparameters of the initial translation model are dynamically adjusted to balance the training speed and performance of the initial translation model. A monitoring cycle is preset to periodically monitor the loss changes during training, save model weights and training logs, and stop training when the performance of the initial translation model reaches the preset standard to obtain a pre-trained translation model.

9. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of verifying the pre-trained translation model through the verification set, fine-tuning the pre-trained translation model according to the verification result, testing the fine-tuned pre-trained translation model through the test set, and evaluating the test result is as follows: Input the data in the validation set into the pre-trained translation model to obtain the translation results of the validation set; Based on the professional standards of TCM translation, the translation results of the validation set were manually evaluated. The evaluation content included the accuracy of terminology translation, the completeness of cultural background transmission and the fluency of the text. The types and frequencies of errors in the translation results were also recorded. Based on manual evaluation results, the translation defects of pre-trained translation models in traditional Chinese medicine translation tasks are analyzed; Based on the analyzed translation defects, the pre-trained translation model is fine-tuned. During the fine-tuning process, the cross-validation method is used to dynamically adjust the ratio of the training set and the validation set. After fine-tuning is completed, the data in the test set is input into the fine-tuned pre-trained translation model to obtain the translation results of the test set; Based on the translation quality evaluation indicators and combined with the professional standards of TCM translation, the translation results of the test set were quantitatively evaluated. The accuracy of term translation, the score of cultural background transmission and the fluency score of the text were calculated to form a comprehensive score. This score is used to comprehensively evaluate the performance of the fine-tuned pre-trained translation model in the TCM translation task and obtain the evaluation results.

10. The method for translating multi-source data of traditional Chinese medicine based on deep learning as claimed in claim 1, characterized in that: The specific process of optimizing the fine-tuned pre-trained translation model according to the evaluation results to obtain the translation optimization model is as follows: Based on the translation defects, error types, and error frequencies recorded in the evaluation results, identify the deficiencies of the pre-trained translation model in TCM terminology translation, cultural background transmission, and text fluency; According to the identified defects, the preprocessed data set is optimized in a targeted manner, and the optimized preprocessed data set is re-enhanced and divided to obtain the optimized training set, optimized verification set, and optimized test set; Based on the optimized preprocessed data set, train the fine-tuned pre-trained translation model, and adjust the hyperparameters and learning strategy of the fine-tuned pre-trained translation model; During the training process, we continuously monitor the loss changes of the fine-tuned pre-trained translation model, optimize the performance indicators on the validation set, and optimize the comprehensive score on the test set to ensure that the model is gradually improved during iterative optimization. When the comprehensive score of the model on the optimized test set reaches the preset optimization standard and shows stable performance in different evaluation dimensions, training is stopped and the translation optimization model is obtained.

Images