Construction method and system of lightweight language translation model in field of international drug registration

Abstract

The invention relates to the technical field of machine translation, and discloses a method and system for constructing a lightweight language translation model in the field of drug international registration, and the method comprises the steps: firstly, constructing a knowledge base comprising a text library and a vector library; then, a retrieval module is constructed, the retrieval module queries translation examples related to the to-be-translated text from the document library and the vector library, and the translation examples are fused and merged in an equal proportion mode to serve as a final query result; then, constructing an instruction data sample, and dividing the instruction data sample into a training sample and a test sample; and finally, training a Qwen-18B-Chat model on the training sample by using a low-rank adapter technology, and inputting a to-be-translated text and a related translation example retrieved in the knowledge base into the trained model by using a retrieval enhancement generation technology to obtain a translation result. The method realizes high-efficiency and high-quality translation of the international drug registration file, and has the advantages of easy deployment, low processing cost and high confidentiality.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation, and in particular to a method and system for constructing a lightweight language translation model in the field of international drug registration. Background Art

[0002] Drug registration documents record in detail the confidential information such as research information, clinical data, quality control standards, etc. accumulated by the drug. Usually, a new drug registration document has more than 200 documents, up to 60,000 to 70,000 pages. In international drug registration, document translation must ensure both high efficiency and high quality, and information security, which is of great significance to improving the efficiency of drug research and development and ensuring the smooth progress of subsequent review and approval work. At present, drug registration documents are submitted in the format specified by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH). The translation problems of this type of documents in my country are mainly manifested as follows: 1. Slow speed: more than 200 registration documents usually take 1-2 months to complete; 2. Poor consistency: The documents contain multidisciplinary content, such as chemistry or biology, medicine, pharmacy, clinical and production quality, and are written by different departments. Therefore, in addition to ensuring that data, charts, text and language are always unified, they must also be consistent with the standards and requirements of various countries' regulations and industries that are updated from time to time. Traditional manual operations are very challenging; 3. High cost: The translation cost of tens of thousands of pages usually costs hundreds of thousands or even millions of RMB, which adds uncertainty to high-risk fields such as new drugs. In addition, the lack of professional translators has led to the cost, quality and speed of translation of new drug registration documents becoming a "pain point" problem that has long been ignored by the industry.

[0003] Existing large language models (LLMs) have performed well in natural language processing tasks, including machine translation. However, due to the high computing resource requirements for training and reasoning these large-parameter models, which in turn incurs high costs, as well as the strong dependence of general LLMs on data quality and quantity, biases and errors, and insufficient legitimacy and privacy of training corpora, LLMs still face many challenges when applied to machine translation, especially in the translation of documents such as drug registration documents, which are highly confidential, time-sensitive, require professional and rigorous expression, and must use professional terminology. Summary of the invention

[0004] In view of the technical problems existing in the above-mentioned prior art, the present invention provides a method and system for constructing a lightweight language translation model in the field of international drug registration, so as to realize high-quality, rapid and automated translation of international drug registration documents.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] In a first aspect, the present invention provides a method for constructing a lightweight language translation model in the field of international drug registration, the method comprising the following steps:

[0007] Step S1, constructing a knowledge base containing translation examples in the field of international drug registration, wherein the knowledge base includes a document base and a vector base;

[0008] Step S2, constructing a retrieval module, wherein the retrieval module respectively searches for translation examples matching the text to be translated based on keywords from the document library, searches for translation examples matching the text to be translated based on semantics from the vector library, and combines the two query results by geometric fusion as the final query result;

[0009] Step S3, constructing instruction data samples and dividing them into training samples and test samples; using the training samples to train the model so that it can better understand and process complex medical terms and expressions, and effectively solve the common problems of inaccurate terminology and semantic understanding deviation in traditional translation models in professional field translation;

[0010] Step S4, use low-rank adapter technology to train the Qwen-1_8B-Chat model on the training samples to obtain the lightweight language translation model PhT-LM in the field of international drug registration.

[0011] Preferably, step S1 is specifically as follows: step S11, obtaining original Chinese and English documents from the official website of the State Drug Administration, international authoritative organizations in the field of pharmacy, and international drug registration textbooks;

[0012] Step S12, splitting and mapping each paragraph of the original Chinese and English documents;

[0013] Step S13, the data obtained in step S12 is deduplicated, reviewed, and a correspondence between Chinese and English texts is established, and error information is corrected and supplemented; the data quality and professionalism of the knowledge base are ensured, errors and biases in the data are removed, and the translation model is prevented from retrieving wrong translation information, which leads to a decrease in the translation quality of the text to be translated. In addition, high-granularity data can make retrieval and matching more accurate;

[0014] Step S14, using the processed data as a document library;

[0015] Step S15, the processed data is encoded into vectors using a text embedding model as a vector library. This knowledge base is constructed to obtain high-quality relevant translation examples and improve translation accuracy and professionalism.

[0016] Preferably, the text embedding model is selected from the mxbai-embed-large-v1 model or the stella-large-zh-v3-1792d model.

[0017] Preferably, step S2 is specifically as follows: step S21, using ES search to search for translation examples matching the text to be translated based on keywords from the document library;

[0018] Step S22, using the vector library to search for translation examples based on semantic matching with the text to be translated from the vector library;

[0019] Step S23, respectively extract the first n / 2 results of the query in the text library and the first n / 2 results of the query in the vector library for fusion as the final query result, where n is the total number of query results. Through the geometric fusion retrieval strategy, translation examples of the retrieved text in the keyword dimension and the semantic matching dimension can be obtained. These translation examples combine the accuracy based on vocabulary matching and the relevance based on semantic understanding, thereby further improving the quality and accuracy of the translation model. This method can not only capture the direct correspondence at the vocabulary level, but also effectively use contextual information to improve the natural fluency and contextual adaptability of the translation.

[0020] Preferably, n is 4.

[0021] Preferably, step S3 is specifically as follows: step S31, defining an instruction template for translation in the field of international registration of drugs;

[0022] Step S32, taking each piece of data in the document library as a text to be translated;

[0023] Step S33, searching for translation examples related to the text to be translated through a search module;

[0024] Step S34, deleting text data in the query results that is identical to the text to be translated;

[0025] Step S35, introducing the text to be translated and the queried relevant translation examples into the instruction template to form a translation sentence pair with instructions, and obtaining an instruction data sample;

[0026] Step S36, randomize the instruction data samples and divide them into training samples and test samples.

[0027] The instruction templates specially defined for translation in the field of drug registration can stimulate the original potential of the model and effectively improve the performance of the model when processing highly specialized texts. The retrieved translation examples and the text to be translated are introduced into the instruction template to form translation sentence pairs with instructions, and training samples and test samples are constructed. Using the training samples to train the model, the model can be trained to pay attention to and utilize the translation example pairs that are closely related to the text to be translated retrieved from different dimensions of the knowledge base, thereby enriching the auxiliary translation information of the text to be translated and improving the translation quality.

[0028] Preferably, the expression for training the Qwen-1_8B-Chat model in step S4 is:

[0029] h=W0x+△Wx=W0x+ABx

[0030] Among them, h is the hidden activation variable, W0 is the parameter of the Qwen-1_8B-Chat model, x is the input sample, And r is less than d, A and B are two updated parameter matrices.

[0031] In a second aspect, the present invention provides a lightweight language translation system in the field of international drug registration, comprising:

[0032] Knowledge base, which is a search base for translation examples of the text to be translated;

[0033] A retrieval module is used to obtain translation examples of the text to be translated from the knowledge base;

[0034] The translation instruction generation module generates translation instructions using instruction templates combined with translation examples of the text to be translated;

[0035] The translation model PhT-LM obtained by the method is used to obtain translation instructions and generate translation text.

[0036] The present invention has the following beneficial effects: (1) The construction of a translation knowledge base in the field of international drug registration can ensure that the translation model obtains high-quality training data, and can also provide high-quality translation examples that are closely related to the text to be translated to assist the model translation, thereby improving the accuracy and professionalism of the model translation. The geometric fusion retrieval strategy combines the advantages of ES retrieval and vector library retrieval, and performs retrieval from the two dimensions of keyword matching and semantic matching, thereby improving the comprehensiveness and accuracy of the retrieval, and aims to provide the model with more suitable translation examples. After obtaining the training samples in the field of drug registration, the basic model is trained using the Low-Rank Adaptation (LoRA) technology, so that the model can better understand and process complex medical terms and expressions, thereby significantly improving the translation quality of the international drug registration text. The lightweight language translation system can be deployed on personal computers and enterprise private servers, which reduces hardware costs, improves model flexibility, and can effectively solve the confidentiality issues of international drug registration documents during the translation process, and realize high-quality, fast and automated translation of international drug registration documents.

[0037] (2) Compared with the general large language model, the low-cost deployment and operation environment of this translation model enables it to be deployed locally on personal computers and small corporate servers, and the data does not need to be uploaded to the cloud, greatly reducing the risk of sensitive information leakage. In addition, the ease of use of this model means that a large number of professional translators are no longer needed to carry out complex document translation work. Even people without a translation background can easily translate Chinese and English documents in the field of international drug registration, significantly reducing the translation costs in the process of drug research and development and marketing.

[0038] (3) By using RAG technology to input the text to be translated together with similar translation examples retrieved from the knowledge base into the trained model, the rich auxiliary information in the translation examples can be fully utilized, enabling the translation model to better understand and process complex language structures and semantic relationships, reducing translation errors caused by lack of context, thereby significantly improving the accuracy and naturalness of the translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The present invention is a flowchart for constructing a lightweight language translation model PhT-LM in the field of international drug registration.

[0040] Figure 2 Build a flowchart for your knowledge base.

[0041] Figure 3 This is the framework diagram of the retrieval module.

[0042] Figure 4 Construct a flow graph for the instruction data sample.

[0043] Figure 5This is a schematic diagram of the LoRA lightweight training method.

[0044] Figure 6 a shows the impact of different numbers of translation examples in the retrieval module on the model translation effect in the English-to-Chinese translation task; b shows the impact of different numbers of translation examples in the retrieval module on the model translation effect in the Chinese-to-English translation task.

[0045] Figure 7 A flowchart of a lightweight language translation system in the field of international drug registration provided by the present invention. DETAILED DESCRIPTION

[0046] The following examples also describe certain specific aspects and embodiments, which are provided for illustrative purposes only and should not be construed as limiting the scope of the invention in any way.

[0047] like Figure 1 As shown, a method for constructing a lightweight language translation model in the field of international drug registration specifically includes the following steps:

[0048] Step S1, construct a knowledge base containing a large number of translation examples in the field of international drug registration, which includes a document library and a vector library, such as Figure 2 As shown;

[0049] Step S11, using Python's Selenium library to write an automated script to obtain original Chinese and English documents from the official website of the State Drug Administration and international authoritative organizations in the field of pharmacy. For data in the international registration textbooks for drugs, the electronic version is directly used;

[0050] Step S12, using Python's Docx library to read the obtained Chinese and English files, split and map each paragraph of the file, and store the corresponding Chinese translation and English translation as two columns of an Excel file;

[0051] Step S13, performing deduplication processing on the data obtained in step S12, eliminating redundant data pairs, and avoiding the existence of the same text content in files from different sources, and then reviewing the format and content of the data, establishing the correspondence between Chinese and English texts and correcting and supplementing error information;

[0052] Step S14, storing the processed data in an Elasticsearch (ES for short) library as a document library;

[0053] Step S15, encode the processed data into a vector using a text embedding model, and then store it in another ES library as a vector library. The text embedding model selects the mxbai-embed-large-v1 model and the stella-large-zh-v3-1792d model.

[0054] Paragraphs of the document are split to generate high-granularity data, making retrieval and matching more accurate. After being reviewed and supplemented with error information correction by professional medical field translators, irrelevant and low-quality data are removed, significantly improving the overall quality of the dataset and preventing the translation quality of the text to be translated from decreasing due to retrieving incorrect information and biases during translation.

[0055] Step S2: Construct a retrieval module. The retrieval module queries translation examples that match the text to be translated based on keywords from the document library and queries translation examples that match the text to be translated based on semantics from the vector library; the two query results are merged by geometric ratio fusion as the final query result, as Figure 3 shown;

[0056] Step S21: ES retrieval includes three parts: text processing, query statement generation, and matching processing. First, use the IK tokenizer to tokenize the text to be translated, breaking it down into smaller units, i.e., words; then, filter out those words that frequently appear in most texts but are not helpful for content understanding, such as "de" (of), "shi" (is), etc., through the stop word list. This step helps to streamline the query keywords and improve the retrieval efficiency and relevance; next, according to the processed query keywords, construct a specific query request that conforms to the ES query specification. After receiving the query request, ES uses the pre-constructed inverted index to quickly locate the text containing specific terms. The inverted index is an efficient data structure that records which texts each term appears in and its occurrence positions. Finally, for each matching text, ES calculates its relevance score to the query based on the BM25 algorithm, sorts the matching texts from high to low by the score, and returns the final query result;

[0057] Step S22: Vector library retrieval, i.e., semantic search, determines the relevance of texts in the library to the search query by analyzing the semantic content of the texts. The text to be translated is encoded as a vector through a text embedding model, and then, according to the vector representation of the text to be translated, the cosine similarity algorithm is used to calculate the cosine value of the angle between its vector representation and each text vector representation in the vector library, which is used to represent the similarity between the two. The closer the cosine value is to 1 and the closer the angle is to 0, the more similar the two vectors are. The closer the cosine value is to 0 and the closer the angle is to 90 degrees, the less similar the two vectors are. Finally, the texts in the vector library are sorted from high to low according to the calculated similarity, and the query result is returned.

[0058] Assume that the vector representation of the text to be translated is x = {x1, x2, x3,..., x n}, and the vector representation of a certain text in the vector library is y = {y1, y2, y3,..., y n}, then the cosine similarity calculation formula between the two texts is:

[0059]

[0060] Step S23: for any specified number of results n, the first n / 2 highest ranking results are equally extracted from the ES search and the vector library search as the final query result. In this embodiment, n is 4.

[0061] Step S3, construct instruction data samples and divide them into training samples and test samples, such as Figure 4 As shown;

[0062] Step S31, defining an instruction template applicable to translation in the field of international registration of drugs;

[0063] Step S32, taking each piece of data in the document library as a piece of text to be translated in the instruction data sample;

[0064] Step S33, searching for translation examples related to the text to be translated from the document library and the vector library through the retrieval module;

[0065] Step S34, deleting text data in the query results that is identical to the text to be translated;

[0066] Step S35, introducing the text to be translated and the queried relevant translation examples into the instruction template to form a translation sentence pair with instructions, and obtaining an instruction data sample;

[0067] Step S36, randomize the instruction data samples and divide them into training samples and test samples.

[0068] A specific example of a command data sample is shown below:

[0069] "instruction":"You are a language expert in the field of pharmacy.\nKnown information:\n'(2) Manufacturing is performed while the Drug Manufacturing Certificate has expired.' can be translated as: (2)Manufacturing is performed while the Drug Manufacturing Certificate has expired.\n'(II) The applicant and the manufacturer have not obtained the corresponding drug production license, or the product has not completed the process verification of commercial large-scale manufacturing;' can be translated as: (II)The applicant and the manufacturer have not obtained the corresponding drug production license, or the product has not completed the process verification of commercial large-scale manufacturing;\n'Article 19 In the case of continuing the drug manufacturing following the expiration of the Drug Manufacturing Certificate, the company shall apply to the original issuing authority for renewal of the Drug Manufacturing Certificate 6 months ahead of the expiration.' can be translated as: Article 19 In the case of continuing the drug manufacturing following the expiration of the Drug Manufacturing Certificate, the company shall apply to the original issuing authority for renewal of the Drug Manufacturing Certificate 6 months ahead of the expiration.\n'(2) the License is not renewed upon expiration of the valid term;' can be translated as: (2)the License is not renewed upon expiration of the valid term;\n\nBased on the above known information, please translate '(2) The Drug Manufacturing Certificate has expired but not been re-issued;' into English. Only the translated sentence is allowed to be output, and no fabricated content is allowed.",

[0070] "output":"(2)The Drug Manufacturing Certificate has expired but not been re-issued;"

[0071] Through training with the above samples, the translation model can fully focus on and utilize the information about "drug production license" and "validity period" in the translation examples to assist in the translation of subsequent texts to be translated, thereby improving the accuracy of the translation.

[0072] Step S4, using low-rank adapter technology to train the Qwen-1_8B-Chat model on the training samples to obtain a lightweight language translation model PhT-LM in the field of international drug registration;

[0073] LoRA optimizes model parameters by introducing a bypass structure, which is formed by multiplying matrix A and matrix B. The dimension of matrix A is d*r, and the dimension of matrix B is r*d. Since the value of r is much smaller than the value of d, the number of parameters of the bypass structure is much less than the parameter W0 of the original model. During the model training process, the parameter W0 of the original model is frozen, and only the parameters A and B of the bypass structure are trained. In actual use, the parameters of the bypass structure are directly merged with the parameters of the original network, that is, the parameters are added, so as to achieve an effect similar to full training while keeping the inference time unchanged.

[0074] That is, for the parameter The parameter update △W can be represented by a low-rank decomposition, namely:

[0075]

[0076] During the training process, the parameter W0 is frozen and only the parameters in A and B are trained. For h = W0x, x is the input sample, h is the hidden activation variable, and the forward propagation process becomes:

[0077] h=W0x+△Wx=W0x+ABx

[0078] Compared with the original parameter scale W0, the parameter scale that needs to be tuned by training samples is greatly reduced. Studies have shown that when using LoRA technology, only 0.01% of the parameters need to be trained to achieve good results. Using this method to train the Qwen-1_8B-Chat model not only ensures the performance of the original model, but also reduces the resources and training time required for training the model, and increases the feasibility of the translation model.

[0079] During the training of the translation model, in order to ensure the stability of the environment, all training was performed on an Ubuntu 22.04 operating system equipped with an 80GB A100 graphics card. The hyperparameters of the training process: maximum length of input sequence (cutoff_len), batch (batch_size), learning rate (learning rate, lr), number of training rounds (num_train_epochs) and the rank number of LoRA (lora_rank) were adjusted. The hyperparameter configuration of the Qwen-1_8B-Chat model was determined to be: cutoff_len = 2048, batch_size = 4, lr = 5e-5, num_train_epochs = 2, lora_rank = 8. This series of hyperparameters was used to train Qwen-1_8B-Chat to obtain a trained translation model PhT-LM.

[0080] The trained model uses the Bilingual Evaluation Understudy (BLEU) and Character n-gram F-score (CHRF) indicators to evaluate the quality of the translation model. The BLEU score focuses on n-gram matching at the lexical level, while the CHRF focuses on n-gram correspondence at the character level. The combined use of these two scoring criteria can comprehensively consider the correspondence of translations at all levels, thereby obtaining more accurate, consistent and professional translation texts.

[0081] Table 1 Evaluation results of the trained translation model and existing models

[0082]

[0083]

[0084] Table 1 shows the evaluation results of the translation model after training. In the English-Chinese test set, PhT-LM achieved the highest scores in the five evaluation indicators of BLEU-1, BLEU-2, BLEU-3, BLEU-4, and CHRF, reaching 69.053, 56.726, 47.744, 40.741, and 51.115 respectively, significantly exceeding ChatGLM3-6B, Chinese-Alpaca-2-7B, and GPT-3.5-turbo general large models. In the Chinese-English translation task, PhT-LM also demonstrated its excellent performance, and each indicator also exceeded other existing general large models, with scores increased by 16% to 65%. The results show that in the document translation task in the field of international drug registration, the PhT-LM model disclosed in the present invention achieved the best results, which is better than the existing general large language model.

[0085] Table 2 Evaluation results of different retrieval strategies

[0086]

[0087] In order to evaluate the effectiveness of the geometric fusion retrieval strategy, two test groups were constructed, covering Chinese-English and English-Chinese tasks respectively. Each test group contains two test samples. In one test sample, the relevant translation examples of the text to be translated are obtained by using ES retrieval, and in the other test sample, the relevant translation examples of the text to be translated are obtained by using vector library retrieval. The evaluation results are shown in Table 2. Whether it is the Chinese-English translation task or the English-Chinese translation task, the model using the geometric fusion retrieval strategy has higher scores in BLEU-1 to BLEU-4 and CHRF indicators than the model using ES retrieval alone or vector library retrieval alone. Therefore, it can be proved that the use of the geometric fusion retrieval strategy can further improve the translation quality of the model, that is, the fusion of ES retrieval results and vector library retrieval results can achieve complementary feature information.

[0088] Figure 6 The effect of different numbers of translation examples in the retrieval module on the model translation effect is shown. By modifying the number n of final query results returned by the retrieval module, test sets with 1, 2, 4, 8, and 16 translation examples were constructed to evaluate the effect of different numbers of translation examples on the performance of the translation model and to determine the optimal number of queries for the retrieval module.

[0089] It can be found that the overall trend of BLEU and CHRF scores is the same for both English-to-Chinese and Chinese-to-English tasks, that is, when the number of translation examples in the translation instruction increases from 1 to 4, the BLEU-1 to BLEU-4 scores and the CHRF scores all show an upward trend, and then stabilize. When more translation examples are added (for example, the number of examples is 16), the BELU score and CHRF score show a downward trend. Therefore, it can be determined that when the final query result contains 4 translation examples, the model has the greatest effect on improving the translation quality of text in the field of drug registration.

[0090] The above description is only a preferred embodiment of the present invention, but is not intended to limit the present invention. Those skilled in the art may make possible changes and modifications to the present invention using the technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for constructing a lightweight language translation model in the field of international drug registration, characterized in that: The method comprises the following steps: Step S1, constructing a knowledge base containing translation examples in the field of international drug registration, wherein the knowledge base includes a document base and a vector base; Step S2, constructing a retrieval module, wherein the retrieval module respectively searches for translation examples matching the text to be translated based on keywords from the document library, searches for translation examples matching the text to be translated based on semantics from the vector library, and combines the two query results by geometric fusion as the final query result; Step S3, constructing instruction data samples and dividing them into training samples and test samples; Step S4, use low-rank adapter technology to train the Qwen-1_8B-Chat model on the training samples to obtain the lightweight language translation model PhT-LM in the field of international drug registration.

2. The method for constructing a lightweight language translation model in the field of international drug registration according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11, obtaining original Chinese and English documents from the official website of the State Drug Administration, international authoritative organizations in the field of pharmacy, and international drug registration textbooks; Step S12, splitting and mapping each paragraph of the original Chinese and English documents; Step S13, the data obtained in step S12 is deduplicated, reviewed, a correspondence between Chinese and English texts is established, and error information is corrected and supplemented; Step S14, using the processed data as a document library; Step S15, encode the processed data into vectors using a text embedding model as a vector library.

3. The method for constructing a lightweight language translation model in the field of international drug registration according to claim 2, characterized in that: The text embedding model is selected from the mxbai-embed-large-v1 model or the stella-large-zh-v3-1792d model.

4. The method for constructing a lightweight language translation model in the field of international drug registration according to claim 1, characterized in that: Step S2 is specifically as follows: Step S21, using ES search to search for translation examples that match the text to be translated based on keywords from the document library; Step S22, using the vector library to search for translation examples based on semantic matching with the text to be translated from the vector library; Step S23, respectively extract the first n / 2 results of the query in the text library and the first n / 2 results of the query in the vector library, and merge them as the final query result, where n is the total number of query results.

5. The method for constructing a lightweight language translation model in the field of international drug registration according to claim 4, characterized in that: The n is 4.

6. The method for constructing a lightweight language translation model in the field of international drug registration according to claim 1, characterized in that: Step S3 is specifically as follows: Step S31, defining an instruction template for translation in the field of international drug registration; Step S32, taking each piece of data in the document library as a text to be translated; Step S33, searching for translation examples related to the text to be translated through a search module; Step S34, deleting text data in the query results that is identical to the text to be translated; Step S35, introducing the text to be translated and the queried relevant translation examples into the instruction template to form a translation sentence pair with instructions, and obtaining an instruction data sample; Step S36, randomize the instruction data samples and divide them into training samples and test samples.

7. The method for constructing a lightweight language translation model in the field of international drug registration according to claim 1, characterized in that: The expression for training the Qwen-1_8B-Chat model in step S4 is: h=W0x+△Wx=W0x+ABx Among them, h is the hidden activation variable, W0 is the parameter of the Qwen-1_8B-Chat model, x is the input sample, And r is less than d, A and B are two updated parameter matrices.

8. A lightweight language translation system for international drug registration, characterized in that: include: Knowledge base, which is a search base for translation examples of the text to be translated; A retrieval module is used to obtain translation examples of the text to be translated from the knowledge base; The translation instruction generation module generates translation instructions using instruction templates combined with translation examples of the text to be translated; The translation model PhT-LM obtained by the method according to any one of claims 1 to 7 is used to obtain translation instructions and generate translation text.

Images