A method and apparatus for generating traditional Chinese medicine prescriptions based on multi-task joint optimization

Abstract

This application provides a method and apparatus for generating traditional Chinese medicine (TCM) prescriptions based on multi-task joint optimization. The method includes: constructing a training set based on historical clinical case data, the training set comprising multiple samples and sample weight labels, each sample containing clinical text features, TCM prescription labels, and prescription set labels; constructing an enhanced training set based on the training set; training a multi-task joint optimization prescription generation model based on the enhanced training set; predicting TCM prescription sequences using an autoregressive approach, with the TCM prescription labels as the supervision signal; a set prediction branch used to predict the presence or absence of each TCM herb in the global TCM dictionary based on the semantic representation of clinical text features, with the prescription set labels as the supervision signal, and introducing sample weight labels into its loss function to enhance attention to low-frequency TCM herbs; training the model by weighted fusion of the loss functions of the above two branches; and generating TCM prescriptions using the trained model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and apparatus for generating traditional Chinese medicine prescriptions based on multi-task joint optimization. Background Technology

[0002] In recent years, artificial intelligence (AI), especially large language models, has demonstrated remarkable generalization capabilities across various fields, including its application in Traditional Chinese Medicine (TCM). For instance, it has brought new opportunities to the task of generating TCM prescriptions. TCM prescriptions are personalized treatment plans formulated by TCM practitioners based on TCM principles and clinical insights, by examining patients' symptoms, constitution, and other relevant data. However, limited by the availability of TCM prescription data and the long-tailed distribution of TCM usage frequency, TCM prescription generation models are not only easily affected by the order of drug arrangement during training but also tend to favor generating high-frequency drugs while ignoring low-frequency drugs, thus affecting the accuracy and comprehensiveness of the generated results.

[0003] Relevant research on TCM prescription generation mainly focuses on predicting TCM prescriptions based on symptom / medical record text. Common approaches can be summarized into two categories: First, prescription generation methods, which serialize prescriptions into drug name sequences and use autoregressive models such as Seq2Seq and Transformer to generate TCM prescriptions based on symptom text. They also mitigate duplicate and missed prescriptions through deduplication constraints or constraint decoding. This involves supervised fine-tuning of pre-trained large language models on case prescription data, or using parameter fine-tuning techniques such as LoRA to achieve TCM prescription generation. Second, TCM prescription recommendation methods, which model prescriptions as set / multi-label results. They characterize the relationship between symptoms and TCM through multi-label learning or graph learning, typically combining multi-graph structures and graph neural networks, and utilizing contrastive pre-training and hierarchical modeling to enhance representation and alleviate data sparsity.

[0004] Although the above methods have made some progress in the generation and recommendation of traditional Chinese medicine prescriptions, they still have at least the following shortcomings:

[0005] First, existing methods for generating traditional Chinese medicine prescriptions typically model prescriptions as ordered drug sequences, implicitly introducing the assumption of a fixed generation order. However, clinical prescriptions focus more on the overall compatibility between Chinese medicines rather than their sequential order. Even with deduplication constraints or coverage mechanisms to mitigate the problems of repetition and omission, these models are still inevitably affected by order dependencies during the training and decoding stages, limiting their ability to fully understand and model the patterns of prescription combinations.

[0006] Second, in both recommendation-based and generative approaches, model optimization objectives are often dominated by data distribution. The long-tailed nature of TCM usage frequency makes models more prone to fitting high-frequency drugs, while failing to learn enough about low-frequency TCMs. Even though some works have enhanced representations through graph structure modeling or contrastive learning, these methods are still limited in their effectiveness in alleviating the imbalance between high- and low-frequency TCMs and improving the coverage of low-frequency TCMs. This results in insufficient diversity and comprehensiveness in the generated results, thus limiting the application value of the models in real clinical scenarios. Summary of the Invention

[0007] In view of this, this application proposes a method and apparatus for generating traditional Chinese medicine prescriptions based on multi-task joint optimization.

[0008] Specifically, this application is implemented through the following technical solution:

[0009] According to a first aspect of the embodiments of this specification, a method for generating traditional Chinese medicine prescriptions based on multi-task joint optimization is provided, comprising the following steps:

[0010] Step S1: Construct a training set based on historical clinical case data. The training set includes multiple samples and sample weight labels. Each sample corresponds to a historical case in the historical clinical case data and includes clinical text features, traditional Chinese medicine prescription labels, and prescription set labels. The traditional Chinese medicine prescription labels indicate the names of the traditional Chinese medicines in the prescription corresponding to the historical case. The prescription set labels are multi-label vectors generated based on a global traditional Chinese medicine dictionary, used to indicate the types of traditional Chinese medicines included in the prescription. The sample weight labels are calculated based on the frequency of each traditional Chinese medicine appearing in the corresponding prescriptions of the historical clinical case data.

[0011] Step S2: For at least a portion of the samples in the training set, keep the clinical text features and prescription set labels of each sample unchanged, randomly rearrange the sequence of Chinese medicine names in the Chinese medicine prescription labels multiple times, and form derived samples by combining the Chinese medicine prescription labels, clinical text features and prescription set labels corresponding to the rearranged sequences, and form an enhanced training set based on the original samples and the derived samples.

[0012] Step S3: Train a multi-task joint optimization prescription generation model based on the enhanced training set. The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone and is configured with a sequence generation branch and an ensemble prediction branch. The sequence generation branch is used to predict the sequence of traditional Chinese medicine prescriptions in an autoregressive manner, and its supervision signal is the traditional Chinese medicine prescription label. The ensemble prediction branch is used to predict the presence or absence of each traditional Chinese medicine in the global traditional Chinese medicine dictionary based on the semantic representation of clinical text features, and its supervision signal is the prescription ensemble label. Sample weight labels are introduced into its loss function to enhance attention to low-frequency traditional Chinese medicines. On the enhanced training set, the model is jointly trained by weighted fusion of the loss functions of the sequence generation branch and the ensemble prediction branch.

[0013] Step S4: Use the trained multi-task joint optimization prescription generation model to generate traditional Chinese medicine prescriptions.

[0014] According to a second aspect of the embodiments of this specification, a traditional Chinese medicine prescription generation device based on multi-task joint optimization is provided, comprising:

[0015] The preprocessing unit is used to construct a training set based on historical clinical case data. The training set includes multiple samples and sample weight labels. Each sample corresponds to a historical case in the historical clinical case data and includes clinical text features, traditional Chinese medicine prescription labels, and prescription set labels. The traditional Chinese medicine prescription labels indicate the names of the traditional Chinese medicines in the prescription corresponding to the historical case. The prescription set labels are multi-label vectors generated based on a global traditional Chinese medicine dictionary, used to indicate the types of traditional Chinese medicines included in the prescription. The sample weight labels are calculated based on the frequency of each traditional Chinese medicine appearing in the corresponding prescriptions of the historical clinical case data.

[0016] An enhancement processing unit is used to, for at least a portion of the samples in the training set, keep the clinical text features and prescription set labels of each sample unchanged, randomly rearrange the sequence of Chinese medicine names in the Chinese medicine prescription labels multiple times, form derived samples by combining the Chinese medicine prescription labels, clinical text features and prescription set labels corresponding to the rearranged sequences, and form an enhanced training set based on the original samples and the derived samples.

[0017] The model training unit is used to train a multi-task joint optimization prescription generation model based on the enhanced training set. The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone and is configured with a sequence generation branch and an ensemble prediction branch. The sequence generation branch is used to predict the sequence of traditional Chinese medicine prescriptions in an autoregressive manner, and its supervision signal is the traditional Chinese medicine prescription label. The ensemble prediction branch is used to predict the presence or absence of each traditional Chinese medicine in the global traditional Chinese medicine dictionary based on the semantic representation of clinical text features, and its supervision signal is the prescription ensemble label. Sample weight labels are introduced into its loss function to enhance attention to low-frequency traditional Chinese medicines. The model is jointly trained on the enhanced training set by weighted fusion of the loss functions of the sequence generation branch and the ensemble prediction branch.

[0018] The model inference unit is used to generate traditional Chinese medicine prescriptions using the trained multi-task joint optimization prescription generation model.

[0019] According to a third aspect of the embodiments of this specification, an electronic device is provided, including a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method described in the first aspect.

[0020] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor of the method described in the first aspect.

[0021] The embodiments of this application have at least the following technical effects:

[0022] This application embodiment uses a data augmentation strategy to perform multiple rounds of random rearrangement of the Chinese herbal medicine (TCM) sequences of at least a portion of the TCM prescription tags in the training set to generate derived text with different prescription orders than the original samples. An augmented training set is constructed based on the original and derived samples. The model trained on this augmented training set can reduce the order dependency interference of non-prescription intrinsic attributes, enabling the model to learn the TCM compatibility and combination rules more stably. Furthermore, this application embodiment constructs a joint optimization framework including sequence generation and set prediction. During model training, while constraining the difference between the TCM prescription sequence generated by the model and the TCM prescription tags based on the loss function of the sequence generation branch, it also predicts the difference between the multi-label prediction results representing the probability of each TCM occurrence and the prescription set tags based on the set prediction branch. Sample weight labels are introduced into this loss function to assign higher training weights to low-frequency TCM, weakening the dominance effect of high-frequency drugs and improving the coverage and recall performance of long-tail low-frequency TCM, thereby enhancing the applicability and reliability of the model in TCM prescription generation scenarios. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0024] Figure 1 This is a schematic diagram of the overall architecture for generating traditional Chinese medicine prescriptions, as illustrated in an exemplary embodiment of this application.

[0025] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the present application of a method for generating traditional Chinese medicine prescriptions based on multi-task joint optimization;

[0026] Figure 3 This is a structural block diagram of an electronic device illustrated in an exemplary embodiment of this application;

[0027] Figure 4 This is a structural block diagram of a traditional Chinese medicine prescription generation device based on multi-task joint optimization, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0030] The related technologies model prescriptions as ordered drug sequences, which leads to interference from the drug arrangement order during the training and decoding stages. This makes it difficult to fully learn the overall compatibility relationship between Chinese medicines, affecting the accurate modeling of prescription combination rules. Furthermore, the long-tailed distribution of the frequency of Chinese medicine use makes the model susceptible to data distribution during training, tending to fit high-frequency drugs while paying insufficient attention to low-frequency Chinese medicines, thus affecting the comprehensiveness and clinical applicability of the generated results.

[0031] Based on this, the technical solution of this application proposes a traditional Chinese medicine prescription generation scheme based on multi-task joint optimization. For example... Figure 1 As shown, this embodiment first desensitizes, denoises, and structures historical clinical case data to construct a standardized dataset. The samples include clinical text features, traditional Chinese medicine (TCM) prescription labels, and prescription set labels. Based on this, a global TCM list is statistically summarized for prescription labeling, and the frequency of each TCM herb is further calculated to generate sample weight labels, providing a foundation for subsequent multi-label independent prediction and imbalanced optimization. The standardized dataset is hierarchically divided according to the distribution characteristics of TCM herbs to obtain training, validation, and test sets. Furthermore, a data augmentation strategy is introduced only for the training set. This involves keeping the clinical text features and prescription set labels of each sample unchanged, and randomly rearranging the sequence of TCM herb names in the prescription labels multiple times to generate derived samples. This allows for the formation of an enhanced training set based on the original and derived samples, thereby reducing the model's dependence on the order of drug arrangement and guiding the model to focus more on the combination relationships between TCM herbs. Finally, a multi-task joint optimization prescription generation model is constructed. This model has two task branches: a sequence generation branch and an ensemble prediction branch. It can retain the sequence generation objective while introducing multi-label independent prediction objectives, and performs weighted optimization based on sample weight labels to reduce the dominant effect of high-frequency traditional Chinese medicines during training, thereby improving the learning and coverage capabilities of low-frequency traditional Chinese medicines. The model in this embodiment can improve overall performance while maintaining prescription generation quality and enhance the recall performance of low-frequency drugs.

[0032] The core idea of ​​multi-task joint optimization in this application is to allow a pre-trained large language model to learn two related tasks simultaneously, enabling these tasks to share the underlying feature extraction process, thereby mutually promoting each other and ultimately improving the model's performance across all tasks. Since the underlying core of these tasks is understanding the patient's condition, sequence patterns based on sequence generation and ensemble prediction allow the model to have a more comprehensive and profound understanding of medical principles. Thus, for a given medical record, the model outputs a complete prescription sequence under the constraint of whether certain Chinese medicines should be prescribed.

[0033] The embodiments described in this specification will now be described in detail.

[0034] This application provides a method for generating traditional Chinese medicine prescriptions based on multi-task joint optimization. Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of this application of a method for generating traditional Chinese medicine prescriptions based on multi-task joint optimization. Figure 2 As shown, the method for generating traditional Chinese medicine prescriptions includes at least the following steps:

[0035] Step S1: Construct a training set based on historical clinical case data. The training set includes multiple samples and sample weight labels. Each sample corresponds to a historical case in the historical clinical case data and includes clinical text features, traditional Chinese medicine prescription labels, and prescription set labels. The traditional Chinese medicine prescription labels indicate the names of the traditional Chinese medicines in the prescription corresponding to the historical case. The prescription set labels are multi-label vectors generated based on a global traditional Chinese medicine dictionary and are used to indicate the types of traditional Chinese medicines included in the prescription. The sample weight labels are calculated based on the frequency of each traditional Chinese medicine appearing in the corresponding prescription in the historical clinical case data.

[0036] Clinical text information for each case is extracted as clinical text features, and the sequence of Chinese herbal medicine names in the prescription is extracted as Chinese herbal medicine prescription tags. A global Chinese herbal medicine dictionary is obtained based on all Chinese herbal medicine prescription tags. A multi-label vector generated based on the global Chinese herbal medicine dictionary is used to indicate the types of Chinese herbal medicines contained in the prescription. Each element in the multi-label vector corresponds to a Chinese herbal medicine in the global Chinese herbal medicine dictionary. If the Chinese herbal medicine appears in the prescription corresponding to the historical case, the value is 1; otherwise, the value is 0.

[0037] To achieve high-quality model fine-tuning, this embodiment first performs standardized preprocessing and structured analysis on historical clinical case data to form a standard dataset that the model can directly use. Then, based on the distribution characteristics of traditional Chinese medicine, the dataset is divided into hierarchical random partitions to obtain training, validation, and test sets.

[0038] The processing of historical clinical case data includes organizing patient complaints, present medical history, detailed diagnoses, and other information, and using the resulting text sequence as the clinical text feature, which is used as model input; extracting Chinese herbal combinations from prescriptions in traditional Chinese medicine cases to form a sequence of Chinese herbal name names as the Chinese herbal prescription label, which is used as a supervision signal for the sequence generation task; and obtaining a global Chinese herbal dictionary by deduplicating historical prescriptions, which is used for prescription label encoding and limiting the range of drugs that the model can output; based on the global Chinese herbal dictionary, each historical prescription is encoded as a 0 / 1 vector of length N, where N is the total number of Chinese herbal medicines in the global Chinese herbal dictionary, 1 indicates that the Chinese herbal medicine appears in the historical prescription, and 0 indicates that it does not appear, thus obtaining the prescription set label corresponding to each sample, which is used for training the set prediction branch.

[0039] This embodiment also counts the frequency of occurrence of each Chinese herbal medicine. Compared with the total sample size A, according to Calculate the weighting coefficient, which is used to weight low-frequency Chinese medicines in the loss calculation to alleviate the long-tail imbalance problem.

[0040] Step S2: For at least a portion of the samples in the training set, keep the clinical text features and prescription set labels of each sample unchanged, randomly rearrange the sequence of Chinese medicine names in the Chinese medicine prescription labels multiple times, and form derived samples by combining the Chinese medicine prescription labels, clinical text features and prescription set labels corresponding to the rearranged sequences, and form an enhanced training set based on the original samples and the derived samples.

[0041] To effectively alleviate the dependence on the order of non-prescription intrinsic attributes during model fine-tuning, this embodiment designs a data augmentation strategy. Data augmentation is achieved by randomly arranging the labels of traditional Chinese medicine prescriptions in the training set to alleviate overfitting of traditional Chinese medicine position information during model training, thereby strengthening the learning of traditional Chinese medicine combinations.

[0042] For example, an enhancement factor k is introduced, where k random shuffling cycles are considered as one enhancement cycle. While maintaining the original clinical text features and prescription set labels, the names of Chinese medicines in the prescription labels of the original sample are randomly rearranged k times. The resulting prescription labels from each rearrangement, along with the original clinical text features and prescription set labels, form a derived sample. This derived sample, along with the original sample, forms the enhanced training set. Since the enhanced training set contains samples with the same clinical text features but different prescription order, the model focuses more on the Chinese medicine combination itself during fine-tuning, thereby improving the accuracy of Chinese medicine prescription generation.

[0043] Step S3: Train a multi-task joint optimization prescription generation model based on the enhanced training set. The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone and is configured with a sequence generation branch and an ensemble prediction branch. The sequence generation branch is used to predict the sequence of traditional Chinese medicine prescriptions in an autoregressive manner, and its supervision signal is the traditional Chinese medicine prescription label. The ensemble prediction branch is used to predict whether each traditional Chinese medicine in the global traditional Chinese medicine dictionary appears based on the semantic representation of clinical text features, and its supervision signal is the prescription ensemble label. Sample weight labels are introduced into its loss function to enhance attention to low-frequency traditional Chinese medicines. On the enhanced training set, joint training is performed by weighted fusion of the loss functions of the sequence generation branch and the ensemble prediction branch.

[0044] During model training, cross-entropy loss is used to optimize the difference between the predicted TCM prescription sequence and the TCM prescription label in the sequence generation branch. Furthermore, the predicted important prescription sequence is transformed into an independent judgment of whether each TCM appears through the ensemble prediction branch. Binary cross-entropy loss is used for multi-label training, and sample weight labels are combined to assign higher weights to the label items corresponding to low-frequency TCM during loss calculation, thereby reducing the dominant influence of high-frequency TCM in the training process. The model is jointly trained by weighted fusion of the loss function of the sequence generation branch and the loss function of the ensemble prediction branch. While ensuring that the predicted TCM prescription sequence conforms to the distribution of training data, the prediction accuracy of the overall TCM prescription set is improved, and the problem of insufficient learning of low-frequency TCM caused by the high-frequency TCM dominating the training process is effectively alleviated.

[0045] Step S4: Use the trained multi-task joint optimization prescription generation model to generate traditional Chinese medicine prescriptions.

[0046] based on Figure 2 As can be seen from the Chinese herbal medicine prescription generation method shown, this embodiment of the application constructs derived samples with the same input but different prescription order by randomly rearranging the sequence of Chinese herbal medicine names in the original samples of the training set, thereby guiding the model to learn the combination relationship of Chinese herbal medicines more fully; and by introducing a joint optimization framework, the loss function of the sequence generation branch and the loss function of the set prediction branch are fused in the model training, which improves the prediction accuracy of the overall set of Chinese herbal medicine prescriptions while ensuring that the prescription sequence generation form conforms to the distribution of training data, and effectively alleviates the problem of insufficient learning of low-frequency Chinese herbal medicines caused by the high-frequency Chinese herbal medicines dominating the training process.

[0047] In some embodiments, the backbone network is used to encode the features of the input clinical text and output hidden features. The backbone network includes an input layer, an embedding layer, and multiple hidden layers. At least some of the linear transformation layers in the multiple hidden layers are provided with a LoRA (Low-Rank Adaptation) module. Here, the LoRA module refers to injecting two low-rank matrices into the linear transformation layer. These two low-rank matrices are multiplied by matrix multiplication and then added to the original weight matrix to achieve fine-tuning of the model.

[0048] Accordingly, the set prediction branch in step S3 is connected to the last layer of the multi-layer hidden layer and is used to perform set prediction on the semantic representation of clinical text features.

[0049] In some embodiments, the ensemble prediction branch in step S3 is configured to perform the following steps:

[0050] Weighted average pooling is applied to the semantic representation output of the last layer of the multi-layer hidden layers;

[0051] A multi-label prediction auxiliary output head is used to linearly map the semantic representation after weighted average pooling, and outputs a set prediction score vector. The set prediction score vector is used to characterize the independent prediction score of whether each Chinese herbal medicine should appear.

[0052] The independent prediction scores are mapped to the probability distribution of each Chinese herbal medicine using the Sigmoid function to output multi-label prediction results.

[0053] In some embodiments, the loss function for the sequence generation branch in step S3 is as follows:

[0054]

[0055] in, The loss function for generating branches is defined by B, where B is the number of samples in the batch, and T is the length of the traditional Chinese medicine prescription sequence in the input samples. For the clinical text features of the b-th sample, Let b be the sequence of target tokens preceding position t in the b-th sample. Let P( be the token of the t-th traditional Chinese medicine in the b-th sample) The model, given a case input and a generated / given prefix, generates a target token. The predicted probability.

[0056] In natural language processing, a token refers to a basic semantic unit after a text sequence has been segmented. For the clinical text features and traditional Chinese medicine prescription sequences processed in this embodiment, a token can correspond to a word or character fragment with complete semantics. For example, after segmentation, the clinical text features are split into several tokens, which serve as the input sequence for the model; each herb name in the traditional Chinese medicine prescription label is also represented as one or more tokens, constituting the target sequence that the model needs to generate.

[0057] The loss function for the ensemble prediction branch in step S3 is as follows:

[0058]

[0059] in, The loss function for predicting branches of the set. A value of 1 indicates that the b-th herb in the n-th dimension of the prescription set label appears in the herbal prescription label, while a value of 0 indicates that it does not appear in the herbal prescription label. The set prediction score vector logits is output by the set prediction auxiliary head. This is the Sigmoid function.

[0060] In some embodiments, step S3, on the enhanced training set, involves jointly training the loss function of the weighted fusion sequence generation branch and the loss function of the ensemble prediction branch, including:

[0061] While freezing the training parameters of the backbone network, update the parameters of the LoRA module and the parameters of the ensemble prediction branch.

[0062] In some embodiments, step S4 includes:

[0063] The target clinical case data is processed to obtain the target clinical text features;

[0064] The target clinical text features are input into the trained multi-task joint optimization prescription generation model, and the traditional Chinese medicine prescription prediction result is obtained based on the output of the sequence generation branch.

[0065] Next, the traditional Chinese medicine prescription generation scheme based on multi-task joint optimization of the above embodiments of this application will be described in detail.

[0066] First, preprocessing of historical clinical case data.

[0067] Historical clinical case data were anonymized, denoised, and structured to form a standard dataset suitable for model fine-tuning. From this dataset, the inputs required for model fine-tuning and labeled elements for supervising model training were constructed. Examples of data elements are shown in Table 1.

[0068]

[0069] Table 1

[0070] After obtaining the standard dataset, it is divided into a training set, a validation set, and a test set. Data augmentation is performed on the training set to alleviate overfitting of the sequence generation model to the order of Chinese medicine herbs.

[0071] In this example, the enhancement factor k=3 is set, and the original training samples are assumed to be... Where X represents clinical text features. This is for labeling traditional Chinese medicine prescriptions. During the data augmentation stage, keeping X unchanged, the output sequence S undergoes three random rearrangements (Shuffle) to generate an augmented training set. The process is as follows:

[0072] (1)

[0073] in, Let represent the sequence of traditional Chinese medicine prescriptions after the i-th random permutation. This is achieved by introducing an enhanced training set. This makes the model more inclined to learn the combination rules of prescription Chinese medicines during training, reducing its dependence on a fixed order.

[0074] After obtaining the enhanced training set through the above steps, the multi-task joint optimization prescription generation model is fine-tuned.

[0075] The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone network. Efficient parameter fine-tuning is achieved by injecting a low-rank adaptation (LoRA) module as a bypass into the linear mapping layer of the backbone network. The backbone network encodes the input text features, which are then passed sequentially through the input layer, embedding layer, and the aforementioned multiple hidden layers infused with LoRA modules to obtain the hidden state output.

[0076] The model has two training branches at the output end. The first is the sequence generation main branch. This branch reuses the language model output head of the large language model, uses an autoregressive approach to generate sequences of Chinese herbal medicine names, and optimizes the difference between the model's output distribution and the Chinese herbal medicine prescription labels using cross-entropy loss. This guides the model to generate prescription sequences that conform to the Chinese herbal medicine name representations in the training data, improving the quality of sequence generation. The loss function for this branch is... The standard cross-entropy loss is used, as shown in the following formula (2):

[0077] (2)

[0078] In formula (2), The loss function for generating branches is defined by B, where B is the number of samples in a batch (i.e., the number of samples input for training in one session), and T is the length of the traditional Chinese medicine prescription sequence in the input samples. For the clinical text features of the b-th sample, Let b be the sequence of target tokens preceding position t in the b-th sample. Let P( be the token of the t-th traditional Chinese medicine in the b-th sample) The model, given a case input and a generated / given prefix, generates a target token. The predicted probability.

[0079] Second is the ensemble prediction branch. This branch connects to the hidden state sequence of the last layer of the backbone network, using the token-level semantic representation as the input feature source for the branch, denoted as... Its dimensions are [B, T, H], where H is the dimension of the hidden vector. The ensemble prediction branch first... Weighted average pooling is performed. Through this feature aggregation method, the original scattered word-level features with dimensions [B,T,H] are smoothly reduced to overall case-level features of dimensions [B,H], thus facilitating the overall judgment of "which Chinese medicines are included in the prescription". Subsequently, a multi-label prediction auxiliary output head is set on the basis of weighted average pooling. The multi-label prediction auxiliary output head is preferably a linear mapping layer Linear(H,N), where N is the dimension of the global Chinese medicine dictionary, and the output set prediction score vector is used. The dimension is [B, N], and the independent prediction score logit is used to characterize whether each Chinese herbal medicine should appear. Then, the prediction score is mapped to the probability distribution of the appearance of each Chinese herbal medicine through the Sigmoid function, thereby forming a multi-label prediction output.

[0080] After obtaining the multi-label prediction output, based on Supervised learning is performed based on the difference between the probability distribution and the prescription set label y; the prescription set label y has dimensions [B, N], with elements ranging from 0 to 1, representing whether the corresponding Chinese medicine appears in the corresponding historical prescription. When calculating the difference between the output of the set prediction branch and the prescription set label y, a weighted binary cross-entropy form can be used to calculate the set prediction loss. And based on sample weight labels In the positive sample items of the ensemble prediction loss, the labels corresponding to low-frequency traditional Chinese medicines are given higher weights, thereby enhancing the learning signal of long-tail low-frequency traditional Chinese medicines and reducing the probability of missed detection. The ensemble prediction loss function is shown in the following formula (3):

[0081] (3)

[0082] in, The loss function for predicting branches of the set. A value of 1 indicates that the b-th herb in the n-th dimension of the prescription set label appears in the herbal prescription label, while a value of 0 indicates that it does not appear in the herbal prescription label. The set prediction score vector logits is output by the set prediction auxiliary head. This is the Sigmoid function, used to map the logit to the probability of existence between 0 and 1.

[0083] The model's joint optimization objective function is a weighted fusion of two loss components, with the sequence generation branch calculating the sequence generation loss. The ensemble prediction branch calculates the ensemble prediction loss. The total loss is obtained by fusing the results according to the weighting coefficient λ. ,in The balancing coefficient is, for example, 0.45 in this embodiment.

[0084] Joint training is performed based on the aforementioned enhanced training set, model structure, and loss function. In each iteration, the large language model backbone network, which shares sample inputs, calculates the cross-entropy loss of the sequence generation branch and the weighted BCE loss of the set prediction branch, respectively. Then, backpropagation is performed based on the joint optimization objective function, using the original pre-trained parameters of the large language model. While keeping the system completely frozen, update the parameters of the low-rank adaptation (LoRA) module injected into the large language model. And the ensemble prediction auxiliary head parameter Haux.

[0085] For example, the joint training process is shown in the algorithm below, where the training set Daug is augmented, the large language model backbone network M is pre-trained, and the original parameters are frozen. Update learnable parameters With the ensemble prediction auxiliary head parameter Haux, The updated LoRA parameters, Haux* is the updated... .

[0086] Method Joint_Optimization_Training:

[0087] Initialize LoRA parameters and BCE Auxiliary Head H aux

[0088] Load W pos from POS_WEIGHT_PATH (Generated in Step 1)

[0089] For each epoch do

[0090] Foreach batch (X, S seq, y multi ) in D aug do

[0091] Last Hidden Logits_Seq = M(X, S) seq, θ, )

[0092] Loss seq = CrossEntropy(Logits seq , S seq )

[0093] Poolstate = MeanPool(Last_Hidden)

[0094] Logits set = H aux (Pool_State)

[0095] Loss set = WeightedBCE(Logits_Set, y multi W pos )

[0096] Total Loss = Loss seq + λ * Loss set

[0097] Backprop(Total Loss )

[0098] Update( H aux )

[0099] End

[0100] End

[0101] return ,Haux*

[0102] Next, the performance of the model in the embodiments of this application will be verified based on experimental simulation.

[0103] To eliminate the impact of hardware environment differences on model performance evaluation results, all experiments were conducted under uniform hardware conditions. The experimental platform was configured with eight NVIDIA A100 GPUs (80GB VRAM per card). This experiment was based on the DeepSeek-R1-Distill-Qwen-7B model with fine-tuning. To verify the performance improvement of the fine-tuned model in the traditional Chinese medicine prescription generation task, this paper compared and evaluated the fine-tuned model with several mainstream open-source large models on the augmented training set. The comparison of the traditional Chinese medicine prescription generation results is shown in Table 2.

[0104]

[0105] Table 2

[0106] Regarding the improvement of long-tail distribution, compared with traditional schemes that do not employ data augmentation and joint optimization frameworks, the baseline model directly fine-tuned in this application embodiment shows a significant advantage in generating low-frequency traditional Chinese medicines (TCMs). The generation proportion of extremely low-frequency TCMs (occurring once) increased by 9.6 percentage points, and the generation proportion of low-frequency TCMs (occurring no more than 10 times) increased by 21.2 percentage points. Simultaneously, the absolute value of the slope of the long-tail distribution curve decreased from 2.47 to 1.98, indicating that the TCM generation frequency distribution changed from steep to gentle, and the generation results provided a more balanced coverage of long-tail drugs. These results demonstrate that the scheme in this application embodiment, while maintaining the quality of prescription generation, helps improve the coverage of low- and medium-frequency drugs in TCM prescription recommendations.

[0107] Figure 3 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 3 At the hardware level, the device includes a processor 302, an internal bus 304, a network interface 306, memory 308, a hardware acceleration device 310, and non-volatile memory 312, and may also include other hardware required for its functions. One or more embodiments of this application can be implemented in software, for example, the processor 302 reads the corresponding computer program from the non-volatile memory 312 into memory 308 and then runs it. Of course, in addition to software implementation, one or more embodiments of this application do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0108] Figure 4 This is a structural block diagram illustrating an exemplary embodiment of a traditional Chinese medicine prescription generation device based on multi-task joint optimization. The traditional Chinese medicine prescription generation device can be applied to, for example... Figure 3 The electronic device shown implements the technical solution of this application. The traditional Chinese medicine prescription generation device includes: a preprocessing unit 410, an enhancement processing unit 420, a model training unit 430, and a model inference unit 440, wherein:

[0109] The preprocessing unit 410 is used to construct a training set based on historical clinical case data. The training set includes multiple samples and sample weight labels. Each sample corresponds to a historical case in the historical clinical case data and includes clinical text features, traditional Chinese medicine prescription labels, and prescription set labels. The traditional Chinese medicine prescription labels indicate the names of traditional Chinese medicines in the prescription corresponding to the historical case. The prescription set labels are multi-label vectors generated based on a global traditional Chinese medicine dictionary and are used to indicate the types of traditional Chinese medicines included in the prescription. The sample weight labels are calculated based on the frequency of each traditional Chinese medicine appearing in the corresponding prescription in the historical clinical case data.

[0110] The enhancement processing unit 420 is used to, for at least a portion of the samples in the training set, keep the clinical text features and prescription set labels of each sample unchanged, randomly rearrange the sequence of Chinese medicine names in the Chinese medicine prescription labels multiple times, form derived samples by combining the Chinese medicine prescription labels corresponding to the rearranged sequences with the clinical text features and prescription set labels, and form an enhanced training set based on the original samples and the derived samples.

[0111] Model training unit 430 is used to train a multi-task joint optimization prescription generation model based on the enhanced training set. The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone and is configured with a sequence generation branch and an ensemble prediction branch. The sequence generation branch is used to predict the sequence of traditional Chinese medicine prescriptions in an autoregressive manner, and its supervision signal is the traditional Chinese medicine prescription label. The ensemble prediction branch is used to predict whether each traditional Chinese medicine in the global traditional Chinese medicine dictionary appears based on the semantic representation of clinical text features, and its supervision signal is the prescription ensemble label. Sample weight labels are introduced into its loss function to enhance attention to low-frequency traditional Chinese medicines. On the enhanced training set, joint training is performed by weighted fusion of the loss functions of the sequence generation branch and the ensemble prediction branch.

[0112] The model inference unit 440 is used to generate traditional Chinese medicine prescriptions using the trained multi-task joint optimization prescription generation model.

[0113] In some embodiments, the backbone network is used to encode the input clinical text features and output hidden features. The backbone network includes an input layer, an embedding layer, and multiple hidden layers. At least some of the linear transformation layers in the multiple hidden layers are provided with LoRA modules. The ensemble prediction branch is connected to the last layer of the multiple hidden layers and is used to perform ensemble prediction on the semantic representation of the clinical text features.

[0114] In some embodiments, the ensemble prediction branch is configured to perform the following steps:

[0115] Weighted average pooling is applied to the semantic representation output of the last layer of the multi-layer hidden layers;

[0116] A multi-label prediction auxiliary output head is used to linearly map the semantic representation after weighted average pooling, and outputs a set prediction score vector. The set prediction score vector is used to characterize the independent prediction score of whether each Chinese herbal medicine should appear.

[0117] The independent prediction scores are mapped to the probability distribution of each Chinese herbal medicine using the Sigmoid function to output multi-label prediction results.

[0118] In some embodiments, the model training unit 430 is configured to update the parameters of the LoRA module and the parameters of the ensemble prediction branch while freezing the training parameters of the backbone network.

[0119] In some embodiments, the model inference unit 440 is used to process target clinical case data to obtain target clinical text features; input the target clinical text features into the trained multi-task joint optimization prescription generation model, and obtain the traditional Chinese medicine prescription prediction result based on the output of the sequence generation branch.

[0120] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0121] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0122] Accordingly, embodiments of this application also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0123] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0124] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0126] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0127] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0128] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0129] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0130] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for generating traditional Chinese medicine prescriptions based on multi-task joint optimization, characterized in that, Includes the following steps: Step S1: Construct a training set based on historical clinical case data. The training set includes multiple samples and sample weight labels. Each sample corresponds to a historical case in the historical clinical case data and includes clinical text features, traditional Chinese medicine prescription labels, and prescription set labels. The traditional Chinese medicine prescription labels indicate the names of the traditional Chinese medicines in the prescription corresponding to the historical case. The prescription set labels are multi-label vectors generated based on a global traditional Chinese medicine dictionary, used to indicate the types of traditional Chinese medicines included in the prescription. The sample weight labels are calculated based on the frequency of each traditional Chinese medicine appearing in the corresponding prescriptions of the historical clinical case data. Step S2: For at least a portion of the samples in the training set, keep the clinical text features and prescription set labels of each sample unchanged, randomly rearrange the sequence of Chinese medicine names in the Chinese medicine prescription labels multiple times, and form derived samples by combining the Chinese medicine prescription labels, clinical text features and prescription set labels corresponding to the rearranged sequences, and form an enhanced training set based on the original samples and the derived samples. Step S3: Train a multi-task joint optimization prescription generation model based on the enhanced training set. The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone and is configured with a sequence generation branch and an ensemble prediction branch. The sequence generation branch is used to predict the sequence of traditional Chinese medicine prescriptions in an autoregressive manner, and its supervision signal is the traditional Chinese medicine prescription label. The ensemble prediction branch is used to predict the presence or absence of each traditional Chinese medicine in the global traditional Chinese medicine dictionary based on the semantic representation of clinical text features, and its supervision signal is the prescription ensemble label. Sample weight labels are introduced into its loss function to enhance attention to low-frequency traditional Chinese medicines. On the enhanced training set, the model is jointly trained by weighted fusion of the loss functions of the sequence generation branch and the ensemble prediction branch. Step S4: Use the trained multi-task joint optimization prescription generation model to generate traditional Chinese medicine prescriptions.

2. The method according to claim 1, characterized in that, The backbone network is used to encode the features of the input clinical text and output hidden features. The backbone network includes an input layer, an embedding layer and multiple hidden layers. At least some of the linear transformation layers in the multiple hidden layers are provided with LoRA modules. The set prediction branch in step S3 is connected to the last layer of the multi-layer hidden layer and is used to perform set prediction on the semantic representation of clinical text features.

3. The method according to claim 2, characterized in that, The ensemble prediction branch in step S3 is configured to perform the following steps: Weighted average pooling is applied to the semantic representation output of the last layer of the multi-layer hidden layers; A multi-label prediction auxiliary output head is used to linearly map the semantic representation after weighted average pooling, and outputs a set prediction score vector. The set prediction score vector is used to characterize the independent prediction score of whether each Chinese herbal medicine should appear. The independent prediction scores are mapped to the probability distribution of each Chinese herbal medicine using the Sigmoid function to output multi-label prediction results.

4. The method according to claim 1, characterized in that, The loss function for the sequence generation branch in step S3 is as follows: ,in, The loss function for generating branches is defined by B, where B is the number of samples in the batch, and T is the length of the traditional Chinese medicine prescription sequence in the input samples. For the clinical text features of the b-th sample, Let b be the sequence of target tokens preceding position t in the b-th sample. Let P( be the token of the t-th traditional Chinese medicine in the b-th sample) The model, given a case input and a generated / given prefix, generates a target token. The predicted probability.

5. The method according to claim 4, characterized in that, The loss function for the ensemble prediction branch in step S3 is as follows: ,in, The loss function for predicting branches of the set. A value of 1 indicates that the b-th herb in the n-th dimension of the prescription set label appears in the herbal prescription label, while a value of 0 indicates that it does not appear in the herbal prescription label. The set prediction score vector logits is output by the set prediction auxiliary head. This is the Sigmoid function.

6. The method according to claim 2, characterized in that, Step S3, on the enhanced training set, involves jointly training the loss function of the weighted fusion sequence generation branch and the loss function of the ensemble prediction branch, including: While freezing the training parameters of the backbone network, update the parameters of the LoRA module and the parameters of the ensemble prediction branch.

7. The method according to claim 1, characterized in that, Step S4 includes: The target clinical case data is processed to obtain the target clinical text features; The target clinical text features are input into the trained multi-task joint optimization prescription generation model, and the traditional Chinese medicine prescription prediction result is obtained based on the output of the sequence generation branch.

8. A traditional Chinese medicine prescription generation device based on multi-task joint optimization, characterized in that, include: The preprocessing unit is used to construct a training set based on historical clinical case data. The training set includes multiple samples and sample weight labels. Each sample corresponds to a historical case in the historical clinical case data and includes clinical text features, traditional Chinese medicine prescription labels, and prescription set labels. The traditional Chinese medicine prescription labels indicate the names of the traditional Chinese medicines in the prescription corresponding to the historical case. The prescription set labels are multi-label vectors generated based on a global traditional Chinese medicine dictionary, used to indicate the types of traditional Chinese medicines included in the prescription. The sample weight labels are calculated based on the frequency of each traditional Chinese medicine appearing in the corresponding prescriptions of the historical clinical case data. An enhancement processing unit is used to, for at least a portion of the samples in the training set, keep the clinical text features and prescription set labels of each sample unchanged, randomly rearrange the sequence of Chinese medicine names in the Chinese medicine prescription labels multiple times, form derived samples by combining the Chinese medicine prescription labels, clinical text features and prescription set labels corresponding to the rearranged sequences, and form an enhanced training set based on the original samples and the derived samples. The model training unit is used to train a multi-task joint optimization prescription generation model based on the enhanced training set. The multi-task joint optimization prescription generation model uses a pre-trained large language model as a shared backbone and is configured with a sequence generation branch and an ensemble prediction branch. The sequence generation branch is used to predict the sequence of traditional Chinese medicine prescriptions in an autoregressive manner, and its supervision signal is the traditional Chinese medicine prescription label. The ensemble prediction branch is used to predict the presence or absence of each traditional Chinese medicine in the global traditional Chinese medicine dictionary based on the semantic representation of clinical text features, and its supervision signal is the prescription ensemble label. Sample weight labels are introduced into its loss function to enhance attention to low-frequency traditional Chinese medicines. The model is jointly trained on the enhanced training set by weighted fusion of the loss functions of the sequence generation branch and the ensemble prediction branch. The model inference unit is used to generate traditional Chinese medicine prescriptions using the trained multi-task joint optimization prescription generation model.

9. An electronic device, characterized in that, include: processor; A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which is executed by a processor according to any one of claims 1 to 7.

Images