Method and system for optimizing traditional Chinese medicine gynecological treatment scheme based on reinforcement learning
Through the optimization method of traditional Chinese medicine gynecological treatment plan based on reinforcement learning, the problems of consistency, data utilization and dynamic adjustment in the formulation and optimization of traditional Chinese medicine gynecological treatment plan were solved, and multi-objective optimization and personalized treatment were achieved, which significantly improved the treatment effect and efficiency.
Patent Information
- Application Number
- CN202410990542.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
There is a difficult guarantee of consistency between diagnosis and prescriptions during the formulation and optimization of traditional Chinese medicine gynecological treatment plans, and it is impossible to effectively utilize a large amount of clinical data and latest research results. Dynamic adjustment of treatment plans lags behind changes in patients' condition, and it is difficult to take into account both symptom improvement, side effects control and cost-effectiveness. and other issues.
Using the Chinese medicine gynecological treatment plan optimization method based on reinforcement learning, pre-process by obtaining patient information, performing Chinese medicine syndrome differentiation, building a reinforcement learning environment, training a multi-objective reinforcement learning model, generating and optimizing treatment plans, and online through clinical feedback Update to form an optimization cycle.
Multi-objective optimization, personalized treatment, safety guarantee, dynamic optimization and long-term efficacy have been achieved, which has improved the effectiveness and efficiency of the treatment plan, reduced the risk of side effects, and improved the quality of diagnosis and treatment and resource utilization efficiency.
Smart Images

Figure CN118942637A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of TCM gynecological treatment plan optimization method, and in particular to a TCM gynecological treatment plan optimization method based on reinforcement learning and a system thereof. Background Art
[0002] TCM gynecology is an important branch of TCM and plays an important role in the diagnosis and treatment of women's diseases. With the development of modern science and technology, TCM gynecology diagnosis and treatment methods are also constantly innovating and improving. However, the current TCM gynecology treatment plan formulation and optimization process still faces many challenges.
[0003] Traditional Chinese medicine gynecological diagnosis and treatment methods mainly rely on the doctor's personal experience and knowledge accumulation. Although this method can achieve personalized treatment to a certain extent, it also has the following problems:
[0004] 1. The consistency of diagnosis and prescription is difficult to ensure, and different doctors may give different diagnoses and treatment plans for the same patient.
[0005] 2. It is difficult to effectively utilize large amounts of clinical data and the latest research results.
[0006] 3. Dynamic adjustments to treatment plans often lag behind changes in the patient’s condition.
[0007] 4. It is difficult to simultaneously take into account multiple goals such as symptom improvement, side effect control and cost-effectiveness.
[0008] In order to improve these problems, some rule-based expert systems have been introduced into TCM gynecological diagnosis and treatment. Such systems provide standardized diagnosis and prescription recommendations by encoding expert knowledge. However, this approach also has limitations:
[0009] 1. The system’s knowledge base is difficult to update in a timely manner and cannot adapt to the rapid development of medical knowledge.
[0010] 2. Rules are often too rigid to handle complex clinical situations and individual differences.
[0011] 3. Lack of ability to learn and self-optimize, and unable to continuously improve from clinical practice.
[0012] In recent years, with the development of artificial intelligence technology, some studies have begun to try to apply machine learning methods to TCM gynecological diagnosis and treatment. Among them, the single-objective reinforcement learning method has achieved certain results in certain specific problems. However, this method still has obvious shortcomings:
[0013] 1. It is difficult to balance multiple treatment goals, and patients often focus too much on a single indicator and ignore other important factors.
[0014] 2. Lack of effective integration of TCM theory and safety standards may result in treatment plans that are inconsistent with TCM theory or pose safety risks.
[0015] 3. Insufficient consideration is given to long-term efficacy, and the focus is mainly on short-term treatment effects.
[0016] 4. It is difficult to effectively deal with the uncertainty and ambiguity of TCM diagnostic information.
[0017] In view of these problems with existing technologies, there is an urgent need for an optimization method for TCM gynecological treatment plans that can comprehensively consider multiple treatment goals, balance safety and effectiveness, and can dynamically optimize and adapt to individual differences. Summary of the invention
[0018] The purpose of the present invention is to provide a method and system for optimizing TCM gynecological treatment plans based on reinforcement learning, aiming to solve the problems of insufficient personalization, difficulty in optimizing long-term efficacy, and complex multi-objective balance in traditional TCM gynecological treatment.
[0019] The present invention provides a method for optimizing a TCM gynecological treatment plan based on reinforcement learning, comprising the following steps:
[0020] Obtain patient information and perform preprocessing to obtain the patient's initial state vector;
[0021] Performing TCM syndrome differentiation based on the patient's initial state vector;
[0022] Initialize the treatment plan according to the TCM syndrome differentiation results;
[0023] Build a reinforcement learning environment, including defining the state space, action space, and reward function;
[0024] Training multi-objective reinforcement learning models;
[0025] Generate and optimize a treatment plan based on the multi-objective reinforcement learning model;
[0026] Collect clinical feedback information;
[0027] Online updating of the multi-objective reinforcement learning model based on the clinical feedback information;
[0028] Evaluate the effectiveness of treatment;
[0029] The multi-objective reinforcement learning model is adjusted according to the treatment effect evaluation result, and the process returns to the step of generating and optimizing the treatment plan based on the multi-objective reinforcement learning model to form an optimization loop.
[0030] Specifically, the steps of obtaining patient information and performing preprocessing include:
[0031] Collect basic patient information I b、Symptom Description I s Tongue I t 、Pulse I p and related inspection resultsI e ;
[0032] Digital processing of collected information: I d =f d (I b ,I s ,I t ,I p ,I e ), where f d is the digital function;
[0033] To standardize: Where μ and σ are the mean and standard deviation of each feature respectively;
[0034] Form the patient's initial state vector: S0 = [I n ; T0], where T0 is the initial timestamp.
[0035] Specifically, the step of performing TCM syndrome differentiation based on the patient's initial state vector includes:
[0036] Construct an improved fuzzy decision tree model IFDT={N,E,A,R}, where N is the node set, E is the edge set, A is the attribute set, and R is the fuzzy rule set;
[0037] Define the fuzzy membership function: Where c is the center value, a and b are adjustable parameters;
[0038] The information gain ratio criterion is introduced to select the optimal split attribute: Where Gain(A) is the information gain, SplitInfoA is the split information;
[0039] The patients were classified according to the IFDT model: T = IFDT (S0), where T was the result of the classification.
[0040] Specifically, the step of initializing the treatment plan according to the results of TCM syndrome differentiation includes:
[0041] Construct a TCM knowledge graph KG = {V, E, R}, where V is the entity set, E is the relationship between entities, and R is the relationship type;
[0042] Treatment plan retrieval based on knowledge graph: P0 = argmax P Sim(KG(T),P), where P0 is the initial treatment plan, Sim is the similarity function, and KG(T) is the subgraph corresponding to T.
[0043] Specifically, the steps of constructing a reinforcement learning environment include:
[0044] Define the state space S = {s|s = [I n ; T; P]}, where I n is the patient's characteristics, T is the time information, and P is the current treatment plan;
[0045] Define the action space A = {a|a = [ΔH; ΔM; ΔA]}, where ΔH, ΔM, and ΔA represent the adjustments to Chinese medicine, acupuncture, and other therapies, respectively;
[0046] Define the reward function:
[0047] R(sa,s′)=w1×ΔSymptom+w2×(-SideEffect)+w3×(-Cost), where ΔSymptom is the degree of symptom improvement, SideEffect is the degree of side effects, Cost is the cost of treatment, and w i is the weight.
[0048] Specifically, the step of training the multi-objective reinforcement learning model includes:
[0049] Constructing an improved multi-objective deep Q-network:
[0050] Where V(s;θ v ) is the state value function, A(s,a;θ a ) is the advantage function, θ={θ v ,θ a} is the network parameter;
[0051] Apply dynamic weight adjustment mechanism:
[0052] Where α is the learning rate, R i (t) is the current reward, is the historical average reward, H(w) is the weight entropy regularization term, and β is the regularization coefficient;
[0053] Introducing the hierarchical attention mechanism:
[0054] h l =Attention(W l ×h l-1 +b l ),Attention(x)=softmax(v T ×tanh(W×x+b))×x,s=Σ l α l ×h l , where h l is the hidden state of the lth layer, Wl 、b l , v is a learnable parameter, α l is the attention weight;
[0055] Application knowledge distillation and experience replay:
[0056] L = (1-λ) × L DQN +λ×L KD +γ×L PER , where L DQN is the DQN loss, L K D is the knowledge distillation loss, L P ER is the priority experience replay loss, λ and γ are balance factors;
[0057] Implement security policy constraints:
[0058] Q′(s,a)=Q(s,a)×S(s,a),S(s,a)=min(μ1(s,a),μ2(s,a),…,μ k (s,a)), where S(s,a) is the security metric function, μ i is the fuzzy membership function of the i-th security rule.
[0059] Specifically, the steps of generating and optimizing a treatment plan based on a multi-objective reinforcement learning model include:
[0060] Generate candidate action set A c ={a|Q′(s,a;θ)>threshold};
[0061] Apply security policy constraints to filter security action set A s ={a∈A c |S(s,a)>safety_threshold};
[0062] Choose the best action
[0063] Generate optimized treatment plan P t =UpdatePlan(P t-1 , a*), where the UpdatePlan function updates the treatment plan P at the previous moment according to the selected action a* t-1 .
[0064] Specifically, the step of online updating the multi-objective reinforcement learning model based on clinical feedback information includes:
[0065] Construct incremental learning module ILM = {θ old , D new ,η}, where θ old is the old model parameter, Dnew is the newly collected data, η is the incremental learning rate;
[0066] Define the knowledge preservation loss L retain =||θ new -θ old || 2 ; Define new knowledge learning loss L new =L IMODQN (D new θ new ), where L IMODQN is the loss function of the multi-objective reinforcement learning model;
[0067] Constructing the comprehensive loss function L total =(1-η)×L retain +η×L new ; Execute the model update algorithm: initialize θ new =θ old ; For each update cycle: From D new Sampling small batches of data; calculating L total ; Update θ using gradient descent new ; If convergence or the maximum number of iterations is reached, the update ends; update model parameters θ = θ new .
[0068] Specifically, the step of evaluating the treatment effect includes:
[0069] Define comprehensive evaluation indicators:
[0070] CE=w s ×f symptom +w e ×(1-f sideeffect )+w t ×f satisfaction +w c ×(1-f cost )
[0071] where w s , w e , w t , w c is the weight of each indicator, f symptom Indicates the degree of symptom improvement, f sideeffect Indicates the degree of side effects, f satisfaction represents patient satisfaction, f cost represents the cost of treatment;
[0072] Introducing fuzzy comprehensive evaluation method: Where W is the weight vector, U i is a single factor evaluation matrix, is the fuzzy composition operator, It is a fuzzy addition operation;
[0073] Calculate the final evaluation result: E = Defuzzify(R), where Defuzzify is a defuzzification function that converts the fuzzy evaluation result into a clear numerical value.
[0074] The TCM gynecological treatment plan optimization system based on reinforcement learning that implements the method described above comprises:
[0075] The patient information acquisition and preprocessing module is used to obtain patient information and preprocess it to obtain the patient's initial state vector;
[0076] A TCM syndrome differentiation module, used for performing TCM syndrome differentiation based on the patient's initial state vector;
[0077] A treatment plan initialization module, used to initialize the treatment plan according to the TCM syndrome differentiation results;
[0078] Reinforcement learning environment building module, used to build reinforcement learning environment, including defining state space, action space and reward function;
[0079] Multi-objective reinforcement learning model training module, used to train multi-objective reinforcement learning models;
[0080] A treatment plan generation and optimization module, used to generate and optimize a treatment plan based on the multi-objective reinforcement learning model;
[0081] A clinical feedback collection module, used to collect clinical feedback information;
[0082] A model online updating module, used for updating the multi-objective reinforcement learning model online based on the clinical feedback information;
[0083] Treatment effect evaluation module, used to evaluate treatment effect;
[0084] The optimization loop control module is used to adjust the multi-objective reinforcement learning model according to the treatment effect evaluation result, and control the system to return to the treatment plan generation and optimization module to form an optimization loop.
[0085] The present invention has the following beneficial effects:
[0086] 1. Multi-objective optimization: Through a multi-objective reinforcement learning model, multiple objectives such as symptom improvement, side effect control and cost-effectiveness are considered simultaneously to achieve comprehensive optimization of the treatment plan.
[0087] 2. Personalized treatment: Combining the individual characteristics and dynamic feedback of patients, highly personalized treatment plan formulation and adjustment are achieved.
[0088] 3. Safety assurance: Introduce a safety policy constraint mechanism to ensure that the generated treatment plan complies with traditional Chinese medicine theory and safety standards, significantly reducing the risk of side effects.
[0089] 4. Dynamic optimization: By collecting clinical feedback in real time and updating it online, dynamic optimization of treatment plans is achieved, which can adapt to changes in the patient's condition in a timely manner.
[0090] 5. Long-term efficacy: By considering long-term rewards and introducing knowledge distillation mechanisms, the long-term effectiveness of treatment options is improved.
[0091] 6. Improve efficiency: The use of artificial intelligence technology greatly shortens the time for generating personalized plans and improves the efficiency of clinical diagnosis and treatment.
[0092] 7. Knowledge accumulation: Through continuous learning and optimization, the system can continuously accumulate and update knowledge and improve the level of diagnosis and treatment.
[0093] 8. Cost-effectiveness: While improving the treatment effect, it effectively controls the treatment cost and improves the utilization efficiency of medical resources.
[0094] In summary, the present invention provides an innovative method for optimizing TCM gynecological treatment plans by integrating TCM theory and modern artificial intelligence technology, which is expected to significantly improve the quality and efficiency of TCM gynecological diagnosis and treatment, and provide patients with more accurate, safer and more effective personalized treatment services. BRIEF DESCRIPTION OF THE DRAWINGS
[0095] Figure 1 This is a reinforcement learning-based TCM gynecological treatment plan optimization method and its system flow chart.
[0096] Figure 2 This is a schematic diagram of the framework of the urinary stone laser lithotripsy parameter optimization system based on deep reinforcement learning of the present invention. DETAILED DESCRIPTION
[0097] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the TCM gynecological treatment plan optimization method and system based on reinforcement learning proposed by the present invention, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0098] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0099] The present invention provides the following technical solution: a method for optimizing a TCM gynecological treatment plan based on reinforcement learning, comprising the following steps:
[0100] 1. Obtain patient information and preprocess it to obtain the patient's initial state vector. In a TCM gynecology clinic, when a patient comes to see a doctor, the doctor will first collect various information about the patient. This information includes the patient's basic information I b (such as age, occupation, etc.), symptom description I s (such as abdominal pain, irregular menstruation, etc.), tongue image I t (such as tongue quality, tongue coating, etc.), pulse condition I p (such as floating pulse, sinking pulse, etc.) and related examination results I e (such as blood routine, B-ultrasound, etc.) Next, the system will digitize this information:
[0101] I d =f d (I b , I s , I t , I p , I e )
[0102] Among them, f d is a digitization function that converts different types of information into numerical vectors. For example, for symptom descriptions, natural language processing techniques can be used to convert text into vectors; for tongue and pulse images, predefined numerical mapping tables can be used for conversion, and then the system will standardize the digitized data:
[0103]
[0104] Among them, μ and σ are the mean and standard deviation of each feature, respectively. This step ensures that different types of data can be compared and analyzed on the same scale.
[0105] Finally, the system forms the patient's initial state vector:
[0106] S0=[I n ; T0]
[0107] Among them, T0 is the initial timestamp, which is used to record the time point of the patient's visit. This initial state vector S0 contains all the relevant information of the patient, providing a basis for subsequent analysis and decision-making.
[0108] 2. Perform TCM syndrome differentiation based on the patient's initial state vector. After obtaining the patient's initial state vector, the system will use the improved fuzzy decision tree model (IFDT) to perform TCM syndrome differentiation.
[0109] The IFDT model is defined as follows:
[0110] IFDT = {N, E, A, R}
[0111] Among them, N is a node set, representing different syndromes or symptoms; E is an edge set, representing the relationship between nodes; A is an attribute set, containing various features for classification; and r is a fuzzy rule set, used to deal with uncertainty in TCM diagnosis.
[0112] In IFDT, we define a fuzzy membership function:
[0113]
[0114] Among them, c is the central value, and a and b are adjustable parameters. This function is used to measure the degree to which a feature belongs to a certain syndrome. For example, for "spleen deficiency syndrome", multiple fuzzy membership functions may be set to evaluate the severity of different symptoms (such as fatigue, loss of appetite, etc.)
[0115] In order to select the optimal splitting attribute, we introduce the information gain ratio criterion:
[0116]
[0117] Among them, Gain(A) is the information gain, and SplitInfo(A) is the split information. This criterion helps us select the most discriminative attributes when building a decision tree, thereby improving the accuracy of classification.
[0118] Finally, we use the IFDT model to classify the patients:
[0119] T=IFDT(S0)
[0120] T is the result of syndrome differentiation. For example, the system may classify the patient into TCM syndromes such as "liver depression and qi stagnation syndrome" and "kidney yang deficiency syndrome".
[0121] 3. Initialize the treatment plan based on the results of TCM syndrome differentiation. After obtaining the results of syndrome differentiation, the system will use the pre-built TCM knowledge graph KG to initialize the treatment plan:
[0122] KG={V,E,r}
[0123] Among them, V is the entity set, including various syndromes, symptoms, drugs, etc.; E is the relationship between entities; r is the relationship type, such as "treatment", "relief", etc.
[0124] The system will search for treatment options based on the knowledge graph:
[0125]
[0126] Among them, P0 is the initial treatment plan, Sim is the similarity function, and KG(T) is the subgraph corresponding to T. This step will find the known treatment plan that is most similar to the patient's syndrome as the initial plan.
[0127] 4. Construct a reinforcement learning environment In order to optimize the treatment plan, the system constructs a reinforcement learning environment. First, define the state space:
[0128] S={s|s=[I n ; T; P]}
[0129] Among them, I n is the patient feature, T is the time information, and P is the current treatment plan. This state space contains the patient’s current condition and the treatment plan being implemented.
[0130] Then define the action space:
[0131] A={a|a=[ΔH; ΔM; ΔA]}
[0132] Among them, ΔH, ΔM, and ΔA represent adjustments to Chinese medicine, acupuncture, and other therapies, respectively. For example, ΔH may include increasing or decreasing the dosage of a certain Chinese medicine, and ΔM may include changing acupuncture points or frequencies.
[0133] Finally, define the reward function:
[0134] R(sa, s′)=w1×ΔSymptom+w2×(-SideEffect)+w3×(-Cost)
[0135] Among them, ΔSymptom is the degree of symptom improvement, SideEffect is the degree of side effects, Cost is the cost of treatment, w1, w2,
[0136] w3 is the weight coefficient. This reward function takes into account the treatment effect, side effects, and costs to balance multiple treatment goals.
[0137] 5. Train the multi-objective reinforcement learning model. After building the reinforcement learning environment, the system starts to train the multi-objective reinforcement learning model. This model is an improved multi-objective deep Q network (IMODQN), and its structure is as follows:
[0138]
[0139] Among them, V(s; θ v ) is the state value function, A(s, a; θ a ) is the advantage function, θ={θ v ,θ a} are network parameters. This structure separates the state value and action advantage, which helps improve learning efficiency and stability.
[0140] In order to better adapt to the characteristics of TCM gynecological treatment, we have introduced the following innovative mechanisms:
[0141] a) Dynamic weight adjustment mechanism:
[0142]
[0143] Where α is the learning rate (usually set between 0.001 and 0.01), R i (t) is the current reward, is the historical average reward, H(w) is the weight entropy regularization term, and β is the regularization coefficient (usually set between 0.001 and 0.1). This mechanism can dynamically adjust the weights of different goals based on feedback during the treatment process. For example, in the early stages of treatment, more emphasis may be placed on symptom improvement, while in the later stages, more emphasis may be placed on side effect control.
[0144] b) Hierarchical Attention Mechanism:
[0145] h l =Attention(W l ×h l-1 +b l )
[0146] Attention(x) = softmax(v T ×tanh(W×x+b))×x
[0147]
[0148] Among them, h l is the hidden state of the lth layer, W l 、b l , v is a learnable parameter, α l is the attention weight. This mechanism can capture the hierarchy of TCM diagnostic information. For example, it can focus on different levels of information such as tongue image and pulse image, and weight them according to their importance.
[0149] c) Knowledge distillation and experience replay:
[0150] L = (1-λ) × L DQN +λ×L KD +γ×L PER
[0151] Among them, L DQN is the DQN loss, L KD is the knowledge distillation loss, L PERis the priority experience replay loss, λ and γ are balancing factors (usually λ is set between 0.1 and 0.5, and γ is set between 0.01 and 0.1). This mechanism combines expert knowledge and historical experience, which helps to accelerate the learning process and improve model performance.
[0152] d) Security policy constraints:
[0153] Q′(s,a)=Q(s,a)×S(s,a)
[0154] S(s,a)=min(μ1(s,a),μ2(s,a),…,μ k (s,a))
[0155] Among them, S(s,a) is the security metric function, μ i is the fuzzy membership function of the ith safety rule. This mechanism ensures that the treatment plan recommended by Al complies with TCM theory and safety standards. For example, rules can be set to avoid overdose or improper combination of certain drugs.
[0156] In the optimization of TCM gynecological treatment plans, the training of multi-objective reinforcement learning models is a core step.
[0157] The "multi-target" here specifically refers to the following key targets that are closely related to TCM gynecological treatment:
[0158] 1. Symptom improvement (ΔSymptom): including the degree of improvement of main symptoms (such as dysmenorrhea, irregular menstruation, abnormal vaginal discharge, etc.) and secondary symptoms (such as low back pain, fatigue, mood swings, etc.).
[0159] 2. Side Effect Control: Reduce the risk of adverse reactions (such as gastrointestinal reactions, rashes, etc.) that may occur during treatment.
[0160] 3. Treatment cost (Cost): Under the premise of ensuring the treatment effect, control the treatment cost, including drug costs and the use of other medical resources.
[0161] 4. Improvement of tongue and pulse (Tongue Pulse): Changes in tongue and pulse indicators, which are important in TCM diagnosis.
[0162] 5. Patient satisfaction: the patient's subjective evaluation of the treatment process and effect.
[0163] There are often trade-offs and conflicts between these goals. For example, adding certain expensive Chinese medicines may improve the effect of symptom improvement, but at the same time increase the cost of treatment; or increasing the drug dosage may accelerate symptom relief, but may increase the risk of side effects. Therefore, it is necessary to balance these goals through multi-objective reinforcement learning.
[0164] In one specific example, consider a patient with dysmenorrhea and anxiety, whose initial treatment is Guizhi Fuling Wan plus acupuncture. During the training:
[0165] 1. Symptom improvement (ΔSymptom): The model learns that increasing the dosage of Angelica sinensis and White Peony Root can better relieve dysmenorrhea symptoms, while combining it with Bupleurum Root and Cyperus rotundus can improve anxiety.
[0166] 2. Side Effect Control: The system uses safety policy constraints to ensure that increasing the drug dosage does not cause significant gastrointestinal discomfort.
[0167] 3. Treatment cost (Cost): The model weighed the cost and effect of using expensive Chinese medicines (such as Panax notoginseng and Salvia miltiorrhiza), and ultimately selected a formula with a higher cost-effectiveness.
[0168] 4. Tongue and pulse improvement (TonguePulse): Through the hierarchical attention mechanism, the model pays special attention to the changes in tongue and pulse indicators related to dysmenorrhea, such as pale red tongue, thin white tongue coating, and stringy and fine pulse.
[0169] 5. Patient Satisfaction: Considering the poor compliance of patients with decoctions, the model recommends using Chinese medicine granules instead. Although the cost is slightly higher, it improves patient satisfaction.
[0170] During the training process, the dynamic weight adjustment mechanism enables the model to flexibly adjust the importance of each goal according to the treatment stage. For example, in the early stage of treatment, more emphasis may be placed on rapid relief of symptoms, while in the later stage, more attention may be paid to long-term efficacy and side effect control.
[0171] The knowledge distillation mechanism ensures that the model does not deviate from the theoretical basis of traditional Chinese medicine during the learning process. For example, when selecting drug combinations, the "monarch, minister, assistant and envoy" principle and the "seven emotions compatibility" theory of traditional Chinese medicine are taken into consideration.
[0172] Through this multi-objective optimization, the model can weigh the pros and cons of various aspects and develop the best personalized treatment plan for patients. This method not only takes into account the immediate symptom improvement, but also focuses on multiple aspects such as side effect control, cost-effectiveness and patient satisfaction, truly realizing the concept of "holistic concept" and "treatment based on syndrome differentiation" in traditional Chinese medicine.
[0173] Compared with the existing technology, the advantages of this method are:
[0174] 1. Optimize multiple conflicting treatment goals simultaneously rather than a single goal.
[0175] 2. Adapt to the needs of different treatment stages through dynamic weight adjustment.
[0176] 3. Combining TCM theoretical knowledge and clinical data, an organic combination of theory and practice is achieved.
[0177] 4. Safety constraints are taken into consideration to ensure that the generated plan complies with the principles of safe use of traditional Chinese medicine.
[0178] 5. Be able to deal with the uncertainty and ambiguity in TCM diagnosis, especially in the judgment of tongue and pulse.
[0179] This multi-objective, dynamic, and safe optimization method provides a new solution for the formulation of TCM gynecological treatment plans.
[0180] 6. Generate and optimize treatment plans based on the multi-objective reinforcement learning model. After training the model, the system uses the following steps to generate and optimize treatment plans:
[0181] a) Generate candidate action sets:
[0182] A c ={a|Q′(s,a;θ)>threshold}
[0183] Where threshold is a preset threshold, usually set to 80% to 90% of the current optimal Q value. b) Apply security policy constraints to filter the security action set:
[0184] A s ={a∈A c |S(s,a)>safety threshold}
[0185] Among them, safety_threshold is usually set between 0.8 and 0.95 to ensure that the selected action is safe enough.
[0186] c) Select the best action:
[0187]
[0188] d) Generate optimized treatment plan:
[0189] P t =UpdatePlan(P t -1, a*)
[0190] Among them, the UpdatePlan function updates the treatment plan P at the previous moment according to the selected action a* t-1 For example, if a* recommends increasing the dosage of a certain Chinese medicine, the UpdatePlan function will adjust the prescription accordingly.
[0191] To better understand the practical application of the present invention, we take a patient with polycystic ovary syndrome (PCOS) as an example. In this step, the system generates and optimizes the treatment plan according to the specific situation of the patient. For example, for our PCOS patient:
[0192] Initial state:
[0193] Main symptoms: oligomenorrhea, acne, hirsutism;
[0194] Tongue: pale and fat tongue with greasy white fur;
[0195] Pulse: deep and slippery;
[0196] The initial solution generated by the system may be:
[0197] 1. Chinese medicine prescription: Xiaoyao powder with modifications (Bupleurum 10g, Angelica 10g, White Peony 10g, Poria 15g, Atractylodes 10g, Paeonia suffruticosa 10g, Gardenia 10g);
[0198] 2. Acupuncture program: Guanyuan, Sanyinjiao, Taichong, 3 times a week;
[0199] 3. Lifestyle recommendations: control diet and increase exercise;
[0200] As treatment progresses, the system may be optimized as follows:
[0201] Add 6g of roasted licorice root to nourish the spleen and stomach and improve metabolism;
[0202] Add 10g peach kernel and 6g safflower to promote blood circulation and remove blood stasis, and improve ovarian microcirculation;
[0203] The acupuncture program adds the Zusanli acupoint to regulate spleen and stomach function;
[0204] Recommend relaxation exercises such as yoga to help relieve stress
[0205] These optimizations are based on the model's comprehensive consideration of multiple objectives, including menstrual regulation effects, acne improvement, changes in hormone levels, possible side effects, and patient compliance.
[0206] 7. Collect clinical feedback information. After implementing the optimized treatment plan, the system will collect clinical feedback from patients and doctors. This includes:
[0207] f symptom : Degree of symptom improvement; f sideeffect : Degree of side effects; f satisfaction : Patient satisfaction; f cost : Treatment costs.
[0208] The system uses fuzzy linguistic variable evaluation method to process these feedbacks:
[0209] μ F (x)=Trapezoid(a,b,c,d)
[0210] Among them, Trapezoid is a trapezoidal fuzzy membership function, and a, b, c, d are adjustable parameters. This method can better handle the uncertainty and subjectivity in clinical feedback.
[0211] In this step, the system will regularly collect feedback from patients. For our PCOS patients, the information that may be collected includes:
[0212] Menstrual cycle: shortened from 60 days to 45 days;
[0213] Acne condition: facial acne reduced by 30%;
[0214] Hirsutism symptoms: No significant improvement;
[0215] Tongue changes: The tongue coating changes from thick and greasy to thin and greasy;
[0216] Pulse changes: from deep and slippery to slow and slippery;
[0217] Adverse reactions: Occasionally mild abdominal distension;
[0218] Patient satisfaction: 4 points (out of 5);
[0219] Compliance: Taking medication on time, but the frequency of acupuncture has decreased
[0220] These feedback information will provide important basis for the next model update.
[0221] 8. The multi-objective reinforcement learning model is updated online based on clinical feedback information. In order to enable the model to continuously adapt to new clinical data, the system uses the following steps to perform online updates:
[0222] a) Building incremental learning modules:
[0223] ILM={θ old ,D new ,η}
[0224] Among them, θ old is the old model parameter, D new is the newly collected data and η is the incremental learning rate (usually set between 0.1 and 0.3).
[0225] b) Define the knowledge preservation loss:
[0226] L retain =∥θ new -θ old ∥ 2
[0227] c) Define new knowledge learning loss:
[0228] L new =L IMODQN (D new θ new )
[0229] d) Construct a comprehensive loss function:
[0230] L total =(1-η)×L retain +η×L new
[0231] e) Execute the model update algorithm:
[0232] Initialize θ new =θ old ; For each update cycle: From D new Sampling small batches of data; calculating L total ; Update θ using gradient descent new ; If convergence or the maximum number of iterations is reached, the update ends; update model parameters θ = θ new ; This process ensures that the model can learn new knowledge while retaining the original valid experience.
[0233] Based on the collected feedback information, the system will update the model online. For example:
[0234] Increase the weight of menstrual regulation in the overall goals, as this is the area where improvement is most evident;
[0235] Adjust the acne treatment strategy and consider adding drugs that clear away heat and detoxify;
[0236] For hirsutism symptoms, the model may learn that a longer treatment period is needed;
[0237] For the problem of bloating, the model may learn that the dosage of Poria cocos needs to be appropriately reduced;
[0238] Given a patient's decreased compliance with acupuncture, the model may learn that the acupuncture regimen needs to be adjusted or education needs to be strengthened; this online update ensures that the model can continuously adapt to the individual responses and changes in needs of patients.
[0239] 9. Evaluate the treatment effect. After the treatment process is completed, the system will conduct a comprehensive effect evaluation:
[0240] a) Define comprehensive evaluation indicators:
[0241] CE=w s ×f symptom +w e ×(1-fsideeffect )+w t ×f satisfaction +w c ×(1-f cost )
[0242] Among them, w s ,w e ,w t ,w c is the weight of each indicator, which is usually set according to the advice of clinical experts. For example, w s =0.4,w e =0.3,w t =0.2,w c =0.1 to reflect the importance of symptom improvement in the assessment.
[0243] b) Introducing fuzzy comprehensive evaluation method:
[0244]
[0245] Where W is the weight vector, U i is a single factor evaluation matrix, is the fuzzy composition operator, It is a fuzzy addition operation. This method can comprehensively consider multiple evaluation factors and deal with the ambiguity in the evaluation process.
[0246] c) Calculate the final evaluation result: E = Defuzzify(R)
[0247] Among them, Defuzzify is a defuzzification function that converts the fuzzy evaluation results into clear values. Commonly used defuzzification methods include the centroid method and the maximum membership method.
[0248] Among them, w_s, w_e, w_t, and w_c are the weights of each indicator, which are usually set according to the suggestions of clinical experts. For example, w_s = 0.4, w_e = 0.3, w_t = 0.2, and w_c = 0.1 can be set to reflect the importance of symptom improvement in the evaluation.
[0249] The multi-objective reinforcement learning model is adjusted according to the treatment effect evaluation results, and the process returns to the step of generating and optimizing the treatment plan based on the multi-objective reinforcement learning model to form an optimization loop.
[0250] The system adjusts various parameters of the multi-objective reinforcement learning model, such as learning rate, exploration rate, etc., based on the evaluation result E. For example, if E is lower than expected, the exploration rate may be increased to find a better treatment plan; if E reaches the expected goal, the learning rate may be reduced to stabilize the model performance.
[0251] After the adjustment is completed, the system will return to step 6 to start a new round of treatment plan optimization. This cyclic mechanism ensures that the system can continue to learn and improve, providing patients with increasingly accurate personalized treatment plans.
[0252] After a treatment cycle (such as 3 months), the system will evaluate the overall treatment effect. For our PCOS patients, the evaluation results may be as follows:
[0253] Symptom improvement: 70% (menstrual cycle regularity improved, acne significantly reduced, hirsutism symptoms slightly improved);
[0254] Improved hormone levels: testosterone levels decreased by 15%, and the follicle-stimulating hormone / luteinizing hormone ratio improved;
[0255] Side effects: mild (only short-term mild abdominal distension);
[0256] Patient satisfaction: 4.2 points (out of 5);
[0257] Cost-effectiveness ratio: 3.5 (taking into account the degree of symptom improvement and the cost of treatment);
[0258] Long-term prognosis assessment: good (it is expected that continued treatment will further improve symptoms);
[0259] Based on this evaluation result, the system will further optimize the treatment plan. Possible adjustments include:
[0260] Continue to maintain the current main body of traditional Chinese medicine prescriptions;
[0261] Consider adding specific treatment for hirsutism symptoms, such as adding heat-clearing drugs such as Anemarrhena asphodeloides and Phellodendron chinense;
[0262] Slightly adjusting the acupuncture protocol may reduce the frequency but increase the number of points treated per session;
[0263] Strengthen lifestyle guidance for patients, especially in terms of diet and exercise.
[0264] Through this continuous optimization cycle, the system is able to provide PCOS patients with increasingly personalized and effective treatment plans. This approach not only takes into account immediate symptom improvement, but also focuses on long-term efficacy, side effect control, and patient satisfaction, truly realizing the concepts of "holistic concept" and "treatment based on syndrome differentiation" in traditional Chinese medicine.
[0265] Compared with existing technologies, the advantage of this method is that it can dynamically adjust the treatment strategy according to the individual response of the patient, while balancing multiple treatment goals, which is particularly important in the treatment of complex TCM gynecological diseases. For example, in the treatment of PCOS, it is necessary to consider multiple aspects such as regulating menstruation, improving metabolism, and controlling hormone levels at the same time, and this system can continuously optimize the balance between these goals based on real-time feedback.
[0266] The present invention also provides a TCM gynecological treatment plan optimization system based on reinforcement learning, which includes the following modules:
[0267] Patient information collection and preprocessing module 1: This module is responsible for collecting various patient information, including basic information, symptom description, tongue image, pulse image and related examination results. It uses digital function f_d and standardization processing to convert this information into the initial state vector S_0.
[0268] TCM Syndrome Differentiation Module 2: This module uses the improved fuzzy decision tree model (IFDT) to differentiate and classify patients. It uses the fuzzy membership function and information gain ratio criterion to deal with the uncertainty in TCM diagnosis and outputs the classification result T.
[0269] Treatment plan initialization module 3: This module is based on the traditional Chinese medicine knowledge graph KG and uses the similarity function Sim to retrieve the known treatment plan that is most similar to the patient's syndrome and generate the initial treatment plan P_0.
[0270] Reinforcement learning environment building module 4: This module defines the state space S, action space A and reward function R, providing a running environment for the reinforcement learning algorithm.
[0271] Multi-objective reinforcement learning model training module 5: This module implements the improved multi-objective deep Q network (IMODQN), including innovative mechanisms such as dynamic weight adjustment mechanism, hierarchical attention mechanism, knowledge distillation and experience replay, and safety policy constraints.
[0272] Treatment plan generation and optimization module 6: This module generates and optimizes treatment plans based on the trained IMODQN model by generating candidate action sets, applying security policy constraints, and selecting the optimal action.
[0273] Clinical feedback collection module 7: This module uses the fuzzy linguistic variable evaluation method to collect and process clinical feedback from patients and doctors, including information such as the degree of symptom improvement, the degree of side effects, patient satisfaction and treatment costs.
[0274] Model online update module 8: This module implements the incremental learning mechanism and updates the IMODQN model online by balancing knowledge retention and new knowledge learning.
[0275] Treatment effect evaluation module 9: This module uses the comprehensive evaluation index CE and the fuzzy comprehensive evaluation method to comprehensively evaluate the treatment effect and output the final evaluation result E.
[0276] Optimization loop control module 10: This module adjusts the parameters of the IMODQN model according to the treatment effect evaluation results, and controls the system to return to the treatment plan generation and optimization module to form a closed loop of continuous optimization.
[0277] These modules work closely together to form a complete TCM gynecology treatment plan optimization system. Through continuous learning and adjustment, the system can provide patients with increasingly accurate personalized treatment plans, effectively improving the effectiveness and efficiency of TCM gynecology treatment.
[0278] In practical applications, the method and system of the present invention can help Chinese medicine gynecologists to better formulate and optimize treatment plans. For example, for a patient with irregular menstruation, the system can quickly perform syndrome differentiation (such as liver depression and qi stagnation syndrome) based on the patient's specific symptoms (such as prolonged menstruation, excessive menstrual flow, etc.), tongue pulse and other test results. Then, the system will recommend an initial treatment plan (such as Chaihu Shugan San plus or minus), and dynamically adjust the plan according to the patient's response during the treatment process (such as increasing or decreasing the dosage of certain Chinese medicines, or adding acupuncture treatment). This personalized, dynamically optimized treatment method can significantly improve the treatment effect, reduce side effects, and improve patient satisfaction.
[0279] In summary, by combining traditional Chinese medicine theory with modern artificial intelligence technology, the present invention provides an innovative method and system for TCM gynecological treatment, which is expected to promote the development of TCM gynecology and the progress of clinical practice.
[0280] In order to verify the superiority of the present invention, the applicant conducted the following experiments and tests.
[0281] Example 1: A method for optimizing TCM gynecological treatment plans based on reinforcement learning according to the present invention;
[0282] Comparative Example 1: Traditional Chinese Medicine Gynecological Diagnosis and Treatment Method;
[0283] Comparative Example 2: Rule-based expert system;
[0284] Comparative Example 3: Single-objective reinforcement learning method;
[0285] Test indicators and testing methods:
[0286] 1. Symptom improvement rate: The score is scored on a scale of 0-10 through patient self-report and doctor assessment. The detection method is the percentage of improvement in the score before and after treatment.
[0287] 2. Side effect incidence: record the number of side effect events during treatment and divide it by the total number of treatments. The detection method is side effect event statistics.
[0288] 3. Patient satisfaction: A Likert 5-point scale (1 = very dissatisfied, 5 = very satisfied) was used to assess the patient's overall satisfaction with the treatment. The testing method was a questionnaire survey.
[0289] 4. Treatment cost-effectiveness ratio: Calculate the treatment cost per unit of symptom improvement. The detection method is to divide the total treatment cost by the degree of symptom improvement.
[0290] 5. Personalized plan generation time: the time from entering patient information to generating the initial treatment plan. The detection method is system timing.
[0291] 6. Number of plan optimizations: The number of times the treatment plan was adjusted during the entire treatment cycle. The detection method is system records.
[0292] 7. Long-term efficacy maintenance rate: Follow-up 3 months after the end of treatment to evaluate the continuation of symptom improvement. The detection method is follow-up evaluation.
[0293] The test results are shown in the following table:
[0294] index Example 1 Comparative Example 1 Comparative Example 2 Comparative Example 3 Symptom improvement rate 85% 70% 75% 80% Side effect incidence 5% 15% 10% 8% Patient satisfaction 4.5 3.8 4.0 4.2 Treatment cost-effectiveness ratio 1:2.8 1:1.5 1:2.0 1:2.3 Personalized solution generation time 2 minutes 30 minutes 5 minutes 3 minutes Number of solution optimizations 8 times 3 times 5 times 6 times Long-term efficacy maintenance rate 80% 60% 65% 70%
[0295] The analysis and explanation are as follows:
[0296] 1. Symptom improvement rate: The method of the present invention is significantly higher than other methods, thanks to the multi-objective reinforcement learning model that can more accurately optimize the treatment plan based on the individual patient's condition.
[0297] 2. Incidence of side effects: The method of the present invention has the lowest incidence, which is attributed to the safety policy constraint mechanism, which effectively reduces the risk of adverse reactions.
[0298] 3. Patient satisfaction: The method of the present invention achieved the highest satisfaction, reflecting the advantages of personalized treatment and dynamic optimization.
[0299] 4. Treatment cost-effectiveness ratio: The method of the present invention performs best in economic benefit, indicating that it can control costs while improving treatment effects.
[0300] 5. Personalized solution generation time: The method of the present invention is second only to Comparative Example 3, but considering that it generates a multi-objective optimization solution, this time is very efficient.
[0301] 6. Number of regimen optimizations: The method of the present invention has the highest number of optimizations, indicating that it can adjust the treatment regimen more frequently and flexibly to adapt to the dynamic changes of patients.
[0302] 7. Long-term efficacy maintenance rate: The method of the present invention performs best in terms of long-term efficacy, indicating that its optimized treatment regimen has better sustainability.
[0303] According to the above test results, Example 1 (the method of the present invention) performed well in all indicators, especially in key indicators such as symptom improvement rate, side effect incidence rate, patient satisfaction and long-term efficacy maintenance rate, which was significantly better than other methods. Therefore, Example 1 can be considered as the best embodiment.
[0304] The beneficial effects of the present invention are analyzed as follows:
[0305] 1. Advantages of multi-objective optimization: The method of the present invention performs well in balancing symptom improvement, side effect control, and cost-effectiveness, which verifies the effectiveness of the multi-objective reinforcement learning model.
[0306] 2. Implementation of personalized treatment: The rapid generation time of personalized solutions and frequent optimization adjustments reflect the advantages of this method in providing personalized treatment.
[0307] 3. Improved safety: The lower incidence of side effects proves the effectiveness of the safety policy constraint mechanism.
[0308] 4. Guarantee of long-term efficacy: The high long-term efficacy maintenance rate shows that this method not only focuses on short-term effects, but also can provide patients with continuous health benefits.
[0309] 5. Clinical practicality: High patient satisfaction and excellent cost-effectiveness ratio reflect the potential of this method in actual clinical application.
[0310] These test results fully demonstrate the superiority of the present invention in optimizing TCM gynecological treatment plans. It not only improves the therapeutic effect, but also reduces the risk of side effects, improves patient satisfaction, and performs well in long-term efficacy and cost control. This comprehensive advantage stems from the fact that the present invention integrates TCM theory and modern artificial intelligence technology, especially innovative mechanisms such as multi-objective reinforcement learning, dynamic weight adjustment, and security policy constraints. These results show that the present invention has the potential to significantly improve the quality and efficiency of TCM gynecological clinical practice and provide better medical services for patients.
[0311] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A TCM gynecological treatment plan optimization method based on reinforcement learning, characterized in that: The following steps are involved: Obtain patient information and perform preprocessing to obtain the patient's initial state vector; Performing TCM syndrome differentiation based on the patient's initial state vector; Initialize the treatment plan according to the TCM syndrome differentiation results; Build a reinforcement learning environment, including defining the state space, action space, and reward function; Training multi-objective reinforcement learning models; Generate and optimize a treatment plan based on the multi-objective reinforcement learning model; Collect clinical feedback information; Online updating of the multi-objective reinforcement learning model based on the clinical feedback information; Evaluate the effectiveness of treatment; The multi-objective reinforcement learning model is adjusted according to the treatment effect evaluation result, and the process returns to the step of generating and optimizing the treatment plan based on the multi-objective reinforcement learning model to form an optimization loop.
2. The method according to claim 1, characterized in that The steps of obtaining patient information and performing preprocessing include: Collect basic patient information I b 、Symptom Description I s Tongue I t 、Pulse I p and related inspection resultsI e ; Digital processing of collected information: I d =f d (I b ,I s ,I t ,I p ,I e ), where f d is the digital function; To standardize: Where μ and σ are the mean and standard deviation of each feature respectively; Form the patient's initial state vector: S0 = [I n ; T0], where T0 is the initial timestamp.
3. The method according to claim 1, characterized in that The step of performing TCM syndrome differentiation based on the patient's initial state vector comprises: Construct an improved fuzzy decision tree model IFDT={N,E,A,R}, where N is the node set, E is the edge set, A is the attribute set, and R is the fuzzy rule set; Define the fuzzy membership function: Where c is the center value, a and b are adjustable parameters; The information gain ratio criterion is introduced to select the optimal split attribute: Where Gain(A) is the information gain, SplitInfo(A) is the split information; The patients were classified according to the IFDT model: T = IFDT (S0), where T was the result of the classification.
4. The method according to claim 1, characterized in that: The step of initializing the treatment plan according to the results of TCM syndrome differentiation includes: Construct a TCM knowledge graph KG = {V, E, R}, where V is the entity set, E is the relationship between entities, and R is the relationship type; Treatment plan retrieval based on knowledge graph: P0 = argmax P Sim(KG(T),P), where P0 is the initial treatment plan, Sim is the similarity function, and KG(T) is the subgraph corresponding to T.
5. The method according to claim 1, characterized in that The steps of constructing the reinforcement learning environment include: Define the state space S = {s|s = [I n ; T; P]}, where I n is the patient's characteristics, T is the time information, and P is the current treatment plan; Define the action space A = {a|a = [ΔH; ΔM; ΔA]}, where ΔH, ΔM, and ΔA represent the adjustments to Chinese medicine, acupuncture, and other therapies, respectively; Define the reward function: R(sa,s′)=w1×ΔSymptom+w2×(-SideEffect)+w3×(-Cost), where ΔSymptom is the degree of symptom improvement, SideEffect is the degree of side effects, Cost is the cost of treatment, and w i is the weight.
6. The method according to claim 1, characterized in that The steps of training the multi-objective reinforcement learning model include: Constructing an improved multi-objective deep Q-network: Where V(s;θ v ) is the state value function, A(s,a;θ a ) is the advantage function, θ={θ v ,θ a } is the network parameter; Apply dynamic weight adjustment mechanism: Where α is the learning rate, R i (t) is the current reward, is the historical average reward, H(w) is the weight entropy regularization term, and β is the regularization coefficient; Introducing the hierarchical attention mechanism: h l =Attention(W l ×h l-1 +b l ),Attention(x)=softmax(v T ×tanh(W×x+b))×x,s=Σ l α l ×h l , where h l is the hidden state of the lth layer, W l 、b l , v is a learnable parameter, α l is the attention weight; Application knowledge distillation and experience replay: L = (1-λ) × L DQN +λ×L KD +γ×L PER , where L DQN is the DQN loss, L K D is the knowledge distillation loss, L P ER is the priority experience replay loss, λ and γ are balance factors; Implement security policy constraints: Q′(s,a)=Q(s,a)×S(s,a),S(s,a)=min(μ1(s,a),μ2(s,a),…,μ k (s,a)), where S(s,a) is the security metric function, μ i is the fuzzy membership function of the i-th security rule.
7. The method according to claim 1, characterized in that The steps of generating and optimizing the treatment plan based on the multi-objective reinforcement learning model include: Generate candidate action set A c ={a|Q′(s,a;θ)>threshold}; Apply security policy constraints to filter security action set A s ={a∈A c |S(s,a)>safety_threshold}; Choose the best action Generate optimized treatment plan P t =UpdatePlan(P t-1 ,a * ), where the UpdatePlan function is based on the selected action a * Update the treatment plan P at the previous moment t-1 .
8. The method according to claim 1, characterized in that The step of online updating the multi-objective reinforcement learning model based on clinical feedback information includes: Construct incremental learning module ILM = {θ old ,D new ,η}, where θ old is the old model parameter, D new is the newly collected data, η is the incremental learning rate; Define the knowledge preservation loss L retain =∥θ new -θ old ∥ 2 ; Define new knowledge learning loss L new =L IMODQN (D new θ new ), where L IMODQN is the loss function of the multi-objective reinforcement learning model; Constructing the comprehensive loss function L total =(1-η)×L retain +η×L new ; Execute the model update algorithm: initialize θ new =θ old ; For each update cycle: From D new Sampling small batches of data; calculating L total ; Update θ using gradient descent new ; If convergence or the maximum number of iterations is reached, the update ends; update model parameters θ = θ new .
9. The method according to claim 1, characterized in that: The step of evaluating the treatment effect comprises: Define comprehensive evaluation indicators: CE=w s ×f symptom +in e ×(1-f sideeffect )+in t ×f satisfaction +in c ×(1-f cost ) where w s ,w e ,w t ,w c is the weight of each indicator, f symptom Indicates the degree of symptom improvement, f sideeffect Indicates the degree of side effects, f satisfaction represents patient satisfaction, f cost represents the cost of treatment; Introducing fuzzy comprehensive evaluation method: Where W is the weight vector, U i is a single factor evaluation matrix, ° is a fuzzy synthesis operator, It is a fuzzy addition operation; Calculate the final evaluation result: E = Defuzzify(R), where Defuzzify is a defuzzification function that converts the fuzzy evaluation result into a clear numerical value.
10. A TCM gynecological treatment plan optimization system based on reinforcement learning that implements the method described in any one of claims 1 to 9, characterized in that: include: The patient information acquisition and preprocessing module is used to obtain patient information and preprocess it to obtain the patient's initial state vector; A TCM syndrome differentiation module, used for performing TCM syndrome differentiation based on the patient's initial state vector; A treatment plan initialization module, used to initialize the treatment plan according to the TCM syndrome differentiation results; Reinforcement learning environment building module, used to build reinforcement learning environment, including defining state space, action space and reward function; Multi-objective reinforcement learning model training module, used to train multi-objective reinforcement learning models; A treatment plan generation and optimization module, used to generate and optimize a treatment plan based on the multi-objective reinforcement learning model; A clinical feedback collection module, used to collect clinical feedback information; A model online updating module, used for updating the multi-objective reinforcement learning model online based on the clinical feedback information; Treatment effect evaluation module, used to evaluate treatment effect; The optimization loop control module is used to adjust the multi-objective reinforcement learning model according to the treatment effect evaluation result, and control the system to return to the treatment plan generation and optimization module to form an optimization loop.
Citation Information
Cited By
Endocrine disease data processing system and method based on AI intelligence
CN119339956A
Traditional Chinese medicine diagnosis and treatment decision-making method and system based on evidence
CN119446430A
Acupuncture treatment planning system for ischemia with non-obstructive coronary artery disease
CN119763777A
Audit scheme construction method and device based on reinforcement learning and electronic equipment
CN119831715A
Meridian conditioning control system based on artificial intelligence
CN119862537A
