A reinforcement learning treatment system for gynecological diseases based on large model double-function guidance and a construction method thereof

Abstract

The application relates to a gynecological disease reinforcement learning treatment system based on a large model double-function guide and a construction method, which comprises the following steps: jointly encoding core indexes selected, generating a discrete-continuous mixed action space suitable for a gynecological disease treatment scene by a medical large model, sampling and selecting a treatment scheme from the action space, inputting a state vector and the treatment scheme into the medical large model to generate a new state vector, defining a multi-target weighted reward function, calculating an instant reward, iteratively optimizing by using a strategy gradient method, and outputting an optimal treatment strategy. The application introduces a probabilistic action space generation module to replace a traditional artificial discretization or random generation mode, solves the contradiction problem between action space construction and convergence efficiency, and performs semantic analysis on core indexes of gynecological diseases by using a medical pre-training large model to generate a feasible treatment scheme set with a probability distribution, so that the clinical rationality of the action is ensured, the space dimension is simplified through probability screening, and strategy convergence is accelerated.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a reinforcement learning treatment system for gynecological diseases based on a large-model dual-function guided approach and its construction method. Background Technology

[0002] With the rapid development of precision medicine, the demand for individualized treatment of gynecological diseases is becoming increasingly urgent. Traditional treatment strategies rely heavily on physicians' clinical experience, which not only suffers from significant subjective limitations and a lack of dynamic optimization capabilities in the treatment process, but also leads to uneven clinical efficacy due to the heterogeneity of treatment levels among different medical institutions. The application of artificial intelligence (AI) technology in the treatment of gynecological diseases also faces many challenges that urgently need to be overcome: real medical data is scarce due to ethical constraints, and conducting strategy trial-and-error on patients carries serious ethical risks; the sharing and use of sensitive data such as patient medical records and images poses a risk of privacy leaks, and traditional encryption methods cannot simultaneously ensure data availability and resistance to attribute-based reasoning attacks; existing medical AIs mostly focus on diagnostic tasks, and treatment strategies are mostly static recommendations, lacking the ability to adaptively adjust based on the dynamic state of patients and prone to blindly exploring the strategy space, making it difficult to adapt to the complex course and significant individual differences in gynecological diseases. While digital twins, large-scale models, and embedding technologies have been applied individually in the medical field, a collaborative framework has not yet been formed to solve the problem of dynamic training of treatment strategies.

[0003] As a core technology for sequential decision optimization, reinforcement learning has shown application potential in the field of gynecological disease treatment, but it still faces three major bottlenecks: First, the challenge of constructing the action space. Gynecological disease treatment involves multi-dimensional combinations of drug types, dosages, and surgical methods. Randomly generated actions are prone to dimensionality explosion, and existing clustering or manual discretization methods cannot balance clinical rationality and spatial simplification. Second, the dilemma of state transition modeling. The dynamic changes of patients' physiological indicators under treatment intervention are highly nonlinear, and the transfer functions of traditional statistical models cannot accurately simulate the complex dynamics of medical scenarios. Third, low convergence efficiency. The high noise of medical data and the high risk of decision-making can easily cause reinforcement learning strategies to get stuck in local optima during iteration, and existing methods lack optimization mechanisms that deeply integrate with clinical knowledge. Summary of the Invention

[0004] The purpose of this invention is to provide a reinforcement learning treatment system and construction method for gynecological diseases based on a large model with dual functions, thereby solving the aforementioned problems in the prior art.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for constructing a reinforcement learning-based treatment system for gynecological diseases guided by a large-scale dual-function model includes the following steps: Step S01: Construct a core indicator system for gynecological diseases and select core indicators according to disease type; Step S02: Construct the encoding module and use the Embedding model to jointly encode the core indicators to obtain the state vector. ; Step S03: Define the conditions for treatment termination and the corresponding judgment criteria; Step S04: Generate a discrete-continuous hybrid action space adapted to gynecological disease treatment scenarios based on a large medical model. ; Step S05: Through the policy function From the action space Treatment plan selection by mid-sample , the state vector Treatment plan Input a large medical model and generate a new state vector. ; Step S06: Define including therapeutic rewards Safety Rewards and economic rewards Multi-objective weighted reward function ; Step S07: Based on the state vector Treatment plan and the new state vector Through a multi-objective weighted reward function Calculate instant rewards Based on the termination condition If the conditions are met, the strategy gradient method is selected for iterative optimization to output the optimal treatment strategy. .

[0006] The beneficial effects of this invention are: the introduction of a "large model-guided probabilistic action space generation module" replaces the traditional manual discretization or random generation method, resolving the contradiction between action space construction and convergence efficiency. By using a medically pre-trained large model to perform semantic parsing of core indicators of gynecological diseases, a set of feasible treatment plans with probability distributions is generated. This ensures the clinical rationality of the actions while simplifying the spatial dimension through probability screening, thus accelerating strategy convergence.

[0007] By leveraging the state transition simulation function of large-scale models, the limitations of fixed rules in traditional statistical modeling are eliminated. The patient's current state vector and the selected treatment plan are input into the large-scale model, which utilizes its ability to fit medical time-series data to dynamically predict the changing trends of core indicators such as blood routine and tumor markers, and adaptively generate new state vectors. This solves the problem that the state transition function in traditional methods cannot match the dynamic changes of individual conditions, making the prediction of state transitions more consistent with the time-series changes of real clinical diagnosis and treatment.

[0008] A two-way feedback mechanism of "policy update - model fine-tuning" is constructed to achieve collaborative optimization of clinical knowledge and iterative data. After each policy update, the state-action-new state trajectory corresponding to the reward value is fed back to the large model for parameter fine-tuning, so that the subsequently generated action space is more in line with clinical efficacy data, solving the problem of insufficient clinical adaptability caused by the "pure data-driven" nature of traditional reinforcement learning.

[0009] This paper adds a module for screening and encoding gynecological disease-specific indicators, which differs from general medical reinforcement learning methods in addressing the disease-specific adaptability issue of treatment strategies. For different gynecological disease types such as inflammation, tumors, and dysfunctional bleeding, corresponding core indicator sets are predefined, and differentiated state vectors are generated through feature encoding, improving the targeting of strategy training.

[0010] Based on the above technical solution, the present invention can be further improved as follows.

[0011] Furthermore, the core indicators in step S01 include demographic information, medical history data, laboratory indicators, imaging data, and reproductive physiological mechanism parameters; Demographic information includes patient age A and BMI index B; Medical history data includes disease type D and disease duration T; Laboratory indicators include hormone levels. inflammatory factors ; Imaging data include lesion size (S), morphological parameters (M), and imaging semantics. Textual semantics ; Reproductive physiological parameters include secretion rate k and binding efficiency. .

[0012] Furthermore, step S02 specifically includes the following steps: Step S21: Continuous data are filled with the mean or median of the same population or with the model prediction value; categorical data and missing image features are filled with 0. Step S22: Normalize continuous data, use 0-1 encoding for binary data, one-hot encoding for multi-class data, and ordered label encoding for ordered discrete data. Semantic indicators and imaging indicators are extracted as fixed-dimensional features using a deep learning model. Step S23: Concatenate all encoded individual features in a fixed order into a one-dimensional original vector; Step S24: Based on the Embedding model, encode the original vector into a state vector with a dimension of 1024. and the state vector As input representations for reinforcement learning agents.

[0013] The further beneficial effect of adopting the above is that, compared with the method of directly using literals as input, the input mode based on the encoded state vector can effectively protect the data and avoid the risk of data leakage from a technical perspective.

[0014] Furthermore, step S03 specifically includes the following steps: Step S31: Define the conditions for treatment termination : ; in," " indicates "OR" logic, meaning treatment will terminate if either scenario is met; Step S32: Establish criteria for determining whether treatment targets are met, safety risks exceed limits, and treatment exceeds the prescribed period.

[0015] Furthermore, the judgment criteria in step S32 include: The criteria for determining treatment success are: for two consecutive treatment cycles, the core efficacy indicators simultaneously meet the criteria of "normal hormone levels + lesion safety," that is: ; The criteria for determining excessive safety risks are: ≥2 instances of mild adverse reactions, or 1 instance of a serious adverse reaction, i.e.: ; The criteria for determining if treatment has exceeded the recommended duration are: the treatment cycle has reached the maximum clinically recommended limit, i.e.: .

[0016] Furthermore, step S04 specifically includes: The medical big data model generates a required action space based on extracted clinical information. Calculate the action space Each treatment plan conditional probability Action space All treatment options conditional probability The sum is 1; the higher the conditional probability value, the better the treatment plan. In the current state vector The higher the degree of compatibility, the better.

[0017] The further beneficial effects of adopting the above are: by generating a probabilistic action space through a large model, the action dimension of traditional gynecological treatment scenarios is reduced from thousands to single digits. Experiments have verified that the convergence speed of the strategy is improved by more than 60%, thus solving the convergence problem of medical reinforcement learning.

[0018] Furthermore, step S06 specifically includes the following steps: Step S61: Define efficacy rewards based on improvements in core patient status indicators The study employs a segmented design, corresponding to three clinical scenarios: "treatment improvement," "partial improvement," and "no improvement." ; Step S62: Define safety rewards based on adverse reaction monitoring : ; in, express The number of adverse reactions that occur in the patient at any given time; The fixed penalty weight representing the number of adverse reactions per unit; Step S63: Define economic rewards based on the cost of treatment actions : ; in, Indicates the execution of an action Total treatment cost; This indicates the cost per unit dose of drug; Indicates drug dosage; This indicates that the treatment method has a fixed cost; This indicates the cost per unit of treatment cycle; Indicates the treatment cycle; This represents the maximum cost threshold for routine clinical treatment. , , These are the unit costs for the drug, the method, and the cycle, respectively. Step S64: Reward based on treatment efficacy Safety Rewards and economic rewards Construct a multi-objective weighted reward function : ; in, This indicates a reward for effective treatment. Indicates a safety reward; Indicates economic reward; , , These represent the weights of the efficacy reward, safety reward, and economic reward, respectively.

[0019] Furthermore, the definitions of the three clinical scenarios in step S61 include: Treatment improvement is defined as: a dual improvement in symptom relief and pathological cure, and estrogen levels. and the size of the lesion All are within the normal range: ; in, This indicates the estrogen level after treatment; This indicates the normal range for estrogen levels; Indicates the condition of the lesions after treatment; Indicates the normal range of the lesion; No improvement is defined as: lesion size Compared to before treatment Increased beyond a security threshold : ; in, Indicates the size of the lesion after treatment; Indicates the size of the lesion before treatment; Indicates the safety threshold; Partial improvement is defined as: failure to achieve improvement in both indicators after treatment, but no significant worsening of lesion size. ; in, It is an estrogen-improving item; This refers to the improvement of lesions; the reward value when the treatment target is met. .

[0020] Furthermore, step S07 specifically includes the following steps: Step S71: Utilize the current trajectory data The parameters of the policy function are updated using the policy gradient formula. : ; Step S72: Complete the trajectory Input the medical big data model, and the medical big data model learns the correspondence between "state-action-reward-new state" in the trajectory and fine-tunes its own parameters; Step S73: If Repeat steps S03 to S07 until... ;like This is the optimal treatment strategy. Output.

[0021] The present invention also discloses a reinforcement learning treatment system for gynecological diseases based on a large model with dual functions, which is constructed using the above-mentioned construction method. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0023] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0024] Example 1 like Figure 1 As shown, a method for constructing a reinforcement learning treatment system for gynecological diseases based on a large-model dual-function guided approach includes the following steps: Step S01: Construct a core indicator system for gynecological diseases, and select core indicators according to disease type (inflammation / tumor / functional bleeding, etc.); Step S02: Construct an encoding module and use the Embedding model to jointly encode the core indicators of gynecological disease patients to obtain a state vector. ; Step S03: Define the conditions for treatment termination and the corresponding judgment criteria; Step S04: Generate a discrete-continuous hybrid action space adapted to gynecological disease treatment scenarios based on a large medical model. ; Step S05: Through the policy function From the action space Treatment plan selection by mid-sample , the state vector Treatment plan Input a large medical model and generate a new state vector based on its time-series prediction capabilities. Changes in patient indicators after simulated treatment intervention: ; Step S06: Define including therapeutic rewards Safety Rewards and economic rewards Multi-objective weighted reward function ; Step S07: Based on the state vector Treatment plan and the new state vector Through a multi-objective weighted reward function Calculate instant rewards , Multi-objective weighted reward function The calculated values ​​are based on the termination condition. If the conditions are met, the strategy gradient method is selected for iterative optimization to output the optimal treatment strategy. .

[0025] A "large-model-guided probabilistic action space generation module" is introduced to replace traditional manual discretization or random generation methods, resolving the contradiction between action space construction and convergence efficiency. By using a medically pre-trained large-scale model to perform semantic parsing of core indicators of gynecological diseases, a set of feasible treatment plans with probability distributions is generated. This ensures the clinical rationality of the actions while simplifying the spatial dimension through probability screening, thus accelerating strategy convergence.

[0026] By leveraging the state transition simulation function of large-scale models, the limitations of fixed rules in traditional statistical modeling are eliminated. The patient's current state vector and the selected treatment plan are input into the large-scale model, which utilizes its ability to fit medical time-series data to dynamically predict the changing trends of core indicators such as blood routine and tumor markers, and adaptively generate new state vectors. This solves the problem that the state transition function in traditional methods cannot match the dynamic changes of individual conditions, making the prediction of state transitions more consistent with the time-series changes of real clinical diagnosis and treatment.

[0027] A two-way feedback mechanism of "policy update - model fine-tuning" is constructed to achieve collaborative optimization of clinical knowledge and iterative data. After each policy update, the state-action-new state trajectory corresponding to the reward value is fed back to the large model for parameter fine-tuning, so that the subsequently generated action space is more in line with clinical efficacy data, solving the problem of insufficient clinical adaptability caused by the "pure data-driven" nature of traditional reinforcement learning.

[0028] This paper adds a module for screening and encoding gynecological disease-specific indicators, which differs from general medical reinforcement learning methods in addressing the disease-specific adaptability issue of treatment strategies. For different gynecological disease types such as inflammation, tumors, and dysfunctional bleeding, corresponding core indicator sets are predefined, and differentiated state vectors are generated through feature encoding, improving the targeting of strategy training.

[0029] Example 2 like Figure 1 As shown, this embodiment is a further improvement on embodiment 1, as detailed below: The core indicators in step S01 include demographic information, medical history data, laboratory indicators, imaging data, and reproductive physiological mechanism parameters; Demographic information includes patient age A and BMI index B; Medical history data includes disease type D and disease duration T; Laboratory indicators include hormone levels. inflammatory factors ; Imaging data include lesion size (S), morphological parameters (M), and imaging semantics. Textual semantics ; Reproductive physiological parameters include secretion rate k and binding efficiency. .

[0030] Example 3 like Figure 1 As shown, this embodiment is a further improvement on embodiment 1, as detailed below: Step S02 specifically includes the following steps: Step S21: Handle the "noise" and "missing" data of the raw data: fill continuous data (height, blood routine) with the mean or median of the same population or with the model prediction value; fill categorical data (medical history, allergy history) and missing imaging features with 0. Step S22: Encode by data type: normalize continuous data, encode binary data with 0-1, encode multi-type data with one-hot encoding, encode ordered discrete data with ordered labels, and extract semantic indicators and imaging indicators as fixed-dimensional features using deep learning models. Step S23: Vector integration: Concatenate all encoded individual features in a fixed order into a one-dimensional original vector, ensuring that the dimension is fixed and the order is traceable; Step S24: Vector Embedding: Based on the Embedding model, the original vector is encoded into a state vector with a dimension of 1024. and the state vector As input representations for reinforcement learning agents.

[0031] Compared to directly using literals as input, the input mode based on encoded state vectors can effectively protect data and avoid the risk of data leakage from a technical perspective.

[0032] By using disease-specific indicator coding and a two-way feedback mechanism, the model's strategy transfer accuracy is improved in various gynecological disease scenarios such as inflammation and tumors, thus solving the problem of insufficient generalization of single-disease models.

[0033] Embedding technology is introduced to process raw patient medical data, transforming the raw state data, which includes demographic information, medical history, laboratory indicators, imaging data, and reproductive physiological parameters, into low-dimensional, semantically unrelated vector representations. During this process, original privacy information is anonymized and hidden. Subsequent processes such as action space generation, policy training, and state transition simulation are all based on the embedded vectors, without needing to access or expose the original privacy data. This effectively avoids the privacy leakage risks caused by the direct participation of raw data in computation in traditional solutions. Simultaneously, the embedded vectors retain the core features of the medical data, without affecting the accuracy of policy training, achieving a synergistic balance between privacy protection and treatment effectiveness.

[0034] Example 4 like Figure 1As shown, this embodiment is a further improvement on embodiment 1, as detailed below: Step S03 specifically includes the following steps: Step S31: Define the conditions for treatment termination This includes treatment achievement, safety risks, and conditions for treatment exceeding the prescribed duration: ; in," " indicates "OR" logic, meaning treatment will terminate if either scenario is met; Step S32: Establish criteria for determining whether treatment targets are met, safety risks exceed limits, and treatment exceeds the prescribed period.

[0035] The judgment criteria in step S32 include: The criteria for determining treatment success are: for two consecutive treatment cycles, the core efficacy indicators simultaneously meet the criteria of "normal hormone levels + lesion safety," that is: ; Treatment target assessment focuses on two core indicators for gynecological disease treatment: estrogen, which characterizes endocrine function. And lesions characterizing the pathological state; according to the clinical principle of consolidating the therapeutic effect, two consecutive cycles (current cycle) are used. and the previous cycle All of these criteria must be met to conclude the treatment and avoid misjudging a cure due to fluctuations in physiological indicators. Cyclic estrogen , Periodic lesion size All come from the state vector .

[0036] The criteria for determining excessive safety risks are: ≥2 instances of mild adverse reactions, or 1 instance of a serious adverse reaction, i.e.: ; In clinical practice, ≥2 mild adverse reactions can significantly reduce patient compliance, while severe adverse reactions are an absolute contraindication and treatment must be terminated immediately. ( (Number of mild adverse reactions per cycle) ( The number of severe adverse reactions per cycle (all derived from the state vector) is derived from the state vector. ; The criteria for determining if treatment has exceeded the recommended duration are: the treatment cycle has reached the maximum clinically recommended limit, i.e.: .

[0037] Fixed maximum treatment cycle If the target is not achieved after 24 weeks (approximately 6 months), the treatment plan needs to be reassessed, such as changing medications or adjusting intervention methods, to avoid the strategy falling into a cycle of long-term ineffective treatment.

[0038] Example 5 like Figure 1 As shown, this embodiment is a further improvement on embodiment 1, as detailed below: Step S04 specifically includes: The preferred large model is the Large Language Model (LLM) specifically designed for the medical field. This model is trained using large-scale medical-specific data and can accurately analyze and professionally process the input patient state vector, outputting individualized treatment plans that conform to clinical diagnosis and treatment standards.

[0039] The medical big data model generates a required action space based on extracted clinical information. This includes 3-8 feasible treatment options and calculation of the action space. Each treatment plan conditional probability Action space All treatment options conditional probability The sum is 1; the higher the conditional probability value, the better the treatment plan. In the current state vector The higher the degree of compatibility, the better.

[0040] The action space is typically a mixed discrete-continuous space, encompassing drug therapy (drug type, dosage, frequency of administration), physical therapy (physical therapy, rehabilitation programs, lifestyle interventions), and surgical intervention (surgical methods, timing of surgery, extent of resection), etc. ; in, Indicates the type of drug; Indicates the treatment method; Indicates drug dosage; A specific action space is shown below: ; Among them, probability The clinical benefits of the proposed protocols were combined with the impact on fertility and patient tolerability. The sum of the probabilities of all protocols was 1.

[0041] By generating a probabilistic action space using a large model, the action dimension in traditional gynecological treatment scenarios is reduced from thousands to single digits. Experiments show that the convergence speed of the strategy is improved by more than 60%, solving the convergence problem of medical reinforcement learning.

[0042] By eliminating low-feasibility options from the action space and combining this with the complication weight constraint in the reward function, the probability of adverse interventions in the trained strategy is reduced.

[0043] Example 6 like Figure 1 As shown, this embodiment is a further improvement on embodiment 1, as detailed below: Step S06 specifically includes the following steps: Step S61: Define efficacy rewards based on improvements in core patient status indicators The study employs a segmented design, corresponding to three clinical scenarios: "treatment improvement," "partial improvement," and "no improvement." ; Step S62: Define safety rewards based on adverse reaction monitoring : ; in, express The number of adverse reactions that occur in the patient at any given time; The fixed penalty weight representing the number of adverse reactions per unit; This is to ensure that safety rewards are non-negative and to avoid confusion with the penalties associated with therapeutic rewards; The definitions of the three clinical scenarios in step S61 include: Treatment improvement is defined as a dual improvement in symptom relief and pathological cure, one indicator of which is estrogen level. and the size of the lesion All are within the normal range: ; in, Indicates after treatment ( Estrogen levels at any given time; Indicates the normal range of estrogen levels; indicates the post-treatment ( The condition of the lesions at any given time; Indicates the normal range of lesions; reward value when treatment targets are met. Defined as a larger positive reward to encourage the strategy to pursue effective treatment options; No improvement is defined as: lesion size Compared to before treatment Increased beyond a security threshold : ; in, Indicates the size of the lesion at (time) after treatment; Indicates before treatment ( The size of the lesion at any given time; Indicates the safety threshold; Partial improvement is defined as: failure to achieve improvement in both indicators after treatment, but no significant worsening of lesion size. ; in, This is the estrogen improvement item; the more significant the improvement, the closer the value of this item is to 1. This is a lesion improvement item; the more significant the lesion reduction, the closer this item's value is to 1; it's also a bonus value when the treatment target is met. ; Step S63: Define economic rewards based on the cost of treatment actions : ; in, Indicates the execution of an action Total treatment cost; This indicates the cost per unit dose of drug; Indicates drug dosage; indicates fixed costs of treatment methods; This indicates the cost per unit of treatment cycle; Indicates the treatment cycle; This represents the maximum cost threshold for routine clinical treatment. , , The unit costs are for the drug, the method, and the cycle, respectively; the design of the economic incentive follows the principle of "the lower the cost, the higher the incentive".

[0044] Step S64: Reward based on treatment efficacy Safety Rewards and economic rewards Construct a multi-objective weighted reward function This is used to quantify the effect of therapeutic actions on improving a patient's condition. ; in, Indicates a reward for therapeutic efficacy; indicates a reward for safety. Indicates economic reward; , , These represent the weights of the efficacy reward, safety reward, and economic reward, respectively.

[0045] Example 7 like Figure 1As shown, this embodiment is a further improvement on embodiment 1, as detailed below: Step S07 specifically includes the following steps: Step S71: Update the policy function using the policy gradient method. : Utilizing current trajectory data The parameters of the policy function are updated using the policy gradient formula. This makes the model more inclined to choose actions that yield higher rewards. ; Step S72: Fine-tuning of large model parameters: Adjusting the complete trajectory Input the medical big data model, and the medical big data model learns the correspondence between "state-action-reward-new state" in the trajectory, fine-tunes its own parameters, optimizes the rationality of subsequent generated treatment actions, and improves the accuracy of predicting state transition results. Step S73: Iterate until the optimal strategy is output: if Repeat steps S03 to S07, and repeat the process of state parsing, action generation, reward calculation, and policy update until... ;like The output will be the optimal treatment strategy, which corresponds to the treatment sequence that can reliably obtain the highest cumulative reward for this type of patient.

[0046] Example 8 A reinforcement learning therapy system for gynecological diseases based on a large model with dual functions is constructed using the construction method of any one of Examples 1 to 7.

[0047] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a reinforcement learning treatment system for gynecological diseases based on a large-model dual-function guided approach, characterized in that, Includes the following steps: Step S01: Construct a core indicator system for gynecological diseases and select core indicators according to disease type; Step S02: Construct an encoding module and use the Embedding model to jointly encode the core indicators to obtain a state vector. ; Step S03: Define the conditions for treatment termination and the corresponding judgment criteria; Step S04: Generate a discrete-continuous hybrid action space adapted to gynecological disease treatment scenarios based on a large medical model. ; Step S05: Through the policy function From the action space Treatment plan selection by mid-sample , the state vector With the treatment plan Input the aforementioned medical model to generate a new state vector. ; Step S06: Define including therapeutic rewards Safety Rewards and economic rewards Multi-objective weighted reward function ; Step S07: Based on the state vector The treatment plan and the new state vector Through a multi-objective weighted reward function Calculate instant rewards Based on the termination condition If the conditions are met, the strategy gradient method is selected for iterative optimization to output the optimal treatment strategy. .

2. The construction method according to claim 1, characterized in that, The core indicators in step S01 include demographic information, medical history data, laboratory indicators, imaging data, and reproductive physiological mechanism parameters; The demographic information includes the patient's age A and BMI index B; The medical history data includes disease type D and disease course T; The laboratory indicators include hormone indicators. inflammatory factors ; The imaging data includes lesion size S, morphological parameters M, and imaging semantics. Textual semantics ; The reproductive physiological mechanism parameters include secretion rate k and binding efficiency. .

3. The construction method according to claim 2, characterized in that, Step S02 specifically includes the following steps: Step S21: Continuous data are filled with the mean or median of the same population or with the model prediction value; categorical data and missing image features are filled with 0. Step S22: Normalize continuous data, use 0-1 encoding for binary data, one-hot encoding for multi-class data, and ordered label encoding for ordered discrete data. Semantic indicators and imaging indicators are extracted as fixed-dimensional features using a deep learning model. Step S23: Concatenate all encoded individual features in a fixed order into a one-dimensional original vector; Step S24: Based on the Embedding model, encode the original vector into a state vector with a dimension of 1024. and the state vector As input representations for reinforcement learning agents.

4. The construction method according to claim 1, characterized in that, Step S03 specifically includes the following steps: Step S31: Define the conditions for treatment termination : ； in," "" indicates "OR" logic, and treatment is terminated if either scenario is met; Step S32: Establish criteria for determining whether treatment targets are met, safety risks exceed limits, and treatment exceeds the prescribed period.

5. The construction method according to claim 4, characterized in that, The determination criteria in step S32 include: The criteria for determining treatment success are: for two consecutive treatment cycles, the core efficacy indicators simultaneously meet the criteria of "normal hormone levels + lesion safety," that is: ； The criteria for determining excessive safety risks are: ≥2 instances of mild adverse reactions, or 1 instance of a serious adverse reaction, i.e.: ； The criteria for determining if treatment has exceeded the recommended duration are: the treatment cycle has reached the maximum clinically recommended limit, i.e.: 。 6. The construction method according to claim 1, characterized in that, Step S04 specifically includes: The medical big data model generates a required action space based on extracted clinical information. Calculate the action space Each treatment plan conditional probability The action space All treatment options conditional probability The sum is 1; the higher the conditional probability value, the better the treatment plan. In the current state vector The higher the degree of compatibility, the better.

7. The construction method according to claim 1, characterized in that, Step S06 specifically includes the following steps: Step S61: Define efficacy rewards based on improvements in core patient status indicators The study employs a segmented design, corresponding to three clinical scenarios: "treatment improvement," "partial improvement," and "no improvement." ； Step S62: Define safety rewards based on adverse reaction monitoring : ； in, express The number of adverse reactions that occur in the patient at any given time; The fixed penalty weight representing the number of adverse reactions per unit; Step S63: Define economic rewards based on the cost of treatment actions : ； in, Indicates the execution of an action Total treatment cost; This indicates the cost per unit dose of drug; Indicates drug dosage; This indicates that the treatment method has a fixed cost; This indicates the cost per unit of treatment cycle; Indicates the treatment cycle; This represents the maximum cost threshold for routine clinical treatment. , , These are the unit costs for the drug, the method, and the cycle, respectively. Step S64: Reward based on the therapeutic effect The security reward and the aforementioned economic rewards Construct a multi-objective weighted reward function : ； in, This indicates a reward for effective treatment. Indicates a safety reward; Indicates economic reward; , , These represent the weights of the efficacy reward, safety reward, and economic reward, respectively.

8. The construction method according to claim 7, characterized in that, The definitions of the three clinical scenarios in step S61 include: Treatment improvement is defined as: a dual improvement in symptom relief and pathological cure, and estrogen levels. and the size of the lesion All are within the normal range: ； in, This indicates the estrogen level after treatment; This indicates the normal range for estrogen levels; Indicates the condition of the lesions after treatment; Indicates the normal range of the lesion; No improvement is defined as: lesion size Compared to before treatment Increased beyond a security threshold : ； in, Indicates the size of the lesion after treatment; Indicates the size of the lesion before treatment; Indicates the safety threshold; Partial improvement is defined as: failure to achieve improvement in both indicators after treatment, but no significant worsening of lesion size. ； in, It is an estrogen-improving item; This refers to the improvement of lesions; the reward value when the treatment target is met. .

9. The construction method according to claim 1, characterized in that, Step S07 specifically includes the following steps: Step S71: Utilize the current trajectory data The parameters of the policy function are updated using the policy gradient formula. : ； Step S72: Complete the trajectory The medical big data model is input, and the medical big data model learns the correspondence between "state-action-reward-new state" in the trajectory and fine-tunes its own parameters; Step S73: If Repeat steps S03 to S07 until... ;like This is the optimal treatment strategy. Output.

10. A reinforcement learning therapy system for gynecological diseases based on a large-model dual-function guided approach, characterized in that, Constructed according to the construction method of any one of claims 1 to 9.

Images