Reinforcement learning-based dual-mode gravel strategy intelligent optimization method and system and readable storage medium

Abstract

The dual-mode lithotripsy strategy intelligent optimization method based on reinforcement learning of the application comprises the following steps: collecting endoscopic visual images, lithotripsy device operating parameters and patient physiological monitoring data during lithotripsy, and performing pretreatment; segmenting and analyzing the pretreated endoscopic visual images, counting the stone fragment size distribution characteristics and calculating the distribution information entropy of the stone fragments at the current moment; calculating the entropy increase rate based on the distribution information entropy of multiple continuous moments, and integrating it with other data into state characteristic quantities; taking the state characteristic quantities as the input of the pre-constructed reinforcement learning model, outputting the mode switching instruction and the parameter adjustment amount of the corresponding mode by the reinforcement learning model; based on the mode switching instruction, the lithotripsy device is switched to the high-energy nucleation breaking mode or the low-energy uniform crushing mode, and the lithotripsy device operating parameters are adjusted according to the parameter adjustment amount, the mode is adjusted through the information entropy change of the stone fragment distribution, and the lithotripsy efficiency, lithotripsy effect and safety are considered.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare technology, and in particular to a method, system, and readable storage medium for intelligent optimization of a dual-mode lithotripsy strategy based on reinforcement learning. Background Technology

[0002] Urinary tract stones are a common urinary system disease. Endourological lithotripsy is currently the mainstream minimally invasive treatment. During the operation, the hardness, fragmentation shape and location of the stone will continuously change dynamically as the lithotripsy progresses. A single fixed lithotripsy method is difficult to adapt to the needs of the entire treatment. Clinically, multiple lithotripsy modes are usually used in combination to complete the operation based on the actual situation during the operation. The switching between modes is used to adapt to the lithotripsy needs at different stages and ensure the smooth operation of the operation.

[0003] Current lithotripsy strategies largely rely on doctors' experience to manually switch working modes, or simply switch modes based on fixed thresholds set before surgery according to static parameters such as stone size and CT values. They lack dynamic perception and intelligent decision-making regarding the real-time lithotripsy status during surgery. Mode switching is delayed and highly subjective, which can easily lead to problems such as low lithotripsy efficiency or excessive energy output. At the same time, the stone fragmentation process is not quantitatively analyzed, making it impossible to balance lithotripsy efficiency, operational stability, and surgical safety. This often results in prolonged operation time, high residual stone rate, and may also increase the risk of urinary tract tissue damage due to unreasonable mode switching or improper energy control. It is difficult to achieve adaptive and refined intelligent lithotripsy control. Summary of the Invention

[0004] In view of this, the present invention proposes a dual-mode rock-breaking strategy intelligent optimization method, system and readable storage medium based on reinforcement learning. The mode is adjusted based on the information entropy change of the fragment distribution during the rock-breaking process, so as to improve the rock-breaking efficiency while ensuring the rock-breaking effect and rock-breaking stability.

[0005] The technical solution of this invention is implemented as follows: A dual-mode intelligent optimization method for rock-breaking strategies based on reinforcement learning includes the following steps: Step S1: Collect endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data during the lithotripsy process, and perform preprocessing. Step S2: Segment and analyze the preprocessed endoscopic visual image, statistically analyze the particle size distribution characteristics of the stone fragments, and calculate the distribution information entropy of the stone fragments at the current moment. Step S3: Calculate the entropy growth rate based on the distributed information entropy at multiple consecutive time points, and integrate the operating parameters of the lithotripsy equipment, patient physiological monitoring data, distributed information entropy, and entropy growth rate as state feature quantities; Step S4: Use the state features as input to the pre-built reinforcement learning model, and let the reinforcement learning model output mode switching instructions and corresponding mode parameter adjustment amounts. Step S5: Based on the mode switching command, control the crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode, and adjust the operating parameters of the crushing equipment according to the parameter adjustment amount.

[0006] Preferably, the specific steps of step S1 are as follows: Step S11: Acquire continuous endoscopic video frame images during the lithotripsy process at a preset sampling frequency and output them as endoscopic visual images; Step S12: Collect the current output energy, impact frequency, and current working mode operating parameters of the stone crushing equipment as the operating parameters of the stone crushing equipment; Step S13: Collect intraoperative renal pelvis pressure and urinary tract mucosal temperature data as physiological monitoring data for the patient; Step S14: Align and match endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data based on timestamps, and remove outliers, invalid frames, and cached data after noise reduction and smoothing.

[0007] Preferably, the specific steps of step S2 are as follows: Step S21: Use a semantic segmentation network to perform pixel-level segmentation on the endoscopic visual image to obtain the stone fragment region and the tissue background region; Step S22: Divide the stone fragments in the stone fragment area into several particle size grades according to the preset particle size range, and count the proportion of the number of stone fragments in each particle size grade to the total number of stone fragments. Step S23: Calculate the distribution entropy of the stone fragments at the current moment based on the proportion of the number of stone fragments of each particle size class to the total number of stone fragments.

[0008] Preferably, the formula for calculating the distribution information entropy is:

[0009] in For distributed information entropy, For the number of particle size grades, For the first The proportion of stone fragments of each size class to the total number of stone fragments.

[0010] Preferably, the specific steps of step S3 are as follows: Step S31: Select a sliding time window of a certain duration and extract the distribution information entropy sequence of multiple consecutive samples sorted in chronological order within the sliding time window; Step S32: Calculate the entropy growth rate within the sliding time window based on the distributed information entropy sequence using the sliding window difference calculation method; Step S33: Integrate the distribution entropy of the stone fragments at the current moment, the entropy growth rate, the operating parameters of the lithotripsy equipment, and the patient's physiological monitoring data into a state feature quantity.

[0011] Preferably, the formula for calculating the entropy growth rate is:

[0012] in The rate of increase of entropy, This represents the total number of valid sampling points within the sliding time window. These are the distribution information entropies of the stone fragments at the sampling times of the first and last valid sampling points within the sliding time window, respectively. The time interval between adjacent valid sampling points.

[0013] Preferably, the specific steps of step S4 are as follows: Step S41: Construct the state space of the reinforcement learning model, wherein the state space includes state features; Step S42: Construct the action space of the reinforcement learning model. The action space includes discrete actions and continuous actions. The discrete actions include maintaining the current rock-breaking mode, switching to the high-energy core-breaking mode, and switching to the low-energy uniform-breaking mode. The continuous actions include the output energy adjustment amount and impact frequency adjustment amount of the high-energy core-breaking mode and the low-energy uniform-breaking mode. Step S43: Construct the reward function of the reinforcement learning model with the goals of maximizing stone crushing efficiency, ensuring the stability of the mode operation, and achieving stone crushing smoothness. Step S44: Input the state feature quantity into the reinforcement learning model, and the reinforcement learning model performs iterative optimization. After the iteration is completed, the discrete action and continuous action are output as mode switching instructions and the parameter adjustment amount of the corresponding mode.

[0014] Preferably, the expression for the reward function is:

[0015] in This is the instant reward value for a single step. This represents the difference in distribution information entropy between two adjacent valid sampling points corresponding to stone fragments. This represents the number of mode switches per unit of time. This represents the real-time volatility of the entropy growth rate. These are preset weighting coefficients.

[0016] Preferably, the formula for calculating the real-time volatility of the entropy growth rate is:

[0017] in Number the sliding time window. For the first The rate of increase of entropy corresponding to each sliding time window This represents the average entropy growth rate across all sliding time windows.

[0018] Preferably, the specific steps of step S5 are as follows: Step S51: After discretizing the mode switching command, control the crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode. Step S52: After converting the parameter adjustment amount into an electronic control signal, control the crushing equipment to adjust the operating parameters of the crushing equipment; Step S53: After the lithotripsy equipment performs its operation, a new round of endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data are collected. The state space of the reinforcement learning model is then updated after recalculating the distribution information entropy and entropy growth rate.

[0019] A dual-mode intelligent optimization system for rock-breaking strategies based on reinforcement learning includes: The multi-source data acquisition unit is used to acquire endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data during the lithotripsy process, and to perform preprocessing. The distribution information entropy calculation unit is used to segment and analyze the preprocessed endoscopic visual image, statistically analyze the particle size distribution characteristics of the stone fragments, and calculate the distribution information entropy of the stone fragments at the current moment. The entropy growth rate calculation unit is used to calculate the entropy growth rate based on the distributed information entropy at multiple consecutive time points, and integrates the operating parameters of the lithotripsy equipment, patient physiological monitoring data, distributed information entropy and entropy growth rate as state feature quantities; The rock-breaking strategy optimization unit is used to take state features as input to a pre-built reinforcement learning model, and the reinforcement learning model outputs mode switching instructions and corresponding mode parameter adjustment amounts. The stone crushing equipment adjustment unit is used to control the stone crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode based on mode switching commands, and to adjust the operating parameters of the stone crushing equipment according to the parameter adjustment amount; The multi-source data acquisition unit, the distributed information entropy calculation unit, the entropy growth rate calculation unit, the stone crushing strategy optimization unit, and the stone crushing equipment adjustment unit are sequentially connected.

[0020] A readable storage medium having a built-in computer program configured to execute a reinforcement learning-based dual-mode rock-breaking strategy intelligent optimization method.

[0021] Compared with the prior art, the beneficial effects of the present invention are: This invention relates to a dual-mode lithotripsy strategy intelligent optimization method, system, and readable storage medium based on reinforcement learning. It collects multi-source data during the lithotripsy process, reflecting the entire process state from different perspectives. After segmenting and analyzing endoscopic visual images, image data containing only stone fragments can be obtained, allowing for statistical analysis of the fragment distribution characteristics. The distribution information entropy can be calculated based on the proportion of stone fragments of different particle sizes, reflecting the degree of fragmentation. The entropy growth rate can then be calculated, characterizing the speed of stone fragmentation. Finally, the obtained data is integrated into state feature quantities, which can be input into a constructed reinforcement learning model. The model can iteratively optimize and output an optimization scheme aimed at maximizing lithotripsy efficiency, mode operation stability, and lithotripsy smoothness. The optimization scheme includes mode switching commands and corresponding parameter adjustments. Finally, the lithotripsy equipment can be adjusted and optimized based on the mode switching commands and parameter adjustments, improving operational stability and surgical safety while ensuring lithotripsy efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 The flowchart shows the intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to the present invention. Figure 2 This is a flowchart of step S1 of the intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to the present invention; Figure 3 This is a flowchart of step S2 of the intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning of the present invention; Figure 4 This is a flowchart of step S3 of the intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning of the present invention; Figure 5 This is a flowchart of step S4 of the intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning of the present invention; Figure 6 This is a flowchart of step S5 of the intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning of the present invention; Figure 7 This is a schematic diagram of the intelligent optimization system for dual-mode rock-breaking strategy based on reinforcement learning according to the present invention. In the diagram, 1 is the multi-source data acquisition unit; 2 is the distributed information entropy calculation unit; 3 is the entropy growth rate calculation unit; 4 is the stone crushing strategy optimization unit; and 5 is the stone crushing equipment adjustment unit. Detailed Implementation

[0024] To better understand the technical content of this invention, a specific embodiment is provided below, and the invention will be further described in conjunction with the accompanying drawings.

[0025] See Figures 1 to 6 The present invention provides a dual-mode rock-breaking strategy intelligent optimization method based on reinforcement learning, comprising the following steps: Step S1: Collect endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data during the lithotripsy process, and perform preprocessing. Step S2: Segment and analyze the preprocessed endoscopic visual image, statistically analyze the particle size distribution characteristics of the stone fragments, and calculate the distribution information entropy of the stone fragments at the current moment. Step S3: Calculate the entropy growth rate based on the distribution information entropy at multiple consecutive time points, and integrate the operating parameters of the lithotripsy equipment, patient physiological monitoring data, distribution information entropy, and entropy growth rate as state feature quantities; Step S4: Use the state features as input to the pre-built reinforcement learning model, and let the reinforcement learning model output mode switching instructions and corresponding mode parameter adjustment amounts. Step S5: Based on the mode switching command, control the crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode, and adjust the operating parameters of the crushing equipment according to the parameter adjustment amount.

[0026] This invention presents a dual-mode lithotripsy strategy intelligent optimization method based on reinforcement learning for the lithotripsy of urinary tract stones. The dual modes include a high-energy core-breaking mode and a low-energy uniform fragmentation mode. The high-energy core-breaking mode operates with higher output energy and lower impact frequency, mainly used to powerfully bombard the intact or dense stone body, quickly opening the main crack and breaking the core structure of the stone, achieving efficient fragmentation. It is suitable for the initial stage of lithotripsy and the stage when the stone is hard. The low-energy uniform fragmentation mode operates with lower output energy and higher impact frequency, mainly used to uniformly refine and break up stones that have already developed cracks, controlling the fragment size within an appropriate range, avoiding excessive fragmentation and escape, and reducing stimulation to surrounding tissues. It is suitable for the middle and late stages of lithotripsy and the pulverization and cleaning stage. Different lithotripsy modes need to be adopted at different stages of lithotripsy to improve lithotripsy efficiency, avoid stone fragment residue, and improve surgical safety.

[0027] During the initial lithotripsy, medical staff select the high-energy core-breaking mode based on the stone's morphology and set the operating parameters of the lithotripsy equipment. After a certain period of lithotripsy, multi-source data can be collected, including endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data. Endoscopic visual images can intuitively represent the stone's fragmentation morphology, fragment size, and distribution, directly reflecting the real-time progress and effect of lithotripsy. The lithotripsy equipment operating parameters represent the current working intensity and status of the equipment, providing a basis for mode switching and parameter adjustment. Patient physiological monitoring data can represent the safety status of the urinary tract tissues during the operation, avoiding tissue damage caused by excessive lithotripsy energy. By collecting multi-source data and preprocessing it, the entire lithotripsy process can be comprehensively and realistically depicted.

[0028] After preprocessing multi-source data, the preprocessed endoscopic visual images can be segmented and analyzed to extract image data containing stone fragments. Then, based on the particle size classification of the stone fragments, their distribution characteristics are statistically analyzed, and the distribution information entropy of the stone fragments at the current moment is calculated. This distribution information entropy corresponds to the distribution of the stone fragments and is used to determine the progress of stone fragmentation. Subsequently, based on the distribution information entropy at multiple consecutive time points, the entropy growth rate can be calculated. This entropy growth rate quantifies the efficiency of stone fragmentation per unit time. Finally, the distribution information entropy, entropy growth efficiency, lithotripsy equipment operating parameters, and patient physiological monitoring data are integrated into the state space of a reinforcement learning model. Through iterative optimization of the reinforcement learning model, two working methods can be adaptively output. The system provides mode switching commands and corresponding parameter adjustments for each mode. Based on these commands, the lithotripsy equipment can be switched to the appropriate working mode. If the equipment is already in the current mode, it remains in that mode. After adjusting the parameters for each mode, the lithotripsy process continues. Throughout the process, multi-source data is continuously collected, and the distribution information entropy and entropy growth rate are calculated. An optimization scheme is dynamically output through a reinforcement learning model to adjust the equipment's working model and operating parameters. This ensures improved lithotripsy efficiency while preventing excessive stone escape that could damage surrounding tissues, thus enhancing surgical safety. It also facilitates the aspiration and removal of stone fragments, reducing residue.

[0029] Preferably, the specific steps of step S1 are as follows: Step S11: Acquire continuous endoscopic video frame images during the lithotripsy process at a preset sampling frequency and output them as endoscopic visual images; Step S12: Collect the current output energy, impact frequency, and current working mode operating parameters of the stone crushing equipment as the operating parameters of the stone crushing equipment; Step S13: Collect intraoperative renal pelvis pressure and urinary tract mucosal temperature data as physiological monitoring data for the patient; Step S14: Align and match endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data based on timestamps, and remove outliers, invalid frames, and cached data after noise reduction and smoothing.

[0030] During lithotripsy, continuous video frames from the endoscope are acquired at a fixed preset frequency to serve as endoscopic visual images. Simultaneously, the operating parameters of the lithotripsy equipment are collected, including the current output energy, impact frequency, and operating parameters of the current working mode. The operating parameters of the current working mode include core energy parameters, pulse timing parameters, mode status parameters, and supporting system parameters. After switching modes, the adjustment amount can be calculated based on the operating parameters of the current lithotripsy equipment. In order to avoid damage to the patient's tissues during lithotripsy, it is necessary to collect the patient's physiological monitoring data during the procedure. The collected multi-source data can be accurately aligned, and then invalid abnormal data such as blurred images, equipment idle, and data abrupt changes are removed. After noise reduction and smoothing, a standardized real-time raw dataset is constructed and cached.

[0031] Preferably, the specific steps of step S2 are as follows: Step S21: Use a semantic segmentation network to perform pixel-level segmentation on the endoscopic visual image to obtain the stone fragment region and the tissue background region; Step S22: Divide the stone fragments in the stone fragment area into several particle size grades according to the preset particle size range, and count the proportion of the number of stone fragments in each particle size grade to the total number of stone fragments. Step S23: Calculate the distribution information entropy of the stone fragments at the current moment based on the proportion of the number of stone fragments of each size class to the total number of stone fragments. The formula for calculating the distribution information entropy is:

[0032] in For distributed information entropy, For the number of particle size grades, For the first The proportion of stone fragments of each size class to the total number of stone fragments, and satisfying the following conditions: .

[0033] Since the distribution characteristics of the stone fragments need to be statistically analyzed laterally, a semantic segmentation network is used to accurately segment the endoscopic visual image to obtain the stone fragment region and the tissue background region in order to improve the accuracy of feature statistics. Then, feature statistics can be performed in the stone fragment region. First, the number and particle size of all stone fragments in the stone fragment region are counted. Then, according to the preset particle size range, the stone fragments are divided into n levels. The number of stone fragments in each particle size level is counted and the proportion of stone fragments in the total stone fragments is calculated. Finally, the distribution information entropy of the stone fragments at the current moment is calculated using the distribution information entropy calculation formula. The larger the distribution information entropy value, the finer and more uniform the stone fragments are, and the higher the degree of stone fragmentation.

[0034] Preferably, the specific steps of step S3 are as follows: Step S31: Select a sliding time window of a certain duration and extract the distribution information entropy sequence of multiple consecutive samples sorted in chronological order within the sliding time window; Step S32: Calculate the entropy growth rate within the sliding time window based on the distributed information entropy sequence using the sliding window difference calculation method. The formula for calculating the entropy growth rate is:

[0035] in The rate of increase of entropy, This represents the total number of valid sampling points within the sliding time window. These are the distribution information entropies of the stone fragments at the sampling times of the first and last valid sampling points within the sliding time window, respectively. The time interval between adjacent valid sampling points; Step S33: Integrate the distribution entropy of the stone fragments at the current moment, the entropy growth rate, the operating parameters of the lithotripsy equipment, and the patient's physiological monitoring data into a state feature quantity.

[0036] After obtaining the distribution information entropy, the entropy growth rate can be calculated using the distribution information entropy sequence within a preset sliding time window. The entropy growth rate characterizes the stone fragmentation and propulsion efficiency per unit time. When the value approaches 0, it is determined that the stone fragmentation has stopped.

[0037] Preferably, the specific steps of step S4 are as follows: Step S41: Construct the state space of the reinforcement learning model, wherein the state space includes state features; Step S42: Construct the action space of the reinforcement learning model. The action space includes discrete actions and continuous actions. The discrete actions include maintaining the current rock-breaking mode, switching to the high-energy core-breaking mode, and switching to the low-energy uniform-breaking mode. The continuous actions include the output energy adjustment amount and impact frequency adjustment amount of the high-energy core-breaking mode and the low-energy uniform-breaking mode. Step S43: With the objectives of maximizing rock-breaking efficiency, ensuring operational stability of the model, and achieving stable rock-breaking performance, construct the reward function for the reinforcement learning model. The expression for the reward function is:

[0038] in This is the instant reward value for a single step. This represents the difference in distribution information entropy between two adjacent valid sampling points corresponding to stone fragments. This represents the number of mode switches per unit of time. These are preset weighting coefficients, corresponding to the weights for crushing efficiency, mode stability, and process stability, respectively, and all are positive numbers. The real-time volatility of the entropy growth rate is calculated using the following formula:

[0039] in Number the sliding time window. For the first The rate of increase of entropy corresponding to each sliding time window The average entropy growth rate across all sliding time windows is used to characterize the stone fragmentation efficiency per unit time through the real-time fluctuation rate of the entropy growth rate. The more drastic the fluctuation, the worse the stability of the stone fragmentation process; the smaller the value, the smoother and more controllable the stone fragmentation process.

[0040] Step S44: Input the state feature quantity into the reinforcement learning model, and the reinforcement learning model performs iterative optimization. After the iteration is completed, the discrete action and continuous action are output as mode switching instructions and the parameter adjustment amount of the corresponding mode.

[0041] The state space of a reinforcement learning model is the integrated state feature quantity. Its action space includes discrete and continuous actions. The discrete and continuous actions output through the action space can be used as optimization schemes. The reward function of a reinforcement learning model consists of three parts, the first part being the positive reward term. Using the increase in the information entropy of the distribution of kidney stone fragments per unit time as the core indicator, the reinforcement learning strategy is guided to prioritize the stone fragmentation process and improve the stone fragmentation efficiency per unit time. The second part is the first negative penalty term. By penalizing the number of mode switching times per unit time, frequent oscillations between the two modes are suppressed, ensuring the continuity of the crushing process and reducing efficiency losses and operational risks caused by frequent equipment start-ups and shutdowns. The third part is the second negative penalty item. By penalizing the real-time fluctuation rate of entropy growth rate, the fluctuation range of the stone crushing process is constrained, avoiding sudden changes in stone crushing efficiency, ensuring a smooth and controllable crushing process, reducing the risk of fragment loss and tissue damage, and ensuring the stability of the overall stone crushing process. The training objective of the reinforcement learning model is to maximize the cumulative reward value and achieve the optimal balance between stone crushing efficiency and process stability. Finally, the output is the mode switching command and the parameter adjustment amount in the corresponding mode.

[0042] Preferably, the specific steps of step S5 are as follows: Step S51: After discretizing the mode switching command, control the crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode. Step S52: After converting the parameter adjustment amount into an electronic control signal, control the crushing equipment to adjust the operating parameters of the crushing equipment; Step S53: After the lithotripsy equipment performs its operation, a new round of endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data are collected. The state space of the reinforcement learning model is then updated after recalculating the distribution information entropy and entropy growth rate.

[0043] The mode switching command and parameter adjustment amount can be discretely converted into electrical signals that can be recognized by the lithotripsy equipment. The lithotripsy equipment then switches modes and adjusts operating parameters. After the mode switching and parameter adjustment are completed, the lithotripsy equipment performs a new round of lithotripsy. At the same time, the steps and methods of this invention can be repeatedly executed during the lithotripsy process, and the state space of the reinforcement learning model is updated. New optimization commands are output according to preset time intervals. While ensuring lithotripsy efficiency, surgical safety is improved, and residual lithotripsy and damage to internal tissues are reduced.

[0044] Reference Figure 7 The present invention provides a dual-mode intelligent optimization system for rock-breaking strategies based on reinforcement learning, comprising: The multi-source data acquisition unit 1 is used to acquire endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data during the lithotripsy process, and to perform preprocessing. Distribution information entropy calculation unit 2 is used to segment and analyze the preprocessed endoscopic visual image, statistically analyze the particle size distribution characteristics of the stone fragments, and calculate the distribution information entropy of the stone fragments at the current moment. Entropy growth rate calculation unit 3 is used to calculate the entropy growth rate based on the distributed information entropy at multiple consecutive time points, and integrates the operating parameters of the lithotripsy equipment, patient physiological monitoring data, distributed information entropy and entropy growth rate as state feature quantities; The rock-breaking strategy optimization unit 4 is used to take the state features as input to the pre-built reinforcement learning model, and the reinforcement learning model outputs mode switching instructions and corresponding mode parameter adjustment amounts. The stone crushing equipment adjustment unit 5 is used to control the stone crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode based on the mode switching command, and to adjust the operating parameters of the stone crushing equipment according to the parameter adjustment amount; The multi-source data acquisition unit 1, the distributed information entropy calculation unit 2, the entropy growth rate calculation unit 3, the stone crushing strategy optimization unit 4, and the stone crushing equipment adjustment unit 5 are connected in sequence.

[0045] A readable storage medium having a built-in computer program configured to execute a reinforcement learning-based dual-mode rock-breaking strategy intelligent optimization method.

[0046] The specific steps of the reinforcement learning-based dual-mode rock-breaking strategy intelligent optimization system and the readable storage medium of the present invention for executing the reinforcement learning-based dual-mode rock-breaking strategy intelligent optimization method are therefore not described in detail here.

[0047] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dual-mode intelligent optimization method for rock-breaking strategies based on reinforcement learning, characterized in that, Includes the following steps: Step S1: Collect endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data during the lithotripsy process, and perform preprocessing. Step S2: Segment and analyze the preprocessed endoscopic visual image, statistically analyze the particle size distribution characteristics of the stone fragments, and calculate the distribution information entropy of the stone fragments at the current moment. Step S3: Calculate the entropy growth rate based on the distribution information entropy at multiple consecutive time points, and integrate the operating parameters of the lithotripsy equipment, patient physiological monitoring data, distribution information entropy, and entropy growth rate as state feature quantities; Step S4: Use the state features as input to the pre-built reinforcement learning model, and let the reinforcement learning model output mode switching instructions and corresponding mode parameter adjustment amounts. Step S5: Based on the mode switching command, control the crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode, and adjust the operating parameters of the crushing equipment according to the parameter adjustment amount.

2. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 1, characterized in that, The specific steps of step S1 are as follows: Step S11: Acquire continuous endoscopic video frame images during the lithotripsy process at a preset sampling frequency and output them as endoscopic visual images; Step S12: Collect the current output energy, impact frequency, and current working mode operating parameters of the stone crushing equipment as the operating parameters of the stone crushing equipment; Step S13: Collect intraoperative renal pelvis pressure and urinary tract mucosal temperature data as physiological monitoring data for the patient; Step S14: Align and match endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data based on timestamps, and remove outliers, invalid frames, and cached data after noise reduction and smoothing.

3. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S21: Use a semantic segmentation network to perform pixel-level segmentation of the endoscopic visual image to obtain the stone fragment region and the tissue background region; Step S22: Divide the stone fragments in the stone fragment area into several particle size grades according to the preset particle size range, and count the proportion of the number of stone fragments in each particle size grade to the total number of stone fragments. Step S23: Calculate the distribution entropy of the stone fragments at the current moment based on the proportion of the number of stone fragments of each size class to the total number of stone fragments; The formula for calculating the distribution information entropy is: in For distributed information entropy, For the number of particle size grades, For the first The proportion of stone fragments of each size class to the total number of stone fragments.

4. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S31: Select a sliding time window of a certain duration and extract the distribution information entropy sequence of multiple consecutive samples sorted in chronological order within the sliding time window; Step S32: Calculate the entropy growth rate within the sliding time window based on the distributed information entropy sequence using the sliding window difference calculation method; Step S33: Integrate the distribution entropy of the stone fragments at the current moment, the entropy growth rate, the operating parameters of the lithotripsy equipment, and the patient's physiological monitoring data into a state feature quantity.

5. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 4, characterized in that, The formula for calculating the entropy growth rate is: in The rate of increase of entropy, This represents the total number of valid sampling points within the sliding time window. These are the distribution information entropies of the stone fragments at the sampling times of the first and last valid sampling points within the sliding time window, respectively. The time interval between adjacent valid sampling points.

6. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 5, characterized in that, The specific steps of step S4 are as follows: Step S41: Construct the state space of the reinforcement learning model, wherein the state space includes state features; Step S42: Construct the action space of the reinforcement learning model. The action space includes discrete actions and continuous actions. The discrete actions include maintaining the current rock-breaking mode, switching to the high-energy core-breaking mode, and switching to the low-energy uniform-breaking mode. The continuous actions include the output energy adjustment amount and impact frequency adjustment amount of the high-energy core-breaking mode and the low-energy uniform-breaking mode. Step S43: Construct the reward function of the reinforcement learning model with the goals of maximizing stone crushing efficiency, ensuring the stability of the mode operation, and achieving stone crushing smoothness; Step S44: Input the state feature quantity into the reinforcement learning model, and the reinforcement learning model performs iterative optimization. After the iteration is completed, the discrete action and continuous action are output as mode switching instructions and the parameter adjustment amount of the corresponding mode.

7. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 6, characterized in that, The expression for the reward function is: in This is the instant reward value for a single step. This represents the difference in distribution information entropy between two adjacent valid sampling points corresponding to stone fragments. This represents the number of mode switches per unit of time. For preset weighting coefficients, The real-time volatility of the entropy growth rate is calculated using the following formula: in Number the sliding time window. For the first The rate of increase of entropy corresponding to each sliding time window This represents the average entropy growth rate across all sliding time windows.

8. The intelligent optimization method for dual-mode rock-breaking strategy based on reinforcement learning according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S51: After discretizing the mode switching command, control the crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode. Step S52: After converting the parameter adjustment amount into an electronic control signal, control the crushing equipment to adjust the operating parameters of the crushing equipment; Step S53: After the lithotripsy equipment performs its operation, a new round of endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data are collected. The state space of the reinforcement learning model is then updated after recalculating the distribution information entropy and entropy growth rate.

9. A system applying the dual-mode rock-breaking strategy intelligent optimization method based on reinforcement learning as described in any one of claims 1-8, characterized in that, include: The multi-source data acquisition unit is used to acquire endoscopic visual images, lithotripsy equipment operating parameters, and patient physiological monitoring data during the lithotripsy process, and to perform preprocessing. The distribution information entropy calculation unit is used to segment and analyze the preprocessed endoscopic visual image, statistically analyze the particle size distribution characteristics of the stone fragments, and calculate the distribution information entropy of the stone fragments at the current moment. The entropy growth rate calculation unit is used to calculate the entropy growth rate based on the distributed information entropy at multiple consecutive time points, and integrates the operating parameters of the lithotripsy equipment, patient physiological monitoring data, distributed information entropy and entropy growth rate as state feature quantities; The rock-breaking strategy optimization unit is used to take state features as input to a pre-built reinforcement learning model, and the reinforcement learning model outputs mode switching instructions and corresponding mode parameter adjustment amounts. The stone crushing equipment adjustment unit is used to control the stone crushing equipment to switch to high-energy core crushing mode or low-energy uniform crushing mode based on mode switching commands, and to adjust the operating parameters of the stone crushing equipment according to the parameter adjustment amount; The multi-source data acquisition unit, the distributed information entropy calculation unit, the entropy growth rate calculation unit, the stone crushing strategy optimization unit, and the stone crushing equipment adjustment unit are sequentially connected.

10. A readable storage medium, characterized in that, The readable storage medium contains a built-in computer program configured to execute the intelligent optimization method for a dual-mode rock-breaking strategy based on reinforcement learning as described in any one of claims 1-8.

Images