Intelligent warming device for intraoperative patient and collaborative control method and system thereof

Abstract

The application provides an intelligent warming device cooperative control method and system for an intraoperative patient, relates to the technical field of cooperative control, and comprises the following steps: acquiring patient multi-site body surface temperature data to calculate pheromone values representing temperature urgency; constructing a deep reinforcement learning intelligent agent, taking the normalized pheromone values as attention prior distribution, fusing the network self-learning weight, generating fusion attention weight to weight aggregate features and make decisions on the power distribution of each warming unit; introducing a virtual repulsive force penalty term based on the target power and the pheromone priority in the loss function to constrain the upper limit of the total power during training, and adaptively adjusting the pheromone calculation parameters. The application realizes precise, cooperative and energy-saving control of differentiated warming needs of multiple sites.

Description

Technical Field

[0001] This invention relates to collaborative control technology, and more particularly to a collaborative control method and system for intelligent warming devices for patients during surgery. Background Technology

[0002] Maintaining stable patient body temperature during surgery is crucial for ensuring surgical safety and postoperative recovery. Current technologies employ rule-based collaborative control strategies, such as allocating a fixed total power budget among different heating units according to a preset priority order or a simple weighted average algorithm. These methods simplify the patient's temperature maintenance problem into multiple independent temperature closed-loop control problems, or rely on static allocation rules set by human experience.

[0003] Because the body's thermoregulation mechanism is impaired under anesthesia, and different body parts exhibit significant differences in heat loss rates, sensitivity to low temperatures, and thermal inertia, independent control strategies relying solely on local temperature feedback are insufficient to achieve coordinated and optimal regulation of overall body temperature. Each heating unit operates independently to pursue its own control objectives, easily leading to unnecessary fluctuations in total power consumption and frequent exceedances of the power supply limit, potentially causing overload risks. Furthermore, it cannot dynamically adjust the warming priority of each area based on the patient's real-time, global body temperature status. In addition, power allocation methods based on fixed rules lack adaptability and cannot learn and respond to complex, non-linear dynamic changes in patient body temperature. When dealing with sudden rapid drops in body temperature or conflicting temperature requirements in different areas, their control effect is often poor, making it difficult to achieve precise, efficient, and safe coordinated warming while meeting global constraints. Summary of the Invention

[0004] This invention provides a method and system for the coordinated control of intelligent warming devices for patients during surgery, which can solve the problems in the prior art.

[0005] A first aspect of the present invention provides a collaborative control method for an intelligent warming device for patients during surgery, comprising: Intraoperative surface temperature data of multiple sites on the patient were acquired, and pheromone values ​​for each site were calculated. The pheromone values, combined with temperature deviation and temperature change rate, characterize the temperature urgency of the corresponding site. A deep reinforcement learning agent is constructed, and the pheromone values ​​are normalized as the prior distribution of attention weights. This distribution is then weighted and fused with the self-learned attention weights of the neural network to generate fused attention weights. The fused attention weights are then used to weight and aggregate features from different parts to generate the power allocation actions for each heating device unit. During the training phase, a virtual repulsion penalty term is introduced into the loss function. When the sum of the target power values ​​of all device units exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power values ​​of each device unit and the priority determined by the pheromone values, and the virtual repulsion penalty term is accumulated. The deep reinforcement learning agent is then updated based on the loss function. The calculation parameters for the pheromone value are adaptively adjusted based on the difference between the prior distribution and the self-learning attention weights.

[0006] The steps for acquiring intraoperative surface temperature data from multiple sites on the patient and calculating pheromone values ​​at each site include: The patient's surface temperature data at multiple sites during surgery is acquired through a distributed temperature sensing unit. Temperature deviation and temperature change rate are calculated for each region. The temperature deviation is the difference between the current temperature and the target temperature, and the temperature change rate is the ratio of the temperature change to the time interval between two adjacent acquisition cycles. The initial pheromone value is obtained by weighting and summing the temperature deviation and temperature change rate after assigning initial weighting coefficients to them respectively. The magnitude of the initial pheromone value quantifies the temperature urgency, and the initial weighting coefficients reflect the relative contribution of temperature deviation and temperature change rate to the temperature urgency. The initial pheromone value is subjected to time decay processing. By introducing a decay factor, the pheromone value at historical moments gradually decreases over time. The initial pheromone value at the current moment is added to the decayed historical pheromone value to obtain the pheromone value at the current moment. The decay factor controls the degree of influence of historical pheromone values ​​on the current decision.

[0007] The steps of constructing a deep reinforcement learning agent, normalizing the pheromone values ​​as the prior distribution of attention weights, and weightedly fusing them with the self-learned attention weights of the neural network to generate fused attention weights include: The pheromone values ​​of each part are normalized to obtain the prior distribution, and the sum of the prior weights of each part in the prior distribution is a unit value. The temperature state features and historical power distribution features of each part are extracted by the neural network encoding module in the deep reinforcement learning agent, and then input into the attention calculation module to generate self-learning attention weights and normalize them. Calculate the consistency metric between the prior distribution and the self-learning attention weights. Subtract the consistency metric from the unit value to obtain the fusion coefficient of the prior distribution. Use the consistency metric as the fusion coefficient of the self-learning attention weights. Then, sum the weights of the prior distribution and the self-learning attention weights using the corresponding fusion coefficients to obtain the preliminary fusion weights. The initial fusion weights are gated and modulated. When the temperature deviation or temperature change rate of each part exceeds the preset condition, the gate value of the corresponding part is set to an enhancement coefficient greater than the unit value. The initial fusion weights are multiplied by the corresponding gate value and normalized to obtain the fusion attention weights.

[0008] The steps for generating the power distribution actions of each insulation device unit after weighted aggregation of features from different parts using the fused attention weights include: Temperature state characteristics, temperature change characteristics, and power state characteristics are extracted for each part. The power state characteristics include the current power value and power change rate of the corresponding heat preservation device unit. The temperature state features, temperature change features, and power state features of each part are weighted and aggregated using the fusion attention weights to generate global temperature state features, global temperature change features, and global power state features. The global features of temperature state, global features of temperature change, and global features of power state are concatenated to form a comprehensive global feature vector, which is then input into the decision network of the deep reinforcement learning agent. The output layer of the decision network generates the power adjustment amount for each heat preservation device unit; the power adjustment amount is added to the current power value of the device unit to obtain the target power value, and the activation function of the output layer constrains the power adjustment amount of each device unit within a preset range.

[0009] During the training phase, a virtual repulsion penalty term is introduced into the loss function. When the sum of the target power values ​​of all device units exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power values ​​of each device unit and the priority determined by the pheromone values, and the virtual repulsion penalty term is accumulated. The steps for updating the deep reinforcement learning agent based on the loss function include: After performing the power allocation action in each training iteration cycle, an instantaneous reward signal is calculated based on the measured body surface temperature data. The pheromone values ​​of each part are normalized to obtain the priority weight of each part. The priority weight is used to characterize the priority order of each device unit in resource allocation. The sum of the target power values ​​of each insulation device unit is calculated as the predicted total power. When the predicted total power exceeds the supply limit, a virtual repulsion force value is calculated for any two insulation device units. The virtual repulsion force value is directly proportional to the product of the target power values ​​of the two device units and inversely proportional to the absolute value of the difference in priority weights between the two device units. The virtual repulsion force values ​​of all device unit pairs are accumulated to obtain the total virtual repulsion force value. The magnitude by which the predicted total power exceeds the supply limit is multiplied by the total virtual repulsion force value to obtain a virtual repulsion force penalty term. The policy loss term and value loss term are calculated based on the instant reward signal; the policy loss term, value loss term, and virtual repulsion penalty term are weighted and summed to form the total loss function; the gradient of the total loss function with respect to the network parameters is calculated through backpropagation, and gradient descent is performed to update the deep reinforcement learning agent.

[0010] The steps for calculating the instant reward signal include: The difference between the current temperature deviation and the temperature deviation of the previous cycle for each part is calculated as the temperature improvement amount; the temperature improvement amount for each part is multiplied by the corresponding priority weight and then summed to obtain the priority-weighted temperature reward. The power constraint penalty is calculated based on the difference between the predicted total power and the supply ceiling; Calculate the change amplitude of the target power value of each device unit in adjacent control cycles and sum them to obtain the total power fluctuation. Calculate the smoothness penalty based on the total power fluctuation. The priority-weighted temperature reward, the power constraint penalty, and the smoothness penalty are combined in a weighted manner to obtain an instant reward signal.

[0011] The step of adaptively adjusting the calculation parameters of the pheromone value based on the difference between the prior distribution and the self-learning attention weights includes: The difference between the prior distribution of each part and the self-learning attention weight is calculated to obtain the local difference value. The absolute values ​​of the local difference values ​​of all parts are averaged to obtain the overall difference measure value. When the overall difference measurement value exceeds the preset difference threshold, the number of parts with positive local difference values ​​and the number of parts with negative local difference values ​​are counted, and the ratio of the absolute value of the difference between the two values ​​to the number of headquarters is calculated as the deviation asymmetry. If the deviation asymmetry exceeds the symmetry threshold, the initial weighting coefficients of temperature deviation and temperature change rate in the pheromone value calculation formula are adjusted. If the deviation asymmetry does not exceed the symmetry threshold, the attenuation factor of historical pheromone values ​​in the pheromone value calculation formula is adjusted. When the overall difference metric does not exceed the preset difference threshold, the calculation parameters of the pheromone value remain unchanged.

[0012] A second aspect of the present invention provides a collaborative control system for an intelligent warming device for patients during surgery, comprising: The pheromone calculation module is used to acquire the surface temperature data of multiple parts of the patient during surgery and calculate the pheromone value of each part. The pheromone value, combined with the temperature deviation and temperature change rate, represents the temperature urgency of the corresponding part. The decision generation module is used to construct a deep reinforcement learning agent. It normalizes the pheromone values ​​as the prior distribution of attention weights and weights them with the self-learned attention weights of the neural network to generate fused attention weights. The fused attention weights are used to weight and aggregate the features of different parts to generate the power allocation actions of each heating device unit. The model training module is used to introduce a virtual repulsion penalty term into the loss function during the training phase. When the sum of the target power values ​​of each device unit exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power value of each device unit and the priority determined by the pheromone value, and the virtual repulsion penalty term is accumulated. The deep reinforcement learning agent is updated based on the loss function. The parameter adaptive adjustment module is used to adaptively adjust the calculation parameters of the pheromone value based on the difference between the prior distribution and the self-learning attention weight.

[0013] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] This method's deep reinforcement learning agent design effectively integrates prior knowledge and autonomous learning capabilities. Pheromones are normalized and used as the prior distribution of attention weights, guiding the model to quickly focus on key areas. These pheromone values ​​are then weighted and fused with the attention weights learned by the neural network to generate the final fused attention weights. This mechanism utilizes expert prior knowledge based on physiological rules to ensure the immediacy and safety of control, while also giving the model the flexibility to learn complex, nonlinear relationships from data. This allows for better weighted aggregation of features from different areas, and based on this, generates refined power allocation actions for each insulation unit.

[0016] The virtual repulsion penalty term introduced during training is crucial for ensuring collaborative operation among multiple devices and managing system resource constraints. When the total target power of all device units exceeds the supply limit, this penalty term calculates and accumulates the virtual repulsion force between units based on their target power and priority determined by pheromone values. This forces the agent to actively learn during training how to dynamically prioritize and optimize power allocation based on the urgency of each component within a limited total power budget. This avoids ineffective power competition or excessive concentration, achieving efficient and rational allocation of system resources at the global level. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the collaborative control method of the intelligent warming device for intraoperative patients according to an embodiment of the present invention; Figure 2 Update the flowchart for the deep reinforcement learning agent during the training phase. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0020] Figure 1 This is a flowchart illustrating the collaborative control method of an intelligent warming device for intraoperative patients according to an embodiment of the present invention. Figure 1 As shown, the method includes: Intraoperative surface temperature data of multiple sites on the patient were acquired, and pheromone values ​​for each site were calculated. The pheromone values, combined with temperature deviation and temperature change rate, characterize the temperature urgency of the corresponding site. A deep reinforcement learning agent is constructed, and the pheromone values ​​are normalized as the prior distribution of attention weights, which are then weighted and fused with the self-learned attention weights of the neural network to generate fused attention weights. The power distribution actions of each insulation device unit are generated by weighting and aggregating features from different parts using the fused attention weights. During the training phase, a virtual repulsion penalty term is introduced into the loss function. When the sum of the target power values ​​of each device unit exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power value of each device unit and the priority determined by the pheromone value, and the virtual repulsion penalty term is accumulated. The deep reinforcement learning agent is updated based on the loss function. The calculation parameters for the pheromone value are adaptively adjusted based on the difference between the prior distribution and the self-learning attention weights.

[0021] In one optional implementation, the steps of acquiring intraoperative surface temperature data from multiple sites on the patient and calculating pheromone values ​​for each site include: The patient's surface temperature data at multiple sites during surgery is acquired through a distributed temperature sensing unit. Temperature deviation and temperature change rate are calculated for each region. The temperature deviation is the difference between the current temperature and the target temperature, and the temperature change rate is the ratio of the temperature change to the time interval between two adjacent acquisition cycles. The initial pheromone value is obtained by weighting and summing the temperature deviation and temperature change rate after assigning initial weighting coefficients to them respectively. The magnitude of the initial pheromone value quantifies the temperature urgency, and the initial weighting coefficients reflect the relative contribution of temperature deviation and temperature change rate to the temperature urgency. The initial pheromone value is subjected to time decay processing. By introducing a decay factor, the pheromone value at historical moments gradually decreases over time. The initial pheromone value at the current moment is added to the decayed historical pheromone value to obtain the pheromone value at the current moment. The decay factor controls the degree of influence of historical pheromone values ​​on the current decision.

[0022] For example, during surgery, multiple non-contact infrared temperature sensing units or attached thermocouple sensors are placed on key areas such as the patient's forehead, chest, abdomen, and limbs to acquire surface temperature data of each area in real time with a sampling period of 1 to 5 seconds. Each sensing unit transmits the measured temperature value to the central controller for processing via a wireless communication module.

[0023] For each monitoring site, the temperature deviation is calculated by subtracting the currently measured temperature from the preset target temperature (usually set between 36.5℃ and 37.0℃). For example, when the measured temperature on the forehead is 35.2℃ and the target temperature is 36.8℃, the temperature deviation is -1.6℃. The rate of temperature change is calculated by extracting temperature data from two adjacent acquisition cycles. If the abdominal temperature is 35.8℃ at time t-1 and 35.5℃ at time t, with a time interval of 2 seconds, then the rate of temperature change is (35.5-35.8) / 2 = -0.15℃ / second. A negative value indicates a decreasing temperature trend.

[0024] The initial pheromone value is calculated using a weighted summation method. Let α be the initial weighting coefficient for temperature deviation and β be the initial weighting coefficient for the rate of temperature change. The initial values ​​of α and β can be set to 0.6 and 0.4 respectively, indicating that the contribution of temperature deviation to the urgency is slightly higher than that of the rate of temperature change. For a certain location, if the absolute value of the temperature deviation is 1.5℃ and the absolute value of the rate of temperature change is 0.2℃ / second, then the initial pheromone value for that location is calculated as 0.6 × 1.5 + 0.4 × 0.2 = 0.98. A larger value indicates a more urgent need for warmth in that location.

[0025] The time-decrease processing introduces a decay factor ρ, with a value ranging from 0.8 to 0.95. In each new acquisition cycle, the pheromone value from the previous moment is multiplied by the decay factor ρ to achieve decay. Specifically, if the pheromone value of a certain region at moment t-1 is 1.2, and the decay factor is set to 0.9, then the decayed historical pheromone value is 1.2 × 0.9 = 1.08. This decayed value is then added to the newly calculated initial pheromone value at moment t (assumed to be 0.85) to obtain the final pheromone value at moment t: 1.08 + 0.85 = 1.93. This accumulation mechanism ensures that the pheromone value of regions that remain at low temperatures continuously accumulates, guaranteeing them higher processing priority.

[0026] In scenarios where multiple body parts are monitored simultaneously, the pheromone values ​​of each part form a numerical vector. When the pheromone value for the chest is 2.1, for the abdomen 1.5, and for the left upper limb 0.8, this vector intuitively reflects the differences in the urgency of keeping warm in different parts. The setting of the attenuation factor needs to balance the ability to remember historical states with the response speed to current changes. A larger attenuation factor (such as 0.95) makes the system pay more attention to the historical cumulative effect, while a smaller attenuation factor (such as 0.8) focuses more on the temperature state at the current moment.

[0027] This pheromone mechanism can quantify the urgency of the insulation needs of different parts of the body, providing a priority basis for subsequent power allocation decisions.

[0028] In one optional implementation, the step of constructing a deep reinforcement learning agent, normalizing the pheromone values ​​as a prior distribution of attention weights, and weightedly fusing them with the self-learned attention weights of the neural network to generate fused attention weights includes: The pheromone values ​​of each part are normalized to obtain the prior distribution, and the sum of the prior weights of each part in the prior distribution is a unit value. The temperature state features and historical power distribution features of each part are extracted by the neural network encoding module in the deep reinforcement learning agent, and then input into the attention calculation module to generate self-learning attention weights and normalize them. Calculate the consistency metric between the prior distribution and the self-learning attention weights, subtract the consistency metric from 1 to obtain the fusion coefficient of the prior distribution, and use the consistency metric as the fusion coefficient of the self-learning attention weights. The preliminary fusion weights are obtained by weighting the prior distribution and the self-learning attention weights separately using the corresponding fusion coefficients and then summing them. The initial fusion weights are gated and modulated. When the temperature deviation or temperature change rate of each part exceeds the preset condition, the gate value of the corresponding part is set to an enhancement coefficient greater than the unit value. The initial fusion weights are multiplied by the corresponding gate value and normalized to obtain the fusion attention weights.

[0029] For example, suppose there are N monitoring sites, and the pheromone values ​​of each site are denoted as the pheromone values ​​of sites 1 to N. Normalization is achieved by dividing by the sum of the pheromone values ​​of all sites, that is, the prior weight of site i is equal to the pheromone value of that site divided by the sum of the pheromone values ​​of all sites, ensuring that the sum of all prior weights equals 1. This prior distribution reflects the initial attention allocation based on the urgency of temperature.

[0030] The deep reinforcement learning agent employs an Actor-Critic architecture, comprising a policy network and a value network. The policy network outputs power allocation actions, while the value network evaluates the value of the current state. The value network uses a three-layer fully connected structure with 128, 64, and 1 neurons in each layer, and the last layer outputs the value estimate of the current state. The neural network encoding module receives raw state data from various body parts. Temperature state features include the current surface temperature, the deviation from the target temperature, and the rate of temperature change over the last five time steps. Historical power allocation features record the power applied to the body part by each heating device unit in the previous three to five time steps. The encoding module uses a three-layer fully connected network with 128, 256, and 128 neurons in each layer, and ReLU activation function. These heterogeneous features are mapped to a unified latent space representation, generating a 128-dimensional feature vector. The attention calculation module then utilizes these feature vectors and employs a multi-head attention mechanism, setting up four attention heads. The query, key, and value vectors of each attention head are set to a dimension of 32. The attention score is obtained by scaling the query vector and key vector by taking the dot product and dividing by the square root of the dimension, followed by softmax normalization. The score is then weighted and summed with the value vector. The outputs of each attention head are concatenated and passed through a linear transformation layer to generate self-learning attention weights for each part, ensuring that the sum of the weights for all parts equals 1.

[0031] To integrate the prior distribution and the self-learning attention weights, the degree of consistency between the two needs to be quantified. Cosine similarity is used as the consistency metric. It is calculated by multiplying the prior weights of each part by their corresponding self-learning attention weights, summing the results, and then dividing by the product of the magnitudes of the two weight vectors. Since both the prior distribution and the self-learning attention weights are normalized, the denominator is always 1, simplifying the consistency metric to the sum of the products of the weights at each corresponding position. When the two weight distributions are highly consistent, this value is close to 1; when the distributions differ significantly, the value decreases. Subtracting the consistency metric from 1 yields the fusion coefficient for the prior distribution, while the consistency metric itself serves as the fusion coefficient for the self-learning attention weights. This dynamic coefficient allocation mechanism ensures that when the agent's self-learning judgment deviates significantly from the prior temperature urgency, it relies more heavily on the guidance of the prior distribution; when the two converge, it increases trust in the agent's autonomous learning results. The initial fusion weights are obtained by weighted summation, that is, the initial fusion weight of each part is equal to the prior weight of that part multiplied by the prior fusion coefficient plus the self-learning weight of that part multiplied by the self-learning fusion coefficient.

[0032] The initial fusion weights are further processed by a gating modulation module to handle emergency situations. Gating conditions are set for each part, specifically including a temperature deviation threshold and a temperature change rate threshold. When the absolute value of the deviation between the actual temperature and the target temperature of a part exceeds a set threshold (e.g., 1.5 degrees Celsius), or the temperature change rate exceeds a threshold (e.g., a decrease of 0.01 degrees Celsius per second), the gating value for that part is set as an enhancement coefficient, typically ranging from 1.2 to 1.5. Parts without triggered conditions retain a gating value of 1. The initial fusion weights are multiplied element-wise by the corresponding gating values ​​to obtain the modulated weights; that is, the modulated weight of each part equals the initial fusion weight of that part multiplied by the corresponding gating value. Since gating enhancement violates the constraint that the total weight sum is 1, normalization is required again. Finally, the fused attention weight equals the modulated weight of each part divided by the sum of the modulated weights of all parts. This gating mechanism ensures that when abnormal temperature deterioration is detected, even if the agent's self-learning attention does not respond sufficiently, attention to that part can still be forcibly increased, enhancing the robustness and safety of control.

[0033] By integrating the aforementioned prior distribution with self-learning attention, it is possible to balance the empirical prior based on temperature urgency with the dynamic adaptability of the agent's autonomous learning. While ensuring priority warmth for critical areas, it allows the agent to adjust its attention allocation strategy according to the surgical progress and individual patient differences, thereby improving the accuracy and adaptability of power allocation.

[0034] In one optional implementation, the step of generating the power allocation action of each heat preservation device unit by weighting and aggregating the features of different parts using the fused attention weights includes: Temperature state characteristics, temperature change characteristics, and power state characteristics are extracted for each part. The power state characteristics include the current power value and power change rate of the corresponding heat preservation device unit. The temperature state features, temperature change features, and power state features of each part are weighted and aggregated using the fusion attention weights to generate global temperature state features, global temperature change features, and global power state features. The global features of temperature state, global features of temperature change, and global features of power state are concatenated to form a comprehensive global feature vector, which is then input into the decision network of the deep reinforcement learning agent. The power adjustment amount of each heat preservation device unit is generated through the output layer of the decision network; The target power value is obtained by adding the power adjustment amount to the current power value of the device unit, and the activation function of the output layer constrains the power adjustment amount of each device unit within a preset range.

[0035] For example, multi-dimensional feature representations are extracted for each monitoring site. Temperature state features include the current surface temperature, the deviation from the target temperature, the normalized temperature, and the historical average temperature of that site over the last five time steps. Temperature change features include the temperature difference between adjacent time steps, the rate of temperature change, and the derivative of the rate of temperature change, used to capture temperature change trends. Power state features include the current power value of the corresponding heating device unit, the proportion of power to the total supply limit, the power change rate over the last five time steps, and the cumulative operating time. A convolutional neural network layer extracts local features from the temperature sequence of each site, resulting in a 64-dimensional temperature state feature vector; a recurrent neural network layer captures the temporal dependence of temperature changes, generating a 32-dimensional temperature change feature vector; and the power state features are mapped to a 32-dimensional power state feature vector through a fully connected layer.

[0036] The features extracted from each monitoring site are weighted and aggregated using the aforementioned fusion attention weights. There are N monitoring sites, and the fusion attention weights for each site satisfy a normalization condition, meaning the sum of the weights for all sites is 1. For temperature state features, the temperature state feature vectors of each site are weighted and summed according to their corresponding fusion attention weights to generate a global temperature state feature vector with a dimension of 64. The generation methods for global temperature change features and global power state features are similar, resulting in feature vectors with a dimension of 32. This weighted aggregation process ensures that sites with higher temperature urgency receive a larger weight in the global features, guaranteeing priority response to key areas in decision-making.

[0037] Three types of global feature vectors are concatenated sequentially to form a comprehensive global feature vector of dimension 128, which is then input into the decision network of the deep reinforcement learning agent. The decision network employs a multilayer perceptron structure, containing two hidden layers with 256 and 128 nodes respectively, using ReLU activation. Batch normalization layers and dropout layers are added between the hidden layers, with a dropout rate set to 0.3 to prevent overfitting. The number of nodes in the output layer of the decision network equals the number of heating device units, and each node outputs the power adjustment amount corresponding to that of its respective device unit. The output layer activation function uses the hyperbolic tangent function, limiting the output value to between -1 and +1, and then multiplying it by a preset maximum power adjustment range, such as 50 watts, to obtain the actual power adjustment amount.

[0038] After obtaining the power adjustment amount for each device unit, this adjustment amount is added to the current power value of the corresponding device unit to obtain the target power value. The target power value is truncated to ensure it falls within the physical power range of the device unit, typically between 0 and 200 watts. By constraining the activation function and limiting the maximum adjustment range, temperature fluctuations caused by excessively drastic power adjustments are avoided, achieving a smooth power allocation strategy. The entire feature extraction, weighted aggregation, and decision generation process is completed within each control cycle, which is set to 30 seconds to ensure rapid response to changes in patient body temperature.

[0039] This invention utilizes the synergistic effect of multidimensional feature extraction and attention-weighted aggregation to enable the decision network to comprehensively consider the temperature status, trend of change, and power history of each part, and combine urgency priority to achieve precise power allocation, thereby improving the targeting and efficiency of thermal insulation control.

[0040] In one optional implementation, during the training phase, a virtual repulsion penalty term is introduced into the loss function. When the sum of the target power values ​​of all device units exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power values ​​of each device unit and the priority determined by the pheromone values, and the virtual repulsion penalty term is accumulated. The step of updating the deep reinforcement learning agent based on the loss function includes: After performing the power allocation action in each training iteration cycle, the temperature reward component is calculated based on the measured body surface temperature data. The pheromone values ​​of each part are normalized to obtain the priority weight of each part. The priority weight is used to characterize the priority order of each device unit in resource allocation. The sum of the target power values ​​of each heat preservation device unit is calculated as the predicted total power. When the predicted total power exceeds the supply limit, a virtual repulsion force value is calculated for any two heat preservation device units. The virtual repulsion force value is directly proportional to the product of the target power values ​​of the two device units and inversely proportional to the absolute value of the difference between the priority weights of the two device units. The virtual repulsion force values ​​of all device unit pairs are summed to obtain the total virtual repulsion force value. The magnitude by which the predicted total power exceeds the supply limit is multiplied by the total virtual repulsion force value to obtain the virtual repulsion force penalty term. Based on the temperature reward component, calculate the strategy loss item and the value loss item; The total loss function is constructed by weighted summing of the policy loss term, value loss term, and virtual repulsion penalty term. The gradient of the total loss function with respect to the network parameters is calculated through backpropagation, and gradient descent is performed to update the deep reinforcement learning agent.

[0041] Combination Figure 2 The training phase update flowchart of the deep reinforcement learning agent is used for illustration. For example, at the beginning of each training iteration during agent training, the deep reinforcement learning agent outputs and executes the power allocation actions of each heating device unit based on the current state. Immediately after execution, measured data of the surface temperature of each part of the body are collected, and the measured temperature is compared with the target temperature range to calculate the temperature reward component. The calculation method for this temperature reward component is as follows: if the temperature of a part of the body enters the target range of 36.5 to 37.0 degrees Celsius, a positive reward of 10 points is given; for every 0.1 degrees Celsius the temperature deviates from the target range, 2 points are deducted. Simultaneously, considering the temperature change trend, if the temperature is moving closer to the target, an additional 1 point is added per second for every 0.1 degrees Celsius increase in the absolute value of the temperature change rate.

[0042] To achieve reasonable allocation in resource-constrained scenarios, the pheromone values ​​of each component are normalized using a softmax function to obtain the priority weight for each component. This priority weight ranges from 0 to 1, and the sum of the weights for all components is 1. A higher value indicates a higher allocation priority for that component in resource competition. The sum of the target power values ​​of all insulation components is calculated as the predicted total power. When the predicted total power does not exceed the supply limit, the virtual repulsion mechanism is not triggered; when the predicted total power exceeds the supply limit, a virtual repulsion penalty term needs to be calculated.

[0043] Specifically, for any two insulation unit units i and j, their virtual repulsion force values ​​are calculated using the following formula: ; Among them, P i and P j These are the target power values ​​for the two device units, w i and w jHere, k represents the corresponding priority weights, and k is a repulsion coefficient constant ranging from 0.01 to 0.1. When the difference in priority weights between two device units is less than 0.01, the difference is set to 0.01 to avoid division by zero errors. This calculation method indicates that the repulsion between device units with larger target power values ​​is stronger, while the repulsion between device units with larger priority differences is weaker. This aligns with the physical intuition that higher-priority units should receive resources first. The virtual repulsion value is calculated for each device unit pair and accumulated to obtain the total virtual repulsion value. The excess range is obtained by subtracting the supply limit from the predicted total power, and multiplying this excess range by the total virtual repulsion value to obtain the virtual repulsion penalty term.

[0044] The strategy loss term and value loss term are calculated based on the temperature reward component and the temporal difference method. Specifically, firstly, the value of the current state and the next state are estimated using a value network to obtain the predicted value. The temperature reward component is multiplied by a discount factor and added together, then the predicted value of the current state is subtracted to obtain the temporal difference error, which is the advantage function value. The discount factor is set to 0.99. The strategy loss term is calculated by taking the logarithm of the action probability distribution output by the strategy network, multiplying it by the advantage function value, taking the negative value, and then averaging over all samples. The value loss term is calculated by multiplying the temperature reward component and the predicted value of the next state by a discount factor as the actual return, and then calculating the mean squared error between this actual return and the predicted value of the current state.

[0045] The total loss function is constructed as a weighted sum of the policy loss term, the value loss term, and the virtual repulsion penalty term, with weight coefficients set to 0.5, 0.5, and 0.3, respectively. The gradient of the total loss function with respect to all neural network parameters is calculated using an automatic differentiation framework. Gradient clipping is employed to limit the gradient norm to within 0.5 to prevent gradient explosion. The Adam optimizer is used to perform gradient descent updates, with a learning rate of 0.0003 and a batch size of 32 samples. After multiple rounds of iterative training, the agent gradually learns to allocate resources rationally according to priority under power-constrained conditions, avoiding insufficient warmth in high-priority areas due to resource contention.

[0046] This invention utilizes the synergistic effect of a virtual repulsion mechanism and priority weights to enable the intelligent agent to automatically reduce the power allocation of low-priority parts when the total power is limited, thereby ensuring the insulation effect of critical parts.

[0047] In one alternative implementation, the step of calculating the instant reward signal includes: The difference between the current temperature deviation of each part and the temperature deviation of the previous cycle is calculated as the temperature improvement amount; The priority-weighted temperature reward is obtained by multiplying the temperature improvement of each part by its corresponding priority weight and then summing the results. The power constraint penalty is calculated based on the difference between the predicted total power and the supply ceiling; Calculate the change amplitude of the target power value of each device unit in adjacent control cycles and sum them to obtain the total power fluctuation. Calculate the smoothness penalty based on the total power fluctuation. The priority-weighted temperature reward, the power constraint penalty, and the smoothness penalty are combined in a weighted manner to obtain an instant reward signal.

[0048] For example, the calculation of temperature improvement reflects the actual impact of the control action on the patient's body surface temperature. For the i-th monitoring site, in the t-th control cycle, the deviation between the current temperature and the target temperature of that site is first obtained, denoted as the current temperature deviation. Simultaneously, the temperature deviation of that site in the previous cycle is obtained, denoted as the historical temperature deviation. The difference between the historical temperature deviation and the current temperature deviation is the temperature improvement for that site; a positive value indicates a decrease in temperature deviation, and a negative value indicates an increase in temperature deviation. Since different sites have varying physiological importance, priority weights for each site need to be calculated based on pheromone values. The pheromone values ​​of each site are normalized using the Softmax function to obtain a priority weight vector. The temperature improvement for each site is multiplied by its corresponding priority weight, and then summed over all sites to obtain the priority-weighted temperature reward. This design ensures that sites with high physiological importance or severe temperature anomalies receive higher attention weights. When the temperature of these key sites is effectively improved, the agent receives more significant positive incentives.

[0049] Power constraint penalties are used to ensure that the total power demand of all insulation unit units does not exceed the supply limit. In each control cycle, after the agent outputs the target power value for each unit, the target power values ​​of all units are summed to obtain the predicted total power. The predicted total power is compared with the system-set supply limit. When the predicted total power exceeds the supply limit, the difference between the predicted total power and the supply limit is calculated and recorded as the excess range. The excess range is multiplied by a penalty coefficient to obtain the power constraint penalty. The penalty coefficient is usually set to a negative value, ranging from -0.5 to -2. This penalty value is negative, and the larger the excess range, the larger the absolute value of the penalty. When the predicted total power does not exceed the supply limit, the power constraint penalty is set to zero, and it does not negatively affect the reward signal.

[0050] The introduction of a smoothness penalty aims to suppress drastic fluctuations in control commands and avoid energy waste and equipment damage caused by frequent and large-scale power adjustments in the heating device. For each device unit, the absolute value of the difference between its target power value in the current cycle and the target power value in the previous cycle is calculated. This value characterizes the power change amplitude of a single device. The power change amplitudes of all device units are summed to obtain the total power fluctuation. This index reflects the control smoothness of the entire system between adjacent control cycles. The total power fluctuation is multiplied by the smoothness penalty coefficient to obtain the smoothness penalty term. The smoothness penalty coefficient is usually set to a negative value, typically ranging from -0.1 to -0.5. The specific value needs to be optimized during training based on the balance between temperature control effectiveness and power fluctuation.

[0051] The final instant reward signal is obtained by weighted combination of the above three components. The specific calculation method is as follows: the instant reward equals the priority-weighted temperature reward multiplied by a weighting coefficient of 1.0, plus the power constraint penalty multiplied by a weighting coefficient of 1.0, plus the smoothness penalty multiplied by a weighting coefficient of 1.0. For example, when the priority-weighted temperature reward of a certain control cycle is 5.0, the power constraint penalty is -3.0, and the smoothness penalty is -0.8, the instant reward is calculated as 5.0×1.0+(-3.0)×1.0+(-0.8)×1.0=1.2.

[0052] This multi-objective weighted reward design enables the agent to pursue temperature regulation accuracy while also considering energy constraints and control stability during training, ultimately achieving safe, efficient, and collaborative control for patient warmth during surgery.

[0053] In one optional implementation, the step of adaptively adjusting the calculation parameters of the pheromone value based on the difference between the prior distribution and the self-learning attention weights includes: The difference between the prior distribution of each part and the self-learning attention weight is calculated to obtain the local difference value. The absolute values ​​of the local difference values ​​of all parts are averaged to obtain the overall difference measure value. When the overall difference measurement value exceeds the preset difference threshold, the number of parts with positive local difference values ​​and the number of parts with negative local difference values ​​are counted, and the ratio of the absolute value of the difference between the two values ​​to the number of headquarters is calculated as the deviation asymmetry. If the deviation asymmetry exceeds the symmetry threshold, the initial weighting coefficients of temperature deviation and temperature change rate in the pheromone value calculation formula are adjusted; if the deviation asymmetry does not exceed the symmetry threshold, the attenuation factor of historical pheromone values ​​in the pheromone value calculation formula is adjusted. When the overall difference metric does not exceed the preset difference threshold, the calculation parameters of the pheromone value remain unchanged.

[0054] For example, during the iterative training of a deep reinforcement learning agent, it is necessary to continuously monitor the consistency between the prior distribution of pheromones calculated based on domain knowledge and the attention weights obtained by the neural network through autonomous learning.

[0055] To achieve adaptive parameter adjustment, the difference between the prior distribution value and the self-learning attention weight value is first calculated for each part, yielding a local difference value for each part. The sign of this local difference value reflects the directional deviation of the two mechanisms in judging the importance of that part; a positive value indicates that the prior distribution assigns a higher weight to that part, and a negative value indicates the opposite. The arithmetic mean of the absolute values ​​of the local differences for all monitored parts is then calculated to obtain the overall difference metric, which quantifies the degree of divergence between prior knowledge and the learning strategy at the global level.

[0056] A preset difference threshold is set as the criterion for judgment. This threshold is usually set between 0.15 and 0.25, and can be set to 0.2. When the overall difference measurement value exceeds this threshold, it indicates that there is a significant divergence between the two mechanisms, and the parameter adjustment mechanism needs to be activated. At this time, the number of parts with positive local difference values ​​is recorded as the number of positive deviation parts, and the number of parts with negative local difference values ​​is recorded as the number of negative deviation parts.

[0057] Calculate the absolute value of the difference between the number of positive and negative deviation sites, and divide this difference by the total number of monitored sites to obtain the deviation asymmetry. This index reflects whether the divergence is concentrated in a certain direction. The symmetry threshold is usually set between 0.4 and 0.6, and can be set to 0.5. If the deviation asymmetry exceeds the symmetry threshold, it indicates that the prior distribution overestimates or underestimates the importance of a certain type of feature site. The root cause of the problem lies in the unreasonable initial weighting coefficients of the temperature deviation term and the temperature change rate term in the pheromone calculation formula. In this case, if the number of positive deviation sites is greater than the number of negative deviation sites, it indicates that the prior distribution generally overestimates the importance of a certain type of feature site. The temperature deviation weighting coefficient should be reduced by 0.05 to 0.1, while the temperature change rate weighting coefficient should be increased by the same amount to keep the sum of the two equal to 1; otherwise, the adjustment should be reversed. If the deviation asymmetry does not exceed the symmetry threshold, it indicates that the positive and negative differences are relatively evenly distributed across sites. The problem lies in the excessively strong or weak influence of historical pheromone values. At this point, if the overall difference measure is between 0.2 and 0.3, the historical pheromone decay factor will be increased by 0.02 to 0.05 to reduce the inertia of historical information; if the overall difference measure is greater than 0.3, the decay factor will be decreased by 0.02 to 0.05 to enhance the continuity of historical trends.

[0058] When the overall difference metric does not exceed the preset difference threshold of 0.2, it indicates that prior knowledge and learning strategy have achieved good synergy, and the calculation parameters of pheromone value are in a reasonable range. At this time, keep the parameters such as temperature deviation weight coefficient, temperature change rate weight coefficient and historical pheromone decay factor unchanged to avoid unnecessary disturbances affecting stability.

[0059] This adaptive adjustment mechanism is executed once after each training cycle. Through closed-loop feedback, it ensures that the pheromone prior always provides effective guidance for deep reinforcement learning. At the same time, it allows the neural network to discover hidden patterns in the data based on the prior, achieving a dynamic balance between domain knowledge and data-driven methods, and improving the robustness and accuracy of the coordinated control of multi-site warming devices in complex surgical environments.

[0060] A second aspect of the present invention provides a collaborative control system for an intelligent warming device for patients during surgery, comprising: The pheromone calculation module is used to acquire the surface temperature data of multiple parts of the patient during surgery and calculate the pheromone value of each part. The pheromone value, combined with the temperature deviation and temperature change rate, represents the temperature urgency of the corresponding part. The decision generation module is used to construct a deep reinforcement learning agent. It normalizes the pheromone values ​​as the prior distribution of attention weights and weights them with the self-learned attention weights of the neural network to generate fused attention weights. The fused attention weights are used to weight and aggregate the features of different parts to generate the power allocation actions of each heating device unit. The model training module is used to introduce a virtual repulsion penalty term into the loss function during the training phase. When the sum of the target power values ​​of each device unit exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power value of each device unit and the priority determined by the pheromone value, and the virtual repulsion penalty term is accumulated. The deep reinforcement learning agent is updated based on the loss function. The parameter adaptive adjustment module is used to adaptively adjust the calculation parameters of the pheromone value based on the difference between the prior distribution and the self-learning attention weight.

[0061] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0062] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0063] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A collaborative control method for an intelligent warming device for intraoperative patients, characterized in that, include: Intraoperative surface temperature data of multiple sites on the patient were acquired, and pheromone values ​​for each site were calculated. The pheromone values, combined with temperature deviation and temperature change rate, characterize the temperature urgency of the corresponding site. A deep reinforcement learning agent is constructed, and the pheromone values ​​are normalized as the prior distribution of attention weights. This distribution is then weighted and fused with the self-learned attention weights of the neural network to generate fused attention weights. The fused attention weights are then used to weight and aggregate features from different parts to generate the power allocation actions for each heating device unit. During the training phase, a virtual repulsion penalty term is introduced into the loss function. When the sum of the target power values ​​of all device units exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power values ​​of each device unit and the priority determined by the pheromone values, and the virtual repulsion penalty term is accumulated. The deep reinforcement learning agent is then updated based on the loss function. The calculation parameters for the pheromone value are adaptively adjusted based on the difference between the prior distribution and the self-learning attention weights.

2. The method according to claim 1, characterized in that, The steps for acquiring intraoperative surface temperature data from multiple sites on the patient and calculating pheromone values ​​at each site include: The temperature deviation is the difference between the current temperature and the target temperature, and the temperature change rate is the ratio of the temperature change to the time interval between two adjacent acquisition cycles. The initial pheromone value is obtained by weighting and summing the temperature deviation and temperature change rate after assigning initial weighting coefficients to them respectively. The magnitude of the initial pheromone value quantifies the temperature urgency, and the initial weighting coefficients reflect the relative contribution of temperature deviation and temperature change rate to the temperature urgency. The initial pheromone value is subjected to time decay processing. By introducing a decay factor, the pheromone value at historical moments gradually decreases over time. The initial pheromone value at the current moment is added to the decayed historical pheromone value to obtain the pheromone value at the current moment. The decay factor controls the degree of influence of historical pheromone values ​​on the current decision.

3. The method according to claim 1, characterized in that, The steps of constructing a deep reinforcement learning agent, normalizing the pheromone values ​​as the prior distribution of attention weights, and weightedly fusing them with the self-learned attention weights of the neural network to generate fused attention weights include: The pheromone values ​​of each part are normalized to obtain the prior distribution; The temperature state features and historical power distribution features of each part are extracted by the neural network encoding module in the deep reinforcement learning agent, and then input into the attention calculation module to generate self-learning attention weights and normalize them. Calculate the consistency metric between the prior distribution and the self-learning attention weights. Subtract the consistency metric from the unit value to obtain the fusion coefficient of the prior distribution. Use the consistency metric as the fusion coefficient of the self-learning attention weights. Then, sum the weights of the prior distribution and the self-learning attention weights using the corresponding fusion coefficients to obtain the preliminary fusion weights. The initial fusion weights are gated and modulated. When the temperature deviation or temperature change rate of each part exceeds the preset condition, the gate value of the corresponding part is set to an enhancement coefficient greater than the unit value. The initial fusion weights are multiplied by the corresponding gate value and normalized to obtain the fusion attention weights.

4. The method according to claim 1, characterized in that, The steps for generating the power distribution actions of each insulation device unit after weighted aggregation of features from different parts using the fused attention weights include: Temperature state characteristics, temperature change characteristics, and power state characteristics are extracted for each part. The power state characteristics include the current power value and power change rate of the corresponding heat preservation device unit. The temperature state features, temperature change features, and power state features of each part are weighted and aggregated using the fusion attention weights to generate global temperature state features, global temperature change features, and global power state features. The global features of temperature state, global features of temperature change, and global features of power state are concatenated to form a comprehensive global feature vector, which is then input into the decision network of the deep reinforcement learning agent. The power adjustment amount of each heat preservation device unit is generated through the output layer of the decision network; the power adjustment amount is added to the current power value of the device unit to obtain the target power value.

5. The method according to claim 1, characterized in that, During the training phase, a virtual repulsion penalty term is introduced into the loss function. When the sum of the target power values ​​of each device unit exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power value of each device unit and the priority determined by the pheromone value, and the virtual repulsion penalty term is accumulated. The steps for updating the deep reinforcement learning agent based on the loss function include: After performing the power allocation action in each training iteration cycle, an instantaneous reward signal is calculated based on the measured body surface temperature data. The priority weight of each part is obtained by normalizing the pheromone values ​​of each part. The sum of the target power values ​​of each insulation device unit is calculated as the predicted total power. When the predicted total power exceeds the supply limit, a virtual repulsion force value is calculated for any two insulation device units. The virtual repulsion force value is directly proportional to the product of the target power values ​​of the two device units and inversely proportional to the absolute value of the difference in priority weights between the two device units. The virtual repulsion force values ​​of all device unit pairs are accumulated to obtain the total virtual repulsion force value. The magnitude by which the predicted total power exceeds the supply limit is multiplied by the total virtual repulsion force value to obtain a virtual repulsion force penalty term. The policy loss term and value loss term are calculated based on the instant reward signal; the policy loss term, value loss term, and virtual repulsion penalty term are weighted and summed to form the total loss function; the gradient of the total loss function with respect to the network parameters is calculated through backpropagation, and gradient descent is performed to update the deep reinforcement learning agent.

6. The method according to claim 5, characterized in that, The steps for calculating the instant reward signal include: The difference between the current temperature deviation and the temperature deviation of the previous cycle for each part is calculated as the temperature improvement amount; the temperature improvement amount for each part is multiplied by the corresponding priority weight and then summed to obtain the priority-weighted temperature reward. The power constraint penalty is calculated based on the difference between the predicted total power and the supply ceiling; Calculate the change amplitude of the target power value of each device unit in adjacent control cycles and sum them to obtain the total power fluctuation. Calculate the smoothness penalty based on the total power fluctuation. The priority-weighted temperature reward, the power constraint penalty, and the smoothness penalty are combined in a weighted manner to obtain an instant reward signal.

7. The method according to claim 2, characterized in that, The step of adaptively adjusting the calculation parameters of the pheromone value based on the difference between the prior distribution and the self-learning attention weights includes: The difference between the prior distribution of each part and the self-learning attention weight is calculated to obtain the local difference value. The absolute values ​​of the local difference values ​​of all parts are averaged to obtain the overall difference measure value. When the overall difference measurement value exceeds the preset difference threshold, the number of parts with positive local difference values ​​and the number of parts with negative local difference values ​​are counted, and the ratio of the absolute value of the difference between the two values ​​to the number of headquarters is calculated as the deviation asymmetry. If the deviation asymmetry exceeds the symmetry threshold, the initial weighting coefficients of temperature deviation and temperature change rate in the pheromone value calculation formula are adjusted. If the deviation asymmetry does not exceed the symmetry threshold, the attenuation factor of historical pheromone values ​​in the pheromone value calculation formula is adjusted.

8. A smart warming device and collaborative control system for intraoperative patients, used to implement the method of any one of claims 1-7, characterized in that, include: The pheromone calculation module is used to acquire the surface temperature data of multiple parts of the patient during surgery and calculate the pheromone value of each part. The pheromone value, combined with the temperature deviation and temperature change rate, represents the temperature urgency of the corresponding part. The decision generation module is used to construct a deep reinforcement learning agent. It normalizes the pheromone values ​​as the prior distribution of attention weights and weights them with the self-learned attention weights of the neural network to generate fused attention weights. The fused attention weights are used to weight and aggregate the features of different parts to generate the power allocation actions of each heating device unit. The model training module is used to introduce a virtual repulsion penalty term into the loss function during the training phase. When the sum of the target power values ​​of each device unit exceeds the supply limit, the virtual repulsion force between the device units is calculated based on the target power value of each device unit and the priority determined by the pheromone value, and the virtual repulsion penalty term is accumulated. The deep reinforcement learning agent is updated based on the loss function. The parameter adaptive adjustment module is used to adaptively adjust the calculation parameters of the pheromone value based on the difference between the prior distribution and the self-learning attention weight.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images