Reinforcement learning control method and system driven by refrigerator temperature prediction

Abstract

The application provides a refrigerator temperature prediction driving reinforcement learning control method and system, multi-dimensional time sequence sensing data in the running process of the refrigerator is collected; the multi-dimensional time sequence sensing data is preprocessed to construct a refrigerator data set; a three-temperature zone prediction model with physical constraints is trained by using the refrigerator data set, the three-temperature zone prediction model is used to output the future period refrigerating chamber predicted temperature, the freezing chamber predicted temperature and the freezing evaporation sensor predicted temperature; the predicted temperature output by the three-temperature zone prediction model is spliced and fused with the multi-dimensional time sequence sensing data to obtain a reinforcement learning state vector; the reinforcement learning state vector is input into a reinforcement learning intelligent agent to obtain a refrigerator multi-component collaborative control instruction; the running state of the refrigerator actuator is adjusted according to the collaborative control instruction; and the multi-dimensional time sequence sensing data in the running process of the refrigerator is collected again to realize continuous prediction type energy-saving control of the refrigerator, and solve the problem of hysteresis of the compressor control.

Description

Technical Field

[0001] This application relates to the field of refrigeration equipment technology, and in particular to a reinforcement learning control method and system for refrigerator temperature prediction-driven control. Background Technology

[0002] Refrigerators, as ubiquitous food storage devices in households, rely on the coordinated operation of multiple components such as compressors, fans, and dampers to maintain the target storage temperature in each compartment. They are core home appliances that balance food preservation and energy consumption. The new national standard places higher demands on the environmental adaptability and autonomous optimization capabilities of smart refrigerators, requiring the devices to adapt to complex operating conditions under different seasons and usage habits, ensuring accurate temperature control while reducing overall energy consumption.

[0003] To address the energy-saving control needs of compressors, some solutions have incorporated reinforcement learning techniques to optimize parameters. For example, a reinforcement learning-based compressor energy-saving operation control method predicts future cycle parameters by collecting multi-dimensional compressor operating data and combines this with current features to input into a reinforcement learning model, outputting compressor control commands. Another method is an intelligent energy-saving control method for variable frequency air conditioners, which inputs real-time state vectors into a pre-trained reinforcement learning agent to adjust parameters such as compressor speed and fan speed to balance energy saving and comfort. A third method is an adaptive multi-strategy deep reinforcement learning-based compressor energy-saving control system that constructs a state space and action space based on compressor operating parameters and combines a multi-dimensional reward function to train a strategy network to output a control strategy.

[0004] The above solutions only focus on the control optimization of a single compressor device, without considering the overall characteristics of the refrigerator's multi-component collaborative operation. Furthermore, they rely solely on current real-time status feedback for control decisions, lacking the ability to predict future temperature changes, resulting in control lag and an inability to adapt to the complex operating requirements of refrigerators with multiple temperature zones and operating conditions. Summary of the Invention

[0005] This application provides a reinforcement learning control method and system for refrigerator temperature prediction-driven control to solve the problem of lag in compressor control.

[0006] The first aspect of this application provides a reinforcement learning control method driven by refrigerator temperature prediction, including: Collect multi-dimensional time-series sensor data during the refrigerator's operation; The multidimensional time-series sensing data is preprocessed to construct a refrigerator dataset; A three-temperature zone prediction model with physical constraints is trained using the refrigerator dataset. The three-temperature zone prediction model is used to output the predicted temperature of the refrigerator compartment, the predicted temperature of the freezer compartment, and the predicted temperature of the freezer evaporator sensor for future time periods. The predicted temperature output by the three-temperature zone prediction model is spliced ​​and fused with the multi-dimensional time-series sensing data to obtain the reinforcement learning state vector. The reinforcement learning state vector is input into the reinforcement learning agent to obtain the multi-component collaborative control command of the refrigerator; The operating status of the refrigerator actuator is adjusted according to the coordinated control command; and multi-dimensional time-series sensor data during the refrigerator's operation are re-acquired to achieve continuous predictive energy-saving control of the refrigerator.

[0007] By collecting and preprocessing multi-dimensional time-series sensor data during refrigerator operation to construct a dataset, a three-temperature zone prediction model with physical constraints is trained to output the predicted temperatures of the refrigerator compartment, freezer compartment, and freezer evaporator sensors for future periods. The predicted temperatures are then spliced ​​and fused with the sensor data to form a reinforcement learning state vector. This vector is then used by a reinforcement learning agent to output multi-component collaborative control commands to adjust the operating state of the actuators. Data is continuously re-collected to achieve predictive energy-saving control. This approach can alleviate the compressor control lag problem, improve the timeliness of refrigerator temperature control response, reduce operating energy consumption, and enhance the system's adjustment stability and energy-saving effect under dynamic operating conditions.

[0008] Optionally, the preprocessing of the multidimensional time-series sensing data to construct a refrigerator dataset includes: The continuously acquired multidimensional time-series sensor data is divided into independent data samples of a preset duration. Remove invalid data samples caused by sensor malfunction or data parsing errors; For independent data samples with missing data, interpolation is used to complete the data; Based on the preset normal operating temperature range of each temperature zone of the refrigerator, independent data samples that exceed the preset threshold for a duration exceeding the temperature limit, as well as independent data samples under extreme environmental conditions, are filtered out to obtain the preprocessed refrigerator dataset.

[0009] By segmenting continuously acquired multidimensional time-series sensor data into independent data samples of preset duration, eliminating invalid data samples caused by sensor malfunctions or data parsing errors, using interpolation to complete data samples with missing data, and filtering out independent data samples whose temperature exceeds the preset threshold for extended periods and those under extreme environmental conditions based on the preset normal operating temperature range of each temperature zone of the refrigerator to obtain a preprocessed refrigerator dataset, the integrity and consistency of multidimensional time-series sensor data can be improved, the interference of abnormal data on subsequent model training and control decisions can be reduced, and data quality and the reliability of reinforcement learning control strategies can be improved.

[0010] Optionally, training a three-temperature-zone prediction model with physical constraints using the refrigerator dataset includes: The refrigerator dataset is divided into a training subset, a validation subset, and a test subset; A sliding window approach is used to construct time-series training samples based on the training subset. Each time-series training sample takes multi-dimensional time-series sensing data of a continuous historical period before the current time as input and the real temperature values ​​of three temperature zones in a future preset period as labels. A multi-output regression model is constructed. The total loss function of the multi-output regression model is obtained by weighted summation of the basic prediction loss and the physical constraint loss. The physical constraint loss is used to constrain the positive correlation between the compressor speed and the temperature drop of the corresponding temperature zone. The multi-output regression model is iteratively trained using the training subset and the validation subset, and the model performance is evaluated using the test subset. The parameters of the trained and evaluated model are then solidified to obtain the three-temperature zone prediction model.

[0011] By dividing the refrigerator dataset into training, validation, and test subsets, a sliding window approach is used to construct time-series training samples based on the training subset. These samples use multi-dimensional time-series sensor data spanning a continuous historical period prior to the current moment as input and are labeled with the actual temperature values ​​of three temperature zones in a preset future time period. A multi-output regression model is constructed, with the total loss function being a weighted sum of the basic prediction loss and the physical constraint loss (the physical constraint loss is used to constrain the positive correlation between compressor speed and the temperature drop magnitude of the corresponding temperature zone). Iterative training is performed using the training and validation subsets, and the model performance is evaluated using the test subset. The parameters of the trained and evaluated model are then solidified to obtain a three-temperature zone prediction model. This approach improves the accuracy of the model's prediction of the refrigerator's thermal processes, enhances the consistency between the prediction results and physical laws, and improves the model's generalization performance across different data subsets.

[0012] Optionally, the physical constraint loss can be constructed in the following ways: Calculate the predicted temperature change and the actual temperature change between adjacent time points; Extract the compressor operating speed at the corresponding moment; A penalty term is constructed: when the compressor speed increases, if the absolute value of the predicted temperature change is less than the absolute value of the actual temperature change, the loss value is increased to force the model to output a prediction result that conforms to the compressor's refrigeration characteristics.

[0013] By calculating the predicted temperature change and the actual temperature change at adjacent time points and extracting the compressor operating speed at the corresponding time points, a penalty term is constructed. When the compressor speed increases, if the absolute value of the predicted temperature change is less than the absolute value of the actual temperature change, the loss value is increased to force the model to output a prediction result that conforms to the compressor's refrigeration characteristics. This can improve the physical consistency between the predicted temperature change trend and the compressor speed, enhance the model's ability to represent the dynamic refrigeration process, reduce the risk of the prediction result deviating from the actual thermodynamic law, and improve the reliability of the three-temperature zone prediction model under varying operating conditions.

[0014] Optionally, after solidifying the trained and evaluated model parameters, the method further includes: With the electric valve opening fixed, switch between refrigeration mode and cooling mode respectively, and cycle through multiple compressor speed levels in each cooling mode; Collect and compare the actual temperature change trends and cooling amplitudes of the three temperature zones under each operating condition with the predicted temperature change trends and cooling amplitudes output by the three-temperature-zone prediction model; If the prediction results conform to the judgment rules that the higher the compressor speed under the same refrigeration flow, the greater the temperature drop, and that the temperature drop of the refrigeration side temperature zone is greater than the temperature drop of the refrigerator compartment when the refrigeration flows to the refrigeration side, and the temperature drop of the refrigerator compartment is greater than the temperature drop of the refrigeration side temperature zone when the refrigeration flows to the refrigeration side, then the model is deemed to have passed the verification; otherwise, return to the model training step to retrain.

[0015] By fixing the electric valve opening after the model parameters are solidified, switching between refrigeration-to-cooling mode and refrigeration-to-cooling mode, and traversing multiple compressor speed levels in each cooling mode, the actual temperature change trend and cooling amplitude of the three temperature zones under each operating condition are collected and compared with the predicted temperature change trend and cooling amplitude output by the three-temperature zone prediction model. If the prediction results conform to the judgment rules that the higher the compressor speed under the same refrigeration flow, the greater the cooling amplitude, and at the same speed, the cooling amplitude of the refrigeration side temperature zone is greater than the cooling amplitude of the refrigeration compartment when refrigeration flows to the refrigerator, and the cooling amplitude of the refrigeration compartment is greater than the cooling amplitude of the refrigeration side temperature zone when refrigeration flows to the refrigerator, then the model is deemed to be qualified. Otherwise, the model training step is returned to retrain. This can improve the physical rationality of the model output under different cooling modes and speed conditions, enhance the consistency of the prediction results in response to the differentiated characteristics of the refrigeration flow, and improve the reliability of the model before solidification and deployment.

[0016] Optionally, the multidimensional time-series sensing data includes the set temperature of the freezer compartment, the set temperature of the refrigerator compartment, the actual temperature of the freezer compartment, the actual temperature of the refrigerator compartment, the temperature of the freezer evaporator sensor, the ambient temperature, humidity, the on / off status of the electric valve, the status of the freezer door, the status of the refrigerator door, the defrosting status, the compressor speed, the speed of the refrigeration fan, the speed of the refrigerator fan, and the speed of the condenser fan.

[0017] By collecting multi-dimensional time-series sensing data such as the set temperature of the freezer compartment, the set temperature of the refrigerator compartment, the actual temperature of the freezer compartment, the actual temperature of the refrigerator compartment, the temperature of the freezer evaporator sensor, the ambient temperature, humidity, the on / off status of the electric valve, the status of the freezer door, the status of the refrigerator door, the defrosting status, the compressor speed, the speed of the freezer fan, the speed of the refrigerator fan, and the speed of the condenser fan, a multi-source information foundation covering thermal state, component operation, and environmental disturbances can be provided for the reinforcement learning control method driven by refrigerator temperature prediction. This improves the comprehensiveness and diversity of the data features on which the model training and decision-making are based, thereby helping to improve the prediction accuracy and the adaptability of the control strategy.

[0018] Optionally, the cooperative control commands output by the reinforcement learning agent include the target speed of the compressor, the refrigeration flow direction of the electric valve, the target speed of the refrigeration fan, the target speed of the refrigerator fan, and the target speed of the condenser fan.

[0019] By using reinforcement learning, the intelligent agent outputs coordinated control commands, including the target speed of the compressor, the cooling flow direction of the electric valve, the target speed of the refrigeration fan, the target speed of the refrigerator fan, and the target speed of the condenser fan. This enables coordinated adjustment of the operating status of multiple components of the refrigerator, improving the matching of actions between the compressor, fan, and cooling flow direction, enhancing the overall operational coordination of the system while meeting temperature control requirements, reducing energy consumption conflicts or response delays that may be caused by independent control of multiple components, and enhancing energy-saving control effects.

[0020] Optionally, the three-temperature zone prediction model is configured to simultaneously predict three target values—refrigerator compartment temperature, freezer compartment temperature, and freezer evaporator sensor temperature—over a certain period in the future, in order to predict the temperature zone trend of the refrigerator.

[0021] By configuring the three-temperature zone prediction model to simultaneously predict three target values—refrigerator compartment temperature, freezer compartment temperature, and freezer evaporator sensor temperature—over a certain period in the future, and thus predicting the temperature trend of the refrigerator, it can provide the reinforcement learning agent with advanced information on temperature changes in multiple temperature zones. This improves the control decision-making ability to perceive the future thermal state of the refrigerator, thereby optimizing the timing of the generation of multi-component collaborative control commands and enhancing the system's response timeliness and adjustment effect to temperature fluctuations under dynamic operating conditions.

[0022] Optionally, the method further includes: Before training a three-temperature zone prediction model with physical constraints using the refrigerator dataset, anomaly verification is performed on the multidimensional time-series sensing data based on the refrigerator operating mechanism and sensor range. Furthermore, after adjusting the operating status of the refrigerator actuator, the actual temperature changes of each compartment of the refrigerator are monitored, and the actual temperature changes are fed back to the reinforcement learning agent.

[0023] Before training a three-temperature-zone prediction model with physical constraints using a refrigerator dataset, anomaly verification is performed on multi-dimensional time-series sensing data based on the refrigerator's operating mechanism and sensor range. After adjusting the operating state of the refrigerator's actuator, the actual temperature changes in each compartment of the refrigerator are monitored and fed back to the reinforcement learning agent to dynamically adjust subsequent control strategies. This improves the effectiveness of training data input, enhances the control strategy's ability to perceive and correct deviations in actual temperature control, and improves the adaptability of the reinforcement learning agent in long-term operation.

[0024] A second aspect of this application provides a reinforcement learning control system driven by refrigerator temperature prediction, applied to the reinforcement learning control method driven by refrigerator temperature prediction described in the first aspect, the system comprising: The sensing and acquisition module is used to collect multi-dimensional time-series sensing data during the operation of the refrigerator; A data processing unit is used to preprocess the multidimensional time-series sensing data to construct a refrigerator dataset; The temperature prediction module is used to train a three-temperature zone prediction model with physical constraints using the refrigerator dataset, and output the predicted temperature of the refrigerator compartment, the predicted temperature of the freezer compartment, and the predicted temperature of the freezer evaporator sensor for future periods based on the real-time collected multi-dimensional time-series sensor data. The feature fusion unit is used to splice and fuse the predicted temperature output by the temperature prediction module with the multi-dimensional time-series sensing data collected by the sensing acquisition module to obtain a reinforcement learning state vector. The reinforcement learning control module is used to receive the reinforcement learning state vector generated by the feature fusion unit and output the multi-component collaborative control command of the refrigerator. An actuator is used to adjust the operating state of the refrigerator according to the coordinated control instructions; The sensing and acquisition module re-acquires multi-dimensional time-series sensing data during the refrigerator's operation after the actuator adjusts the operating state, so that the system can achieve continuous predictive energy-saving control.

[0025] By setting up a sensing and acquisition module to collect multi-dimensional time-series sensor data during the refrigerator's operation, the data processing unit preprocesses the multi-dimensional time-series sensor data to construct a refrigerator dataset. The temperature prediction module uses the refrigerator dataset to train a three-temperature zone prediction model with physical constraints and outputs the predicted temperatures of the refrigerator compartment, freezer compartment, and freezer evaporator sensor for future periods based on the real-time collected multi-dimensional time-series sensor data. The feature fusion unit splices and fuses the predicted temperatures with the multi-dimensional time-series sensor data collected by the sensing and acquisition module to obtain a reinforcement learning state vector. The reinforcement learning control module receives the reinforcement learning state vector and outputs collaborative control commands for multiple refrigerator components. The actuator adjusts the refrigerator's operating state according to the collaborative control commands. Furthermore, the sensing and acquisition module re-collects multi-dimensional time-series sensor data after the actuator adjusts the operating state. This improves the refrigerator system's ability to predict temperature changes under multi-component collaborative operation conditions, enhances the system's operational coordination under dynamic conditions, and improves the sustainability of energy-saving control effects.

[0026] As can be seen from the above technical solutions, this application provides a reinforcement learning control method and system driven by refrigerator temperature prediction. This involves collecting multi-dimensional time-series sensing data during refrigerator operation; preprocessing the multi-dimensional time-series sensing data to construct a refrigerator dataset; training a three-temperature zone prediction model with physical constraints using the refrigerator dataset, whereby the three-temperature zone prediction model outputs predicted temperatures for the refrigerator compartment, freezer compartment, and freezer evaporator sensor for future time periods; concatenating and fusing the predicted temperatures output by the three-temperature zone prediction model with the multi-dimensional time-series sensing data to obtain a reinforcement learning state vector; inputting the reinforcement learning state vector into a reinforcement learning agent to obtain collaborative control commands for multiple refrigerator components; adjusting the operating state of the refrigerator actuators according to the collaborative control commands; and re-collecting multi-dimensional time-series sensing data during refrigerator operation to achieve continuous predictive energy-saving control of the refrigerator, thus solving the problem of lag in compressor control. Attached Figure Description

[0027] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the reinforcement learning control method for refrigerator temperature prediction driven by an embodiment of this application; Figure 2 This is a connection diagram of a reinforcement learning control system driven by refrigerator temperature prediction provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application.

[0030] To address the issue of lag in compressor control, see [link / reference]. Figure 1 This application provides a reinforcement learning control method driven by refrigerator temperature prediction in some embodiments, including: S100: Collects multi-dimensional time-series sensor data during the refrigerator's operation.

[0031] In some embodiments, the multidimensional time-series sensing data includes the freezer compartment set temperature, the refrigerator compartment set temperature, the freezer compartment actual temperature, the refrigerator compartment actual temperature, the freezer evaporator sensor temperature, the ambient temperature, humidity, the electric valve on / off status, the freezer door status, the refrigerator door status, the defrosting status, the compressor speed, the refrigeration fan speed, the refrigerator fan speed, and the condenser fan speed.

[0032] It should be understood that the scope of the multi-dimensional time-series sensor data collected here during refrigerator operation is not limited to a single refrigerator or specific operating conditions. Instead, it covers full operational data from different devices, installation locations, seasons, and ambient temperatures. This ensures the dataset encompasses all operating conditions encountered in daily refrigerator use, such as door opening and retrieval, changes in ambient temperature and humidity, and defrosting cycles, preventing bias in the dataset towards a single operating condition that could lead to deviations in subsequent model training. Furthermore, splitting the data into independent samples according to a three-day cycle preserves the complete cyclical patterns of refrigerator temperature changes and component operation. This also facilitates subsequent data normalization and the division into training, validation, and test sets, meeting the sample size requirements for subsequent model training. The various multi-dimensional time-series data listed above comprehensively cover environmental inputs, state inputs, and component outputs that affect internal refrigerator temperature changes, fully supporting the generation of subsequent temperature prediction and control strategies.

[0033] S200: Preprocess multidimensional time-series sensing data to construct a refrigerator dataset.

[0034] In some embodiments, preprocessing of multidimensional time-series sensing data to construct a refrigerator dataset includes: The continuously acquired multidimensional time-series sensor data is divided into independent data samples of a preset duration. Remove invalid data samples caused by sensor malfunction or data parsing errors; For independent data samples with missing data, interpolation is used to complete the data; Based on the preset normal operating temperature range of each temperature zone of the refrigerator, independent data samples that exceed the preset threshold for a duration exceeding the temperature limit, as well as independent data samples under extreme environmental conditions, are filtered out to obtain the preprocessed refrigerator dataset.

[0035] It should be understood that, firstly, the object of this step is the continuously acquired multi-dimensional time-series sensor data. This type of data comes from various temperature and environmental sensors deployed in the refrigerator and is sequential data generated continuously in time order. Therefore, the first step is to divide it into independent data samples of a preset duration (such as 24 hours) to facilitate subsequent sample-by-sample verification processing.

[0036] Secondly, for the segmented independent data samples, we first combine the refrigerator's own operating mechanism and the design range of the corresponding sensors to quickly identify invalid abnormal data caused by sensor failure, data transmission or parsing errors. Such data cannot reflect the refrigerator's true operating status and is directly removed as invalid data samples and not included in the refrigerator dataset constructed subsequently.

[0037] Third, short-term, intermittent data loss is common during the data collection process. This does not significantly affect the overall validity of the sample. Therefore, it is not necessary to remove the entire sample. Instead, data at the missing locations can be filled in using interpolation to obtain a complete and usable sample.

[0038] Finally, after completing the basic anomaly removal and missing data completion, we further validate the samples based on the preset normal operating temperature ranges for different components and temperature zones of the refrigerator. These preset ranges include: 2-8℃ for the refrigerator compartment, -24℃ to -16℃ for the freezer compartment, -25℃ to -18℃ for the freezing evaporator sensor during cooling, and 5℃ to 10℃ during defrosting. If the temperature in a sample exceeds the corresponding range for a duration exceeding our preset threshold (e.g., 2 hours), it indicates that the operating state of that sample deviates from normal conditions, and this sample will be removed entirely. Simultaneously, samples collected under extreme ambient temperatures do not meet the training requirements of conventional control scenarios and are also removed. All the qualified samples obtained constitute the preprocessed refrigerator dataset, which can be used for subsequent model training. This step improves the integrity and consistency of multi-dimensional time-series sensor data, reduces the interference of abnormal data on subsequent model training and control decisions, and improves data quality and the reliability of reinforcement learning control strategies.

[0039] S300: Train a three-temperature zone prediction model with physical constraints using a refrigerator dataset.

[0040] The three-temperature zone prediction model is used to output the predicted temperatures of the refrigerator compartment, freezer compartment, and freezer evaporator sensor for future time periods.

[0041] In some embodiments, training a three-temperature-zone prediction model with physical constraints using a refrigerator dataset includes: The refrigerator dataset is divided into a training subset, a validation subset, and a test subset; A sliding window approach is used to construct time-series training samples based on a training subset. Each time-series training sample takes multi-dimensional time-series sensing data of a continuous historical period before the current moment as input and the real temperature values ​​of three temperature zones in a future preset period as labels. A multi-output regression model is constructed. The total loss function of the multi-output regression model is obtained by weighted summation of the basic prediction loss and the physical constraint loss. The physical constraint loss is used to constrain the positive correlation between the compressor speed and the temperature drop of the corresponding temperature zone. The multi-output regression model is iteratively trained using training and validation subsets, and the model performance is evaluated using a test subset. The parameters of the trained and evaluated model are then solidified to obtain a three-temperature zone prediction model.

[0042] It should be understood that when constructing time-series training samples, this embodiment uses multi-dimensional time-series sensing data from 60 consecutive minutes prior to the current moment as input for a single training sample, with the corresponding label being the actual temperature of the three temperature zones in the next 3 minutes. This setting adapts to the slow and lagging characteristics of refrigerator temperature changes, ensuring that the input features can cover sufficient information on changes in operating conditions, while also meeting the needs of the temperature control strategy for short-term temperature prediction.

[0043] This embodiment introduces a physical constraint loss into the total loss function of the multi-output regression model. This is an optimization to address the problem that existing data-driven models are prone to deviating from fundamental physical laws and having unreliable prediction results in abnormal scenarios. Since the compressor is the core refrigeration component in a refrigerator, under normal operating conditions, a higher compressor speed corresponds to greater refrigeration power, and a greater temperature drop in the corresponding temperature zone per unit time. This physical law does not change with operating conditions. Therefore, in addition to the basic prediction loss, an extra physical constraint term is added to penalize samples in the prediction results that violate the law that "compressor speed is positively correlated with the temperature drop in the corresponding temperature zone." This allows the model to learn feature mappings that conform to the basic physical laws of refrigeration, avoiding prediction results that violate common sense and improving the model's prediction stability under extreme and rare operating conditions.

[0044] An early stopping strategy is employed during model training. Training is terminated when the prediction loss on the validation set no longer decreases after a certain number of consecutive training iterations. This prevents overfitting on the training set and reduces unnecessary training iterations, improving training efficiency. After training is complete, the model's prediction accuracy and stability are validated using a test set under various typical refrigerator operating conditions, including normal operation, frequent door opening, high and low temperature environments, and high and low loads. Once the performance meets the requirements, the optimal model parameters are then embedded into the refrigerator's control system, ensuring that the model can achieve real-time inference output with low computational consumption at the refrigerator level.

[0045] In some embodiments, the physical constraint loss is constructed in the following ways: Calculate the predicted temperature change and the actual temperature change between adjacent time points; Extract the compressor operating speed at the corresponding moment; A penalty term is constructed: when the compressor speed increases, if the absolute value of the predicted temperature change is less than the absolute value of the actual temperature change, the loss value is increased to force the model to output a prediction result that conforms to the compressor's refrigeration characteristics.

[0046] Specifically, the basic prediction loss uses the mean squared error loss, calculating the errors between the predicted and actual temperature values ​​from the sensors in the refrigerator compartment, freezer compartment, and freeze evaporator compartment, respectively, and then summing them to obtain the overall basic prediction loss. That is: ; in, The basic prediction loss is denoted by N; N is the total number of time-series training samples; i is the sample index, with a value range of i=1, 2, 3, ..., N; , , These represent the actual temperatures of the refrigerator compartment, freezer compartment, and freeze-evaporation sensor for the i-th sample. , , These are the predicted temperatures of the refrigerator compartment, freezer compartment, and freeze-evaporation sensor for the i-th sample, respectively.

[0047] The physical constraint loss is calculated based on time-series data to determine the temperature change between adjacent time points, while simultaneously extracting the compressor operating speed at the corresponding time point; a penalty term is designed based on the positive correlation between operating speed and temperature drop: operating speed The larger, the temperature The larger the decrease, the greater the loss; if this pattern is violated, losses will occur. That is: ; in, is the physical constraint loss; N is the total number of time-series training samples; i is the sample index, with a value range of i=1, 2, 3, ..., N; It predicts the single-step change in temperature; It is the actual one-step change in temperature.

[0048] Weighting coefficients are assigned to the basic prediction loss and the physical constraint loss, and the weighted sum is used to obtain the total loss function. That is: ; in, This is the total loss function; and The weights can be adjusted according to the operating conditions; the standard weight is... =0.7, =0.3.

[0049] During training, backpropagation is used to optimize model parameters, ensuring that the model output conforms to the physical characteristics of compressor refrigeration while maintaining prediction accuracy.

[0050] S400: The predicted temperature output by the three-temperature zone prediction model is spliced ​​and fused with multi-dimensional time-series sensing data to obtain the reinforcement learning state vector.

[0051] S500: Input the reinforcement learning state vector into the reinforcement learning agent to obtain the collaborative control instructions for multiple components of the refrigerator.

[0052] In some embodiments, the cooperative control commands output by the reinforcement learning agent include the target speed of the compressor, the refrigeration flow direction of the electric valve, the target speed of the refrigeration fan, the target speed of the refrigerator fan, and the target speed of the condenser fan.

[0053] It should be understood that these parameters work together to match the current temperature, set temperature, and temperature change trend of different temperature zones in the refrigerator, and dynamically adjust to different cooling needs: when multiple temperature zones need cooling at the same time, the cooling capacity will be allocated according to the priority of the temperature gap of each temperature zone, and the flow opening of the electric valve will be adjusted in conjunction with the corresponding compressor speed and the speed of each fan to meet the needs of rapid cooling while avoiding excessive cooling and energy waste; when only a single temperature zone needs cooling, the component parameters of the corresponding cooling flow direction will be adjusted in a targeted manner to maintain the temperature stability of other temperature zones that do not need cooling, so as to achieve a balance between temperature control accuracy and energy consumption control.

[0054] S600: Adjusts the operating status of the refrigerator actuator according to the coordinated control command; and re-collects multi-dimensional time-series sensor data during the refrigerator's operation to achieve continuous predictive energy-saving control of the refrigerator.

[0055] By collecting and preprocessing multi-dimensional time-series sensor data during refrigerator operation to construct a dataset, a three-temperature zone prediction model with physical constraints is trained to output the predicted temperatures of the refrigerator compartment, freezer compartment, and freezer evaporator sensors for future periods. The predicted temperatures are then spliced ​​and fused with the sensor data to form a reinforcement learning state vector. This vector is then used by a reinforcement learning agent to output multi-component collaborative control commands to adjust the operating state of the actuators. Data is continuously re-collected to achieve predictive energy-saving control. This approach can alleviate the compressor control lag problem, improve the timeliness of refrigerator temperature control response, reduce operating energy consumption, and enhance the system's adjustment stability and energy-saving effect under dynamic operating conditions.

[0056] In some embodiments, after the trained and evaluated model parameters are solidified, the method further includes: With the electric valve opening fixed, switch between refrigeration mode and cooling mode respectively, and cycle through multiple compressor speed levels in each cooling mode; Collect and compare the actual temperature change trends and cooling amplitudes of the three temperature zones under various operating conditions with the predicted temperature change trends and cooling amplitudes output by the three-temperature-zone prediction model; If the prediction results conform to the judgment rules that the higher the compressor speed under the same refrigeration flow, the greater the temperature drop, and that the temperature drop of the refrigeration side temperature zone is greater than the temperature drop of the refrigerator compartment when the refrigeration flows to the refrigeration side, and the temperature drop of the refrigerator compartment is greater than the temperature drop of the refrigeration side temperature zone when the refrigeration flows to the refrigeration side, then the model is deemed to have passed the verification; otherwise, return to the model training step to retrain.

[0057] It should be understood that this test fixed all external influencing variables such as ambient temperature, ambient humidity, and door opening status, retaining only two variables: cooling flow direction and compressor speed. This eliminates external interference and accurately verifies the model's ability to identify the differences in the core operating conditions of dual-circuit cooling, ensuring the reliability of the model validation results. The "three temperature zones" specifically refer to the refrigerator compartment, freezer compartment, and variable temperature compartment. The judgment rules are entirely based on the inherent physical mechanism of the refrigerator's dual-circuit cooling, directly verifying the model's physical adaptability and avoiding the risk of misjudgment inherent in single-dimensional validation. This ensures that the output model truly possesses the ability to distinguish the differentiated characteristics of dual-circuit cooling, guaranteeing the accuracy of subsequent reinforcement learning control input state predictions from the source.

[0058] In some embodiments, the three-temperature zone prediction model is configured to simultaneously predict three target values—refrigerator compartment temperature, freezer compartment temperature, and freezer evaporator sensor temperature—over a future period of time in order to predict the temperature zone trend of the refrigerator.

[0059] It should be understood that increasing the prediction of the temperature of the freeze evaporator sensor can more accurately capture the changes in the heat exchange state of the evaporator in the dual-circuit refrigeration system. This helps the reinforcement learning agent to more comprehensively grasp the actual working condition of the refrigerator's refrigeration circuit, avoid temperature prediction deviations caused by factors such as evaporator frosting and heat exchange efficiency fluctuations, and further improve the accuracy and rationality of subsequent control commands.

[0060] In some embodiments, the method further includes: Before training a three-temperature zone prediction model with physical constraints using a refrigerator dataset, anomaly verification is performed on the multi-dimensional time-series sensing data based on the refrigerator's operating mechanism and sensor range. Furthermore, after adjusting the operating status of the refrigerator's actuator, the actual temperature changes in each compartment of the refrigerator are monitored, and the actual temperature changes are fed back to the reinforcement learning agent to dynamically adjust subsequent control strategies.

[0061] Before training a three-temperature-zone prediction model with physical constraints using a refrigerator dataset, anomaly verification is performed on multi-dimensional time-series sensing data based on the refrigerator's operating mechanism and sensor range. After adjusting the operating state of the refrigerator's actuator, the actual temperature changes in each compartment of the refrigerator are monitored and fed back to the reinforcement learning agent to dynamically adjust subsequent control strategies. This improves the effectiveness of training data input, enhances the control strategy's ability to perceive and correct deviations in actual temperature control, and improves the adaptability of the reinforcement learning agent in long-term operation.

[0062] See Figure 2 This application also provides a reinforcement learning control system driven by refrigerator temperature prediction, applied to the reinforcement learning control method driven by refrigerator temperature prediction provided in the above embodiments. The system includes: The sensing and acquisition module is used to collect multi-dimensional time-series sensing data during the operation of the refrigerator.

[0063] Specifically, the multi-sensor acquisition module covers the core state dimensions of the refrigerator's operation across all scenarios: ambient temperature and humidity reflect the external heat load conditions of the refrigerator, and the cold and heat load is the core external factor affecting the refrigerator's energy consumption and temperature control stability; the temperature of each compartment directly represents the actual state of the current temperature control target; the temperature of the evaporator sensor, the operating status of each component, and the door opening / closing and defrosting status reflect the current working stage and load changes inside the refrigerator. All acquired data will be synchronously output to the subsequent data preprocessing unit to provide complete raw input for subsequent processing.

[0064] The data processing unit is used to preprocess multidimensional time-series sensor data to construct a refrigerator dataset.

[0065] Specifically, the data preprocessing unit removes abnormal data caused by signal interference or occasional malfunctions during sensor acquisition. It also uses adjacent time-series interpolation to fill in missing data caused by short-term sensor disconnection. Finally, it performs a value range rationality check on all temperature data and removes erroneous data that exceeds the normal operating temperature range of the refrigerator. This ensures that the data input to subsequent modules is clean and reliable, and avoids erroneous data from interfering with prediction and decision-making effects.

[0066] The temperature prediction module is used to train a three-temperature zone prediction model with physical constraints using a refrigerator dataset, and outputs the predicted temperature of the refrigerator compartment, the predicted temperature of the freezer compartment, and the predicted temperature of the freezer evaporator sensor for future periods based on real-time acquired multi-dimensional time-series sensor data.

[0067] Specifically, the temperature prediction module differs from the unconstrained temperature prediction model. During model training and inference, the temperature prediction module embeds the corresponding physical relationship constraints between compressor speed and cooling amplitude to avoid the model outputting unreasonable prediction results that violate the actual physical laws of refrigerator refrigeration. At the same time, it can output the future prediction values ​​of three temperature zones—refrigerator temperature, freezer temperature, and freezer evaporation temperature—at once, providing complete future temperature trend information for subsequent decision-making.

[0068] The feature fusion unit is used to concatenate and fuse the predicted temperature output by the temperature prediction module with the multi-dimensional time-series sensing data collected by the sensing acquisition module to obtain the reinforcement learning state vector.

[0069] Specifically, the feature fusion unit horizontally splices together all the actual operating state features of the refrigerator at the current moment and the predicted temperature features of the three future temperature zones output by the temperature prediction module, integrating them into a unified-dimensional comprehensive state vector. This allows the reinforcement learning agent to simultaneously acquire current state information and future temperature change trend information, supporting the decision-making model to make better control judgments.

[0070] The reinforcement learning control module is used to receive the reinforcement learning state vector generated by the feature fusion unit and output the collaborative control command for multiple components of the refrigerator.

[0071] Specifically, the reinforcement learning control module takes the comprehensive state vector as input and refrigerator energy saving and temperature control stability as optimization objectives. Through reinforcement learning training completed offline in advance, it infers the optimal control parameters of the compressor, electric valves and various fans that are adapted to the current state, which is the optimal control strategy. Under the premise of ensuring that the temperature control meets the user's set requirements, it maximizes the reduction of the refrigerator's overall operating energy consumption.

[0072] The actuator is used to adjust the refrigerator's operating status according to the coordinated control instructions; Among them, the sensing and acquisition module re-acquires multi-dimensional time-series sensing data during the refrigerator's operation after the actuator adjusts the operating state, so that the system can achieve continuous predictive energy-saving control.

[0073] Specifically, the actuator receives the optimal control strategy output by the reinforcement learning control module and directly issues corresponding control commands to each controlled component to adjust the compressor's operating speed, the opening degree of the electric valve, and the operating speed of each fan, thereby changing the refrigerator's cooling power and cold capacity distribution, and realizing the implementation of the control strategy.

[0074] By setting up a sensing and acquisition module to collect multi-dimensional time-series sensor data during the refrigerator's operation, the data processing unit preprocesses the multi-dimensional time-series sensor data to construct a refrigerator dataset. The temperature prediction module uses the refrigerator dataset to train a three-temperature zone prediction model with physical constraints and outputs the predicted temperatures of the refrigerator compartment, freezer compartment, and freezer evaporator sensor for future periods based on the real-time collected multi-dimensional time-series sensor data. The feature fusion unit splices and fuses the predicted temperatures with the multi-dimensional time-series sensor data collected by the sensing and acquisition module to obtain a reinforcement learning state vector. The reinforcement learning control module receives the reinforcement learning state vector and outputs collaborative control commands for multiple refrigerator components. The actuator adjusts the refrigerator's operating state according to the collaborative control commands. Furthermore, the sensing and acquisition module re-collects multi-dimensional time-series sensor data after the actuator adjusts the operating state. This improves the refrigerator system's ability to predict temperature changes under multi-component collaborative operation conditions, enhances the system's operational coordination under dynamic conditions, and improves the sustainability of energy-saving control effects.

[0075] In some embodiments, the system further includes an offline support unit, which is an offline dataset and model solidification unit, used to provide training data and solidify model parameters for the temperature prediction module.

[0076] As can be seen from the above technical solutions, the embodiments of this application provide a reinforcement learning control method and system driven by refrigerator temperature prediction. This involves collecting multi-dimensional time-series sensing data during refrigerator operation; preprocessing the multi-dimensional time-series sensing data to construct a refrigerator dataset; training a three-temperature zone prediction model with physical constraints using the refrigerator dataset, which outputs the predicted temperatures of the refrigerator compartment, freezer compartment, and freezer evaporator sensor for future periods; concatenating and fusing the predicted temperatures output by the three-temperature zone prediction model with the multi-dimensional time-series sensing data to obtain a reinforcement learning state vector; inputting the reinforcement learning state vector into a reinforcement learning agent to obtain collaborative control commands for multiple refrigerator components; adjusting the operating state of the refrigerator actuators according to the collaborative control commands; and re-collecting the multi-dimensional time-series sensing data during refrigerator operation to achieve continuous predictive energy-saving control of the refrigerator and solve the problem of lag in compressor control.

Claims

1. A reinforcement learning control method driven by refrigerator temperature prediction, characterized in that, include: Collect multi-dimensional time-series sensor data during the refrigerator's operation; The multidimensional time-series sensing data is preprocessed to construct a refrigerator dataset; A three-temperature zone prediction model with physical constraints is trained using the refrigerator dataset. The three-temperature zone prediction model is used to output the predicted temperature of the refrigerator compartment, the predicted temperature of the freezer compartment, and the predicted temperature of the freezer evaporator sensor for future time periods. The predicted temperature output by the three-temperature zone prediction model is spliced ​​and fused with the multi-dimensional time-series sensing data to obtain the reinforcement learning state vector. The reinforcement learning state vector is input into the reinforcement learning agent to obtain the multi-component collaborative control command of the refrigerator; The operating status of the refrigerator actuator is adjusted according to the coordinated control command; and multi-dimensional time-series sensor data during the refrigerator's operation are re-acquired to achieve continuous predictive energy-saving control of the refrigerator.

2. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 1, characterized in that, The preprocessing of the multidimensional time-series sensing data to construct a refrigerator dataset includes: The continuously acquired multidimensional time-series sensor data is divided into independent data samples of a preset duration. Remove invalid data samples caused by sensor malfunction or data parsing errors; For independent data samples with missing data, interpolation is used to complete the data; Based on the preset normal operating temperature range of each temperature zone of the refrigerator, independent data samples that exceed the preset threshold for a duration exceeding the temperature limit, as well as independent data samples under extreme environmental conditions, are filtered out to obtain the preprocessed refrigerator dataset.

3. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 1, characterized in that, The step of training a three-temperature zone prediction model with physical constraints using the refrigerator dataset includes: The refrigerator dataset is divided into a training subset, a validation subset, and a test subset; A sliding window approach is used to construct time-series training samples based on the training subset. Each time-series training sample takes multi-dimensional time-series sensing data of a continuous historical period before the current time as input and the real temperature values ​​of three temperature zones in a future preset period as labels. A multi-output regression model is constructed. The total loss function of the multi-output regression model is obtained by weighted summation of the basic prediction loss and the physical constraint loss. The physical constraint loss is used to constrain the positive correlation between the compressor speed and the temperature drop of the corresponding temperature zone. The multi-output regression model is iteratively trained using the training subset and the validation subset, and the model performance is evaluated using the test subset. The parameters of the trained and evaluated model are then solidified to obtain the three-temperature zone prediction model.

4. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 3, characterized in that, The physical constraint loss is constructed in the following ways: Calculate the predicted temperature change and the actual temperature change between adjacent time points; Extract the compressor operating speed at the corresponding moment; A penalty term is constructed: when the compressor speed increases, if the absolute value of the predicted temperature change is less than the absolute value of the actual temperature change, the loss value is increased to force the model to output a prediction result that conforms to the compressor's refrigeration characteristics.

5. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 3, characterized in that, After the parameters of the trained and evaluated model are solidified, the process also includes: With the electric valve opening fixed, switch between refrigeration mode and cooling mode respectively, and cycle through multiple compressor speed levels in each cooling mode; Collect and compare the actual temperature change trends and cooling amplitudes of the three temperature zones under each operating condition with the predicted temperature change trends and cooling amplitudes output by the three-temperature-zone prediction model; If the prediction results conform to the judgment rules that the higher the compressor speed under the same refrigeration flow, the greater the temperature drop, and that the temperature drop of the refrigeration side temperature zone is greater than the temperature drop of the refrigerator compartment when the refrigeration flows to the refrigeration side, and the temperature drop of the refrigerator compartment is greater than the temperature drop of the refrigeration side temperature zone when the refrigeration flows to the refrigeration side, then the model is deemed to have passed the verification; otherwise, return to the model training step to retrain.

6. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 1, characterized in that, The multidimensional time-series sensing data includes the set temperature of the freezer compartment, the set temperature of the refrigerator compartment, the actual temperature of the freezer compartment, the actual temperature of the refrigerator compartment, the temperature of the freezer evaporator sensor, the ambient temperature, humidity, the on / off status of the electric valve, the status of the freezer door, the status of the refrigerator door, the defrosting status, the compressor speed, the speed of the refrigeration fan, the speed of the refrigerator fan, and the speed of the condenser fan.

7. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 1, characterized in that, The cooperative control commands output by the reinforcement learning agent include the target speed of the compressor, the refrigeration flow direction of the electric valve, the target speed of the refrigeration fan, the target speed of the refrigerator fan, and the target speed of the condenser fan.

8. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 1, characterized in that, The three-temperature zone prediction model is configured to simultaneously predict three target values ​​for a certain period of time in the future: the refrigerator compartment temperature, the freezer compartment temperature, and the temperature of the freezer evaporation sensor, in order to predict the trend of the refrigerator's temperature zones.

9. The reinforcement learning control method for refrigerator temperature prediction driven according to claim 1, characterized in that, The method further includes: Before training a three-temperature zone prediction model with physical constraints using the refrigerator dataset, anomaly verification is performed on the multidimensional time-series sensing data based on the refrigerator operating mechanism and sensor range. Furthermore, after adjusting the operating status of the refrigerator actuator, the actual temperature changes of each compartment of the refrigerator are monitored, and the actual temperature changes are fed back to the reinforcement learning agent to dynamically adjust the subsequent control strategy.

10. A reinforcement learning control system driven by refrigerator temperature prediction, characterized in that, The reinforcement learning control method for refrigerator temperature prediction driven by any one of claims 1-9, the system comprising: The sensing and acquisition module is used to collect multi-dimensional time-series sensing data during the operation of the refrigerator; A data processing unit is used to preprocess the multidimensional time-series sensing data to construct a refrigerator dataset; The temperature prediction module is used to train a three-temperature zone prediction model with physical constraints using the refrigerator dataset, and output the predicted temperature of the refrigerator compartment, the predicted temperature of the freezer compartment, and the predicted temperature of the freezer evaporator sensor for future periods based on the real-time collected multi-dimensional time-series sensor data. The feature fusion unit is used to splice and fuse the predicted temperature output by the temperature prediction module with the multi-dimensional time-series sensing data collected by the sensing acquisition module to obtain a reinforcement learning state vector. The reinforcement learning control module is used to receive the reinforcement learning state vector generated by the feature fusion unit and output the multi-component collaborative control command of the refrigerator. An actuator is used to adjust the operating state of the refrigerator according to the coordinated control instructions; The sensing and acquisition module re-acquires multi-dimensional time-series sensing data during the refrigerator's operation after the actuator adjusts the operating state.

Images