Refrigerating machine room control system evaluation optimization method and system based on reinforcement learning, medium, program product and terminal

Abstract

The invention provides a refrigeration machine room control system evaluation optimization method and system based on reinforcement learning, a medium, a program product and a terminal, and the method comprises the steps: collecting the operation data of a refrigeration machine room control system in real time, and carrying out the preprocessing of the operation data; key performance indexes of the control system are extracted based on the preprocessed operation data; the key performance indexes comprise response time, fault rate, operation stability and maintenance frequency; constructing a health degree calculation formula based on the key performance indexes of the control system, and performing calculation to obtain the health degree of the control system; and evaluating the health degree of the control system by adopting a reinforcement learning method so as to optimize and adjust the refrigerating machine room control system according to an evaluation result. According to the method, multiple key health degree indexes such as response time, failure rate, operation stability and maintenance frequency are comprehensively considered, the health condition of the control system is quantitatively evaluated, an optimization adjustment strategy of the control system is guided in combination with a reinforcement learning method, and a scientific basis is provided for preventive maintenance and optimized operation.

Description

Technical Field

[0001] The present application relates to the field of refrigeration technology, and in particular to a reinforcement learning-based refrigeration room control system evaluation and optimization method, system, medium, program product and terminal. Background Art

[0002] In modern refrigeration rooms, the control system plays a central role. It is not only responsible for starting and stopping the refrigeration equipment, but also involves regulating the operating status of the equipment. These control operations have a direct impact on the overall efficiency and reliability of the system. As the technical complexity of refrigeration equipment increases, the performance indicators of the control system, such as response time, failure rate, operating stability and maintenance frequency, become particularly critical. At the same time, the evaluation and maintenance of the health of the control system is becoming increasingly important.

[0003] At present, the evaluation methods of control system health mainly include the following aspects:

[0004] (1) Response time monitoring: Monitor the execution time of control instructions to ensure that the system can respond quickly to operational requirements.

[0005] (2) Failure rate statistics: Evaluate system reliability by analyzing historical failure records and failure rate statistics.

[0006] (3) Operation stability monitoring: Continuously monitor the stability of system operation to ensure its long-term effective operation.

[0007] (4) Maintenance frequency statistics: Evaluate the health status of the system by recording and analyzing the maintenance frequency of the system.

[0008] Although the above methods can reflect some of the health conditions of the control system to a certain extent, they have some limitations in practical applications:

[0009] (1) Singleness problem: Existing evaluation methods usually only focus on a single performance indicator and cannot fully reflect the overall health of the control system.

[0010] (2) Lag problem: Some evaluation methods rely on historical data, which may result in the inability to capture the real-time health status of the system in a timely manner.

[0011] (3) Lack of comprehensive evaluation: Currently, there is a lack of a method that can comprehensively consider multiple key indicators such as response time, failure rate, operating stability and maintenance frequency to achieve a comprehensive evaluation of the health of the control system.

[0012] (4) Insufficient early warning capabilities: Most existing methods do not have the ability to warn of potential problems, making it difficult to provide effective guidance for preventive maintenance. This may result in failures not being discovered and handled in a timely manner, affecting the efficiency and reliability of system operation. Summary of the invention

[0013] In view of the above-mentioned shortcomings of the prior art, the present invention provides a refrigeration room control system evaluation and optimization method, system, medium, program product and terminal based on reinforcement learning, which are used to solve the problems that the evaluation of the refrigeration room control system in the prior art is mainly focused on a single indicator and lacks a comprehensive evaluation of the health status of the refrigeration room control system.

[0014] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a refrigeration room control system evaluation and optimization method based on reinforcement learning, including: real-time collection of operating data of the refrigeration room control system, and preprocessing of the operating data; extracting key performance indicators of the control system based on the preprocessed operating data; the key performance indicators include: response time, failure rate, operating stability and maintenance frequency; constructing a health calculation formula based on the key performance indicators of the control system and performing calculations to obtain the health of the control system; and using a reinforcement learning method to evaluate the health of the control system so as to optimize and adjust the refrigeration room control system according to the evaluation results.

[0015] In some embodiments of the first aspect of the present application, the health calculation formula is: CH = k1 × N RE +k2×N FR +k3×N OS +k4×N MF ; Among them, N RE is the normalized response efficiency, N FR is the normalized failure rate, N OS is the normalized operating stability, N MF is the normalized maintenance frequency, k1, k2, k3 and k4 are the corresponding weight coefficients respectively.

[0016] In some embodiments of the first aspect of the present application, the calculation formula of the normalized response efficiency is: N RE =NT / AT; where NT is the standard response time and AT is the actual response time.

[0017] In some embodiments of the first aspect of the present application, the calculation formula of the normalized failure rate is: N FR =AF / NF; where AF is the actual failure rate and NF is the standard failure rate.

[0018] In some embodiments of the first aspect of the present application, the calculation formula of the normalized operating stability is: N OS =AO / NO; AO is the actual operating time and NO is the standard operating time.

[0019] In some embodiments of the first aspect of the present application, the calculation formula of the normalized maintenance frequency is: N MF =AM / NM; AM is the actual maintenance times and NM is the standard maintenance times.

[0020] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a refrigeration room control system evaluation and optimization system based on reinforcement learning, including: an operation data acquisition module, used to collect the operation data of the refrigeration room control system in real time and pre-process the operation data; an indicator extraction module, used to extract the key performance indicators of the control system based on the pre-processed operation data; the key performance indicators include: response time, failure rate, operation stability and maintenance frequency; a health calculation module, used to construct a health calculation formula based on the key performance indicators of the control system and perform calculations to obtain the health of the control system; an optimization and adjustment module, used to use a reinforcement learning method to evaluate the health of the control system, so as to optimize and adjust the refrigeration room control system according to the evaluation results.

[0021] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the reinforcement learning-based refrigeration room control system evaluation and optimization method is implemented.

[0022] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the refrigeration room control system evaluation and optimization method based on reinforcement learning.

[0023] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the reinforcement learning-based refrigeration room control system evaluation and optimization method.

[0024] As described above, the reinforcement learning-based refrigeration room control system evaluation and optimization method, system, medium, program product and terminal provided in the present application have the following beneficial effects: The present invention quantitatively evaluates the health status of the control system by comprehensively considering multiple key health indicators such as response time, failure rate, operation stability and maintenance frequency, and combines reinforcement learning methods to guide the optimization adjustment strategy of the control system, providing a scientific basis for preventive maintenance and optimized operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1Shown is a flow chart of a refrigeration room control system evaluation and optimization method based on reinforcement learning in one embodiment of the present application.

[0026] Figure 2 Shown is a structural schematic diagram of a refrigeration room control system evaluation and optimization system based on reinforcement learning in one embodiment of the present application.

[0027] Figure 3 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

[0030] <1> The Analytic Hierarchy Process (AHP) is a structured technique used to deal with complex decision-making problems. It decomposes the problem into multiple levels by constructing a hierarchical model, usually including the goal level, the criterion level, and the solution level. The AHP method is particularly suitable for complex decision-making problems involving multiple criteria and solutions. It can provide a systematic and structured decision support tool. Through AHP, decision makers can deal with decision-making problems more objectively and scientifically, and improve the quality and efficiency of decision-making.

[0031] <2> Reinforcement Learning (RL) is a method of machine learning in which an agent learns how to choose actions to maximize cumulative rewards from feedback by interacting with the environment. The basic components of reinforcement learning are as follows: (1) Environment: the external environment in which the agent is located, including the current state and feedback information; (2) State: describes the current situation or condition of the system or agent; (3) Action: the action that the agent can take in a given state; (4) Reward: the feedback returned by the environment after the agent takes a certain action, which serves as a guiding signal for learning. The core of reinforcement learning is to learn the optimal strategy through continuous trial and error, and gradually optimize the actions taken by the agent in different states to obtain the maximum cumulative reward.

[0032] The embodiment of the present invention provides a method and system for evaluating and optimizing a refrigeration room control system based on reinforcement learning. By integrating various data, a health calculation model is constructed to provide a comprehensive and systematic health evaluation method for the performance and reliability of the refrigeration room, thereby improving the operating efficiency and reliability of the refrigeration room.

[0033] It should be understood that the reinforcement learning-based refrigeration room control system evaluation and optimization method in the embodiment of the present application can be applied to controllers such as ARM (Advanced RISC Machines), FPGA (Field Programmable Gate Array), SoC (System on Chip), DSP (Digital Signal Processing) or MCU (Microcontroller Unit); it can also be applied to personal computers such as desktop computers, laptops, tablet computers, smart phones, smart bracelets, smart watches, smart helmets, smart TVs, personal digital assistants (PDA for short); it can also be applied to servers, and the servers can be arranged on one or more physical servers according to various factors such as function and load, or they can be composed of distributed or centralized server clusters, which is not limited in this embodiment.

[0034] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1 The flowchart of a refrigeration room control system evaluation and optimization method based on reinforcement learning in an embodiment of the present invention is shown. The refrigeration room control system evaluation and optimization method based on reinforcement learning in this embodiment mainly includes the following steps:

[0035] Step S11: collect the operating data of the refrigeration room control system in real time and pre-process the operating data. Specifically, collect the data of each sensor and control strategy in the refrigeration room control system, and perform normalization and outlier processing.

[0036] It should be noted that any refrigeration room will have a refrigeration room control system to monitor and manage the various equipment and processes in the refrigeration room. The refrigeration room control system usually includes components such as sensors, controllers, actuators, safety systems, and remote monitoring and control. Among them, sensors are used to collect real-time data of key parameters such as temperature, humidity, pressure, flow, etc.; controllers are used to automatically adjust the operating status of refrigeration equipment, such as compressors, pumps, valves, etc. according to the data collected by sensors; actuators are used to receive instructions from controllers and perform actual control operations on equipment, such as switching, adjusting flow, etc.; safety systems are used to ensure the safe operation of the refrigeration room, including overload protection, leak detection, emergency shutdown, etc.; remote monitoring and control are used to allow operators to monitor and control the operation of the refrigeration room from a remote location.

[0037] Refrigeration room control systems mainly include local control and algorithmic control. Specifically, local control usually refers to the direct manual control of the equipment in the refrigeration room by the on-site administrator or operator. This control method allows for rapid response because there is almost no delay between decision-making and execution. Local control includes two types of operations: start-stop and regulation. Local control relies on manual judgment and operation, so it may not be as accurate or efficient as automated control in some cases.

[0038] Algorithmic control refers to the management of equipment and systems in the refrigeration room through pre-set algorithms or software programs. This control method can achieve a high degree of automation, reduce human errors and improve efficiency. Algorithmic control can optimize the operation of the refrigeration room based on data and analysis. Algorithmic control includes two types of operations: start and stop and adjustment. Algorithmic control usually requires more complex system design and maintenance, but can provide more stable and predictable performance. In actual applications, refrigeration rooms usually combine local control and algorithmic control to achieve optimal operating efficiency and reliability.

[0039] The operation data in this embodiment includes sensor data, data transmission information, and equipment failure data. The operation data is collected in real time from the refrigeration room control system. These data may come from various sensors, such as temperature sensors, pressure sensors, flow sensors, etc., data information transmitted by the equipment itself, and logs and event records of the control system.

[0040] Furthermore, the collected operation data is preprocessed by data cleaning and normalization. The purpose of data cleaning is to remove noise and outliers in the data. The process of data cleaning usually includes: checking missing values, removing erroneous data, and format unification. Among them, checking missing values ​​is to identify and process missing values ​​in the data set, which may be processed by filling, deleting or interpolating; removing erroneous data is to identify and delete or correct errors or outliers in the data set; format unification is to ensure that all data is stored in a consistent format, such as date and time format.

[0041] Data normalization is scaling the data to a specific range, such as 0 to 1 or -1 to 1. This helps balance the scales of different features in data analysis and machine learning models. In this embodiment, the reading ranges between different running data may vary greatly, such as the reading ranges of different sensors.

[0042] Step S12: extracting key performance indicators of the control system based on the pre-processed operation data; the key performance indicators include: response time, failure rate, operation stability and maintenance frequency.

[0043] Step S13: construct a health calculation formula based on the key performance indicators of the control system and perform calculations to obtain the health of the control system. Calculate the key performance indicators in the control system and calculate the health, and use the health as the input of reinforcement learning.

[0044] In one embodiment, the health calculation formula is:

[0045] CH=k1×N RE +k2×N FR +k3×N OS +k4×N MF ; (Formula 1)

[0046] Among them, N RE is the normalized response efficiency, N FR is the normalized failure rate, N OS is the normalized operating stability, N MF is the normalized maintenance frequency, k1, k2, k3 and k4 are the corresponding weight coefficients. In this embodiment, k1, k2, k3 and k4 are preferably set to: 0.3, 0.3, 0.2, 0.2, that is, CH = 0.3 × N RE +0.3×N FR +0.2×N OS +0.2×N MF The weight coefficient can be set according to actual conditions and is not limited in this embodiment.

[0047] It should be explained that the health indicators involved in the health calculation formula in the embodiment of the present invention include: response efficiency, failure rate, operation stability and maintenance frequency, and the final control system health value is obtained by calculation based on each health indicator. Among them, each health indicator in the health calculation formula is a normalized value to facilitate the subsequent health value calculation.

[0048] Specifically, k1, k2, k3 and k4 are weight coefficients corresponding to each health index, k1 is the weight coefficient corresponding to the normalized response efficiency, k2 is the weight coefficient corresponding to the normalized failure rate, k3 is the weight coefficient corresponding to the normalized operation stability, and k4 is the weight coefficient corresponding to the normalized maintenance frequency. The weight coefficient is determined based on the key indicators and specific application scenarios of the refrigeration room control system in actual operation. In this implementation, the weight coefficient can be determined by the following steps:

[0049] (1) Expert experience method: Collect the opinions of experts in the field and preliminarily determine the relative importance of each health indicator based on the actual operation experience of the control system. Experts usually assign weights to each health indicator based on their professional judgment and historical experience.

[0050] (2) Hierarchy Analysis Method: By constructing a health index hierarchy model, the factors affecting the health of the control system are decomposed into several levels to form a hierarchical structure. The judgment matrix is ​​constructed using the paired comparison method to calculate the weight of each health index. At the same time, the rationality of the judgment matrix is ​​ensured through consistency testing.

[0051] (3) Data-driven method: Utilizing a large amount of historical operating data of the refrigeration room control system, statistical analysis and machine learning algorithms are used to analyze the impact of each health indicator on the health of the control system. This method can reveal patterns and relationships in the data, thereby dynamically adjusting the weight coefficient to make it closer to the actual situation of the refrigeration room control system.

[0052] Finally, the determination of weight coefficients k1, k2, k3 and k4 combines expert experience and data analysis results to ensure the scientificity and rationality of the evaluation method. For example, the initial weight coefficients of the refrigeration room control system in this embodiment are determined according to the above steps and are set to: 0.3, 0.3, 0.2, 0.2 in sequence.

[0053] Furthermore, the acquisition of each health index is obtained by calculating the preprocessed operation data. The specific calculation method of each normalized health index is as follows:

[0054] In one embodiment, the calculation formula of the normalized response efficiency is:

[0055] N RE=NT / AT; (Formula 2)

[0056] Among them, NT is the standard response time and AT is the actual response time. The standard response time refers to the preset response time of the refrigeration room control system to a specific event or condition change under ideal or standard conditions. The actual response time refers to the actual response time of the monitored control system to the same event or condition change in actual operation. By calculating the ratio between the standard response time and the actual response time, the normalized response efficiency of the refrigeration room control system can be obtained. The larger the normalized response efficiency value, the shorter the actual response time and the more timely the response of the refrigeration room control system.

[0057] In one embodiment, the calculation formula of the normalized failure rate is:

[0058] N FR =AF / NF; (Formula 3)

[0059] Among them, AF is the actual failure rate and NF is the standard failure rate. The standard failure rate is the preset failure rate of the equipment in the refrigeration room under normal working conditions, and the actual failure rate is the actual failure rate of the equipment in the current operating time period. The normalized failure rate of the refrigeration room control system is obtained by calculating the ratio between the actual failure rate and the standard failure rate. The smaller the ratio, the lower the failure rate and the more reliable the control system.

[0060] In one embodiment, the calculation formula of the normalized operating stability is:

[0061] N OS =AO / NO; (Formula 4)

[0062] Among them, AO is the actual operating time and NO is the standard operating time. The standard operating time refers to the preset operating time of the control system under ideal conditions, which is usually based on design parameters or expected trouble-free operating time; the actual operating time is the actual monitored operating time of the control system, which refers to the actual operating time length of the control system without failure or other interruptions. The normalized operating stability of the refrigeration room control system is obtained by calculating the ratio between the actual operating time and the standard operating time. The larger the ratio, the closer the actual operating time is to or exceeds the standard operating time, which usually means that the refrigeration room control system is more stable.

[0063] In one embodiment, the calculation formula for the normalized maintenance frequency is:

[0064] N MF =AM / NM; (Formula 5)

[0065] AM is the actual maintenance frequency, and NM is the standard maintenance frequency. The standard maintenance frequency refers to the ideal maintenance frequency of the control system within the preset time period, and the actual maintenance frequency refers to the actual maintenance frequency of the control system within the preset time period. By calculating the ratio between the actual maintenance frequency and the standard maintenance frequency, the normalized maintenance frequency of the refrigeration room control system is obtained. The larger the ratio, the higher the maintenance frequency, and the healthier the control system.

[0066] Step S14: using a reinforcement learning method to evaluate the health of the control system, so as to optimize and adjust the refrigeration room control system according to the evaluation result.

[0067] Specifically, a reinforcement learning algorithm is used, with health as the core of its reward function. After each operation (such as starting and stopping equipment or adjusting parameters), the health of the control system will be used as feedback (reward), and the reinforcement learning algorithm will continuously optimize and adjust the control strategy through this feedback. The various elements of reinforcement learning are: (1) State: real-time operating data of each device in the control system (such as temperature, load, pressure, etc.); (2) Action: equipment start and stop, fan speed adjustment, water supply temperature adjustment, etc.; (3) Reward: real-time feedback based on the health score of the control system. For example, if the health score is high, a positive reward is obtained, which encourages the control system to repeat similar operations. If the health score is low, a negative reward is obtained, and the control system optimizes and adjusts the control strategy.

[0068] Furthermore, the reinforcement learning model is applied to the monitoring platform of the refrigeration room control system, which can detect the status data of the equipment in real time and optimize and adjust the control strategy. The embodiment of the present invention ensures the real-time and accuracy of the control system health score by continuously feeding back the operating parameters through the control system. The reinforcement learning model dynamically adjusts the control parameters of the refrigeration room control system based on the real-time control system health score, such as optimizing the start and stop of the chiller, adjusting the fan speed of the cooling tower, etc. Through continuous real-time feedback, the system performance is continuously optimized.

[0069] It should be emphasized that in actual equipment control systems and health assessment systems, by introducing real-time assessment of the health of the control system, problems that may lead to failures can be discovered in a timely manner, and appropriate measures can be taken to solve the problems, so as to extend the service life of the equipment and maintain its performance level, while improving the reliability of the equipment. Through reinforcement learning methods, the control system can be trained and learn how to adopt the optimal maintenance strategy under different health states to minimize maintenance costs and maximize equipment efficiency, so that the equipment operation and maintenance strategy can be dynamically adjusted to maintain the control system in a high health state and avoid health deterioration. Among them, the specific process of combining reinforcement learning with health is as follows:

[0070] (1) Health extension of state space: The state space of reinforcement learning contains the core parameters of system operation, such as pressure and temperature. Incorporating health into the state space of reinforcement learning means that health is an additional dimension that describes the health status of the device or the current control system. At this time, the reinforcement learning agent not only considers performance but also the health status of the control system when making decisions.

[0071] (2) Adding health factors to the reward function: In reinforcement learning, the reward function is a key part of designing the behavior of the intelligent agent. Health is directly integrated into the reward function as the long-term optimization goal of the control system. That is, while maintaining high performance, the device must also ensure that its health remains within a reasonable range. Reward design includes: (a) Positive rewards: Positive rewards are obtained when system performance is improved and operation goals are achieved; (b) Health rewards: Additional rewards are given when the health is high to encourage the system to maintain a high health state; (c) Punishment mechanism: When the health drops to a certain threshold, negative rewards are given to punish incorrect operations.

[0072] (3) Health prediction and adaptive adjustment: The health of a control system may change over time, due to environmental changes, operating load, and other factors. Reinforcement learning agents can evaluate changes in system health in real time through historical operating data or sensor information and adjust their operating strategies accordingly. In this way, agents can take preventive measures before the health of the system deteriorates, preventing the control system from entering a low health state.

[0073] (4) Balance between long-term health and short-term performance optimization: The reinforcement learning model needs to find a balance between short-term performance and long-term health. Over-optimization of short-term efficiency may lead to rapid aging of equipment, while over-protection of health may reduce production efficiency. By designing a reward function, the reinforcement learning model can make a trade-off between long-term benefits (maintaining high health) and short-term benefits (improving production efficiency).

[0074] (5) Adaptive adjustment based on health: Reinforcement learning agents can learn to control the dynamic changes in system health, gradually adapt to changes in equipment or environment, and dynamically adjust control strategies. For example, when the health of a device deteriorates, the agent can reduce the system load or adjust the operation mode to extend the life of the device.

[0075] For example, suppose there is a large refrigeration room containing a large number of long-term running equipment, including chillers, cooling towers, cooling pumps, freezing pumps and other equipment. Multiple sensors are installed on each device to detect parameters such as temperature, flow, and pressure, and there are sophisticated electronic control devices to adjust them. Due to the complex operation mode of the equipment, it is easy to cause control disorders or inefficiency, thereby increasing energy consumption and shortening equipment life.

[0076] In the embodiment of the present invention, by taking the health of the control system as the optimization goal of reinforcement learning and using the control system health score to guide the dynamic adjustment of the control strategy of the control system, it can be ensured that the system operates in an efficient state and reduce energy waste; through real-time monitoring and adjustment of the health of the control system, frequent start and stop of equipment and excessive load operation are reduced, and the service life of key equipment is extended; the control system health assessment can detect potential faults or abnormalities in advance, so that the control system can arrange maintenance in advance to avoid unplanned downtime and emergency repairs; the control system health score of the present invention can provide decision-making basis for managers, helping them to better plan equipment operation and maintenance strategies, improve operational efficiency and reduce maintenance costs.

[0077] It should be noted that when the health value of the refrigeration room control system is lower than the preset health threshold, the control system will automatically generate an early warning message to alert the user and provide specific maintenance suggestions to guide the user to perform equipment inspection and maintenance to restore or maintain the health of the refrigeration room control system.

[0078] Furthermore, the optimization results of the refrigeration room control system based on reinforcement learning can be evaluated to set optimization goals. This means determining the direction and measures for improvement based on the current performance, reliability, and efficiency status of the system. Specific optimization goals include minimizing energy consumption, maximizing cooling output, and extending service life.

[0079] The process of minimizing energy consumption is as follows: using health assessment results to identify links with high energy consumption; implementing an energy management system to optimize equipment operating parameters, such as compressor load, pump operating speed, etc.; using frequency conversion technology to adjust equipment operating frequency to match actual needs; and regularly maintaining and upgrading equipment to ensure its operating efficiency.

[0080] The process of maximizing cooling output is to analyze system performance data to determine the factors that affect cooling output; optimize the refrigeration cycle, including improving the flow and distribution of refrigerant; reduce equipment downtime through predictive maintenance to ensure continuous operation; and use advanced control algorithms, such as fuzzy logic or artificial intelligence, to optimize system response.

[0081] The way to extend service life is to identify potential wear and failure risks through health assessment results; and implement regular and predictive maintenance plans to reduce unexpected downtime and repairs.

[0082] At the same time, the current operating status and potential problems of the refrigeration room can be determined according to the optimization results of the refrigeration room control system based on reinforcement learning, and optimization strategies can be formulated. Optimization algorithms such as genetic algorithms and particle swarm optimization can be used to find the optimal operating parameters and maintenance plans.

[0083] It should be noted that the embodiment of the present invention monitors the operating data of the refrigeration room control system in real time, and then obtains the health assessment result, evaluates and analyzes the current health of the control system based on the assessment result, and determines the operating efficiency, stability and potential risks of the control system. According to the health assessment result, the operating parameters such as the load of the compressor, the flow rate of cooling water, etc. are automatically or semi-automatically adjusted to optimize the system performance. The present invention can also construct a health prediction model to warn of potential faults in advance, adjust the optimization strategy, formulate a preventive maintenance plan or adjust the operating mode to avoid faults.

[0084] At the same time, intuitive displays and regular reports can be provided through the visual interface, allowing operators to easily monitor system status and optimization effects, and users can intuitively understand the comprehensive health status of the refrigeration room control system in real time. Specifically, data visualization technology can also be used, such as dashboards, charts, etc. to display health assessment results, so that users can quickly and intuitively view the current status and historical performance of the refrigeration room control system. Users can more effectively monitor and manage the operating status of the refrigeration room control system, discover and solve problems in a timely manner, and improve the reliability and efficiency of the control system.

[0085] It should be emphasized that the present invention provides a comprehensive and accurate reinforcement learning-based refrigeration room control system evaluation and optimization method. By comprehensively considering multiple key health indicators such as response time, failure rate, operation stability and maintenance frequency, the health status of the control system is quantitatively evaluated, providing a scientific basis for preventive maintenance and optimized operation.

[0086] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects, and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0087] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0088] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0089] Figure 2 is a schematic block diagram of a refrigeration room control system evaluation and optimization system 200 based on reinforcement learning provided in an embodiment of the present application. Figure 2 As shown, the system includes an operation data acquisition module 210 , an indicator extraction module 220 , a health calculation module 230 and an optimization and adjustment module 240 .

[0090] The operation data acquisition module 210 is used to collect the operation data of the refrigeration room control system in real time and pre-process the operation data;

[0091] An index extraction module 220 is used to extract key performance indicators of the control system based on the pre-processed operation data; the key performance indicators include: response time, failure rate, operation stability and maintenance frequency;

[0092] A health calculation module 230 is used to construct a health calculation formula based on key performance indicators of the control system and perform calculations to obtain the health of the control system;

[0093] The optimization and adjustment module 240 is used to evaluate the health of the control system by using a reinforcement learning method, so as to optimize and adjust the refrigeration room control system according to the evaluation result.

[0094] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0095] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0096] Figure 3 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 3 As shown in FIG. 1 , the computer device includes: at least one processor 301, a memory 302, at least one network interface 303 and a user interface 305. The various components in the device are coupled together via a bus system 304. It can be understood that the bus system 304 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 304 is not described in detail. Figure 3 In the specification, various buses are labeled as bus systems.

[0097] The user interface 305 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0098] It is understood that the memory 302 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0099] The memory 302 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 300. Examples of these data include: any executable program for operating on the electronic terminal 300, such as an operating system 3021 and an application 3022; the operating system 3021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 3022 may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The evaluation and optimization method of the refrigeration room control system based on reinforcement learning provided in the embodiment of the present invention may be included in the application 3022.

[0100] The method disclosed in the above embodiment of the present invention can be applied to the processor 301, or implemented by the processor 301. The processor 301 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 301 or the instruction in the form of software. The above processor 301 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 301 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 301 can be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0101] In an exemplary embodiment, the electronic terminal 300 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0102] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes the refrigeration room control system evaluation and optimization method based on reinforcement learning in any of the embodiments shown.

[0103] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium, which stores a program code. When the program code runs on a computer, the computer executes the refrigeration room control system evaluation and optimization method based on reinforcement learning of any of the embodiments shown.

[0104] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0105] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0110] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs), etc.).

[0111] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0113] In summary, the present application provides a refrigeration room control system evaluation and optimization method, system, medium, program product and terminal based on reinforcement learning, including: real-time collection of the operation data of the refrigeration room control system, and preprocessing of the operation data; extracting the key performance indicators of the control system based on the preprocessed operation data; the key performance indicators include: response time, failure rate, operation stability and maintenance frequency; constructing a health calculation formula based on the key performance indicators of the control system and calculating to obtain the health of the control system; using a reinforcement learning method to evaluate the health of the control system, so as to optimize and adjust the refrigeration room control system according to the evaluation results. The present invention quantitatively evaluates the health status of the control system by comprehensively considering multiple key health indicators such as response time, failure rate, operation stability and maintenance frequency, and combines the reinforcement learning method to guide the optimization adjustment strategy of the control system, providing a scientific basis for preventive maintenance and optimized operation. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0114] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A refrigeration room control system evaluation and optimization method based on reinforcement learning, characterized in that: include: Collecting the operating data of the refrigeration room control system in real time and preprocessing the operating data; Extract key performance indicators of the control system based on preprocessed operation data; The key performance indicators include: response time, failure rate, operation stability and maintenance frequency; Based on the key performance indicators of the control system, a health calculation formula is constructed and calculated to obtain the health of the control system; The health of the control system is evaluated by a reinforcement learning method, so as to optimize and adjust the refrigeration room control system according to the evaluation result.

2. The refrigeration room control system evaluation and optimization method based on reinforcement learning according to claim 1 is characterized in that: The health calculation formula is: CH=k1×N RE +k2×N FR +k3×N OS +k4×N MF ; Among them, N RE is the normalized response efficiency, N FR is the normalized failure rate, N OS is the normalized operating stability, N MF is the normalized maintenance frequency, k1, k2, k3 and k4 are the corresponding weight coefficients respectively.

3. The refrigeration room control system evaluation and optimization method based on reinforcement learning according to claim 2 is characterized in that: The calculation formula of the normalized response efficiency is: N RE =NT / AT; Among them, NT is the standard response time and AT is the actual response time.

4. The refrigeration room control system evaluation and optimization method based on reinforcement learning according to claim 2 is characterized in that: The calculation formula of the normalized failure rate is: N FR =AF / NF; Among them, AF is the actual failure rate and NF is the standard failure rate.

5. The refrigeration room control system evaluation and optimization method based on reinforcement learning according to claim 2 is characterized in that: The calculation formula of the normalized operation stability is: N OS =ON / NO; Among them, AO is the actual operating time and NO is the standard operating time.

6. The refrigeration room control system evaluation and optimization method based on reinforcement learning according to claim 2 is characterized in that: The calculation formula for the normalized maintenance frequency is: N MF =AM / NM; Among them, AM is the actual maintenance number and NM is the standard maintenance number.

7. A refrigeration room control system evaluation and optimization system based on reinforcement learning, characterized in that: include: An operation data acquisition module is used to collect the operation data of the refrigeration room control system in real time and pre-process the operation data; An index extraction module is used to extract key performance indicators of the control system based on pre-processed operation data; The key performance indicators include: response time, failure rate, operation stability and maintenance frequency; A health calculation module is used to construct a health calculation formula based on the key performance indicators of the control system and perform calculations to obtain the health of the control system; The optimization and adjustment module is used to evaluate the health of the control system by using a reinforcement learning method, so as to optimize and adjust the refrigeration room control system according to the evaluation result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the refrigeration room control system evaluation and optimization method based on reinforcement learning described in any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The computer program product includes computer program codes, and when the computer program codes are executed on a computer, the computer is enabled to implement the refrigeration room control system evaluation and optimization method based on reinforcement learning as claimed in any one of claims 1 to 6.

10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the refrigeration room control system evaluation and optimization method based on reinforcement learning as described in any one of claims 1 to 6.

Images