An intelligent supervision system for high-risk operations based on reinforcement learning

Abstract

The application relates to the technical field of construction operation supervision, in particular to an intelligent high-risk operation supervision system based on reinforcement learning, which comprises a data storage module, a data acquisition module used for collecting operation scene data in a target operation area in real time, a data analysis module used for determining operation environment features, operation equipment features and operation behavior features and determining operation target representation values in the target operation area, a model simulation module used for constructing an operation digital twin model of the target operation area based on historical operation scene data and historical execution actions, simulating an operation process to obtain simulated execution actions, and a risk assessment module used for determining key execution actions based on the operation target representation values and a preset reinforcement learning model, determining a supervision risk category and obtaining target execution actions. The application can improve the supervision accuracy and real-time performance of high-risk operation processes and realize dynamic adjustment of supervision strategies.

Description

Technical Field

[0001] This invention relates to the field of construction operation supervision technology, and in particular to an intelligent supervision system for high-risk operations based on reinforcement learning. Background Technology

[0002] High-risk operations, such as chemical production, working at heights, and mining, involve complex and variable working environments with numerous potential hazards. The operating status of equipment, changes in the working environment, and the behavior of workers can all significantly impact operational safety. Traditional monitoring methods rely primarily on manual inspections and simple monitoring equipment, which are insufficient for real-time and comprehensive understanding of the various conditions at the work site, making it prone to regulatory loopholes and leading to safety accidents.

[0003] With the rapid development of information technology, technologies such as big data, artificial intelligence, and digital twins have gradually matured and been applied in various fields. In the supervision of high-risk operations, these technologies can be used to better process and analyze large amounts of operational data, enabling real-time monitoring, prediction, and intelligent decision-making of the operational process. Reinforcement learning, as an important branch of artificial intelligence, can find optimal decision-making strategies through continuous learning and optimization in complex environments, providing a new approach and method for the supervision of high-risk operations.

[0004] Chinese patent application publication number CN117911953A discloses an artificial intelligence-based power operation risk monitoring and identification system and method, including: Step S10: Collecting real-time image data of the current power operation site, and initially extracting personnel features, equipment features, and scene features at the operation site; Step S20: Determining the personnel identity category based on the personnel features to obtain the corresponding first personnel risk level, and obtaining the corresponding environmental risk level based on the equipment features and scene features; when the first personnel risk level is lower than the environmental risk level, proceed directly to step S40, otherwise proceed to step S30; Step S30: Performing secondary extraction on the personnel features and equipment features based on the personnel identity category and scene features; wherein, the secondary extracted personnel features and equipment features are different depending on the different personnel identity categories and scene features; determining the corresponding second personnel risk level based on the secondary extracted personnel features and equipment features; when the second personnel risk level is higher than a threshold, proceeding to step S40, otherwise returning to step S10; Step S40: Providing a high-risk warning to the corresponding personnel. The above steps have the following disadvantages compared to the above technical solutions.

[0005] Existing technologies have the following problems: they rely solely on feature extraction and threshold determination based on single image data, lack the ability to learn and optimize dynamically, and cannot adjust their strategies in a timely manner once the working environment changes or new risk factors emerge, resulting in inaccurate risk assessments or lagging regulatory measures. Summary of the Invention

[0006] To address this, the present invention provides an intelligent monitoring system for high-risk operations based on reinforcement learning, which overcomes the problems of existing technologies that rely solely on feature extraction and threshold judgment based on single image data, lacking dynamic learning and optimization capabilities, resulting in inaccurate risk assessment and lagging regulatory measures.

[0007] To achieve the above objectives, the present invention provides an intelligent monitoring system for high-risk operations based on reinforcement learning, comprising:

[0008] The data storage module is used to store historical operation scene data and historical execution actions of the target operation equipment within the target operation area. The operation scene data includes operation environment data, operation data of the operation equipment, and behavior data of the operators.

[0009] The data acquisition module is used to collect work scene data in the target work area in real time;

[0010] The data analysis module, which is connected to the data acquisition module, is used to determine the characteristics of the work environment based on the work environment data within the target time period, to determine the characteristics of the work equipment based on the operation data of the work equipment within the target time period, and to determine the characteristics of the work behavior based on the behavior data of the workers within the target time period, and to determine the target representation value of the work within the target work area based on the characteristics of the work environment, the characteristics of the work equipment, and the characteristics of the work behavior.

[0011] The model simulation module is connected to the data storage module and the data acquisition module respectively. It is used to construct a digital twin model of the target work area based on the historical work scenario data and the historical execution actions, and to simulate the work process based on the current work environment data, the current work equipment operation data, the current work personnel behavior data and the digital twin model of the work to obtain the simulated execution actions.

[0012] The risk assessment module is connected to the data analysis module and the model simulation module respectively. It is used to determine key execution actions based on the operation target representation value and the preset reinforcement learning model, and to determine the regulatory risk category based on the comparison results between the simulated execution actions and the key execution actions, so as to obtain the target execution actions.

[0013] Furthermore, the data analysis module includes:

[0014] An environmental analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the work environment based on the work environment data within the target time period;

[0015] The equipment analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the operating equipment based on the operating data of the operating equipment within the target time period.

[0016] The behavior analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the work behavior based on the behavior data of the workers within the target time period;

[0017] The target analysis unit is connected to the environment analysis unit, the equipment analysis unit, and the behavior analysis unit, respectively, and is used to determine abnormal operation characteristics based on the operation environment characteristics, the operation equipment characteristics, and the operation behavior characteristics, and to determine the operation target representation value in the target operation area based on the abnormal operation characteristics.

[0018] Furthermore, the model simulation module includes:

[0019] The model building unit is connected to the data storage module and is used to generate several action simulation scenarios based on the historical operation scenario data and the historical execution actions, and to build a digital twin model of the target operation area based on each of the action simulation scenarios.

[0020] The action simulation unit is connected to the data acquisition module and the model building unit respectively. It is used to generate a current operation simulation scenario based on the current operation environment data, the current operation equipment operation data and the current operation personnel behavior data, and to simulate the operation process based on the current operation simulation scenario and the operation digital twin model to obtain simulated execution actions.

[0021] Furthermore, the risk assessment module includes:

[0022] An action analysis unit, which is connected to the data analysis module, is used to determine key execution actions based on the task target representation value and a preset reinforcement learning model;

[0023] The action evaluation unit is connected to both the model simulation module and the action analysis unit, and is used to determine action evaluation features based on the comparison results between the simulated action and the key action.

[0024] A risk analysis unit, connected to the action assessment unit, is used to determine the regulatory risk category based on the action assessment characteristics, and to determine the target action to be performed based on the regulatory risk category.

[0025] Furthermore, the environmental analysis unit constructs an environmental change curve based on the work environment data within the target time period, and determines the environmental abrupt change region based on the environmental change curve to identify the characteristics of the work environment.

[0026] Furthermore, the equipment analysis unit determines several related equipment parameters based on the equipment operation data within the target time period, and determines the equipment characteristics based on each related equipment parameter.

[0027] Furthermore, the behavior analysis unit determines the characteristics of the work behavior based on the comparison results of the worker behavior data within the target time period and the standard behavior data.

[0028] Furthermore, the target analysis unit determines the target representation value of the operation within the target operation area based on the comparison results between the abnormal operation features and the standard operation features.

[0029] Furthermore, the risk analysis unit determines the regulatory risk category based on the action assessment characteristics and a preset assessment characteristic comparison table, wherein the regulatory risk category includes a high-impact risk category and a low-impact risk category.

[0030] Furthermore, the risk analysis unit determines the target action based on the regulatory risk category, including:

[0031] If the regulatory risk category is a low-impact risk category, then the simulated execution action will be determined as the target execution action;

[0032] If the regulatory risk category is a high-impact risk category, then the key execution action will be identified as the target execution action.

[0033] Compared with existing technologies, the advantages of this invention are as follows: By setting up a data storage module to store historical work scenario data and historical execution actions, this invention provides data support for subsequent work process simulation and risk assessment. By setting up a data acquisition module to collect work scenario data within the target work area in real time, it ensures the timeliness and relevance of regulatory decisions. By setting up a data analysis module to determine the characteristics of the work environment, equipment, and behavior, and further determine the target representation values ​​within the target work area, this invention performs correlation analysis on different types of work data, comprehensively considering the impact of multiple aspects such as the work environment, equipment, and personnel on the work process, thereby improving the accuracy of supervision of high-risk work processes. By setting up a model simulation module to construct a digital twin model of the work based on historical work scenario data and digital twin technology, and combining it with current work data to simulate the work process, this invention achieves accurate mapping between the physical and virtual scenarios, enabling the prediction of possible situations during the work process, avoiding the amplification of risks due to blind operation, and improving the model's fit and prediction accuracy to the actual work process. By setting up a risk assessment module, combining operational target representation values ​​and reinforcement learning models, operational risks can be automatically and intelligently assessed, key execution actions can be identified, and regulatory risk categories and target execution actions can be determined based on the comparison results between simulated execution actions and key execution actions. This avoids the subjectivity and inaccuracy of manual assessment, improves the efficiency and accuracy of risk assessment, and dynamically assesses operational risks based on real-time operational data and simulation results. This enables dynamic adjustment of regulatory strategies, improving the system's adaptability and risk response capabilities.

[0034] Furthermore, the data analysis module of this invention, by setting up an environmental analysis unit, determines the characteristics of the working environment based on the working environment data within the target time period, accurately extracting features reflecting the state of the working environment and enabling timely detection of environmental anomalies. By setting up an equipment analysis unit, based on the operating data of the working equipment within the target time period, it can accurately monitor the operating status of the equipment and extract equipment features, helping to promptly detect potential equipment failures or abnormal operating conditions. By setting up a behavior analysis unit, it analyzes the behavioral data of the workers within the target time period, accurately identifying the behavioral characteristics of the workers, assessing the behavioral risks of the workers, and helping to promptly detect violations, fatigue, or unsafe behaviors. By setting up a target analysis unit, comprehensively considering the characteristics of the working environment, the characteristics of the working equipment, and the behavioral characteristics of the workers, it can identify abnormal working characteristics, such as operating high-risk equipment in harsh environments or personnel violations leading to equipment malfunctions, thereby determining the target value of the work within the target working area, quantifying the overall risk status within the target working area, and improving the accuracy and effectiveness of regulatory decisions.

[0035] Furthermore, the model simulation module of this invention, by setting up a model building unit, can generate multiple action simulation scenarios based on historical work scenario data and historical execution actions, thereby constructing a digital twin model of the target work area. This accurately reflects various factors and interrelationships during the work process, providing a highly realistic virtual environment for subsequent simulation and analysis. By setting up an action simulation unit, based on current work environment data, current equipment operation data, and current personnel behavior data, a current work simulation scenario is generated. Combined with the work digital twin model, the work process is simulated to obtain simulated execution actions. This can be dynamically adjusted according to the current work situation, ensuring the timeliness and accuracy of the simulation results. The combination of the model building unit and the action simulation unit enables the system to perform accurate modeling and real-time simulation based on historical and real-time data, improving the efficiency and accuracy of supervision.

[0036] Furthermore, the risk assessment module of this invention, by setting up an action analysis unit, can identify key execution actions based on the target representation value of the operation and a preset reinforcement learning model. The preset reinforcement learning model can learn the optimal decision-making strategy based on historical and real-time data, thereby providing intelligent support for the determination of key execution actions. By setting up an action evaluation unit, the simulated execution actions and key execution actions can be compared to generate action evaluation features, quantifying the differences between the executed actions and the simulated actions. By setting up a risk analysis unit, based on the action evaluation features, the regulatory risk category can be determined, which helps to clarify the nature and severity of the risk and provides a basis for taking corresponding regulatory measures. According to the regulatory risk category, the target execution action is determined, ensuring that the most appropriate action can be taken under different risk conditions to reduce risk and ensure operational safety, thereby improving the efficiency and accuracy of supervision. Attached Figure Description

[0037] Figure 1 This is a structural block diagram of a high-risk operation intelligent monitoring system based on reinforcement learning, according to an embodiment of the present invention.

[0038] Figure 2 This is a structural block diagram of the data analysis module in an embodiment of the present invention;

[0039] Figure 3 This is a structural block diagram of the model simulation module in an embodiment of the present invention;

[0040] Figure 4 This is a structural block diagram of the risk assessment module in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0042] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0043] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0044] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0045] Please see Figures 1-4 As shown, Figure 1 This is a structural block diagram of a high-risk operation intelligent monitoring system based on reinforcement learning, according to an embodiment of the present invention. Figure 2 This is a structural block diagram of the data analysis module in an embodiment of the present invention; Figure 3 This is a structural block diagram of the model simulation module in an embodiment of the present invention; Figure 4 This is a structural block diagram of the risk assessment module in an embodiment of the present invention; an embodiment of the present invention provides an intelligent monitoring system for high-risk operations based on reinforcement learning, comprising:

[0046] The data storage module is used to store historical operation scene data and historical execution actions of the target operation equipment within the target operation area. The operation scene data includes operation environment data, operation data of the operation equipment, and behavior data of the operators.

[0047] During implementation, the operational environment data includes parameters such as ambient temperature, humidity, air pressure, toxic gas concentration, light intensity, and wind speed. The operational equipment data includes parameters such as equipment temperature, voltage, current, power, and vibration frequency. The personnel behavior data includes parameters such as identification, tool identification, protective equipment identification, and equipment operation identification. Actions performed include, but are not limited to, turning on / off operational equipment, adjusting operational equipment, issuing on-site alarms, and sending remote alarms. In practical applications, these settings can be customized based on actual conditions. For example, they can be configured according to the following: "Action Subject (person / system), timestamp, associated object (equipment / area / personnel), action content, execution result (success / failure / partial completion), and associated risks (e.g., whether the hazard was eliminated after the action)."

[0048] The data acquisition module is used to collect work scene data in the target work area in real time;

[0049] Specifically, the data acquisition module includes an environmental monitoring unit for real-time acquisition of work environment data within the target work area, an equipment monitoring unit for real-time acquisition of operation data of several work devices within the target work area, and a behavior monitoring unit for real-time acquisition of worker behavior data within the target work area.

[0050] In implementation, the specific structure of the environmental monitoring unit, equipment monitoring unit, and behavior monitoring unit is not limited, as this is existing technology and will not be elaborated further.

[0051] The data analysis module, which is connected to the data acquisition module, is used to determine the characteristics of the work environment based on the work environment data within the target time period, to determine the characteristics of the work equipment based on the operation data of the work equipment within the target time period, and to determine the characteristics of the work behavior based on the behavior data of the workers within the target time period, and to determine the target representation value of the work within the target work area based on the characteristics of the work environment, the characteristics of the work equipment, and the characteristics of the work behavior.

[0052] Specifically, the data analysis module includes:

[0053] An environmental analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the work environment based on the work environment data within the target time period;

[0054] Specifically, the environmental analysis unit constructs an environmental change curve based on the work environment data within the target time period, and determines the environmental abrupt change region based on the environmental change curve to determine the characteristics of the work environment.

[0055] In implementation, an environmental change curve is constructed based on the operational environment data within the target time period. Time is used as the independent variable, and environmental temperature, humidity, air pressure, toxic gas concentration, light intensity, and wind speed are used as dependent variables. Data points within the target time period are sequentially connected to form the environmental change curve. This curve is then compared to a standard environmental change curve (implementers can set standard environmental data based on actual conditions or historical data of environmental parameters from qualified operational processes, and construct a standard environmental change curve accordingly) to determine the anomaly index corresponding to any environmental change curve. For example, correlation coefficients (such as Pearson correlation analysis) are calculated. If the anomaly index corresponding to any environmental change curve is less than a preset anomaly index (preferably, the preset anomaly index ranges from 0.6 to 0.7), the environmental change curve is marked. A difference analysis is then performed between the marked environmental change curve and the standard environmental change curve. Implementers can set difference thresholds for each operational environment parameter according to actual conditions. When the environmental difference between the marked environmental change curve and the standard environmental change curve at any collection time point exceeds the difference threshold, an environmental abrupt change is identified, and the area exceeding the difference threshold is designated as the environmental abrupt change region. For data such as ambient temperature, humidity, air pressure, toxic gas concentration, light intensity, and wind speed, operational environment parameters (e.g., temperature) in areas with abrupt environmental changes are identified as abnormal operational environment parameters. Correlation analysis is then performed on these abnormal parameters to determine the characteristics of the operational environment. For example, data analysis can be conducted based on historical operational environment data. An environmental training dataset can be constructed using abnormal operational environment parameters that have passed compliance checks. Operational environment characteristics during the work process are then labeled to train an initial neural network model, resulting in an environmental analysis model. Inputting the identified abnormal operational environment parameters into the environmental analysis model yields the operational environment characteristics output by the model.

[0056] The equipment analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the operating equipment based on the operating data of the operating equipment within the target time period.

[0057] Specifically, the equipment analysis unit determines several related equipment parameters based on the operating data of the equipment within the target time period, and determines the characteristics of the equipment based on each related equipment parameter.

[0058] In implementation, statistical analysis is performed on the operating data of the equipment within the target time period to calculate the correlation coefficients (such as Pearson correlation coefficient, Spearman rank correlation coefficient, etc.) between the operating parameters of each piece of equipment. Operating parameters with any correlation coefficient greater than a preset correlation coefficient are identified as associated operating parameters. These associated parameters are then clustered to obtain several cluster groups, each containing several operating parameters. The cluster group with the most operating parameters is identified as the key cluster group, which is input into the equipment analysis model to obtain the operating equipment characteristics output by the model. In practical applications, data analysis can be performed based on historical operating data. Key cluster groups that passed the qualification test in historical data can be used to construct an equipment training dataset, and the operating equipment characteristics during the operation process can be labeled to train the initial neural network model, resulting in the equipment analysis model.

[0059] The behavior analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the work behavior based on the behavior data of the workers within the target time period;

[0060] Specifically, the behavior analysis unit determines the characteristics of the work behavior based on the comparison results of the worker behavior data within the target time period and the standard behavior data.

[0061] In implementation, statistical analysis is performed on the worker behavior data within the target time period to calculate the similarity between the worker behavior data at any collection time point and the standard behavior data (the actual implementers can set the standard behavior data based on the actual situation or the worker behavior specifications in the qualified operation process in historical data). (In practical application, the similarity can be calculated using any similarity calculation method, such as cosine similarity calculation method, Euclidean distance calculation method, etc.). The collection time point corresponding to any worker behavior parameter with a similarity less than the preset similarity (preferably, the preset similarity range is set to 0.65 to 0.75) is determined as the key collection time point. The worker behavior parameters corresponding to each key collection time point are analyzed to determine the similarity between each key collection time point. The worker behavior parameters corresponding to the key collection time points with a similarity greater than the preset similarity are determined as key worker behavior parameters. The worker behavior parameters are then input into the behavior analysis model to obtain the work behavior characteristics output by the behavior analysis model. In practical applications, data analysis can be performed based on the behavior data of operators in historical data. A behavior training dataset can be constructed by taking the key behavior parameters of operators who have passed the qualification test from historical data, and the behavior characteristics of operators in the operation process can be labeled to train the initial neural network model and obtain the behavior analysis model.

[0062] The target analysis unit is connected to the environment analysis unit, the equipment analysis unit, and the behavior analysis unit, respectively, and is used to determine abnormal operation characteristics based on the operation environment characteristics, the operation equipment characteristics, and the operation behavior characteristics, and to determine the operation target representation value in the target operation area based on the abnormal operation characteristics.

[0063] Specifically, the target analysis unit determines the target representation value of the operation within the target operation area based on the comparison results between the abnormal operation characteristics and the standard operation characteristics.

[0064] In implementation, abnormal operation characteristics YA1, YA2, ..., YA j , ..., YA m YB1, YB2, ..., YB i , ..., YB n YC1, YC2, ..., YC g , ..., YC h Standard operating procedure features EA1, EA2, ..., EA j , ..., EA m EB1, EB2, ..., EB i , ..., EB n EC1, EC2, ..., EC g EC h Then the target performance value ZB = sqrt(∑ m j＝1 (YA j -EA j ) 2 +∑ n i＝1 (YB i -EB i ) 2 +∑ h g＝1 (YC g -EC g ) 2 ), where j = 1, 2, ..., m; i = 1, 2, ..., n; g = 1, 2, ..., h; m is the number of environmental features, n is the number of equipment features, h is the number of behavioral features, and (m+n+h) is the number of operational features. YA j For the j-th environmental feature value, YB i YC is the feature value of the i-th device. g For the g-th behavioral feature value, EA j For the j-th standard environmental characteristic value, EB i For the i-th standard device characteristic value, EC gLet g be the characteristic value of the g-th standard behavior, and sqrt() be the preset square root determination function.

[0065] This invention's data analysis module, through an environmental analysis unit, determines operational environment characteristics based on operational environment data within a target time period, accurately extracting features reflecting the operational environment's state and enabling timely detection of environmental anomalies. By setting up an equipment analysis unit, it accurately monitors equipment operating status and extracts equipment features based on operational equipment data within the target time period, facilitating the timely detection of potential equipment malfunctions or abnormal operation. By setting up a behavior analysis unit, it analyzes worker behavior data within the target time period, accurately identifying worker behavior characteristics and assessing behavioral risks, thus helping to promptly detect violations, fatigue, or unsafe behaviors. By setting up a target analysis unit, it comprehensively considers operational environment characteristics, equipment characteristics, and worker behavior characteristics, identifying abnormal operational features, such as operating high-risk equipment in harsh environments or personnel violations leading to equipment malfunctions. This allows for the determination of operational target representation values ​​within the target operational area, quantifying the overall risk status within the target operational area and improving the accuracy and effectiveness of regulatory decisions.

[0066] The model simulation module is connected to the data storage module and the data acquisition module respectively. It is used to construct a digital twin model of the target work area based on the historical work scenario data and the historical execution actions, and to simulate the work process based on the current work environment data, the current work equipment operation data, the current work personnel behavior data and the digital twin model of the work to obtain the simulated execution actions.

[0067] Specifically, the model simulation module includes:

[0068] The model building unit is connected to the data storage module and is used to generate several action simulation scenarios based on the historical operation scenario data and the historical execution actions, and to build a digital twin model of the target operation area based on each of the action simulation scenarios.

[0069] The action simulation unit is connected to the data acquisition module and the model building unit respectively. It is used to generate a current operation simulation scenario based on the current operation environment data, the current operation equipment operation data and the current operation personnel behavior data, and to simulate the operation process based on the current operation simulation scenario and the operation digital twin model to obtain simulated execution actions.

[0070] In implementation, each action simulation scenario includes the working environment of the target work area, the operation status of the work equipment, and the behavior of the workers. Each action simulation scenario has a corresponding execution action. The work process is simulated based on each action simulation scenario to generate a digital twin model of the target work area.

[0071] In one specific embodiment, several training samples can be generated based on historical work scenario data and corresponding action simulation scenarios. The machine learning model can then be trained using these training samples to obtain a scenario analysis model. Inputting current work environment data, current equipment operation data, and current personnel behavior data into the scenario analysis model can generate the current work simulation scenario.

[0072] Understandably, a digital twin model of a work operation can simulate the work process based on the current work simulation scenario, identify potential anomalies or problems based on the simulation results, and generate optimization suggestions, such as adjusting equipment operating parameters, optimizing work processes, or adjusting personnel behavior to ensure work safety, thereby obtaining simulated execution actions.

[0073] This invention's model simulation module, through the establishment of a model building unit, can generate multiple action simulation scenarios based on historical work scenario data and historical execution actions. This constructs a digital twin model of the target work area, accurately reflecting various factors and interrelationships during the work process, providing a highly realistic virtual environment for subsequent simulation and analysis. By setting up an action simulation unit, based on current work environment data, current equipment operation data, and current personnel behavior data, a current work simulation scenario is generated. This scenario is then combined with the work digital twin model to simulate the work process, resulting in simulated execution actions. The simulation can be dynamically adjusted according to the current work situation, ensuring the timeliness and accuracy of the simulation results. The combination of the model building unit and the action simulation unit enables the system to perform accurate modeling and real-time simulation based on historical and real-time data, improving the efficiency and accuracy of supervision.

[0074] The risk assessment module is connected to the data analysis module and the model simulation module respectively. It is used to determine key execution actions based on the operation target representation value and the preset reinforcement learning model, and to determine the regulatory risk category based on the comparison results between the simulated execution actions and the key execution actions, so as to obtain the target execution actions.

[0075] Specifically, the risk assessment module includes:

[0076] An action analysis unit, which is connected to the data analysis module, is used to determine key execution actions based on the task target representation value and a preset reinforcement learning model;

[0077] In implementation, the target representation value is input into a preset reinforcement learning model to obtain the key execution actions output by the preset reinforcement learning model. The target representation value is used as the state input, and an action space can be constructed based on historical execution actions, including all possible execution actions, such as adjusting equipment parameters and optimizing personnel operations. A reward function is set, for example, a positive reward is given for improved work efficiency, and a negative reward is given for increased equipment failure rate. The key execution action is the optimal action with the highest reward value corresponding to the target representation value.

[0078] The action evaluation unit is connected to both the model simulation module and the action analysis unit, and is used to determine action evaluation features based on the comparison results between the simulated action and the key action.

[0079] In practice, motion evaluation features are used to assess the differences between simulated actions and key actions.

[0080] In a specific embodiment, action evaluation characteristics can be quantitatively determined through action matching evaluation, effect evaluation, and risk evaluation. For action matching evaluation, each dimension of the executed action can be assigned a value and quantified based on historical executed actions, thus simulating the executed actions R1, R2, ..., R p , ..., R q With key execution actions T1, T2, ..., T p ,…,T q Action matching evaluation value MP = (∑ q p＝1 R p ×T p ) / (sqrt(∑ q p＝1 (R p ) 2 )×sqrt(∑ q p＝1 (T p ) 2Where p = 1, 2, ..., q; q is the dimension of the action; for effect evaluation, the work efficiency of the simulated action and the key action after the execution of the first preset time period is evaluated based on the work digital twin model, and the ratio of the work efficiency of the simulated action to the work efficiency of the key action is determined as the effect evaluation value; for risk evaluation, the abnormal situation of the work after the execution of the simulated action and the key action is evaluated based on the work digital twin model, and the ratio of the number of abnormalities of the simulated action to the number of abnormalities of the key action is determined as the risk evaluation value. Therefore, the action evaluation features include action matching evaluation value, effect evaluation value, and risk evaluation value. In actual implementation, preferably, the first preset time period is set to a range of 1h to 3h, and the second preset time period is set to a range of 3h to 5h.

[0081] A risk analysis unit, connected to the action assessment unit, is used to determine the regulatory risk category based on the action assessment characteristics, and to determine the target action to be performed based on the regulatory risk category.

[0082] Specifically, the risk analysis unit determines the regulatory risk category based on the action assessment characteristics and a preset assessment characteristic comparison table, wherein the regulatory risk category includes a high-impact risk category and a low-impact risk category.

[0083] In implementation, the action assessment characterization value DZ can be determined based on action assessment characteristics, where DZ = (MP × FP) / XP, and MP is the action matching assessment value, FP is the risk assessment value, and XP is the effect assessment value. Practitioners can set up a preset assessment characteristic comparison table based on historical data of action assessment characteristics that have passed the compliance test. For example, if the action assessment characterization value is greater than the preset assessment characterization value, the regulatory risk category is determined to be a high-impact risk category; if the action assessment characterization value is less than or equal to the preset assessment characterization value, the regulatory risk category is determined to be a low-impact risk category. Preferably, the preset assessment characterization value is set to a range of 0.9 to 1.5.

[0084] Specifically, the risk analysis unit determines the target action based on the regulatory risk category, including:

[0085] If the regulatory risk category is a low-impact risk category, then the simulated execution action will be determined as the target execution action;

[0086] If the regulatory risk category is a high-impact risk category, then the key execution action will be identified as the target execution action.

[0087] This invention's risk assessment module, through the establishment of an action analysis unit, identifies key execution actions based on the target representation value of the operation and a pre-set reinforcement learning model. The pre-set reinforcement learning model learns the optimal decision-making strategy based on historical and real-time data, thus providing intelligent support for determining key execution actions. By setting up an action evaluation unit, it compares simulated execution actions with key execution actions, generating action evaluation features and quantifying the differences between the actual and simulated actions. Through the risk analysis unit, based on the action evaluation features, it determines the category of regulatory risks, helping to clarify the nature and severity of the risks and providing a basis for taking appropriate regulatory measures. Based on the category of regulatory risks, it determines the target execution actions, ensuring that the most appropriate actions are taken under different risk conditions to reduce risks and ensure operational safety, thereby improving the efficiency and accuracy of supervision.

[0088] Specifically, this invention stores historical work scenario data and historical actions through a data storage module, providing data support for subsequent work process simulation and risk assessment. A data acquisition module collects work scenario data within the target work area in real time, ensuring the timeliness and relevance of regulatory decisions. A data analysis module identifies work environment characteristics, equipment characteristics, and behavioral characteristics, further determining the target performance values ​​within the target work area. Correlation analysis of different types of work data, comprehensively considering the impact of the work environment, equipment, and personnel on the work process, improves the accuracy of supervision of high-risk work processes. A model simulation module constructs a digital twin model of the work based on historical work scenario data and digital twin technology, and simulates the work process using current work data. This achieves a precise mapping between the physical and virtual scenarios, predicting potential situations during the work process, avoiding amplified risks due to blind operation, and improving the model's fit and prediction accuracy to the actual work process. By setting up a risk assessment module, combining operational target representation values ​​and reinforcement learning models, operational risks can be automatically and intelligently assessed, key execution actions can be identified, and regulatory risk categories and target execution actions can be determined based on the comparison results between simulated execution actions and key execution actions. This avoids the subjectivity and inaccuracy of manual assessment, improves the efficiency and accuracy of risk assessment, and dynamically assesses operational risks based on real-time operational data and simulation results. This enables dynamic adjustment of regulatory strategies, improving the system's adaptability and risk response capabilities.

[0089] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A high-risk operation intelligent supervision system based on reinforcement learning, characterized in that, include: The data storage module is used to store historical operation scene data and historical execution actions of the target operation equipment within the target operation area. The operation scene data includes operation environment data, operation data of the operation equipment, and behavior data of the operators. The data acquisition module is used to collect work scene data in the target work area in real time; The data analysis module, which is connected to the data acquisition module, is used to determine the characteristics of the work environment based on the work environment data within the target time period, to determine the characteristics of the work equipment based on the operation data of the work equipment within the target time period, and to determine the characteristics of the work behavior based on the behavior data of the workers within the target time period, and to determine the target representation value of the work within the target work area based on the characteristics of the work environment, the characteristics of the work equipment, and the characteristics of the work behavior. The model simulation module is connected to the data storage module and the data acquisition module respectively. It is used to construct a digital twin model of the target work area based on the historical work scenario data and the historical execution actions, and to simulate the work process based on the current work environment data, the current work equipment operation data, the current work personnel behavior data and the digital twin model of the work to obtain the simulated execution actions. The risk assessment module is connected to the data analysis module and the model simulation module respectively. It is used to determine key execution actions based on the operation target representation value and the preset reinforcement learning model, and to determine the regulatory risk category based on the comparison results between the simulated execution actions and the key execution actions, so as to obtain the target execution action. The key execution action is the optimal action with the highest reward value output by the preset reinforcement learning model when the operation target representation value is used as the state input of the preset reinforcement learning model. The model simulation module includes: The model building unit is connected to the data storage module and is used to generate several action simulation scenarios based on the historical operation scenario data and the historical execution actions, and to build a digital twin model of the target operation area based on each of the action simulation scenarios. The action simulation unit is connected to the data acquisition module and the model building unit respectively. It is used to generate a current operation simulation scenario based on the current operation environment data, the current operation equipment operation data and the current operation personnel behavior data, and to simulate the operation process based on the current operation simulation scenario and the operation digital twin model to obtain simulated execution actions. 2.The reinforcement learning based intelligent supervision system for high-risk operations according to claim 1, wherein, The data analysis module includes: An environmental analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the work environment based on the work environment data within the target time period; The equipment analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the operating equipment based on the operating data of the operating equipment within the target time period. The behavior analysis unit, which is connected to the data acquisition module, is used to determine the characteristics of the work behavior based on the behavior data of the workers within the target time period; The target analysis unit is connected to the environment analysis unit, the equipment analysis unit, and the behavior analysis unit, respectively, and is used to determine abnormal operation characteristics based on the operation environment characteristics, the operation equipment characteristics, and the operation behavior characteristics, and to determine the operation target representation value in the target operation area based on the abnormal operation characteristics. 3.The reinforcement learning based intelligent supervision system for high-risk operations according to claim 2, wherein, The risk assessment module includes: An action analysis unit, which is connected to the data analysis module, is used to determine key execution actions based on the task target representation value and a preset reinforcement learning model; The action evaluation unit is connected to both the model simulation module and the action analysis unit, and is used to determine action evaluation features based on the comparison results between the simulated action and the key action. A risk analysis unit, connected to the action assessment unit, is used to determine the regulatory risk category based on the action assessment characteristics, and to determine the target action to be performed based on the regulatory risk category.

4. The intelligent monitoring system for high-risk operations based on reinforcement learning according to claim 3, characterized in that, The environmental analysis unit constructs an environmental change curve based on the work environment data within the target time period, and determines the environmental change abrupt change region based on the environmental change curve to determine the characteristics of the work environment.

5. The reinforcement learning based intelligent supervision system for high-risk operations of claim 4, wherein, The equipment analysis unit determines several related equipment parameters based on the operating data of the equipment within the target time period, and determines the characteristics of the equipment based on each related equipment parameter.

6. The reinforcement learning based intelligent supervision system for high-risk operations of claim 5, wherein, The behavior analysis unit determines the characteristics of the work behavior based on the comparison results of the worker behavior data within the target time period and the standard behavior data.

7. The intelligent monitoring system for high-risk operations based on reinforcement learning according to claim 6, characterized in that, The target analysis unit determines the target representation value of the operation within the target operation area based on the comparison results between the abnormal operation features and the standard operation features. 8.The reinforcement learning based high-risk operation intelligent supervision system according to claim 7, wherein, The risk analysis unit determines the regulatory risk category based on the action assessment characteristics and a preset assessment characteristic comparison table, wherein the regulatory risk category includes a high-impact risk category and a low-impact risk category. 9.The reinforcement learning based high-risk operation intelligent supervision system according to claim 8, wherein, The risk analysis unit determines the target action based on the regulatory risk category, including: If the regulatory risk category is a low-impact risk category, then the simulated execution action will be determined as the target execution action; If the regulatory risk category is a high-impact risk category, then the key execution action will be identified as the target execution action.

Images