Complex multi-mode resource intelligent scheduling method and system based on reinforcement learning

Abstract

The invention discloses a complex multi-mode resource intelligent scheduling method and system based on reinforcement learning, and relates to the technical field of resource scheduling, and the method comprises the steps: obtaining environment state monitoring data of multiple types of monitoring equipment in a current time step, and extracting data features of each piece of environment state monitoring data; calculating to obtain a state change instruction of each monitoring device by using each data feature and the operation state of each device in the multiple types of monitoring devices; adjusting the operation state of each monitoring device based on the state change instruction; obtaining an abnormity judgment result corresponding to the environment state monitoring data according to each data feature until the abnormity judgment result exceeds a threshold value and the environment real state is abnormal; the invention aims to solve the practical problems of high comprehensive overhead, tense resource allocation, complex scheduling strategy and the like caused by multiple types of equipment in the process of continuously monitoring the environment by multiple types of equipment in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and more specifically, to a method and system for intelligent scheduling of complex multimodal resources based on reinforcement learning. Background Art

[0002] With the continuous advancement of equipment technology and intelligence, environmental monitoring using equipment has entered a more complex stage. This complexity is reflected in both the diversity and number of external environments that need to be monitored, and the variety and number of environmental monitoring devices. For example, in the field of power inspections, the expansion of inspection scope has led to greater differences between different monitoring points. The number of monitoring points is also increasing, and the monitoring devices now include a wide range of equipment, from basic optical and thermal sensors to advanced drones and satellites.

[0003] Typically, each monitoring point is equipped with low-power, basic sensors to continuously monitor the external environment. If the sensor data indicates an anomaly, more accurate but also more expensive monitoring equipment, such as drones or autonomous vehicles, is activated to further confirm the potentially problematic monitoring point. However, as system complexity increases, this hierarchical monitoring strategy faces challenges: low reliability and judgment value of low-level information, high cost and difficulty in scheduling high-level equipment, and high overall system overhead. Furthermore, as environmental complexity increases, integrated monitoring systems struggle to ensure the timeliness and accuracy of environmental monitoring information over large areas with limited resources, often requiring frequent use of advanced equipment and manual patrols. While adding a wider range of monitoring equipment can alleviate these issues to some extent, the challenge remains how to integrate monitoring information from multiple sources and schedule various types of equipment to obtain more accurate environmental information over long periods of operation with minimal system overhead. Summary of the Invention

[0004] The purpose of the present invention is to provide a complex multimodal resource intelligent scheduling method and system based on reinforcement learning to solve practical problems such as high comprehensive overhead, tight resource allocation, and complex scheduling strategies caused by the large number of equipment types in the process of continuous environmental monitoring by multiple types of equipment in complex environments.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: In one aspect, the present application provides a method for intelligent scheduling of complex multimodal resources based on reinforcement learning, comprising the following specific steps: Obtain environmental status monitoring data of multiple types of monitoring equipment in the current time step, and extract data features of each environmental status monitoring data; Utilizing various data features and the operating status of each device in the multi-type monitoring devices, a state change instruction for each monitoring device is calculated; Adjust the operating status of each monitoring device based on the status change instruction; The abnormality judgment result corresponding to the environmental status monitoring data is obtained according to each data feature, until the abnormality judgment result exceeds the threshold and the actual state of the environment is abnormal.

[0006] On the basis of the above technical solution, the present invention can also be improved as follows.

[0007] Furthermore, the above scheduling method is implemented through a resource scheduling framework, which includes a multimodal data source, an environmental data encoder, an environmental state determiner, and a device scheduler, wherein: Environmental status monitoring data and the operating status of monitoring equipment are obtained through multimodal data sources; data features are obtained through environmental data encoders; abnormality judgment results are obtained through environmental status judges; and status change instructions are obtained and executed through equipment schedulers.

[0008] Furthermore, the device scheduler is trained in the following way: The system continues to run until the abnormal judgment result after executing the state change instruction does not exceed the preset judgment boundary standard, and the system ends the operation. The operating state reward of each monitoring device at all time steps is calculated based on the operating state of each monitoring device during the system operation and the preset reward function; Based on the loss function preset by reinforcement learning, the loss function value of the scheduler is calculated using the running state rewards of all the above time steps, and the parameters of the device scheduler are updated through backpropagation and gradient descent; The above process is repeated until the parameter update amount of the device scheduler meets the training end condition, and the scheduler that meets the training end condition is determined as the device scheduler.

[0009] Furthermore, the above reward function is specifically: ; Where, represents the running state reward in the current time step, represents the duration of the current time step, Indicates the operating status of the i-th monitoring device, To stop, To run, represents the operating cost of the i-th monitoring device, Indicates the preset maximum reward value, Indicates abnormal judgment results, and ; represents the real state of the environment, and ; Indicates the preset judgment boundary standard, It indicates the additional reward or penalty caused by the mismatch between abnormal judgment and the actual state of the environment.

[0010] In a second aspect, the present application provides a complex multimodal resource intelligent scheduling system based on reinforcement learning, which is applied to a complex multimodal resource intelligent scheduling method based on reinforcement learning in any one of the first aspects, including: The feature extraction module is used to obtain the environmental status monitoring data of multiple types of monitoring equipment in the current time step and extract the data features of each environmental status monitoring data; An instruction calculation module is used to calculate a state change instruction for each monitoring device by using various data features and the operating status of each device in the multi-type monitoring device; An instruction execution module, configured to adjust the operating state of each monitoring device based on the state change instruction; The abnormality judgment module is used to obtain the abnormality judgment result corresponding to the environmental status monitoring data according to various data features until the abnormality judgment result exceeds the threshold and the actual state of the environment is abnormal.

[0011] Furthermore, the scheduling in the above scheduling system is implemented through a resource scheduling framework, which includes a multimodal data source, an environmental data encoder, an environmental state determiner, and a device scheduler, wherein: Environmental status monitoring data and the operating status of monitoring equipment are obtained through multimodal data sources; data features are obtained through environmental data encoders; abnormality judgment results are obtained through environmental status judges; and status change instructions are obtained and executed through equipment schedulers.

[0012] Furthermore, the device scheduler is trained in the following way: Based on the operating status and abnormality judgment results of each monitoring device after executing the state change instruction, the operating status reward of the monitoring device in the current time step is calculated using a preset reward function; Based on the preset loss function, the loss function value of the initial scheduler is calculated using the running status reward. If the loss function value does not meet the training end condition, the scheduler parameters of the initial scheduler are adjusted. Based on the initial scheduler after the scheduler parameters are adjusted, the loss function value is calculated using the running status reward of the next time step until the loss function value meets the training end condition. The initial scheduler that meets the training end condition is determined as the device scheduler.

[0013] Furthermore, the above reward function is specifically: ; Where, represents the running state reward in the current time step, represents the duration of the current time step, Indicates the operating status of the i-th monitoring device, To stop, To run, represents the operating cost of the i-th monitoring device, Indicates the preset maximum reward value, Indicates abnormal judgment results, and ; represents the real state of the environment, and ; Indicates the preset judgment boundary standard, It indicates the additional reward or penalty caused by the mismatch between abnormal judgment and the actual state of the environment.

[0014] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods in the first aspect when executing the computer program.

[0015] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute any one of the methods in the first aspect.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: 1. The resource scheduling framework provided by the present invention can accurately judge the environmental status in real time by combining the environmental data obtained by the device through the use of a trained environmental status judge; by using a trained reinforcement learning-based device scheduler, it can continuously and intelligently control various types of devices in the data source by combining the environmental data obtained by the device and the operating status of the device in real time; thereby, while ensuring that the system provides real-time environmental status judgment, the device is scheduled and the overall operating overhead of the system is reduced.

[0017] 2. This invention addresses the high energy consumption, low efficiency, and scheduling difficulties inherent in complex, multi-device environmental monitoring systems. It proposes an intelligent system architecture framework that minimizes the overall overhead of long-term system operation while ensuring accurate environmental status assessment results. This framework boasts versatility, ease of construction, low energy consumption, and wide applicability, making it suitable for systems comprised of diverse environmental monitoring equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings: Figure 1 A flowchart of a scheduling method according to an embodiment of the present invention; Figure 2 A connection diagram of a scheduling system according to an embodiment of the present invention; Figure 3 Schematic diagram of a resource scheduling framework according to an embodiment of the present invention; Figure 4 Schematic diagram of the connection of electronic equipment in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0022] In the description of the embodiments of the present invention, "a plurality of" means at least two.

[0023] Example 1: In order to solve the problems of high energy consumption, low efficiency, and difficult scheduling in the current environmental monitoring system composed of complex and multi-type equipment, due to the explosive development of artificial intelligence technology, in many fields, through the application of artificial intelligence technologies such as deep learning and reinforcement learning, many problems that were previously difficult to tackle have been solved to a certain extent; the versatility and ability to handle complex situations brought by artificial intelligence technology can well assist in solving problems such as data processing and equipment scheduling in the application context of complex environmental monitoring of multi-type equipment, and on this basis further achieve the purpose of reducing system overhead and optimizing system management, thereby substantially promoting the development of environmental monitoring towards intelligence and automation.

[0024] This embodiment provides a complex multimodal resource intelligent scheduling method based on reinforcement learning; specifically, the scheduling method is implemented through a resource scheduling framework, such as Figure 3As shown in Figure 1, the resource scheduling framework includes a multimodal data source, an environmental data encoder, an environmental state judger, and a device scheduler, where: Environmental status monitoring data and the operating status of monitoring equipment are obtained through multimodal data sources; data features are obtained through environmental data encoders; abnormality judgment results are obtained through environmental status judges; and status change instructions are obtained and executed through equipment schedulers.

[0025] Furthermore, the resource scheduling framework is implemented through the following steps when scheduling: Figure 1 As shown, the following specific steps are included: S1, obtain the environmental status monitoring data of multiple types of monitoring equipment in the current time step, and extract the data features of each environmental status monitoring data.

[0026] S2, using various data features and the operating status of each device in the multi-type monitoring devices, calculate and obtain the state change instruction of each monitoring device.

[0027] S3, adjusting the operating status of each monitoring device based on the status change instruction.

[0028] S4, obtain the abnormality judgment result corresponding to the environmental status monitoring data according to each data feature, until the abnormality judgment result exceeds the threshold and the actual state of the environment is abnormal; wherein, when the abnormality judgment result exceeds the threshold and the actual state of the environment is abnormal, it indicates that the environment is abnormal and corresponding manpower is required to conduct abnormality investigation. In this investigation process, steps S1-S4 can be omitted. When the manpower investigation is completed and the abnormality is resolved, steps S1-S4 are re-executed.

[0029] Specifically, with the addition of inspection methods like drones and intelligent robots, current power inspections feature a rich variety of inspection equipment, but a limited number of high-cost devices. Considering the often large coverage areas required for power inspections, this places high demands on the system's ability to identify potential risk points and dispatch high-cost resources. Supported by basic, low-cost sensors, the system can collect large amounts of long-term, continuous data on factors such as temperature, humidity, and wind speed. Combined with historical inspection results, the system can assess the risk level of each location, screen out low-risk points, prioritize and focus inspections on high-risk points, and obtain more accurate data. This improves the efficiency of risk point detection within limited resources, achieving the dual goals of optimizing data acquisition costs (such as equipment overhead and time costs) and improving data acquisition efficiency (such as monitoring range and data accuracy).

[0030] The device scheduler is trained in the following way: S11, the system continues to run until the abnormality judgment result after executing the state change instruction does not exceed the preset judgment boundary standard, the system ends the operation, and calculates the operating state reward of the monitoring device at all time steps based on the operating state of each monitoring device during the system operation and the preset reward function; S12, based on the loss function preset by reinforcement learning, the loss function value of the scheduler is calculated using the running state rewards of all the above time steps, and the parameters of the device scheduler are updated through back propagation and gradient descent; S13, repeating S11-S12 until the parameter update amount of the device scheduler meets the training end condition, and determining the scheduler that meets the training end condition as the device scheduler.

[0031] Furthermore, the above reward function is specifically: ; Where, represents the running state reward in the current time step, represents the duration of the current time step, Indicates the operating status of the i-th monitoring device, To stop, To run, represents the operating cost of the i-th monitoring device, Indicates the preset maximum reward value, Indicates abnormal judgment result, and ; represents the real state of the environment, and ; Indicates the preset judgment boundary standard, It indicates the additional reward or penalty caused by the mismatch between abnormal judgment and the actual state of the environment.

[0032] The hardware environment required in the above resource scheduling framework is as follows: 1. A multimodal data source consisting of several devices, each of which can independently and continuously generate device data and aggregate it into a continuous data stream; 2. A data aggregation and computing unit (mobile or server platform), which can connect to the data source to obtain real-time data streams and has sufficient storage and processing capabilities (depending on the data rate of the data stream) to complete the computing work required by the environmental data encoder, environmental status determiner, and device scheduler.

[0033] Furthermore, different devices are connected to the data center, generating continuous time series data that can be described in the form of (time point, data point); the data center needs to continuously receive and store time series data from different devices and save them separately, taking into account the practical constraints of storage size.

[0034] Specifically, the environmental data encoder, environmental state determiner, and device scheduler within the resource scheduling framework, based on the present invention's network structure requirements for each component within the framework and after determining the input and output sizes based on actual conditions, can initialize system parameters and connect to the data link for training. The environmental data encoder can be constructed based on a multi-layer LSTM network; the environmental state determiner can be constructed using a combination of a multi-layer feedforward neural network and a Sigmoid function layer; and the device scheduler can be constructed using a corresponding neural network structure based on the selected reinforcement learning algorithm.

[0035] When training the above-mentioned environment data encoder, environment state judger and device scheduler, the training data is in the form of tuples (time Data generated by the data source, at any time The status of all devices in the data source, at all times The real state of the environment) is the time series data of the data points; the time series data starts at the time of system startup and ends at the time when both the system and the real state of the environment determine that the monitoring point has an abnormality. The time interval between data points should be the shortest interval between all device data collection. Therefore, you can continuously obtain real environment data through the established real data source, or you can continuously obtain simulated data by building a simulated environment. The training process can be: ①, Determine the time slice size w After that, the data source acquisition length is w The training data is sent to the environment data encoder, and the device status at the last moment is sent to the device scheduler. ②. The environmental data encoder calculates the environmental state characteristics based on the data transmitted in ① and transmits them to the environmental state judge and device scheduler.

[0036] ③. The environment state judge calculates the environment state judgment result based on the environment state characteristics transmitted in ②, and updates the parameters of the environment state judge and environment data encoder based on the actual environment state and loss function at the last moment.

[0037] ④、The device scheduler combines the environmental state features transmitted in ② and the data transmitted in ① into an input vector and calculates the action . At the same time, rewards are obtained according to the defined reward function Wait for the data source to complete the action. Thus, after the device scheduler obtains the input vector of the next cycle, it is combined with the previous data to form (input vector, action , reward , the input vector at the next moment). Parameters are updated according to the reinforcement learning algorithm and loss function.

[0038] Among them, when the environmental data encoder, environmental state judger, and device scheduler are trained and executed, they can be described as follows: ①, according to the time slice size w , the data source acquisition length is w The data source data is sent to the environment data encoder, and the device status at the last moment is sent to the device scheduler; ②. The environmental data encoder calculates the environmental state characteristics based on the data transmitted in ① and transmits them to the environmental state judge and device scheduler.

[0039] ③. The environmental state judge calculates the environmental state judgment result based on the environmental state characteristics transmitted in ②.

[0040] ④、The device scheduler combines the environmental state features transmitted in ② and the data transmitted in ① into an input vector and calculates the action , the data source adjusts the equipment operating status according to the instructions.

[0041] Example 2: This application provides a complex multimodal resource intelligent scheduling system based on reinforcement learning, which is applied to a complex multimodal resource intelligent scheduling method based on reinforcement learning in Example 1, such as Figure 2 As shown, including: The feature extraction module is used to obtain the environmental status monitoring data of multiple types of monitoring equipment in the current time step and extract the data features of each environmental status monitoring data; An instruction calculation module is used to calculate a state change instruction for each monitoring device by using various data features and the operating status of each device in the multi-type monitoring device; An instruction execution module, configured to adjust the operating state of each monitoring device based on the state change instruction; The abnormality judgment module is used to obtain the abnormality judgment result corresponding to the environmental status monitoring data according to various data features until the abnormality judgment result exceeds the threshold and the actual state of the environment is abnormal.

[0042] Furthermore, the scheduling in the above scheduling system is implemented through a resource scheduling framework, which includes a multimodal data source, an environmental data encoder, an environmental state determiner, and a device scheduler, wherein: Environmental status monitoring data and the operating status of monitoring equipment are obtained through multimodal data sources; data features are obtained through environmental data encoders; abnormality judgment results are obtained through environmental status judges; and status change instructions are obtained and executed through equipment schedulers.

[0043] Furthermore, the device scheduler is trained in the following way: Based on the operating status and abnormality judgment results of each monitoring device after executing the state change instruction, the operating status reward of the monitoring device in the current time step is calculated using a preset reward function; Based on the preset loss function, the loss function value of the initial scheduler is calculated using the running status reward. If the loss function value does not meet the training end condition, the scheduler parameters of the initial scheduler are adjusted. Based on the initial scheduler after the scheduler parameters are adjusted, the loss function value is calculated using the running status reward of the next time step until the loss function value meets the training end condition. The initial scheduler that meets the training end condition is determined as the device scheduler.

[0044] Furthermore, the above reward function is specifically: ; Where, represents the running state reward in the current time step, represents the duration of the current time step, Indicates the operating status of the i-th monitoring device, To stop, To run, represents the operating cost of the i-th monitoring device, Indicates the preset maximum reward value, Indicates abnormal judgment results, and ; represents the real state of the environment, and ; Indicates the preset judgment boundary standard, It indicates the additional reward or penalty caused by the mismatch between abnormal judgment and the actual state of the environment.

[0045] Example 3: This application provides an electronic device, such as Figure 4 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method in embodiment 1 is implemented.

[0046] Example 4: The present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute the method in Example 1.

[0047] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A complex multimodal resource intelligent scheduling method based on reinforcement learning, characterized in that: The specific steps include: Acquire environmental status monitoring data of multiple types of monitoring equipment in the current time step, and extract data features of each of the environmental status monitoring data; Utilizing each of the data features and the operating status of each of the multiple types of monitoring devices, a state change instruction for each of the monitoring devices is calculated; adjusting the operating status of each monitoring device based on the status change instruction; An abnormality judgment result corresponding to the environmental status monitoring data is obtained according to each of the data features, until the abnormality judgment result exceeds a threshold and the actual state of the environment is abnormal.

2. The method for intelligent scheduling of complex multimodal resources based on reinforcement learning according to claim 1, characterized in that: The scheduling method is implemented through a resource scheduling framework, which includes a multimodal data source, an environmental data encoder, an environmental state determiner, and a device scheduler, wherein: The environmental status monitoring data and the operating status of the monitoring equipment are obtained through a multimodal data source; the data features are obtained through an environmental data encoder; the abnormality judgment result is obtained through an environmental status judge; and the state change instruction is obtained and executed through a device scheduler.

3. The method for intelligent scheduling of complex multimodal resources based on reinforcement learning according to claim 2, characterized in that: The device scheduler is trained in the following way: The system continues to run until the abnormal judgment result after executing the state change instruction does not exceed the preset judgment boundary standard. The system ends the operation and calculates the operating state reward of the monitoring device at all time steps based on the operating state of each monitoring device during the system operation and the preset reward function; Based on the loss function preset by reinforcement learning, the scheduler's loss function value is calculated using the running state rewards of all the above time steps, and the parameters of the device scheduler are updated through backpropagation and gradient descent; The above process is repeated until the parameter update amount of the device scheduler meets the training end condition, and the scheduler that meets the training end condition is determined as the device scheduler.

4. The method for intelligent scheduling of complex multimodal resources based on reinforcement learning according to claim 3, characterized in that: The reward function is specifically: ； Where, represents the running state reward in the current time step, represents the duration of the current time step, Indicates the operating status of the i-th monitoring device, To stop, To run, represents the operating cost of the i-th monitoring device, Indicates the preset maximum reward value, Indicates abnormal judgment results, and ; represents the real state of the environment, and ; Indicates the preset judgment boundary standard, It indicates the additional reward or penalty caused by the mismatch between abnormal judgment and the actual state of the environment.

5. A complex multimodal resource intelligent scheduling system based on reinforcement learning, applied to a complex multimodal resource intelligent scheduling method based on reinforcement learning according to any one of claims 1 to 4, characterized in that: include: A feature extraction module is used to obtain environmental status monitoring data of multiple types of monitoring equipment in the current time step and extract data features of each of the environmental status monitoring data; An instruction calculation module, configured to calculate a state change instruction for each of the monitoring devices by utilizing each of the data features and the operating state of each of the multiple types of monitoring devices; An instruction execution module, configured to adjust the operating state of each monitoring device based on the state change instruction; The abnormality judgment module is used to obtain the abnormality judgment result corresponding to the environmental status monitoring data according to each of the data features, until the abnormality judgment result exceeds a threshold and the actual state of the environment is abnormal.

6. The complex multimodal resource intelligent scheduling system based on reinforcement learning according to claim 5 is characterized in that: The scheduling in the scheduling system is implemented through a resource scheduling framework, which includes a multimodal data source, an environmental data encoder, an environmental state determiner, and a device scheduler, wherein: The environmental status monitoring data and the operating status of the monitoring equipment are obtained through a multimodal data source; the data features are obtained through an environmental data encoder; the abnormality judgment result is obtained through an environmental status judge; and the state change instruction is obtained and executed through a device scheduler.

7. The complex multimodal resource intelligent scheduling system based on reinforcement learning according to claim 6 is characterized in that: The device scheduler is trained in the following way: Based on the operating state of each monitoring device after executing the state change instruction and the abnormality judgment result, a preset reward function is used to calculate the operating state reward of the monitoring device in the current time step; Based on a preset loss function, the loss function value of the initial scheduler is calculated using the running status reward. If the loss function value does not meet the training end condition, the scheduler parameters of the initial scheduler are adjusted, and based on the initial scheduler after the scheduler parameters are adjusted, the loss function value is calculated using the running status reward of the next time step until the loss function value meets the training end condition. The initial scheduler that meets the training end condition is determined as the device scheduler.

8. The complex multimodal resource intelligent scheduling system based on reinforcement learning according to claim 7 is characterized in that: The reward function is specifically: ； Where, represents the running state reward in the current time step, represents the duration of the current time step, Indicates the operating status of the i-th monitoring device, To stop, To run, represents the operating cost of the i-th monitoring device, Indicates the preset maximum reward value, Indicates abnormal judgment results, and ; represents the real state of the environment, and ; Indicates the preset judgment boundary standard, It indicates the additional reward or penalty caused by the mismatch between abnormal judgment and the actual state of the environment.

Images