Production behavior identification method for non-electricity-consuming enterprises based on DQN algorithm

Abstract

This invention discloses a method for identifying the production behavior of non-electricity-consuming enterprises based on the DQN algorithm. The method comprises the following steps: A. establishing an environmental model for identifying the production behavior of non-electricity-consuming enterprises using the gym library of Open AI; B. constructing a classification prediction model for exploring action plans for identifying enterprise production behavior, inputting a training set into the classification prediction model, and completing model training; C. optimizing the classification prediction model using a validation set to obtain an optimized classification prediction model; D. validating the performance of the classification prediction model using a test set; and E. saving the trained classification prediction model and deploying it to a terminal to identify the production behavior of the enterprise in real time, providing a reference for environmental protection process managers to identify the enterprise's production status. This invention can improve the shortcomings of existing technologies and can more accurately identify the production behavior of non-electricity-consuming enterprises.

Description

Technical Field

[0001] The present invention relates to the technical field of identifying enterprise production behavior, and in particular to a method for identifying production behavior of non-electricity-consuming enterprises based on a DQN algorithm. Background Art

[0002] In recent years, environmental issues have garnered increasing attention. To protect the environment and reduce pollutant emissions, environmental regulators have implemented production suspensions and restrictions on businesses. Efficiently ensuring that businesses adhere to production schedules is a major challenge for supervisory and law enforcement personnel. Data mining technology is applied and plays a significant role in various fields. With the rapid development of society, human demands for the environment are increasing, and environmental pollution is impacting human life. Therefore, the application of data mining technology in the field of environmental protection and improving environmental quality has become a hot topic of public concern.

[0003] In the field of environmental protection, the application of data mining technology to identify the production behavior of polluting enterprises can provide an important reference for environmental protection process management. While a relatively comprehensive solution has been developed for identifying production behavior based on electricity consumption data, identifying production behavior based on non-electricity signals such as temperature, vibration, and radar is still in its infancy. There are two types of production behavior for non-electricity-using enterprises: production and non-production. Identifying the production behavior of non-electricity-using enterprises is a binary data classification problem. The data samples collected by sensors, consisting of signals such as vibration frequency and amplitude, exhibit class imbalance. When training on unbalanced datasets, traditional machine learning models relatively easily learn features from frequently occurring categories, but they inadequately mine features from less frequently occurring categories. This indicates that machine learning classification algorithms are ineffective when data distribution is imbalanced. The key to identifying the production behavior of non-electricity-using enterprises is how to reduce the occurrence of misjudgments. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for identifying the production behavior of non-electricity-consuming enterprises based on the DQN algorithm, which can solve the shortcomings of the existing technology and can more accurately identify the production behavior of non-electricity-consuming enterprises.

[0005] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows.

[0006] A method for identifying production behavior of non-electricity-consuming enterprises based on a DQN algorithm includes the following steps:

[0007] A. Use Open AI's gym library to build an environmental model for identifying production behaviors of non-electricity-consuming enterprises;

[0008] B. Build a classification prediction model for enterprise production behavior identification and action plan exploration, input the training set into the classification prediction model, and complete the model training;

[0009] C. Use the validation set to optimize the classification prediction model to obtain the optimized classification prediction model;

[0010] D. Use the test set to verify the performance of the classification prediction model;

[0011] E. Save the trained classification prediction model and deploy it to the terminal to identify the company's production behavior in real time, providing a reference for environmental protection process managers to identify the company's production status.

[0012] Preferably, step A includes the following steps:

[0013] A1. Use a relational database table to store sensor device information, including 3D vibration frequency, 3D vibration amplitude, pipe wall temperature, and CPU temperature, and then build a static information model of the sensor device.

[0014] A2. Use Python to design a class that identifies the production status of non-electricity-consuming enterprises and builds an environmental model. This class includes states and behaviors, with states representing discrete conditions in the environment. Implement the step function in the class to change the environmental state based on the agent's identified behavior and calculate the reward value for that behavior, thus building a single-step interaction model between identified behavior and environmental changes.

[0015] A3. Construct an environment reward method for identification behavior, and then train the optimal disposal strategy. For the situation where the data samples of enterprises in production status are less than the data samples of enterprises in non-production status, and the data samples are unbalanced, reformulate the design rules of the reward and punishment function in the interaction between the intelligent agent and the environment, increase the intensity of rewards and punishments, and improve the classification and prediction performance of minority classes in unbalanced data.

[0016] Preferably, step B includes the following steps:

[0017] B1. Initialize the action value function;

[0018] B2. Construct object sets, action sets, and action sets;

[0019] B3. Use the linear decay greedy strategy to select actions, that is, the action is selected according to the action value function with a high probability, and the action is selected randomly with a low probability.

[0020] B4. In the current state, after selecting action a, perform action a in the environment, and the environment returns the corresponding reward and the next state;

[0021] B5. Store the experience sequence obtained through exploration into the experience pool. The experience pool consists of: state, behavior, reward, next state, and completed events.

[0022] B6. Use a greedy strategy to select actions to obtain the optimal strategy in the environment;

[0023] B7. Extract experience sequence samples in batches from the experience pool to train the deep Q network.

[0024] Preferably, step B7 includes the following steps:

[0025] B71. Randomly initialize two networks with exactly the same structure but different parameters as the training network and the target network respectively;

[0026] B72. Each time, we extract several experience sequences from the experience pool. The experience sequence is a four-tuple consisting of state, behavior, reward, and next state. We input the current state into the training network to obtain the predicted value of the action selected in that state. We input the next state into the target network to obtain the actual value of the action selected in that state.

[0027] B73. Use the gradient descent method to reduce the mean square error between the predicted value obtained by the training network and the actual value obtained by the target network. DQN continuously approaches the convergence value and the model training is completed.

[0028] Preferably, step C comprises the following steps:

[0029] C1. Input the validation set into the model. The performance of the model on the validation data is used as a feedback signal. The G-means value is used as the performance indicator of the model. The G-means value is proportional to the recognition accuracy. The formula is as follows:

[0030] ,

[0031] TP is a positive sample determined by the model to be positive, that is, the enterprise status in the sample data is production and the model determines it to be production; FN is a positive sample determined by the model to be negative, that is, the enterprise status in the data sample is production but the model determines it to be non-production; TN is a negative sample determined by the model to be negative, that is, the enterprise status in the data sample is non-production and the model determines it to be non-production; FP is a negative sample determined by the model to be positive, that is, the enterprise status in the data sample is non-production but the model determines it to be production;

[0032] C2. Adjust the model configuration based on the performance indicators of the model on the validation set;

[0033] C3. When the highest performance index value is obtained, the optimized model is obtained.

[0034] Preferably, step D comprises the following steps:

[0035] D1. Input the test set into the model and obtain the G-means value output by the model;

[0036] D2. Evaluate the recognition accuracy of the recognition model based on the obtained G-means value.

[0037] Preferably, step E comprises the following steps:

[0038] E1. Save the trained classification model;

[0039] E2. Install the inference engines OpenPPL and OpenCV;

[0040] E3. OpenCV reads the collected real-time monitoring data;

[0041] E4. Create and register the engines you want to use. Each engine corresponds to an inference backend, and read the model saved in step E1.

[0042] E5. Perform model reasoning;

[0043] E6. Obtain the identification result of the model.

[0044] The beneficial effects brought about by adopting the above technical solution are: the present invention completes decision optimization through reinforcement learning, and adopts the DQN algorithm to construct an intelligent agent learning model for exploring the identification scheme of the production behavior of non-electricity-consuming enterprises. The present invention regards the identification of the production behavior of non-electricity-consuming enterprises as a processing process of an intelligent agent. By modeling the data information of the sensor equipment, the data characteristics of the sensor equipment are further comprehensively mined, and the production behavior of non-electricity-consuming enterprises is identified more accurately. This method has important practical significance and theoretical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of an embodiment of the present invention;

[0046] Figure 2 This is the production behavior identification process of non-electricity-consuming enterprises based on the DQN algorithm in the embodiment of the present invention;

[0047] Figure 3 This is the process of constructing an environmental model for identifying the production behavior of non-electricity-consuming enterprises in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Reference Figure 1-3 , a specific embodiment of the present invention includes the following steps,

[0049] A. Use Open AI's gym library to build an environmental model for identifying production behaviors of non-electricity-consuming enterprises;

[0050] B. Build a classification prediction model for enterprise production behavior identification and action plan exploration, input the training set into the classification prediction model, and complete the model training;

[0051] C. Use the validation set to optimize the classification prediction model to obtain the optimized classification prediction model;

[0052] D. Use the test set to verify the performance of the classification prediction model;

[0053] E. Save the trained classification prediction model and deploy it to the terminal to identify the company's production behavior in real time, providing a reference for environmental protection process managers to identify the company's production status.

[0054] Step A includes the following steps:

[0055] A1. Use a relational database table to store sensor device information, including 3D vibration frequency, 3D vibration amplitude, pipe wall temperature, and CPU temperature, and then build a static information model of the sensor device.

[0056] A2. Use Python to design a class that identifies the production status of non-electricity-consuming enterprises and builds an environmental model. This class includes states and behaviors, with states representing discrete conditions in the environment. Implement the step function in the class to change the environmental state based on the agent's identified behavior and calculate the reward value for that behavior, thus building a single-step interaction model between identified behavior and environmental changes.

[0057] A3. Construct an environment reward method for identification behavior, and then train the optimal disposal strategy. For the situation where the data samples of enterprises in production status are less than the data samples of enterprises in non-production status, and the data samples are unbalanced, reformulate the design rules of the reward and punishment function in the interaction between the intelligent agent and the environment, increase the intensity of rewards and punishments, and improve the classification and prediction performance of minority classes in unbalanced data.

[0058] Step B includes the following steps,

[0059] B1. Initialize the action value function;

[0060] B2. Construct object sets, action sets, and action sets;

[0061] B3. Use the linear decay greedy strategy to select actions, that is, the action is selected according to the action value function with a high probability, and the action is selected randomly with a low probability.

[0062] B4. In the current state, after selecting action a, perform action a in the environment, and the environment returns the corresponding reward and the next state;

[0063] B5. Store the experience sequence obtained through exploration into the experience pool. The experience pool consists of: state, behavior, reward, next state, and completed events.

[0064] B6. Use a greedy strategy to select actions to obtain the optimal strategy in the environment;

[0065] B7. Extract experience sequence samples in batches from the experience pool to train the deep Q network.

[0066] Step B7 includes the following steps:

[0067] B71. Randomly initialize two networks with exactly the same structure but different parameters as the training network and the target network respectively;

[0068] B72. Each time, we extract several experience sequences from the experience pool. The experience sequence is a four-tuple consisting of state, behavior, reward, and next state. We input the current state into the training network to obtain the predicted value of the action selected in that state. We input the next state into the target network to obtain the actual value of the action selected in that state.

[0069] B73. Use the gradient descent method to reduce the mean square error between the predicted value obtained by the training network and the actual value obtained by the target network. DQN continuously approaches the convergence value and the model training is completed.

[0070] Step C includes the following steps:

[0071] C1. Input the validation set into the model. The performance of the model on the validation data is used as a feedback signal. The G-means value is used as the performance indicator of the model. The G-means value is proportional to the recognition accuracy. The formula is as follows:

[0072] ,

[0073] TP is a positive sample determined by the model to be positive, that is, the enterprise status in the sample data is production and the model determines it to be production; FN is a positive sample determined by the model to be negative, that is, the enterprise status in the data sample is production but the model determines it to be non-production; TN is a negative sample determined by the model to be negative, that is, the enterprise status in the data sample is non-production and the model determines it to be non-production; FP is a negative sample determined by the model to be positive, that is, the enterprise status in the data sample is non-production but the model determines it to be production;

[0074] C2. Adjust the model configuration based on the performance indicators of the model on the validation set;

[0075] C3. When the highest performance index value is obtained, the optimized model is obtained.

[0076] Step D includes the following steps:

[0077] D1. Input the test set into the model and obtain the G-means value output by the model;

[0078] D2. Evaluate the recognition accuracy of the recognition model based on the obtained G-means value.

[0079] Step E includes the following steps:

[0080] E1. Save the trained classification model;

[0081] E2. Install the inference engines OpenPPL and OpenCV;

[0082] E3. OpenCV reads the collected real-time monitoring data;

[0083] E4. Create and register the engines you want to use. Each engine corresponds to an inference backend, and read the model saved in step E1.

[0084] E5. Perform model reasoning;

[0085] E6. Obtain the identification result of the model.

[0086] This invention provides a method for identifying the production behavior of non-electricity-consuming enterprises based on the DQN algorithm. The basic idea is to treat the identification of non-electricity-consuming enterprises' production behavior as a sequential decision problem. An intelligent agent uses a linearly decaying greedy strategy to select actions to explore and exploit the environment. A deep Q-network is used to obtain the optimal strategy. Then, actions are selected based on the optimal strategy. The generated experience sequence is stored in an experience pool. The Q-network is trained using batches of data from the experience pool. Finally, the trained deep reinforcement learning model is saved and deployed to a terminal, which then identifies the production status of polluting enterprises in real time. This method can model the production status of enterprises and the intelligent agent that explores optimization solutions. After deploying the trained model to the terminal, the production behavior of non-electricity-consuming enterprises can be identified based on real-time monitoring data. This identification method, based on the DQN algorithm, continuously interacts with the intelligent agent. The environment rewards the agent for different classifications, reducing the false positive rate and saving manpower and resources for supervisory departments. This invention can provide a reference for environmental management personnel in identifying enterprise production behavior and is of great significance for ensuring that enterprises implement production suspension and restriction measures within specified timeframes, thereby improving environmental quality.

[0087] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0088] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying production behavior of non-power-consuming enterprises based on DQN algorithm, characterized by The following steps are involved: A. Use Open AI's gym library to build an environmental model for identifying production behaviors of non-electricity-consuming enterprises; A1. Use a relational database table to store sensor device information, including 3D vibration frequency, 3D vibration amplitude, pipe wall temperature, and CPU temperature, and then build a static information model of the sensor device. A2. Use Python to design a class that identifies the production status of non-electricity-consuming enterprises and builds an environmental model. This class includes states and behaviors, with states representing discrete conditions in the environment. Implement the step function in the class to change the environmental state based on the agent's identified behavior and calculate the reward value for that behavior, thus building a single-step interaction model between identified behavior and environmental changes. A3. Build an environment reward system for identifying behaviors, thereby training the optimal response strategy. For cases where data samples from companies in production are fewer than those from companies in non-production states, resulting in data imbalance, re-establish the design rules for the reward and penalty functions during the interaction between the agent and the environment, increase the intensity of rewards and penalties, and improve the classification and prediction performance of minority classes in imbalanced data. B. Build a classification prediction model for enterprise production behavior identification and action plan exploration, input the training set into the classification prediction model, and complete the model training; C. Use the validation set to optimize the classification prediction model to obtain the optimized classification prediction model; D. Use the test set to verify the performance of the classification prediction model; E. Save the trained classification prediction model and deploy it to the terminal to identify the company's production behavior in real time, providing a reference for environmental protection process managers to identify the company's production status.

2. The method for identifying production behavior of non-electricity-consuming enterprises based on the DQN algorithm according to claim 1 is characterized by: In step B The following steps are included: B1. Initialize the action value function; B2. Construct object sets, action sets, and action sets; B3. Use the linear decay greedy strategy to select actions, that is, the action is selected according to the action value function with a high probability, and the action is selected randomly with a low probability. B4. In the current state, after selecting action a, perform action a in the environment, and the environment returns the corresponding reward and the next state; B5. Store the experience sequence obtained through exploration into the experience pool. The experience pool consists of: state, behavior, reward, next state, and completed events. B6. Use a greedy strategy to select actions to obtain the optimal strategy in the environment; B7. Extract experience sequence samples in batches from the experience pool to train the deep Q network.

3. The method for identifying production behavior of non-electricity-consuming enterprises based on the DQN algorithm according to claim 2 is characterized by: In step B7, The following steps are included: B71. Randomly initialize two networks with exactly the same structure but different parameters as the training network and the target network respectively; B72. Each time, we extract several experience sequences from the experience pool. The experience sequence is a four-tuple consisting of state, behavior, reward, and next state. We input the current state into the training network to obtain the predicted value of the action selected in that state. We input the next state into the target network to obtain the actual value of the action selected in that state. B73. Use the gradient descent method to reduce the mean square error between the predicted value obtained by the training network and the actual value obtained by the target network. DQN continuously approaches the convergence value and the model training is completed.

4. The method for identifying production behavior of non-electricity-consuming enterprises based on the DQN algorithm according to claim 3 is characterized by: In step C, The following steps are included: C1. Input the validation set into the model. The performance of the model on the validation data is used as a feedback signal. The G-means value is used as the performance indicator of the model. The G-means value is proportional to the recognition accuracy. The formula is as follows: TP is a positive sample determined by the model to be positive, that is, the enterprise status in the sample data is production and the model determines it to be production; FN is a positive sample determined by the model to be negative, that is, the enterprise status in the data sample is production but the model determines it to be non-production; TN is a negative sample determined by the model to be negative, that is, the enterprise status in the data sample is non-production and the model determines it to be non-production; FP is a negative sample determined by the model to be positive, that is, the enterprise status in the data sample is non-production but the model determines it to be production; C2. Adjust the model configuration based on the performance indicators of the model on the validation set; C3. When the highest performance index value is obtained, the optimized model is obtained.

5. The method for identifying production behavior of non-electricity-consuming enterprises based on the DQN algorithm according to claim 4 is characterized by: Step D includes the following steps: D1. Input the test set into the model and obtain the G-means value output by the model; D2. Evaluate the recognition accuracy of the recognition model based on the obtained G-means value.

6. The method for identifying production behavior of non-electricity-consuming enterprises based on the DQN algorithm according to claim 5 is characterized by: In step E, The following steps are included: E1. Save the trained classification model; E2. Install the inference engines OpenPPL and OpenCV; E3. OpenCV reads the collected real-time monitoring data; E4. Create and register the engines you want to use. Each engine corresponds to an inference backend, and read the model saved in step E1. E5. Perform model reasoning; E6. Obtain the identification result of the model.

Images