Rod pumped well energy consumption optimization control method, device and system based on edge calculation

Abstract

The invention discloses a rod-pumped well energy consumption optimization control method, device and system based on edge calculation. According to the method, the system environment state of a rod-pumped well system is obtained in real time, a current control strategy is obtained through a DDPG-based rod-pumped well energy consumption optimization control model according to the current system environment state and based on a random strategy, and the current control strategy is sent to the rod-pumped well system; after the rod-pumped well system executes the current control strategy, the system environment state and the instant return at the next moment are obtained, and the system environment state and the instant return are integrated with the current system environment state and the current control strategy into specific state transfer data; repeating the process of obtaining the state transition data until a set quantity of state transition data is obtained; then multiple pieces of state transition data are randomly extracted, and parameters of the rod pumped well energy consumption optimization control model are updated; and the process of obtaining the state transition data and updating the parameters is repeated until the preset optimization condition is met, and the optimal energy consumption optimization control strategy of the rod-pumped well is obtained.

Description

Technical Field

[0001] The present application relates to an energy consumption optimization control method, device and system for pumping unit wells based on edge computing. Background Art

[0002] Pumping unit wells are important equipment in oil extraction. The production sites of most pumping unit wells are in remote areas or environments with few people. In this environment, the basic communication network is not as stable as in urban areas, resulting in the inability to achieve real-time data collection, real-time transmission and feedback of on-site data. If all data analysis and control programs are directly centralized on the cloud platform, it will be difficult to meet the requirements of immediacy. In addition, the energy consumption problem of pumping unit wells has always been a concern. The traditional control system of pumping unit wells cannot process complex data in real time and make optimal decisions, so it is impossible to optimize energy consumption in real time. Summary of the Invention

[0003] In order to better achieve the optimization control of the energy consumption of pumping unit wells, the embodiments of the present application provide an energy consumption optimization control method, device and system for pumping unit wells based on edge computing.

[0004] In a first aspect, the embodiments of the present application provide an energy consumption optimization control method for pumping unit wells based on edge computing, which is applied to the cloud-edge collaboration layer. The method includes:

[0005] Obtain the system environment state of the pumping unit well system in real time;

[0006] Through the energy consumption optimization control model of the pumping unit well based on DDPG, according to the current system environment state, based on a random policy, obtain the current control policy and send it to the pumping unit well system;

[0007] Obtain the system environment state and immediate reward at the next moment after the pumping unit well system executes the current control policy;

[0008] Integrate the current system environment state, current control policy, immediate reward and system environment state at the next moment into a state transition data;

[0009] Repeat the process of obtaining the state transition data until a set number of state transition data are obtained;

[0010] Randomly extract multiple state transition data from the set number of state transition data, and update the parameters of the energy consumption optimization control model of the pumping unit well;

[0011] Repeat the process of obtaining the state transition data and updating the parameters until the preset optimization condition is met, and obtain the optimal energy consumption optimization control strategy for the pumping unit well.

[0012] In an alternative implementation of the embodiment of the present application, the energy consumption optimization control model of the pumping well includes an actor network and a critic network; randomly extracting a plurality of state transition data from the set number of state transition data to update the parameters of the energy consumption optimization control model of the pumping well, including:

[0013] Randomly extract a plurality of state transition data from the set number of state transition data, and according to the system environment state at the next moment in each state transition data, obtain the corresponding control strategy at the next moment through the actor network;

[0014] Through the critic network, according to the current system environment state and the current control strategy and the system environment state and control strategy at the next moment, obtain the corresponding liquid production of the pumping well in the current period and the liquid production of the pumping well in the next period;

[0015] Calculate the TD error according to the liquid production of the pumping well in the current period and the liquid production of the pumping well in the next period;

[0016] Update the weight parameters of the critic network according to the TD error;

[0017] Update the weight parameters of the actor network according to the liquid production of the pumping well in the current period.

[0018] In an alternative implementation of the embodiment of the present application, the updating the weight parameters of the critic network according to the TD error includes:

[0019] Update the weight parameters of the critic network through the following formula 1 according to the TD error:

[0020] ω t+1 = ω t + α ω [r t + γQ(s t+1 , a t+1 ) - Q(s t , a t )] Formula 1;

[0021] In the formula, ω t+1 is the weight parameter of the critic network at the (t + 1)-th moment; ω t is the weight parameter of the critic network at the t-th moment; α ω is the learning rate of the critic network; r t + γQ(s t+1 , a t+1 ) - Q(s t , a t ) is the TD error.

[0022] In an alternative implementation manner of the embodiment of the present application, updating the weight parameters of the actor network according to the liquid production volume of the pumping unit well in the current period includes:

[0023] Updating the weight parameters of the actor network according to the liquid production volume of the pumping unit well in the current period through the following formula 2:

[0024]

[0025] In the formula, θ t+1 is the weight parameter of the actor network at the (t + 1)-th moment; θ t is the weight parameter of the actor network at the t-th moment; α θ is the learning rate of the actor network; is the deterministic policy gradient.

[0026] In an alternative implementation manner of the embodiment of the present application, obtaining the immediate reward through the following formula 3:

[0027]

[0028] In the formula, R is the immediate reward; -Energy(s t , a t ) is the energy consumption of the pumping unit system in the t-th time period; Q is the liquid production volume of the pumping unit well in the period; γ is the liquid production volume penalty coefficient; is the upper limit of the liquid production volume of the pumping unit well in the period; Q is the lower limit of the liquid production volume of the pumping unit well in the period.

[0029] In an alternative implementation manner of the embodiment of the present application, the real-time acquisition of the system environment state of the pumping unit well system includes:

[0030] Based on a preset electrical parameter module, the motor operating current and voltage frequency of the pumping unit well system are acquired in real time;

[0031] Based on a motor speed sensor, the motor rotation angle of the pumping unit well system is acquired in real time;

[0032] Based on a temperature sensor and a pressure sensor, the changes in temperature and pressure of the pumping unit well system are acquired in real time;

[0033] Based on a dynamometer card sensor, the dynamometer card parameters of the pumping unit well system are acquired in real time.

[0034] In a second aspect, the embodiment of the present application provides an energy consumption optimization control device for a pumping unit well based on edge computing, which is applied to the cloud-edge collaboration layer. The device includes:

[0035] A real-time acquisition module, configured to acquire the system environment state of the pumping unit well system in real time;

[0036] A random policy module, configured to obtain a current control policy based on a random policy according to the current system environment state through an energy consumption optimization control model of the pumping unit well based on DDPG, and send the current control policy to the pumping unit well system;

[0037] A first determination module, configured to obtain the system environment state and the immediate reward at the next moment after the pumping unit well system executes the current control policy;

[0038] An integration module, configured to integrate the current system environment state, the current control policy, the immediate reward, and the system environment state at the next moment into a state transition data;

[0039] A repeated acquisition module, configured to repeat the process of obtaining the state transition data until a set number of state transition data is obtained;

[0040] An update module, configured to randomly extract multiple state transition data from the set number of state transition data, and update the parameters of the energy consumption optimization control model of the pumping unit well;

[0041] An iteration module, configured to repeat the process of obtaining the state transition data and updating the parameters until a preset optimization condition is met, so as to obtain an optimal energy consumption optimization control policy for the pumping unit well.

[0042] In a third aspect, an embodiment of the present application provides an energy consumption optimization control system for a pumping unit well based on edge computing. The system includes an intelligent perception layer, a communication layer, a cloud-edge collaboration layer, and an application layer:

[0043] The intelligent perception layer is configured to collect the system environment state of the pumping unit well system;

[0044] The communication layer is communicatively connected to the intelligent perception layer and the cloud-edge collaboration layer;

[0045] The cloud-edge collaboration layer is configured to obtain an energy consumption optimization control policy for the pumping unit well based on the energy consumption optimization control method for the pumping unit well based on edge computing as described in the first aspect;

[0046] The application layer is communicatively connected to the cloud-edge collaboration layer, and is configured to display the energy consumption optimization control situation of the pumping well.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the energy consumption optimization control method for the pumping unit well based on edge computing as described above is implemented.

[0048] In a fifth aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned energy consumption optimization control method for pumping unit wells based on edge computing.

[0049] In a sixth aspect, an embodiment of the present application provides a computer program product containing instructions. When the computer program product runs on a computer device, it causes the computer device to execute the above-mentioned energy consumption optimization control method for pumping unit wells based on edge computing.

[0050] In a seventh aspect, an embodiment of the present application provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a computer program or instructions to implement the above-mentioned energy consumption optimization control method for pumping unit wells based on edge computing.

[0051] The beneficial effects of the above technical solutions provided by the embodiments of the present application at least include:

[0052] The energy consumption optimization control method for pumping unit wells based on edge computing provided by the embodiments of the present application obtains the system environment state of the pumping unit well system in real time, and randomly gives a control strategy through an energy consumption optimization control model for pumping unit wells based on DDPG, obtaining the system environment state and immediate reward at the next moment after the pumping unit well system executes the control strategy. Repeating the above process, the parameters of the energy consumption optimization control model for pumping unit wells are continuously updated through the method of reinforcement learning, and finally the optimal energy consumption optimization control strategy for pumping unit wells is obtained, realizing the energy consumption optimization of pumping unit wells. This method has strong computing power and real-time performance, can better adapt to complex data environments and real-time control requirements, reduces the energy consumption of pumping unit wells, improves efficiency, reduces the need for manual intervention, and moreover, can effectively reduce the energy consumption of pumping unit wells without the need to transform existing pumping units, having high practical value and promotion prospects.

[0053] Other features and advantages of the present application will be described in the following specification, and part of them will become obvious from the specification or be understood by implementing the present application. The objectives and other advantages of the present application can be achieved and obtained through the structures specifically pointed out in the written specification, claims, and drawings.

[0054] The technical solutions of the present application will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0055] The drawings are used to provide a further understanding of the present application, and constitute a part of the specification. They are used together with the embodiments of the present application to explain the present application, and do not constitute a limitation to the present application; in the drawings:

[0056] Figure 1 Schematic diagram of the steps of the energy consumption optimization control method for pumping wells based on edge computing provided by the embodiments of the present application;

[0057] Figure 2 Schematic diagram of the decision-making process of the MDP model for pumping wells provided by the embodiments of the present application;

[0058] Figure 3 Schematic diagram of the AC framework of the energy consumption optimization control model for pumping wells provided by the embodiments of the present application;

[0059] Figure 4 Optimization control flow chart of the energy consumption optimization control model for pumping wells provided by the embodiments of the present application;

[0060] Figure 5 Schematic diagram of the structure of the energy consumption optimization control device for pumping wells based on edge computing provided by the embodiments of the present application;

[0061] Figure 6 Schematic diagram of the structure of the energy consumption optimization control system for pumping wells based on edge computing provided by the embodiments of the present application;

[0062] Figure 7 Schematic diagram of the overall architecture of the energy consumption optimization control system for pumping wells based on edge computing provided by the embodiments of the present application. Detailed implementation manners

[0063] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0064] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0065] It should also be understood that the term "and / or" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0066] As used in the specification of this application and the appended claims, the term "if" may be construed as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" depending on the context.

[0067] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0068] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0069] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not mean the order of execution is prior or posterior, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0070] In order to illustrate the technical solution of this application, the following specific embodiments are used for illustration.

[0071] The inventors found that in the prior art, it has been difficult to obtain a high-quality solution to the energy consumption problem of pumping wells. The traditional control system of pumping wells cannot process complex data in real time and make optimal decisions, so the optimization of energy consumption cannot be achieved in real time. In recent years, with the development of edge computing technology, its powerful computing power and real-time performance have provided a new solution for the energy consumption optimization of pumping wells. However, how to apply edge computing technology to the energy consumption optimization control of pumping wells still requires further research and exploration. Based on this, after further research and development, the inventors made this application to provide a method, device, and system for optimizing the energy consumption control of pumping wells based on edge computing.

[0072] Embodiment 1

[0073] In the embodiment of the present application, the inventor proposes an energy consumption optimization control system for pumping wells based on edge computing. The main components of this system are, from bottom to top, an intelligent perception layer, a communication layer, a cloud-edge collaboration layer, and an application layer. In the cloud-edge collaboration layer of this system, variable frequency control of the pumping well (i.e., the pumping well system) is implemented on the edge side, and relevant coordination management and computing support are implemented on the cloud computing side. On the cloud computing side, the cloud-edge collaboration layer performs tasks such as data analysis and collaborative management of different data sources, such as access to pumping wells, permission verification, and optimization service for the pumping frequency of the optimal pumping well. On the edge side, by perceiving, analyzing, and processing data of each pumping well, the collaborative computing task is further completed, the message push and control of the pumping well to be optimized are realized, and moreover, the energy consumption optimization control model of the pumping well constructed based on DDPG is embedded into the energy consumption controller. Through an energy consumption optimization control method for pumping wells based on edge computing, variable frequency control of the pumping well system is realized on the edge side.

[0074] In the above energy consumption controller, it includes three functions: industrial control data acquisition, data storage and forwarding, and edge inference application. Among them, the industrial control data acquisition function: supports flexible configuration of port communication, acquisition block (acquisition point) protocol and information, and networking information for 1 network port, 3 serial ports, and Zigbee, is compatible with mainstream Modbus multi-protocol adaptation in the oilfield field, integrates multiple communication methods such as 4G / Zigbee / wired, and meets the real-time and efficient acquisition access of industrial control data of PLC, instruments, and RTUs in the oil and gas field and the monitoring of the operation data of the energy consumption controller; the data storage and forwarding function: supports parsing and mapping the data collected at the oil and gas production site to local storage. At the same time, the industrial control data centrally stored in the energy consumption controller or the inference result data generated in real time by the energy consumption optimization control model of the pumping well can be pushed to the intelligent Internet of Things system or a third-party system in the production network through MQTT message forwarding, and supports data parsing, function calculation, and resume transmission after network interruption; the edge inference application function: supports uploading the energy consumption optimization control model of the pumping well based on DDPG to the current energy consumption controller, and online start-stop, parameter configuration, and version management of the energy consumption optimization control model of the pumping well. During the production process, according to the environmental parameters of the pumping well, the energy consumption optimization control model of the pumping well based on DDPG can be used to obtain the optimal production parameters and achieve the lowest energy consumption of the pumping well.

[0075] The following specifically introduces the energy consumption optimization control method for pumping wells based on edge computing in the cloud-edge collaboration layer:

[0076] The embodiment of the present application provides an energy consumption optimization control method for pumping wells based on edge computing. Refer to Figure 1 As shown, this method includes:

[0077] S101: Real-time obtain the system environment state of the pumping well system.

[0078] In the embodiments of the present application, the real-time acquisition of the system environment state of the pumping unit well system includes:

[0079] Based on a preset electrical parameter module, the motor running current and voltage frequency of the pumping unit well system are acquired in real time;

[0080] Based on a motor speed sensor, the motor rotation angle of the pumping unit well system is acquired in real time;

[0081] Based on a temperature sensor and a pressure sensor, the changes in temperature and pressure of the pumping unit well system are acquired in real time;

[0082] Based on a dynamometer card sensor, the dynamometer card parameters of the pumping unit well system are acquired in real time.

[0083] In the embodiments of the present application, the intelligent perception layer in the energy consumption optimization control system of the pumping unit well, namely the intelligent perception device, is mainly responsible for collecting the perception information of the production and production status of each pumping unit well and the feedback and control of the corresponding task units, including a preset electrical parameter module, a motor speed sensor, a temperature sensor, a pressure sensor, and a dynamometer card sensor. The preset electrical parameter module collects parameters such as the motor running current and voltage frequency, and monitors the driving magnitude in real time; the motor speed sensor monitors the operation of the motor according to the collected motor rotation angle; the temperature sensor and the pressure sensor mainly reflect the changes in the equipment and downhole during the operation of the pumping unit: the dynamometer card sensor mainly collects the parameters required to generate the dynamometer card. The intelligent perception layer can obtain information such as the production status of the pumping unit well and the fluid parameters in the wellbore, and perceive and provide real-time feedback on the variable frequency control environment of the pumping unit well.

[0084] The system environment state data of the pumping unit well system collected by various sensors in the intelligent perception layer includes real-time dynamometer card data and oilfield production parameters. Among them, the real-time dynamometer card data mainly includes load, displacement, maximum load, and minimum load, and the oilfield production parameters include comprehensive electrical parameters, oil pressure, casing pressure, back pressure, power consumption, stroke, and pumping frequency.

[0085] The data of the system environment state of the pumping unit well system acquired in real time by the intelligent perception layer in the above manner are transmitted in real time to the cloud-edge collaboration layer through the communication layer for processing.

[0086] S102: Through the energy consumption optimization control model of the pumping unit well based on DDPG, according to the current system environment state, a current control strategy is obtained based on a random strategy and sent to the pumping unit well system.

[0087] In the embodiments of the present application, according to the characteristics of the data information obtained by the intelligent perception layer, the inventor designs the neural network structure of the energy consumption optimization control model for the pumping well, selects to construct the energy consumption optimization control model for the pumping well based on DDPG, uses the Markov decision process model (MDP) to implement the decision-making process between the agent and the environmental state during the reinforcement learning process, and adopts the actor-critic (AC) framework to implement it. Moreover, the neural network parameters and the system environmental state of the energy consumption optimization control model for the pumping well are initialized.

[0088] In the embodiments of the present application, the premise of combining the energy-saving optimization control of the pumping well with reinforcement learning is to establish a pumping well MDP model. The MDP model is usually described as a quadruple [S, A, P, R]. Therefore, the inventor defines that in the pumping well MDP model, S is the state space represented by the pumping well and external environmental parameters; A is the action space composed of available control instructions of the edge layer of the pumping well; P is the transition probability between different environmental states of the pumping well system; R is the immediate reward that can be obtained by taking different control actions a, usually in the form of a function with a penalty term. This model is as Figure 2 shown. The energy consumption optimization control model for the pumping well embedded in the edge energy consumption controller gives the current control action, that is, the current control strategy a t according to the current system environmental state s t . After the pumping well system executes this control strategy, a new state s t+1 is generated, and the edge energy consumption controller obtains an immediate reward r t .

[0089] In the embodiments of the present application, the inventor specifies the specific parameters of the pumping well MDP model according to the actual situation of the data obtained by the intelligent perception layer and the evaluation conditions of the periodic liquid production and energy consumption of the pumping well as follows:

[0090] 1) State space S: The surface dynamometer card and the stroke frequency law within the current control period;

[0091] 2) Action space A: According to the principle that the action selection conforms to the actual controlled variable, and in combination with the actual situation of the research object, the stroke of the pumping well is selected, and the frequency corresponding to the motor is used as the control action.

[0092] 3) Transition probability P: The transition probability depends on the true state of the system environment after the control action is executed, and the agent needs to make an unbiased estimate of it by relying on multiple Monte Carlo samplings.

[0093] 4) Immediate reward R: The liquid production and the current energy consumption of the pumping well are the main indicators for evaluating the operation of the pumping well.

[0094] In the embodiments of the present application, the energy consumption optimization control model of the pumping unit well constructed based on DDPG is implemented using the AC framework. The AC framework is as shown in Figure 3 where the actor network gives actions in different environmental states, and the critic network evaluates the quality of this action. The two are in a relationship of mutual supervision and mutual learning. Essentially, the actor network learns the state value function from the environment, and the critic network learns the state-action value function. After the intelligent agent completes the learning task, only the actor network is needed to complete a specific reinforcement learning task.

[0095] In the embodiments of the present application, the system environmental state of the pumping unit well system transmitted by the intelligent perception layer is obtained in real time. Through the energy consumption optimization control model of the pumping unit well, according to the current system environmental state, a current control strategy (that is, the control strategy for the next moment issued at the current moment) can be obtained based on a random strategy and sent to the pumping unit well system.

[0096] S103: Obtain the system environmental state and the immediate reward at the next moment after the pumping unit well system executes the current control strategy.

[0097] In the embodiments of the present application, when the pumping unit well system executes the current control strategy, the system environmental state will change randomly. At the same time, the energy consumption optimization control model of the pumping unit well will obtain the system environmental state at the next moment and an immediate reward.

[0098] In the embodiments of the present application, the immediate reward is obtained through the following formula 3:

[0099]

[0100] In the formula, R is the immediate reward; -Energy(s t , a t ) is the energy consumption of the pumping unit system in the t-th time period; Q is the liquid production of the pumping unit well in the period; γ is the liquid production penalty coefficient; is the upper limit of the liquid production of the pumping unit well in the period; Q is the lower limit of the liquid production of the pumping unit well in the period.

[0101] In the embodiments of the present application, when setting the MDP model of the pumping unit well, the liquid production of the pumping unit well and the current energy consumption are used as the main indicators for evaluating the operation of the pumping unit well. In formula 3, -Energy(s t , a t) is the energy consumption of the pumping unit system in the t-th time period. Since the smaller the energy consumption, the better, it is generally taken as a negative value; Q is the liquid production of the pumping unit well within the period. The higher the production, the better. However, in the actual production process, too large fluctuations in production will have a certain impact on the formation. In order to ensure that the oil well production is within a certain range, a penalty function γ regarding Q needs to be added; Take the maximum value of the inflow performance relationship (IPR), that is, the maximum production corresponding to a pressure of 0. Q Take the average liquid production of normal production in the recent 10 days.

[0102] S104: Integrate the current system environment state, the current control strategy, the immediate reward, and the system environment state at the next moment into a state transition data.

[0103] In the embodiment of the present application, the received current system environment state s t , the current control strategy a t , the immediate reward rt and the system environment state s at the next moment t+1 are integrated into a state transition data, that is, integrated into the data format of (s t , a t , r t , s t+1 ).

[0104] S105: Repeat the above process of obtaining state transition data until a set number of state transition data is obtained.

[0105] In the embodiment of the present application, the above steps S102 to S104 are repeatedly executed, that is, continuously obtain the current system environment state, and give the corresponding current control strategy. After the pumping unit well system executes the current control strategy, the system environment state changes, and the system environment state and the immediate reward at the next moment are obtained, and the current system environment state, the current control strategy, the immediate reward, and the system environment state at the next moment are integrated into a state transition data. Repeat the above process to continuously obtain the state transition data (s t , a t , r t , s t+1 ) until a state transition data of the set data is obtained.

[0106] S106: Randomly extract multiple state transition data from the set number of state transition data, and update the parameters of the energy consumption optimization control model of the pumping unit well.

[0107] In the embodiment of the present application, when a set number of state transition data is obtained, randomly extract N state transition data, and use the gradient descent method to update the parameters of the energy consumption optimization control model of the pumping unit well once.

[0108] In the embodiment of the present application, the energy consumption optimization control model of the pumping well includes an actor network and a critic network; randomly extracting a plurality of state transition data from the set number of state transition data to update the parameters of the energy consumption optimization control model of the pumping well, including:

[0109] Randomly extract a plurality of state transition data from the set number of state transition data, and according to the system environment state at the next moment in each state transition data, obtain the corresponding control strategy at the next moment through the actor network;

[0110] Through the critic network, according to the current system environment state, the current control strategy, the system environment state at the next moment, and the control strategy at the next moment, obtain the corresponding liquid production of the pumping well in the current cycle and the liquid production of the pumping well in the next cycle;

[0111] Calculate the TD error according to the liquid production of the pumping well in the current cycle and the liquid production of the pumping well in the next cycle;

[0112] Update the weight parameters of the critic network according to the TD error;

[0113] Update the weight parameters of the actor network according to the liquid production of the pumping well in the current cycle.

[0114] In the embodiment of the present application, updating the weight parameters of the critic network according to the TD error includes:

[0115] Update the weight parameters of the critic network through the following formula 1 according to the TD error:

[0116] ω t+1 = ω t + α ω [r t + γQ(S t+1 , a t+1 ) - Q(s t , a t )] Formula 1;

[0117] In the formula, ω t+1 is the weight parameter of the critic network at the (t + 1)-th moment; ω t is the weight parameter of the critic network at the t-th moment; α ω is the learning rate of the critic network; r t + γQ(s t+1 , a t+1 ) - Q(s t , a t ) is the TD error.

[0118] In the embodiment of the present application, the weight parameters of the actor network are updated according to the fluid production of the pumping well in the current cycle, including:

[0119] According to the current cycle pumping well fluid production, the weight parameters of the actor network are updated through the following formula 2:

[0120]

[0121] Where, θ t+1 is the weight parameter of the actor network at the t+1th moment; θ t is the weight parameter of the actor network at the tth moment; α θ is the learning rate of the actor network; is a deterministic policy gradient.

[0122] In the embodiment of the present application, N pieces of state transition data are randomly selected from the acquired set number of state transition data, and for each randomly selected piece of state transition data (s t ,a t ,r t ,s t+1 ), the actor network will provide the critic network with the next moment's control strategy a according to the next moment's system environment state t+1 . The critic network is based on the current system environment state and the current control strategy (s t ,a t ) and the system environment state at the next moment and the control strategy at the next moment (s t+1 ,a t+1 ), and obtain the corresponding current cycle pumping well liquid production Q(s t ,a t ) and the next cycle of pumping well liquid production Q(s t+1 ,a t+1 ) and calculate the TD deviation. The critic network updates the weight parameters according to the calculated TD deviation through the above formula 1, and the actor network updates the current cycle pumping well liquid production Q(s t ,a t ), update the weight parameters through the above formula 2, and thus complete the parameter update of the energy consumption optimization control model of the pumping well.

[0123] In the embodiment of the present application, as Figure 3 is a schematic diagram of the AC framework. The policy module in the figure is the actor network. According to the current system environment state s t , randomly give the current control strategy, i.e. action a t , the action transition probability in the MDP model is expressed in state st Execute action a next t Then transfer to state s t+1 The probability that after the pumping unit well system executes the current control strategy, it returns the immediate reward, that is, the reward r t To the evaluation module (i.e., the critic network), the critic network calculates the TD deviation and accumulates the return Q t Return it to the actor network

[0124] S107: Repeat the above process of obtaining state transition data and updating parameters until the preset optimization conditions are met, and obtain the optimal energy consumption optimization control strategy for the pumping unit well

[0125] In the embodiments of the present application, it is necessary to continuously iterate and update the parameters of the energy consumption optimization control model of the pumping unit well to obtain the optimal energy consumption optimization control strategy for the pumping unit well. In each iteration and update process, steps S102 - S105 need to be executed. After obtaining the preset number of state transition data, randomly select N state transition data to update the parameters of the energy consumption optimization control model of the pumping unit well. Repeat the process of obtaining state transition data and updating parameters for iteration. As the number of iterations gradually increases, the parameters of the neural network will gradually converge and stabilize, and the control actions given by the energy consumption optimization control model of the pumping unit well, that is, the control strategy, will become more and more stable. When the cumulative return of the system no longer changes significantly, it is considered that the learning process ends. At this time, the optimal strategy has been obtained, and the updated and optimized energy consumption optimization control model of the pumping unit well can be put into formal use. In addition, when the maximum number of iterations is reached, regardless of whether the optimal strategy has been learned at this time, the reinforcement learning task will be forcibly ended, and the strategy at the maximum number of iterations will be used as the optimal energy consumption optimization control strategy for the pumping unit well

[0126] In the embodiments of the present application, as Figure 4 shown, it is the optimization control flow chart of the energy consumption optimization control model of the pumping unit well. In the initialization stage of the figure, first, according to the input energy consumption optimization data of the pumping unit well (including stroke frequency, liquid production, and current power consumption, etc.), set the neural network used by the model, and initialize the parameters of the neural network and the system environment of the pumping unit well. The energy consumption optimization control model in the energy consumption controller gives the current control strategy (i.e., the control action a t ) randomly according to the current system environment state s t . After the pumping unit well system executes this control action, it updates the system environment state s at the next moment t+1 , calculates the immediate reward r of this control action t , and then converts and stores the current system environment state, current control strategy, immediate reward, and system environment state at the next moment as a state transition data (s t , a t , rt , s t+1 ), repeat the above process. When the obtained state transition data reaches the set quantity, that is, when the target number of times is reached, update the neural network parameters of the energy consumption optimization control model of the pumping unit well in the edge energy consumption controller, and iteratively update until the preset optimization condition of low energy consumption and high production is reached, then end the optimization to obtain the optimal energy consumption optimization control strategy for the pumping unit well.

[0127] In the embodiment of the present application, in order to verify the feasibility and effectiveness of the energy consumption optimization control method for pumping unit wells provided by the present application, experiments were carried out in a certain oilfield. 10 pumping unit wells were selected for testing, and the energy consumption data before and after using this method to optimize and control the pumping unit wells were compared. The experimental results show that using this method can effectively reduce the energy consumption of pumping unit wells, and the average energy saving rate can reach more than 10%. At the same time, the experiment also found that different pumping unit wells have different responses to the optimization control strategy, and for some high-energy-consuming pumping unit wells, the optimization effect obtained by using this method is more significant.

[0128] The energy consumption optimization control method for pumping unit wells based on edge computing provided by the embodiment of the present application obtains the system environment state of the pumping unit well system in real time, and randomly gives a control strategy through the energy consumption optimization control model of the pumping unit well based on DDPG to obtain the system environment state and immediate reward at the next moment after the pumping unit well system executes this control strategy. Repeat the above process, and continuously update the parameters of the energy consumption optimization control model of the pumping unit well through the method of reinforcement learning, and finally obtain the optimal energy consumption optimization control strategy for the pumping unit well to realize the energy consumption optimization of the pumping unit well. This method has strong computing power and real-time performance, can better adapt to complex data environments and real-time control requirements, reduces the energy consumption of pumping unit wells, improves efficiency, reduces the need for manual intervention, and moreover, does not require modification of the existing pumping units to effectively reduce the energy consumption of pumping unit wells, and has high practical value and popularization prospects.

[0129] Embodiment 2

[0130] Based on the same inventive concept, the embodiment of the present application also provides an energy consumption optimization control device for pumping unit wells based on edge computing. Referring to Figure 5 as shown, this device includes:

[0131] A real-time acquisition module 101, which is used to acquire the system environment state of the pumping unit well system in real time;

[0132] A random policy module 102, which is used to obtain the current control strategy based on a random policy through the energy consumption optimization control model of the pumping unit well based on DDPG according to the current system environment state and send it to the pumping unit well system;

[0133] A determination module 103, configured to obtain the system environment state and the immediate reward at the next moment after the pumping unit well system executes the current control strategy;

[0134] An integration module 104, configured to integrate the current system environment state, the current control strategy, the immediate reward, and the system environment state at the next moment into a state transition data;

[0135] A repeated acquisition module 105, configured to repeat the process of obtaining the state transition data until a set number of state transition data are obtained;

[0136] An update module 106, configured to randomly extract a plurality of state transition data from the set number of state transition data to update the parameters of the pumping unit well energy consumption optimization control model;

[0137] An iteration module 107, configured to repeat the process of obtaining the state transition data and performing parameter update until a preset optimization condition is met, so as to obtain an optimal pumping unit well energy consumption optimization control strategy.

[0138] Embodiment III

[0139] Based on the same inventive concept, an embodiment of the present application further provides a pumping unit well energy consumption optimization control system based on edge computing. Referring to Figure 6 as shown, the system includes an intelligent perception layer 1, a communication layer 2, a cloud-edge collaboration layer 3, and an application layer 4:

[0140] The intelligent perception layer 1 is configured to collect the system environment state of the pumping unit well system;

[0141] The communication layer 2 is communicatively connected to the intelligent perception layer 1 and the cloud-edge collaboration layer 3;

[0142] The cloud-edge collaboration layer 3 is configured to obtain a pumping unit well energy consumption optimization control strategy based on the pumping unit well energy consumption optimization control method as described in Embodiment I;

[0143] The application layer 4 is communicatively connected to the cloud-edge collaboration layer 3 and is configured to display the pumping well energy consumption optimization control situation.

[0144] In an embodiment of the present application, as Figure 7As shown in the figure, the intelligent perception layer 1 is an intelligent perception device, which is mainly responsible for collecting the perception information of the production and production status of each pumping well, as well as the feedback and control of the corresponding task units, including a preset electrical parameter module, a motor speed sensor, a temperature sensor, a pressure sensor, and a dynamometer sensor. The preset electrical parameter module collects parameters such as the running current and voltage frequency of the motor, and monitors the driving force in real time; the motor speed sensor monitors the running condition of the motor according to the collected motor rotation angle; the temperature sensor and the pressure sensor mainly reflect the changes in the equipment and downhole during the operation of the pumping unit: the dynamometer sensor mainly collects the parameters required to generate the dynamometer diagram. The intelligent perception layer 1 can obtain information such as the production status of the pumping well and the fluid parameters in the wellbore, and perceive and provide real-time feedback on the variable frequency control environment of the pumping well.

[0145] The communication layer 2 is a communication device, which is mainly responsible for the information communication between the devices in the energy consumption optimization control system of the pumping well. Specifically, it is used to communicate and connect with the intelligent perception layer 1 and the cloud-edge collaboration layer 3, and is responsible for the information collaboration of the production and variable frequency of the pumping well and the information interaction within the energy consumption optimization control system of the pumping well. It mainly communicates with the cloud-edge collaboration layer 3 through a combination of wireless and wired methods, and the physical security of the device needs to be considered. The communication layer 2 mainly communicates based on the MQTT protocol, and pushes messages to the intelligent Internet of Things system or third-party system of the production network through MQTT message forwarding, supporting data parsing, function calculation, and resume data transmission after network interruption, realizing cross-platform message subscription and sending, which not only ensures the low-power consumption requirements of the lightweight and streamlined edge side, but also provides performance guarantee for high-concurrency million-level connections on the cloud side.

[0146] The cloud-edge collaboration layer 3 is a cloud-edge collaboration device, which is mainly responsible for the variable frequency control of the pumping well implemented on the edge side, as well as the relevant coordination management and computing support implemented on the cloud computing side. It is also responsible for the information communication in the energy consumption optimization control system of the pumping well, mainly the information collaboration of the production and variable frequency of the pumping well and the information interaction within the system.

[0147] On the cloud computing side, the cloud-edge collaboration layer performs tasks such as data analysis and collaborative management of different data sources, such as the access of pumping wells, permission verification, and the optimization service of the pumping times of the best pumping wells; on the edge side, through the perception, analysis, and processing of data from each pumping well, the collaborative computing task is further completed, realizing the message push and control of the pumping wells to be optimized. And, the energy consumption optimization control model of the pumping well constructed based on DDPG is embedded in the energy consumption controller, and through an energy consumption optimization control method of the pumping well based on edge computing, the variable frequency control of the pumping well system is realized on the edge side.

[0148] Among them, for the specific method of obtaining the optimal energy consumption optimization control strategy of the pumping well according to the energy consumption optimization control model of the pumping well based on DDPG in the cloud-edge collaboration layer 3, reference can be made to Embodiment 1, which will not be elaborated here.

[0149] The application layer 4 mainly refers to the intelligent control platform for oil and gas production energy. Operators can monitor the energy consumption of pumping wells in real time through the Web side, and query and process the energy consumption optimization situation.

[0150] In the operation process of the energy consumption optimization control system for pumping wells in this application embodiment, first, various sensors deployed in the intelligent perception layer 1 collect real-time data during the production process of the pumping well, that is, the real-time system environment state. Then, the energy consumption controller on the edge side of the cloud-edge collaboration layer 3 preliminarily processes the data according to business rules and models. The edge side can provide computing offloading by using the designed edge technology, and can achieve real-time response of the devices in the intelligent perception layer 1 after communication protocol parsing and data format conversion. At the same time, on the edge side, abnormal detection and filtering processing can be performed on the data collected by the intelligent perception layer 1, reducing the disadvantages of storing a large amount of anomalies in the traditional cloud computing mode and the drawback of effective data being submerged, improving the response and reducing latency, and further completing collaborative computing tasks, realizing message push and control of the pumping wells to be optimized, and embedding the energy consumption optimization control model for pumping wells based on DDPG into the energy consumption controller, realizing variable frequency control of the pumping wells at the edge layer. On the cloud computing side, the cloud-edge collaboration layer 3 performs work on data analysis and collaborative management of different data sources, such as access to pumping wells, permission verification, and stroke frequency optimization services for the best pumping wells.

[0151] Embodiment 4

[0152] Based on the same inventive concept, this application embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the energy consumption optimization control method for pumping wells based on edge computing as described in Embodiment 1 above.

[0153] Embodiment 5

[0154] Based on the same inventive concept, this application embodiment also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the energy consumption optimization control method for pumping wells based on edge computing as described in Embodiment 1 above.

[0155] Embodiment 6

[0156] Based on the same inventive concept, this application embodiment also provides a computer program product containing instructions. When the computer program product runs on a computer device, it causes the computer device to execute the energy consumption optimization control method for pumping wells based on edge computing as described in Embodiment 1 above.

[0157] Embodiment 7

[0158] Based on the same inventive concept, an embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a computer program or instruction to implement the energy consumption optimization control method for pumping wells based on edge computing as described in the first embodiment above.

[0159] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified function in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0161] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified function in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the specified function in one Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0163] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. A method for optimizing energy consumption of a pumping well based on edge computing, characterized in that: Applied to the cloud-edge collaboration layer, including: Obtain the system environment status of the pumping well system in real time; Through the DDPG-based pumping well energy consumption optimization control model, according to the current system environment state, based on the random strategy, the current control strategy is obtained and sent to the pumping well system; Obtaining the system environment state and instant feedback at the next moment after the pumping well system executes the current control strategy; Integrate the current system environment state, the current control strategy, the immediate feedback and the system environment state at the next moment into a state transfer data; Repeat the above process of obtaining state transition data until a set amount of state transition data is obtained; Randomly extracting a plurality of state transition data from the set number of state transition data, and updating the parameters of the pumping well energy consumption optimization control model; Repeat the above process of obtaining state transfer data and updating parameters until the preset optimization conditions are met, and obtain the optimal pumping well energy consumption optimization control strategy.

2. The method according to claim 1, characterized in that The pumping well energy consumption optimization control model comprises an actor network and a critic network; the step of randomly extracting a plurality of state transition data from the set number of state transition data and updating the parameters of the pumping well energy consumption optimization control model comprises: Randomly extracting a plurality of state transition data from the set number of state transition data, and obtaining a corresponding control strategy at the next moment through the actor network according to the system environment state at the next moment in each state transition data; Through the critic network, according to the current system environment state and the current control strategy and the next moment system environment state and the next moment control strategy, the corresponding current cycle pumping well liquid production and the next cycle pumping well liquid production are obtained; Calculating the TD deviation according to the liquid production of the pumping well in the current cycle and the liquid production of the pumping well in the next cycle; According to the TD deviation, updating the weight parameters of the critic network; The weight parameters of the actor network are updated according to the fluid production of the pumping well in the current period.

3. The method according to claim 2, characterized in that The updating of the weight parameters of the critic network according to the TD deviation includes: According to the TD deviation, the weight parameters of the critic network are updated by the following formula 1: ω t+1 = ω t + α ω [r t + γQ(s t+1 , a t+1 ) - Q(s t , a t )] Formula 1; In the formula, ω t+1 is the weight parameter of the critic network at the t+1th moment; ω t is the weight parameter of the critic network at the tth moment; α ω is the learning rate of the critic network; r t +γQ(s t+1 ,a t+1 )-Q(s t ,a t ) is the TD deviation.

4. The method according to claim 2, characterized in that The updating of the weight parameters of the actor network according to the fluid production of the pumping well in the current period includes: According to the fluid production of the pumping well in the current cycle, the weight parameters of the actor network are updated by the following formula 2: In the formula, θ t+1 is the weight parameter of the actor network at the t+1th moment; θ t is the weight parameter of the actor network at the tth moment; α θ is the learning rate of the actor network; is the deterministic policy gradient.

5. The method according to claim 1, characterized in that The instant return is obtained by the following formula 3: Where R is the immediate return; -Energy(s t ,a t ) is the energy consumption of the pumping system in the tth time period; Q is the liquid production of the pumping well in the period; γ is the liquid production penalty coefficient; is the upper limit of the liquid production of the pumping well within the cycle; Q It is the lower limit of the liquid production of the pumping well within the cycle.

6. The method according to claim 1, characterized in that The real-time acquisition of the system environment status of the pumping well system includes: Based on the preset electrical parameter module, the motor operating current and voltage frequency of the pumping well system are obtained in real time; Based on the motor speed sensor, the motor rotation angle of the pumping well system is obtained in real time; Based on temperature sensors and pressure sensors, the temperature and pressure changes of the pumping well system are obtained in real time; Based on the dynamometer sensor, the dynamometer parameters of the pumping well system are obtained in real time.

7. An energy consumption optimization control device for a pumping well based on edge computing, characterized in that: Applied to the cloud-edge collaboration layer, including: A real-time acquisition module is used to obtain the system environment status of the pumping well system in real time; A random strategy module is used to obtain a current control strategy based on a random strategy and send it to the pumping well system according to the current system environment state through a pumping well energy consumption optimization control model based on DDPG; The first determination module is used to obtain the system environment state and instant feedback at the next moment after the pumping well system executes the current control strategy; An integration module, used to integrate the current system environment state, current control strategy, immediate feedback and the system environment state at the next moment into a state transfer data; A repeated acquisition module is used to repeat the above process of obtaining the state transition data until a set amount of state transition data is obtained; An updating module, used for randomly extracting a plurality of state transition data from the set number of state transition data, and updating the parameters of the pumping well energy consumption optimization control model; The iteration module is used to repeat the above process of acquiring state transfer data and updating parameters until the preset optimization conditions are met, thereby obtaining the optimal pumping well energy consumption optimization control strategy.

8. An energy consumption optimization control system for oil pumping wells based on edge computing, characterized in that: Including intelligent perception layer, communication layer, cloud-edge collaboration layer and application layer: The intelligent sensing layer is used to collect the system environment status of the pumping well system; The communication layer is communicatively connected with the intelligent perception layer and the cloud-edge collaboration layer; The cloud-edge collaboration layer is used to obtain an energy consumption optimization control strategy for a pumping well based on the energy consumption optimization control method for a pumping well based on edge computing as claimed in claim 1; The application layer is communicated with the cloud-edge collaboration layer to display the energy consumption optimization control status of the oil pumping wells.

9. A computer-readable storage medium, in which a computer program is stored. When the program is executed by a processor, the processor executes the energy consumption optimization control method for a pumping well based on edge computing as described in any one of claims 1 to 6.

10. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the energy consumption optimization control method for a pumping well based on edge computing as described in any one of claims 1 to 6 is implemented.

11. A computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the method for optimizing energy consumption of a pumping well based on edge computing as described in any one of claims 1 to 6.

12. A chip, comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a computer program or instruction to implement the energy consumption optimization control method for a pumping well based on edge computing as described in any one of claims 1 to 6.

Images