A pig house environment control method and device based on digital twinning and reinforcement learning, and a medium

Abstract

The application discloses a pig house environment control method and device based on digital twinning and reinforcement learning, and a medium, and relates to the technical field of large-scale breeding. The method comprises the following steps: extracting a plurality of data subsets from a data set; for each data subset, inputting a sample pig house state at the last time in the data subset into a network of a reinforcement learning model, obtaining a virtual environment control instruction, updating a device state, inputting the updated data subset into a digital twinning model, and obtaining an updated pig house internal environment state; updating an experience replay pool based on the updated data subset and the updated pig house internal environment state, training the network until all data subsets are traversed, and determining the trained network as an environment control model; inputting a pig house state at a current time into the environment control model, obtaining an environment control instruction at the current time, and thereby controlling the pig house environment. The application can improve the accuracy and real-time performance of pig house environment regulation and control.

Description

Technical Field

[0001] This application relates to the field of large-scale breeding technology, and in particular to a method, equipment and medium for controlling pig house environment based on digital twins and reinforcement learning. Background Technology

[0002] In large-scale pig farming, the high stocking density and relatively enclosed space of pigsties make it easy for the animals' daily activities and metabolism to deteriorate the internal environment. To improve animal comfort and production efficiency, pigsties are typically equipped with environmental control equipment, including fans, evaporative cooling pads, and heaters, supplemented by manual or mechanical manure removal. Fans are used for ventilation, evaporative cooling pads achieve cooling in summer through water evaporation, heaters provide heating in winter, and manure removal helps maintain cleanliness and reduce harmful gases. Through proper control, pigsties can maintain a suitable production environment, thereby ensuring the benefits of intensive farming.

[0003] In some cases, environmental control in large-scale pig farms relies heavily on manual experience or simple methods such as threshold control and timed control. While these methods can provide some regulation, they are insufficient for real-time, dynamic, and precise adjustments, leaving room for improvement in environmental suitability and energy conservation. Theoretically, reinforcement learning can achieve intelligent environmental control through pre-trained models. However, directly using real pig farms for training can lead to environmental parameters deviating from suitable ranges during the trial-and-error process, causing stress and even disease in pigs, resulting in economic losses. Therefore, obtaining reliable data for reinforcement learning training without affecting actual production has become a pressing technical problem.

[0004] In summary, while the control strategies provided above can play a certain role in regulating the environment and improving animal living comfort, they still cannot regulate the pigsty environment in a real-time, dynamic, and efficient manner. Therefore, there are shortcomings and room for improvement in terms of environmental suitability control and energy conservation and emission reduction. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, and medium for controlling pig house environment based on digital twins and reinforcement learning, which can improve the accuracy and real-time performance of pig house environment control.

[0006] To achieve the above objectives, this application provides the following solution.

[0007] Firstly, this application provides a method for controlling the pigsty environment based on digital twins and reinforcement learning, including: A digital twin model of the pigsty environment is constructed; the digital twin model is obtained by training an improved LSTM model using sample pigsty state sequences. A dataset is constructed, and multiple data subsets are extracted from the dataset; the dataset includes a sequence of sample pigsty states; the sequence of sample pigsty states includes: the state of the pigsty environment, the state of the equipment, the state of the pigsty environment, the number of pigs, and the average age of the pigs; For each subset of data, the state of the pigsty at the last moment in that subset is input into the reinforcement learning model. The network receives control commands for the virtual environment; The device status at the last moment in the data subset is updated based on the virtual environment control instructions to obtain the updated data subset; The updated subset of data is input into the digital twin model to obtain the updated environmental status of the pigsty. The experience replay pool of the reinforcement learning model is updated based on the updated data subset and the updated state of the pigsty environment to obtain the updated experience replay pool; the experience replay pool includes multiple elements; each element includes: virtual environment control instructions, rewards, and sample pigsty states at two adjacent time points; When the number of elements in the updated experience replay pool reaches a set number, random sampling is performed from the updated experience replay pool, and the... The network is trained to obtain the trained version. network; Until all data subsets have been traversed, the trained data will be processed. The network was identified as an environmental control model; Get the current status of the pigsty; The current state of the pigsty is input into the environmental control model to obtain the environmental control command for the current moment, and the pigsty environment is controlled according to the environmental control command for the current moment.

[0008] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the pigsty environment control method based on digital twins and reinforcement learning described in the first aspect.

[0009] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the pigsty environment control method based on digital twins and reinforcement learning described in the first aspect.

[0010] According to the specific embodiments provided in this application, this application has the following technical effects: This application trains an improved LSTM model using sample pigsty state sequences to obtain a digital twin model, achieving the pre-training of a digital twin model with the same changing patterns as a real pigsty, and in the reinforcement learning model... During the network training process, the final sample pigsty state from the data subset is first input. The network receives virtual environment control commands and then updates the device state at the last moment in the data subset using these commands, resulting in an updated data subset. This updated data subset is then input into the digital twin model to obtain the updated pigsty environment state. The updated data subset and the updated pigsty environment state are then used to update the experience replay pool. Finally, when the number of elements in the updated experience replay pool reaches a set number, random sampling training is performed from the updated experience replay pool. The network, after being trained The network reduces the need for collecting data on actual pigsty actions, avoiding the economic losses associated with directly using real pigsties to train reinforcement learning agents. Furthermore, by extracting the state sequences of sample pigsties multiple times, the reinforcement learning model can be trained over multiple preset time periods based on these state sequences, improving the accuracy of the environmental control model. In practical applications, inputting the current pigsty state sequence into the environmental control model generates the current environmental control commands, enabling real-time control of the pigsty environment and thus improving the accuracy and real-time performance of pigsty environmental regulation. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a pigsty environmental control method based on digital twins and reinforcement learning, which is provided in this application.

[0013] Figure 2 This is a schematic diagram of the overall process of a pigsty environmental control method based on digital twins and reinforcement learning provided in this application.

[0014] Figure 3 A schematic diagram illustrating the training process of the digital twin model provided in this application.

[0015] Figure 4 A schematic diagram of the structure of the improved LSTM model provided in this application.

[0016] Figure 5The reinforcement learning model provided in this application A schematic diagram of the network structure.

[0017] Figure 6 The reinforcement learning model provided in this application A schematic diagram of the overall process of network training.

[0018] Figure 7 The reinforcement learning model provided in this application A detailed flowchart illustrating the network training process.

[0019] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application applies to longitudinally ventilated pig houses in most regions. To achieve precise ventilation and environmental control, the pig house should be equipped with a fan with adjustable speed, a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, an ammonia sensor, and a wet curtain. It should also be equipped with a mechanical louvered fan to control the air intake area and a mechanical manure scraper for manure cleaning.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In one exemplary embodiment, such as Figure 1 and Figure 2 As shown, a method for controlling the pig farm environment based on digital twins and reinforcement learning is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is described using a server as an example, and includes the following steps S1 to S8.

[0024] Step S1: Construct a digital twin model of the pigsty environment; the digital twin model is obtained by training an improved LSTM model using sample pigsty state sequences.

[0025] As one feasible approach, step S1 specifically includes steps S11 to S14: Step S11: Based on the training period, randomly extract the sample pig house state sequence to obtain the sample pig house state sequence and the corresponding sample pig house environmental state during the training period.

[0026] Specifically, such as Figure 3 As shown, operational data from the pigsty was collected to obtain a sample pigsty state sequence. This was done at a certain sampling interval. The condition of the sample pigsties was sampled and recorded using computer equipment. Due to seasonal changes and the approximately four-month breeding cycle of fattening pigs, experienced pig farmers can first raise two or more rounds of fattening pigs normally, with a sampling interval. Data sampling will take eight months or longer, covering half of the four-season cycle, thus satisfying the requirement for extracting local climate information. Therefore, let the total duration of the sampling phase be... E Then the number of samples And let the obtained dataset be D The dataset consists of data sequences at different times, including: (1) temperature sequences: The number is 8, (2) humidity sequence: The number is 8, (3) carbon dioxide concentration sequence: The number is 8, (4) ammonia concentration sequence: The number is 8, (5) Water pump operating status sequence: The quantity is 2, (6) the louvered fan state sequence: The number is 4, (7) Fan operating status sequence: The number is 4, (8) manure cleaning interval sequence (9) Sequence of the number of pigs in the shed (10) Average age sequence of pigs in the shed: The sequence data of the first... Position, such as Right now The first moment i The ammonia concentration at each sensor location was recorded. Simultaneously, the outdoor temperature and humidity at each sampling time were recorded on my country's official weather website, generating (11) an outdoor temperature sequence. (12) Outdoor humidity sequence: .

[0027] The sample pigsty state sequences required to construct the digital twin model. After obtaining a sufficiently long and effective dataset, the following operation is performed: traverse the water pump operation sequences. 1. Visor state sequence Fan operation sequence manure cleaning interval sequence If at a certain sampling time t exist: or or or Then add the index of the sampling time to the array. After the traversal is complete, assume The number of data points is Therefore, it can be concluded that the number of data points that did not undergo any changes during the operation is [number missing]. One, and the remaining ones not added to The remaining indices of the array constitute Array. Let... ,from and Randomly selected from each Each index forms DatasetIndex And according to a 4:1 ratio, DatasetIndex Randomly divided into TrainDatasetIndex (i.e., the sequence of sample pigsty states and the corresponding environmental states within the sample pigsty during the training period) and TestDatasetIndex .

[0028] Step S12: Input the sample pigsty state sequence during the training period into the improved LSTM model to obtain the predicted environmental state inside the pigsty.

[0029] As an feasible approach, the improved LSTM model includes: a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, and an LSTM network layer connected in sequence; the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer all include convolutional layers and max pooling layers.

[0030] Specifically, such as Figure 4 As shown, the improved LSTM model includes: (1) a size of 15 1) A 1-D convolutional layer with 64 kernels; (2) Size 2 1 maximum pooling layer; (3) Size 7 1-D convolutional layer with 64 kernels; (4) Size 2 1 maximum pooling layer; (5) size 3 1-D convolutional layer with 32 kernels; (6) Size 2 1. Max pooling layer; (7) Single layer, LSTM network layer composed of 64 LSTM neurons. The input and output data of the network are marked in the figure. The training period is from time [t-179] to time t. The input data is the sample pig house state sequence from time [t-179] to time t. There are 48 data vectors in the sample pig house state sequence, including: sensor readings at each point, the operating status of equipment such as water pumps, the number of pigs in the house, etc. The output data is the reading prediction of each sensor (temperature, humidity, carbon dioxide concentration and ammonia concentration) at time t+1, a total of 32 data.

[0031] Step S13: Based on the environmental conditions inside the sample pigsty and the predicted environmental conditions inside the pigsty, calculate the error value using a loss function.

[0032] Step S14: Iteratively optimize the parameters of the improved LSTM model based on the error value until the error value is less than or equal to the preset error value, and obtain a digital twin model of the pig house environment.

[0033] Specifically, an improved LSTM model is trained so that it can predict future environmental data based on historical data with a set prediction accuracy. The training method of the improved LSTM model is as follows: (1) Construct the network using a common deep learning framework and randomly initialize the network parameters; (2) From TrainDatasetIndex Select a time index t, and extract the data from the 179 times before time t, the data at time t, and the sensor data at time t+1 from the sample pig house state sequence; (3) Input the data at time t and the data from the previous 179 times into the network, calculate the predicted value of the sensor data at time t+1, and use the difference between the predicted value and the true value as the error, and train the network parameters using the trainer provided by the deep learning framework; (4) Repeat steps (2) and (3), and extract the data from the sample pig house state sequence. TrainDatasetIndex All data are put into training; (5) In the same form, TestDatasetIndex The data corresponding to the index in the network is input to calculate the corresponding sensor prediction value, and the absolute value of the difference between all prediction values ​​and the true value is calculated and summed to obtain the total error E; (6) Set a certain error limit ,like If the network training is incomplete, repeat steps (2), (3), (4), and (5) until... This means that the twin network training has been completed, and a digital twin model of the pigsty environment has been obtained. .

[0034] Using the aforementioned data acquisition methods and neural network techniques, a digital twin model of the pigsty environment was established. The model can calculate the predicted value of the pig house environment at the next moment based on the current environmental state, equipment operating parameters and historical data. It can: (1) replace the real pig house environment and simulate the changes in the pig house environment under a certain working mode; (2) predict the future changes in the pig house environment in the real world and provide a basis for precision breeding.

[0035] Step S2: Construct a dataset and extract multiple data subsets from the dataset; the dataset includes a sequence of sample pigsty statuses; the sequence of sample pigsty statuses includes: the status of the pigsty's internal environment, the status of the equipment, the status of the pigsty's external environment, the number of pigs, and the average age of the pigs.

[0036] As one feasible approach, step S2 specifically includes steps S21 to S23: Step S21: Obtain the sample pigsty state sequence and construct a dataset; the pigsty internal environment state includes: temperature, humidity, carbon dioxide concentration and ammonia concentration; equipment state includes: water pump operation status, louver fan status, fan operation status and manure cleaning interval; the pigsty external environment state includes: outdoor temperature and outdoor humidity.

[0037] Step S22: Select multiple times as the last time.

[0038] Step S23: Based on the last moment, extract data from the dataset multiple times over preset time periods to obtain multiple data subsets; the preset time periods are... Time's up The period of time; The time is the last moment.

[0039] Step S3: For each data subset, input the sample pigsty state at the last moment in the data subset into the reinforcement learning model. The network receives control commands for the virtual environment.

[0040] Specifically, such as Figure 6 As shown, the control actions (i.e., virtual environment control commands) of the equipment in the pigsty are designed, where: the fan movement is represented by three actions {-1, 0, 1}. Assume that the current operating state of a certain fan is... Then -1 represents Decrease by 0.1, if If it is already 0, it will not decrease further; 0 represents... Remain unchanged, 1 represents Increase by 0.1, if If the value is already 1, it will not increase further; the pump's action is {-1, 0, 1}. Assume a certain pump's current operating state is... Then -1 represents Decrease by 0.1, if If it is already 0, it will not decrease further; 0 represents... Remain unchanged, 1 represents Increase by 0.1, if If the value is already 1, it will not increase further; the automatic blind action is {-1, 0, 1}. Assume a certain blind opening degree is... -1 represents Reduce by 10°, if If it is already 0, it will not decrease further; 0 represents... Remain unchanged, 1 represents Increase by 10°, if If the value is already 90, then no further increases will be made; the manure cleaning operation action set {0, 1} represents whether the manure cleaning operation will not be performed in the next moment or will be performed in the next moment, respectively.

[0041] Specifically, let the reinforcement learning model of the pigsty be an agent. Then, for the agent, its state (i.e., the state of the sample pigsty) is... for:

[0042]

[0043]

[0044]

[0045] .

[0046] Right now It is a 48-dimensional state space vector, and the value and meaning of each variable in the vector are as described in the previous section on data collection.

[0047] Specifically, different equipment is divided into different groups for coordinated control, as follows: Fan group 1 contains two fans, fan group 2 contains one fan, and fan group 3 contains two fans; the water pump group contains two water pumps for the pigsty; louver group 1 contains two louvers, and louver group 2 contains two louvers. Assume that the selectable action of fan group 1 is... Assume fan unit 2 is Assume fan unit 3 is The action set is {-1, 0, 1}. When a certain fan unit performs an action, all fans within the fan unit simultaneously perform the same action, and the action execution logic of the fan unit and the individual fans is the same. Let the water pump group be... The action set is {-1, 0, 1}. When the pump group performs a certain action, all pumps in the group simultaneously perform that action, and the execution logic of the pump group's action is the same as that of a single pump's action; let louver group 1 be... , The action set is {-1, 0, 1}. When a venetian blind group performs an action, all blinds within the group perform that action, and the action execution logic for the venetian blind group is the same as for a single blind. Let the action of the manure cleaning operation be... The action set is {0, 1}.

[0048] Let the Agent action be ,but for:

[0049] .

[0050] Right now It is a 7-dimensional vector, and the total action space size is There are a total of 1458 actions. The action design is as follows: act=[-1,-1,-1,-1,-1,-1,0] is the 1st action, act=[-1,-1,-1,-1,-1,-1,1] is the 2nd action, act=[-1,-1,-1,-1,-1,0,0] is the 3rd action, act=[-1,-1,-1,-1,-1,0,1] is the 4th action, and so on, with act=[1,1,1,1,1,1,1,1] being the 1458th action.

[0051] Among them, reinforcement learning models The structure of the network is as follows Figure 5 As shown, this network is a four-layer fully connected neural network, consisting of an input layer, a hidden layer, and an output layer. The input layer has one layer with 48 neurons, representing the input of the state vector. The hidden layer contains two layers of neurons, each with 128 neurons. The output layer has one layer with 1458 neurons, each neuron representing the action value of the i-th action given the current state input. .

[0052] Step S4: Update the device status at the last moment in the data subset based on the virtual environment control instructions to obtain the updated data subset.

[0053] As one feasible approach, step S4 specifically includes: updating the device status based on virtual environment control commands to obtain the virtual device status; and replacing the virtual device status in the data subset. The device status at any given time is used to obtain an updated subset of data.

[0054] Step S5: Input the updated data subset into the digital twin model to obtain the updated environmental status of the pigsty.

[0055] Step S6: Update the experience replay pool of the reinforcement learning model based on the updated data subset and the updated state of the pigsty environment to obtain the updated experience replay pool; the experience replay pool includes multiple elements; each element includes: virtual environment control instructions, rewards, and sample pigsty states at two adjacent time points.

[0056] As one feasible approach, step S6 specifically includes steps S61 to S64: Step S61: Obtain The external environment status of the pigsty and the status of virtual devices at time +1. The number of pigs at time +1 and The average age of pigs at time +1.

[0057] Step S62: Based on the updated environmental status of the pigsty, The external environmental conditions of the pigsty at time +1 The number of pigs at time +1 and The average age of pigs at time +1 is obtained. The state of the sample pigsty at time +1.

[0058] Step S63: Based on The environmental conditions inside the pigsty at any given time are calculated. Total reward at any given moment.

[0059] As a feasible approach, The formula for calculating the total reward at each moment is: ; in, for Total reward at each moment; A reward for temperature; for Temperature at any moment; A reward for humidity; for Humidity at any given moment; A reward for carbon dioxide; for Carbon dioxide at any given moment; A reward for ammonia concentration; for The ammonia concentration at a given time.

[0060] Specifically, the lower limit of the comfort temperature for this category of pigs is set as follows: The upper limit of the comfortable temperature is The lower critical temperature is The high critical temperature is The lower limit of comfortable humidity is The upper limit of comfortable humidity is Low critical humidity is High critical humidity is The upper limit of ammonia concentration is The upper limit of carbon dioxide concentration is For the collected sensor data, the state reward at the current moment is calculated using the following formula: .

[0061] .

[0062] .

[0063] .

[0064] Step S64: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Real-time sample pigsty status, virtual environment control commands, Momentary rewards and The sample pigsty state at time +1 is stored as an element in the experience replay pool, resulting in an updated experience replay pool.

[0065] Step S7: When the number of elements in the updated experience replay pool reaches the set number, randomly sample from the updated experience replay pool and... The network is trained to obtain the trained version. network.

[0066] As one feasible approach, step S7 specifically includes steps S71 to S74: Step S71: Randomly sample the elements in the updated experience replay pool to obtain multiple elements.

[0067] Step S72: For each element, apply the reinforcement learning model... Networks calculate and predict environmental control commands.

[0068] Step S73: Based on the predicted environmental control commands, use the loss function and The network calculates the estimated loss value.

[0069] Step S74: With the goal of minimizing the estimated loss value, perform... The network parameters are used for training to obtain the trained network. network.

[0070] Step S8: Continue until all data subsets have been traversed, and then process the trained data. The network was identified as an environmental control model.

[0071] Specifically, such as Figure 7As shown, the entire training process of the environmental control model includes the above steps S1 to S8, and the refinement steps are as follows: (1) Initialize a Q network as shown in Figure 5 using the method of random initialization, and set this network as , and copy the parameters of the network to construct an identical network ; Construct an experience replay pool R, and set the number of elements stored in R to . In the initial state, = 0; Let the user input the total number of sequences required for setting, the number of selected sequences is , set the maximum number of steps per time T, set the update time step C, and set the lower limit of the data volume ; (2) Make the following judgment: Whether it satisfies < . If not, save the current network , end the training, and is the network of the target intelligent control Agent; If satisfied, execute the subsequent steps; (3) Set the current time step t = 1, and randomly select a , and extract the data subset from the data set D; (4) Make a judgment: Whether t < T. If not, let increase by 1, and return to step (2); If satisfied, execute the subsequent steps; (5) Use the data at the moment as the state and input it into . Select the action with the maximum action value in the Q network output with the ε-greedy strategy. After obtaining , according to the regulation values of each device in , generate new device regulation values, including 1. The new operating state of the wet curtain water pump: , ; 2. The new opening degree state of the inlet air shutter: , ; 3. The new operating state of the fan: , ; 4. The new manure cleaning interval ; And replace the corresponding device values in the subset with the above-mentioned new device regulation values; (6) Input into , that is, the digital twin model, to obtain the predicted environmental state at the next moment, , ; (7) Use the predicted output of , the data subset middle Device values ​​in, dataset D New state variables are constructed using the number of pigs, average age of pigs, outdoor temperature, and outdoor relative humidity. (8) According to the state Calculate the reward value and will The elements consisting of the four variables are added as a whole to the experience replay pool R, and the data subset is reconstructed. (9) Determine whether the number of elements in the experience replay pool R is sufficient. > If satisfied, proceed to step (10); if not satisfied, proceed to step (14); (10) Select from R One data point, The sampling size set by the user is obtained (11) Using the formula Calculate the value of each sampled element. (12) By minimizing the target loss Backpropagation update Network parameters; (13) Determine whether t is an integer multiple of C, and if so, update The parameters, let If not, proceed directly to step (14); (14) increment t by 1 and proceed to step (4).

[0072] Through the above Network training methods can improve efficiency by appropriately setting the total number of sequences. Maximum number of steps in a single update (T), update time step (C), lower limit of data volume. Sampling quantity A convergent parameter ε is obtained using a greedy parameter. The network is used by intelligent agents. The agent can then... A network that calculates the sum of all neurons in a given state. Output the value and select the one with the largest value. The output neuron represents the environmental control command, which is the optimal control action for the current state of the pigsty.

[0073] Step S9: Obtain the current status of the pigsty.

[0074] Step S10: Input the current state of the pigsty into the environmental control model to obtain the current environmental control command, and control the pigsty environment according to the current environmental control command.

[0075] After completing steps S1 to S8 above, the environmental control model is deployed in a real pigsty. The specific environmental control process is as follows: (1) Collect important parameters at the current moment, namely the various types of data mentioned above, including: temperature, humidity, carbon dioxide concentration, ammonia concentration at each sensor point; operating status of wet curtain pump; louver opening status; fan operating status; manure cleaning interval; number of pigs in the shed; average age of pigs in the shed; outdoor temperature and humidity.

[0076] (2) Use the data collected in (1) to construct the current state of the pigsty.

[0077] (3) Input the current state of the pigsty into the deployed environmental control model. The environmental control model will generate the optimal environmental control command for the current time based on the current state of the pigsty.

[0078] (4) Adjust the operating status of the equipment in the pigsty according to the environmental control instructions at the current moment. The specific adjustment rules are described in the action design above, and obtain the best operating status of the pigsty equipment at this moment.

[0079] (5) Operate the equipment in the pigsty according to the optimal operating state of the pigsty equipment at this moment, and maintain a sampling interval. ΔH The time.

[0080] (6) At this point, a basic control process has been completed, and the time has been updated to the next moment. Return to step (1) and execute the above process again in sequence.

[0081] The beneficial effects of the pigsty environmental control method based on digital twins and reinforcement learning proposed in this application are mainly reflected in the following aspects: 1. It can avoid economic losses from agent training: By pre-training a digital twin model with the same changing patterns as a real pigsty, and then using the digital twin model to construct an agent training scheme, during the training process of the reinforcement learning model, the action at time t is generated based on the state at time t, and the reward is calculated. The state at time t+1 is generated through the digital twin model, and the elements are put into the experience replay pool. The reinforcement learning model is then trained using the elements in the experience replay pool. This reduces the work of collecting actions from the real pigsty and avoids directly using the real pigsty to train the reinforcement learning model, thus avoiding economic losses.

[0082] 2. Coordinated control capability of multiple environmental factors: By using a specific reward function calculation method, that is, the rewards of temperature, humidity, carbon dioxide concentration and ammonia concentration are considered simultaneously in this application to calculate the total reward. The reinforcement learning model can coordinately control four environmental variables, improve the accuracy of the parameters of the reinforcement learning model, thereby improving the accuracy of pig house environmental regulation and making the pig house environment more comfortable.

[0083] 3. High real-time performance: It can automatically control the equipment related to the pig house environment through computer equipment without much human intervention. It can adjust the dynamic environment in real time. Compared with manually setting equipment operating values, threshold control, and timed control, it has a stronger dynamic adaptability and improves the real-time performance of pig house environment control.

[0084] 4. High accuracy: A reinforcement learning model that can be applied to the control of the pig house environment is constructed. This reinforcement learning model can obtain the optimal operation for the next operation time in the pig house based on the input outdoor temperature and humidity, indoor temperature and humidity, ammonia concentration, carbon dioxide concentration, as well as the operation status of the fans, wet curtain pumps, and manure scrapers in the pig house, the number of pigs, and the average age of the pigs in the house, thereby improving the accuracy of the control of the pig house environment.

[0085] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores datasets. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a pig farm environmental control method based on digital twins and reinforcement learning.

[0086] Those skilled in the art will understand that Figure 8The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0087] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0088] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0089] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0090] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0091] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for controlling the pigsty environment based on digital twins and reinforcement learning, characterized in that, The pigsty environment control method based on digital twins and reinforcement learning includes: A digital twin model of the pigsty environment is constructed; the digital twin model is obtained by training an improved LSTM model using sample pigsty state sequences. A dataset is constructed, and multiple data subsets are extracted from the dataset; the dataset includes a sequence of sample pigsty states; the sequence of sample pigsty states includes: the state of the pigsty environment, the state of the equipment, the state of the pigsty environment, the number of pigs, and the average age of the pigs; For each subset of data, the state of the pigsty at the last moment in that subset is input into the reinforcement learning model. The network receives control commands for the virtual environment; The device status at the last moment in the data subset is updated based on the virtual environment control instructions to obtain the updated data subset; The updated subset of data is input into the digital twin model to obtain the updated environmental status of the pigsty. The experience replay pool of the reinforcement learning model is updated based on the updated data subset and the updated state of the pigsty environment to obtain the updated experience replay pool; the experience replay pool includes multiple elements; each element includes: virtual environment control instructions, rewards, and sample pigsty states at two adjacent time points; When the number of elements in the updated experience replay pool reaches a set number, random sampling is performed from the updated experience replay pool, and the... The network is trained to obtain the trained version. network; Until all data subsets have been traversed, the trained data will be processed. The network was identified as an environmental control model; Get the current status of the pigsty; The current state of the pigsty is input into the environmental control model to obtain the environmental control command for the current moment, and the pigsty environment is controlled according to the environmental control command for the current moment.

2. The pigsty environment control method based on digital twins and reinforcement learning according to claim 1, characterized in that, Constructing a digital twin model of the pigsty environment specifically includes: Based on the training period, the sample pigsty state sequence is randomly extracted to obtain the sample pigsty state sequence and the corresponding sample pigsty internal environment state during the training period. The sample pigsty state sequence during the training period is input into the improved LSTM model to obtain the predicted environmental state inside the pigsty. Based on the sample pigsty environmental conditions and the predicted pigsty environmental conditions, the error value is calculated using a loss function; The parameters of the improved LSTM model are iteratively optimized based on the error value until the error value is less than or equal to a preset error value, thus obtaining a digital twin model of the pigsty environment.

3. The pigsty environment control method based on digital twins and reinforcement learning according to claim 2, characterized in that, The improved LSTM model includes: a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, and an LSTM network layer connected in sequence; the first feature extraction layer, the second feature extraction layer, and the third feature extraction layer all include convolutional layers and max pooling layers.

4. The pigsty environment control method based on digital twins and reinforcement learning according to claim 1, characterized in that, Construct a dataset and extract multiple data subsets from the dataset, specifically including: Obtain the state sequence of sample pigsties and construct a dataset; the internal environmental state of the pigsties includes: temperature, humidity, carbon dioxide concentration and ammonia concentration; the equipment state includes: water pump operation status, louver fan status, blower operation status and manure cleaning interval; the external environmental state of the pigsties includes: outdoor temperature and outdoor humidity; Select multiple times as the final time; Based on the final moment, the dataset is extracted multiple times over preset time periods to obtain multiple data subsets; the preset time periods are... Time's up The time period of time; the mentioned The time is the last moment.

5. The pigsty environment control method based on digital twins and reinforcement learning according to claim 4, characterized in that, The device status at the last moment in the data subset is updated based on the virtual environment control instructions to obtain the updated data subset, specifically including: The device status is updated based on the virtual environment control commands to obtain the virtual device status; Replace the data subset with the virtual device state. The device status at any given time is used to obtain an updated subset of data.

6. The pigsty environment control method based on digital twins and reinforcement learning according to claim 5, characterized in that, The experience replay pool is updated based on the updated data subset and the updated environmental conditions within the pigsty, resulting in an updated experience replay pool, which specifically includes: Get The external environmental conditions of the pigsty at time +1 The number of pigs at time +1 and Average age of pigs at time +1; Based on the updated pigsty environment status and virtual device status The external environmental conditions of the pigsty at time +1 The number of pigs at time +1 and The average age of pigs at time +1 is obtained. The state of the sample pigsty at time +1; based on The environmental conditions inside the pigsty at any given time are calculated. Total reward at each moment; Will Real-time sample pigsty status, virtual environment control commands, Momentary rewards and The sample pigsty state at time +1 is stored as an element in the experience replay pool, resulting in an updated experience replay pool.

7. The pigsty environment control method based on digital twins and reinforcement learning according to claim 6, characterized in that, Random sampling is performed from the updated experience replay pool, and the... The network is trained to obtain the trained version. Networks, specifically including: Randomly sample elements from the updated experience replay pool to obtain multiple elements; For each element, a reinforcement learning model is used. Networks calculate and predict environmental control commands; Based on the predicted environmental control instructions, a loss function and the... The network calculates the estimated loss value; With the goal of minimizing the estimated loss value, for The network parameters are used for training to obtain the trained network. network.

8. The pigsty environment control method based on digital twins and reinforcement learning according to claim 6, characterized in that, The formula for calculating the total reward at each moment is: ； in, for Total reward at each moment; A reward for temperature; for Temperature at any moment; A reward for humidity; for Humidity at any given moment; A reward for carbon dioxide; for Carbon dioxide at any given moment; A reward for ammonia concentration; for The ammonia concentration at a given time.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the pig house environment control method based on digital twins and reinforcement learning as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pigsty environment control method based on digital twins and reinforcement learning as described in any one of claims 1-8.

Images