A plant rhizosphere irrigation control system and method based on PID structure reinforcement learning

Abstract

The application is a plant root washing control system and method based on PID structure reinforcement learning, belonging to the field of intelligent equipment and control technology. The system comprises a multi-modal sensor array, a PID intelligent controller and an actuator. The PID intelligent controller is composed of a watering reinforcement learning agent, a washing reinforcement learning agent, a grey prediction model, a physical effect predictor and an action fusioner. The washing reinforcement learning agent generates the original action as the proportional link, the grey prediction model predicts the trend to generate the correction as the differential link, the physical effect predictor evaluates the physical effect to generate the correction as the integral link, and the action fusioner weights and fuses the three into the final instruction. The method comprises system initialization, state perception, mode judgment, action generation and execution, and online learning. The application combines reinforcement learning with grey prediction and physical model, reduces data dependence, improves control smoothness and physical consistency, and realizes efficient and lossless root washing control.

Description

Technical Field

[0001] This invention relates to a root flushing control system and method for plant boxes based on PID-like structural reinforcement learning, and belongs to the field of intelligent equipment and control technology, and is particularly applicable to a root flushing control system for plant boxes based on PID-like structural reinforcement learning. Background Technology

[0002] Plant roots are vital organs for absorbing water and nutrients, and their morphological characteristics directly influence plant growth and ecological adaptability. In plant ecology, soil science, and agricultural science research, accurate observation and analysis of plant root morphology are crucial for understanding plant physiological and ecological characteristics, interspecific relationships, and plant-soil feedback mechanisms. Efficient and non-destructive root flushing techniques not only provide researchers with complete root samples but also offer technical support for root phenotypic analysis in precision agriculture, thus playing a significant role in advancing plant science research and agricultural technology.

[0003] Currently, various automated root rinsing devices have emerged on the market, such as rinsing devices based on high-pressure water guns and cleaning devices utilizing the eddy current principle. These devices have improved rinsing efficiency to some extent, but still have prominent problems such as high risk of root damage and large water consumption. Invention patent (CN120036160A) proposes "a plant experimental planting box for root rinsing and its control method," which improves experimental efficiency to some extent by setting nozzles and rinsing holes on the box and using pressure and humidity sensors to achieve automatic watering and root rinsing; however, the control method of this scheme mainly adopts simple threshold control and fuzzy control, still requiring human assistance to control and adjust the rinsing process. In recent years, with the development of artificial intelligence technology, reinforcement learning methods have begun to be applied to the field of root rinsing control. Existing reinforcement learning control methods mainly achieve automated control of the rinsing process through closed-loop feedback between sensor data and actuators. However, these methods have significant limitations: First, traditional reinforcement learning algorithms heavily rely on large amounts of experimental data for training, leading to high data acquisition costs and long training cycles in practical applications. Second, purely data-driven methods ignore the constraints of domain knowledge and physical laws, resulting in insufficient model generalization ability and requiring the collection of large amounts of new data for training when the environment changes. Third, existing machine learning methods have limited ability to model physical processes such as fluid dynamics, making it difficult to ensure the accuracy and reliability of control with limited data. Furthermore, traditional reinforcement learning is prone to producing actions that do not conform to physical laws during the exploration process, which not only affects system stability but also increases the risk of root damage.

[0004] In summary, existing root flushing technologies, especially in the application of machine learning, still suffer from problems such as excessive reliance on data, insufficient utilization of physical laws, poor smoothness of actions, and difficulty in multi-objective optimization. Therefore, there is an urgent need for an intelligent control system that can reduce the dependence on large amounts of training data, combine data-driven learning with physical model constraints, and perform temporal smoothing and forward-looking optimization of control actions to achieve efficient, precise, and non-destructive control of the plant root flushing process. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a plant box root flushing control system and method based on PID-like reinforcement learning. This system combines a grey prediction model, a physical effect predictor, and a reinforcement learning framework to construct a novel intelligent controller with a proportional-integral-derivative (PID) structure, reducing the data dependence of machine learning model training and achieving refined, adaptive, and forward-looking optimal control of the flushing process.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A plant box root flushing control system based on PID-like reinforcement learning is characterized by comprising a multimodal sensor array, a PID-like intelligent controller, and an actuator; wherein the PID-like intelligent controller is further composed of a watering reinforcement learning agent, a flushing reinforcement learning agent, a grey prediction model, a physical effect predictor, and an action fusion unit.

[0008] The multimodal sensor array is deployed on the plant box to collect environmental status data in real time, including but not limited to: pressure sensor, humidity sensor, camera and flow meter; its output is respectively connected to the watering reinforcement learning agent, rinsing reinforcement learning agent and physical effect predictor in the simulated PID intelligent controller.

[0009] Preferably, the plant box is a plant experimental planting box for root rinsing proposed in the invention patent (CN120036160A).

[0010] The aforementioned PID-like intelligent controller is deployed on a processor, with its input end connected to a multimodal sensor array and its output end connected to an actuator, used to generate precise control commands based on environmental conditions.

[0011] The watering reinforcement learning agent is a DDPG (Deep Deterministic Policy Gradient) reinforcement learning network. Its input is connected to a multimodal sensor array, and its output is directly connected to an actuator. This agent is specifically designed for daily watering, outputting watering action commands based on soil moisture sensor data.

[0012] The aforementioned flushing reinforcement learning agent is a DDPG reinforcement learning network, with the same network structure as the watering reinforcement learning agent, but it is trained and deployed independently. Its input is connected to a multimodal sensor array, and its output is connected to a gray prediction model, a physical effect predictor, and an action fusion unit, respectively. As the "proportional (P) element" of the controller in the root flushing mode, it generates immediate initial control actions based on the deviation between the current system state and the target cleanliness.

[0013] The gray prediction model is a GM(1,1) gray model, whose input is connected to the flushing reinforcement learning agent and whose output is connected to the action fusion unit. It acts as the "differential (D) link" of the controller, predicting future trends based on historical action sequences and generating differential correction quantities for smooth control.

[0014] The physical effect predictor is a lightweight feedforward neural network. Its inputs are connected to a multimodal sensor array and a flushing reinforcement learning agent, respectively, and its output is connected to an action fusion unit. As the "integral (I) link" of the controller, it learns the mapping from "state-action" to "physical effect index" to quickly evaluate the cumulative physical effects of historical actions and generate an integral correction quantity to ensure the consistency of long-term physical laws.

[0015] The action fusion unit is a fully connected network. Its input is connected to the flushing reinforcement learning agent, the gray prediction model, and the physical effect predictor, respectively, and its output is connected to the actuator. It is responsible for intelligently weighting and fusing the outputs of the three links of P, I, and D to generate the final control command in the root flushing mode.

[0016] The actuator is a controllable pressure and flow water pump, connected to the processor and driven by processor instructions.

[0017] Furthermore, the pressure sensor is a MEMS piezoresistive pressure sensor, embedded in the four walls of the plant box in a grid array; the humidity sensor is a capacitive humidity sensor, arranged in a grid in the soil layer in the middle of the plant box; the flow meter is a turbine flow sensor installed at the water supply pipe and the inlet of the T-junction branch; the camera is an industrial-grade CMOS camera, installed on the outside of the box, equipped with an adjustable pan-tilt head; all sensors are connected to the processor via an RS485 bus.

[0018] Furthermore, the system has two working modes: daily watering mode and root flushing mode;

[0019] In the daily watering mode, the system uses soil moisture sensor data to directly output control commands to the actuator to complete the watering process.

[0020] In root flushing mode, the system employs a complete pseudo-PID control loop consisting of the flushing reinforcement learning agent, gray prediction model, physical effect predictor, and action fusion unit to achieve precise and non-destructive control of the root flushing process.

[0021] A root flushing control method for plant boxes based on PID-like reinforcement learning is characterized by the following steps:

[0022] Step S1: System initialization, establishing and pre-training a simulated PID intelligent controller;

[0023] Step S2: Use the multimodal sensor array to sense and construct the environmental state vector in real time;

[0024] Step S3: Determine the current task mode: If it is the daily watering mode, proceed to step S3-A; if it is the root flushing mode, proceed to step S3-B.

[0025] Step S3-A: The watering reinforcement learning agent directly outputs the watering action based on the environmental state vector, and then proceeds to step S4;

[0026] Step S3-B: Based on the state vector, use the PID-like intelligent controller to generate flushing actions using the reinforcement learning agent, gray prediction model, physical effect predictor, and action fusion unit, and then proceed to step S4.

[0027] Step S4: The plant box actuator performs a watering or rinsing action;

[0028] Step S5: Perform online learning and parameter updates on the model in the simulated PID intelligent controller;

[0029] Step S6: Repeat steps S2 to S5 until the task termination condition is met.

[0030] The input to the simulated PID intelligent controller is the environmental state vector S. t The output is the flushing action A. t The environmental state vector S t Includes, but is not limited to, sensor readings, image features, and system status; wherein, sensor readings are pressure sensor data P directly read from the sensor. t Humidity sensor data H t Flow meter reading F t The image feature is the root density D extracted using image segmentation methods. t Root length L t Branching complexity B t Soil coverage C tThe root density is the ratio of root pixels per unit area; the root length is the total length of the skeleton calculated after refining the root system to a single pixel width; the branch complexity is the ratio of pixels occupied by endpoints and intersections; the soil coverage is the ratio of pixels occupied by soil; the system state is the flushing time T. t Cumulative water consumption W t The aforementioned rinsing action A t This refers to the pressure and flow rate of the water pump.

[0031] Furthermore, step S1 specifically includes:

[0032] S101: Start the system and initialize the sensors, actuators, and processor in the multimodal sensor array.

[0033] S102: The processor loads a pre-trained physical effect predictor; wherein the physical effect predictor is trained offline using computational fluid dynamics (CFD) simulation data or historical experimental data;

[0034] S103: The processor loads the pre-trained watering reinforcement learning agent and the rinsing reinforcement learning agent; wherein the watering reinforcement learning agent and the rinsing reinforcement learning agent are trained offline using historical experimental data.

[0035] Furthermore, step S2 specifically includes:

[0036] S201: Multimodal sensor array acquires sensor data in real time;

[0037] S202: The processor preprocesses the sensor data from the multimodal sensor array and constructs the environmental state vector S. t =[P t H t F t D t L t B t C t T t W t ].

[0038] Furthermore, the step S3-B of generating the rinsing action using a PID-like reinforcement learning model specifically includes the following sub-steps:

[0039] S301 (P-stage - Instant Decision): The reinforcement learning agent is flushed based on the current environmental state vector S. t This generates a preliminary, primitive control action. This action represents a proportional, immediate response to the deviation from the current state.

[0040] S302 (D-stage - Trend Prediction): The grey prediction model reads the final action sequence from the most recent m historical moments {A} t-m A t-1 Establish the GM(1,1) grey differential equation, predict the trend of the action at the next moment, and generate the differential correction ΔA. t This correction is used to smooth the control process and suppress overshoot.

[0041] S303 (I-stage - Physical Integral Correction): The physical effect predictor uses the current state S t and the original actions output by the reinforcement learning agent Using this as input, the shear stress on the root surface is calculated via forward propagation. Soil stripping efficiency and water flow utilization rate Three physical index scalars are used, and the deviations of the physical indexes from the preset ideal target values ​​are linearly weighted to generate an integral correction quantity:

[0042]

[0043] Where α, β, and γ are preset weighting coefficients, which are adjusted according to plant type and experimental objectives; τ target A target shear stress threshold is set artificially; η target To set a target for efficiency stripping; ξ target Target water flow utilization rate is set for humans.

[0044] S304 (Motion Blending): The motion blender will... ΔA t , and I t The final execution action is generated through a learnable, weighted, fully connected network by fusion. Linear(·) is a fully connected network operation.

[0045] Furthermore, step S4 specifically includes:

[0046] S401: The processor will finally execute action A. t It is converted into specific control instructions and sent to the actuator.

[0047] S402: Drive the actuator to perform the rinsing operation and rinse the plant roots.

[0048] S403: After the actuator performs the action, the multimodal sensor array acquires a new state vector S. t+1 And calculate the reward R based on the change in root cleanliness. t .

[0049] Furthermore, step S5 specifically includes:

[0050] S501: Transfer the empirical tuple (S t A t R t S t+1 Store in the experience replay buffer;

[0051] S502: Periodically sample data from the buffer to update the policy network and value network of the flushing reinforcement learning agent.

[0052] Preferably, in step S5, the physical effect predictor is fine-tuned using newly collected real system data to narrow the gap between its simulation and the real world, achieving continuous adaptation from simulation to reality.

[0053] Furthermore, the termination condition for the rinsing task is that all pressure sensor readings are below the threshold, or the maximum rinsing time is reached, or the vision system determines that the roots have been cleaned; the termination condition for the watering task is that the soil moisture reaches a predetermined target value.

[0054] An electronic device includes at least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0055] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the above-described PID-like reinforcement learning-based root flushing control method for plant boxes.

[0056] Finally, the present invention also discloses a computer-readable storage medium storing computer instructions for causing a processor to execute the above-described PID-like reinforcement learning-based root flushing control method for plant boxes.

[0057] The beneficial effects of this invention are as follows: It provides a plant box root flushing control system and method based on PID-like reinforcement learning. Based on a PID-like intelligent decision fusion mechanism, it organically combines the adaptive decision-making ability of reinforcement learning, the trend prediction ability of grey prediction, and the physical law constraint ability of physical effect predictors. This method not only significantly reduces the dependence on large-scale training data and improves the model's generalization ability, but also effectively suppresses oscillations in the control process by introducing "differential" and "integral" correction terms, ensuring the smoothness and physical consistency of the action. Thus, while achieving efficient flushing, it minimizes the risk of root damage and water consumption, providing a novel and superior solution for the intelligent control of plant root flushing. Attached Figure Description

[0058] To make the objectives and technical solutions of this invention clearer, the following figures are provided for illustration:

[0059] Figure 1 This is an architecture diagram of a plant box root flushing control system based on PID-like reinforcement learning in this invention.

[0060] Figure 2 This is a flowchart of a plant box root flushing control method based on PID-like structural reinforcement learning in this invention.

[0061] Figure 3 This is a schematic diagram of a plant box using a PID-like structural reinforcement learning approach for watering tasks in Embodiment 3 of the present invention; wherein, 401 is a nozzle; 402 is the planting box body; 403 is a pressure sensor array; 404 is a humidity sensor array; 405 is a vision sensor; 408 is a rinsing hook; 409 is a base; 410 is a caster wheel; 4021 is the box body; and 4022 is a rinsing hole array.

[0062] Figure 4 This is a schematic diagram of a plant box using a PID-like structural reinforcement learning approach for a rinsing task in Embodiment 3 of the present invention; wherein, 401 is a nozzle; 402 is the planting box body; 403 is a pressure sensor array; 404 is a humidity sensor array; 405 is a vision sensor; 406 is a coarse filter; and 407 is a fine filter.

[0063] Figure 5 This is a schematic diagram of the electronic device in Embodiment 4 of the present invention. Detailed Implementation

[0064] To make the objectives and technical solutions of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0065] Example 1: For standardized experimental planting boxes (dimensions: 500mm long × 300mm wide × 400mm high), in order to reduce labor costs and improve efficiency and accuracy, a device capable of automatic watering and rinsing is needed. This invention provides "a plant box root rinsing control system based on PID-like reinforcement learning".

[0066] A root flushing control system for plant boxes based on PID-like reinforcement learning, combined with Figure 1 The device is characterized by comprising a multimodal sensor array 1, a simulated PID intelligent controller 2, and an actuator 3; wherein the simulated PID intelligent controller 2 is further composed of a watering reinforcement learning agent 21, a rinsing reinforcement learning agent 22, a gray prediction model 23, a physical effect predictor 24, and an action fusion unit 25.

[0067] The multimodal sensor array 1 is deployed on the plant box to collect environmental status data in real time, including but not limited to: pressure sensor, humidity sensor, camera and flow meter; its output is respectively connected to the watering reinforcement learning agent 21, the rinsing reinforcement learning agent 22 and the physical effect predictor 24 in the simulated PID intelligent controller 2.

[0068] The simulated PID intelligent controller 2 is deployed on the processor. Its input is connected to the multimodal sensor array 1, and its output is connected to the actuator 3. It is used to generate precise control commands according to the environmental conditions.

[0069] The watering reinforcement learning agent 21 is a DDPG reinforcement learning network. Its input is connected to the multimodal sensor array 1, and its output is directly connected to the actuator 3. This agent is specifically designed for daily watering mode and outputs watering action commands based on soil moisture sensor data.

[0070] The flushing reinforcement learning agent 22 is a DDPG reinforcement learning network, with the same network structure as the watering reinforcement learning agent 21, but it is trained and deployed independently. Its input is connected to the multimodal sensor array 1, and its output is connected to the gray prediction model 23, the physical effect predictor 24, and the action fusion unit 25, respectively. As the "proportional (P) element" of the controller in the root flushing mode, it generates the original control action with an immediate response based on the deviation between the current system state and the target cleanliness.

[0071] The gray prediction model 23 is a GM(1,1) gray model, with its input connected to the flushing reinforcement learning agent 22 and its output connected to the action fusion unit 25. It acts as the "differential (D) link" of the controller, predicting future trends based on historical action sequences to generate differential correction quantities for smooth control.

[0072] The physical effect predictor 24 is a lightweight feedforward neural network. Its input is connected to the multimodal sensor array 1 and the flushing reinforcement learning agent 22, respectively, and its output is connected to the action fusion unit 25. As the "integral (I) link" of the controller, it learns the mapping from "state-action" to "physical effect index" to quickly evaluate the cumulative physical effect of historical actions and generate an integral correction quantity to ensure the consistency of long-term physical laws.

[0073] The action fusion unit 25 is a fully connected network. Its input is connected to the flushing reinforcement learning agent 22, the gray prediction model 23 and the physical effect predictor 24 respectively, and its output is connected to the actuator 3. It is responsible for intelligently weighting and fusing the outputs of the three links of P, I and D to generate the final control command in the root flushing mode.

[0074] The actuator 3 is a controllable pressure and flow water pump, which is connected to the processor and driven by the processor's instructions.

[0075] Furthermore, the pressure sensor is a MEMS piezoresistive pressure sensor, embedded in the four walls of the plant box in a grid array; the humidity sensor is a capacitive humidity sensor, arranged in a grid in the soil layer in the middle of the plant box; the flow meter is a turbine flow sensor installed at the water supply pipe and the inlet of the T-junction branch; the camera is an industrial-grade CMOS camera, installed on the outside of the box, equipped with an adjustable pan-tilt head; all sensors are connected to the processor via an RS485 bus.

[0076] Furthermore, the system has two working modes: daily watering mode and root flushing mode;

[0077] In the daily watering mode, the system uses soil moisture sensor data to directly output control commands from the watering reinforcement learning agent 21 to the actuator 3 to complete the watering.

[0078] In the root flushing mode, the system activates a complete pseudo-PID control loop consisting of the flushing reinforcement learning agent 22, the gray prediction model 23, the physical effect predictor 24, and the action fusion unit 25 to achieve precise and non-destructive control of the root flushing process.

[0079] Example 2: For experimental planting boxes designed and manufactured in a standardized manner, in order to reduce labor costs and improve efficiency and accuracy, a control method that can automatically water and rinse is needed to be applied to the device described in Example 1. The present invention provides "a plant box root rinsing control method based on PID-like reinforcement learning".

[0080] A root flushing control method for plant boxes based on PID-like reinforcement learning, combined with Figure 2 Its characteristic is that it includes the following steps:

[0081] Step S1: System initialization, establishing and pre-training a simulated PID intelligent controller 2. Specifically,

[0082] S101: Start the system and initialize the sensors, actuators 3 and processor in the multimodal sensor array 1.

[0083] S102: The processor loads the pre-trained physical effect predictor 24; wherein the physical effect predictor 24 is trained offline using computational fluid dynamics (CFD) simulation data or historical experimental data. Specifically,

[0084] Based on the planting box dimensions (500mm long × 300mm wide × 400mm high) of Example 1, a geometric model was constructed in CFD software. By changing the root density parameters (sparse: density < 0.1, medium: 0.1 ≤ density ≤ 0.3, dense: density > 0.3) and soil type parameters (sand, loam, clay), 100 different geometric configurations were generated.

[0085] Secondly, the computational fluid dynamics (CFD) software OpenFOAM was used to simulate different operating conditions. Operating parameters included: pump pressure: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8 MPa; pump flow rate: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 L / min; flushing hole opening combinations: simulating different strategies such as single hole opening, simultaneous opening of multiple holes, and sequential opening from bottom to top; root density: sparse, medium, dense; soil type: sandy soil, loam, clay. A total of 8000 simulation conditions were performed: 100 geometric configurations × 8 × 10. Each simulation lasted 10 seconds, recording complete flow field data.

[0086] After the simulation is completed, post-processing is used to calculate the physical indices for each working condition: root surface shear stress, soil stripping efficiency, and water utilization rate. The input parameters (environmental state vector + action) and the output physical indices for each working condition are then combined to form training sample pairs to construct the dataset. 80% is randomly selected as the training set, and 20% as the test set.

[0087] The physical effects predictor 24 uses a three-layer fully connected network (input layer 128 → hidden layer 64 → output layer 3), trained for 200 epochs using the Adam optimizer (learning rate 0.001) and a batch size of 64.

[0088] S103: The processor loads the pre-trained watering reinforcement learning agent 21 and rinsing reinforcement learning agent 22; wherein the watering reinforcement learning agent 21 and rinsing reinforcement learning agent 22 are trained offline using historical experimental data.

[0089] The input of the simulated PID intelligent controller 2 is the environmental state vector S. t The output is the flushing action A. t The environmental state vector S t This includes sensor readings, image features, and system status; where the sensor readings are the pressure sensor data P directly read from the sensor. t Humidity sensor data H t Flow meter reading F t The image feature is the root density D extracted using image segmentation methods. t Root length L t Branching complexity Bt Soil coverage C t The root density is the ratio of root pixels per unit area; the root length is the total length of the skeleton calculated after refining the root system to a single pixel width; the branch complexity is the ratio of pixels occupied by endpoints and intersections; the soil coverage is the ratio of pixels occupied by soil; the system state is the flushing time T. t Cumulative water consumption W t The aforementioned rinsing action A t This refers to the pressure and flow rate of the water pump.

[0090] Step S2: Real-time data sensing and construction of an environmental state vector using the multimodal sensor array 1. Specifically,

[0091] S201: Multimodal sensor array 1 acquires sensor data in real time;

[0092] S202: The processor preprocesses the sensor data from the multimodal sensor array 1 and constructs the environmental state vector S. t =[P t H t F t D t L t B t C t T t W t ].

[0093] Step S3: Determine the current task mode: If it is the daily watering mode, proceed to step S3-A; if it is the root flushing mode, proceed to step S3-B.

[0094] Step S3-A: The watering reinforcement learning agent 21 directly outputs the watering action based on the environmental state vector, and then proceeds to step S4. Specifically,

[0095] The watering reinforcement learning agent 21 reads the current environment state vector and propagates it forward through the policy network of DDPG to generate watering action instructions. The processor converts the watering action instructions into control signals and sends them to the actuator 3 to complete the watering. After watering, the multimodal sensor array 1 collects new environment state vectors and calculates reward values ​​based on changes in soil moisture, storing them in the experience playback buffer for subsequent model updates.

[0096] Step S3-B: Based on the state vector, generate the flushing action using the flushing reinforcement learning agent 22, gray prediction model 23, physical effect predictor 24, and action fusion unit 25 in the simulated PID intelligent controller 2, and then proceed to step S4. Specifically, this includes the following sub-steps:

[0097] S301 (P-stage - Instant Decision): The reinforcement learning agent 22 is flushed based on the current environmental state vector S. t This generates a preliminary, primitive control action. This action represents a proportional, immediate response to the deviation from the current state.

[0098] The aforementioned flushing reinforcement learning agent 22 employs a DDPG deep deterministic policy gradient network, which consists of two parts: a policy network (Actor) and a value network (Critic).

[0099] Policy Network (Actor): Maps state vectors to deterministic actions. The network structure is: input layer (dimension equal to the dimension of the environment state vector) → hidden layer 1 (400 neurons, ReLU activation) → hidden layer 2 (300 neurons, ReLU activation) → output layer (dimension equal to the dimension of the action vector, Tanh activation). The output range is normalized to [-1, 1] and then linearly mapped to the actual physical range.

[0100] The Critic network evaluates the value of state-action pairs. The network structure is as follows: input layer (environment state vector dimension + action dimension) → hidden layer 1 (400 neurons, ReLU activation) → hidden layer 2 (300 neurons, ReLU activation) → output layer (1 neuron, linear activation). The output Q-value represents the expected cumulative reward.

[0101] Specifically, the policy network μ of DDPG in reinforcement learning agent 22 is flushed. θ (S t ) The state vector S t Mapping to the action space:

[0102]

[0103] Where W1 and W2 are weights and b1 and b2 are biases, the tanh activation function restricts the action to the range of [-1, 1], and then maps it to the actual pressure and flow range through a linear transformation.

[0104] S302 (D-stage - Trend Prediction): Grey prediction model 23 reads the final action sequence of the most recent m historical moments {A t-m A t-1 Establish the GM(1,1) grey differential equation, predict the trend of the action at the next moment, and generate the differential correction ΔA. t This correction is used to smooth the control process and suppress overshoot.

[0105] Specifically, the process of establishing the GM(1,1) model is as follows:

[0106] S3021: For the original sequence A(0) (t)=A t-m A t-m+1 ,…,A t-1 Generate A by performing an accumulation once (1) (t);

[0107] S3022: Establish the grey differential equation:

[0108] S3023: Estimate parameters a and b using the least squares method;

[0109] S3024: Solve the differential equation to obtain the predicted value A (1) (t), then subtract and restore to get (t),

[0110] S3025: Calculate the differential correction:

[0111] S303 (I-stage - Physical Integral Correction): Physical effect predictor 24 uses the current state S t and the original actions output by the reinforcement learning agent Using this as input, the shear stress on the root surface is calculated via forward propagation. Soil stripping efficiency and water flow utilization rate Three physical index scalars are used, and the deviations of the physical indexes from the preset ideal target values ​​are linearly weighted to generate an integral correction quantity:

[0112]

[0113] Where α, β, and γ are preset weighting coefficients, which are adjusted according to plant type and experimental objectives; τ target A target shear stress threshold is set artificially; η target To set a target for efficiency stripping; ξ target Target water flow utilization rate is set for humans.

[0114] Integral correction I t The physical meaning lies in the fact that, in order to ensure that the current action conforms to physical laws and achieves the optimization goal, the direction and magnitude of the original action need to be adjusted. When I t A positive value indicates that the rinsing intensity needs to be increased; a negative value indicates that the rinsing intensity needs to be reduced; and a value close to zero indicates that the original action is close to optimal.

[0115] S304 (Motion Blending): Motion Blender 25 will ΔA t , and I t The final execution action is generated through a learnable, weighted, fully connected network by fusion. Linear(·) is a fully connected network operation.

[0116] Step S4: The plant box actuator 3 performs a rinsing action. Specifically,

[0117] S401: The processor will finally execute action A. t It is converted into specific control instructions and sent to actuator 3.

[0118] S402: Drive actuator 3 to perform rinsing work and rinse the plant roots.

[0119] S403: After the actuator 3 performs the action, the multimodal sensor array 1 acquires a new state vector S. t+1 And calculate the reward R based on the change in root cleanliness. t .

[0120] The reward function R t =Φ(S t+1 )-Φ(S t The design employs a potential energy reward mechanism to ensure policy invariance, where the potential energy function... W max The maximum allowable water consumption is represented by k1, k2, and k3, which are weighting coefficients.

[0121] Step S5: Perform online learning and parameter updates on the model in the simulated PID intelligent controller 2. Specifically,

[0122] S501: Transfer the empirical tuple (S t A t R t S t+1 Store in the experience replay buffer;

[0123] S502: Periodically sample data from the buffer to update the policy network and value network of the flushing reinforcement learning agent 22. The update rules for DDPG are as follows: the value network is updated by minimizing the TD error, and the policy network is updated by deterministic policy gradients.

[0124] Preferably, in step S5, the physical effect predictor 24 is fine-tuned using newly collected real system data to narrow the gap between its simulation and the real world, achieving continuous adaptation from simulation to reality; the fine-tuning uses the mean squared error loss function as follows: The superscript (i) corresponds to the i-th time point, and N is the maximum time period.

[0125] Step S6: Repeat steps S2 to S5 until the task termination condition is met. The termination condition for the flushing task is that all pressure sensor readings are below the threshold of 0.01 MPa, or the maximum flushing time of 300 seconds is reached, or the vision system determines that the root system has been cleaned. t <0.02; The termination condition for the watering task is that the soil moisture reaches the predetermined target value.

[0126] Example 3: Applied to Examples 1 and 2, this invention observes the root morphology of a plant growing in culture soil with different compositions. To reduce labor costs and improve efficiency and accuracy, a device capable of automatic watering and rinsing is needed. This invention provides "a plant box with PID-like structural reinforcement learning".

[0127] Combination Figure 3 A plant experimental planting box for root rinsing is composed of a nozzle 401, a planting box body 402, a pressure sensor array 403, a humidity sensor array 404, a vision sensor 405, a coarse filter 406, a fine filter 407, a rinsing hook 408, a base 409, a moving wheel 410, and a water pipe 411; the planting box body 402 is composed of a box body (4021) and a rinsing hole array (4022).

[0128] There are two nozzles 401, which are installed on both sides of the top of the box body (4021) and are supplied with water through water pipes 411 connected to them.

[0129] The housing 4021 has six flushing holes 4022 arranged vertically at equal intervals, each flushing hole 4022 being individually and sealed to a water pipe 411. The vertical distribution of the flushing holes allows the system to perform targeted flushing of soil at different depths.

[0130] The pressure sensor array 403 consists of six MEMS piezoresistive pressure sensors, which are installed near each flushing hole 4022 and at the bottom of the housing, respectively, for real-time monitoring of soil pressure at different heights. The output of the pressure sensor array 403 is connected to the PID-like intelligent controller 2 via an RS485 bus.

[0131] The humidity sensor array 404 consists of three capacitive humidity sensors, evenly distributed in the soil layer in the middle of the housing, for monitoring soil moisture. The output of the humidity sensor array 404 is connected to the simulated PID intelligent controller 2 via an RS485 bus.

[0132] The aforementioned visual sensor 405 uses an industrial-grade CMOS camera, which is installed on the outside of the housing body 4021 and equipped with an adjustable gimbal. It is connected to the simulated PID intelligent controller 2 via an RS485 bus to acquire root system images in real time, providing data support for root system morphology feature extraction and cleanliness determination.

[0133] When watering and planting, the base 409 fits into the bottom of the box body 4021 and can be detached and pulled out; four rinsing hooks 408 are installed on the bottom of the box body 4021; four casters 410 are installed on the four corners of the base 409, which can be used for the free movement of the planting box.

[0134] Combination Figure 4 When rinsing the roots, remove the base 409 and suspend the coarse filter 406 and fine filter 407 at the bottom of the housing 4021 via the rinsing hook 408. These are used to collect plant roots and prevent root loss during rinsing. The coarse filter 406 has a 2mm pore size to intercept large soil particles; the fine filter 407 has a 0.5mm pore size to intercept fine soil particles. The filtered water returns to the water pump inlet through the return pipe, achieving water recycling.

[0135] The actuator 3 is a controllable pressure (0-0.8MPa) and flow (0-10L / min) water pump, which is connected to the water pipe 411 and the simulated PID intelligent controller 2 respectively. Driven by the processor instructions, it supplies water to the nozzle 401 and the flushing hole array 4022 through the water pipe 411.

[0136] Example 4: For the scenario in Example 1, Figure 5 A schematic diagram of an electronic device 90 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0137] Electronic devices can also refer to various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0138] like Figure 5As shown, the electronic device 90 includes at least one processor 91 and a memory, such as a read-only memory (ROM) 92 or a random access memory (RAM) 93, communicatively connected to the at least one processor 91. The memory stores computer programs executable by the at least one processor. The processor 91 can perform various appropriate actions and processes based on the computer program stored in the ROM 92 or loaded into the RAM 93 from storage unit 98. The RAM 93 can also store various programs and data required for the operation of the electronic device 90. The processor 91, ROM 92, and RAM 93 are interconnected via a bus 94. An input / output (I / O) interface 95 is also connected to the bus 94.

[0139] Multiple components in electronic device 90 are connected to I / O interface 95, including: input unit 96, such as keyboard, mouse, etc.; output unit 97, such as various types of displays, speakers, etc.; storage unit 98, such as disk, optical disk, etc.; and communication unit 99, such as network card, modem, wireless transceiver, etc. Communication unit 99 allows electronic device 90 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0140] Processor 91 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 91 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 91 performs the various methods and processes described above, such as a PID-like structural reinforcement learning-based root flushing control method for plant boxes.

[0141] In some embodiments, a plant box root flushing control method based on PID-like structural reinforcement learning can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 98. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 90 via ROM 92 and / or communication unit 99. When the computer program is loaded into RAM 93 and executed by processor 91, one or more steps of the plant box root flushing control method based on PID-like structural reinforcement learning described above can be performed. Alternatively, in other embodiments, processor 91 can be configured to perform a plant box root flushing control method based on PID-like structural reinforcement learning by any other suitable means (e.g., by means of firmware).

[0142] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0144] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0146] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0147] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0148] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A root flushing control system for plant boxes based on PID-like reinforcement learning, characterized in that, It consists of a multimodal sensor array (1), a simulated PID intelligent controller (2), and an actuator (3); The simulated PID intelligent controller (2) is deployed on the processor. Its input end is connected to the multimodal sensor array (1), and its output end is connected to the actuator (3). It consists of a watering reinforcement learning agent (21), a flushing reinforcement learning agent (22), a gray prediction model (23), a physical effect predictor (24), and an action fusion unit (25), which are used to generate precise control commands according to the environmental conditions. The multimodal sensor array (1) is deployed on the plant box to collect environmental status data in real time, including but not limited to: pressure sensor, humidity sensor, camera and flow meter; Its output terminals are respectively connected to the watering reinforcement learning agent (21), the flushing reinforcement learning agent (22) and the physical effect predictor (24) in the simulated PID intelligent controller (2); The watering reinforcement learning agent (21) has its input end connected to the multimodal sensor array (1) and its output end directly connected to the actuator (3); it is used for daily watering mode and outputs watering action instructions based on soil moisture sensor data. The input end of the flushing reinforcement learning agent (22) is connected to the multimodal sensor array (1), and the output end is connected to the gray prediction model (23), the physical effect predictor (24) and the action fusion unit (25) respectively; based on the deviation between the current system state and the target cleanliness, the original control action with immediate response is generated; The gray prediction model (23) is connected to the flushing reinforcement learning agent (22) at its input end and connected to the action fusion unit (25) at its output end; it predicts future trends based on historical action sequences and generates differential correction quantities for smooth control. The input of the physical effect predictor (24) is connected to the multimodal sensor array (1) and the flushing reinforcement learning agent (22) respectively, and the output is connected to the action fusion unit (25). By learning the mapping from "state-action" to "physical effect index", the cumulative physical effect of historical actions is quickly evaluated, and an integral correction amount is generated to ensure the consistency of long-term physical laws. The action fusion unit (25) is a fully connected network. Its input is connected to the flushing reinforcement learning agent (22), the gray prediction model (23), and the physical effect predictor (24), respectively, and its output is connected to the actuator (3). It is responsible for weighted fusion of the outputs of the flushing reinforcement learning agent (22), the gray prediction model (23), and the physical effect predictor (24) to generate the final control command in the root flushing mode. The actuator (3) is a controllable pressure and flow water pump, which is connected to the processor and driven by the processor's instructions.

2. The plant box root flushing control system based on PID-like reinforcement learning as described in claim 1, characterized in that, The pressure sensor is a MEMS piezoresistive pressure sensor, embedded in the four walls of the plant box in a grid array; the humidity sensor is a capacitive humidity sensor, arranged in a grid in the soil layer in the middle of the plant box; the flow meter is a turbine flow sensor installed at the water supply pipe and the inlet of the T-junction branch; the camera is an industrial-grade CMOS camera, installed on the outside of the box, equipped with an adjustable pan-tilt head; all sensors are connected to the processor via an RS485 bus.

3. The plant box root flushing control system based on PID-like reinforcement learning as described in claim 1, characterized in that, The watering reinforcement learning agent (21) and the rinsing reinforcement learning agent (22) are both DDPG (Deep Deterministic Policy Gradient) networks, which are trained and deployed independently.

4. The plant box root flushing control system based on PID-like reinforcement learning as described in claim 1, characterized in that, The gray prediction model (23) is the GM(1,1) gray model.

5. A root flushing control system for a plant box based on PID-like reinforcement learning as described in claim 1, characterized in that, The physical effect predictor (24) is a feedforward neural network.

6. A root flushing control system for a plant box based on PID-like reinforcement learning as described in claim 1, characterized in that, The motion fusion unit (25) is a fully connected network.

7. A root flushing control method for a plant box using a simulated PID structured reinforcement learning approach, applied to the root flushing control system for a plant box according to claims 1-6, characterized in that... Includes the following steps: Step S1: System initialization, establishing a simulated PID intelligent controller (2) and pre-training; Step S2: Use the multimodal sensor array (1) to sense and construct the environmental state vector in real time; Step S3: Determine the current task mode: If it is the daily watering mode, proceed to step S3-A; if it is the root flushing mode, proceed to step S3-B. Step S3-A: The watering reinforcement learning agent (21) directly outputs the watering action based on the environmental state vector and proceeds to step S4; Step S3-B: Based on the state vector, generate the flushing action using the simulated PID intelligent controller (2), and proceed to step S4; Step S4: The plant box actuator (3) performs a watering or rinsing action; Step S5: Perform online learning and parameter updates on the model in the simulated PID intelligent controller (2); Step S6: Repeat steps S2 to S5 until the task termination condition is met; The input of the simulated PID intelligent controller (2) is the environmental state vector S. t The output is the flushing action A. t The environmental state vector S t Includes, but is not limited to, sensor readings, image features, and system status; wherein, sensor readings are pressure sensor data P directly read from the sensor. t Humidity sensor data H t Flow meter reading F t The image feature is the root density D extracted using image segmentation methods. t Root length L t Branching complexity B t Soil coverage C t The root density is the ratio of root pixels per unit area; the root length is the total length of the skeleton calculated after refining the root system to a single pixel width; the branch complexity is the ratio of pixels occupied by endpoints and intersections; the soil coverage is the ratio of pixels occupied by soil; the system state is the flushing time T. t Cumulative water consumption W t The aforementioned rinsing action A t The pressure and flow rate of the water pump; The rinsing task is terminated when all pressure sensor readings are below a threshold, or when the maximum rinsing time is reached, or when the root system is determined to be clean; the watering task is terminated when the soil moisture reaches a predetermined target value.

8. The method for controlling root flushing in a plant box using PID-like reinforcement learning as described in claim 7, characterized in that, The specific steps of S1 are as follows: S101: Start the system and initialize the sensors, actuators (3) and processor in the multimodal sensor array (1); S102: The processor loads the pre-trained physical effect predictor (24); wherein the physical effect predictor (24) is trained offline using computational fluid dynamics (CFD) simulation data or historical experimental data; S103: The processor loads the pre-trained watering reinforcement learning agent (21) and rinsing reinforcement learning agent (22); wherein the watering reinforcement learning agent (21) and rinsing reinforcement learning agent (22) are trained offline using historical experimental data.

9. The method for controlling root flushing in a plant box using a PID-like structured reinforcement learning approach according to claim 7, characterized in that, The step S3-B, which uses a PID-like reinforcement learning model to generate the rinsing action, specifically includes the following sub-steps: S301: Rinse the reinforcement learning agent (22) based on the current environment state vector S t This generates a preliminary, primitive control action. S302: Grey prediction model (23) reads the final action sequence of the most recent m historical moments {A t-m A t-1 Establish the GM(1,1) grey differential equation, predict the trend of the action at the next moment, and generate the differential correction quantity; S303: Physical Effect Predictor (24) with current state S t and the original actions output by the reinforcement learning agent Using this as input, the shear stress on the root surface is calculated via forward propagation. Soil stripping efficiency and water flow utilization rate Three physical index scalars are used, and the deviations of the physical indexes from the preset ideal target values ​​are linearly weighted to generate an integral correction quantity I. t ; S304: Motion Fusioner (25) will ΔA t , and I t The final execution action is generated through a learnable, weighted, fully connected network by fusion. Linear(·) is a fully connected network operation.

10. The method for controlling root flushing in a plant box using PID-like reinforcement learning as described in claim 7, characterized in that, In step S5, the physical effect predictor (24) is fine-tuned using newly collected real system data to narrow the gap between its simulation and the real world, and to achieve continuous adaptation from simulation to reality.

11. An electronic device, characterized in that, The electronic device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a plant box root flushing control method based on PID-like structural reinforcement learning as described in any one of claims 7 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the plant box root flushing control method based on PID-like reinforcement learning as described in any one of claims 7 to 10.

Images