Agricultural robot crop growth detection method based on touch feedback

Abstract

This invention discloses a method for detecting crop growth using agricultural robots based on tactile feedback, belonging to the field of agricultural robot technology. The method includes: spatial multi-point measurement of low-frequency and high-frequency impedance, leaf thickness, and hardness at multiple plant sites; extraction of delay time and opening / closing rate using a dark-light-dark cycle; compression of leaves with 0.05~0.3N and monitoring impedance recovery to extract residual impedance difference and half-life; feature fusion followed by input to a multi-task network to output stress type and severity. In a blind test of 120 tomato plants, this method achieved an accuracy of 94.2%, superior to the 63.3% of the static impedance method. This invention also provides a matching end effector and temperature compensation. This method can operate at night without damaging plants and is suitable for on-site inspection using agricultural robots.

Description

Technical Field

[0001] This invention belongs to the field of agricultural robots and crop growth detection technology, specifically relating to a method for detecting crop growth using agricultural robots based on tactile feedback. Background Technology

[0002] Tactile feedback-based crop growth detection using agricultural robots is an important direction for precision agriculture, among which bioimpedance sensing technology has attracted widespread attention because it can reflect the physiological state of plants non-destructively and rapidly.

[0003] In fact, water stress, nitrogen deficiency, and early diseases can all cause changes in plant tissue impedance values. Moreover, the measurement results are affected by various factors such as ambient temperature, measurement location, and electrode contact status, making it difficult to distinguish the true cause of impedance changes and causing measurement results to be easily confused. Most studies rely on single static impedance values ​​or static impedance spectra, lacking the utilization of the dynamic response process of plants, resulting in poor detection specificity. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting crop growth in agricultural robots based on tactile feedback to solve the above-mentioned technical problems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a crop growth detection method for agricultural robots based on tactile feedback, comprising four steps: spatial multi-point measurement, dynamic light excitation, dynamic force excitation, feature fusion and stress recognition; The spatial multi-point measurement specifically involves the end effector of the agricultural robot sequentially collecting electrophysiological response parameters and static mechanical parameters from multiple parts of the plant to form a spatial distribution feature vector. The dynamic photoexcitation specifically involves applying time-varying light excitation, including red and blue light, to the target part of the plant, and continuously collecting the time series of electrophysiological response parameters during the excitation period to extract photoresponse characteristic parameters that reflect stomatal movement dynamics. The dynamic force excitation specifically refers to: applying a preset pressure to the target part of the plant for a brief mechanical squeeze, continuously collecting the recovery time series of electrophysiological response parameters after the pressure is released, and extracting dynamic recovery characteristic parameters that reflect cell turgor pressure and cell membrane integrity. The feature fusion and stress identification specifically involves inputting the spatial distribution feature vector, light response feature parameters, and dynamic recovery feature parameters into a pre-trained classification model, which then outputs the crop's stress type and stress severity. The classification model uses a multilayer perceptron. The number of nodes in the input layer is the total number of features. The hidden layer consists of two layers with 64 nodes each. The output layer uses softmax to output three stress types: water stress, nitrogen deficiency, and early disease, and three severity levels: mild, moderate, and severe.

[0006] Preferably, in the spatial multi-point measurement, at least two different parts of the plant are sampled, and the electrophysiological response parameters include low-frequency impedance value and high-frequency impedance value. The measurement frequency of the low-frequency impedance value is 0.5kHz to 5kHz, and the measurement frequency of the high-frequency impedance value is 50kHz to 200kHz. The static mechanical parameters include leaf thickness and leaf hardness.

[0007] The time-varying illumination excitation is a cyclical excitation of "first from dark to light, then from light to dark", with a dark adaptation time of 20 to 60 seconds, an illumination time of 30 to 90 seconds, and a recovery time from dark to dark of 60 to 120 seconds. The illumination source is a dimmable LED, including red and blue light bands.

[0008] The preset pressure for brief mechanical extrusion is 0.05N to 0.3N, and the extrusion duration is 3 to 10 seconds.

[0009] In the spatial multi-point measurement step, different parts of the plant include the top new leaves, the middle functional leaves, and the bottom old leaves. Dynamic light excitation and dynamic force excitation are selected on the middle functional leaves in step one.

[0010] It also includes an environmental compensation step: collecting microenvironmental data through temperature and humidity sensors close to the leaves, using a pre-established compensation model to correct the electrophysiological response parameters to the baseline value at a standard temperature of 25℃, and then constructing a spatial distribution feature vector from the collected electrophysiological response parameters after environmental compensation correction.

[0011] Compared with the prior art, the beneficial effects of the present invention are: 1. Compared with the traditional static impedance method, this invention captures the dynamic response of plants under light and force stimulation, and takes into account the differences between different leaf positions and the response curve of the same leaf over time. Compared with using only single-point static data, it has fewer missed and false judgments, and can improve the overall identification accuracy of water stress, nitrogen deficiency and early diseases.

[0012] 2. All detection steps of this invention can be completed in a single contact action, with shorter time per plant. It can operate at night to avoid interference from ambient light and does not damage the plants, making it easy to integrate into existing agricultural robots. Attached Figure Description

[0013] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0014] Figure 1 This is a flowchart of the present invention, which will be described below in conjunction with... Figure 1 The technical solutions in the embodiments of the present invention will be clearly and completely described.

[0015] 1. System Preparation An agricultural robot with a tracked chassis, equipped with a six-degree-of-freedom collaborative robotic arm, and a multimodal end effector installed at the end of the robotic arm. Its internal structure includes: The base connected to the robotic arm contains a spring potentiometer-type elastic suspension mechanism and a miniature force feedback unit. The spring stiffness of the elastic suspension mechanism is 0.2 N / mm, and the stroke is 5 mm, which is used to control the contact force within 0.05 N ± 0.02 N.

[0016] The contact head, fixed to the lower end of the elastic suspension mechanism, integrates the following components: Interdigitated electrodes: 50μm thick polyimide film substrate, conductive silicone material with volume resistivity <10Ω·cm, 5 pairs of interdigitated fingers, finger length 5mm, finger width 0.2mm, finger spacing 0.5mm, and surface coated with a hydrophilic coating.

[0017] LED light source: Two surface-mount LEDs with wavelengths of 660nm for red light and 450nm for blue light, with light intensity ranging from 0-500μmol / m². 2 Adjust within a range of / s.

[0018] Mechanical sensors: A combination of a 1N range thin-film pressure sensor and a 0.01mm resolution miniature displacement sensor is used to measure contact force, blade thickness, and hardness.

[0019] The temperature and humidity sensor is attached next to the electrode. It also includes a light shield with a black silicone skirt, which is installed around the contact head and can press the blade surface to block ambient light during daytime operation.

[0020] 2. Conduct multi-point spatial measurement work The robot locates the target plant using a vision system, designating the first unfolded leaf from top to bottom as the newest top leaf, the third and fourth leaves as the middle functional leaves, and the sixth and seventh leaves as the older bottom leaves. The end effector moves sequentially above each leaf and, relying on an elastic suspension mechanism, gently presses the non-midrib area on the upper surface of the leaf with a contact force of 0.05N. At each contact point, the robot performs the following measurements: Apply a 1kHz, 50mV sinusoidal signal and acquire low-frequency impedance. A 100kHz, 50mV sinusoidal signal was applied, and the high-frequency impedance was collected. The pressure-displacement curve was recorded. The downward stroke of the contact head was read by the displacement sensor. The displacement increment from the initial contact to when the pressure reaches 0.05N was calculated by combining the pressure sensor data. This displacement increment is the blade thickness T. The slope of the pressure-displacement curve near 0.05N was calculated by combining the pressure sensor data. This slope is the blade hardness H. To be on the safe side, each part was measured 3 times and the arithmetic mean was taken. A total of 12 original measurement values ​​were obtained from the three parts, which constituted the spatial distribution feature vector S.

[0021] A 1kHz sinusoidal signal and a 100kHz sinusoidal signal were applied. The frequency selection of 1kHz and 100kHz was based on the impedance spectrum scanning results in the preliminary experiment. The impedance near 1kHz mainly reflects the extracellular fluid ion migration and is sensitive to both water stress and nutrient deficiency. The impedance near 100kHz is mainly affected by the cell membrane capacitance. When the membrane integrity is impaired, this value decreases significantly. Thickness and stiffness are related to cell turgor pressure and cell wall rigidity. These three parameters can complement each other.

[0022] 3. Dynamic optical excitation and extraction of photoresponse characteristic parameters After the static parameters are collected, keep the electrode contact state unchanged and proceed directly to the dynamic light excitation stage. The dynamic light excitation test is carried out on the middle functional blade. The operation time is preferably arranged at night. If the operation is carried out during the day, the light shield needs to be used to press against the blade surface to reduce interference.

[0023] The specific timing sequence is as follows: Dark adaptation: LED off for 30 seconds, recording every 0.2 seconds during this period. Use the average of the last 10 seconds as the baseline. .

[0024] Photoexcitation: Turn on the red LED to emit light at a wavelength of 660nm with an intensity of 200μmol / m². 2 A red light of / s is emitted for 60 seconds, and is recorded every 0.2 seconds. .

[0025] Dark recovery: Turn off LED and continue recording. 90 seconds.

[0026] Four optical response characteristic parameters were extracted from the acquired time series: delay time, opening rate, closing rate, and amplitude variation. Delay time From the start of light excitation to The interval between the time points when the first consecutive decrease exceeds three standard deviations of the baseline noise, the opening rate. To fit a straight line from the end of the delay to the point where the impedance reaches its minimum, the slope is taken as the opening velocity and the closing velocity as the closing velocity. For the interval from the end of optical excitation to the return of impedance to a stable state, a straight line is fitted to calculate the slope, and the amplitude change ΔZ is the baseline average value minus the impedance average value within 10 seconds before the end of optical excitation.

[0027] In addition, to obtain a specific response to nitrogen deficiency, blue LEDs emitting light at a wavelength of 450 nm and an intensity of 200 μmol / m² were used after the original leaves had rested for 1 minute. 2 / s of blue light, then repeat the above process.

[0028] In data analysis, nitrogen-deficient plants exhibit a brief increase in impedance approximately 10-15 seconds after blue light exposure, while healthy plants, plants under water stress, or diseased plants do not show this phenomenon. Therefore, the amplitude of the second peak under blue light excitation is calculated relative to the maximum positive bias of the baseline. It is also one of the characteristics of light response.

[0029] 4. Extraction of dynamic force excitation and mechanical recovery characteristic parameters The dynamic force excitation is also performed on the middle functional blade, preferably using a different blade. The end effector presses the blade vertically downward with a pressure of 0.1 N, holds for 5 seconds, and then releases the pressure at a rate of approximately 0.05 N / s. Recording is performed continuously for 30 seconds from the moment of release. The sampling frequency is 10Hz. The residual impedance difference and recovery half-life are extracted from the recovery curve. Residual impedance difference Δ 30 seconds after release Subtracting the baseline measured during the dark adaptation phase, this value is close to 0 if the leaf has fully recovered; if cell turgor pressure is lost or the membrane is damaged, it shows a positive bias, and the recovery half-life is [not specified]. To begin from the release To impedance recovery to + Δ The required time and pressure range should be selected as 0.05~0.3N, because the contact stability is poor and the impedance reading fluctuates greatly when it is below 0.05N, while irreversible damage may occur to tender leaves when it is above 0.3N. The duration of 5 seconds can produce a clear recovery curve in most cases, without excessively increasing the detection time of a single plant.

[0030] 5. Environmental compensation steps During each measurement, the temperature and humidity sensor recorded the temperature T and relative humidity RH near the leaf surface. Through constant temperature chamber experiments, it was found that the impedance of tomato leaves at 1kHz has an approximately linear relationship with temperature in the range of 15~35℃: for every 10℃ increase in temperature, the impedance decreases by about 9%, and the effect of humidity on impedance in the range of 30%-80% is less than 3%.

[0031] Based on this, a temperature compensation formula is established: Where α = 0.009 / ℃, all measured and All values ​​were corrected to the 25℃ reference value using the formula described above.

[0032] 6. Implementation of Feature Fusion and Stress Recognition The features obtained from the above steps are fused together, resulting in 12 dimensions of spatial features, 4 dimensions of red light excitation features, 1 dimension of blue light second peak features, and 2 dimensions of force recovery features, for a total of 19 dimensions. Since there is a certain correlation between the features, principal component analysis is used to reduce the dimensionality to 15 principal components.

[0033] The classification model employs a multi-task neural network with an attention mechanism. The input layer has 15 nodes, and a 15×15 attention weight matrix is ​​calculated from the attention layer to weight and reorganize the input features. The shared hidden layer has 128 nodes, with the ReLU activation function and a dropout rate of 0.3. There are two output branches: a classification branch and a severity branch. The classification branch has four nodes: healthy, water stress, nitrogen deficiency, and disease, which are activated by softmax. The severity branch has three nodes: mild, moderate, and severe, which are activated by softmax.

[0034] The loss function is defined as ,in For cross-entropy, To account for mean squared error, training data were collected from 120 tomato plants in an artificial climate chamber, divided into 4 groups of 30 plants each: the healthy group with normal water and fertilizer, the water stress group with soil moisture content of 40% field capacity, the nitrogen deficiency group with nitrogen application of 30% of the conventional amount, and the disease group inoculated with tomato early blight pathogen. Each plant in each group was measured 3 times according to the above procedure, resulting in a total of 360 samples, of which 240 were used for training and 120 were used for testing.

[0035] The model was trained using the Adam optimizer with a learning rate of [missing information]. With a batch size of 32 and 100 training rounds, to improve the generalization ability to compound stress, a small number of samples with a single stress were first used for contrastive learning pre-training under the SimCLR framework, and then fine-tuned on the entire training set.

[0036] Test results: On 120 independent test samples, the overall classification accuracy was 94.2%, with the following accuracy rates for each category: healthy 96.7%, water stress 96.7%, nitrogen deficiency 93.3%, and disease 90.0%. As a control experiment, the accuracy rate using only static 1kHz impedance values ​​combined with the kNN classifier was 63.3%; the accuracy rate using static impedance spectroscopy combined with PCA-SVM was 78.3%. These results demonstrate the advantage of the method of this invention in distinguishing easily confused stress types.

[0037] Taking tomato plants as an example, a complete testing process is as follows: The robot moves to the target plant and uses visual positioning to locate the new leaves at the top, the functional leaves in the middle, and the old leaves at the bottom. The end effector contacts three blades in sequence, measuring the performance at each location. , T and H are recorded, and temperature is compensated to obtain a spatial feature vector; the actuator returns to the central functional leaf, performs a "dark-light-dark" cycle of red light, and extracts... , , After a 1-minute rest period, the middle leaves of the plant, which had been stimulated with red light, were stimulated with blue light to extract ΔZ. Micro-force compression-release was performed on the central functional blade, and the results were recorded. Recovery curve, extract Δ and The 19-dimensional features are reduced in dimensionality using PCA and then input into the neural network. The output result is obtained after about 0.2 seconds. Finally, the robot records the plant position and diagnosis result and moves to the next plant.

[0038] This method can also be applied to other crops, but parameters need to be fine-tuned according to crop characteristics. Taking maize at the large trumpet stage as an example, because maize leaves are relatively hard and have a waxy layer on their surface, the contact force is increased to 0.12N, and tiny bumps are made on the surface of the interdigitated electrodes to pierce the waxy layer. The leaf position selection is changed to the third leaf from the base, the ear leaf, and the second leaf from the top. Because maize stomata have a slow stomatal response, the light exposure time in dynamic light excitation is extended to 90 seconds. For plants inoculated with maize leaf spot disease, early identification can be achieved 3 days before the onset of symptoms through mechanical recovery half-life and light response delay time, with a detection accuracy of 88.9%.

Claims

1. A method for detecting crop growth in agricultural robots based on tactile feedback, characterized in that, It includes four steps: spatial multi-point measurement, dynamic optical excitation, dynamic force excitation, feature fusion, and stress identification. The spatial multi-point measurement specifically involves the end effector of the agricultural robot sequentially collecting electrophysiological response parameters and static mechanical parameters from multiple parts of the plant to form a spatial distribution feature vector. The dynamic photoexcitation specifically involves applying time-varying light excitation, including red and blue light, to the target part of the plant, and continuously collecting the time series of electrophysiological response parameters during the excitation period to extract photoresponse characteristic parameters that reflect stomatal movement dynamics. The dynamic force excitation specifically refers to: applying a preset pressure to the target part of the plant for a brief mechanical squeeze, continuously collecting the recovery time series of electrophysiological response parameters after the pressure is released, and extracting dynamic recovery characteristic parameters that reflect cell turgor pressure and cell membrane integrity. The feature fusion and stress identification specifically involves inputting the spatial distribution feature vector, light response feature parameters, and dynamic recovery feature parameters into a pre-trained classification model, which then outputs the crop's stress type and stress severity. The classification model uses a multilayer perceptron, with the number of input layer nodes equal to the total number of features.

2. The method according to claim 1, characterized in that: In the spatial multi-point measurement, at least two different parts of the plant are collected. The electrophysiological response parameters include low-frequency impedance value and high-frequency impedance value. The measurement frequency of low-frequency impedance value is 0.5kHz to 5kHz, and the measurement frequency of high-frequency impedance value is 50kHz to 200kHz. The static mechanical parameters include leaf thickness and leaf hardness.

3. The method according to claim 2, characterized in that: The time-varying illumination excitation is a cyclical excitation of "from dark to light, then from light to dark", wherein the dark adaptation time is 20-60 seconds, the illumination time is 30-90 seconds, the recovery time from dark is 60-120 seconds, and the illumination source is a dimmable LED that includes red and blue light bands.

4. The method according to claim 3, characterized in that: The preset pressure of the brief mechanical compression is 0.05N to 0.3N, and the compression duration is 3 seconds to 10 seconds.

5. The method according to claim 4, characterized in that: In the spatial multi-point measurement step, different parts of the plant include the top new leaves, the middle functional leaves, and the bottom old leaves. Dynamic light excitation and dynamic force excitation are performed on the middle functional leaves.

6. The method according to claim 1, characterized in that: It also includes an environmental compensation step: collecting microenvironmental data through temperature and humidity sensors close to the leaves, using a pre-established compensation model to correct the electrophysiological response parameters to the baseline value at a standard temperature of 25℃, and then constructing a spatial distribution feature vector from the collected electrophysiological response parameters after environmental compensation correction.

Images