Soil state detection method and device based on plant leaf phenotypic characteristics

Abstract

The invention provides a soil state detection method and device based on plant leaf phenotypic features, and the method comprises the steps: collecting historical data, and carrying out the preprocessing operation, such as contour extraction, feature extraction and feature dimension reduction; constructing a model data set based on the preprocessed data; training through the model data set to obtain a soil environment prediction model; and collecting real-time data, inputting a processing result into the soil environment prediction model, and outputting a detection result of at least one soil environment parameter. According to the method, the soil environment prediction result is obtained by shooting the image and analyzing the plant leaf phenotypic characteristics, so that the cost is reduced on one hand, and the problem of damage to the soil on the other hand is avoided; furthermore, based on image analysis, the method has the advantage of high universality, and the model can quickly adapt to soil state detection requirements of different crops by adjusting training data and parameters; in combination with multiple algorithms, redundant information can be reduced, and the generalization ability of the model can be remarkably improved while the precision is kept.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural detection, and particularly relates to a method and device for detecting soil conditions based on phenotypic characteristics of plant leaves. Background Art

[0002] Soil, as the basic carrier for crop growth, parameters such as its moisture, pH value, organic matter content, and mineral nutrients (N / P / K) directly determine the physiological state and yield quality of crops. Each year, a large amount of grain production is reduced due to soil nutrient imbalance, directly causing serious economic losses. Precision agriculture can solve soil problems to a certain extent. It requires dynamic monitoring of soil conditions to achieve on-demand irrigation and precision fertilization, which is of strategic significance for ensuring food security and sustainable agricultural development.

[0003] In the prior art, soil condition detection methods mainly rely on direct measurement by sensors. Although this method is accurate, there are still corresponding problems:

[0004] (1) High cost: In direct detection technologies (sensors / sampling analysis), sampling and analysis are carried out through contact sensors. For example, detecting NH4 + , NO3 - and other ions by an ion-selective electrode (ISE), or measuring moisture by a time domain reflectometer (TDR). Although the detection accuracy is high, the equipment cost is high, especially when applied in large-scale farmland, the cost increase is more significant.

[0005] (2) Complex wiring: Sensors usually need to be connected to a data acquisition system through wired or wireless networks; wired sensors are limited by the cable length (effective distance < 500 meters), and although wireless sensors (such as LoRa) support remote transmission, when the node spacing > 200 meters, the error rate rises to 15%. Therefore, in practical applications, the overall project is complex, especially in farmland with complex terrain.

[0006] (3) Limited coverage: Due to the limited number of sensors, each sensor can only cover a limited area around it, resulting in relatively sparse soil condition data points in the entire farmland and data blind spots.

[0007] (4) Complex data management: The data generated by a large number of sensors requires complex processing and management systems, increasing the technical difficulty and operation and maintenance costs.

[0008] Accordingly, indirect detection technologies such as remote sensing or spectral detection have emerged in the prior art. Among them, remote sensing monitoring includes aerial remote sensing: retrieving soil organic matter through multispectral satellite images; ground spectroscopy: detecting soil moisture using near-infrared (NIR) spectroscopy. Although such technologies do not require destructive testing of the soil, remote sensing detection has the problem of insufficient resolution. For example, the spatial resolution of satellite remote sensing is ≥10 meters, and it is impossible to identify soil heterogeneity at the field plot scale (<100 square meters); ground spectroscopy requires manual handheld devices to measure point by point, with an efficiency <50 points / hour, making it difficult to cover large areas of farmland. In addition, spectral detection methods also have the problem of serious environmental interference. For example, soil spectra are significantly affected by vegetation coverage and surface roughness. When the vegetation coverage rate >30%, the inversion error of organic matter increases by 40%.

[0009] In recent years, indirect detection methods based on plant phenotypic characteristics have gradually received attention. As an important organ for plants to interact with the environment, the phenotypic characteristics (color, morphology, texture, etc.) of plant leaves can reflect changes in soil conditions. However, existing research based on plant phenotypic characteristics mostly focuses on the detection of single environmental factors and lacks a comprehensive assessment of soil conditions. In addition, existing methods have problems such as insufficient accuracy and poor generalization ability in feature extraction and model construction, making it difficult to meet the needs of precision agriculture.

[0010] In summary, there is still a need for a low-cost, high-precision, and easy-to-deploy soil condition detection method and device that can comprehensively and real-time monitor soil conditions through plant leaf phenotypic characteristics. Summary of the Invention

[0011] In view of the problems existing in the prior art, the present invention provides a soil condition detection method and device based on plant leaf phenotypic characteristics to solve the above problems.

[0012] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0013] On the one hand, the present invention provides a soil condition detection method based on plant leaf phenotypic characteristics, including the following steps:

[0014] Collect historical leaf images and corresponding soil environmental data;

[0015] Perform image preprocessing on the historical leaf images and obtain a preprocessing data set; at least contour extraction, feature extraction, and feature dimensionality reduction are provided in the steps of the image preprocessing;

[0016] Based on the preprocessing data set and the corresponding soil environmental data, construct a model data set;

[0017] Train a random forest model through the model data set to obtain a soil environmental prediction model;

[0018] Collect real-time leaf images for the image preprocessing, and input the processing results into the soil environment prediction model to output the detection results of at least one soil environment parameter.

[0019] Further, the image preprocessing includes the following steps:

[0020] The contour extraction: Remove the background information of the leaf image based on the image segmentation algorithm, and perform image enhancement through the local image enhancement algorithm; Optimize the contour of the leaf image through the gradient vector flow algorithm;

[0021] The feature extraction: Extract the high-dimensional feature vectors of the leaf image with optimized contour through the convolutional neural network model;

[0022] The feature dimensionality reduction: Reduce the dimensionality of the high-dimensional feature vectors through the principal component analysis method to obtain low-dimensional feature vectors.

[0023] Even further, the contour extraction further includes the following steps:

[0024] Construct a leaf shape recognition model;

[0025] Output the shape recognition result of the leaf image based on the leaf shape recognition model, and the shape recognition result is the prior information of the leaf image;

[0026] Set a prior addition term in the energy function of the gradient vector flow algorithm, and the prior addition term is the weighted prior information of the leaf image.

[0027] Even further, in the convolutional neural network model, extract the local and global features of the enhanced leaf image with optimized contour through multiple convolutional layers with different sizes, and dynamically set the feature weights based on the channel attention mechanism.

[0028] Even further, in the principal component analysis method, set a dynamic regularization term and set a local structure constraint.

[0029] Further, set a multi-task learning mechanism in the random forest model and establish a correlation constraint between multiple tasks;

[0030] The dynamic weights of multiple tasks in the random forest model are set based on the dynamic changes of the model input data.

[0031] Further, the soil environment data includes one or more combinations of soil temperature and humidity, air temperature and humidity, light intensity, wind speed, stem diameter, soil pH value, and soil electrical characteristic parameters.

[0032] Further, the historical leaf images include images of a single leaf under different time periods, different lighting conditions, and different climate conditions, and the images of the single leaf also include images taken from at least two shooting angles.

[0033] On the other hand, the present invention also provides a soil condition detection device based on plant leaf phenotypic characteristics, which mainly includes a camera component, a display component, a data acquisition module, and a data processing module;

[0034] The data acquisition module: is used to acquire leaf images;

[0035] The data processing module: is used to perform image preprocessing on the leaf images, and input the preprocessing results into a soil environment prediction model to output detection results;

[0036] The camera component is connected to the data processing module through the data acquisition module; the data processing module is also connected to the display component.

[0037] Furthermore, it further includes an energy storage component, a communication module, and a soil detection component;

[0038] The energy storage component: is used to supply power to the soil condition detection device;

[0039] The communication module: is used to wirelessly transmit or receive data;

[0040] The soil detection component: is used to acquire corresponding soil environment data.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] By analyzing the plant leaf phenotypic characteristics by taking plant leaf images, the present invention significantly reduces the hardware cost and system complexity, and indirectly detects the soil condition, effectively avoiding the high cost of traditional sensors and the problem of soil damage, protecting the soil structure and reducing the interference to crops, and is particularly suitable for long-term monitoring and large-area farmland management; further, based on image analysis, corresponding models can be separately trained for different crops, making it have the advantage of strong versatility, and by adjusting the training data and parameters, the model can quickly adapt to the soil condition detection needs of different crops, showing a wide range of application prospects and good scalability; furthermore, by combining multiple algorithms, redundant information can be reduced, and while maintaining the accuracy, the generalization ability of the model can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is the flowchart of the method of the present invention;

[0045] Figure 2 It is the schematic diagram of the system structure of the device in the specific embodiment of the present invention;

[0046] Figure 3 It is the schematic diagram of the test area of the device in the specific embodiment of the present invention;

[0047] Figure 4 It is the three-dimensional perspective view of the device in the specific embodiment of the present invention;

[0048] Figure 5 It is the human-computer interaction interface of the display component in the specific embodiment of the present invention.

[0049] In the figure: 1. Soil temperature and humidity sensor; 2. Air temperature and humidity sensor; 3. Light intensity sensor; 4. Wind speed sensor; 5. Stem diameter sensor; 6. Sprinkler irrigation system; 7. Soil state detection device; 8. Ventilation system; 9. Light control system; 10. Soil electrical property sensor; 11. Main control box; 701. Switch; 702. Power charging port; 703. Heat dissipation component; 704. Touch display screen; 705. Camera; 706. Shell; 707. Main control chip; 708. USB interface. Specific Embodiments

[0050] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0051] It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0052] It should be noted that the methods used in the present invention are all conventional methods unless otherwise specified; the raw materials and devices used are all conventional commercially available products unless otherwise specified.

[0053] It should also be noted that, for the convenience of understanding in the specific embodiments of the present invention, the method steps are described in a certain order. However, those skilled in the art can adjust the order of the steps according to actual needs. Therefore, this cannot be used as a limiting condition. Further, in the description of the following specific embodiments, without special instructions, the upper and lower subscripts of each parameter should be understood as the distinguishing marks of similar identifiers according to common explanations, representing the parameters of the subscript-related or corresponding devices, and cannot be understood as specific models or special marks.

[0054] This embodiment proposes a method for detecting soil conditions based on the phenotypic characteristics of plant leaves, as Figure 1 shown, which specifically includes the following steps:

[0055] S1. Acquisition of multi-dimensional environmental information and leaf images.

[0056] Collect historical leaf images and corresponding soil environment data for constructing the training set of the model. Among them, the most common way to query historical information is to intercept leaf images from surveillance videos. However, due to the lack of some multi-dimensional soil environment data and the fact that the resolution of surveillance images does not meet the training requirements, a local historical information collection scenario is provided in this embodiment. Specifically, as Figure 3 shown, a micro-meteorological station is built, which is respectively equipped with a soil temperature and humidity sensor 1, an air temperature and humidity sensor 2, a light intensity sensor 3, a wind speed sensor 4, a stem diameter sensor 5, a sprinkler irrigation system 6, a soil condition detection device 7, a ventilation system 8, a light regulation system 9, and a soil electrical property sensor 10 in a fixed area. The control units of the above components are all connected to the main control box 11. In this area, the system can comprehensively and real-time obtain multi-dimensional information of the soil and the environment, providing rich data support for the correlation analysis of leaf images and soil humidity. Optionally, in this embodiment, the soil environment data includes soil temperature and humidity, air temperature and humidity, light intensity, wind speed, stem diameter, soil pH value, and soil electrical property parameters (calculating soil nutrient components based on soil conductivity and dielectric constant), as well as air flow rate information.

[0057] When acquiring leaf image information, the device in this embodiment will collect images of a single leaf at different times, under different lighting conditions, and under different climate conditions, and there are at least two images taken at different shooting angles in the images of the single leaf. During the specific execution, a shooting strategy with multiple time periods and multiple environmental conditions is adopted to ensure the comprehensiveness and diversity of the data. The shooting will be carried out at different times of the day, including morning, noon, afternoon, and evening, to capture the changes of the leaf under different light intensities. This will help analyze the influence of lighting conditions on the phenotypic characteristics of the leaf and further improve the correlation between soil conditions and plant growth characteristics. In addition, shooting will also be carried out under different weather conditions, including sunny, cloudy, and rainy days, so as to increase the diversity of the dataset and ensure that the model can cope with various environmental changes during training. At the same time, to ensure the high quality and consistency of the images, a high-precision camera is used for shooting to ensure the image resolution and detail clarity. To eliminate the influence brought by the changes in shooting angles and positions, all shooting devices will be stably fixed through a fixed bracket or tripod, and a precise angle adjustment device is used to ensure the consistency of the camera position during each shooting. In addition, multiple cameras will be arranged at different positions for multi-camera shooting to capture images of the leaf from different angles, further improving the richness and accuracy of the data.

[0058] The sampling rate of soil environment data is set to be collected once every 10 minutes to ensure the real-time and accuracy of the data. And the entire data collection process will last for a relatively long time, covering multiple growth stages of the plant from seedling to maturity, ensuring that the data can reflect the growth dynamics of the plant. In terms of light control, the sunshade or reflector of the light regulation system 9 is used to adjust the natural light, avoiding the influence of too strong or too weak light on the image quality, and shooting separately under shadow and direct sunlight to obtain leaf images under different light intensities, thereby improving the adaptability of the data. In terms of soil condition monitoring, to ensure that the changes in soil conditions are accurately recorded, the environmental factors of the soil can also be adjusted and controlled in various ways, such as: controlling the soil humidity by adding water through the sprinkler irrigation system 6 to simulate different soil moisture conditions; adjusting the nitrogen, phosphorus, and potassium content in the soil by applying fertilizers to reflect the changes in soil fertility; monitoring the change of soil pH value by adding different soil acidity and alkalinity reagents; obtaining the wind speed information in the soil surrounding environment by controlling the wind speed of the ventilation system 8 at different levels, so as to comprehensively obtain various environmental data affecting plant growth and leaf characteristics.

[0059] S2. Data preprocessing.

[0060] Perform image preprocessing on the historical leaf images collected above and obtain a preprocessing dataset. Among them, the steps of image preprocessing at least include contour extraction, feature extraction, and feature dimensionality reduction; specifically, the image preprocessing includes the following steps:

[0061] S21. Contour extraction: Remove the background information of the leaf image based on the image segmentation algorithm, and perform image enhancement through the local image enhancement algorithm to obtain the enhanced leaf image; Optimize the contour of the enhanced leaf image through the gradient vector flow algorithm; Specifically, use the GrabCut algorithm to remove the background of the leaf image, and through the preliminary annotation of the foreground and background, accurately separate the leaf area, remove irrelevant background noise, and extract a clearer leaf image. Adjust all leaf images to a unified size of 224×224 pixels to ensure the consistency of the images when input into the model and avoid errors caused by different input sizes of the model. Apply the Contrast Limited Adaptive Histogram Equalization (CLAHE) method to uniformly adjust the brightness and contrast of the images, improve the low-contrast areas caused by uneven illumination in the leaf images, make the features of the leaves more prominent, and facilitate subsequent feature extraction. Apply image enhancement techniques to generate more samples by performing various transformations on the original data, so as to expand the training dataset, reduce the overfitting phenomenon, and improve the generalization ability of the model, including: rotation, translation, flipping, scaling, and noise addition, etc.

[0062] Furthermore, to optimize the contour of the above leaf image through the gradient vector flow algorithm, the GVF (Gradient Vector Flow) Snake algorithm can be used. It is a classic image segmentation algorithm widely used in medical image processing, target tracking, edge detection and other fields. Its core idea is to evolve the initial contour to the target edge by minimizing the energy. However, the traditional GVF Snake algorithm has many limitations when dealing with the contours of plant leaves. First of all, plant leaf images usually contain complex textures and noises, and the traditional GVF Snake algorithm is easily interfered by these noises, resulting in inaccurate contour segmentation; Secondly, the traditional GVF Snake algorithm only relies on the gradient information of the image and lacks prior knowledge of the leaf shape. The shape of plant leaves usually has certain regularity, and the traditional algorithm cannot utilize this prior information, resulting in the segmentation result not conforming to the true shape of the leaves; Moreover, the traditional GVF Snake algorithm uses fixed energy weights and cannot adapt to the local feature changes of the leaf edges; In addition, the traditional GVF Snake algorithm is mainly applicable to medical images and simple target tracking, while in the phenotypic analysis of plant leaves, due to the complex shape and rich edge details of the leaves, the performance of the traditional algorithm is poor.

[0063] To solve the above problems, obtain the leaf contour, reduce the interference of noise on the segmentation result, and improve the segmentation accuracy; This embodiment proposes an improved GVF algorithm for leaf contour optimization, namely the LCO-GVF (Leaf Contour Optimized GVF, LCO-GVF) algorithm. The specific improvement contents include:

[0064] (1) The traditional GVF Snake algorithm only relies on the gradient information of the image and lacks prior knowledge of the leaf shape. Therefore, in this embodiment, the shape prior information of the leaf is generated through a pre-trained leaf shape model and added as a constraint term to the GVF energy function to solve the problem of prior absence. Specifically, a leaf shape recognition model is constructed: a neural network model is trained by establishing a dataset of typical leaf images - leaf shapes (labels), so as to form a corresponding leaf shape recognition model. Then, based on the leaf shape recognition model, the shape recognition result of the aforementioned leaf image is output, and this is used as the prior information of the leaf image;

[0065] The shape prior constraint term makes the segmentation result more conform to the true shape of the leaf by minimizing the distance between the current contour points and the prior contour points. Thus, the leaf shape prior constraint term E shape has the following formula:

[0066]

[0067] In the formula, C i is the current contour point; C prior is the contour point generated by the leaf shape prior model; n is the number of contour points; i is the position ordinal number.

[0068] The shape prior constraint can effectively reduce the interference of noise on the segmentation result, making the segmentation contour smoother and more conform to the true shape of the leaf.

[0069] (2) The traditional GVF Snake algorithm uses fixed energy weights and cannot adapt to the local feature changes of the leaf edge. Based on this, in this embodiment, the weights of the energy function are dynamically adjusted according to the local features of the leaf edge. Specifically, the dynamic energy adjustment term dynamically adjusts the energy weight of each contour point by calculating the complexity of the local edge, enabling the algorithm to better capture the detailed features of the leaf. The formula is:

[0070]

[0071] In the formula, E dynamic is the dynamic energy adjustment term; w i is the weight dynamically adjusted according to the local features of the leaf edge; E local is the local energy function, representing the local edge feature of the current contour point.

[0072] Weight calculation:

[0073]

[0074] In the formula, complexity i represents the edge complexity at the i-th contour point, and α is a parameter controlling the weight change speed.

[0075] The dynamic energy adjustment mechanism can adaptively adjust the energy weights according to the local features of the blade edge, enabling the algorithm to have higher segmentation accuracy in complex edge regions.

[0076] (3) Set a prior addition term in the energy function of the gradient vector flow algorithm, which is the prior information of the weighted blade image, so as to introduce the blade shape prior constraint and the dynamic energy adjustment mechanism into the traditional GVF energy function to form the improved energy function E LCO-GVF , and its function formula is:

[0077] E LCO-GVF = E GVF + γ1E shape + γ2E dynamic

[0078] In the formula, E GVF is the traditional GVF energy function; γ1 and γ2 are weight coefficients used to balance the contributions of each term.

[0079] Thus, through optimization, the improved gradient vector flow algorithm (LCO-GVF algorithm) of this embodiment can better process complex blade edges, significantly improving the segmentation accuracy. And it enables the algorithm to still maintain high segmentation accuracy in complex situations such as uneven illumination and leaf surface spots. In addition, the improved algorithm can adaptively adjust the energy weights according to the local features of the blade edge, enabling the algorithm to better capture the detailed features of the blade.

[0080] S22. Feature extraction: Extract the high-dimensional feature vector of the blade image with optimized contours through a convolutional neural network model. Specifically, the convolutional neural network model of this embodiment is the EfficientNetV2-MAFFM model, which is used to extract the high-dimensional features of the blade image.

[0081] To solve the many problems existing in the traditional EfficientNetV2 model when dealing with plant leaf phenotypic features, including: insufficiently fine feature extraction, high computational complexity, lack of dynamic adaptability, etc. This embodiment proposes an improved EfficientNetV2-MAFFM model, which extracts the local and global features of the enhanced blade map with optimized contours through multiple convolutional layers of different sizes, and dynamically sets the feature weights based on the channel attention mechanism. The specific steps include:

[0082] (a) Traditional multi-scale feature extraction methods usually use fixed convolution kernel sizes and weights and cannot dynamically adjust according to the local and global information of the input features. Based on this, in this embodiment, by introducing a channel attention mechanism and adaptive weight adjustment, the model can adaptively select the most relevant scale for fusion according to the input features, and multiple convolutional layers with different sizes are used to extract local and global features.

[0083] Multi-scale feature extraction:

[0084] F 1x1 = Conv 1×1 (F in )

[0085] F 3x3 = Conv 3×3 (F in )

[0086] F 5x5 = Conv 5×5 (F in )

[0087] In the formula, F in is the input feature; Conv 1x1 , Conv 3x3 , Conv 5x5 are convolutional layers of 1×1, 3×3, and 5×5 respectively.

[0088] Channel attention mechanism:

[0089] w 1x1 = σ(FC(GAP(F 1×1 )))

[0090] w 3x3 = σ(FC(GAP(F 3×3 )))

[0091] w 5x5 = σ(FC(GAP(F 5×5 )))

[0092] In the formula, GAP is the global average pooling layer; FC is a lightweight fully connected network; w 1×1 , w 3×3 , w 5×5 are the weights of the features extracted by the convolutional layers of 1×1, 3×3, and 5×5 respectively; σ is the Sigmoid activation function.

[0093] Feature fusion:

[0094] F out = w 1×1 ·F 1×1 + w 3×3 ·F3×3 +w 5×5 ·F 5×5

[0095] Wherein, F out is the fused feature.

[0096] During the blade feature extraction process, the channel attention mechanism is combined with multi-scale feature extraction. By dynamically adjusting the weights of each scale feature, the model can adaptively select the most relevant scale for fusion according to the input features. This method not only improves the accuracy of feature extraction but also significantly reduces the computational complexity.

[0097] (b) To ensure the lightweight of the MAFFM module, the following design is also adopted in this embodiment:

[0098] Traditional convolution operations require independent convolution calculations for each channel during calculation, resulting in a high computational complexity. In this embodiment, depthwise separable convolution is adopted, which decomposes the convolution operation into depth convolution and pointwise convolution, significantly reducing the number of parameters and the amount of calculation. The expression is:

[0099] F out = DepthwiseConv(F out ) + PointwiseConv(F out )

[0100] Wherein, DepthwiseConv is the depth convolution, which performs independent convolution operations on each input channel; PointwiseConv is the pointwise convolution, which performs 1×1 convolution operations on the output of the depth convolution.

[0101] In addition, traditional channel attention mechanisms usually use fully connected layers to calculate channel weights, resulting in a high computational complexity. In this embodiment, a lightweight channel attention mechanism is introduced, and the channel weights are calculated through global average pooling and a lightweight fully connected network, significantly reducing the amount of calculation.

[0102] w = σ(FC(GAP(F)))

[0103] Wherein, GAP is the global average pooling layer; FC is the lightweight fully connected network.

[0104] Combining depthwise separable convolution with a lightweight channel attention mechanism during the blade feature extraction process can significantly reduce the number of parameters and the amount of calculation of the model while maintaining the accuracy of feature extraction, making it suitable for deployment on resource-constrained devices.

[0105] (c) Traditional feature fusion methods usually use fixed weights or simple weighted summation, and cannot adaptively adjust the fusion strategy according to the input features. Based on this, in this embodiment, by introducing dynamic weight adjustment and multi-scale feature interaction, the model can adaptively select the most relevant features for fusion according to the input features. The expression is as follows:

[0106]

[0107] In the formula, w i is the weight of the i-th scale feature; α ij is the interaction weight between the i-th and j-th scale features; ⊙ represents element-wise multiplication; F i and F j are the features after fusing the i-th and j-th scale features respectively.

[0108] An adaptive feature fusion mechanism is proposed during the leaf feature extraction process. By dynamically adjusting the weights of each scale feature and the interaction weights, the model can adaptively select the most relevant features for fusion according to the input features. This method not only improves the accuracy of feature extraction but also significantly enhances the generalization ability of the model.

[0109] During the process of improving the EfficientNetV2-MAFFM model, from the above content, it can be seen that for the traditional weighted summation and feature concatenation methods, it is found that although the weighted summation is simple, it cannot adapt to the local feature changes of leaf images, while feature concatenation, although retaining more information, is difficult to be actually applied due to high-dimensional redundancy and computational complexity problems. Subsequently, a channel attention mechanism is introduced to dynamically adjust the feature weights through global average pooling and a lightweight fully connected network. Although the accuracy is improved, there are still problems of high computational complexity and insufficient cross-scale feature interaction. For this reason, a cross-scale feature interaction mechanism is further introduced to complement different scale features through element-wise multiplication, significantly improving the ability to capture leaf edges and textures in complex backgrounds. However, the computational cost of the interaction module is large. Therefore, a dynamic weight adjustment mechanism is introduced to dynamically adjust the weights based on local features, which not only improves the accuracy but also reduces the computational complexity. Finally, this embodiment integrates cross-scale interaction and dynamic weight adjustment, and proposes an adaptive feature fusion mechanism, whose feature extraction accuracy on leaf images is significantly improved, the computational efficiency is significantly optimized, and it is suitable for deployment on resource-constrained devices. This improvement process has gone through multiple experiments and optimizations, gradually solving the limitations of traditional methods and significantly improving the performance of the model in the task of plant leaf phenotypic feature extraction.

[0110] The improved EfficientNetV2-MAFFM model in this embodiment significantly enhances the model's ability to extract leaf phenotypic features through multi-scale feature extraction, dynamic weight adjustment, and lightweight design, while maintaining high computational performance. However, EfficientNetV2-MAFFM extracts local and global features of the leaf through multi-scale convolutional layers and uses the Adaptive Feature Fusion Module (MAFFM) to dynamically adjust the weights of features at different scales to generate a high-dimensional feature vector. Directly using these features will lead to high computational complexity and model overfitting problems. Therefore, it is necessary to perform dimensionality reduction on the extracted high-dimensional feature vector.

[0111] S23. Feature dimensionality reduction: The high-dimensional feature vector is reduced in dimension through the principal component analysis method to obtain a low-dimensional feature vector. Specifically, this embodiment uses the Dynamic Local Sparse PCA (DLSPCA) algorithm to solve the limitations of traditional principal component analysis (PCA) when dealing with plant leaf phenotypic features, such as: global linear assumption, lack of sparsity and local adaptability, inability to dynamically adjust, etc.

[0112] The specific steps include:

[0113] A. Traditional PCA performs a linear combination of all features, lacking sparsity, resulting in the features after dimensionality reduction still containing a large amount of redundant information. Based on this, the present invention introduces a dynamic L1 regularization term into the objective function, making the features after dimensionality reduction have dynamic sparsity, thereby reducing redundant information.

[0114] The objective function of traditional PCA is:

[0115]

[0116] In the formula, X is the input data matrix; U is the projection matrix; ∥·∥ F is the Frobenius norm.

[0117] After introducing the dynamic L1 regularization term, the improved objective function is:

[0118]

[0119] In the formula, λ is the regularization parameter, controlling the intensity of sparsity; θ k is the dynamic sparse weight; μ k is the kth principal component.

[0120] Among them, the calculation formula of the dynamic sparse weight θ k is:

[0121]

[0122] where importance k is the importance score of the k-th principal component; α is a parameter that controls the change rate of the dynamic sparse weight.

[0123] B. Traditional PCA assumes that the data is globally linear, while the phenotypic characteristics of plant leaves usually have non-linearity and local structure. Based on this, in this embodiment, by introducing local structure constraints into the objective function, the features after dimensionality reduction can better retain the local structure of the data.

[0124] Thus, the improved objective function is:

[0125]

[0126] where W ij is the similarity weight between data points x i and x j ; γ is the strength parameter of the local structure constraint.

[0127] Similarity weight calculation:

[0128]

[0129] where σ is a parameter that controls the similarity decay rate (different from the aforementioned Sigmoid activation function).

[0130] C. Traditional PCA uses a fixed number of principal components and cannot adaptively adjust the dimensionality reduction strategy according to the local structure of the data. Based on this, in this embodiment, by introducing an adaptive threshold, the model can adaptively select the number of principal components according to the local structure of the data, and the improved objective function is:

[0131]

[0132] where φ k is the adaptive threshold.

[0133] Among them, the calculation formula of the adaptive threshold φ k is:

[0134]

[0135] where importance k is the importance score of the k-th principal component; β is a parameter that controls the change rate of the adaptive threshold.

[0136] In the process of improving DLSPCA, many limitations of traditional PCA in processing plant leaf phenotypic characteristics were found. First of all, traditional PCA assumes that the data is globally linear, while the high-dimensional features of leaf images usually have non-linearity and local structure, resulting in a large amount of useful information being lost after dimensionality reduction. Therefore, a local structure constraint was introduced. The similarity weights between data points were calculated through a Gaussian kernel function and added to the objective function of PCA, significantly improving the ability to retain the local structure of the features after dimensionality reduction. However, there is still redundant information in the features after dimensionality reduction, especially in the high-dimensional leaf image features, and the dimensionality reduction effect is not ideal. Therefore, a dynamic L1 regularization term was further introduced. By calculating the importance scores of each principal component and dynamically adjusting the sparse weights, the features after dimensionality reduction have stronger sparsity, effectively reducing redundant information.

[0137] In addition, traditional PCA uses a fixed number of principal components and cannot adapt to the local structure changes of different datasets, resulting in poor generalization ability. Therefore, in this embodiment, an adaptive principal component selection mechanism is introduced. By dynamically adjusting the number of principal components, the model can adaptively select the most relevant principal components according to the local structure of the data. After multiple experiments, it was found that the performance of traditional PCA and sparse PCA on leaf images was not ideal. Finally, by introducing local structure constraints and dynamic sparse constraints, DLSPCA is significantly superior to traditional methods in terms of dimensionality reduction effect and is particularly suitable for the dimensionality reduction task of plant leaf phenotypic characteristics.

[0138] After that, based on the preprocessed dataset and the corresponding soil environment data, a model dataset was constructed and divided into a training set, a validation set, and a test set according to the ratio of 70-15-15. Ensure that the model can effectively learn, adjust, and evaluate, while avoiding overfitting and ensuring the generalization ability of the model.

[0139] S3. Generate a model.

[0140] Train a random forest model through the model dataset. Among them, in this embodiment, in order to solve the limitations of the traditional random forest model in processing plant leaf phenotypic characteristics, including: being unable to handle multiple related tasks simultaneously, resulting in limited generalization ability of the model; lacking task interrelationship, resulting in limited performance of the model; and being unable to dynamically adjust task weights, an Adaptive Multi-Task Random Forest (AMTRF) was proposed. By introducing a multi-task learning mechanism, task interrelationship modeling, and dynamic task weight adjustment, the model performance was significantly improved. The specific steps include:

[0141] S31: Use the low-dimensional feature vectors after DLSPCA dimensionality reduction as input data, prepare the label data Y(1), Y(2), …, Y(M) for multiple tasks, and divide the data set into a training set, a validation set, and a test set.

[0142] S32: Initialize the parameters of the AMTRF model, including: the number of decision trees T, the maximum tree depth max_depth for each task, the task-interrelationship weight λ mn , and the dynamic task weight adjustment parameter α. And initialize the dynamic weight ω (m) for each task, with the initial value set to 1.

[0143] S33: Model training;

[0144] S331. For each task m, construct T decision trees using the training set data. Select the optimal splitting point by using the local adaptive splitting strategy, and optimize the inter-task relationship according to the inter-task relationship modeling mechanism.

[0145] The inter-task relationship of the traditional random forest model is:

[0146] Corr(y (m) ,y (n) ) = 0

[0147] In the formula, Corr(y (n) ,y (n) ) is the relationship between the m-th task and the n-th task.

[0148] In this embodiment, an inter-task relationship modeling mechanism is introduced, and the improved inter-task relationship is:

[0149]

[0150] In the formula, λ mn is the inter-task relationship weight between the m-th task and the n-th task; is the relationship between the t-th decision tree and the m-th task and the n-th task.

[0151] S332. During the training process, dynamically adjust the task weights according to the importance scores of each task. The task weights of the traditional random forest are:

[0152] ω (m) = 1

[0153] In the formula, ω (m) is the weight of the m-th task.

[0154] In this embodiment, a dynamic task weight adjustment mechanism is introduced, and the improved task weight is:

[0155]

[0156] where importance (m) is the importance score of the m-th task; α is a parameter that controls the change rate of the dynamic task weights.

[0157] S333. Establish a multi-task learning mechanism. By simultaneously processing multiple related tasks, the model can better capture the correlation between tasks, thereby improving the generalization ability of the model.

[0158] It is known that the output of the traditional random forest model is:

[0159]

[0160] where y is the final output of the model; T is the number of decision trees; y t is the output of the t-th decision tree.

[0161] In this embodiment, a multi-task learning mechanism is introduced, and the improved output is:

[0162]

[0163] where y (m) is the final output of the m-th task; y t (m) is the output of the t-th decision tree for the m-th task.

[0164] Thus, by simultaneously optimizing the loss functions of all tasks, the loss function Loss is obtained as:

[0165]

[0166] where Loss (m) is the loss function of the m-th task

[0167] S334. Input the low-dimensional feature vectors of the test set into the trained AMTRF model. For each task m, use the corresponding decision tree for prediction to obtain the prediction results. Use the dynamic weight voting mechanism to synthesize the prediction results of all decision trees to obtain the final prediction result as:

[0168]

[0169] where is the prediction result of the t-th decision tree for the m-th task; is the dynamic weight of the t-th decision tree for the m-th task.

[0170] In the process of improving AMTRF, when using the traditional random forest model to handle multiple related tasks, it is found that it cannot handle multiple tasks simultaneously, resulting in poor generalization ability. Therefore, in this embodiment, a multi-task learning mechanism is introduced. By optimizing multiple related tasks simultaneously and making full use of the correlation between tasks, the generalization ability of the model is significantly improved. However, the traditional random forest lacks the modeling of the correlation between tasks, resulting in poor performance of the model when dealing with multiple tasks. Therefore, a mechanism for modeling the correlation between tasks is further introduced. By calculating the correlation weights between tasks, the correlation between tasks is optimized, thereby improving the performance of the model.

[0171] In addition, the traditional random forest uses fixed task weights and cannot adjust the weights according to the dynamic changes of the input data, resulting in poor adaptability of the model. Therefore, this embodiment also introduces a dynamic task weight adjustment mechanism. By calculating the importance scores of each task, the task weights are dynamically adjusted, enabling the model to adaptively adjust the task weights according to the changes in the input data. After multiple experiments, it is found that simple task weighting and task concatenation methods perform poorly on leaf phenotypic characteristics. Finally, by introducing the modeling of the correlation between tasks and dynamic weight adjustment.

[0172] S4. Model evaluation.

[0173] Evaluate the AMTRF model to ensure its reliability and accuracy in practical applications.

[0174] After the model training is completed, use the test set to evaluate the performance of the AMTRF model. Through theoretical analysis and comparison with the performance of existing methods, the effectiveness of the model is verified.

[0175] By introducing a multi-task learning mechanism, the modeling of the correlation between tasks, and dynamic task weight adjustment, the AMTRF model significantly improves its performance. The multi-task learning mechanism enables the model to handle multiple related tasks simultaneously, making full use of the correlation between tasks and enhancing the generalization ability of the model; the modeling of the correlation between tasks, by introducing the correlation constraints between tasks, enables the model to better capture the correlation between tasks, thereby improving the performance of the model; the dynamic task weight adjustment, by dynamically adjusting the weights of tasks, enables the model to adjust the task weights according to the dynamic changes of the input data, thereby enhancing the generalization ability of the model.

[0176] In the corresponding comparative example, the average accuracy of the traditional random forest model in the multi-task prediction task of plant leaf phenotypic characteristics is about 85%, while for the AMTRF model obtained in this embodiment, the accuracy can be increased to 92% or more. By adopting depthwise separable convolution and lightweight channel attention mechanism, the number of model parameters and computational complexity are significantly reduced, making it suitable for deployment on resource-constrained devices. By adjusting the training data and parameters, the model can quickly adapt to the soil condition detection requirements of different crops, showing broad application prospects and good scalability. In practical applications, the present invention can effectively reduce the dependence on traditional soil sensors, significantly reducing the hardware cost and system complexity.

[0177] Through the above training, a soil environment prediction model can be obtained.

[0178] S5. Model deployment and detection.

[0179] Deploy the soil environment prediction model into the soil condition detection device, collect real-time leaf images for the aforementioned image preprocessing, and input the processing results into the soil environment prediction model to output the detection results of at least one soil environment parameter, thereby realizing convenient, fast and highly accurate soil condition detection.

[0180] On the other hand, this embodiment also provides a soil condition detection device based on plant leaf phenotypic characteristics for deploying the aforementioned soil environment prediction model to realize soil condition detection.

[0181] As Figure 2 shown, the device of this embodiment specifically runs all the main algorithms and control logics in the form of multi-module cooperation. The modules used include: data acquisition module, display control module, communication module. In addition, there are other functional modules for expanding the system functions, including automatic irrigation module, intelligent ventilation module, lighting management module, alarm and notification module, and AI module; the device configuration also includes a camera component and a display component.

[0182] Among them, the data acquisition module further includes an image acquisition module and an environment detection module.

[0183] The image acquisition module is connected to the camera component for collecting leaf images. The camera component uses a high-resolution camera to take images of the leaves of plant seedlings as input data for processing. The camera is controlled by Python code to take pictures and save the images.

[0184] The environment detection module is connected to the soil detection component for collecting corresponding soil environment data. The soil detection component further includes a soil temperature and humidity sensor, a light intensity sensor, an air temperature and humidity sensor, a soil pH sensor, a soil nitrogen, phosphorus and potassium sensor, a stem diameter sensor, and a wind speed sensor.

[0185] The display control module is connected to the display component. The display component uses a touch screen to display the system operating status, sensor data, and prediction results, and provides a user interaction interface on the display component, such as Figure 5 shown. The display component is a touch screen, through which real-time data can be viewed and system functions can be controlled.

[0186] Based on the core processing unit of the device, a data processing module is also provided, which is used to perform image preprocessing on the blade images and input the preprocessing results into the soil environment prediction model to output prediction results; the camera component is connected to the data processing module through the data acquisition module; the data processing module is also connected to the display component.

[0187] Furthermore, the device is also wirelessly connected to the cloud server through the communication module, and uploads the collected data and detection results to the cloud through the communication module. Personnel can remotely access and manage through the terminal; at the same time, personnel can also view system data, receive alerts, and remotely control the system through portable communication devices (such as mobile phones, tablets, customized terminals) in real time.

[0188] In addition, there are other functional modules for expanding system functions, including an automatic irrigation module, an intelligent ventilation module, a lighting management module, an alarm and notification module, and an AI module.

[0189] Among them, the automatic irrigation module automatically controls irrigation according to the data of the soil humidity sensor and the prediction results of the model; the intelligent ventilation module intelligently controls the ventilation system of the greenhouse according to the data of the air temperature, humidity, and air flow rate sensors; the lighting management module adjusts the brightness and working hours of the supplementary light according to the data of the light intensity sensor; the alarm and notification module, when the system detects an abnormal situation, pushes a notification to the user through the mobile application to take timely measures to prevent the crops from being damaged; the AI module deploys the front end of the general large language model (invoking the API of the large language model) on the device to provide natural language processing and interaction functions, and provides personalized planting suggestions and management strategies according to the environmental sensor data and user needs.

[0190] To meet the portable use conditions, such as Figure 4 shown, in the specific implementation process, an integrated design is carried out. Raspberry Pi 4 Model B is used as the core of the device, including a carrier board, a main control chip 707, a storage medium, and the main accessories are integrated inside its shell 706: a camera 705, a heat dissipation component 703, and a display component. This device has strong computing power and rich interfaces, and can meet the needs of image processing, model inference, and data display.

[0191] The camera 705 uses a high-resolution light sensor to capture images of plant leaves. The camera 705 is connected via the USB interface 708 of the Raspberry Pi to ensure the efficiency and stability of image acquisition. In the device, the onboard storage medium can be used to support the storage of captured images and detection results locally for a period of time, thereby supporting local information backtracking.

[0192] like Figure 5 As shown, the display component of the device is equipped with a touch screen 704, which is used to display the human-computer interaction interface of the system, providing the operating status, the captured leaf images, and the soil detection results predicted by the model. The interface design is simple and beautiful, ensuring that users can quickly understand and use it. The touch screen 704 is connected via the HDMI interface of the Raspberry Pi and provides interactive functions. Furthermore, in order to facilitate operation, a switch 701 for controlling shooting is separately set on the housing 706, so as to realize the function of manual shutter, and can take into account the function of power on / off by using the logic of long and short presses.

[0193] In addition, the device is equipped with a power charging port 702 and an energy storage component (such as a rechargeable lithium battery), so as to support long-term outdoor use without relying on an external power source. Simply aim the device at the plant leaves and press the shooting button to obtain the predicted results of the soil state within a few seconds. The operation is simple and fast. The power charging port 702 is a DC interface, which can be used with an adapter to connect to the mains to quickly charge the internal energy storage component.

[0194] Furthermore, the aforementioned communication module can specifically adopt a Wi-Fi or Bluetooth function module to wirelessly transmit data to a host computer for further data analysis or data distribution, and by utilizing wireless communication technology, the irrigation module, ventilation module, and light source module can be remotely controlled through the device to start and stop.

[0195] Working process:

[0196] The camera automatically focuses and captures high-resolution images to ensure that the image quality meets the model input requirements. After the image is processed, it is input into the CNN-PCA-RF model to extract leaf phenotypic features and predict soil conditions. The model outputs the current soil condition. The prediction results are displayed in real time on the touch screen, and detailed information on the current soil condition can be viewed through the interface.

[0197] It should be noted that in the above examples (such as Figure 5As shown (in the figure), taking soil humidity as the detection target is mainly for convenient demonstration and explanation. At the same time, in order to quickly understand the accuracy of the detection results in the initial stage, the device also collects the measured data of soil sensors in the area through the environmental detection module and feeds it back to the corresponding human-computer interaction interface. Thus, it can be understood that the technical solution of this application is not limited to the prediction of soil humidity. By adjusting the training data and model parameters, it is also applicable to predicting other soil state indicators, including soil pH, nutrient content, etc. And after the model completes the calibration test, it is not necessary to configure corresponding sensors in the actual site.

[0198] Example 1;

[0199] The trained model was deployed to a Raspberry Pi-based portable soil state detection device and field tests were conducted.

[0200] First, the portable device was used to take pictures of tomato leaves under different environmental conditions (including sunny days, cloudy days, and rainy days), and the prediction results of the soil state were obtained in real time. At the same time, a high-precision soil humidity sensor was used to measure the soil humidity of the same plot as the true value for comparison. Multiple tests were conducted at different time periods (including morning, noon, and evening) and different weather conditions (including sunny days, cloudy days, and rainy days), and a total of 500 groups of data were collected.

[0201] By comparing the predicted values of the AMTRF model with the true values of the soil humidity sensor, the prediction error and precision rate were calculated. The experimental results showed that the average prediction error of the AMTRF model was ±2.5%, and the precision rate was above 92%.

[0202] Specifically, under sunny conditions, the prediction error of the model was ±2.1% and the precision rate was 94.3%; under cloudy conditions, the prediction error was ±2.6% and the precision rate was 93.5%; under rainy conditions, the prediction error was ±2.8% and the precision rate was 92.8%. In addition, the root mean square error (RMSE) and coefficient of determination (R 2 ) were 0.028 and 0.946, indicating that the prediction results of the model had a high degree of consistency with the true values.

[0203] Through multiple field tests and data analysis, the prediction error of the AMTRF model under different environmental conditions was controlled within ±3.0%, the precision rate exceeded 92%, and the root mean square error (RMSE) and coefficient of determination (R 2 ) also reached a relatively high level. This indicates that the AMTRF model has high reliability and accuracy in practical applications and can meet the needs of precision agriculture for soil state monitoring. In addition, the real-time prediction function of the portable device was highly consistent with the measurement results of the soil humidity sensor, further verifying the feasibility and practicality of the device in practical applications.

[0204] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than limiting the protection scope of the present invention. Any simple modification or equivalent replacement made by those of ordinary skill in the art to the technical solution of the present invention shall not depart from the essence and scope of the technical solution of the present invention.

Claims

1. A method for detecting soil status based on plant leaf phenotypic characteristics, characterized in that, It includes the following steps: Collect historical leaf images and corresponding soil environment data; Perform image preprocessing on the historical leaf images and obtain a preprocessing dataset; at least contour extraction, feature extraction, and feature dimension reduction are provided in the steps of the image preprocessing; Based on the preprocessing dataset and the corresponding soil environment data, construct a model dataset; Train a random forest model through the model dataset to obtain a soil environment prediction model; Collect real-time leaf images for the image preprocessing, and input the processing results into the soil environment prediction model to output the detection results of at least one soil environment parameter.

2. The soil condition detection method based on plant leaf phenotypic characteristics according to claim 1, wherein The image preprocessing includes the following steps: The contour extraction: Remove the background information of the leaf image based on an image segmentation algorithm, and perform image enhancement through a local image enhancement algorithm; optimize the contour of the leaf image through a gradient vector flow algorithm; The feature extraction: Extract high-dimensional feature vectors of the leaf image with optimized contour through a convolutional neural network model; The feature dimension reduction: Reduce the dimension of the high-dimensional feature vectors through the principal component analysis method to obtain low-dimensional feature vectors.

3. The soil condition detection method based on plant leaf phenotypic characteristics according to claim 2, wherein The contour extraction further includes the following steps: Construct a leaf shape recognition model; Based on the leaf shape recognition model, output the shape recognition result of the leaf image, and the shape recognition result is the prior information of the leaf image; Set a prior addition term in the energy function of the gradient vector flow algorithm, and the prior addition term is the weighted prior information of the leaf image.

4. The soil condition detection method based on plant leaf phenotypic characteristics according to claim 2, wherein In the convolutional neural network model, extract the local and global features of the enhanced leaf image with optimized contour through multiple convolutional layers with different sizes, and dynamically set feature weights based on the channel attention mechanism.

5. The soil condition detection method based on plant leaf phenotypic characteristics according to claim 2, characterized in that, In the principal component analysis method, set a dynamic regularization term and set local structure constraints.

6. The soil condition detection method based on plant leaf phenotypic characteristics according to claim 1, characterized in that Set a multi-task learning mechanism in the random forest model and establish a correlation constraint between multi-tasks; The dynamic weights of the multi-tasks in the random forest model are set based on the dynamic changes of the model input data.

7. The method for detecting soil condition based on plant leaf phenotypic characteristics according to claim 1, characterized in that, The soil environment data includes one or a combination of soil temperature and humidity, air temperature and humidity, light intensity, wind speed, stem diameter, soil pH value, and soil electrical characteristic parameters.

8. The method for detecting soil condition based on plant leaf phenotypic characteristics according to claim 1, wherein The historical leaf images include images of a single leaf at different times, under different lighting conditions, and under different climate conditions, and at least two images taken from different shooting angles are also provided in the images of the single leaf.

9. A soil condition detection device based on the phenotypic characteristics of plant leaves, characterized in that, It includes a camera component, a display component, a data acquisition module, and a data processing module; The data acquisition module: is used to collect leaf images; The data processing module: is used to perform image preprocessing on the leaf images and input the preprocessing results into the soil environment prediction model to output detection results; The camera component is connected to the data processing module through the data acquisition module; the data processing module is also connected to the display component.

10. The soil condition detection device based on plant leaf phenotypic characteristics according to claim 9, wherein It further includes an energy storage component, a communication module, and a soil detection component; The energy storage component: is used to supply power to the soil status detection device; The communication module: is used to wirelessly transmit or receive data; The soil detection component: is used to collect corresponding soil environment data.

Images