Crop maturity recognition method based on deep learning

Abstract

The application provides a crop maturity recognition method based on deep learning, which realizes accurate recognition of crop maturity through the cooperative work of a multi-source data acquisition device, a data processing module, a data analysis module and a computer terminal; the specific steps include: acquiring appearance feature data through one-time data collection, data preprocessing, analyzing crop maturity probability based on the preprocessed data and collecting data for the second time, analyzing crop maturity according to the maturity feature data, generating and executing a crop harvesting scheme, and finally optimizing the recognition model through the trained model; the method combines the color feature, texture feature, morphological feature and time correction factor of crops to calculate the crop maturity probability, and when the maturity probability reaches the preset value, the subsequent steps are performed, thereby reducing the misjudgment risk caused by subjective bias of artificial experience and invalid data redundancy, and improving the accuracy and efficiency of crop maturity recognition.

Description

Technical Field

[0001] This invention belongs to the field of smart agriculture technology, and specifically discloses a crop maturity recognition method based on deep learning. Background Technology

[0002] Accurate identification of crop maturity is a core aspect of agricultural production. Accurate maturity identification can prevent premature or late harvesting, reducing yield losses due to misjudgment. Combined with maturity distribution data, irrigation and fertilization strategies can be adjusted accordingly, avoiding resource waste. Traditional crop maturity identification methods mainly rely on manual experience or single sensor data, which have the following drawbacks:

[0003] Highly subjective: It relies on human experience to make judgments. Farmers or experts make subjective assessments by observing characteristics such as color and shape with the naked eye. It is easily affected by lighting conditions and individual experience differences, resulting in a high error rate and making it impossible to achieve large-scale application.

[0004] Data limitations: Existing technologies often rely on single features, such as color, and lack multi-source feature fusion. Key indicators are not fully explored and cannot comprehensively reflect maturity.

[0005] Poor timeliness: Static models are difficult to adapt to dynamic changes in crop growth and environmental disturbances, resulting in low prediction accuracy;

[0006] Lack of feedback mechanism: Existing technologies are mostly one-way predictions, and do not use the actual results for model iteration and optimization, resulting in a decline in accuracy after long-term application;

[0007] Therefore, it is necessary to invent a crop maturity identification method based on deep learning to solve the above problems. Summary of the Invention

[0008] To overcome the aforementioned deficiencies in the prior art, this invention provides a deep learning-based crop maturity identification method. Through the collaborative work of multi-source data acquisition devices, a data processing module, a data analysis module, and a computer terminal, it achieves accurate identification of crop maturity. The specific steps include: acquiring appearance feature data in a primary data collection phase; data preprocessing; analyzing crop maturity probability based on the preprocessed data and conducting a secondary data collection phase; analyzing crop maturity based on maturity feature data; generating and executing a crop harvesting plan; and finally, optimizing the identification model through model training. This effectively solves the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a crop maturity identification method based on deep learning, comprising a multi-source data acquisition device, a data processing module, a data analysis module, and a computer terminal, specifically including the following steps:

[0010] S1. The multi-source data acquisition device performs a data acquisition to obtain the appearance feature data of crops in the area to be detected; the appearance feature data is crop images and radar cloud point data of the area to be detected.

[0011] S2. The data processing module processes the acquired appearance feature data to obtain preprocessed data.

[0012] S3. The data analysis module calculates the crop maturity probability of the area to be detected based on the preprocessed data. When the maturity probability reaches a preset value, the multi-source data acquisition device performs secondary data acquisition to obtain the maturity characteristic data of the crop in the area to be detected. The maturity characteristic data includes: the compliance rate of key chemical components that can reflect crop maturity and the compliance rate of key physical properties that can reflect crop maturity.

[0013] S4. The data analysis module calculates the crop maturity of the area to be detected based on the maturity characteristic data.

[0014] S5. The computer terminal generates a crop harvesting plan based on maturity and weather data, distributes the harvesting plan, and provides feedback on the harvesting results; the harvesting results include: crop quality grade and the percentage of mature crops.

[0015] S6. Using crop maturity as a label and chemical composition, physical properties, and harvest results as features, the model is trained to automatically learn and optimize the weight coefficients in the maturity analysis model calculation formula.

[0016] Based on the above embodiments, the maturity probability calculation method is as follows: In the formula, F c F is the color feature value. t F is the texture feature value. m ε(t) is the morphological feature value, ω is the time correction factor, and ω is the weight coefficient, which is dynamically adjusted based on the deep learning model.

[0017] Based on the above embodiments, the formula for calculating the color feature value is as follows: In the formula, A t S represents the total number of pixels in the crop image. g ΔH represents the number of pixels in the image that represent the typical color of the crop at the maturity stage, and ΔH is the angular difference between the mean H channel value of the current image and the standard H value of the mature sample.

[0018] Based on the above embodiments, the formula for calculating the texture feature value is as follows: , in the formula F is the entropy of multi-scale texture information extracted through local binary patterns. tmax The maximum texture feature value in historical data, Gabor maxHere, k represents the maximum Gabor filter energy, k represents different detection levels or neighborhood scales, and n represents the total number of different detection levels or neighborhood scales. en γ is the Gabor filter energy, and γ is the weighting coefficient;

[0019] Based on the above embodiments, the formula for calculating the morphological feature value is as follows: , N in the formula g N represents the number of mature fruits detected per unit canopy area. max A represents the theoretical maximum number of mature fruits per unit area. c D represents the effective area of ​​the canopy. c D represents the current number of days since conception. m σ represents the theoretical maturity days, and σ is the standard deviation of the Gaussian function used to adjust the morphological feature weights.

[0020] Based on the above embodiments, the formula for calculating the time correction factor is as follows: β is the environmental response rate, which is fitted using historical data.

[0021] Based on the above embodiments, the formula for calculating ΔH is as follows: In the formula, H m H is the standard H value for mature samples. c This represents the mean value of the H channels in the current image.

[0022] Based on the above embodiments, the maturity calculation formula for the region to be detected is as follows: In the formula, n represents the number of chemical components that reflect crop maturity, m represents the number of physical characteristics that reflect crop maturity, and C... a To reflect the compliance rate of chemical components in crops, P b To reflect the rate of compliance with physical characteristics that indicate crop maturity, The weighting coefficients corresponding to the chemical composition compliance rate that reflect crop maturity are as follows: The weighting coefficients are used to reflect the rate at which the physical characteristics of crop maturity meet the standards. and The sum is 1.

[0023] The technical effects and advantages of this invention are as follows:

[0024] 1. By combining multi-source data such as crop color features, texture features, morphological features and time correction factors, and using a deep learning model to dynamically adjust the weights of each feature, this invention can comprehensively reflect the complex changing patterns of crop maturity. Compared with traditional methods that rely on a single feature, it significantly reduces the risk of misjudgment caused by the one-sidedness of features and improves the accuracy and reliability of maturity identification.

[0025] 2. A phased data acquisition strategy is adopted, with a primary acquisition of appearance features and a secondary acquisition of maturity features. The secondary data acquisition is triggered only when the maturity probability reaches a preset threshold. This mechanism effectively avoids redundant acquisition and processing of invalid data, reduces the waste of computing resources, and further improves analysis efficiency and system response speed by focusing on key maturity feature data.

[0026] 3. By using actual harvest results along with data such as chemical composition and physical properties as model training features, and combining this with dynamic weight adjustment, a closed-loop system of "data acquisition—analysis—execution—feedback" is formed. This mechanism enables the model to continuously adapt to dynamic changes in crop growth and environmental disturbances, maintaining high prediction accuracy over the long term, and solving the problems of poor timeliness and declining long-term application performance of traditional static models. Attached Figure Description

[0027] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0029] Figure 2 This is a flowchart illustrating the overall steps of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention provides, for example Figure 1 The crop maturity identification method based on deep learning shown is characterized by including a multi-source data acquisition device, a data processing module, a data analysis module, and a computer terminal;

[0032] In a more specific application of the present invention, the multi-source data acquisition device is used to acquire appearance feature data and maturity feature data required for primary and secondary data acquisition. The appearance feature data is crop images of the area to be detected, and the maturity feature data includes: the compliance rate of key chemical components that reflect crop maturity and the compliance rate of key physical properties that reflect crop maturity. For example, taking corn as the detection target, the compliance rate of the key chemical components specifically includes: the compliance rate of starch and moisture content of corn kernels, and the compliance rate of key physical properties specifically includes: the compliance rate of corn kernel hardness, the compliance rate of 100-kernel weight, and the compliance rate of color change. The multi-source data acquisition device can specifically be a combination of a high-resolution RGB camera, a near-infrared spectrometer, a lidar, an environmental sensor, and a dielectric sensor.

[0033] The data processing module is responsible for image denoising, normalization, color space conversion (e.g., RGB to HSV), and spectral data filtering; it uses the OpenCV library to perform Gaussian filtering for denoising, crop redundant backgrounds, and extract plant ROI regions; it converts the image to HSV space and calculates the mean value of the H channel; it normalizes the image pixel values ​​to [0, 1] and standardizes the spectral data according to the historical maximum value.

[0034] The data analysis module is used to calculate the crop maturity probability and crop maturity degree of the area to be detected based on the preprocessed data after preprocessing by the data processing module.

[0035] The computer terminal is used to generate crop harvesting plans based on maturity and weather data, distribute the harvesting plans, and provide feedback on the harvesting results.

[0036] like Figure 2 As shown, the specific steps include the following:

[0037] S1. The multi-source data acquisition device performs a data acquisition to obtain the appearance feature data of crops in the area to be detected.

[0038] In a preferred embodiment of this application, the appearance feature data consists of crop images of the area to be detected and radar cloud point data.

[0039] Furthermore, the crop images of the area to be detected are obtained by using an RGB camera, and the radar cloud point data are obtained by scanning the canopy structure with a lidar.

[0040] S2. The data processing module processes the acquired appearance feature data to obtain preprocessed data.

[0041] In a preferred embodiment of this application, the specific process for processing the acquired appearance feature data is as follows: Gaussian filtering is performed using the OpenCV library for noise reduction; redundant background is cropped, and the plant ROI region is extracted; the image is converted to HSV space, the mean of the H channel is calculated, and the number of pixels S representing the typical colors of the crop maturity stage in the image is counted. g The image pixel values ​​are normalized to [0, 1]. Statistical outlier filtering using the PCL library is used to remove flying point noise. A canopy height model is generated based on cloud point data, and the effective canopy area A is calculated. c Density clustering was used to identify fruit point cloud clusters and count the number of clusters per unit area, which was then used as N. g The estimated value;

[0042] S3. The data analysis module calculates the crop maturity probability in the area to be detected based on the preprocessed data. When the maturity probability reaches the preset value, the multi-source data acquisition device performs secondary data acquisition to obtain the maturity characteristic data of the crop in the area to be detected.

[0043] In a preferred embodiment of this application, the preset value is 0.4, and the theoretical value of the maturity probability is in the range [0, 0.5].

[0044] In a preferred embodiment of this application, the maturity probability is calculated as follows: In the formula, F c F is the color feature value. t For texture feature values, F m ε(t) is the morphological feature value, ω is the time correction factor, and ω is the weight coefficient, which is dynamically adjusted based on the deep learning model.

[0045] In a preferred embodiment of this application, the formula for calculating the color feature value is as follows: In the formula, A t S represents the total number of pixels in the crop image. g ΔH represents the number of pixels in the image that represent the typical color of the crop at the maturity stage, and ΔH is the angular difference between the mean H channel value of the current image and the standard H value of the mature sample.

[0046] In a preferred embodiment of this application, the formula for calculating the texture feature value is as follows: , in the formula F is the entropy of multi-scale texture information extracted through local binary patterns. tmax The maximum texture feature value in historical data, Gabor max Here, k represents the historical maximum energy of the Gabor filter, and n represents the total number of different detection levels or neighborhood scales. enLet γ be the Gabor filter energy, and γ be the weighting coefficient. For cereal crops, γ∈[0.3,0.4] has universality.

[0047] In a preferred embodiment of this application, the formula for calculating the morphological feature value is as follows: , N in the formula g N represents the number of mature fruits detected per unit canopy area. max A represents the theoretical maximum number of mature fruits per unit area. c D represents the effective area of ​​the canopy. c D represents the current number of days since conception. m σ represents the theoretical maturity days, and σ is the standard deviation of the Gaussian function used to adjust the morphological feature weights.

[0048] In a preferred embodiment of this application, the formula for calculating the time correction factor is as follows: β is the environmental response rate, which is fitted using historical data.

[0049] In a preferred embodiment of this application, the formula for calculating ΔH is: In the formula, H m H is the standard H value for mature samples. c This represents the mean value of the H channels in the current image.

[0050] In the preferred embodiment of this application, the maturity characteristic data of the crop in the area to be detected specifically include: starch compliance rate, moisture compliance rate, hardness compliance rate, 100-kernel weight compliance rate, and color change compliance rate of corn kernels;

[0051] Furthermore, the starch compliance rate was calculated by matching the spectral characteristics detected by the near-infrared spectrometer with the starch content calibration model. The moisture compliance rate was calculated by combining the near-infrared spectrometer with the dielectric sensor through the linear relationship between the dielectric constant and the moisture content. The hardness compliance rate was calculated by direct sampling and detection using a portable hardness tester. The compliance rate of 100-grain weight was obtained by manual sampling and weighing.

[0052] Example of calculating crop maturity probability:

[0053] Suppose the test data for a cornfield is as follows:

[0054] Image total pixels A t =480000, number of mature color pixels (S) g =218400, current H channel mean H c =32°, mature sample standard H value H m =35°, using 3 detection scales, i.e., n=3, k=1, the multi-scale texture information entropy value is 4.2, k=2, the multi-scale texture information entropy value is 5.1, k=3, the multi-scale texture information entropy value is 3.8, and the maximum texture feature value F in historical data.tmax =10, Gabor filter energy Gabor en =38.6, the historical maximum value of Gabor filter energy. max =82.4, weight γ=0.35, number of mature fruits N g =22560, the theoretical maximum number of mature fruits per unit area, N max =520, canopy area A c =48m², current number of days of maternity leave D c =102, Theoretical Maturity Days D m =115, Gaussian function standard deviation σ=15, environmental response rate β=0.08, color feature weight ω c =0.4, texture feature weight ω t =0.3, morphological feature weight ω m =0.3;

[0055] Calculation process:

[0056]

[0057] Color characteristic value:

[0058] ,

[0059] Texture feature values:

[0060] F t =(4.2 / 1 2 +5.1 / 2 2 +3.8 / 3 2 ) / 10 + 0.35 × (38.6 / 82.4) = 0.59 + 0.164 ≈ 0.754,

[0061] Morphological characteristic values:

[0062] F m =(22560 / (520×48))×exp[-(102-115) 2 / (2×15 2 )]≈0.904×0.687≈0.621,

[0063] Time correction factor:

[0064] ,

[0065] Probability of maturity:

[0066] P m =[(0.447×0.4)+(0.3×0.754)+(0.3×0.621)]×(1-0.326)=

[0067] (0.179+0.226+0.186)×0.674≈0.40;

[0068] Conclusion: P m =0.4, the current maturity probability has reached the target, triggering a second data collection.

[0069] S4. The data analysis module calculates the crop maturity of the area to be detected based on the maturity characteristic data.

[0070] In a preferred embodiment of this application, the maturity calculation formula for the region to be detected is as follows: In the formula, n represents the number of chemical components that reflect crop maturity, m represents the number of physical characteristics that reflect crop maturity, and C... a To reflect the compliance rate of chemical components in crops, P b To reflect the rate of compliance with physical characteristics that indicate crop maturity, The weighting coefficients corresponding to the chemical composition compliance rate that reflect crop maturity are as follows: The weighting coefficients are used to reflect the rate at which the physical characteristics of crop maturity meet the standards. and The sum is 1.

[0071] Furthermore, the chemical composition compliance rate includes: starch compliance rate and moisture compliance rate of corn kernels; the physical characteristics compliance rate that can reflect crop maturity includes: hardness compliance rate, 100-kernel weight compliance rate and color change compliance rate.

[0072] Example of maturity calculation:

[0073] Suppose the test data for a cornfield is as follows:

[0074] Starch compliance rate: 85%, weighting coefficient corresponding to starch compliance rate: 0.3.

[0075] Moisture content compliance rate: 90%, weighting coefficient corresponding to moisture content compliance rate: 0.2.

[0076] Hardness compliance rate: 88%, weighting coefficient corresponding to hardness compliance rate: 0.15.

[0077] 100-kernel weight compliance rate: 92%, weighting coefficient corresponding to 100-kernel weight compliance rate: 0.2.

[0078] Color transformation compliance rate: 95%, weighting coefficient corresponding to color transformation compliance rate: = 0.15;

[0079] Calculation process:

[0080] M=0.85×0.3+0.9×0.2+0.88×0.15+0.92×0.2+0.95×0.15=0.8935;

[0081] Conclusion: The current corn maturity rate in the tested area is 89.35%, indicating that the crop is close to full maturity.

[0082] S5. The computer terminal generates a crop harvesting plan based on maturity and weather data, distributes the harvesting plan, and provides feedback on the harvesting results.

[0083] In a preferred embodiment of this application, the crop harvesting method includes:

[0084] When 0 < M ≤ 40%, it indicates that the crops in the current area are not mature and harvesting needs to be delayed, and field management needs to be strengthened.

[0085] When 40% < M ≤ 70%, it means that the crops in the current area are partially mature. If the weather is stable in the future, we will continue to wait for them to mature. If there is severe weather, we will selectively harvest mature crops in batches.

[0086] When 70% < M ≤ 90%, it indicates that the crops in the current area are mature, and mechanical / manual harvesting should be organized immediately.

[0087] When 90% < M ≤ 100%, it indicates that the crops in the current area are overripe. Mechanical and manual labor should work together to complete the harvest as soon as possible.

[0088] S6. Using crop maturity as a label and maturity characteristic data and harvest results as features, the model is trained to automatically learn and optimize the weight coefficients in the maturity analysis model calculation formula.

[0089] In the preferred embodiment of this application, the harvest results specifically include crop quality grade and the percentage of mature crops;

[0090] Furthermore, the process of automatically learning and optimizing the weight coefficients in the maturity analysis model calculation formula through model training is as follows:

[0091] The model automatically learns the influence strength of each feature on maturity through training, introduces an attention mechanism to dynamically allocate weights, and adds the quality grade of each harvest and the proportion of mature crops as new features to the training set; if the deviation between the predicted maturity and the measured quality is >10%, the model is triggered to retrain.

[0092] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A crop maturity recognition method based on deep learning, characterized by, The application relates to a multi-source data acquisition device, a data processing module, a data analysis module and a computer terminal, and specifically comprises the following steps: S1. The multi-source data acquisition device performs first data acquisition to obtain appearance feature data of crops in a detection area; the appearance feature data is crop pictures and radar cloud point data of the detection area; S2. The data processing module processes the obtained appearance feature data to obtain preprocessed data; S3. The data analysis module calculates the crop maturity probability of the detection area according to the preprocessed data, when the maturity probability reaches a preset value, the multi-source data acquisition device performs second data acquisition to obtain maturity feature data of crops in the detection area; the maturity feature data includes a key chemical component standard reaching rate which can reflect the crop maturity and a key physical property standard reaching rate which can reflect the crop maturity; S4. The data analysis module calculates the crop maturity of the detection area according to the maturity feature data; S5. The computer terminal generates a crop harvesting scheme according to the maturity and weather data, issues the harvesting scheme and feeds back a harvesting result; the harvesting result includes a crop quality grade and a mature crop proportion; S6. The crop maturity is taken as a label, chemical components, physical properties and harvesting result contents are taken as features, a training model is used to automatically learn and optimize weight coefficients in a maturity analysis model calculation formula. 2.The deep learning-based crop maturity identification method of claim 1, wherein: The mature probability calculation mode is: , in the formula F c is a color feature value, F t is a texture feature value, F m is a morphological feature value, and epsilon(t) is a time correction factor, and omega is a weight coefficient, which is dynamically adjusted based on a deep learning model. 3.The deep learning-based crop maturity identification method of claim 2, wherein: The color characteristic value is calculated by the following formula: , wherein A t is the total number of pixels of the crop image, S g is the number of pixels of the typical color of the crop in the mature stage in the crop image, and ΔH is the angle difference between the average value of the H channel of the current image and the standard H value of the mature sample. 4.The deep learning-based crop maturity identification method of claim 2, wherein: The formula for calculating the texture feature value is: , wherein is the multi-scale texture information entropy extracted by the local binary pattern, F tmax is the maximum value of the texture feature value in the historical data, Gabor max is the maximum value of the Gabor filter energy, k represents different detection levels or neighborhood scales, n is the total number of different detection levels or neighborhood scales, Gabor en is the Gabor filter energy, and γ is a weight coefficient. 5.The deep learning-based crop maturity identification method of claim 2, wherein: The calculation formula of the morphological characteristic value is: , wherein N g is the number of mature fruits detected in a unit canopy area, N max is the maximum number of theoretical mature fruits per unit area, A c is the effective area of the canopy, D c is the current growth days, D m is the theoretical mature days, and σ is the standard deviation of a Gaussian function for adjusting the weight of the morphological characteristics. 6.The deep learning-based crop maturity identification method of claim 2, wherein: The calculation formula of the time correction factor is: β is the environmental response rate, fitted by historical data. 7.The deep learning-based crop maturity identification method of claim 3, wherein: The calculation formula of the ΔH is: , in which H m is the standard H value of the mature sample, and H c is the H channel average value of the current image. 8.The deep learning-based crop maturity identification method of claim 1, wherein: The maturity calculation formula of the region to be detected is: In the formula, n is the number of chemical components reflecting the maturity of the crops, m is the number of physical properties reflecting the maturity of the crops, C a is the standard rate of chemical components reflecting the maturity of the crops, P b is the standard rate of physical properties reflecting the maturity of the crops, is the weight coefficient corresponding to the standard rate of chemical components reflecting the maturity of the crops, is the weight coefficient corresponding to the standard rate of physical properties reflecting the maturity of the crops, and The sum is 1.

Images