Detection method, device and system for crop growth period and storage medium

Abstract

The embodiment of the invention provides a crop growth period detection method, device and system and a storage medium. The method comprises the following steps: acquiring a first crop image and a second crop image of crops in an agricultural area at a first focal length and a second focal length at a preset time node; preprocessing the first and second crop images to obtain a plurality of first and second sub-crop images; the plurality of first sub-crop images and the plurality of second sub-crop images are respectively input into a heading and flowering rotation detection model so as to output a detection result, and the heading and flowering rotation detection model is constructed by taking an object detection segmentation model architecture as a basic model architecture and comprises a feature extraction module, a middle layer module and a classifier module which are connected in sequence; the feature extraction module comprises a RepVGG-B2 convolutional network, a depth separable convolutional network and a parameter-free attention module, the intermediate layer module comprises a plurality of parameter-free attention modules, and the classifier module comprises a plurality of adaptive feature fusion ASFF modules; determining whether the growth period of the crop is an initial heading period or a flowering period according to a detection result.

Description

Technical Field

[0001] The present application relates to the field of agricultural technology, and in particular to a method, device, system and storage medium for detecting the growth period of crops. Background Art

[0002] Rice is one of my country's three major staple crops, and its yield and quality are highly valued. To maintain and improve rice yield and quality, accurate monitoring, identification, and analysis of the rice growth period are essential, leveraging information from rice fields. The entire rice growth period includes transplanting, greening, tillering, jointing, heading, flowering, and maturity. The rice growth period is crucial agricultural information, and identifying key growth periods plays a crucial role in rice field production and management. By monitoring rice growth stages, farmers can implement pest and disease control measures during critical periods, minimizing losses and ensuring high-quality rice production.

[0003] Rice heading and flowering are two growth periods that are difficult to observe. Existing technologies usually use phenotypes such as average temperature and sunshine hours in the planting area, combined with their own constructed mathematical models, to predict the heading and flowering periods of rice at different latitudes. However, this prediction method is actually invisible and intangible, and is greatly affected by environmental factors, making the prediction of rice heading and flowering periods inaccurate. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, system and storage medium for detecting the growth period of crops, so as to solve the technical defect of inaccurate prediction of the heading and flowering periods of rice in the prior art.

[0005] In order to achieve the above-mentioned objectives, the present application provides, in a first aspect, a method for detecting the growth period of crops, the detection method comprising:

[0006] Acquire a first crop image of crops in an agricultural area at a first focal length and a second crop image at a second focal length at a preset time point, wherein the first focal length is smaller than the second focal length;

[0007] Preprocessing the first crop image and the second crop image respectively to obtain a plurality of first sub-crop images and a plurality of second sub-crop images;

[0008] Inputting a plurality of first sub-crop images and a plurality of second sub-crop images into a spikelet and flowering rotation detection model respectively, so as to output a detection result through the spikelet and flowering rotation detection model, wherein the spikelet and flowering rotation detection model is constructed based on an object detection and segmentation model architecture, and the spikelet and flowering rotation detection model includes a feature extraction module, an intermediate layer module, and a classifier module connected in sequence, wherein the feature extraction module includes a RepVGG-B2 convolutional network, a depthwise separable convolutional network, and a parameter-free attention module, the intermediate layer module includes multiple parameter-free attention modules, and the classifier module includes multiple adaptive feature fusion (ASFF) modules;

[0009] According to the test results, it is determined that the growth period of the crop is the heading stage or the flowering stage.

[0010] In an embodiment of the present application, determining whether the growth period of a crop is the heading stage or the flowering stage based on the detection result includes: when the detection result is the crop area of the heading crop in the first sub-crop image, determining the area ratio between the sum of the crop areas of the heading crops in multiple first sub-crop images and the image area of the first crop image; when the area ratio reaches a first preset threshold, determining that the growth period of the crop enters the heading stage.

[0011] In an embodiment of the present application, the detection method also includes: when the area ratio reaches a second preset threshold, determining that the growth period of the crop enters the heading period, wherein the second preset threshold is greater than the first preset threshold; when the area ratio reaches a third preset threshold, determining that the growth period of the crop enters the full heading period, wherein the third preset threshold is greater than the second preset threshold.

[0012] In an embodiment of the present application, determining whether the growth period of a crop is the heading stage or the flowering stage based on the detection results includes: when the detection result is any marked second sub-crop image, determining that the growth period of the crop enters the flowering stage, wherein any marked second sub-crop image is generated after the heading and flowering rotation detection model marks any crop in the flowering state when it is detected that any crop in the second sub-crop image is in the flowering state.

[0013] In an embodiment of the present application, the detection method also includes: when it is determined that the growth period of the crop has entered the heading stage, the preset time node is used to determine the starting node of the heading stage; when it is determined that the growth period of the crop has entered the flowering stage, the preset time node is used to determine the starting node of the flowering stage.

[0014] In an embodiment of the present application, the detection method also includes a training step of an earing and flowering rotation detection model, and the training step includes: obtaining a historical first crop image of the agricultural area at a first focal length during the historical earing period and a historical second crop image at a second focal length during the historical flowering period; preprocessing the historical first crop image and the historical second crop image respectively to obtain a plurality of historical first sub-crop images and a plurality of historical second sub-crop images; dividing the plurality of historical first sub-crop images and the plurality of historical second sub-crop images into a first training set, a first test set, a second training set and a second test set according to a preset ratio; taking the object detection and segmentation model architecture as the basic model, and constructing a training set to be trained according to the binary cross entropy function, the KL divergence function, the ProbIoU function and the DFL function. The method comprises the following steps: pre-training and testing the ear and flower rotation detection model to be trained through the first training set and the first test set, and pre-training and testing the ear and flower rotation detection model to be trained through the second training set and the second test set; for any historical first sub-crop image in the first test set, when the deviation value between the loss value of any historical first sub-crop image and the loss value of the previous first sub-crop image meets the first preset deviation value, and for any historical second sub-crop image in the second test set, when the second deviation value between the loss value of any historical second sub-crop image and the loss value of the previous historical second sub-crop image meets the preset bias interpolation value, it is determined that the trained ear and flower rotation detection model is obtained.

[0015] In an embodiment of the present application, the detection method also includes: determining the binary cross entropy function as the classification loss function of the ear and flowering rotation detection model to be trained, and determining the KL divergence function, ProbIoU function and DFL function as the rotation bounding box regression loss function of the ear and flowering rotation detection model to be trained, so as to obtain the total loss function of the ear and flowering rotation detection model.

[0016] In an embodiment of the present application, the binary cross entropy function is determined as the classification loss function of the ear and flowering rotation detection model to be trained, and the KL divergence function, ProbIoU function and DFL function are determined as the rotation bounding box regression loss function of the ear and flowering rotation detection model to be trained, so as to obtain the total loss function of the ear and flowering rotation detection model as shown in formula (1):

[0017] Total_Loss=BCE_Loss+λ1·KLD_Loss+λ2·ProbIoU_Loss+λ3·DFL_Loss(1)

[0018] Among them, Total_Loss is the total loss function, BCE_Loss is the binary cross entropy function, KLD_Loss is the KL divergence function, ProbIoU_Loss is the ProbIoU function, DFL_Loss is the DFL function, and λ1, λ2 and λ3 are constant coefficients.

[0019] In an embodiment of the present application, the intermediate layer module also includes multiple Concat modules for connecting to the feature extraction module, and the parameter-free attention module is used between the feature extraction module and the Concat module.

[0020] In an embodiment of the present application, the intermediate layer module also includes multiple C2F2_3d neural network modules, and each adaptive feature fusion ASFF module in the multiple adaptive feature fusion ASFF modules is used to weightedly sum the different features of each sub-crop image output by the corresponding connected C2F2_3d neural network module to obtain a feature map after the fusion of features.

[0021] In an embodiment of the present application, the ear formation and flowering rotation detection model includes an initial ear detection model and a flowering detection model. The initial ear detection model is used to output the detection results of multiple first sub-crop images, and the flowering detection model is used to output the detection results of multiple second sub-crop images.

[0022] A second aspect of the present application provides a detection device for determining the growth period of a crop, comprising:

[0023] a memory configured to store instructions;

[0024] The processor is configured to call the instructions from the memory and implement the above-mentioned method for detecting the growth period of crops when executing the instructions.

[0025] A third aspect of the present application provides a detection system for determining the growth period of crops, comprising the above-mentioned detection device for determining the growth period of crops.

[0026] A fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon. When the instructions are executed by a processor, the processor is configured to execute the above-mentioned method for the crop growth period.

[0027] The above technical solution obtains a first crop image of crops in an agricultural area at a first focal length and a second crop image at a second focal length at a preset time point, wherein the first focal length is smaller than the second focal length; preprocesses the first crop image and the second crop image to obtain multiple first sub-crop images and multiple second sub-crop images; and inputs the multiple first sub-crop images and multiple second sub-crop images into a heading and flowering rotation detection model to output detection results. The heading and flowering rotation detection model is constructed based on the object detection and segmentation model architecture and includes a feature extraction module, an intermediate layer module, and a classifier module connected in sequence. The feature extraction module includes a RepVGG-B2 convolutional network, a depthwise separable convolutional network, and a parameter-free attention module. The intermediate layer module includes multiple parameter-free attention modules. The classifier module includes multiple adaptive feature fusion (ASFF) modules. The crop growth period is determined to be the heading stage or the flowering stage based on the detection results. This method effectively improves the prediction accuracy of the heading stage and flowering stage.

[0028] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present application but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0030] Figure 1 The following schematically shows a flow chart of a method for detecting the growth period of crops according to an embodiment of the present application;

[0031] Figure 2 A schematic diagram of capturing rice images using a camera with a variable-focus function according to an embodiment of the present application is shown;

[0032] Figure 3 The following schematically shows a structural diagram of a model for detecting ear formation, flowering and rotation according to an embodiment of the present application;

[0033] Figure 4 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0034] To make the purpose, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application and are not intended to limit the embodiments of the present application. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present application without making creative efforts are within the scope of protection of this application.

[0035] In addition, if there are descriptions involving "first" and "second" in the embodiments of the present application, the descriptions of "first" and "second" are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0036] Figure 1 The following schematically shows a flow chart of a method for detecting the growth period of crops according to an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a method for detecting the growth period of crops, which may include the following steps:

[0037] Step 101 : acquiring a first crop image of crops in an agricultural area at a first focal length and a second crop image at a second focal length at a preset time point, wherein the first focal length is smaller than the second focal length.

[0038] In the embodiments of the present application, it should be noted that the agricultural area refers to an area engaged in agricultural production activities, generally refers to an area centered on farming and mainly planting food crops and cash crops, also known as an agricultural area. Crops refer to crops planted in an agricultural area, such as rice, wheat, etc. The preset time node can be arbitrarily selected according to actual production needs, in the format of a certain year, month, day, and time. The smaller the focal length, the wider the framing range, the wider the field of view of the captured image, and the smaller the proportion of the scenery in the image. The larger the focal length, the narrower the framing range, the smaller the field of view of the captured image, and the larger the proportion of the scenery in the image. In this technical solution, the rice ears are more obvious, so the first focal length can be set to far focus for shooting the rice ears. At the same time, since the rice flowers are small and not obvious, the second focal length can be set to near focus for shooting the flowering of the crop. Taking the crop as rice and the preset time node as 8:00 on August 8, 2008 as an example, the processor can respectively obtain images of the rice in the agricultural area at a first focal length and a second focal length at 8:00 on August 8, 2008.

[0039] It should be understood that in order to obtain an image of the target object, it is usually necessary to obtain it through an image acquisition device. Taking rice as an example, the states of rice such as the beginning of earing and flowering, which are highly delicate features, need to be observed from different distances of the rice plant. Therefore, in this technical solution, in order to be able to capture the details of the crop from different observation distances, a zoom function camera can be selected as the image acquisition device. The zoom function camera is a device that can automatically adjust the focal length according to demand to capture the details of the crop plant. Taking a single piece of rice farmland as an example, if Figure 2 The figure shows a schematic diagram of a variable-focus camera capturing images of rice fields. Specifically, after installing the variable-focus camera in a field, the camera can be set to a first focal length of far focus and a second focal length of near focus to capture images of rice fields. The images captured at the first focal length can be used to detect rice ear formation, while the images captured at the second focal length can be used to detect rice flowering.

[0040] Step 102 : Preprocess the first crop image and the second crop image respectively to obtain a plurality of first sub-crop images and a plurality of second sub-crop images.

[0041] In the embodiment of the present application, it should be noted that preprocessing may refer to image cropping. Specifically, after the processor obtains the first crop image at the first focal length and the second crop image at the second focal length, the processor may perform image cropping on both the first crop image and the second crop image. For example, the original image may be screenshoted with a resolution of 1024x1024 to obtain four sub-images at the upper left corner, upper right corner, lower left corner, and lower right corner, respectively.

[0042] Step 103, respectively input multiple first sub-crop images and multiple second sub-crop images into the ear and flowering rotation detection model to output detection results through the ear and flowering rotation detection model. The ear and flowering rotation detection model is constructed based on the object detection and segmentation model architecture. The ear and flowering rotation detection model includes a feature extraction module, an intermediate layer module and a classifier module connected in sequence. The feature extraction module includes a RepVGG-B2 convolutional network, a depthwise separable convolutional network and a parameter-free attention module. The intermediate layer module includes multiple parameter-free attention modules. The classifier module includes multiple adaptive feature fusion ASFF modules.

[0043] In the embodiment of the present application, it should be noted that after the processor obtains a plurality of first sub-crop images and a plurality of second sub-crop images by cropping, the plurality of first sub-crop images and the plurality of second sub-crop images can be respectively input into the ear and flowering rotation detection model to output the detection results through the ear and flowering rotation detection model. Among them, the ear and flowering rotation detection model is constructed with the object detection segmentation model architecture as the basic model architecture. The object detection segmentation model can refer to yolov8, which is the latest version of the yolo deep learning algorithm series. Its core idea is to transform the target detection problem into a regression problem, and directly predict the category and position of the target through a single convolutional neural network, thereby realizing real-time target detection.

[0044] In this technical solution, if Figure 3 As shown, a schematic diagram of the structure of a knot, blossom, and rotation detection model is provided. The knot, blossom, and rotation detection model may include a feature extraction module, an intermediate layer module, and a classifier module connected in sequence. The feature extraction module may refer to the Backbone module, whose primary function is to extract high-level features from the input data. These features are then passed to subsequent network layers to complete specific tasks such as classification, detection, and segmentation. The intermediate layer module may refer to the Neck module, which further processes and fuses the features extracted by the Backbone module to improve the feature representation and enhance the model's detection capabilities for objects of different scales. The classifier module may refer to the Head module, which is the last layer in a deep learning model and is responsible for mapping features processed by the Backbone and Neck modules to the final output space. The structure of the Head module may vary depending on the task. For example, in image classification tasks, the Head module is typically a fully connected layer or a softmax classifier. In object detection tasks, the Head module may include a bounding box regressor and a classifier.

[0045] It should be noted that the Backbone module may include a RepVGG-B2 convolutional network, a deep separable convolutional network and a parameter-free attention module. Compared with the traditional Backbone module structure, this technical solution selects RepVGG-B2 to replace the ordinary convolution in the traditional Backbone module to improve the Backbone module's detection performance for small objects. The deep separable convolutional network may refer to a Sep-Conv network, and the parameter-free attention module may refer to a SimAM attention module. Specifically, SimAM is a similarity-based attention module that can selectively weight the input information, thereby improving the accuracy of the attention mechanism while ensuring computational efficiency. The Neck module may include multiple SimAM attention modules, specifically, such as Figure 3 As shown in the figure, the Neck module can include three SimAM attention modules. The Head module can include multiple adaptive feature fusion (ASFF) modules. The ASFF module can be used to solve the scale variation problem in target detection. ASFF solves the inconsistency between different feature scales by adaptively fusing feature maps of different scales, thereby enhancing the scale invariance of features.

[0046] In an embodiment of the present application, the intermediate layer module also includes multiple Concat modules for connecting to the feature extraction module, and the parameter-free attention module is used between the feature extraction module and the Concat module.

[0047] In this embodiment, it should be noted that Figure 3 As shown in the figure, the Neck module can also include multiple Concat modules for connecting with the feature extraction module, and the SimAM attention module is used between the feature extraction module and the Concat module. The Concat module is used to combine feature maps at different levels to improve the accuracy and efficiency of target detection. By splicing feature maps of different scales, the Concat module can capture more contextual information and enhance the model's ability to understand the target.

[0048] In an embodiment of the present application, the intermediate layer module also includes multiple C2F2_3d neural network modules, and each adaptive feature fusion ASFF module in the multiple adaptive feature fusion ASFF modules is used to weightedly sum the different features of each sub-crop image output by the corresponding connected C2F2_3d neural network module to obtain a feature map after the fusion of features.

[0049] In this embodiment, it should be noted that the Neck module can also include three C2F2_3d neural network modules. Therefore, the Head module can include three adaptive feature fusion ASFF modules corresponding to each C2F2_3d neural network module. Each adaptive feature fusion ASFF module can be used to perform weighted summation on the different features of each sub-crop image output by the corresponding connected C2F2_3d neural network module to obtain a feature map after the fusion of features.

[0050] It should be noted that, taking the ASFF-1 module as an example:

[0051] By performing weighted summation on the first three C2F2_3d (80x80, 40x40, 20x20) outputs with different features, the calculation formula is as follows:

[0052] y=αx′1+βx′2+γx′3

[0053] Among them, the feature fusion in ASFF adopts the direct addition method, and the feature sizes of X1, X2, and X3 output from the three C2F2_3d structures are different, so it is necessary to make the size and number of channels of the feature map the same by upsampling or downsampling. That is, in the above formula, x1 ′ 、x2 ′ 、x ′ 3 is the feature map of X1, X2, and X3 after upsampling or downsampling (X1 does not need to be upsampled in ASFF-1). Then, the features from different layers are multiplied by the weight parameters α, β, and γ and added together to obtain the new fused feature ASFF-1. Similarly, ASFF-2 and ASFF-3 can be obtained.

[0054] The weight parameters α, β, and γ are obtained by performing a 1×1 convolution on the resized feature maps of X1, X2, and X3. The parameters α, β, and γ are concat-based and then softmaxed to ensure that they are in the range [0, 1] and sum to 1.

[0055] In an embodiment of the present application, the ear formation and flowering rotation detection model includes an initial ear detection model and a flowering detection model. The initial ear detection model is used to output the detection results of multiple first sub-crop images, and the flowering detection model is used to output the detection results of multiple second sub-crop images.

[0056] In this embodiment, it should be noted that the earing-flowering rotation detection model may include a beginning-of-earing detection model and a flowering detection model, wherein the beginning-of-earing detection model and the flowering detection model have the same structure but different parameters. Specifically, the multiple first sub-crop images are input only to the beginning-of-earing detection model, which outputs detection results for the multiple first sub-crop images. The multiple second sub-crop images are input only to the flowering detection model, which outputs detection results for the multiple second sub-crop images.

[0057] Step 104: Determine whether the crop's growth period is the heading stage or the flowering stage based on the detection result.

[0058] In the embodiment of the present application, it should be noted that after the processor inputs multiple first sub-crop images and multiple second sub-crop images into the heading and flowering rotation detection model respectively, and outputs the detection results through the heading and flowering rotation detection model, the processor can further determine whether the growth period of the crop enters the heading stage or the flowering stage based on the detection results output by the model.

[0059] In an embodiment of the present application, determining whether the growth period of a crop is the heading stage or the flowering stage based on the detection result includes: when the detection result is the crop area of the heading crop in the first sub-crop image, determining the area ratio between the sum of the crop areas of the heading crops in multiple first sub-crop images and the image area of the first crop image; when the area ratio reaches a first preset threshold, determining that the growth period of the crop enters the heading stage.

[0060] In this embodiment, it should be noted that after the processor inputs multiple first sub-crop images and multiple second sub-crop images into the heading and flowering rotation detection model, and the heading and flowering rotation detection model outputs a detection result, if the detection result is the crop area of the heading crop in the first sub-crop image, the processor further determines the area ratio between the sum of the crop areas of the heading crops in the multiple first sub-crop images and the image area of the first crop image, that is, determines the proportion of the heading crops in the first crop image. If the area ratio reaches a first preset threshold, it can be considered that the crop's growth period has entered the heading stage. The first preset threshold can be set based on historical experience, specifically, including but not limited to 0.1. Therefore, if the area ratio reaches 0.1, it can be considered that the crop's growth period has entered the heading stage. If the area ratio is less than 0.1, it can be considered that the crop's growth period has not yet entered the heading stage. The heading stage is the beginning of rice heading, when the first ear appears at the top of the rice plant. Rice at this stage requires sufficient sunlight and water to ensure normal ear development.

[0061] In an embodiment of the present application, the detection method also includes: when the area ratio reaches a second preset threshold, determining that the growth period of the crop enters the heading period, wherein the second preset threshold is greater than the first preset threshold; when the area ratio reaches a third preset threshold, determining that the growth period of the crop enters the full heading period, wherein the third preset threshold is greater than the second preset threshold.

[0062] In this embodiment, it should be noted that the second preset threshold and the third preset threshold can be set according to historical experience. In this technical solution, the second preset threshold can include but is not limited to being set to 0.5, and the third preset threshold can include but is not limited to being set to 0.8. Therefore, if the area ratio reaches 0.5, it can be considered that the crop's growth period has entered the heading period, and if the area ratio reaches 0.8, it can be considered that the crop's growth period has entered the full heading period. Among them, the heading period follows the initial heading period, and when 10% of the rice ears are exposed from the flag leaf sheath, it is called the heading period. At this stage, a large number of rice ears are pulled out from the top of the rice plant, forming a lush scene. At this time, the rice's demand for nutrients reaches its peak, and it is necessary to apply fertilizer in time to meet the growth needs. The full heading period refers to the period when the ears of most rice plants in the whole field have been pulled out. The rice growth at this stage gradually tends to be stable, but it is still necessary to maintain a suitable growth environment to ensure the smooth development of the ear.

[0063] In an embodiment of the present application, determining whether the growth period of a crop is the heading stage or the flowering stage based on the detection results includes: when the detection result is any marked second sub-crop image, determining that the growth period of the crop enters the flowering stage, wherein any marked second sub-crop image is generated after the heading and flowering rotation detection model marks any crop in the flowering state when it is detected that any crop in the second sub-crop image is in the flowering state.

[0064] In this embodiment, it should be noted that after the processor inputs multiple first sub-crop images and multiple second sub-crop images into the heading and flowering rotation detection model, and the heading and flowering rotation detection model outputs a detection result, if the detection result is any marked second sub-crop image, the processor can determine that the crop's growth period has entered the flowering period. The marked second sub-crop image is generated by the heading and flowering rotation detection model after it detects that any crop in the second sub-crop image is in the flowering state and marks the crop in the flowering state.

[0065] In an embodiment of the present application, the detection method also includes: when it is determined that the growth period of the crop has entered the heading stage, the preset time node is determined as the starting node of the heading stage; when it is determined that the growth period of the crop has entered the flowering stage, the preset time node is determined as the starting node of the flowering stage.

[0066] In this embodiment, it should be noted that this technical solution collects the first crop image of the crops in the agricultural area at the first focal length at a preset time node, that is, a certain year, month, day, and time. Therefore, when the processor determines that the growth period of the crop enters the earing stage based on the detection results, the preset time node can be determined as the starting node of the earing stage. Similarly, when the processor determines that the growth period of the crop enters the flowering stage based on the detection results, the preset time node can be determined as the starting node of the flowering stage.

[0067] In an embodiment of the present application, the detection method also includes a training step of an earing and flowering rotation detection model, and the training step includes: obtaining a historical first crop image of the agricultural area at a first focal length during the historical earing period and a historical second crop image at a second focal length during the historical flowering period; preprocessing the historical first crop image and the historical second crop image respectively to obtain a plurality of historical first sub-crop images and a plurality of historical second sub-crop images; dividing the plurality of historical first sub-crop images and the plurality of historical second sub-crop images into a first training set, a first test set, a second training set and a second test set according to a preset ratio; taking the object detection and segmentation model architecture as the basic model, and constructing a to-be-trained model according to the binary cross entropy function, the KL divergence function, the ProbIoU function and the DFL function. A spikelet and flowering rotation detection model; pre-training and test parameter adjustment of the spikelet and flowering rotation detection model to be trained using a first training set and a first test set, and pre-training and test parameter adjustment of the spikelet and flowering rotation detection model to be trained using a second training set and a second test set; for any historical first sub-crop image in the first test set, when the deviation value between the loss value of any historical first sub-crop image and the loss value of the previous first sub-crop image meets the first preset deviation value, and for any historical second sub-crop image in the second test set, when the second deviation value between the loss value of any historical second sub-crop image and the loss value of the previous historical second sub-crop image meets the preset bias interpolation value, it is determined that the trained spikelet and flowering rotation detection model is obtained.

[0068] In this embodiment, it should be noted that the earing and flowering rotation detection model can be obtained by training historical image data. Specifically, taking rice as an example, the processor can obtain the historical first rice image of the agricultural area at the first focal length during the historical earing period and the historical second rice image at the second focal length during the historical flowering period, so as to train the earing detection model through the historical first rice image and the flowering detection model through the historical second rice image. Secondly, the historical first rice image is cropped using a resolution of 1024x1024 to obtain four historical first sub-images, namely the upper left corner, the upper right corner, the lower left corner and the lower right corner, and the historical second rice image is cropped in the same way. For multiple historical first sub-images, rolabelimg is used to mark the rotation target detection frame, and the marked object is a category of rice ear. The data collected every day in the data set is randomly sampled and divided into training set and test set in a 4:1 manner. The training set and test set divided every day are summarized to obtain the final first training set and first test set. Similarly, for multiple historical second sub-images, rolabelimg is used to annotate rotation object detection boxes, labeling the objects as either flowering or non-flowering rice ears. The data collected daily in the dataset is randomly sampled and divided into training and test sets in a 4:1 ratio. The resulting training and test sets are then combined to form the final second training and test sets. The processor can then construct a trained flowering and rotation detection model using the yolov8 architecture as the base model, along with the binary cross-entropy function, KL divergence function, ProbIoU function, and Dependency Factor (DFL) function. The trained flowering and rotation detection model is then pre-trained and tested using the first training set and the first test set, respectively, and pre-trained and tested using the second training set and the second test set, respectively. The binary cross-entropy function can refer to the BCE loss function, a commonly used loss function primarily used for binary classification tasks. The BCE loss function measures the difference between the model's predicted probability distribution and the true label. The KL divergence function measures the difference between two probability distributions P and Q, also known as relative entropy. The ProbIoU function is a similarity metric based on the Hellinger distance, primarily used for calculating the loss of oriented bounding boxes (OBBs) in object detection. ProbIoU mitigates boundary discontinuities by considering center point differences, covariance, and aspect ratio consistency. It is suitable for calculating the Probability Intersection over Union (ProbIoU) between oriented bounding boxes. The DFL function can evaluate model quality by calculating density statistics of certain features in fMRI datasets. These features can include activation patterns, network connectivity patterns, wavelet coefficients, and other features. Including these features ensures the model's robustness to neurodegeneration and data bias across different scanners.For any historical first-sub-crop image in the first test set, if the deviation between the loss value of any historical first-sub-crop image and the loss value of the previous first-sub-crop image meets the first preset bias interpolation value, the initial ear detection model can be considered trained. Simultaneously, for any historical second-sub-crop image in the second test set, if the second deviation between the loss value of any historical second-sub-crop image and the loss value of the previous historical second-sub-crop image meets the preset bias interpolation value, the flowering detection model can be considered trained. When both the initial ear detection model and the flowering detection model are trained, it can be determined that a fully trained ear formation and flowering rotation detection model has been obtained.

[0069] In an embodiment of the present application, the detection method also includes: determining the binary cross entropy function as the classification loss function of the ear and flowering rotation detection model to be trained, and determining the KL divergence function, ProbIoU function and DFL function as the rotation bounding box regression loss function of the ear and flowering rotation detection model to be trained, so as to obtain the total loss function of the ear and flowering rotation detection model.

[0070] In this embodiment, it should be noted that this technical solution can determine the BCE loss function as the classification loss function of the ear of flowering rotation detection model to be trained, and determine the KL divergence function, ProbIoU function and DFL function as the rotation bounding box regression loss function of the ear of flowering rotation detection model to be trained, so as to obtain the total loss function of the ear of flowering rotation detection model.

[0071] In the embodiment of the present application, the binary cross entropy function is determined as the classification loss function of the ear and flowering rotation detection model to be trained, and the KL divergence function, ProbIoU function and DFL function are determined as the rotation bounding box regression loss function of the ear and flowering rotation detection model to be trained, so as to obtain the total loss function of the ear and flowering rotation detection model as shown in formula (1):

[0072] Total_Loss=BCE_Loss+λ1·KLD_Loss+λ2·ProbIoU_Loss+λ3·DFL_Loss(1)

[0073] Among them, Total_Loss is the total loss function, BCE_Loss is the binary cross entropy function, KLD_Loss is the KL divergence function, ProbIoU_Loss is the ProbIoU function, DFL_Loss is the DFL function, and λ1, λ2 and λ3 are constant coefficients.

[0074] (3) In this embodiment, it should be noted that λ1, λ2, and λ3 can be set to 0.4, 0.4, and 0.2, respectively. Using the improved Soft-NMS, ProbIoU is used to replace the original IoU calculation to enhance the effect of Soft-NMS. ProbIoU takes into account the confidence and overlap of the box, and can more accurately evaluate the relationship between the rotation boxes. The calculation formula is as follows:

[0075] ProbIoU=Ainter / (∈+Aunion)xCbox1xCbox2

[0076] Where Ainter represents the intersection area of two boxes, Aunion represents the union area of two boxes, ε prevents the denominator from being zero, and Cbox1 and Cbox2 represent the confidence levels of the two predictions. Convert the Torch model to an Onnx model with an input batch size of 4. Four images (top left, top right, bottom left, and bottom right) should be input simultaneously during model inference.

[0077] The above technical solution obtains a first crop image of crops in an agricultural area at a first focal length and a second crop image at a second focal length at a preset time point, wherein the first focal length is smaller than the second focal length; preprocesses the first crop image and the second crop image to obtain multiple first sub-crop images and multiple second sub-crop images; and inputs the multiple first sub-crop images and multiple second sub-crop images into a heading and flowering rotation detection model to output detection results. The heading and flowering rotation detection model is constructed based on the object detection and segmentation model architecture and includes a feature extraction module, an intermediate layer module, and a classifier module connected in sequence. The feature extraction module includes a RepVGG-B2 convolutional network, a depthwise separable convolutional network, and a parameter-free attention module. The intermediate layer module includes multiple parameter-free attention modules. The classifier module includes multiple adaptive feature fusion (ASFF) modules. The crop growth period is determined to be the heading stage or the flowering stage based on the detection results. This method effectively improves the prediction accuracy of the heading stage and flowering stage.

[0078] Based on the actual application scenarios of farmland, the present invention uses a zoomable visible light camera to collect real data, combines modern artificial intelligence technology, and improves the deep learning target detection algorithm to detect the rice heading and flowering nodes. By taking advantage of the camera's zoom capability, the possibility of detection is increased and the uncontrollability is reduced, thereby improving the prediction accuracy in many aspects.

[0079] The present application provides a detection device for determining the growth period of crops, comprising:

[0080] a memory configured to store instructions;

[0081] The processor is configured to call the instructions from the memory and implement the above-mentioned method for detecting the growth period of crops when executing the instructions.

[0082] An embodiment of the present application provides a machine-readable storage medium having instructions stored thereon. When the instructions are executed by a processor, the processor is configured to execute the above-mentioned method for crop growth period.

[0083] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected via a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store method data for the crop growth period. The network interface A02 of the computer device is used to communicate with an external terminal via a network connection. When the computer program B02 is executed by the processor A01, it implements a method for the crop growth period.

[0084] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0085] An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, method steps for crop growth period are implemented.

[0086] The present application also provides a computer program product which, when executed on a data processing device, is adapted to execute the program of initializing the method steps for the crop growth phase.

[0087] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0089] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0091] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0092] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0093] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0094] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0095] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting the growth period of crops, characterized in that: The detection method comprises: Acquire a first crop image of crops in an agricultural area at a first focal length and a second crop image at a second focal length at a preset time point, wherein the first focal length is smaller than the second focal length; preprocessing the first crop image and the second crop image respectively to obtain a plurality of first sub-crop images and a plurality of second sub-crop images; The plurality of first sub-crop images and the plurality of second sub-crop images are respectively input into a spikelet, flowering, and rotation detection model to output a detection result through the spikelet, flowering, and rotation detection model, wherein the spikelet, flowering, and rotation detection model is constructed based on an object detection and segmentation model architecture as a basic model architecture, and the spikelet, flowering, and rotation detection model includes a feature extraction module, an intermediate layer module, and a classifier module connected in sequence, wherein the feature extraction module includes a RepVGG-B2 convolutional network, a depthwise separable convolutional network, and a parameter-free attention module, the intermediate layer module includes multiple parameter-free attention modules, and the classifier module includes multiple adaptive feature fusion (ASFF) modules; The growth period of the crop is determined to be the initial heading period or the flowering period according to the detection result.

2. The method for detecting the growth period of crops according to claim 1, wherein: Determining whether the crop growth period is the heading stage or the flowering stage according to the detection result includes: In a case where the detection result is the crop area of the ear-bearing crop in the first sub-crop image, determining an area ratio between the sum of the crop areas of the ear-bearing crops in the plurality of first sub-crop images and the image area of the first crop image; When the area ratio reaches a first preset threshold, it is determined that the growth period of the crop enters the initial heading stage.

3. The method for detecting the growth period of crops according to claim 2, wherein: The detection method further comprises: When the area ratio reaches a second preset threshold, determining that the growth period of the crop enters the heading period, wherein the second preset threshold is greater than the first preset threshold; When the area ratio reaches a third preset threshold, it is determined that the growth period of the crop enters the heading stage, wherein the third preset threshold is greater than the second preset threshold.

4. The method for detecting the growth period of crops according to claim 1, wherein: Determining whether the crop growth period is the heading stage or the flowering stage according to the detection result includes: In the case where the detection result is any marked second sub-crop image, it is determined that the growth period of the crop enters the flowering period, wherein any marked second sub-crop image is generated after the ear-bearing and flowering rotation detection model marks any crop in the flowering state when it detects that any crop in the second sub-crop image is in the flowering state.

5. The method for detecting the growth period of crops according to claim 1, wherein: The detection method further comprises: When it is determined that the crop's growth period has entered the initial heading stage, the preset time node is determined as the starting node of the initial heading stage; When it is determined that the growth period of the crop enters the flowering period, the preset time node is determined as the starting node of the flowering period.

6. The method for detecting the growth period of crops according to claim 1, wherein: The detection method further includes a training step of the spike-forming, flowering-rotation detection model, and the training step includes: Acquire a historical first crop image of the agricultural area at the first focal length during a historical heading stage and a historical second crop image at the second focal length during a historical flowering stage; Preprocessing the historical first crop image and the historical second crop image respectively to obtain a plurality of historical first sub-crop images and a plurality of historical second sub-crop images; dividing the plurality of historical first sub-crop images and the plurality of historical second sub-crop images into a first training set, a first test set, a second training set, and a second test set according to a preset ratio; The object detection and segmentation model architecture is used as the basic model, and the ear and flowering rotation detection model to be trained is constructed based on the binary cross entropy function, KL divergence function, ProbIoU function and DFL function; Pre-training and testing the to-be-trained ear-bone-flowering rotation detection model using the first training set and the first test set, and pre-training and testing the to-be-trained ear-bone-flowering rotation detection model using the second training set and the second test set, respectively; For any historical first sub-crop image in the first test set, when the deviation value between the loss value of any historical first sub-crop image and the loss value of the previous first sub-crop image meets the first preset bias interpolation value, and for any historical second sub-crop image in the second test set, when the second deviation value between the loss value of any historical second sub-crop image and the loss value of the previous historical second sub-crop image meets the preset bias interpolation value, it is determined that the trained ear and flowering rotation detection model is obtained.

7. The method for detecting the growth period of crops according to claim 6, wherein: The detection method further comprises: The binary cross entropy function is determined as the classification loss function of the ear and flowering rotation detection model to be trained, and the KL divergence function, ProbIoU function and DFL function are determined as the rotation bounding box regression loss function of the ear and flowering rotation detection model to be trained, so as to obtain the total loss function of the ear and flowering rotation detection model.

8. The method for detecting the growth period of crops according to claim 7, wherein: The binary cross entropy function is determined as the classification loss function of the to-be-trained ear-and-flowering rotation detection model, and the KL divergence function, the ProbIoU function, and the DFL function are determined as the rotation bounding box regression loss function of the to-be-trained ear-and-flowering rotation detection model, so as to obtain the total loss function of the ear-and-flowering rotation detection model as shown in formula (1): Total_Loss=BCE_Loss+λ1·KLD_Loss+λ2·ProbIoU_Loss+λ3·DFL_Loss(1) Among them, the Total_Loss is the total loss function, BCE_Loss is the binary cross entropy function, KLD_Loss is the KL divergence function, ProbIoU_Loss is the ProbIoU function, DFL_Loss is the DFL function, and λ1, λ2 and λ3 are constant coefficients.

9. The method for detecting the growth period of crops according to claim 1, wherein: The intermediate layer module also includes multiple Concat modules for connecting with the feature extraction module, and the parameter-free attention module is used between the feature extraction module and the Concat module.

10. The method for detecting the growth period of crops according to claim 1, wherein: The intermediate layer module also includes multiple C2F2_3d neural network modules, and each adaptive feature fusion ASFF module in the multiple adaptive feature fusion ASFF modules is used to weighted sum the different features of each sub-crop image output by the corresponding connected C2F2_3d neural network module to obtain a feature map after fusion of the features.

11. The method for detecting the growth period of crops according to claim 1, wherein: The ear and flowering rotation detection model includes an initial ear detection model and a flowering detection model. The ear detection model is used to output the detection results of the multiple first sub-crop images, and the flowering detection model is used to output the detection results of the multiple second sub-crop images.

12. A detection device for determining the growth period of crops, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the method for detecting the growth period of crops according to any one of claims 1 to 11 when executing the instructions.

13. A detection system for determining the growth period of crops, characterized in that: It comprises the detection device for determining the growth period of crops according to claim 12.

14. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the processor is configured to perform the method for crop growth period according to any one of claims 1 to 11.

Images