Intelligent crop growth state monitoring method and device based on deep learning

Abstract

The invention relates to the technical field of intelligent agriculture, in particular to an intelligent crop growth state monitoring method and device based on deep learning, and the method comprises the steps: regularly obtaining pictures of crops in seedling trays in a greenhouse; wherein the seedling tray is divided into a plurality of grids, and is used for performing crop seeding on each grid; on the basis of the trained grid detection model, identifying grids in the crop photo, and counting the number of all the grids; on the basis of the trained crop detection model, identifying leaves of crops in the grids in the crop picture, and marking the contour of each leaf; the crop detection model comprises a seedling detection model and an adult seedling detection model, and calculating various growth state indexes of the crops according to the counted number of all the grids and the identification result of the leaves of the crops in the grids. Therefore, automatic identification and tracking of the growth state of the crop plant are realized, personal errors are reduced, and powerful support is provided for precision agriculture.

Description

Technical Field

[0001] The present application relates to the field of smart agricultural technology, and in particular to a method and device for monitoring the growth status of smart crops based on deep learning. Background Art

[0002] During the growth of agricultural products, farmers need to closely monitor the growth status of crops. Traditional crop monitoring methods usually rely on people's subjective experience and judgment, which is not suitable for situations where crops are planted over a wide range. Because traditional farmland management methods are time-consuming and labor-intensive, farmers are prone to neglecting the poor growth status of crops and are unable to detect abnormal conditions of crops in time, which has various adverse effects on crop products and quality. Therefore, a more efficient, accurate and objective monitoring method is needed. Summary of the invention

[0003] In order to overcome the deficiencies in the prior art, the present application provides an intelligent crop growth status monitoring method and device based on deep learning, which can realize automatic identification and tracking of the growth status of crop plants, thereby reducing human errors and providing strong support for precision agriculture.

[0004] In a first aspect, the present application provides an intelligent crop growth status monitoring method based on deep learning, comprising the following steps:

[0005] Regularly obtain photos of crops in a seedling tray in a greenhouse; wherein the seedling tray is divided into a plurality of grids for sowing crops in each of the grids;

[0006] Based on the trained grid detection model, the grids in the crop photo are identified, and the number of all grids is counted; and based on the trained crop detection model, the leaves of the crops in the grids in the crop photo are identified, and the outline of each leaf is marked; the crop detection model includes a seedling detection model and a seedling detection model, wherein the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model, and the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model;

[0007] According to the number of all grids counted and the leaf recognition results of the crops in the grids, various growth status indicators of the crops are calculated.

[0008] In one possible implementation, the grid detection model is constructed based on an improved YOLOv9 target detection network, wherein the improved YOLOv9 target detection network includes replacing the standard convolutional layer with an AKConv layer in the head layer of the original YOLOv9 target detection network; the seedling detection model is constructed based on a RAUNet neural network, and is post-processed using a watershed image segmentation algorithm to obtain the contours of leaves of different seedlings; and the seedling detection model is constructed based on a DynaMask network.

[0009] In a possible implementation, the trained grid detection model / seedling detection model / seedling detection model is obtained in the following manner, including the following steps:

[0010] Dividing the acquired crop photo dataset into a crop photo training dataset and a crop photo test dataset according to a preset ratio;

[0011] Training the constructed grid detection model / seedling detection model / seedling detection model based on the crop photo training data set to adjust model parameters and obtain an initial grid detection model / seedling detection model / seedling detection model;

[0012] Input the crop photo test data set into the obtained initial grid detection model / seedling detection model / seedling detection model, and compare the output detection results with the annotated actual results, and verify the detection accuracy of the obtained initial grid detection model / seedling detection model / seedling detection model according to the comparison results;

[0013] If the detection accuracy meets the preset requirements, the initial grid detection model / seedling detection model / mature seedling detection model is used as the trained grid detection model / seedling detection model / mature seedling detection model.

[0014] In a possible implementation, the crop photo dataset is obtained in the following manner, including the following steps:

[0015] Using a track camera deployed in the greenhouse to take a plurality of crop photos, and performing image augmentation processing on the plurality of crop photos taken to obtain a crop photo set;

[0016] Data annotation is performed on each crop photo in the crop photo set to obtain a crop photo data set; wherein the data annotation includes annotating the grids, seedlings, and mature seedlings in each crop photo one by one.

[0017] In a possible implementation, the crop photo is first divided into overlapping small blocks, and the divided small blocks are processed one by one based on a trained grid detection model and a trained crop detection model to generate local detection results, and then the generated local detection results are spliced ​​using a non-maximum suppression method to obtain a global detection result of the crop photo.

[0018] In a possible implementation, the growth status indicators of crops include germination rate, seedling rate, seedling rate and leaf age, and are calculated as follows:

[0019]

[0020] Among them, according to the crop growth cycle, the number of sprouts, the number of seedlings, and the number of mature seedlings are obtained from the leaf recognition results of the crops in the grid, and the daily growth number of sprouts is obtained according to historical records, and the total number of leaves is counted based on the daily growth number of sprouts.

[0021] In a possible implementation, the growth status indicator of the crop also includes the number of yellow leaves, and is calculated as follows:

[0022] Convert the acquired crop photo into the HSV color space and determine the position of the yellow element in the crop photo;

[0023] According to the recognition results of the leaves of the crops in the grid and the determined positions of the yellow elements in the crop photos, the proportion of the yellow elements in each leaf is calculated, and when the proportion is greater than a set threshold, the leaf is determined to be a yellow leaf.

[0024] In a second aspect, the present application provides an intelligent crop growth status monitoring device based on deep learning, the device comprising:

[0025] An acquisition module is used to regularly acquire photos of crops in a seedling tray in a greenhouse; wherein the seedling tray is divided into a plurality of grids, and crops are sown in each of the grids;

[0026] A detection module, for identifying grids in the crop photo based on a trained grid detection model and counting the number of all grids; and for identifying leaves of crops in the grids in the crop photo based on the trained crop detection model and marking the outline of each leaf; the crop detection model includes a seedling detection model and a mature seedling detection model, wherein the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model, and the mature seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model;

[0027] The calculation module is used to calculate various growth status indicators of crops according to the number of all grids counted and the leaf recognition results of crops in the grids.

[0028] In a third aspect, the present application provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the deep learning-based intelligent crop growth status monitoring method as described in any one of the first aspects are performed.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the deep learning-based intelligent crop growth status monitoring method as described in any one of the first aspects.

[0030] The present embodiment provides an intelligent crop growth status monitoring method and device based on deep learning, which regularly obtains crop photos in a seedling tray in a greenhouse; wherein the seedling tray is divided into multiple grids for sowing crops in each grid; based on a trained grid detection model, the grids in the crop photos are identified, and the number of all grids is counted; and based on a trained crop detection model, the leaves of the crops in the grids in the crop photos are identified, and the outline of each leaf is marked; the crop detection model includes a seedling detection model and a seedling detection model, wherein the seedling leaves in the grids in the crop photos are identified based on the trained seedling detection model, and the seedling leaves in the grids in the crop photos are identified based on the trained seedling detection model; and various growth status indicators of the crops are calculated based on the counted number of all grids and the identification results of the leaves of the crops in the grids. Thus, the automatic identification and tracking of the growth status of crop plants is realized, human errors are reduced, and strong support is provided for precision agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A flowchart of an intelligent crop growth status monitoring method based on deep learning according to an embodiment of the present application is shown;

[0033] Figure 2 A schematic diagram showing a partially labeled crop photo when training a grid detection model according to an embodiment of the present application;

[0034] Figure 3 A schematic diagram showing a partially labeled crop photo when training a seedling detection model according to an embodiment of the present application;

[0035] Figure 4 A schematic diagram showing the detection process of the seedling detection model according to an embodiment of the present application on the input crop photo;

[0036] Figure 5 A schematic diagram showing the detection process of the seedling detection model according to an embodiment of the present application on the input crop photo;

[0037] Figure 6 A schematic diagram of the structure of an intelligent crop growth status monitoring device based on deep learning according to an embodiment of the present application is shown;

[0038] Figure 7 A structural block diagram of an electronic device described in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0040] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0041] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0042] Existing methods rely on farmers to monitor the growth status of crops. In the case of large-scale crop planting, this method will consume a lot of manpower and material resources, with high labor input costs, and it is difficult to achieve higher product benefits; farmers conduct carpet-style manual monitoring of the growth status of crops in farmland, which is time-consuming and has a long cycle, and it is impossible to detect abnormal conditions in farmland in time; the method of judging the production status of large-scale crop planting based on farmers' personal experience will cause fatigue to the personnel themselves, which is prone to missed detection and wrong detection; in the case of large-scale planting areas, the inspection cycle relying on manual operation methods is long, and it is difficult to form effective crop precision management. In view of this, the present application provides an intelligent crop growth status monitoring method and device based on deep learning, which can realize the automatic identification and tracking of the growth status of crop plants, reduce human errors, and provide strong support for precision agriculture.

[0043] In one embodiment, see the attached specification Figure 1 The present application provides a method for monitoring the growth status of crops based on deep learning, comprising the following steps:

[0044] S1. Regularly obtain photos of crops in a seedling tray in a greenhouse; wherein the seedling tray is divided into a plurality of grids for sowing crops in each of the grids;

[0045] S2. Based on the trained grid detection model, the grids in the crop photo are identified, and the number of all grids is counted; and based on the trained crop detection model, the leaves of the crops in the grids in the crop photo are identified, and the outline of each leaf is marked; the crop detection model includes a seedling detection model and a seedling detection model, wherein the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model, and the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model;

[0046] S3. Calculate various growth status indicators of crops according to the counted number of all grids and the identification results of the leaves of crops in the grids.

[0047] This application applies deep learning technology to the agricultural production process. By using the captured photos to monitor the growth status of crops, various indicators of crops are intelligently analyzed, making the crop growth status monitoring method more intelligent, accurate and efficient.

[0048] In order to accurately monitor the growth status of crops, in step S1 of the present application, a track camera is used to take photos of crops grown in a grid in a greenhouse, and it is necessary to ensure that the track camera takes photos regularly at a fixed position in the greenhouse. This process needs to consider factors such as the camera's position setting and shooting time interval to cover all seedling tray areas. Among them, the track can provide a stable shooting platform to keep the camera stable during movement, thereby effectively avoiding the jitter and shaking problems that may occur during handheld shooting, and improving the clarity and stability of the picture. Since the shooting process is more stable and smooth, it also reduces the time and resources wasted due to repeated shooting.

[0049] In the stage of analyzing crop photos, an intelligent monitoring model for crop growth status (including the grid detection model, seedling detection model and seedling detection model mentioned in step S2) is constructed, and trained in the following manner: the acquired crop photo data set is divided into a crop photo training data set and a crop photo test data set according to a preset ratio; the constructed grid detection model / seedling detection model / seedling detection model is trained based on the crop photo training data set to adjust the model parameters and obtain an initial grid detection model / seedling detection model / seedling detection model; the crop photo test data set is input into the obtained initial grid detection model / seedling detection model / seedling detection model, and the output detection result is compared with the actual result of the annotation, and the detection accuracy of the obtained initial grid detection model / seedling detection model / seedling detection model is verified according to the comparison result; if the detection accuracy meets the preset requirements, the initial grid detection model / seedling detection model / seedling detection model is used as the trained grid detection model / seedling detection model / seedling detection model. The specific training process should be a technical means well known to those skilled in the art, and will not be repeated here.

[0050] Among them, a track camera deployed in the greenhouse is used to take multiple crop photos, and the multiple crop photos taken are subjected to image augmentation processing to obtain a crop photo set; this is because the more crop photos there are, the stronger the performance of the intelligent monitoring model of crop growth status. Through image augmentation processing, not only the amount of training data of the intelligent monitoring model of crop growth status can be increased, but also its generalization performance can be provided. Among them, image augmentation processing includes methods such as translation, rotation, shearing, scaling, salt and pepper noise, Gaussian noise, Gaussian blur and brightness enhancement; then, data annotation is performed on each crop photo in the crop photo set to obtain a crop photo data set; for example, the grids, seedlings, and mature seedlings in the crop photos are manually annotated one by one, and the bounding box position and category are marked.

[0051] The grid detection model is used to count the number of grids in the crop photos, so as to facilitate the subsequent calculation of the germination rate, seedling rate, and seedling rate. In one embodiment, a dataset of 471 crop photos with a size of 640x640 is constructed, and the visualization of some annotated crop photos is shown in the attached manual. Figure 2 Each crop photo has been accurately annotated manually, and the annotation information includes the location information of the seedling tray grid. This dataset is critical for the training of the grid detection model because it contains a variety of greenhouse scenes and changes in seedling tray types.

[0052] Taking into account the computational complexity and in order to solve the problem that the distances between each grid and the camera are different in crop photo shooting, the sizes presented in the photo are also different. Traditional convolution has limitations in processing image data with non-fixed size of target grids. This application adopts an improved YOLOv9 target detection network as the main body of the grid detection model. Specifically, AKConv is introduced into the YOLOv9 target detection network to replace the standard convolution layer with the AKConv layer in the head layer of the YOLOv9 target detection network, thereby improving its detection performance for targets of different shapes and sizes.

[0053] The growth status of crops can be divided into the seedling stage and the seedling stage. Since the size of crops in these two periods is very different, it is necessary to establish separate detection models (seedling detection model and seedling detection model) in the seedling stage and the seedling stage respectively to meet the needs of intelligent crop growth status monitoring.

[0054] Among them, in the seedling stage of crops, especially when they are just sprouting, the buds are too small to mark their outlines, so we can only use the feature of the buds showing green to perform preliminary segmentation and obtain preliminary labeling results. And because color segmentation is very sensitive to the determined color threshold, the effect is poor when the light changes or the seedling tray turns green, so it is necessary to manually remove this part of the data to prevent the data set from being contaminated. In one embodiment, a data set of 493 crop photos of size 512×512 is constructed, and the visualization of some labeled crop photos is shown in the attached manual. Figure 3As shown. And the RAUNet (Residual Attention U-Ne) neural network is used to construct the main body of the seedling detection model. Among them, RAUNet is a U-Net network structure that combines residual learning and attention mechanism, which is mainly used to solve complex problems in image segmentation tasks. It introduces residual learning and attention mechanism on the basis of U-Net to further improve the accuracy and efficiency of image segmentation. However, the result output by RAUNet cannot obtain instance information, so this application uses the watershed algorithm to post-process the semantic segmentation result to obtain the instance information of the leaf. The watershed algorithm is a classic algorithm for image segmentation, which draws on the concept of watershed in geography and is used to separate different areas in the image. First, the distance from each pixel in the image to the nearest background pixel is calculated to obtain a distance transformation map. This distance transformation map reflects the distance relationship between different areas in the image and the background. Farther areas will have larger distance values, while closer areas will have smaller distance values. Then, find the local maximum point from the distance transformation map, which usually represents the starting point of the potential segmentation area. These local maximum points are used as seed points for the watershed algorithm to segment the image into different regions. The local maximum points found are then marked as different regions, and each mark is assigned a unique integer value. These marks will be used in the watershed algorithm to distinguish different segmented regions. The watershed algorithm is then executed using the distance transformation map and the marked seed points. The algorithm determines the boundaries of the segmented regions by simulating the process of water flowing from mountain peaks (seed points) to low-lying areas (local minimum points). The places where the water flows converge form segmentation lines, dividing different regions. Ultimately, the watershed algorithm segments the image into different regions and produces a segmentation mask in which each pixel is assigned to a region, or the final instance segmentation result. The detection process of the seedling detection model on the input crop photos is shown in the attached manual. Figure 4 As shown, contours of different colors in the output leaf instance segmentation result represent different leaves.

[0055] Among them, the leaf occlusion is more serious in the seedling stage of crops, and the leaf shapes are different. The simple watershed algorithm is no longer suitable for post-processing of mature leaves. At the same time, the leaf area is larger and the outline is clearer, which is more friendly to labeling. The instance segmentation algorithm model can be used to handle this task. In one embodiment, 42 640×640 crop photos are annotated by manual labeling (labeling leaf outlines), and further expanded by data augmentation methods to construct a crop photo dataset with a larger data volume. And the main body of the seedling detection model is constructed based on the DynaMask network, which can effectively handle the detection of complex targets and instance segmentation tasks. Among them, the detection process of the seedling detection model on the input crop photos is as shown in the attached manual. Figure 5As shown in the figure: the crop photo is sent to the convolutional neural network to obtain the feature map of the crop photo and the corresponding multiple regions of interest; the generated regions of interest are then sent to the RPN network to obtain multiple candidate boxes, and the mapping area of ​​the original image coordinates of the candidate boxes on the feature map is the ROI, and the foreground or background classification and bounding box regression of the crop photo are completed, and the correct ROI is retained; finally, the Mask information of the corresponding category is extracted through the Sigmoid activation function; finally, the Mask is scaled to the size of the prediction box and moved to the original image area.

[0056] In addition, it should be noted that due to the large size of the crop photos taken by the track camera, due to resource limitations such as video memory, the entire crop photo cannot be sent to the neural network for processing at one time. In order to overcome this limitation, the present application adopts an image segmentation (tiling) method, firstly divides the crop photo into overlapping small blocks, and processes the segmented small blocks one by one based on the trained grid detection model and the trained crop detection model, generates local detection results, and then uses the non-maximum suppression method to splice the generated local detection results to obtain the complete global detection results of the crop photo. Specifically, first, the crop photo (3840x2180) is divided into small blocks, usually overlapping small blocks, to ensure that the target object crossing the small block boundary is not missed. The size of these small blocks is usually determined by the resource limitations of the video memory and the processor. Then, these small blocks are processed one by one to generate local detection results. Next, these local detection results need to be spliced ​​to reconstruct the global detection results of the entire crop photo. This can be achieved by superimposing the detection results of each small block according to its position, and gradually constructing the global detection results of the entire crop photo. In this process, the overlapping parts between small blocks need to be specially processed to avoid repeated calculation of targets. Once the global detection results of the entire crop photo are obtained, multiple overlapping frames usually appear at the boundary of the image stitching, and these frames may have different detection results for the same target. In order to remove overlapping detection results, the non-maximum suppression (NMS) method is adopted. The NMSNMS algorithm first sorts the confidence scores of several detection frames in descending order and selects the frame with the highest confidence score; secondly, the IoU value of this frame and the remaining detection frames is calculated. If the value is greater than or equal to the set threshold (the default setting in the experiment is 0.5), the confidence score of the detection frame is set to 0 and the detection frame is discarded; then the above cycle is repeated until all redundant frames of the algorithm are removed.

[0057] In step S3, the output results of the intelligent monitoring model of crop growth status are integrated with historical records, so that various parameters of the current seedling tray area can be obtained, including germination rate, seedling rate, seedling rate, leaf age, number of yellow leaves and other data. These data will help farmers and agricultural professionals better understand the growth status of crops and take necessary measures in time to improve yield and quality.

[0058] Among them, the grid detection model can be used to obtain the total number of grids in the crop photo. The crop detection model can be used to obtain the position and outline of each leaf. Combining the spatial position relationship between the two, it can be determined whether there is a leaf in the grid. For the judgment of budding, as long as there is a leaf in the grid, it is considered to be budding. According to the crop growth cycle, the calculation formulas for budding rate, seedling rate, and seedling rate are as follows:

[0059]

[0060] Since the crop photos taken every day have deviations, it is difficult to find the same leaf in two crop photos. Therefore, the average leaf age of the leaves in the entire crop photo is calculated. The calculation method is as follows:

[0061]

[0062] In other embodiments, crop growth status indicators such as uniformity and the number of yellow leaves can also be calculated. The uniformity is calculated by dividing the total leaf area by the area of ​​the seedling tray; when calculating the number of yellow leaves, the acquired crop photo is first converted to the HSV color space, and the position of the yellow element in the crop photo can be determined according to the interval occupied by the yellow color, and then the proportion of the yellow element in each leaf is calculated in combination with the connected domain of the instance segmentation. When the proportion exceeds a certain threshold, it can be considered as a yellow leaf.

[0063] It can be seen that the intelligent crop growth status monitoring method based on deep learning provided by the present application adopts a grid detection model constructed based on the improved YOLOv9 target detection network, a seedling detection model constructed based on the RAUNet neural network, and a seedling detection model constructed based on the DynaMask network. It can complete the automatic monitoring of various growth status indicators of crops in a short time, which is faster and more accurate than traditional manual monitoring methods.

[0064] Based on the same inventive concept, an embodiment of the present application also provides an intelligent crop growth status monitoring device based on deep learning. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned intelligent crop growth status monitoring method based on deep learning in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0065] As the instruction manual Figure 6 As shown, an embodiment of the present application provides an intelligent crop growth status monitoring device based on deep learning, and the device includes:

[0066] The acquisition module 601 is used to regularly acquire photos of crops in the seedling tray in the greenhouse; wherein the seedling tray is divided into a plurality of grids, and crops are sown in each of the grids;

[0067] The detection module 602 is used to identify the grids in the crop photo based on the trained grid detection model and count the number of all grids; and to identify the leaves of the crops in the grids in the crop photo based on the trained crop detection model and mark the outline of each leaf; the crop detection model includes a seedling detection model and a mature seedling detection model, wherein the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model, and the mature seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model;

[0068] The calculation module 603 is used to calculate various growth status indicators of crops according to the number of all grids counted and the leaf recognition results of crops in the grids; wherein the growth status indicators of crops include germination rate, seedling rate, seedling rate, leaf age and number of yellow leaves.

[0069] In one embodiment, the grid detection model is constructed based on an improved YOLOv9 target detection network, and the improved YOLOv9 target detection network includes replacing the standard convolution layer with an AKConv layer in the head layer of the original YOLOv9 target detection network; the seedling detection model is constructed based on a RAUNet neural network, and is post-processed using a watershed image segmentation algorithm to obtain the contours of different seedling leaves; the seedling detection model is constructed based on a DynaMask network.

[0070] In one embodiment, the device further comprises:

[0071] A training module is used to divide the acquired crop photo data set into a crop photo training data set and a crop photo test data set according to a preset ratio; based on the crop photo training data set, a constructed grid detection model / seedling detection model / seedling detection model is trained to adjust model parameters and obtain an initial grid detection model / seedling detection model / seedling detection model; the crop photo test data set is input into the obtained initial grid detection model / seedling detection model / seedling detection model, and the output detection result is compared with the marked actual result, and the detection accuracy of the obtained initial grid detection model / seedling detection model / seedling detection model is verified according to the comparison result; if the detection accuracy meets the preset requirements, the initial grid detection model / seedling detection model / seedling detection model is used as the trained grid detection model / seedling detection model / seedling detection model.

[0072] In one embodiment, the device further comprises:

[0073] The acquisition module is used to take multiple crop photos using a track camera deployed in a greenhouse, and perform image augmentation processing on the multiple crop photos taken to obtain a crop photo set; perform data annotation on each crop photo in the crop photo set to obtain a crop photo data set; wherein the data annotation includes annotating the grids, seedlings, and mature seedlings in each crop photo one by one.

[0074] In one embodiment, the device further comprises:

[0075] The stitching module is used to first divide the crop photo into overlapping small blocks, and process the divided small blocks one by one based on the trained grid detection model and the trained crop detection model to generate local detection results, and then use the non-maximum suppression method to stitch the generated local detection results to obtain the global detection result of the crop photo.

[0076] The present application provides an intelligent crop growth status monitoring device based on deep learning, which regularly obtains crop photos in a seedling tray in a greenhouse through an acquisition module; wherein the seedling tray is divided into multiple grids for sowing crops in each grid; through a detection module, based on a trained grid detection model, the grids in the crop photos are identified, and the number of all grids is counted; and based on a trained crop detection model, the leaves of the crops in the grids in the crop photos are identified, and the outline of each leaf is marked; the crop detection model includes a seedling detection model and a seedling detection model, wherein the seedling leaves in the grids in the crop photos are identified based on the trained seedling detection model, and the seedling leaves in the grids in the crop photos are identified based on the trained seedling detection model; and the various growth status indicators of the crops are calculated by the calculation module according to the number of all grids counted and the leaf recognition results of the crops in the grids. Thus, the automatic identification and tracking of the growth status of crop plants is realized, human errors are reduced, and strong support is provided for precision agriculture.

[0077] Based on the same concept of the present invention, the specification is attached Figure 7 As shown, the structure of an electronic device 700 provided in an embodiment of the present application includes: at least one processor 701, at least one network interface 704 or other user interface 703, a memory 705, and at least one communication bus 702. The communication bus 702 is used to realize the connection and communication between these components. The electronic device 700 optionally includes a user interface 703, including a display (for example, a touch screen, LCD, CRT, holographic imaging (Holographic) or projection (Projector), etc.), a keyboard or a pointing device (for example, a mouse, a trackball (trackball), a touch pad or a touch screen, etc.).

[0078] The memory 705 may include a read-only memory and a random access memory, and provides instructions and data to the processor 701. A portion of the memory 705 may also include a non-volatile random access memory (NVRAM).

[0079] In some implementations, the memory 705 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:

[0080] Operating system 7051, including various system programs for implementing various basic services and processing hardware-based tasks;

[0081] The application module 7052 includes various application programs, such as a launcher, a media player, a browser, etc., which are used to implement various application services.

[0082] In an embodiment of the present application, by calling the program or instructions stored in the memory 705, the processor 701 is used to execute steps in a smart crop growth status monitoring method based on deep learning, which can realize automatic identification and tracking of the growth status of crop plants, thereby reducing human errors and providing strong support for precision agriculture.

[0083] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the intelligent crop growth status monitoring method based on deep learning are executed.

[0084] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the above-mentioned intelligent crop growth status monitoring method based on deep learning can be executed.

[0085] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0086] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0087] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0088] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0089] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for monitoring the growth status of crops based on deep learning, characterized in that: The method comprises the following steps: Regularly obtain photos of crops in a seedling tray in a greenhouse; wherein the seedling tray is divided into a plurality of grids for sowing crops in each of the grids; Based on the trained grid detection model, the grids in the crop photo are identified, and the number of all grids is counted; and based on the trained crop detection model, the leaves of the crops in the grids in the crop photo are identified, and the outline of each leaf is marked; the crop detection model includes a seedling detection model and a seedling detection model, wherein the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model, and the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model; According to the number of all grids counted and the leaf recognition results of the crops in the grids, various growth status indicators of the crops are calculated.

2. According to claim 1, a method for monitoring the growth status of crops based on deep learning, characterized in that: in, The grid detection model is constructed based on an improved YOLOv9 target detection network, wherein the improved YOLOv9 target detection network includes replacing the standard convolution layer with an AKConv layer in the head layer of the original YOLOv9 target detection network; the seedling detection model is constructed based on a RAUNet neural network, and a watershed image segmentation algorithm is used for post-processing to obtain the contours of leaves of different seedlings; the seedling detection model is constructed based on a DynaMask network.

3. According to claim 2, a method for monitoring the growth status of crops based on deep learning, characterized in that: The trained grid detection model / seedling detection model / seedling detection model is obtained in the following manner, including the following steps: Dividing the acquired crop photo dataset into a crop photo training dataset and a crop photo test dataset according to a preset ratio; Training the constructed grid detection model / seedling detection model / seedling detection model based on the crop photo training data set to adjust model parameters and obtain an initial grid detection model / seedling detection model / seedling detection model; Input the crop photo test data set into the obtained initial grid detection model / seedling detection model / seedling detection model, and compare the output detection results with the annotated actual results, and verify the detection accuracy of the obtained initial grid detection model / seedling detection model / seedling detection model according to the comparison results; If the detection accuracy meets the preset requirements, the initial grid detection model / seedling detection model / mature seedling detection model is used as the trained grid detection model / seedling detection model / mature seedling detection model.

4. According to claim 3, a method for monitoring the growth status of crops based on deep learning, characterized in that: The crop photo dataset is obtained in the following way, including the following steps: Using a track camera deployed in the greenhouse to take a plurality of crop photos, and performing image augmentation processing on the plurality of crop photos taken to obtain a crop photo set; Data annotation is performed on each crop photo in the crop photo set to obtain a crop photo data set; wherein the data annotation includes annotating the grids, seedlings, and mature seedlings in each crop photo one by one.

5. According to claim 4, a method for monitoring the growth status of crops based on deep learning, characterized in that: in, First, the crop photo is divided into overlapping small blocks, and the divided small blocks are processed one by one based on the trained grid detection model and the trained crop detection model to generate local detection results, and then the generated local detection results are spliced ​​using the non-maximum suppression method to obtain the global detection result of the crop photo.

6. The method for monitoring the growth status of crops based on deep learning according to claim 5, characterized in that: in, The growth status indicators of crops include germination rate, seedling rate, seedling rate and leaf age, and are calculated as follows: Among them, according to the crop growth cycle, the number of sprouts, the number of seedlings, and the number of mature seedlings are obtained from the leaf recognition results of the crops in the grid, and the daily growth number of sprouts is obtained according to historical records, and the total number of leaves is counted based on the daily growth number of sprouts.

7. The method for monitoring the growth status of crops based on deep learning according to claim 5, characterized in that: in, The growth status indicator of crops also includes the number of yellow leaves, and is calculated as follows: Convert the acquired crop photo into the HSV color space and determine the position of the yellow element in the crop photo; According to the recognition results of the leaves of the crops in the grid and the determined positions of the yellow elements in the crop photos, the proportion of the yellow elements in each leaf is calculated, and when the proportion is greater than a set threshold, the leaf is determined to be a yellow leaf.

8. An intelligent crop growth status monitoring device based on deep learning, characterized in that: The device comprises: An acquisition module is used to regularly acquire photos of crops in a seedling tray in a greenhouse; wherein the seedling tray is divided into a plurality of grids, and crops are sown in each of the grids; A detection module, for identifying grids in the crop photo based on a trained grid detection model and counting the number of all grids; and for identifying leaves of crops in the grids in the crop photo based on the trained crop detection model and marking the outline of each leaf; the crop detection model includes a seedling detection model and a mature seedling detection model, wherein the seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model, and the mature seedling leaves in the grids in the crop photo are identified based on the trained seedling detection model; The calculation module is used to calculate various growth status indicators of crops according to the number of all grids counted and the leaf recognition results of crops in the grids.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the deep learning-based intelligent crop growth status monitoring method as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the deep learning-based intelligent crop growth status monitoring method as described in any one of claims 1 to 7.

Images