Statistical method, system and equipment for ear emergence rate and leaf width of rice ear and medium
By using the rice detection and segmentation model based on the Transformer model, the rice images were detected and segmented, and the problems of inefficiency and low accuracy of traditional methods were solved, and efficient and accurate statistics of rice ear ejaculation rate and leaf width were achieved.
Patent Information
- Application Number
- CN202411705506.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-18
AI Technical Summary
Traditional methods of stating the ear yield rate and leaf width of rice are inefficient and prone to errors, and rely on human resources, resulting in high costs. The overall algorithm model of existing machine vision technology methods is complex, and the detection and segmentation effects of small targets are poor, resulting in low statistical accuracy.
The rice detection and segmentation model based on the Transformer model is used to detect and segment the rice images, and the characteristic information of rice ears and rice leaves is directly obtained, and the ear rate and leaf width are calculated by counting these characteristic information.
The statistical accuracy of rice ear yield and leaf width is improved, labor and time costs are reduced, model structure is simplified, and statistical efficiency is improved.
Smart Images
Figure CN119648656A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine vision technology, and in particular to a statistical method, system, equipment and medium for rice ear emergence rate and leaf width. Background Art
[0002] The traditional method of counting ear emergence rate and leaf width is manual counting, which is inefficient, error-prone, and consumes a lot of manpower and time costs.
[0003] The existing statistical methods for rice ear emergence rate and leaf width involving machine vision technology usually use traditional target detection and image segmentation models to detect and segment rice ears and leaves, and then perform corresponding data statistical processing. This method can only stack the target detection model and the image segmentation model in sequence to complete the task, which leads to the complexity of the overall algorithm model. In addition, the traditional visual model relies on convolutional neural networks to extract image features, which is less effective when facing smaller targets such as rice ears and leaves, resulting in low statistical accuracy of rice ear emergence rate and leaf width. Summary of the invention
[0004] The present application aims to propose a statistical method, system, equipment and medium for rice ear emergence rate and leaf width, which can improve the statistical accuracy of rice ear emergence rate and leaf width.
[0005] In a first aspect, an embodiment of the present application provides a statistical method for rice ear emergence rate and leaf width, the method comprising:
[0006] Collecting a rice image to be segmented, and preprocessing the rice image to be segmented to obtain a preprocessed rice image;
[0007] Inputting the preprocessed rice image into a trained rice detection and segmentation model for image detection and image segmentation to obtain feature information of rice ears and rice leaves in the rice image to be segmented, wherein the rice detection and segmentation model is constructed based on a Transformer model;
[0008] The ear emergence rate and leaf width corresponding to the rice image to be segmented are statistically calculated according to the feature information.
[0009] Compared with the prior art, the first aspect of the present application has the following beneficial effects:
[0010] The method collects a rice image to be segmented, and pre-processes the rice image to be segmented to obtain a pre-processed rice image; the pre-processed rice image is input into a trained rice detection and segmentation model for image detection and image segmentation to obtain feature information of rice ears and rice leaves in the rice image to be segmented, and the rice detection and segmentation model is constructed based on a Transformer model; according to the feature information, the ear emergence rate and leaf width corresponding to the rice image to be segmented are statistically calculated. In this way, the feature information of rice ears and rice leaves in the rice image to be segmented is directly obtained through the trained rice detection and segmentation model, which can improve statistical efficiency and reduce error rate, reduce labor cost and time cost, and only need to train a rice detection and segmentation model constructed based on the Transformer model to obtain detection results and segmentation results, and then the ear emergence rate and leaf width corresponding to the rice image to be segmented are statistically calculated, which improves statistical efficiency while improving the statistical accuracy of the ear emergence rate and leaf width.
[0011] In some embodiments, the trained rice detection and segmentation model is obtained by:
[0012] Collecting a plurality of rice images, and preprocessing the plurality of rice images to obtain a plurality of preprocessed rice images;
[0013] Construct a hybrid matching loss function including cross entropy loss, GIoU loss and mask loss;
[0014] Based on the hybrid matching loss function, the constructed rice detection and segmentation model is trained using the preprocessed multiple rice images to obtain the trained rice detection and segmentation model.
[0015] In some embodiments, the trained rice detection and segmentation model includes a backbone feature extraction network, a pixel encoder and a Transformer decoder, and the preprocessed rice image is input into the trained rice detection and segmentation model for image detection and image segmentation to obtain feature information of rice ears and rice leaves in the rice image to be segmented, including:
[0016] Using the backbone feature extraction network to extract features from the preprocessed rice image to obtain a multi-scale feature map;
[0017] Performing position encoding on the multi-scale feature map to obtain a position-encoded feature map;
[0018] Performing a flattening operation on the position-encoded features to obtain a flattened feature map;
[0019] Inputting the flattened feature map and the multi-scale feature map into the pixel encoder to obtain an encoding result, a pixel embedding map, an initial content and an initial anchor point;
[0020] Inputting the encoding result, the initial content and the initial anchor point into the Transformer decoder to obtain a decoding result;
[0021] Image detection and image segmentation are performed according to the pixel embedding map and the decoding result to obtain feature information of rice ears and rice leaves in the rice image to be segmented.
[0022] In some embodiments, the backbone feature extraction network includes a linear embedding layer, a plurality of Patch Merging and a plurality of Swin Transformer blocks, and the backbone feature extraction network is used to extract features from the preprocessed rice image to obtain a multi-scale feature map, including:
[0023] Dividing the preprocessed rice image into a plurality of image blocks;
[0024] Inputting the plurality of image blocks into the linear embedding layer to obtain a linear embedding result;
[0025] Inputting the linear embedding result into a first Swin Transformer block to obtain an output feature map of the first Swin Transformer block;
[0026] Inputting the output feature map of the first Swin Transformer block into the first Patch Merging to obtain the output feature map of the first Patch Merging;
[0027] The output feature map of the first Patch Merging is input into the next Sw in Transformer block connected to the first Patch Merging to obtain the output feature map of the next Swin Transformer block, and the output feature map of the next Sw in Transformer block is used as the input of the next Patch Merging connected to the next Swin Transformer block until the output feature map of the last Swin Transformer block is obtained to obtain a multi-scale feature map.
[0028] In some embodiments, the pixel encoder includes a plurality of encoder layers and a unified query selection module, and the flattened feature map and the multi-scale feature map are input into the pixel encoder to obtain an encoding result, a pixel embedding map, an initial content, and an initial anchor point, including:
[0029] Encoding the flattened feature map and the multi-scale feature map using the multiple encoder layers to obtain an encoding result;
[0030] Inputting the encoding result into the unified query selection module to obtain initial content and initial anchor point;
[0031] The feature map in the encoding result is added to the feature map in the multi-scale feature map to obtain a pixel embedding map.
[0032] In some implementations, performing image detection and image segmentation according to the pixel embedding map and the decoding result to obtain feature information of rice ears and rice leaves in the rice image to be segmented includes:
[0033] Perform a dot product between the pixel embedding map and the content query embedding in the Transformer decoder to obtain a mask;
[0034] Perform image segmentation according to the mask to obtain segmentation results of rice ears and rice leaves in the rice image to be segmented;
[0035] Inputting the decoding result into the linear layer for image detection to obtain detection results of rice ears and rice leaves in the rice image to be segmented;
[0036] The segmentation result and the detection result are used as feature information of rice ears and rice leaves in the rice image to be segmented.
[0037] In some implementations, calculating the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information includes:
[0038] Determining the number of rice ears and the number of rice leaves according to the detection results in the characteristic information;
[0039] Dividing the number of rice ears by the number of rice leaves, and calculating the ear emergence rate corresponding to the rice image to be segmented;
[0040] According to the segmentation result in the feature information, a rectangular frame is used to frame the rice leaf, and the leaf length of the rice leaf is obtained;
[0041] The number of horizontal pixel points at half of the leaf length is counted, and the number of horizontal pixel points is converted into an actual distance to calculate the leaf width corresponding to the rice image to be segmented.
[0042] In a second aspect, the present application also provides a statistical system for rice ear emergence rate and leaf width, the system comprising:
[0043] A rice image preprocessing unit, used for collecting a rice image to be segmented, and preprocessing the rice image to be segmented to obtain a preprocessed rice image;
[0044] An image detection and segmentation unit, used for inputting the preprocessed rice image into a trained rice detection and segmentation model to perform image detection and image segmentation, and obtaining feature information of rice ears and rice leaves in the rice image to be segmented, wherein the rice detection and segmentation model is constructed based on a Transformer model;
[0045] The ear emergence rate and leaf width statistical unit is used to calculate the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information.
[0046] In the third aspect, an embodiment of the present application also provides an electronic device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a statistical method for rice ear emergence rate and leaf width as described above.
[0047] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a statistical method for rice ear emergence rate and leaf width as described above.
[0048] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0050] Figure 1 It is a flow chart of an embodiment of a statistical method for rice ear emergence rate and leaf width provided by the present application;
[0051] Figure 2 It is a structural schematic diagram of a backbone feature extraction network in a best embodiment of a statistical method for rice ear emergence rate and leaf width provided by the present application;
[0052] Figure 3 It is a structural schematic diagram of a Transformer-based deep learning model in the best embodiment of the statistical method for rice ear emergence rate and leaf width provided in the present application;
[0053] Figure 4 It is a schematic diagram of the effect of extracting rice ear and rice leaf information features in the best embodiment of the statistical method for rice ear emergence rate and leaf width provided by the present application;
[0054] Figure 5 It is a structural schematic diagram of an embodiment of a statistical system for rice ear emergence rate and leaf width provided by the present application;
[0055] Figure 6 It is a schematic diagram of the structure of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION
[0056] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.
[0057] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0058] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0059] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.
[0060] The traditional method of counting ear emergence rate and leaf width is manual counting, which is inefficient, error-prone, and consumes a lot of manpower and time costs.
[0061] The existing statistical methods for rice ear emergence rate and leaf width involving machine vision technology usually use traditional target detection and image segmentation models to detect and segment rice ears and leaves, and then perform corresponding data statistical processing. This method can only stack the target detection model and the image segmentation model in sequence to complete the task, which leads to the complexity of the overall algorithm model. In addition, the traditional visual model relies on convolutional neural networks to extract image features, which is less effective when facing smaller targets such as rice ears and leaves, resulting in low statistical accuracy of rice ear emergence rate and leaf width.
[0062] In order to solve the problem of low statistical accuracy of the rice ear emergence rate and leaf width, the present application proposes a statistical method, system, equipment and medium for the rice ear emergence rate and leaf width.
[0063] Reference Figure 1 The present application embodiment provides a statistical method for rice ear emergence rate and leaf width, the method comprising the following steps:
[0064] Step S100, collecting a rice image to be segmented, and preprocessing the rice image to be segmented to obtain a preprocessed rice image;
[0065] Step S200, inputting the preprocessed rice image into the trained rice detection and segmentation model to perform image detection and image segmentation, and obtaining feature information of rice ears and rice leaves in the rice image to be segmented. The rice detection and segmentation model is constructed based on the Transformer model;
[0066] Step S300: Calculate the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information.
[0067] In this embodiment, by collecting the rice image to be segmented, and preprocessing the rice image to be segmented, the preprocessed rice image is obtained; the preprocessed rice image is input into the trained rice detection and segmentation model for image detection and image segmentation, and the characteristic information of the rice ear and the rice leaf in the rice image to be segmented is obtained, and the rice detection and segmentation model is constructed based on the Transformer model; according to the characteristic information, the ear emergence rate and the leaf width corresponding to the rice image to be segmented are statistically obtained. In this way, the characteristic information of the rice ear and the rice leaf in the rice image to be segmented is directly obtained through the trained rice detection and segmentation model, which can improve the statistical efficiency and reduce the error rate, reduce the manpower cost and time cost, and only need to train a rice detection and segmentation model constructed based on the Transformer model to obtain the detection result and the segmentation result, and then the ear emergence rate and the leaf width corresponding to the rice image to be segmented are statistically obtained, which improves the statistical efficiency and improves the statistical accuracy of the ear emergence rate and the leaf width.
[0068] The above-mentioned acquisition of the rice image to be segmented may be performed by using a drone, or by using a camera or other acquisition device to acquire the rice image to be segmented, and this embodiment does not impose any specific limitation.
[0069] The above-mentioned preprocessing of the rice image to be segmented to obtain the preprocessed rice image may be a preprocessing method such as de-noising and image enhancement to preprocess the segmented rice image to obtain the preprocessed rice image. The de-noising and image enhancement of this embodiment may be technologies known to those skilled in the art, and will not be described in detail in this embodiment.
[0070] The trained rice detection and segmentation model may be a model trained using a large number of rice images.
[0071] The above Transformer model is essentially an Encoder-Decoder architecture. Therefore, the middle part of the Transformer can be divided into two parts: the encoding component and the decoding component Transformer model (Encoder-Decoder architecture mode), where the encoding component consists of multiple layers of encoders (Encoder), and the decoding component also consists of the same number of layers of decoders (Decoder).
[0072] In some embodiments, the trained rice detection and segmentation model is obtained by:
[0073] Collecting a plurality of rice images, and preprocessing the plurality of rice images to obtain a plurality of preprocessed rice images;
[0074] Construct a hybrid matching loss function including cross entropy loss, GIoU loss and mask loss;
[0075] Based on the hybrid matching loss function, multiple preprocessed rice images are used to train the constructed rice detection and segmentation model to obtain a trained rice detection and segmentation model.
[0076] In this embodiment, multiple rice images are collected and preprocessed to obtain multiple preprocessed rice images; a hybrid matching loss function including cross entropy loss, GIoU loss and mask loss is constructed; based on the hybrid matching loss function, the constructed rice detection and segmentation model is trained using the preprocessed multiple rice images to obtain a trained rice detection and segmentation model. In this way, the hybrid matching loss function can make the prediction result of the trained rice detection and segmentation model more accurate.
[0077] The above-mentioned acquisition of multiple rice images may be achieved by using a drone as a tool for collecting rice image data, and a large number of rice images may be obtained by adjusting parameters such as the magnification and position of the drone lens.
[0078] The preprocessing of the plurality of rice images to obtain the preprocessed plurality of rice images may be preprocessing the plurality of rice images such as denoising, segmentation and image enhancement to obtain the preprocessed plurality of rice images.
[0079] The cross entropy loss mentioned above can be a loss function commonly used in classification problems in machine learning. It is a standard loss function for classification prediction in target detection tasks, which is used to calculate the difference between the predicted category and the true category.
[0080] The above GIoU loss can be a standard loss function for box prediction in object detection tasks, which is used to calculate the difference between the predicted box and the true box.
[0081] The above mask loss can adopt Dice loss, which is the loss function used in segmentation tasks to calculate the difference between the predicted mask and the true mask.
[0082] The above construction includes a hybrid matching loss function of cross entropy loss, GIoU loss and mask loss, which can be constructed by adding the cross entropy loss, GIoU loss and mask loss.
[0083] In some embodiments, the trained rice detection and segmentation model includes a backbone feature extraction network, a pixel encoder and a Transformer decoder, and the preprocessed rice image is input into the trained rice detection and segmentation model for image detection and image segmentation to obtain feature information of rice ears and rice leaves in the rice image to be segmented, including:
[0084] The backbone feature extraction network is used to extract features from the preprocessed rice image to obtain a multi-scale feature map;
[0085] Position encoding is performed on the multi-scale feature map to obtain a position-encoded feature map;
[0086] Flatten the position-encoded features to obtain a flattened feature map;
[0087] Input the flattened feature map and the multi-scale feature map into the pixel encoder to obtain the encoding result, pixel embedding map, initial content and initial anchor point;
[0088] Input the encoding result, initial content and initial anchor point into the Transformer decoder to obtain the decoding result;
[0089] Image detection and image segmentation are performed according to the pixel embedding map and the decoding result to obtain the characteristic information of rice ears and rice leaves in the rice image to be segmented.
[0090] In this embodiment, a trained rice detection and segmentation model including a backbone feature extraction network, a pixel encoder and a Transformer decoder is used for image detection and image segmentation, which can obtain more accurate image detection results and image segmentation results, thereby obtaining more accurate feature information of rice ears and rice leaves in the rice image to be segmented.
[0091] In some embodiments, the backbone feature extraction network includes a linear embedding layer, multiple Patch Merging and multiple Switch Transformer blocks, and the backbone feature extraction network is used to extract features from the preprocessed rice image to obtain a multi-scale feature map, including:
[0092] Divide the preprocessed rice image into multiple tiles;
[0093] Input multiple tiles into the linear embedding layer to obtain linear embedding results;
[0094] Input the linear embedding result into the first Swin Transformer block to obtain the output feature map of the first Swin Transformer block;
[0095] Input the output feature map of the first Swin Transformer block into the first Patch Merging to obtain the output feature map of the first Patch Merging;
[0096] The output feature map of the first Patch Merging is input into the next SwinTransformer block connected to the first Patch Merging to obtain the output feature map of the next Swin Transformer block, and the output feature map of the next Swin Transformer block is used as the input of the next Patch Merging connected to the next Swin Transformer block until the output feature map of the last Swin Transformer block is obtained to obtain a multi-scale feature map.
[0097] In this embodiment, feature extraction is performed on the preprocessed rice image through a linear embedding layer, multiple Patch Merging blocks, and multiple Swin Transformer blocks, so that multi-level features, namely, multi-scale features, can be extracted. Each scale can provide different information for learning, laying a good data foundation for obtaining a rice detection and segmentation model with more accurate detection and segmentation results in the later stage.
[0098] In some embodiments, the pixel encoder includes a plurality of encoder layers and a unified query selection module, and the flattened feature map and the multi-scale feature map are input into the pixel encoder to obtain the encoding result, the pixel embedding map, the initial content and the initial anchor point, including:
[0099] Multiple encoder layers are used to encode the flattened feature map and the multi-scale feature map to obtain an encoding result;
[0100] Input the encoding result into the unified query selection module to obtain the initial content and initial anchor point;
[0101] The feature map in the encoding result is added to the feature map in the multi-scale feature map to obtain the pixel embedding map.
[0102] In this embodiment, multiple encoder layers are used to encode the flattened feature map and the multi-scale feature map to obtain an encoding result; the encoding result is input into a unified query selection module to obtain an initial content and an initial anchor point; the feature map in the encoding result is added to the feature map in the multi-scale feature map to obtain a pixel embedding map. In this way, by using multiple encoder layers and a unified query selection module to extract multiple features, a rich feature representation is provided for subsequent object detection and segmentation tasks, thereby improving the accuracy of the detection results and segmentation results.
[0103] In some implementations, image detection and image segmentation are performed according to the pixel embedding map and the decoding result to obtain feature information of rice ears and rice leaves in the rice image to be segmented, including:
[0104] Take the dot product of the pixel embedding map and the content query embedding in the Transformer decoder to get the mask;
[0105] Perform image segmentation according to the mask to obtain the segmentation results of rice ears and rice leaves in the rice image to be segmented;
[0106] The decoding result is input into the linear layer for image detection to obtain the detection results of rice ears and rice leaves in the rice image to be segmented;
[0107] The segmentation results and detection results are used as the feature information of rice ears and rice leaves in the rice image to be segmented.
[0108] In this embodiment, a mask is obtained by performing a dot product between a pixel embedding map and a content query embedding in a Transformer decoder; image segmentation is performed according to the mask to obtain segmentation results of rice ears and rice leaves in the rice image to be segmented; the decoding result is input into a linear layer for image detection to obtain detection results of rice ears and rice leaves in the rice image to be segmented; the segmentation result and the detection result are used as feature information of rice ears and rice leaves in the rice image to be segmented. In this way, only a constructed rice detection and segmentation model needs to be trained to directly obtain detection results and segmentation results, which can improve statistical efficiency and reduce error rate, and reduce labor costs and time costs.
[0109] In some implementations, counting the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information includes:
[0110] Determine the number of rice ears and the number of rice leaves according to the detection results in the characteristic information;
[0111] Divide the number of rice ears by the number of rice leaves to calculate the ear emergence rate corresponding to the rice image to be segmented;
[0112] According to the segmentation result in the feature information, the rice leaf is framed by a rectangular frame, and the leaf length of the rice leaf is obtained;
[0113] The number of horizontal pixels at half the length of the leaf is counted, and the number of horizontal pixels is converted into actual distance to calculate the leaf width corresponding to the rice image to be segmented.
[0114] In this embodiment, accurate detection results and segmentation results are obtained to statistically calculate the ear emergence rate and leaf width corresponding to the rice image to be segmented, thereby improving the statistical efficiency and the statistical accuracy of the ear emergence rate and leaf width.
[0115] The above-mentioned determination of the number of rice ears and the number of rice leaves based on the detection results in the feature information may be that the detection results include the detected categories and the number of categories, and therefore the number of rice ears and the number of rice leaves can be known.
[0116] The above-mentioned conversion of the number of horizontal pixel points into actual distance and counting the leaf width corresponding to the rice image to be segmented can be carried out by adopting any solution in the prior art known to technical personnel in this field, and the number of horizontal pixel points is converted into actual distance. The calculated actual distance can be used to count the leaf width corresponding to the rice image to be segmented.
[0117] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below:
[0118] The traditional method of counting ear emergence rate and leaf width is manual counting. The disadvantages of relying entirely on manual counting of ear emergence rate and leaf width are: low statistical efficiency and prone to errors, because manual counting can only rely on staff to count rice ears and rice leaves one by one, and the counting process is extremely susceptible to personal subjective judgment. It also consumes a lot of manpower and time costs, especially in the large-scale rice field data collection process, which requires the participation of a large number of staff, which greatly increases various costs.
[0119] At present, the statistical methods for ear emergence rate and leaf width involving machine vision technology usually use traditional target detection and image segmentation models to detect and segment rice ears and rice leaves respectively, and then perform corresponding data statistical processing. That is, traditional visual models are not compatible with target detection and image segmentation tasks, so the target detection model and image segmentation model can only be stacked in sequence to complete the task.
[0120] This method of counting the ear rate and leaf width using traditional detection and segmentation models based on deep learning has the following disadvantages: low efficiency, the data needs to be detected and segmented by trained detection models and segmentation models in turn, and the process is cumbersome. The labeling cost is high. Since a detection model and a segmentation model need to be trained, the same batch of data needs to be labeled twice to meet the training requirements of the detection model and the segmentation model respectively. The accuracy of detection and segmentation is extremely dependent on the quality and quantity of data, and the performance is not ideal when faced with insufficient sample size or poor data quality. Model updating and maintenance are difficult because two deep learning models need to be updated and maintained at the same time.
[0121] In order to solve the problems existing in the above-mentioned traditional methods of counting ear emergence rate and leaf width and the methods of counting ear emergence rate and leaf width based on traditional detection and segmentation models of deep learning, this embodiment adopts a deep learning model based on Transformer to replace the two models of target detection and image segmentation of the existing method, which not only reduces the complexity of the overall model, but also introduces the Transformer model with powerful feature extraction capabilities, thereby improving the overall efficiency and accuracy of the model. The specific technical solutions of this embodiment include the following contents:
[0122] (1) Collect RGB images of rice (i.e., rice images).
[0123] This embodiment uses a drone as a tool for collecting rice RGB image data, and obtains a large amount of rice image data by adjusting parameters such as the drone lens magnification.
[0124] (2) Construct a dataset.
[0125] The collected rice RGB image data is used to extract and calibrate rice ears and leaves, and a sample data set of rice ears and leaves is established. According to the sample data set, the training set, validation set and test set are obtained. The specific implementation is as follows: the data annotation tool LabelMe is used to mark the rice ears and leaves in the collected rice images. The marking content mainly includes the input label, image size and storage path information, and the XML format file generated by the marking information corresponds to the jpg format file.
[0126] (3) Preprocess the image data in the sample dataset.
[0127] In order to eliminate the information irrelevant to rice ears and leaves in the collected rice images, enhance the detectability of relevant information and simplify the data to the maximum extent, the collected original image data is preprocessed by denoising, segmentation and image enhancement.
[0128] 1) Grayscale and filtering: Grayscale the collected color images, that is, process the three-color images into monochrome images, convert the RGB model into a grayscale L model, and use the rgb2gray function of matlab to grayscale the original color image of the sample to simplify the image information. Use the medfilt2 method of matlab to perform median filtering on the grayscale image to eliminate the impulse noise caused by shooting, lighting, and shadows in the sample image.
[0129] 2) Ostu threshold segmentation: remove the background of the sample image to facilitate the subsequent extraction of feature parameters of the sample image.
[0130] 3) Image data enhancement: Using five data enhancement methods, including HSV transformation, flipping, perspective, translation, and collage, on sample images can significantly improve the diversity and complexity of image data, which helps to improve the generalization ability and robustness of the model.
[0131] (4) Construct a Transformer-based deep learning model (i.e., rice detection and segmentation model).
[0132] Establish a Transformer-based deep learning model and use the preprocessed data set in step (3) to train and verify the model. Figure 2 and Figure 3 , Figure 2 This is the structural diagram of the backbone feature extraction network. Figure 2 Image represents the input image, Patch Partition represents image partitioning, Linear Embedding represents linear embedding layer, Swin Transformer Blocks represents Swin Transformer block, Patch Merging represents image merging, stage represents stage, and Multi-level feature maps represents multi-level features. Figure 3 This is a schematic diagram of the structure of the Transformer-based deep learning model. Figure 3Image in it represents the input image, Multi-Scale Features represents multi-level features, Encoder Layers represents multiple encoder layers, United Query Selection represents unified query selection, DecoderLayers represents multiple decoder layers, Hybid Matching represents hybrid matching, United DN represents unified denoising training, upsample represents upsampling, Init Contents represents initial content, Init Anchors represents initial anchors, Queryembedding represents query encoding (i.e. content query embedding), Maskes represents predicted masks, Boxes represents predicted boxes, Classes represents classification results, Position Embedding represents position encoding, and Flatten represents flattening operation. The model structure of the Transformer-based deep learning model is as follows:
[0133] 1) Use Swin Transformer as the backbone feature extraction network, and set the embedding dimension and window size to 192 and 12 respectively. The input image first passes through the Patch splitting module and is divided into non-overlapping small blocks called patches. For RGB images, each small block is regarded as a Patch Token. Then the Patch Tokens enter the Linear Embedding linear embedding layer and are projected to obtain the Linear Embedding. Then, these PatchTokens are sent to multiple Swin Transformer Blocks. Each Block includes two main self-attention mechanisms: W-MSA (Wi ndow-based Multi-Head Self-Attention) and SW-MSA (Shifted Window Multi-Head Self-Attention). After each Swin Transformer Block, it will pass through the Patch Merging (for patch merging) layer, and the number of Tokens will be reduced through the Patch Merging layer to produce a hierarchical representation. And before and after each Swin Transformer Block, LayerNorm and Dropout operations will be applied to stabilize the training and improve the generalization ability of the model. Finally, the multi-level features of the image (i.e., multi-scale feature maps) are obtained.
[0134] 2) Pixel encoder. The multi-level features of the image extracted by Swin Transformer come from different layers of the backbone network, usually including feature representations of different resolutions. First, the multi-scale feature map output by Swin Transformer is positionally encoded to generate a feature map with position information. The feature map with position information is flattened and combined with the multi-scale feature map output by Swin Transformer. Figure 1 The dense features are input into the encoder layer (the pixel encoder contains multiple encoder layers) to obtain dense features (i.e., the encoding results output by the pixel encoder). After the dense features are input into the unified query selection module, the initial content and initial anchor information are obtained. The initial content and initial anchor information need to be input into the Transformer decoding later. Then feature fusion is performed to obtain a high-resolution pixel embedding map by fusing the 1 / 4 resolution feature map in the backbone with the upsampled 1 / 8 resolution feature map in the Transformer encoder. And the Transformer encoder is used to further enhance the features. Finally, mask-enhanced box initialization is performed. This method improves the performance of the model by using the predicted mask to initialize the decoder box. Through the above steps, the model effectively processes the features extracted by the backbone and generates multi-scale feature maps, providing rich feature representations for subsequent target detection and segmentation tasks.
[0135] 3) Transformer decoder. The Transformer decoder receives the multi-scale feature map (including Figure 3 The decoder is composed of multiple decoder layers, each of which includes a self-attention mechanism and an encoder-decoder attention mechanism. In the self-attention layer, each position query (Q) of the decoder is compared with other position queries to calculate an attention score. This allows the model to take into account all position information in the input sequence when generating output. The encoder-decoder attention layer allows the decoder to utilize the output information of the encoder. In this process, the decoder's query (Q) interacts with the encoder's key (K) and value (V) to obtain information related to the current decoding step. After the self-attention and encoder-decoder attention layers, the decoder output is passed through a forward network that can further process and refine the feature representation.
[0136] The mask prediction branch uses the predicted masks to initialize the decoder's boxes, thereby improving the performance of the model. This branch involves dot-producting the pixel embedding map with the content query embedding to predict a set of binary masks, i.e., performing pixel embedding mapping, and dot-producting each content query embedding in the decoder with the pixel embedding map to obtain the output mask. This process involves convolutional layers that map the channel dimension to the Transformer hidden dimension, and a simple interpolation function that performs feature map upsampling. Finally, the output of the Transformer decoder passes through a linear layer that maps the embedding at each position to the probability of each word in the predicted vocabulary to complete the object detection and segmentation task.
[0137] 4) Loss function. The loss function applied by the Transformer-based deep learning model includes the following parts: A. Cross entropy loss, which is the standard loss function for classification prediction in target detection tasks, used to calculate the difference between the predicted category and the true category. B. GIoU loss, which is the standard loss function for box prediction in target detection tasks, used to calculate the difference between the predicted box and the true box. C. Mask loss, usually using Dice loss. This is the loss function used in segmentation tasks, used to calculate the difference between the predicted mask and the true mask. D. Hybrid matching loss, since the predicted box and mask are two parallel heads in a loosely coupled manner, a hybrid matching loss is introduced to encourage more accurate and consistent matching results. The hybrid matching loss is calculated as: HMLoss = Lce + Lgiou + Ldice, where HMLoss represents hybrid matching loss, Lce represents cross entropy loss, Lgiou represents GIoU loss, and Ldice represents mask loss.
[0138] (5) Extract rice ears and rice leaves information from rice RGB images.
[0139] Based on the deep learning model constructed in step (4) (i.e., the rice detection and segmentation model), the information features of rice ears and rice leaves in the rice RGB image are extracted, and then the ear rate is calculated by counting the number of rice ears and rice leaves appearing in the image, and the leaf width of the rice leaf is calculated using the segmented rice leaf information.
[0140] Specifically, the effect of extracting rice ear and leaf information features through the rice detection segmentation model is shown in the figure below: Figure 4 As shown in the figure. The calculation of ear emergence rate is: the number of rice ears detected in the rice RGB image is divided by the number of rice leaves detected in the rice RGB image. The calculation of rice leaf width is: after obtaining the rice leaf segmentation result, use a rectangular frame to frame the target, first obtain the leaf length, then count the number of horizontal pixels at 1 / 2 of the leaf length, and convert the number of horizontal pixels into actual distance to calculate the leaf width.
[0141] It should be noted that the leaf width is calculated by converting the number of horizontal pixels into the actual distance in this embodiment, which can be any solution in the prior art known to those skilled in the art, and this embodiment does not make any specific limitation. For example, the number of horizontal pixels of the line connecting the two points with the longest distance is taken as the maximum leaf width.
[0142] Compared with the prior art, the technical solution of this embodiment has the following advantages:
[0143] This embodiment uses a Transformer-based deep learning model to accurately detect and segment the characteristic information of rice ears and rice leaves on rice images collected by drones, and uses this to calculate and count the data of rice heading rate and rice leaf width, completely avoiding the influence of human subjective factors on the statistical heading rate and leaf width, greatly improving the credibility of the statistics of rice heading rate and leaf width data, while also improving the statistical efficiency and accuracy of rice heading rate and leaf width data, and significantly reducing labor costs.
[0144] Reference Figure 5 The embodiment of the present application further provides a rice ear emergence rate and leaf width statistics system, the system comprising a rice image preprocessing unit 100, an image detection and segmentation unit 200 and a rice ear emergence rate and leaf width statistics unit 300, wherein:
[0145] The rice image preprocessing unit 100 is used to collect the rice image to be segmented and preprocess the rice image to be segmented to obtain the preprocessed rice image;
[0146] The image detection and segmentation unit 200 is used to input the preprocessed rice image into the trained rice detection and segmentation model to perform image detection and image segmentation, and obtain feature information of rice ears and rice leaves in the rice image to be segmented. The rice detection and segmentation model is constructed based on the Transformer model;
[0147] The ear emergence rate and leaf width statistical unit 300 is used to calculate the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information.
[0148] It should be noted that since the statistical system for rice ear emergence rate and leaf width in this embodiment and the statistical method for rice ear emergence rate and leaf width mentioned above are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the system embodiment and will not be described in detail here.
[0149] Reference Figure 6 The embodiment of the present application further provides an electronic device, the electronic device comprising:
[0150] at least one memory;
[0151] at least one processor;
[0152] at least one program;
[0153] The programs are stored in the memory, and the processor executes at least one program to implement the statistical method of rice ear emergence rate and leaf width mentioned above in the present disclosure.
[0154] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.
[0155] The electronic device according to the embodiment of the present application is described in detail below.
[0156] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;
[0157] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes the statistical method of rice ear emergence rate and leaf width in the embodiment of the present disclosure.
[0158] Input / output interface 1800, used to implement information input and output;
[0159] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);
[0160] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );
[0161] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .
[0162] The embodiments of the present disclosure also provide a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned statistical method for rice ear emergence rate and leaf width.
[0163] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0164] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.
[0165] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present disclosure and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0166] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0167] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0168] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0169] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0170] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0172] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0173] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs. The above is a detailed description of the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Within the scope of knowledge possessed by ordinary technicians in the relevant technical field, various changes can be made without departing from the purpose of the present application.
[0174] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.
Claims
1. A statistical method for rice ear emergence rate and leaf width, characterized in that: The method comprises: Collecting a rice image to be segmented, and preprocessing the rice image to be segmented to obtain a preprocessed rice image; Inputting the preprocessed rice image into a trained rice detection and segmentation model for image detection and image segmentation to obtain feature information of rice ears and rice leaves in the rice image to be segmented, wherein the rice detection and segmentation model is constructed based on a Transformer model; The ear emergence rate and leaf width corresponding to the rice image to be segmented are statistically calculated according to the feature information.
2. The statistical method for rice ear emergence rate and leaf width according to claim 1, characterized in that: The trained rice detection and segmentation model is obtained by: Collecting a plurality of rice images, and preprocessing the plurality of rice images to obtain a plurality of preprocessed rice images; Construct a hybrid matching loss function including cross entropy loss, GIoU loss and mask loss; Based on the hybrid matching loss function, the constructed rice detection and segmentation model is trained using the preprocessed multiple rice images to obtain the trained rice detection and segmentation model.
3. The statistical method for rice ear emergence rate and leaf width according to claim 1, characterized in that: The trained rice detection and segmentation model includes a backbone feature extraction network, a pixel encoder and a Transformer decoder. The pre-processed rice image is input into the trained rice detection and segmentation model for image detection and image segmentation to obtain feature information of rice ears and rice leaves in the rice image to be segmented, including: Using the backbone feature extraction network to extract features from the preprocessed rice image to obtain a multi-scale feature map; Performing position encoding on the multi-scale feature map to obtain a position-encoded feature map; Performing a flattening operation on the position-encoded features to obtain a flattened feature map; Inputting the flattened feature map and the multi-scale feature map into the pixel encoder to obtain an encoding result, a pixel embedding map, an initial content and an initial anchor point; Inputting the encoding result, the initial content and the initial anchor point into the Transformer decoder to obtain a decoding result; Image detection and image segmentation are performed according to the pixel embedding map and the decoding result to obtain feature information of rice ears and rice leaves in the rice image to be segmented.
4. The statistical method for rice ear emergence rate and leaf width according to claim 3, characterized in that: The backbone feature extraction network includes a linear embedding layer, multiple Patch Merging and multiple Swin Transformer blocks. The backbone feature extraction network is used to extract features from the preprocessed rice image to obtain a multi-scale feature map, including: Dividing the preprocessed rice image into a plurality of image blocks; Inputting the plurality of image blocks into the linear embedding layer to obtain a linear embedding result; Inputting the linear embedding result into a first Swin Transformer block to obtain an output feature map of the first Swin Transformer block; Inputting the output feature map of the first Swin Transformer block into the first Patch Merging to obtain the output feature map of the first Patch Merging; The output feature map of the first Patch Merging is input into the next Sw in Transformer block connected to the first Patch Merging to obtain the output feature map of the next Swin Transformer block, and the output feature map of the next Sw in Transformer block is used as the input of the next Patch Merging connected to the next Swin Transformer block until the output feature map of the last Swin Transformer block is obtained to obtain a multi-scale feature map.
5. The statistical method for rice ear emergence rate and leaf width according to claim 3, characterized in that: The pixel encoder includes a plurality of encoder layers and a unified query selection module, wherein the flattened feature map and the multi-scale feature map are input into the pixel encoder to obtain an encoding result, a pixel embedding map, an initial content, and an initial anchor point, including: Encoding the flattened feature map and the multi-scale feature map using the multiple encoder layers to obtain an encoding result; Inputting the encoding result into the unified query selection module to obtain initial content and initial anchor point; The feature map in the encoding result is added to the feature map in the multi-scale feature map to obtain a pixel embedding map.
6. The statistical method for rice ear emergence rate and leaf width according to claim 3, characterized in that: The performing of image detection and image segmentation according to the pixel embedding map and the decoding result to obtain feature information of rice ears and rice leaves in the rice image to be segmented includes: Perform a dot product between the pixel embedding map and the content query embedding in the Transformer decoder to obtain a mask; Perform image segmentation according to the mask to obtain segmentation results of rice ears and rice leaves in the rice image to be segmented; Inputting the decoding result into the linear layer for image detection to obtain detection results of rice ears and rice leaves in the rice image to be segmented; The segmentation result and the detection result are used as feature information of rice ears and rice leaves in the rice image to be segmented.
7. The statistical method for rice ear emergence rate and leaf width according to claim 6, characterized in that: The step of calculating the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information includes: Determining the number of rice ears and the number of rice leaves according to the detection results in the characteristic information; Dividing the number of rice ears by the number of rice leaves, and calculating the ear emergence rate corresponding to the rice image to be segmented; According to the segmentation result in the feature information, a rectangular frame is used to frame the rice leaf, and the leaf length of the rice leaf is obtained; The number of horizontal pixel points at half of the leaf length is counted, and the number of horizontal pixel points is converted into an actual distance to calculate the leaf width corresponding to the rice image to be segmented.
8. A statistical system for rice ear emergence rate and leaf width, characterized in that: The system comprises: A rice image preprocessing unit, used for collecting a rice image to be segmented, and preprocessing the rice image to be segmented to obtain a preprocessed rice image; An image detection and segmentation unit, used for inputting the preprocessed rice image into a trained rice detection and segmentation model to perform image detection and image segmentation, and obtaining feature information of rice ears and rice leaves in the rice image to be segmented, wherein the rice detection and segmentation model is constructed based on a Transformer model; The ear emergence rate and leaf width statistical unit is used to calculate the ear emergence rate and leaf width corresponding to the rice image to be segmented according to the feature information.
9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the statistical method for rice ear emergence rate and leaf width as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the statistical method for rice ear emergence rate and leaf width as described in any one of claims 1 to 7.