A method and system for identifying drifting and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model

Abstract

The present invention discloses a method and system for identifying floating and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model, which belongs to the field of crop target detection and identification, and is used to improve the detection accuracy of floating and missed rice seedlings. Technical points: Using a drone to obtain pictures of machine-transplanted rice seedlings as a sample data set, and performing data preprocessing to obtain a self-built data set of rice seedlings. A network detection model is established using the residual network ResNet50 fused with the FPN feature pyramid; the improved Faster-RCNN model is optimized and trained; the trained Faster-RCNN improved algorithm is used to classify and identify rice seedlings, and the seedling line fitting is performed using the least squares method to determine the location of missed rice seedlings. The improved Faster-RCNN network model is used to classify and detect the number of floating rice seedlings, the number of surviving rice seedlings, the number of missing leaves, and the number of missed rice seedlings, and a higher recognition accuracy can be obtained under complex conditions.

Description

Technical Field

[0001] The present invention relates to the fields of crop target detection and recognition, computer target detection, and more particularly to a method and system for identifying drifted and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model. Background Art

[0002] Rice is one of the world's most important food crops and my country's leading crop, accounting for over 40% of total grain production. In 2023, my country's rice planting area reached 28.9491 million hectares, and rice production reached 206.6032 million tons. Transplanting is a critical step in rice cultivation, and its quality determines rice yield. Mechanized transplanting is highly efficient, saves water and fertilizer, and ensures more uniform seedlings. Even spacing between plants facilitates ventilation, increasing or stabilizing crop yields and gradually becoming the primary method of rice cultivation. However, when encountering uneven fields, residual straw, or improper transplanter operation, seedlings can be easily missed or drifted, necessitating later replanting. During replanting, seedling identification primarily relies on manual assessment, a traditional method that is not only time-consuming and labor-intensive, but also suffers from low efficiency and lack of objectivity. Using deep learning to extract features through training and learning of data sets can quickly and accurately identify seedlings. In terms of accuracy and real-time performance, it far exceeds traditional working methods and is an effective method for detecting drifting and missed seedlings transplanted by machines.

[0003] For example, the prior art with document number CN111932551A discloses a method for detecting the missed planting rate of a rice transplanter, which includes the following steps: (1) an image acquisition module acquires a color image of a rice seedling field and transmits it to an image processing and analysis module; (2) the image processing and analysis module processes the color image of the rice seedling field to obtain an optimized seedling binary image and obtain the connected area of ​​the seedling; (3) the pixel projection method is used to accumulate the white pixel values ​​of each row and column of the optimized seedling binary image to determine the left and right boundaries and the upper and lower boundaries of the desired target area, and the optimized seedling binary image is divided into multiple target area units; (4) after the image processing and analysis module continuously processes multiple images, the number of blank target area units and the number of all target area units are counted; (5) the ratio of the number of blank target area units to the number of all target area units is calculated to obtain the missed planting rate. This prior art realizes the detection of the missed planting rate of a rice transplanter. This prior art does not use a deep learning network model to perform target recognition on rice seedling images.

[0004] Therefore, it is urgent to propose a method for identifying floating and missed rice seedlings based on deep learning, so as to realize the rapid promotion of intelligent identification technology for floating and missed rice seedlings. Summary of the Invention

[0005] The technical problems to be solved by the present invention are:

[0006] In view of this, the present invention provides a method and system for identifying drifting and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model. This method greatly improves the time-consuming and labor-intensive problems of traditional manual seedling replacement, and improves the detection accuracy of rice seedlings by designing a deep learning network model.

[0007] The technical solution adopted by the present invention to solve the above technical problems is:

[0008] A method and system for identifying drifting and missed rice seedlings during machine transplanting based on an improved Faster-RCNN model, the specific steps are as follows:

[0009] Step S1: The UAV obtains image data of rice seedlings planted in the field;

[0010] Step S2: pre-processing and labeling the rice seedlings in the field to obtain a pre-processed image;

[0011] Step S3: Selecting the original Faster-RCNN (Faster-Regions with CNN features) deep learning model and optimizing it according to the characteristics of rice seedlings to obtain an improved Faster-RCNN model;

[0012] Step S4: importing the preprocessed rice seedling dataset into the improved Faster-RCNN model for iterative training;

[0013] Step S5: importing the rice seedling image to be tested into the trained Faster-RCNN model to perform rice seedling detection;

[0014] Step S6: fitting the predicted rice seedlings using the least squares method to find the positions of the missed rice seedlings.

[0015] Furthermore, the specific contents of step 1 include:

[0016] Step S11: The drone is adjusted to a corresponding cruise constant speed photography mode, and the plot where the drone is operating is selected on the remote control display for data collection;

[0017] Step S12: The flight parameters of the drone are as follows: the shooting distance between the drone and the rice seedlings is 1.5 meters to 2 meters, the DJI GS Pro mode is used for surveying and mapping aerial photography, the drone flight speed is 1 meter / s, the shooting angle is -90 degrees, the shooting interval is 2 seconds, and the front-to-back overlap rate is 60%.

[0018] Step S13: The drone is a DJI Mavic 3T equipped with a DJI RC-N1 remote controller and four original batteries, which are sufficient for a day's flight operation.

[0019] Furthermore, the specific contents of step 2 include:

[0020] Step S21: sorting the data of the collected field rice seedlings, removing the unrecognizable blurred images and the images of land that were taken improperly;

[0021] Step S22: adjusting the brightness of the reflective area of ​​the rice seedling image reflected from the water surface, and smoothing the rice seedling image with a complex soil environment using a median filter;

[0022] Step S23: Perform reasonable scaling operations on the image, adjust the pixels, and unify the dataset size to facilitate subsequent model training;

[0023] Step S24: performing operations such as flipping and mirroring on the rice seedling images with fewer categories in the dataset to increase the richness of the dataset;

[0024] Step S25: Divide the sorted data set into a training data set and a test data set in a ratio of 8:2;

[0025] Step S26: Label the cleaned rice seedling dataset according to the proposed standard. The proposed standard is: using a rectangular frame as a reference, record the coordinates of the upper left corner and lower right corner of the rice seedling to completely include the rice seedling. According to the definitions of normal seedlings, floating seedlings, and missed seedlings in GB / T 6243-2017 "Test Method for Rice Transplanters", qualified seedlings, floating seedlings, and seedlings with less than 4 leaves are classified and labeled.

[0026] The construction of the improved Faster-RCNN network model specifically includes: selecting the two-stage network model Faster-RCNN in the deep learning network according to the characteristics of single rice seedling identification in the field;

[0027] Replace the backbone network VGG16 in the Faster-RCNN network with the residual network Res Net50;

[0028] Replace the region of interest pooling RoIPooling layer in the Faster-RCNN network with the RoIAlign layer;

[0029] The main difference between floating seedlings and qualified seedlings is that the roots of floating seedlings are not planted in the soil. In order to effectively distinguish the differences between floating seedlings and qualified seedlings, the FPN feature pyramid is added to extract the feature information of rice seedlings to obtain more accurate rice seedling information.

[0030] Furthermore, the specific contents of step 3 include:

[0031] Step S31: inputting the pre-processed field rice seedling image data into the backbone network of the model head to extract feature information from the image data set to generate a multi-scale feature fusion information graph. The backbone network of the improved model is the residual network ResNet50;

[0032] Step S32: Use the FPN feature pyramid to perform multi-scale fusion of the rice seedling image feature information. By constructing feature pyramids at multiple scales to extract rice seedling features at different scales, ResNet50 can accurately extract spatial information when performing feature extraction on rice seedlings.

[0033] Step S33: The multi-scale feature fusion information graph obtained from the backbone network is used to generate a large number of candidate boxes through the region proposal network RPN, and then the obtained candidate boxes are mapped to the multi-scale feature fusion information graph to obtain a new feature matrix;

[0034] Step S34: Use the bilinear interpolation method in RoIAlign to process the feature matrix obtained by the region proposal network to obtain a 7×7 feature map;

[0035] Step S35: The feature information map is classified by the fully connected layer, and the offset of each target is obtained by bounding box regression to obtain a more accurate anchor point box and accurately locate the target.

[0036] Furthermore, the specific contents of step 4 include:

[0037] Step S41: importing the preprocessed rice seedling dataset into the improved Faster-RCNN network model and performing data classification;

[0038] Step S42: Design and adjust the training parameters of the deep learning network model, including model parameters such as the number of iterations and learning rate;

[0039] Step S43: pre-training the improved Faster-RCNN network model using the ImageNet dataset to obtain pre-training weights, and then combining the pre-training weights with the pre-processed rice seedling training dataset to separately train and improve the RPN network layer parameters in the Faster-RCNN network model;

[0040] Step S44: Fix the trained RPN network parameters, use the object proposal boxes generated from the RPN network to train the Fast-RCNN network parameters, and fine-tune the fully connected layers in the Fast-RCNN network;

[0041] Step S45: The RPN network and the Fast-RCNN network share the front-end network parameters at the same time to form a unified network.

[0042] Furthermore, the optimized training parameters of the improved Faster-RCNN network model include the following specific contents:

[0043] Import the pre-trained image dataset into the deep learning network model training network. In each iterative training, randomly import a batch of images as the dataset for feature information extraction. Use backpropagation and batch normalization for optimization training. The iterative training process is performed i times, and the update mode is set to "step".

[0044] Furthermore, the specific contents of step 5 include:

[0045] Step S51: importing the rice seedlings to be tested into the improved Faster-RCNN network model;

[0046] Step S52: The image to be tested is input into the backbone network residual network ResNet50 to extract feature information, and the feature information is fused through the FPN feature pyramid to generate a multi-scale feature fusion information graph;

[0047] Step S53: Generate candidate boxes using the RPN network, and map the generated candidate boxes to the multi-scale feature fusion information graph to obtain a new feature matrix;

[0048] Step S54: Use RoIAlign to interpolate the newly generated feature matrix to generate a 7×7 feature map;

[0049] Step S55: The feature information map is classified by the fully connected layer, and the precise position coordinates of each target are obtained by bounding box regression, and the target is accurately positioned.

[0050] Furthermore, the specific contents of step 6 include:

[0051] The improved Faster-RCNN network model predicts the specific location of rice seedlings and marks the candidate box, and finds the center point and position coordinates of the candidate box;

[0052] The least square method is used to fit the rice seedlings in a straight line, fitting the horizontal and vertical lines respectively, and the central intersection coordinates are obtained as the predicted points of the seedlings;

[0053] The predicted center point that falls within the candidate frame is identified as the rice seedling position, and the center point without a candidate frame is recorded as the rice seedling missed planting position.

[0054] A system for identifying floating and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model. The system has a program module corresponding to the steps of any of the above technical solutions, and when running, executes the steps of the above-mentioned method for identifying floating and missed rice seedlings in machine-transplanted rice seedlings based on the improved Faster-RCNN model.

[0055] A computer-readable storage medium stores a computer program configured to, when called by a processor, implement the steps of a method for identifying drifting and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model.

[0056] A device for identifying floating and missed rice seedlings includes a processor and a memory for storing data. When the processor executes a stored program, it implements the steps of a method for identifying floating and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model. This is equivalent to providing a computer device including a processor and a memory for storing data, characterized in that when the processor executes the stored program, it implements the method for identifying floating and missed rice seedlings.

[0057] The device for identifying floating and missed rice seedlings also includes a camera, which is installed on a rice transplanter to identify and locate floating and missed rice seedlings in real time, thereby controlling the rice transplanter to accurately replant rice seedlings.

[0058] Summarizing the above steps: the present invention uses a deep learning network model to identify floating and missed rice seedlings, and studies a more time-saving and labor-saving operation method using a deep learning network under the conditions of traditional manual identification; the traditional Faster-RCNN network model is improved, the original VGG network model is discarded, and the residual network ResNet50 is selected as the backbone network of the model to perform feature extraction on the rice seedling dataset, and the FPN feature pyramid is used to perform feature fusion to generate a multi-scale feature fusion information graph; RoI Align replaces the original RoIPooling, and the bilinear interpolation method is used to replace the original two quantization operations. Finally, the maximum pooling operation is performed to improve the accuracy of target recognition.

[0059] The present invention has the following beneficial technical effects: the remote sensing data set and monitoring method of rice seedlings extracted based on drones can greatly improve work efficiency. Deep learning is currently the most promising modern method for image recognition. Applying this technology to rice seedling identification can improve the efficiency and quality of rice seedling replenishment, and achieve high-quality and high-yield rice. The present invention uses drones to obtain field rice seedling pictures as sample data sets, performs data preprocessing and annotation on the rice field samples, and performs image cleaning to obtain preprocessed image data. A deep learning network detection model is established and improved using the residual network ResNet50 and the FPN feature pyramid. The preprocessed rice seedling images are used to optimize and train the improved Faster-RCNN network model; the trained Faster-RCNN network model recognizes the rice seedling images to be tested, and the recognized images are fitted using the least squares method to find the positions where rice seedlings are missed. The present invention adopts a deep learning recognition method to significantly improve the existing rice seedling recognition technology. Through the improved Faster-RCNN network model, the number of floating rice seedlings, the number of surviving rice seedlings, the number of missing leaves and the number of missed plantings are classified and identified, and a high recognition accuracy can still be achieved under complex conditions.

[0060] The research on identifying floating and missed rice seedlings according to the present invention has the following advantages:

[0061] 1. Compared with the traditional rice seedling random sampling identification method, the present invention provides a host computer method based on a deep learning network model to perform target recognition on rice seedling pictures, which can reduce the impact of the traditional large workload and heavy tasks on outdoor workers, not only saving manpower, but also improving the work efficiency of seedling replacement operations.

[0062] 2. The present invention uses the residual network ResNet50 and the FPN feature pyramid as the backbone network of the Faster-RCNN network model. This network can train very deep neural networks, avoid the gradient vanishing problem, and improve the accuracy and generalization ability of the model.

[0063] 3. The present invention adopts the bilinear interpolation method to replace the original two quantization operations and replaces the original RoIPooling with RoI Align, thereby improving the accuracy of target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of the method for identifying floating and missed rice seedlings based on deep learning of the present invention.

[0065] Figure 2 This is a structural diagram of the improved Faster-RCNN network model of the present invention.

[0066] Figure 3This is a structural diagram of the backbone sub-network residual network ResNet50 of the present invention.

[0067] Figure 4 This is a structural diagram of the backbone subnetwork FPN feature pyramid of the present invention.

[0068] Figure 5 This is a rice seedling classification prediction result diagram and a missed seedling position prediction diagram in the present invention.

[0069] Figure 6 This is a graph showing the four evaluation indicators of the improved Faster-RCNN model in the present invention.

[0070] Table 1 is a statistical table of the rice seedling identification test results. DETAILED DESCRIPTION

[0071] The following is combined with Figure 1-5 The implementation of the inventive method is described below:

[0072] The present invention provides a method and system for identifying drifting and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model, comprising the following steps:

[0073] S1. This embodiment uses a drone to obtain image data of rice seedlings in a field.

[0074] S11 selected rice planting fields in Qiqihar Tailai Farm and Heilongjiang Beidahuang Group Qixing Farm to collect rice data information. The system used drones to take low-altitude photos of the rice seedlings at a distance of 1.5 to 2 meters. The image acquisition system included a DJI Mavic 3 drone, a 90-degree adjustable camera, a computer, a removable hard drive, etc. The drone's flight speed was 1m / s, the shooting angle was -90 degrees, the photo interval was 2s, and the front-to-back overlap rate was 60%. The system photographed rice seedlings 5 ​​to 7 days after transplanting with a rice transplanter, and collected images of rice seedlings in different shapes, different soil environments, and different lighting conditions, increasing the diversity of the rice seedling dataset and avoiding overfitting in model training.

[0075] S2. This embodiment pre-processes and labels rice seedlings in the field to obtain pre-processed images.

[0076] The step S2 specifically includes:

[0077] Step S21: Data preprocessing: Import the initial rice seedling dataset, discard images that are blurred, show fields, have no rice examples, or have complex rice leaves that cannot be separated; adjust the brightness of the reflective area of ​​the rice seedling images that reflect on the water surface, and use median filtering to smooth the rice seedling images with complex soil environments; perform flipping, mirroring, and other operations on the rice seedling images with fewer categories in the dataset to increase the richness of the dataset, and adjust the images to the optimal size for model training through cropping, scaling, and other operations.

[0078] Step S22: Data labeling: The professional labeling data software labelImg is used for data labeling. According to the national standard "Technical Specifications for Rice Transplanters" and the photographed rice dataset, the rice seedlings are classified into qualified seedlings, floating seedlings, and seedlings with less than 4 leaves. The individual rice seedlings are marked in a rectangular frame from the upper left to the lower right.

[0079] Among them: the standards for various types of rice seedlings are as follows:

[0080] Floating seedlings: After planting, the seedlings have their roots not buried in the soil but floating on the water surface.

[0081] Leaf age: the number of leaves visible on the main stem of rice;

[0082] Missing planting: There are no holes for rice seedlings after planting;

[0083] Qualified seedlings: Seedlings that are not floating, broken, or cut in the planting hole are qualified.

[0084] S3. Select the original Faster-RCNN deep learning model and optimize it according to the characteristics of rice seedlings.

[0085] The deep learning model used in this example is the two-stage object recognition network model, Faster-RCNN, which evolved from R-CNN (Regions with CNN features) and Fast R-CNN. The improved Faster-RCNN model consists of four components: a backbone feature extraction network (Res Net50), a feature pyramid (FPN), a region proposal network (RPN), a region of interest pooling (RoIAlign), and object classification and regression, forming an end-to-end network architecture.

[0086] The step S3 specifically includes:

[0087] Step S31: Import the pre-processed image into the backbone feature extraction network to extract image features and generate a multi-scale fusion information graph. The backbone feature extraction network of this embodiment uses the residual network ResNet50 as the feature information extraction subnetwork; the specific image preprocessing operation in this example is: due to the limitations of the deep learning network model and computer memory, the input image size is usually small, such as a rectangular image in the form of M×N such as 224×224. In this example, the image taken by the drone has a high clarity. First, Adobe Photoshop is used to split and compress the original image to obtain a pre-processed image with a preset value of 500 on one side, and the other preset pixel value is cut to obtain the corresponding ratio N. After the image is processed, smaller objects in the original image can be displayed more clearly, and the network model is maximized during the training process.

[0088] Step S32: During the feature extraction process of rice seedlings by the residual network ResNet50, deeper feature information will be lost as the number of network layers increases. Therefore, the FPN feature pyramid is used to perform multi-scale fusion of the feature information of the rice seedling image. By constructing feature pyramids at multiple scales to extract rice seedling features at different scales, ResNet50 can accurately extract spatial information when extracting features of rice seedlings.

[0089] Step S33: The feature information graph obtained by the feature extraction sub-network ResNet50 is imported into the FPN feature pyramid for feature fusion. The feature pyramid is a network structure with two reverse branches connected horizontally, one side is a bottom-up forward line, and the other is a top-down line. After multiple upsampling and fusion in the last step, prediction is performed using the generated features, in which each layer is performed independently. Figure 3 As shown in the figure, the residual network ResNet50 is divided into 5 stages, namely c1, c2, c3, c4, and c5, where c1 is a 7×7 convolution layer with regularization, excitation function, and maximum pooling; c2 is composed of three residual blocks, regularization, and activation functions; similarly, c3 is composed of four residual blocks, regularization, and activation functions; c4 is composed of three residual blocks, regularization, and activation functions; c5 is composed of six residual blocks, regularization, and activation functions. C1 to C5 together constitute the residual network ResNet50. Figure 4As shown in the figure, it is the network structure diagram of the FPN feature pyramid. The upsampling results in the top-down line are fused with the feature information graph of the same size obtained in the bottom-up line by using horizontal links. After fusion, each feature fusion result is convolved with a 3*3 convolution kernel to obtain the feature layer outputs P2, P3, P4, and P5 in sequence; among them, P6 is obtained by downsampling P5, among which P1 to P5 are used for the Fast-RCNN network structure part, and P6 is used for the prediction of RPN in the Faster-RCNN network model.

[0090] Step S34: The feature information image obtained by the feature information extraction network is transferred to the region proposal network. The region proposal network generates a series of detection boxes for each feature point on the feature information image to detect whether each box contains target information. The non-maximum suppression operation filters out candidate boxes with high repetition rates, ultimately generating approximately 2,000 regions of interest. An anchor box mechanism is introduced into the region proposal network. The entire feature information map is scanned in the form of a sliding window, traversing each pixel in the image and establishing nine anchor boxes of different proportions as the initial candidate boxes of the model; the nine anchor boxes have three shapes with aspect ratios of (1:1, 1:2, and 2:1). When the sliding window slides to a feature point, nine anchor boxes are automatically generated. The region proposal network RPN has two parallel branches. One branch uses softmax classification to determine whether the prior anchor box belongs to the foreground or background. The second branch performs coordinate regression positioning on the anchor box, calculates the target anchor box to better match the actual annotation box, and obtains the final coordinate position parameters (x, y, w, h).

[0091] Step S35: In the region of interest pooling layer, the bilinear interpolation method in the RoIAlign layer is used to calculate the position of the prior box output from the region proposal network, replacing the quantization operation in the RoIPooling layer with the idea of ​​linear operation. The RoIAlign layer maps the feature image to the image, traverses all regions of interest, and obtains the floating-point coordinate position of the candidate box of the predicted point. The region of interest is evenly divided into 7×7 cells, and the coordinates of the four positions are calculated as four sampling points. The sampling points are then max-pooled to obtain more accurate position information.

[0092] Step S36: After the feature information image is pooled in the region of interest, the uniformly processed feature map is passed to the Softmax classifier and regressor. The resulting information image is classified and identified. The regressor is used to process the information image to obtain its precise location. The non-maximum suppression algorithm is used to eliminate possible duplicate target detection frames within the image range. The scores of the detection frames in the image are traversed to obtain the candidate frame with the highest detection target ratio, which is the highest scoring candidate frame. The position coordinates of the detection target are obtained through subsequent network calculations, and the center point of the target frame image is obtained, which is the location of the detection target.

[0093] S4. In this embodiment, the specific parameters for importing the pre-processed rice seedling dataset into the improved Faster-RCNN model for iterative training are:

[0094] Eight pre-training image datasets were randomly selected and combined with the training data to form a batch of datasets, which were imported into the deep learning network model for iterative training. The back-propagation algorithm was used for optimization training. The number of iterations of optimization training was E = 500, and the initial learning rate lr was set to 0.001. After every 100 iterations of training, the learning rate was reduced to one-tenth of the original value. The momentum parameter was set to 0.92, and the weight decay was 0.0005. The entire training iteration process was completed until the loss function tended to be stable, at which point the model was considered to have converged.

[0095] S5. In this embodiment, the rice seedling image to be tested is imported into the trained Faster-RCNN model to perform rice seedling detection. The prediction result is as follows: Figure 5 shown.

[0096] The rice seedlings to be tested are introduced into the improved Faster-RCNN network model;

[0097] The image to be tested is input into the backbone network residual network ResNet50 to extract feature information and generate a multi-scale fusion information graph;

[0098] Use the RPN network to generate candidate boxes, map the generated candidate boxes to the multi-scale fusion information graph, and obtain a new feature matrix;

[0099] Use RoI Align to interpolate the newly generated feature matrix to generate a 7×7 feature map;

[0100] The feature information map is classified by the fully connected layer, and the bounding box regression is used to obtain the precise anchor box of each target and accurately locate the target.

[0101] S6, such as Figure 5 The figure shows the prediction of the missed planting position of rice seedlings based on the improved Faster-RCNN model in step 6 of this embodiment;

[0102] The improved Faster-RCNN network model predicts the specific position of the rice seedlings and marks the candidate frame, and finds the center point and position coordinates of the candidate frame; the least squares method is used to perform straight line fitting on the rice seedlings, fitting the horizontal and vertical lines respectively, and the center intersection coordinates are obtained as the predicted point of the seedling; the predicted center point falling within the candidate frame is identified as the seedling position, and the center point without the candidate frame is recorded as the missed planting position of the rice seedling.

[0103] Figure 5 The rice seedling classification prediction result diagram and the missed rice seedling position prediction diagram in the present invention are shown in FIG. Figure 5 middle:

[0104] (a) shows the rice seedling identification diagram;

[0105] (b) Identification diagram of rice seedlings with less than 4 leaves;

[0106] (c) represents the original image recognition of the missing position;

[0107] (d) shows the prediction map of the missing position.

[0108] from Figure 5 As can be seen in the figure, the improved Faster-RCNN model is capable of classifying and identifying rice seedlings in the field, clearly identifying the three types of rice seedlings. To identify missed planting locations, the method simulates the movement of a rice transplanter in the field to replant seedlings. The coordinates of the missed planting locations (xi, yi) are named based on the y-axis, which ranges from 0 to 1, and the x-axis, which ranges from 0 to 1.

[0109] It can be seen from the provided drawings that the present invention is suitable for the identification of floating and missed rice seedlings, reflecting the practicality of the algorithm of the present invention. The method of the present invention has verified the technical effects claimed by the present invention through simulation experiments and practical applications: it greatly improves the time-consuming and labor-intensive problems of traditional manual seedling replacement, and improves the detection accuracy of rice seedlings by designing a deep learning network model, which facilitates the rapid promotion and application of intelligent identification technology for floating and missed rice seedlings.

[0110] S7. The classification and recognition results of rice seedlings based on the improved Faster-RCNN model in this embodiment are as follows: Figure 6 As shown;

[0111] To ensure comprehensive, accurate, and multi-angle analysis of the improved Faster-RCNN model, this paper uses multi-angle evaluation indicators to analyze the test results, namely precision P (Precision), recall R (Recall), F1 score (F1-Score), and mean average precision mAP (mean average precision).

[0112] Figure 6 The four index curves of the rice seedling classification and identification results in the present invention are shown in FIG. Figure 6 middle:

[0113] (a) shows the recognition accuracy of the improved Faster-RCNN model;

[0114] (b) represents the recognition recall rate of the improved Faster-RCNN model;

[0115] (c) represents the F1 value of the improved Faster-RCNN model recognition index;

[0116] (d) shows the improved Faster-RCNN model recognition index mAP@0.5.

[0117] from Figure 6 The performance of the feature parameters was analyzed based on four evaluation indicators: recognition accuracy, recall rate, F1 score, and mAP@0.5. The improved Faster-RCNN model achieved an accuracy of 89.2%, a recall rate of 89.1%, an F1 score of 93.8%, and an overall mAP@0.5 evaluation indicator of 93.8%. These data indicate that the present invention has a high overall accuracy for rice seedling recognition and can provide technical support for field machine transplanting and seedling replacement programs.

[0118] S8. The improved Faster-RCNN model of this embodiment classifies and identifies the rice seedling images to obtain predicted values ​​and actual values ​​of qualified rice seedlings, floating seedlings, and missed seedlings among the rice seedlings;

[0119] To detect the actual values ​​of the rice missed planting rate and floating seedling rate in the field, four working fields were selected and divided into nine blocks on average. Five of them were selected as test samples using the five-point sampling method, and each block contained approximately 200 rice seedling holes. The actual values ​​obtained in the field were compared with the improved Faster-RCNN model in the present invention to classify and identify rice seedling images, and the predicted values ​​of qualified seedlings, floating seedlings, and missed planting among the rice seedlings were obtained for comparative analysis.

[0120] Table 1 shows that the predicted values ​​for normal, floating, and missed rice seedlings are 95.03%, 1.52%, and 1.42%, respectively. The industry standards for floating and missed rice seedling rates in NY / T989-2020, "Quality of Rice Planting Machinery Operations," are ≤3% for floating and ≤5% for missed rice seedlings. Calculations based on a sample of 20 plots and 3,725 rice seedlings revealed an actual qualified rate of 96.37%, a floating rate of 1.24%, and a missed rate of 1.19%. Comparison of these results shows that the relative error between the predicted and actual values ​​for rice seedlings is small, meeting the accuracy requirements for field seedling replacement by rice transplanters and enabling successful classification and identification of rice seedlings.

[0121] Table 1

[0122]

[0123] The algorithm (method) proposed in the present invention is the underlying technical core of the present invention, and various products can be derived based on the algorithm.

[0124] Based on the algorithm (method) proposed in the present invention, a system for identifying floating and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model is developed using a programming language. The system has program modules corresponding to the steps of the above technical solution, and executes the steps of the above-mentioned method for identifying floating and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model during operation.

[0125] The developed system (software) computer program is stored on a computer-readable storage medium. The computer program is configured to, when called by a processor, implement the steps of the aforementioned method for identifying drifted and missed rice seedlings during machine-transplanted rice seedlings based on the improved Faster-RCNN model. This materializes the present invention on a carrier, becoming a computer program product.

[0126] A device for identifying rice seedlings that have been left or missed in transplantation includes a processor and a memory for storing data. When the processor executes the stored program, it implements a method for identifying rice seedlings that have been left or missed in transplantation using a machine based on an improved Faster-RCNN model. Specifically, the present invention provides a computer device including a processor and a memory for storing data, including a central processing unit (CPU + GPU), a system bus, a power adapter, a display, and other necessary computing and storage devices. When the processor executes the stored program, it implements the method for identifying rice seedlings that have been left or missed in transplantation.

[0127] A device for identifying floating and missed rice seedlings is used as a terminal product applied by the present invention.

[0128] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The computer programs (also referred to as programs, software, software applications, or code) of the present invention include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0130] It should be understood that the various forms of the above-described processes may be used, and steps may be reordered, added, or deleted. For example, the steps described in this application may be performed in parallel, sequentially, or in a different order. As long as the desired results of the technical solutions disclosed in this application can be achieved, they are all within the scope of protection of the present invention.

Claims

1. A method for identifying drifting and missed rice seedlings in machine-transplanted rice seedlings based on an improved Faster-RCNN model, characterized in that: The method comprises: UAVs acquire image data of rice seedlings in the field; Preprocessing and labeling the rice seedling images in the field to obtain preprocessed images; The original Faster-RCNN deep learning model was selected and optimized according to the characteristics of rice seedlings to obtain an improved Faster-RCNN model. The preprocessed rice seedling image dataset was imported into the improved Faster-RCNN model for iterative training; The rice seedling images to be tested are imported into the trained Faster-RCNN model to predict rice seedlings; The least square method is used to fit the predicted rice seedling image data to find the location of the missed rice seedlings. Use drones to obtain images of rice seedlings, including: The drone was set to the appropriate cruise and fixed-speed photography mode, and the plot of land where the drone would be operating was selected on the drone's remote controller for data collection. The drone's flight parameters were as follows: a shooting distance of 1.5 to 2 meters between the drone and the rice seedlings, and aerial photography using DJI GS Pro mode. The drone's flight speed was 1 m / s, the shooting angle was -90°, the shooting interval was 2 seconds, and the front-to-back overlap ratio was 60%. The data set collected from the field rice seedlings is preprocessed and labeled, and the data set preprocessing includes: Data collection of rice seedlings from the field was sorted, and unrecognizable blurry images and images of land that were taken improperly were removed. To address the issue of water surface reflection, the brightness of the reflective area is adjusted; rice seedlings with turbidity and complex soil environment are processed using median filtering to adjust the smoothness of the image; Perform reasonable scaling operations on the images, adjust the pixels, and unify the dataset size to facilitate subsequent model training; The rice seedling images with fewer categories in the dataset are flipped and mirrored to increase the richness of the dataset; The sorted data set is divided into training data set and test data set in a ratio of 8:2; The cleaned rice seedling dataset was annotated according to the proposed standard. The proposed standard is as follows: using a rectangular frame as the reference, recording the coordinates of the upper left corner and lower right corner of the rice seedling to completely enclose the rice seedling, and classifying and annotating qualified seedlings, floating seedlings, and seedlings with less than 4 leaves; Improve the construction of the Faster-RCNN network model, including: According to the characteristics of single rice seedling identification in the field, the two-stage network model Faster-RCNN in the deep learning network was selected; Replace the backbone network VGG16 in the Faster-RCNN network with the residual network Res Net50; Replace the region of interest pooling (RoI Pooling) layer in the Faster-RCNN network with the RoI Align layer; The optimized training of the improved Faster-RCNN network model specifically includes: The preprocessed rice seedling dataset is imported into the improved Faster-RCNN network model and data classification is performed; Design and adjust the training parameters of the improved Faster-RCNN network model, including the number of iterations and learning rate model parameters; The improved Faster-RCNN network model was pre-trained using the ImageNet dataset to obtain pre-trained weights. The pre-trained weights were then combined with the pre-processed rice seedling training dataset to train separately to improve the RPN network layer parameters in the Faster-RCNN network model. The trained RPN network parameters are fixed, the object proposal boxes generated from the RPN network are used to train the Fast-RCNN network parameters, and the fully connected layers in the Fast-RCNN network are fine-tuned. The RPN network and the Fast-RCNN network share the front-end network parameters to form a unified network; The improved Faster-RCNN model is optimized and trained using preprocessed image data, specifically: Import the pre-trained image dataset into the improved Faster-RCNN model. In each iterative training, a batch of images is randomly imported as the dataset for feature information extraction. Backpropagation and batch normalization are used for optimization training. The iterative training process is performed i times, and the update mode is set to "step". The prediction of rice seedlings to be tested includes: The rice seedlings to be tested are introduced into the improved Faster-RCNN network model; The image to be tested is input into the backbone network residual network ResNet50 to extract feature information, and the FPN feature pyramid is used to perform feature fusion to generate a multi-scale feature fusion information graph; Use the RPN network to generate candidate boxes, map the generated candidate boxes to the multi-scale feature fusion information graph, and obtain a new feature matrix; Use RoI Align to interpolate the newly generated feature matrix to generate a 7×7 feature map; The feature information map is classified by the fully connected layer, and the precise position coordinates of each target are obtained by bounding box regression, and the target is accurately located; The method of predicting the position of missed rice seedling planting specifically includes: The improved Faster-RCNN network model predicts the specific location of rice seedlings and marks the candidate box, and finds the center point and position coordinates of the candidate box; The least square method is used to fit the rice seedlings in a straight line, fitting the horizontal and vertical lines respectively, and the central intersection coordinates are obtained as the predicted points of the seedlings; The predicted center point that falls within the candidate frame is identified as the rice seedling position, and the center point without a candidate frame is recorded as the rice seedling missed planting position.

2. The method for identifying floating and missed rice seedlings according to claim 1, wherein: The drone in question is a DJI Mawic 3T equipped with a DJI RC-N1 remote controller and comes with four original batteries.

3. A system for identifying drifting and missed rice seedlings during machine transplanting based on an improved Faster-RCNN model, characterized by: The system has a program module corresponding to the steps of any one of claims 1-2, and executes the steps of the above-mentioned method for identifying drifted and missed rice seedlings in machine-transplanted rice seedlings based on the improved Faster-RCNN model when running.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the method for identifying drifted and missed machine-transplanted rice seedlings based on the improved Faster-RCNN model according to any one of claims 1 to 2 when called by a processor.

5. A device for identifying floating and missed rice seedlings, comprising a processor and a memory for storing data, characterized in that: When the processor executes the stored program, the steps of the method for identifying drifting and missed rice seedlings in machine-transplanted rice seedlings based on the improved Faster-RCNN model according to any one of claims 1 to 2 are implemented.

Images