Pepper plug seeding quality detection method and storage medium

Abstract

The invention relates to a pepper hole tray sowing quality detection method and a storage medium, the method combines single lattice training and whole tray detection for quality detection, and the method comprises the following steps: collecting images of pepper seeds after hole tray sowing, extracting single lattice images, and making a training set and a test set; constructing a deep learning target detection network, introducing a relative pixel loss function, training the deep learning target detection network by using the training set data, and judging a training iteration result; loading an optimal weight file of training iteration into a target detection algorithm, inputting a whole-tray image of the pepper seeds after hole tray sowing, carrying out whole-tray detection, and setting an image size adjustment step length, observation detection time and a size adjustment parameter when the precision is better; the single-seed rate, the reseeding rate and the miss-seeding rate of the pepper seeds sown in each tray are obtained by combining the specification of the to-be-detected pepper hole tray; and calculating the average single seed rate, the average reseeding rate and the average miss-seeding rate. The invention further provides a storage medium used for executing the method.

Description

Technical Field

[0001] The present invention relates to a sowing quality detection technology, and in particular to a pepper plug tray sowing quality detection method and storage medium based on deep learning. Background Art

[0002] Pepper seeds are small and flat, and are prone to missed and repeated sowing during automated sowing, which affects the quality of seedlings and their subsequent growth. Existing sowing quality detection mainly relies on manual observation and sampling calculation. During the operation, both eyes need to pay attention to the sowing situation at all times, which leads to problems such as long-term work intensity, low efficiency, large errors and lack of real-time feedback.

[0003] In recent years, with the development of computer vision and deep learning technology, target detection methods based on image recognition have gradually become a research hotspot. Deep learning algorithms use convolutional neural networks to predict seeding quality results, can calculate the number and location information of detection targets, and have the characteristics of high precision and high efficiency. There are two main deep learning target detection algorithms for hole tray seeding quality detection: "whole tray training + whole tray detection" and "single grid training + single grid detection + result splicing". Whole tray training requires more labeled data, which will increase training costs, while single grid training can use less labeled data and improve training efficiency; but whole tray detection is more efficient than single grid detection, and its post-processing is more convenient. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a pepper plug tray sowing quality detection method and a storage medium in view of the above-mentioned defects of the prior art.

[0005] In order to achieve the above object, the present invention provides a pepper plug tray sowing quality detection method, wherein the pepper plug tray sowing quality detection is performed by combining single grid training and whole tray detection, and comprises the following steps:

[0006] S100, collecting images of pepper seed plug trays after sowing and extracting single-grid images, performing data enhancement on the single-grid images, marking the outlines of pepper seeds in the single grids, and preparing training sets and test sets;

[0007] S200, constructing a deep learning target detection network, introducing a relative pixel loss function into the deep learning target detection network, training the deep learning target detection network using training set data, and determining a training iteration result;

[0008] S300, loading the optimal weight file of the training iteration into the target detection algorithm of the deep learning target detection network, inputting the whole plate image after the pepper seed plug plate is sown, performing whole plate detection, and setting the image size adjustment step length to obtain the size adjustment parameter when the detection time and accuracy are better;

[0009] S400, setting the weight and size adjustment parameters of the target detection algorithm according to the size adjustment parameters obtained in step S300, inputting the online collected image into the constructed target detection algorithm to perform sowing quality detection, outputting the number of seeds in the whole plate and the coordinate center point, and calculating the single seed rate, reseeding rate and missed seeding rate of the pepper seeds sown in each plate in combination with the specifications of the pepper plug tray to be detected; and

[0010] S500, based on the calculation result of step S400, calculate the average single seed rate, average reseeding rate and average missed seeding rate of all pepper seeds under the current sowing task to detect the current overall sowing quality.

[0011] The above-mentioned pepper plug tray sowing quality detection method further includes the steps of:

[0012] S600, calculating the consistency variation coefficient of each row under the current sowing task as a judgment basis for subsequent quality control.

[0013] In the above-mentioned pepper plug tray sowing quality detection method, step S400 further comprises:

[0014] S401, converting the pepper plug tray sowing quality detection model into ONNX open neural network exchange format, and combining with TensorRT acceleration library to achieve efficient reasoning;

[0015] S402, dividing and calculating the detected pepper seed coordinate center points, and constructing a Matrix seed distribution matrix corresponding to the hole tray grid positions; and

[0016] S403, if the number of seeds detected in the same area is 0, it is determined to be missed seeding; if the number of seeds detected in the same area is 1, it is determined to be a single seed; if the number of seeds detected in the same area is greater than 1, it is determined to be reseeding.

[0017] In the above-mentioned pepper plug tray sowing quality detection method, step S100 further comprises:

[0018] S101, removing invalid areas from the selected hole tray sowing image according to the main distribution area of ​​the hole tray to obtain a processed image; and

[0019] S102, using the OpenCV library, setting a template matching script, extracting single-frame images from the processed images, and fusing the randomly cropped enriched data to obtain multiple single-frame images.

[0020] In the above-mentioned pepper plug tray sowing quality detection method, step S100 further comprises:

[0021] S103, performing data enhancement on the extracted single-frame image by means of brightness transformation, random flipping, contrast enhancement and random scaling, with the amplification ratio being 25% to 35%;

[0022] S104, dividing the enhanced overall single-frame image data into a training set and a test set according to 8:2; and

[0023] S105, using Labelme software to label the pepper seeds in each single-frame image in the training set to generate a .txt file.

[0024] In the above-mentioned pepper plug tray sowing quality detection method, the .txt file includes category, the center coordinates of the horizontal coordinate of the marked object, the center coordinates of the vertical coordinate of the marked object, width and height information.

[0025] In the above-mentioned pepper plug tray sowing quality detection method, step S200 further comprises:

[0026] S201, the average pixel size of pepper seeds in the dataset is 30 pixels × 30 pixels. A targeted relative pixel loss function is introduced to improve the recognition accuracy and the offset tolerance of the IoU measurement of small target positions;

[0027] S202, perform training, observe the result curve, and determine whether the target detection algorithm has been trained and converged, and whether the accuracy meets the requirements;

[0028] S203, loading the trained weight file into the target detection algorithm, and testing it on the test set to observe the actual detection effect; and

[0029] S204: If the test result does not meet the actual requirements, increase the number of training times, or re-produce and expand the data set, and re-train and test according to steps S201 to S203 until a weight file that meets the required accuracy is trained.

[0030] In the above-mentioned pepper plug tray sowing quality detection method, step S201 further comprises:

[0031] S2011, set the center point of the target real box annotated in the dataset to X o , Y o , the relative width and height is w o 、h o ; The center point of the network prediction box is X z , Y z , the relative width and height is w z 、h z ;

[0032] S2012. The Euclidean distance between the center point of the network prediction box and the center point of the target real box is: Dc 2 =(X z -X o ) 2 +(Y z -Y o ) 2 ;

[0033] S2013, set the width and height measurements to D l 2 =(w z -w o ) 2 +(h z -h o ) 2 To avoid the prediction box overlap IoU being 0;

[0034] S2014, set the maximum boundary dimension uniform divisor to D f 2 =max(w z ,w o ) 2 +max(h z ,h o ) 2 , to eliminate the influence of dimension;

[0035] S2015, then the relative pixel loss Pxi.Loss is:

[0036]

[0037] S2016. Use the exponential function of e to normalize the relative pixel loss, and the result is a similarity metric with a range of (0, 1]:

[0038]

[0039] Where λ is a constant normalization coefficient.

[0040] In the above-mentioned pepper plug tray sowing quality detection method, step S300 further comprises:

[0041] S301, loading the optimal weight file into the target detection algorithm;

[0042] S302, based on the number of downsampling times k of the target detection algorithm, determine that the step length of the whole disk size change is 2 k pixels, adjust the original width w and height h in proportion, and adjust the ratio R a =2 k f ceil (max(w,h) / 2 k )+n) / max(w,h), where fceil (*) is a rounding up function, n is a variable coefficient for size adjustment, and its value is an integer; and

[0043] S303, changing the size adjustment variable coefficient n, observing the algorithm reasoning time and accuracy, and selecting the size adjustment variable coefficient value with the highest detection accuracy under the required detection speed.

[0044] In order to better achieve the above-mentioned purpose, the present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program is configured to execute the above-mentioned pepper plug tray sowing quality detection method when running.

[0045] The technical effects of the present invention are:

[0046] The present invention can automatically detect and calculate the sowing quality of pepper seeds in plug trays after sowing. In order to facilitate integrated application and improve detection efficiency and accuracy, a deep learning framework is used to implement pepper seed target detection tasks. By improving the deep learning algorithm, it is adapted to the detection of small-target pepper seeds with irregular shapes, which is convenient for expansion and application. In addition, the algorithm is trained using single-grid images, which effectively reduces the number of samples and significantly improves the efficiency of the implementation of the work in the pre-stage; the whole tray image is used for result prediction to ensure the algorithm reasoning speed. The present invention provides a quick and feasible technical solution for the detection of various characteristics of pepper plug tray sowing quality, improves the task operation effect, and can take into account both accuracy and real-time performance; the relative pixel loss function Pxi.Loss for small-target pepper seed recognition is introduced, which can effectively reduce the sensitivity of the prediction frame positioning offset. Compared with the single Euclidean distance metric or the original bounding box loss and other judgment methods, the introduction of the maximum boundary dimension unified divisor term for balancing can maintain the advantage of higher weight for small target boxes under the same distance offset; its deep learning network is portable, and is converted into the ONNX general format and run on the host computer, combined with the TensorRT acceleration library to achieve efficient reasoning; the computer-side host computer software is written in Python, which is convenient for cross-platform porting and provides an end-to-end computer application.

[0047] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A working principle diagram of an embodiment of the present invention;

[0049] Figure 2A-2C It is a schematic diagram of a process of acquiring a single cell image of a hole tray according to an embodiment of the present invention;

[0050] Figure 3A-3BA schematic diagram of a single cell image of a hole tray and a marked file extracted according to an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a relative pixel loss function according to an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of the training results of the deep learning target detection algorithm for the single-grid image data of the plug tray after sowing peppers according to one embodiment of the present invention;

[0053] Figure 6 This is a schematic diagram of the operation of adjusting the image size of the entire disk during the entire disk detection according to an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of transplanting a target detection model according to an embodiment of the present invention to a host computer software;

[0055] Figure 8 A-8B is a schematic diagram of the detection and counting results of an embodiment of the present invention;

[0056] Fig. 9 This is a schematic diagram of calculating the coefficient of variation of consistency for each row;

[0057] Figures 10A-10B This is a rendering of the effect of detecting the sowing quality of pepper seed plug trays in a whole tray image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The structural principle and working principle of the present invention are described in detail below in conjunction with the accompanying drawings:

[0059] See also Figure 1 , Figure 1 The working principle diagram of an embodiment of the present invention. The pepper plug tray sowing quality detection method of the present invention combines single grid training and whole tray detection to perform pepper plug tray sowing quality detection, and includes the following steps:

[0060] Step S100, collecting images of pepper seed plug trays after sowing and extracting single-grid images, performing data enhancement on the single-grid images by random flipping, color conversion, random scaling and translation, marking the outline of the pepper seeds in the single grid, and preparing training sets and test sets;

[0061] Step S200: construct a deep learning target detection network, introduce a relative pixel loss function into the deep learning target detection network, train the deep learning target detection network using training set data, and determine the training iteration result;

[0062] Step S300, loading the optimal weight file of the training iteration into the target detection algorithm of the deep learning target detection network, inputting the whole plate image after the pepper seed plug plate is sown, performing whole plate detection, and setting the image size adjustment step length to obtain the size adjustment parameter when the detection time and accuracy are better;

[0063] Step S400, perform pepper plug tray sowing quality detection by taking pictures, use the pepper plug tray sowing quality detection model to identify the sowing quality of the whole pepper tray to be detected, set the target detection algorithm weight and size adjustment parameter according to the size adjustment parameter obtained in step S300, input the online collected image into the constructed target detection algorithm for sowing quality detection, output the number of seeds in the whole tray and the coordinate center point, and calculate the single seed rate, reseeding rate and missed seeding rate of the pepper seeds sown in each tray in combination with the specifications of the pepper plug tray to be detected; that is, set the target detection algorithm weight and size adjustment parameter (algorithm hyperparameter, that is, in machine learning) according to the above results. In deep learning and deep learning algorithms, parameters that need to be set in advance before starting to train the model. These parameters are not learned through the model training process itself, but are set based on experience, domain knowledge, experiments, or some heuristic methods. They control the overall behavior and performance of the algorithm, such as the speed of learning, the complexity of the model, etc.), load it into the host computer software, design a computer application, identify the sowing quality of the entire tray of peppers, output the number of seeds in the entire tray, the coordinate center point, and combine the specifications of the hole tray used (such as a 72-hole standard seedling tray: 12 rows × 6 columns), and then calculate the single-seed rate, re-seeding rate, and missed sowing rate of the pepper seeds sown in each tray; and

[0064] Step S500, based on the calculation result of step S400, calculate the average single seed rate, average reseeding rate and average missed seeding rate of all pepper seeds under the current sowing task to detect the current overall sowing quality.

[0065] This embodiment also includes the steps of:

[0066] Step S600: Calculate the consistency variation coefficient of each row under the current sowing task as a basis for subsequent quality control.

[0067] Wherein, step S100 further includes:

[0068] Step S101, removing invalid areas from the selected hole tray sowing image according to the main distribution area of ​​the hole tray to obtain a processed image;

[0069] Step S102: using the OpenCV library, setting a template matching script, extracting single-frame images from the processed images, and fusing the randomly cropped enriched data to obtain multiple single-frame images;

[0070] Step S103, performing data enhancement on the extracted single-frame image by means of brightness transformation, random flipping, contrast enhancement and random scaling, with the amplification ratio being 25% to 35%;

[0071] Step S104, dividing the enhanced overall single-frame image data into a training set and a test set according to 8:2; and

[0072] Step S105: Labelme software is used to label the pepper seeds in each single-frame image in the training set to generate a .txt file. The .txt file includes the category, the center coordinates of the horizontal coordinate of the labeled object, the center coordinates of the vertical coordinate of the labeled object, and the width and height information.

[0073] Step S200 further includes:

[0074] Step S201, the average pixel size of pepper seeds in the data set is 30 pixels × 30 pixels, and a targeted relative pixel loss function is introduced to improve the recognition accuracy and the offset tolerance of the IoU measurement of small target positions;

[0075] Step S202: Perform training, observe the result curve, and determine whether the target detection algorithm has converged and whether the accuracy meets the requirements;

[0076] Step S203, loading the trained weight file into the target detection algorithm, and testing it on the test set to observe the actual detection effect; and

[0077] Step S204: If the detection result does not meet the actual requirements, increase the number of training times, or re-produce and expand the data set, and re-train and test according to steps S201 to S203 until a weight file that meets the required accuracy is trained.

[0078] Step S201 further includes:

[0079] Step S2011: Set the center point of the target real box annotated by the dataset to X o , Y o , the relative width and height is w o 、h o ; The center point of the network prediction box is X z , Y z , the relative width and height is w z 、h z ;

[0080] Step S2012: The Euclidean distance between the center point of the network prediction frame and the center point of the target real frame is: D c 2 =(X z -X o ) 2+(Y z -Y o ) 2 ;

[0081] Step S2013: Set the width and height dimensions to D l 2 =(w z -w o ) 2 +(h z -h o ) 2 To avoid the prediction box overlap IoU being 0;

[0082] Step S2014: Set the maximum boundary dimension uniform divisor item to D f 2 =max(w z ,w o ) 2 +max(h z ,h o ) 2 , to eliminate the influence of dimension;

[0083] Step S2015, the relative pixel loss Pxi.Loss is:

[0084]

[0085] Step S2016: normalize the relative pixel loss using the exponential function of e, and the result is a similarity metric with a range of (0, 1]:

[0086]

[0087] Where λ is a constant normalization coefficient.

[0088] Step S300 further includes:

[0089] Step S301, loading the optimal weight file into the target detection algorithm;

[0090] Step S302: Based on the number of downsampling times k of the target detection algorithm, determine the step length of the whole disk size change to be 2 k pixels, adjust the original width w and height h in proportion, and adjust the ratio R a =2 k f ceil (max(w,h) / 2 k )+n) / max(w,h), where f ceil (*) is a rounding up function, n is a variable coefficient for size adjustment, and its value is an integer; and

[0091] Step S303: change the size adjustment variable coefficient n, observe the algorithm reasoning time and accuracy, and select the size adjustment variable coefficient value with the highest detection accuracy under the required detection speed.

[0092] Step S400 further includes:

[0093] Step S401, converting the pepper plug tray sowing quality detection model into ONNX open neural network exchange format, and combining with TensorRT acceleration library to achieve efficient reasoning;

[0094] Step S402, dividing and calculating the detected pepper seed coordinate center points, and constructing a Matrix seed distribution matrix corresponding to the hole tray grid positions; and

[0095] Step S403: if the number of seeds detected in the same area is 0, it is determined to be missed seeding; if the number of seeds detected in the same area is 1, it is determined to be a single seed; if the number of seeds detected in the same area is greater than 1, it is determined to be reseeding.

[0096] The pepper plug tray sowing quality detection method of the present invention is described in detail with specific embodiments below:

[0097] Example 1

[0098] See also Figure 1 The present invention adopts the method of "single grid training + whole plate detection" and introduces a new relative pixel distance Pix.Loss into the training loss function, which specifically includes:

[0099] Capturing images of pepper seed plug trays after sowing, specifically, collecting images of pepper seed plug trays after sowing, wherein the images are from images of pepper seed plug trays after sowing captured on the spot, according to the technical conditions of greenhouse vegetable plug tray precision seeder NY / T1823-2009, a total of 456 images were collected, of which images of 72-hole and 128-hole standard seedling trays accounted for 50% each, and all images were taken in actual seedling operation scenes;

[0100] Single-frame image extraction is performed on the collected whole-disk post-broadcast image, including:

[0101] The selected images of the hole tray after sowing are used to remove invalid areas according to the main distribution area of ​​the hole tray;

[0102] Using the OpenCV library, a template matching script was written to extract single-frame images from the processed images, and random cropping operations were integrated to improve data richness, thereby obtaining 8064 single-frame images.

[0103] The acquired single-frame images are data enhanced, the outlines of pepper seeds in the single frame are annotated, and training sets and test sets are prepared, specifically including: data enhancement of the extracted single-frame images by brightness transformation, random flipping, contrast enhancement and random scaling, with an amplification ratio of 25% to 35%, and a total of 10,682 single-frame images after amplification; the enhanced overall single-frame image data is divided into training sets and test sets according to 8:2; Labelme software is used to annotate the pepper seeds in each single-frame image in the training set, and a .txt file is generated, which contains the following information: category, horizontal coordinate center of the annotated object, vertical coordinate center coordinate of the annotated object, width, height, etc.

[0104] The relative pixel loss function is introduced into the deep learning target detection network. The constructed deep learning target detection network is trained using the training set data to judge the training iteration results: the pixel size of pepper seeds in the data set is 30 pixels × 30 pixels on average. Compared with large and medium-sized targets (corn, rice seeds, etc.), the position of the prediction box of the small target of pepper seeds is slightly deviated, and the IoU (intersection over union, i.e., the ratio between the intersection of the label box and the prediction box and the union of the two) will drop significantly. A targeted relative pixel loss function is introduced to improve the recognition accuracy and improve the problem of low tolerance of IoU measurement of small target position offset, specifically:

[0105] Note that the center point of the target real box annotated by the dataset is X o , Y o , the relative width and height is w o 、h o ; The center point of the network prediction box is X z , Y z , the relative width and height is w z 、h z ; Then the Euclidean distance between the two center points is: D c 2 =(X z -X o ) 2 +(Y z -Y o ) 2 ; The width and height are measured as D l 2 =(w z -w o ) 2 +(h z -h o ) 2 , to avoid the problem of no overlap of prediction boxes and IoU being 0; the maximum boundary dimension unified divisor is: D f 2 =max(w z ,w o ) 2+max(h z ,h o ) 2 , to eliminate the impact of dimension; the relative pixel loss Pxi.Loss is:

[0106]

[0107] The relative pixel loss is normalized using the exponential function of e, and the result is a similarity metric with a range of (0, 1]: Where λ is a constant normalization coefficient.

[0108] Then, training is performed, and the result curve is observed to determine whether the target detection algorithm has converged and whether the accuracy meets the requirements; the trained weight file is loaded into the target detection algorithm, and tested on the test set to observe the actual detection effect; if the detection result does not meet the actual requirements, consider increasing the number of training times, or remaking and expanding the data set, and retraining and predicting according to the above steps until a weight file that meets the required accuracy is trained; the optimal weight file of the training iteration is loaded into the target detection algorithm, the whole disk image is input, and the whole disk detection is performed, and the image size adjustment step is set at the same time, and the size adjustment parameters when the detection time and accuracy are better are observed, including:

[0109] Load the optimal weight file into the target detection algorithm; considering that there is a large size change between the single-frame image and the whole disk image, the algorithm's downsampling times k is used as the benchmark, and the step size change of the whole disk is determined to be 2 k pixels, and then adjust the original width w and height h in proportion, where the adjustment ratio R a =2 k f ceil (max(w,h) / 2 k )+n) / max(w,h), where f ceil (*) is a rounding up function, n is a variable coefficient for size adjustment, which is determined through experiments and takes an integer value; by changing the variable coefficient n for size adjustment (such as taking a value of [-10, 20]), observing the algorithm reasoning time and accuracy, the variable coefficient value for size adjustment with the highest detection accuracy at the required detection speed is selected.

[0110] According to the above results, the weights and size adjustment parameters (algorithm hyperparameters) of the target detection algorithm are set. Combined with the acquisition device, the online acquired images are input into the constructed target detection algorithm for sowing quality detection, and the number of seeds in the whole plate and the coordinate center point are output. Combined with the specifications of the plug tray used (such as 72-hole standard seedling tray: 12 rows × 6 columns), the single-seed rate, re-seeding rate and missed seeding rate of the pepper seeds sown in each tray are calculated. Specifically, the computer host software is set to detect the quality of pepper plug tray sowing by taking pictures; the implemented pepper plug tray sowing quality detection model is converted into ONNX (OpenNeuralNetwork The computer-side host software is written in Python, which is convenient for cross-platform porting. After the code is successfully compiled, it is exported to the .exe executable file format. After installing the software, the detection can be completed by equipping a mobile camera to collect images or uploading collected images. After clicking "Start Task", the software completes the model preloading. The middle picture is the detection picture. Click the "Detect" button, and the detection results of each picture are displayed on the right, and the left is the current detection results of all pictures. Average value; divide the detected pepper seed coordinate center points into regions and calculate them. If the hole tray used is a 72-hole standard seedling tray (12 rows × 6 columns), construct the Matrix[12,6] seed distribution matrix (corresponding to 72 hole tray grid positions): If the number of seeds detected in the same area is 0, it is judged as missed seeding; if the number of seeds detected in the same area is 1, it is judged as a single seed (qualified); if the number of seeds detected in the same area is greater than 1, it is judged as reseeding, then it is judged as reseeding. Click the "Export Data" button and the test results will be exported to the local device in the form of an Excel data table.

[0111] Based on the above results, the sowing quality (average single seed rate, average reseeding rate and average missed sowing rate) of all pepper seeds sown under the current sowing task is calculated to reflect the current overall sowing effect. The consistency coefficient of variation of each row is calculated as a basis for subsequent quality control.

[0112] The present invention can automatically detect and calculate the sowing quality of pepper seeds in plug trays after sowing. In order to facilitate integrated application and improve detection efficiency and accuracy, a deep learning framework is used to implement pepper seed target detection tasks, and a relative pixel loss function is introduced to enhance the detection accuracy of small targets. In addition, single-grid images are used for algorithm training, which effectively reduces the number of samples and the amount of annotation tasks, and significantly improves the efficiency of the implementation of the work in the pre-stage; the whole tray image is used for result prediction to ensure the algorithm reasoning speed. A quick and feasible method for detecting various characteristics of pepper plug tray sowing quality is provided, which improves the task operation effect. At the same time, the algorithm can be transplanted through the host computer software, which can be cross-platform, providing technical support for the integration of automatic detection of sowing quality.

[0113] Example 2

[0114] See also Figure 2A-2C , Figure 2A-2C This is a schematic diagram of a process for acquiring a single cell image of a hole tray according to an embodiment of the present invention. Figure 2A-2C The figure shows the cropping of the collected whole plate image and the extraction of single-grid images. Since the whole plate image collected by the camera has a high consistency and the image generally contains background areas other than the hole plate, based on the OpenCV library, the fixed area is cropped by the average relative coordinate position of the "upper left corner" and the "lower right corner" to remove the invalid background area. Then, based on template matching, as shown in formula 2-1, the pixel-level sliding single-grid template is matched to the whole plate image. According to formula (2-1), the normalized correlation coefficient R between the matching area of ​​the whole plate image S and the template image M is obtained. During the experiment, the matching confidence T is set to 0.14. If R ≥ T, it is considered that there is a match at this position, and the matching success rate is nearly 84%.

[0115]

[0116] In the formula, x p ,y p ——matching coordinate starting point, px;

[0117] U M 、V M ——Template size, px;

[0118] E(M)——mean of template image;

[0119] ——With (x p ,y p ) is the starting point and its size is U M ×V M The mean of the region.

[0120] See also Figure 3A-3B , Figure 3A-3B This is a schematic diagram of a single cell image of a hole tray and a marked file extracted according to an embodiment of the present invention. Figure 3A The figure shows the extracted single grid image of the hole plate and the corresponding Labelme annotation file. The annotation information of each image (Class, x0, y0, L w , L h ) represent: target category number, center point relative horizontal coordinate, center point relative vertical coordinate, relative width and relative height, such as Figure 3B shown.

[0121] See also Figure 4 , Figure 4This is a schematic diagram of the relative pixel loss function of an embodiment of the present invention. Since the error caused by pixel loss is sensitive to the intersection-over-union ratio (IoU) (the ratio of the intersection of the predicted frame and the real frame pixels to the union of the two) when the small target is predicted to be offset, a targeted relative pixel loss function is introduced to improve the recognition accuracy and improve the problem of low tolerance of IoU in measuring the position offset of small targets. Specifically:

[0122] Note the center point X of the target real box annotated by the dataset o , Y o , relative width and height w o 、h o ; Network prediction box center point X z , Y z , relative width and height w z 、h z ; Set the Euclidean distance between the two center points to:

[0123] D c 2 =(X z -X o ) 2 +(Y z -Y o ) 2 ; (2-2)

[0124] Set the width and height dimensions to avoid the problem of no overlap of prediction boxes and IoU of 0:

[0125] D l 2 =(w z -w o ) 2 +(h z -h o ) 2 ; (2-3)

[0126] Set the maximum bounding dimension uniform divisor term to eliminate the effect of dimension:

[0127] D f 2 =max(w z ,w o ) 2 +max(h z ,h o ) 2 ; (2-4)

[0128] Then the relative pixel loss Pxi.Loss is:

[0129]

[0130] The relative pixel loss is normalized using the exponential function of e, and the result is a similarity metric with a range of (0, 1]:

[0131]

[0132] Where λ is the constant normalization coefficient;

[0133] The target detection network loss function after setting the relative pixel loss is (taking the YOLOv8nano network as an example):

[0134] Loss = l box +l cls +l pxi ; (2-7)

[0135] In the formula, l box is the positioning loss; l cls is the confidence loss; l pxi is the relative pixel loss.

[0136] See also Figure 5 , Figure 5 This is a schematic diagram of the training results of the deep learning target detection algorithm of one embodiment of the present invention on the single-grid image data of the plug tray after sowing peppers. The figure shows the training results of the improved YOLO v8nano algorithm, which is implemented based on the Pytorch framework. YOLO v8nano has the advantages of light weight and fast detection speed, and its accuracy can meet the needs of actual operation scenarios. The horizontal coordinates of all images are the number of training rounds, and 100 rounds of iterations are performed. The vertical coordinates are the values ​​of each indicator, among which it can be seen that the positioning loss, confidence loss and relative pixel loss converge with the increase in the number of training rounds, and the smaller the effect, the better; the precision rate is the proportion of correctly predicted samples to the positive samples found; the recall rate is the proportion of correctly predicted positive samples to all positive samples; the average precision rate mAP50 and mAP50-95 are the prediction accuracies when the intersection-over-union ratio IoU is greater than 50% and between 50% and 95%; the two weight files best.pt and last.pt of the algorithm network are obtained through training.

[0137] See also Figure 6 , Figure 6 The whole disk image size adjustment operation diagram of the whole disk detection in one embodiment of the present invention is shown in the figure. The whole disk image is first adjusted by formula (2-8) to determine the adjustment step size Step is 2 k (k is the number of downsampling times of the algorithm, YOLOv8nano is 5), that is, the adjustment step size is 32 pixels. In order to make each pixel of the entire disk image valid, it is necessary to ensure that it is divisible by 32. Within a suitable range, the appropriate size method helps to improve the detection accuracy, such as making the variable coefficient n∈[-10,20], and then conducting experiments to comprehensively consider the detection time and accuracy to determine the final feasible variable coefficient n:

[0138]

[0139] See also Figure 7 , Figure 7 This is a schematic diagram of transplanting the target detection model of an embodiment of the present invention to the host computer software, converting the implemented pepper plug tray sowing quality detection model into the ONNX (Open Neural Network Exchange) open neural network exchange format, and combining with the TensorRT acceleration library to achieve efficient reasoning; wherein the computer-side host computer software is written in Python language, which is convenient for cross-platform transplantation, and the code is exported to the .exe executable file format after successful compilation; after installing the software, the detection can be completed by equipping a movable camera to collect images or uploading collected pictures, and after clicking "Start Task", the software completes model preloading, the middle picture is the detection screen, click the "Detect" button, the detection result of each picture is displayed on the right, and the average value of all current detection results is on the left.

[0140] See also Figure 8 A-8C, Figure 8 A-8C is a schematic diagram of the detection counting results of an embodiment of the present invention. Figure 8 A is the image to be detected, Figure 8 B is the solution result of obtaining the single seed rate, reseeding rate and missed seeding rate after pepper sowing. The solution process is:

[0141] Prediction result Location(xi,yi)

[0142] Seeding quantity matrix Matrix[X,Y]

[0143] Position coordinates Xcal, Ycal

[0144] Whole plate size:

[0145] Sx = ImageSize[0]

[0146] Sy=ImageSize[1]

[0147] *Normalized coordinate position:

[0148] Xcal=(xi / Sx)*6 #column position

[0149] Ycal=(yi / Sy)*12 #row position

[0150] *Calculation result matrix:

[0151] Matrix[X,Y]←[Xcal,Ycal]#Relative coordinate position

[0152] CountX,Y[N]+=1#The number of cells corresponding to

[0153] That is, the normalized coordinate method is used to normalize the center coordinates of the seed tray after sowing. Taking the 72-hole standard seedling tray (12 rows × 6 columns) as an example, the width of the entire tray is obtained by the OpenCV library as S x = ImageSize[0], height is S y = ImageSize[1], perform normalized matching, then the normalized position coordinates of each seed detection: column position is X cal =(x i / S x )*6, row position is Y cal＝ (y i / S y )*12, in order to record the relative coordinate position and form the sowing quantity result matrix Matrix[X cal ,Y cal ], the number of seeds in the corresponding cell Count X,Y [N], such as Figure 8 As shown in Figure B, the single-grain rate (single-grain hole number / total number of holes), reseeding rate (reseeding hole number / total number of holes) and missed seeding rate (missed seeding hole number / total number of holes) are further obtained, and then the average single-grain rate, average reseeding rate and average missed seeding rate of the total seeding hole tray are calculated.

[0154] See also Fig. 9 , Fig. 9 This is a schematic diagram of calculating the consistency coefficient of variation of each row. The calculation of the consistency coefficient of variation of each row is as follows:

[0155] First, the number of seeds sown in each cell is obtained through algorithm detection, and the number of seeds N corresponding to each row is obtained by summing them up based on the row. y , where y represents the row number and x represents the column number. Taking a 72-hole (12×6) standard seedling tray as an example, the maximum number of rows in each tray is 12, and the number of columns is 6, that is, Y=12, X=6, then the number of seeds in the first row N1 can be calculated by the following formula N1=N (1,1) +N (2,1) +N (3,1) +N (4,1) +N (5,1) +N (6,1) ;

[0156]

[0157] Calculate the average number of seeds per row

[0158] Calculate the standard deviation S:

[0159] Calculate the coefficient of variation CV:

[0160] By calculating the coefficient of variation of consistency of each row of the plug tray after sowing, the uniformity of sowing can be evaluated. The smaller the coefficient of variation, the smaller the difference between sowing rows, the higher the sowing quality, which helps to ensure that crops can obtain relatively consistent growth conditions during the growth process and improve the overall seedling quality. For drum sowing production lines, this coefficient can be used to evaluate the sowing performance of the equipment. If the coefficient of variation is large, it may mean that there are problems with the sowing mechanism of the equipment, such as unstable operation of the seed meter and errors in the transmission components, thus providing a basis for the debugging and improvement of the equipment. Understanding the consistency of each row of the plug tray after sowing helps optimize the production process. For example, parameters such as sowing speed and seed type can be reasonably adjusted according to the size of the coefficient of variation to improve production efficiency and quality.

[0161] See also Figures 10A-10B , Figures 10A-10B This is a rendering of an embodiment of the present invention for detecting the sowing quality of pepper seed plug trays in a whole tray image. Fig. 10A The test results of pepper after sowing in 72-hole standard seedling tray; Fig. 10B This is the detection effect of pepper after sowing in a 128-hole standard seedling tray. It can be seen that the method based on deep learning combined with single-grid training and whole-plate detection not only improves the efficiency in the training stage, but also has very high detection accuracy in the detection stage. Combined with pixel loss judgment, it has high-quality detection effects.

[0162] The invention is suitable for online detection of pepper sowing quality for a seedling sowing production line, and completes image acquisition after pepper seed plug tray sowing; in view of the structural characteristics of the plug tray itself (the whole plate image can be regarded as being composed of single-grid images spliced ​​together), single-grid image extraction is performed on the collected source image; then data enhancement is performed on the single-grid image of the plug tray, and after the pepper seeds in the single-grid image are marked, a single-grid image data set of the plug tray is formed, and a weight file is obtained by training; the weight file is loaded into a deep learning target detection algorithm to obtain a plug tray sowing quality detection network; during detection, the collected whole plate image is directly detected, and by adjusting the whole plate image size variation coefficient, the whole plate image adjustment size parameter when the target detection algorithm has the best detection effect is selected; after the adjustment size parameter is set in the target detection algorithm, the online detection of the collected whole plate image is completed, so as to obtain data such as the single seed rate, reseeding rate, missed seeding rate and seed center point of the pepper seeds sown in the plug tray. It solves the problems of low efficiency of manual calculation of sowing quality in existing seedling and sowing production lines when sowing irregular seeds such as peppers, low training and detection efficiency or low accuracy of existing hole tray detection methods, and failure to achieve a balance between the two. Through "single-grid training + whole tray detection", the training efficiency of the deep learning target detection algorithm is improved, and the algorithm detection speed and detection accuracy are improved.

[0163] Correspondingly, based on the same inventive concept, the present invention also provides a storage medium, the storage medium stores a computer program, and the computer program is configured to execute the above-mentioned pepper plug tray sowing quality detection method when running. Through the description of the above implementation mode, it is easy for those skilled in the art to understand that the example implementation mode described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation mode of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (for example, a CD-ROM, a U disk, a mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal or a network device, etc.) to perform the method according to the implementation mode of the present invention.

[0164] In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present invention described in the above "Exemplary Method" section of this specification. Optionally, in this embodiment, the above storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0165] The program product for implementing the above method according to an embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a host computer or a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0166] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0167] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0168] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0169] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0170] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting the quality of pepper plug tray sowing, characterized in that: Combining single-grid training with whole-plate testing to test pepper plug tray sowing quality includes the following steps: S100, collecting images of pepper seed plug trays after sowing and extracting single-grid images, performing data enhancement on the single-grid images, marking the outlines of pepper seeds in the single grids, and preparing training sets and test sets; S200, constructing a deep learning target detection network, introducing a relative pixel loss function into the deep learning target detection network, training the deep learning target detection network using training set data, and determining a training iteration result; S300, loading the optimal weight file of the training iteration into the target detection algorithm of the deep learning target detection network, inputting the whole plate image after the pepper seed plug plate is sown, performing whole plate detection, and setting the image size adjustment step length to obtain the size adjustment parameter when the detection time and accuracy are better; S400, according to the size adjustment parameters obtained in step S300, the weight and size adjustment parameters of the target detection algorithm are set, the online collected image is input into the constructed target detection algorithm to perform sowing quality detection, the number of seeds in the whole plate and the coordinate center point are output, and the single seed rate, reseeding rate and missed seeding rate of the pepper seeds sown in each plate are calculated in combination with the specifications of the pepper plug tray to be detected; as well as S500, based on the calculation result of step S400, calculate the average single seed rate, average reseeding rate and average missed seeding rate of all pepper seeds under the current sowing task to detect the current overall sowing quality.

2. The pepper plug tray sowing quality detection method according to claim 1, characterized in that: Also includes the steps: S600, calculating the consistency variation coefficient of each row under the current sowing task as a judgment basis for subsequent quality control.

3. The pepper plug tray sowing quality detection method according to claim 1, characterized in that: Step S400 further includes: S401, converting the pepper plug tray sowing quality detection model into ONNX open neural network exchange format, and combining with TensorRT acceleration library to achieve efficient reasoning; S402, dividing and calculating the detected pepper seed coordinate center points, and constructing a Matrix seed distribution matrix corresponding to the hole tray grid positions; and S403, if the number of seeds detected in the same area is 0, it is determined to be missed seeding; if the number of seeds detected in the same area is 1, it is determined to be a single seed; if the number of seeds detected in the same area is greater than 1, it is determined to be reseeding.

4. The pepper plug tray sowing quality detection method according to claim 1, characterized in that: Step S100 further includes: S101, removing invalid areas from the selected hole tray sowing image according to the main distribution area of ​​the hole tray to obtain a processed image; and S102, using the OpenCV library, setting a template matching script, extracting single-frame images from the processed images, and fusing the randomly cropped enriched data to obtain multiple single-frame images.

5. The pepper plug tray sowing quality detection method according to claim 4, characterized in that: Step S100 further includes: S103, performing data enhancement on the extracted single-frame image by means of brightness transformation, random flipping, contrast enhancement and random scaling, with the amplification ratio being 25% to 35%; S104, dividing the enhanced overall single-frame image data into a training set and a test set according to 8:2; and S105, using Labelme software to label the pepper seeds in each single-frame image in the training set to generate a .txt file.

6. The pepper plug tray sowing quality detection method according to claim 5, characterized in that: The .txt file includes information about the category, the center coordinates of the horizontal coordinate of the labeled object, the center coordinates of the vertical coordinate of the labeled object, and the width and height.

7. The pepper plug tray sowing quality detection method according to claim 1, characterized in that: Step S200 further includes: S201, the average pixel size of pepper seeds in the dataset is 30 pixels × 30 pixels. A targeted relative pixel loss function is introduced to improve the recognition accuracy and the offset tolerance of the IoU measurement of small target positions; S202, perform training, observe the result curve, and determine whether the target detection algorithm has been trained and converged, and whether the accuracy meets the requirements; S203, loading the trained weight file into the target detection algorithm, and testing it on the test set to observe the actual detection effect; and S204: If the test result does not meet the actual requirements, increase the number of training times, or re-produce and expand the data set, and re-train and test according to steps S201 to S203 until a weight file that meets the required accuracy is trained.

8. The pepper plug tray sowing quality detection method according to claim 7, characterized in that: Step S201 further includes: S2011, set the center point of the target real box annotated in the dataset to X o , Y o , the relative width and height is w o 、h o ; The center point of the network prediction box is X z , Y z , the relative width and height is w z 、h z ; S2012. The Euclidean distance between the center point of the network prediction box and the center point of the target real box is: D c 2 =(X z -X o ) 2 +(Y z -Y o ) 2 ; S2013, set the width and height measurements to D l 2 =(w z -w o ) 2 +(h z -h o ) 2 To avoid the prediction box overlap IoU being 0; S2014, set the maximum boundary dimension uniform divisor term to D f 2 =max(w z ,w o ) 2 +max(h z ,h o ) 2 , to eliminate the influence of dimension; S2015, then the relative pixel loss Pxi.Loss is: S2016. Use the exponential function of e to normalize the relative pixel loss, and the result is a similarity metric with a value range of (0, 1]: Where λ is a constant normalization coefficient.

9. The pepper plug tray sowing quality detection method according to claim 1, characterized in that: Step S300 further includes: S301, loading the optimal weight file into the target detection algorithm; S302, based on the number of downsampling times k of the target detection algorithm, determine that the step length of the whole disk size change is 2 k pixels, adjust the original width w and height h in proportion, and adjust the ratio R a =2 k f ceil (max(w,h) / 2 k )+n) / max(w,h), where f ceil (*) is a rounding up function, n is a variable coefficient for size adjustment, and its value is an integer; and S303, changing the size adjustment variable coefficient n, observing the algorithm reasoning time and accuracy, and selecting the size adjustment variable coefficient value with the highest detection accuracy under the required detection speed.

10. A storage medium, characterized in that: The storage medium stores a computer program, and the computer program is configured to execute the pepper plug tray sowing quality detection method according to any one of claims 1 to 9 when running.

Images