Tobacco seedling growing period growth state discrimination and emergence rate detection method and related device

Abstract

The invention discloses a tobacco seedling growing period growth state judgment and emergence rate detection method and a related device, and relates to the technical field of tobacco seedling growing. The tobacco seedling growing period growth state discrimination and emergence rate detection method comprises the following steps: collecting seedling tray images of each seedling age in a seedling growing process; preprocessing and segmenting the seedling tray image to generate a seedling hole lattice map; and performing seedling category and positioning frame labeling on the seedling hole sub-graph by utilizing a Labelimg tool, randomly segmenting a data set, performing model training on labeled seedling hole sub-graph data based on a deep learning algorithm, and training a tobacco seedling raising period target judgment and emergence rate detection model. The invention solves the problems that the current population growth detection of crops in the seedling stage is mostly limited to a fixed seedling age state, although the prior art can realize the growth state grading of tobacco seedlings in different cells, but is limited to the fixed seedling age, and cannot be expanded to the growth state monitoring of the whole growth period of the tobacco seedlings, or is limited to the growth detection of crops in the single adult stage; and the method can not be expanded to multi-plant group growth state monitoring.

Description

Technical Field

[0001] The invention relates to the technical field of tobacco seedling cultivation, and in particular to a method for determining the growth state of tobacco seedlings during the cultivation period and detecting the emergence rate, and related devices. Background Art

[0002] In the process of tobacco production, "three-tenths of the seedlings are harvested, and half of the good seedlings are harvested", which fully demonstrates that the cultivation of seedlings is an inevitable condition for the success of tobacco production. Due to the large differences in the sowing and transplanting periods of tobacco seedlings at different altitudes, and the different needs of tobacco seedlings for the ecological environment and nutrients at each growth stage, there is a lack of consistent reference for whether the growth rate of tobacco seedlings in the corresponding stage is fast or slow. Relying on manual supervision and control intervention is a common phenomenon in tobacco seedling cultivation operations in most tobacco-producing areas. During the seedling raising period, a large number of frequent operations such as warming, supplementary lighting, ventilation, etc. still need to be completed manually, which requires high technical management of seedling management personnel, and it is urgent to establish a growth and development model for the seedling raising period.

[0003] The growth and development status of tobacco seedlings includes four seedling stages: the emergence stage, the small cross stage, the large cross stage, and the seedling stage. In recent years, with the advancement of big data and artificial intelligence algorithms, image acquisition and algorithm recognition capabilities have been greatly improved. Images of the tobacco seedling emergence process can be collected through Internet of Things devices (temperature and humidity sensors in the greenhouse, surveillance cameras in the greenhouse, etc.), and a deep learning visual algorithm can be used to preliminarily construct a tobacco seedling development status discrimination model, analyze the growth and development status of the seedling stage, realize real-time reporting of seedling conditions, and calculate the tobacco seedling emergence rate. It is an effective way to reduce tobacco seedling transplanting losses and improve the quality of tobacco seedlings.

[0004] Regarding the problem of identifying the growth and development status of tobacco seedlings, the current related research includes references [1] He Yan. Research on the identification algorithm of tobacco seedlings in plug trays based on machine vision [D]. Southwest University, 2019. In order to solve the problem of automatic thinning during transplanting, this paper implements the segmentation of tobacco seedlings in plug trays based on the AdaBoost algorithm, and then uses multi-layer perceptron and AlexNet to identify and classify tobacco seedlings in plug trays. It mainly focuses on the identification of empty holes and multiple strong tobacco seedlings in the plug tray. The accuracy rates of the two algorithms are 96.75% and 97.58% respectively, but they have very high requirements on the selected features. Different features have a greater impact on the experimental results. In addition, they only focus on whether the plug tray is empty and whether there are multiple plants, but do not pay attention to the seedling stage of the tobacco seedlings, and do not consider the different morphologies of tobacco seedlings under different growth and development states.

[0005] References [2] He Yan, Zhu Shiping, Xia Zhilin, et al. Research on automatic thinning algorithm of plug tray tobacco seedlings based on machine vision [J]. Journal of Yunnan Agricultural University (Natural Science Edition), 2019. In order to solve the problem of fast automatic thinning, this paper proposes to divide the plug tray into cells based on the k-means clustering algorithm, calculate the circularity, aspect ratio and rectangularity of the tobacco seedlings, use the area and perimeter of the seedlings, and set a suitable threshold at different growth stages to identify single plants, multiple plants and holes. The correct recognition rate of the number of tobacco seedlings reached more than 97.04%, the position of the holes reached 100%, and the average accuracy of the thinning position and strong seedling recognition reached 94.76% and 89.58% respectively, achieving the purpose of automatic thinning. Although the differences of tobacco seedlings under different growth and development states are taken into account, it is highly dependent on the setting of the threshold and does not have generalization performance. In addition, the recognition rate of the recognition algorithm based on shape features is low and not robust. And it is impossible to automatically determine the seedling stage to which the tobacco seedlings belong.

[0006] References [3] Wu Cong, Guo Zhiqiang, Yang Jie. Research on classification algorithm of seedlings in trays based on residual network with improved attention mechanism [J]. Laser & Optoelectronics Progress, 2022, 59(22): 10. DOI: 10.3788 / LOP202259.2210002. This paper proposes to introduce channel attention mechanism and spatial attention mechanism into residual network ResNet, and uses seven kinds of seedling images of trays, such as eggplant seedlings, cauliflower seedlings, pepper seedlings, eggplant seedlings, celery seedlings, watermelon seedlings, and cucumber seedlings, as data samples. The seedling dataset is obtained through preprocessing operations such as image correction, image segmentation, and data expansion. Then, the randomly erased data samples are input into the improved residual network for training, realizing multi-classification of seedlings in trays. This paper proposes a binary polynomial method to complete the correction of tray images, and uses AdaBoost classifier to obtain tray segmentation. The scheme of data set preprocessing is relatively complicated. The proposed ResNet algorithm based on CBAM attention mechanism solves the classification and recognition of seedlings of various different crops. However, this scheme only selects images with seedlings of roughly the same size when acquiring seedling tray images, and removes seedling trays with too small seedlings and seedling trays with large seedlings and obstructions. Therefore, this method cannot take into account the morphological changes of seedlings during their growth and development. This model can only be used to determine the type of seedlings in a very short period of time during the growth of seedlings, and has no practical application value.

[0007] In summary, existing research focuses on the positioning of empty holes in seedling trays and the detection of multiple plants in one hole in the scenario of tobacco seedling transplanting and thinning. The seedling growth process of crops is a complex and dynamic physiological and biochemical metabolic process. At present, the group growth detection of seedling crops is mostly limited to a fixed seedling age state. For example, in the literature [4] Li Tao, Ren Ling, Hu Bin, et al. Improved YOLOv5s and transfer learning for the grading detection of tomato seedlings in seedling trays [J]. Transactions of the Chinese Society of Agricultural Engineering, 2023, 39(23): 174-184, an improved YOLOv5s model is proposed to classify 20-day-old tomato seedlings in seedling trays into four levels: empty holes, weak seedlings, strong seedlings and diseased seedlings. In the literature [5] Zhang Xiuhua, Jing Maokai, Yuan Yongwei, et al. Tomato seedling classification based on improved YOLOv3-Tiny [J]. Transactions of the Chinese Society of Agricultural Engineering, 2022, 38(01): 221-229. It is proposed to classify 10-day-old tomato seedling trays into empty holes, weak seedlings and strong seedlings based on the YOLOv3-Tiny algorithm. Although the growth status of tobacco seedlings in different holes can be graded, it is limited to a fixed seedling age and cannot be extended to growth monitoring of tobacco seedlings throughout their growth period, or limited to growth detection of single adult crops. It lacks or ignores continuous growth monitoring of groups of crops at various seedling stages. The machine vision-based morphological detection and monitoring technology of single crops is relatively mature, but there is little research on the extraction of key growth parameters of seedling crop groups and the growth monitoring of crop groups throughout their growth period. No effective solution has been proposed so far. Summary of the invention

[0008] Purpose of the invention: To provide a method and related device for determining the growth status of tobacco seedlings and detecting the emergence rate during the seedling raising period, so as to solve the above-mentioned problems existing in the prior art.

[0009] Technical solution: A method for distinguishing the growth status and detecting the emergence rate of tobacco seedlings during the seedling raising period, comprising: S1, collecting images of seedling trays at various seedling age states during the seedling raising process; S2, preprocessing the seedling tray images and segmenting them to generate a number of seedling hole grid maps; S3, using the Labelimg tool to label the seedling categories and positioning frames of a number of seedling hole sub-maps and randomly divide the data set; S4, training a model for the labeled seedling hole sub-map data set based on a deep learning algorithm to train a tobacco seedling target discrimination and emergence rate detection model during the seedling raising period.

[0010] Preferably, collecting images of seedling trays at various seedling ages during the seedling raising process includes:

[0011] Through on-site monitoring of the development process of the seedling trays at the target base, original seedling tray images with preset pixels are regularly collected to obtain several original images of the seedling trays during the entire growth period from seedling emergence to seedling maturity;

[0012] According to the definition of seedling stage, the original images are classified into four seedling stage categories; among them, the original images include: seedling stage, small cross stage, large cross stage and seedling stage.

[0013] Preferably, the seedling tray image is preprocessed and segmented to generate a number of seedling hole grid images, including:

[0014] The original image is skew-calibrated and cropped by using seedling tray edge detection and affine transformation algorithm to obtain a standard square seedling tray. Each seedling tray has 10 rows and 20 columns, with a total of 200 seedling holes, which are cut into 200 hole grid images according to the resolution. The resolution of the hole grid image is 80*80.

[0015] The seedling trays were processed using an image cutting algorithm, and each cropped seedling tray image was cut into 200 seedling hole sub-images;

[0016] Based on the normalized super-green feature method and the maximum inter-class variance method, the green tobacco seedling leaf area and seedling tray border area of ​​the whole tray were extracted, and the linear mapping relationship between the tobacco seedling leaf area rate and the whole tray seedling progress was analyzed, and the whole tray seedling progress based on the leaf area rate was derived.

[0017] As a preferred method, the Labelimg tool is used to label the seedling categories and positioning frames of several seedling hole sub-graphs and randomly split the data set, including:

[0018] The Labelimg tool was used to label the seedling stage category and position of the seedlings in each small hole sub-graph, and an XML format labeling file of each sub-graph was obtained; the labeling categories included: seedling stage, small cross stage, and large cross stage;

[0019] The labeled seedling hole sub-maps are randomly divided into training sets and validation sets according to the preset ratio.

[0020] As a preferred method, a model training is performed on the labeled seedling hole sub-map dataset based on a deep learning algorithm to train a tobacco seedling seedling period target discrimination and emergence rate detection model, including:

[0021] Based on the collected image data sets of the same batch of seedling raising cycle, the daily average leaf area rate of multiple seedling tray images was calculated, and the proportion of actual seedling raising days to the total actual seedling raising cycle days was taken as the seedling raising progress of the whole tray, and a linear mapping relationship between the leaf area rate average and the seedling raising progress of the whole tray was established.

[0022] For the tobacco leaf area extracted from a single hole after block division, the Gini coefficient was used to analyze the distribution of leaf area ratio to estimate the growth consistency of the whole tray;

[0023] Based on the YOLOv5s algorithm model, the block detection strategy, SimAM attention mechanism module and SPD-conv downsampling strategy were introduced to classify and locate the growth status of single-hole tobacco seedlings, so as to derive the growth status indicators of the whole tray of tobacco seedlings; among them, the growth status indicators of the whole tray of tobacco seedlings include: the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period.

[0024] As a preferred method, a block detection strategy is introduced based on the YOLOv5s algorithm model, including:

[0025] The entire seedling tray is divided into 200 single-hole sub-images by blocking, so that the proportion of tobacco seedlings in the input hole grid sub-image is adjusted to no less than 1 / 10. When the input resolution is scaled to 640*640, the minimum resolution of the tobacco seedling size in the seedling stage is about 64*64, which is within the detection range of the YOLOv5s model.

[0026] As a preferred method, based on the YOLOv5s algorithm model, the block detection strategy, SimAM attention mechanism module and SPD-conv downsampling strategy are introduced to classify and locate the growth status of single-hole tobacco seedlings, so as to derive the growth status indicators of the whole tray of tobacco seedlings; among which, the growth status indicators of the whole tray of tobacco seedlings include: the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period, and also include:

[0027] Determine the growth period of the entire tray and the proportion of tobacco seedlings in each period. When more than 50% of the individual plants in the holes enter the preset period, it is determined that the entire tray has entered that period.

[0028] As a preferred method, the growth period of the whole tray and the proportion of tobacco seedlings in each period are determined. When more than 50% of the single plants in the holes enter the preset period, the whole tray is considered to have entered the period, including:

[0029] The counting results of the 200 sub-images are accumulated to obtain the seedling data. When more than one target object is detected in the sub-image, the count is 1, otherwise it is 0, and the count is not repeated. Whether there is a seedling in each hole is output. If no seedling is detected in a single hole grid, it represents an empty hole;

[0030] The prediction results of 200 models were summarized and counted, and the number of seedlings in the seedling emergence stage, small cross stage, large cross stage and holes were represented by a, b, c and d respectively. The proportion of tobacco seedlings in each stage was represented by Count and summarize the emergence rate of the whole tray;

[0031] When the percentage of tobacco seedlings in the preset period exceeds 50%, the growth period of the entire tray is defined as that period.

[0032] As a preferred method, the growth period of the whole tray and the proportion of tobacco seedlings in each period are determined. When more than 50% of the single plants in the holes enter the preset period, it is determined that the whole tray has entered the period, and the following is also included:

[0033] Visualization of evaluation indicators of the model training process and visualization of test set output results.

[0034] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a device for determining the growth status of tobacco seedlings during the seedling raising period and detecting the emergence rate is provided.

[0035] The device for distinguishing the growth status of tobacco seedlings during the seedling raising period and detecting the emergence rate according to the present application includes: a seedling tray image acquisition module, which is used to collect seedling tray images at various seedling age states during the seedling raising process; a preprocessing module, which is used to preprocess the seedling tray images and segment them to generate a number of seedling hole grid maps; a model training module, which is used to perform model training on the labeled seedling hole sub-map data set based on a deep learning algorithm, and train a tobacco seedling seedling raising period target discrimination and emergence rate detection model.

[0036] Beneficial effect: In the embodiment of the present application, a method based on deep learning image recognition is adopted, by collecting images of seedling trays at various seedling ages during the seedling raising process; preprocessing the seedling tray images and segmenting them to generate a number of seedling hole grid maps; using the Labelimg tool to annotate the several seedling hole sub-maps and divide the data set; based on the deep learning algorithm, model training is performed on the annotated seedling hole sub-map data set to train a target discrimination and seedling emergence rate detection model for the tobacco seedling raising period, thereby achieving the purpose of training and generating a target discrimination and seedling emergence rate detection model for the tobacco seedling raising period, thereby realizing accurate and efficient identification and positioning of a large number of pre- The technical effect of defining category object instances further solves the problem that the current group growth detection of seedling crops is mostly limited to fixed seedling age status. Although the existing technology can realize the growth status classification of tobacco seedlings in different holes, it is limited to fixed seedling age and cannot be extended to growth monitoring of tobacco seedlings throughout the growth period, or is limited to the growth detection of single adult crops. It lacks or ignores the continuous growth monitoring of group crops at various seedling stages and ages. The single crop morphology detection and monitoring technology based on machine vision is relatively mature, while the extraction of key growth parameters of seedling crop groups and the growth monitoring of crop groups throughout the growth period are less studied technical issues. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is an example diagram of a cropped whole seedling tray according to the method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate of the seedlings according to the embodiment of the present application;

[0038] Figure 2 It is an analysis of leaf area ratio of the method for distinguishing the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to the preferred embodiment of the present application, analyzing the leaf area ratio (a) and the frame area ratio (b) of the seedling raising tray at different growth periods;

[0039] Figure 3This is an example of a seedling hole sub-diagram after annotation according to the method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate of the seedlings in the embodiment of the present application;

[0040] Figure 4 It is a flow chart of an algorithm for evaluating the growth status of tobacco seedling trays according to the method for determining the growth status of tobacco seedlings during the seedling raising period and detecting the emergence rate of the seedlings in the embodiment of the present application;

[0041] Figure 5 It is a flow chart of a method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to an embodiment of the present application;

[0042] Figure 6 It is a model structure diagram of the improved YOLOv5s according to the method for distinguishing the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate in the embodiment of the present application;

[0043] Figure 7 It is a visualization of the model output comparison results of the tobacco seedling growth state discrimination and emergence rate detection method according to the embodiment of the present application, and the comparison of the detection results before and after the improvement of YOLOv5s;

[0044] Figure 8 It is a comprehensive index diagram for judging the growth and development status of tobacco seedlings according to the seedling tray picture according to the method for judging the growth status of tobacco seedlings and detecting the emergence rate during the seedling raising period of the embodiment of the present application;

[0045] Fig. 9 It is a flow chart of another method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to an embodiment of the present application;

[0046] Fig.10 is a schematic diagram of the structure of a tobacco leaf baking state determination device according to an embodiment of the present application; and

[0047] Fig.11 It is a structural schematic diagram of an electronic device according to the method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate of the seedlings according to the embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0049] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] According to an embodiment of the present invention, a method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate is provided. Figure 4-6 As shown in FIG. 9 , the method includes the following steps S101 to S104:

[0051] Step S101, collecting images of seedling trays at different seedling ages during the seedling raising process;

[0052] By collecting a large number of seedling tray images, accurate image collection can be achieved, thus providing a data basis for subsequent analysis and processing.

[0053] According to an embodiment of the present invention, preferably, collecting images of seedling trays at various seedling ages during the seedling raising process includes:

[0054] Through on-site monitoring of the development process of the seedling trays at the target base, original seedling tray images with preset pixels are regularly collected to obtain several original images of the seedling trays during the entire growth period from seedling emergence to seedling maturity;

[0055] According to the definition of seedling stage, the original images are classified into four seedling stage categories; among them, the original images include: seedling stage, small cross stage, large cross stage and seedling stage;

[0056] For example, we started collecting images from the first day of seedling emergence, and regularly took RGB images of the same batch of seedling trays floating in the greenhouse every day. The image resolution was about 3000*4000 pixels. We continuously monitored the images of the same batch of seedlings at different stages of their development. From March 15 to May 8, we collected a large number of original images of multiple batches of complete seedling periods. A total of 256 original images of seedling trays from emergence to maturity were collected.

[0057] The original images are classified into four seedling stage categories according to the definition of the seedling stage. The classification of the seedling stage categories can be performed by software or professional technicians to classify the original images, which is not limited in this application.

[0058] Step S102, pre-processing the seedling tray image, and segmenting it to generate a plurality of seedling hole grid images;

[0059] By preprocessing the seedling tray image and generating several seedling hole sub-images, the accuracy of the image information can be ensured, and the image can be segmented into multiple required image styles, thereby providing guarantee for subsequent image processing.

[0060] According to an embodiment of the present invention, preferably, the original image is skew-calibrated and the seedling tray is cropped by using seedling tray edge detection and affine transformation algorithm to obtain a standard square seedling tray; wherein each seedling tray has 10 rows and 20 columns, a total of 200 seedling holes, which are cut into 200 hole grid images according to the resolution, and the hole grid image resolution is 80*80;

[0061] The seedling trays were processed using an image cutting algorithm, and each cropped seedling tray image was cut into 200 seedling hole sub-images;

[0062] Based on the normalized super-green feature method and the maximum inter-class variance method, the green tobacco seedling leaf area and seedling tray border area of ​​the whole tray were extracted, and the linear mapping relationship between the tobacco seedling leaf area rate and the whole tray seedling progress was analyzed, and the whole tray seedling progress based on the leaf area rate was derived.

[0063] like Figure 1 As shown, the original image data is preprocessed and the sub-image and leaf area are extracted by hole segmentation;

[0064] Specifically, since the floating seedling tray is manually collected by mobile phone aerial photography, it is inevitable that the seedling tray area in the image is skewed and the size of the seedling tray area is inconsistent due to the difficulty in controlling the angle and height. The seedling tray edge detection and affine transformation technology are used to skew the original image and crop the seedling tray. Among them, the affine transformation maps the point on the image to a point on the output image to achieve operations such as image translation, rotation, scaling, and shearing. Using the affine transformation method, the image resolution of the output seedling tray area after calibration is set to 1600*800, and the four points on the edge of the seedling tray area in the original image are mapped to the four edge coordinate points of the output image, which are [0,0], [0,1600], [800,0], [800,1600] respectively. The skewed seedling tray area in the original image is rotated into a horizontal rectangular image to obtain a standard square seedling tray. Each seedling tray has 10 rows and 20 columns, with a total of 200 seedling holes. Cut into 200 hole grid pictures according to the resolution, and the hole grid picture resolution is 80*80.

[0065] Since the growth rate of tobacco seedlings in each cell in the seedling tray is inconsistent and the number of tobacco seedlings in each cell ranges from 0 to 2, the model needs to be constructed based on the single cell image in the seedling tray. Since the seedling tray image after affine transformation is a regular grid, and each grid has the same size and shape, the resolution of the entire seedling tray image is 1600*800, with 20 cells per row and 10 cells per column. Using image cutting technology, each cropped entire seedling tray image is cut into 200 seedling cell sub-images, and the resolution of the cell sub-image is 80*80.

[0066] In order to comprehensively quantify the growth and development progress, the ExG and Otsu methods were used to extract the green tobacco seedling leaf area and seedling tray border area of ​​the whole seedling tray, which facilitated the statistical analysis of the leaf area rate of the whole seedling tray image samples at different seedling stages. Figure 2 shown.

[0067] Step S103, using the Labelimg tool to label the seedling categories and positioning frames of several seedling hole sub-graphs and randomly divide the data set;

[0068] By adopting the preset algorithm to label all the seedling hole sub-images and dividing the data set at the same time, a good image labeling effect can be achieved. At the same time, the images are divided into corresponding data sets according to the preset requirements, thereby providing accurate image data for the subsequent model training.

[0069] According to an embodiment of the present invention, preferably, the Labelimg tool is used to mark the seedling stage category and position of the seedlings in each small hole sub-graph, and an XML format marking file of each sub-graph is obtained; wherein the marking categories include: seedling stage, small cross stage, large cross stage;

[0070] The labeled seedling hole sub-maps are randomly divided into training sets and validation sets according to the preset ratio.

[0071] It can achieve accurate image annotation effects, and at the same time, it can also achieve the effect of distinguishing data sets for the annotated images, thereby facilitating reading and operation.

[0072] like Figure 3 As described, the Labelimg tool is used to annotate the seedling stage category and position of the seedlings in each small hole sub-graph, and an XML format annotation file is obtained for each sub-graph. The annotation categories include seedling stage (E), small cross stage (X), and large cross stage (D). When the number of tobacco seedling leaves is ≤3, it is marked as seedling stage E. When the number of tobacco seedling leaves = 4 and presents a cross shape, two of which are slightly larger than the other two, it is marked as small cross stage X. When the number of tobacco seedling leaves is ≥5, 4 are in a cross shape, two of which are significantly larger than the other two, and the fifth leaf appears, it is defined as large cross stage D. When the entire tobacco seedling cannot be fully displayed in the hole grid, and the coverage of the tobacco seedling leaves obviously exceeds the hole grid area, it is the seedling stage, and the seedling stage hole grid sub-graph is not marked.

[0073] In the seedling rate detection model, 67 original seedling tray images in the seedling period folder were used, and 13,212 small hole sub-images were obtained after preprocessing and cutting. After labelimg tool annotation, the invalid sub-images without seedlings and those blocked by surrounding leaves were removed, and a total of 9,549 sub-images with target box annotations were obtained. The labeled seedling holes;

[0074] The training set and validation set are randomly split in a ratio of 8:2.

[0075] Step S104, performing model training on the labeled seedling hole sub-image data set based on a deep learning algorithm, and training a tobacco seedling seedling raising period target discrimination and emergence rate detection model;

[0076] The seedling hole sub-maps to be labeled and related data set information are input into the deep learning model for model training, and the tobacco seedling target discrimination and emergence rate detection models are trained.

[0077] According to an embodiment of the present invention, preferably, a model training is performed on the labeled seedling hole sub-graph data set based on a deep learning algorithm to train a tobacco seedling seedling period target discrimination and emergence rate detection model, including:

[0078] Based on the collected image data sets of the same batch of seedling raising cycle, the daily average leaf area rate of multiple seedling tray images was calculated, and the proportion of actual seedling raising days to the total actual seedling raising cycle days was taken as the seedling raising progress of the whole tray, and a linear mapping relationship between the leaf area rate average and the seedling raising progress of the whole tray was established.

[0079] For the tobacco leaf area extracted from a single hole after block division, the Gini coefficient was used to analyze the distribution of leaf area ratio to estimate the growth consistency of the whole tray;

[0080] Based on the YOLOv5s algorithm model, the block detection strategy, SimAM attention mechanism module and SPD-conv downsampling strategy were introduced to classify and locate the growth status of single-hole tobacco seedlings, so as to derive the growth status indicators of the whole tray of tobacco seedlings; among them, the growth status indicators of the whole tray of tobacco seedlings include: the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period.

[0081] Specifically, the block detection strategy processes the image in blocks. The SimAM attention mechanism module and the SPD-conv downsampling strategy are both plug-and-play module components that are directly inserted into the original YOLOv5 model structure.

[0082] 1. Block detection strategy

[0083] The block detection strategy divides the image into multiple blocks and performs independent detection on each block to improve the recognition accuracy of the model in dense tobacco seedling scenes. This strategy usually requires the following steps in the application of YOLOv5s:

[0084] Image segmentation: The whole tray of tobacco seedlings image is divided into multiple equal-sized areas, each of which contains relatively few tobacco seedlings, reducing the occlusion and overlap problems of seedlings in dense scenes.

[0085] Single block detection: Each divided area is input into the YOLOv5s model for detection. This can reduce the detection complexity and improve the model's ability to distinguish individual tobacco seedlings.

[0086] Merge results: By recording the relative position coordinates of each block, the detection results in each block area are reassembled into the detection results in the entire plate image, so as to deduce the growth status of the entire plate of tobacco seedlings.

[0087] Advantages: The block detection strategy can reduce false detections and missed detections caused by dense tobacco seedlings, and improve the accuracy of the model in identifying the status of single-hole tobacco seedlings.

[0088] 2. SimAM Attention Mechanism

[0089] The SimAM (Simple Attention Module) attention mechanism can enhance the model's attention to key features. SimAM automatically adjusts the network's weights in different channels and spatial positions through the coupling of spatial and channel information, improving the effectiveness of feature extraction. The steps to combine SimAM with YOLOv5s are as follows:

[0090] Embedding location: In the feature extraction stage of YOLOv5s, the SimAM module is embedded in the backbone network or detection head to weight the multi-scale feature maps so that the model can give priority to the growth details of the tobacco seedlings.

[0091] Weight update: SimAM constructs an adaptive weight function, calculates the importance of each channel and spatial position, and adaptively weights the feature map, so that the model pays more attention to the key growth status characteristics of the seedlings; such as leaf size, color, shape, etc.

[0092] Output optimization: The addition of SimAM can suppress the impact of background interference information, while enhancing the ability to recognize tobacco seedling characteristics, further improving the accuracy of classification and positioning.

[0093] Advantages: The SimAM module enables the model to focus on the key features of tobacco seedlings, optimize the model's anti-interference ability in complex scenarios, and improve the accuracy of distinguishing different growth states.

[0094] 3. SPD-conv downsampling strategy

[0095] SPD-conv (Sub-pixel Downsampling Convolution) is a new downsampling method that efficiently reduces the dimension of images through sub-pixel convolution. SPD-conv is different from traditional pooling or stride convolution. It retains more local information through special pixel rearrangement and convolution operations. The implementation steps of SPD-conv are as follows:

[0096] Replace the downsampling layer: In the backbone network or feature extraction stage of YOLOv5s, SPD-conv is used to replace the traditional strided convolution or pooling layer to retain the subtle structural information in the tobacco seedling image.

[0097] Pixel rearrangement: SPD-conv achieves the dimensionality reduction effect of spatial information by rearranging some pixels of the image into different channels, while reducing information loss and enhancing the integrity of the model in feature extraction.

[0098] Feature enhancement: While performing dimensionality reduction, SPD-conv further processes feature information through convolution, enabling the model to retain more detail information during downsampling and improve classification and positioning accuracy.

[0099] Advantages: SPD-conv can effectively avoid the information loss problem caused by traditional downsampling, enhance the model's ability to retain tobacco seedling details, and make the positioning of tobacco seedlings in different growth states more accurate.

[0100] The overall implementation process of the single-hole tobacco seedling classification and positioning task based on the YOLOv5s model and combined with the above three improved methods is as follows:

[0101] Data preprocessing and block detection: The whole plate of tobacco seedlings image is preprocessed in blocks, and then the block images are input into the detection head of the YOLOv5s model.

[0102] Embedding SimAM module in the feature extraction stage: Introducing the SimAM attention module in the feature extraction process increases the model's attention to the target seedlings in dense seedling scenes.

[0103] SPD-conv downsampling strategy: SPD-conv is used in the downsampling part of the model to retain more detailed information to accurately classify and locate the growth status of each single-hole tobacco seedling.

[0104] Classification and result merging: The model performs classification and positioning in each block image, and restores these detection results to the entire plate image to derive the overall growth status indicators of the tobacco seedlings.

[0105] According to an embodiment of the present invention, preferably, a block detection strategy is introduced based on the YOLOv5s algorithm model, including:

[0106] The entire seedling tray is divided into 200 single-hole sub-images by blocking, so that the proportion of tobacco seedlings in the input hole grid sub-image is adjusted to no less than 1 / 10. When the input resolution is scaled to 640*640, the minimum resolution of the tobacco seedling size in the seedling stage is about 64*64, which is within the detection range of the YOLOv5s model.

[0107] According to an embodiment of the present invention, preferably, based on the YOLOv5s algorithm model, a block detection strategy, a SimAM attention mechanism module and an SPD-conv downsampling strategy are respectively introduced to classify and locate the growth status of a single-hole tobacco seedling, so as to derive the growth status index of the whole tray of tobacco seedlings; wherein the growth status index of the whole tray of tobacco seedlings includes: the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period, and also includes:

[0108] Determine the growth period of the entire tray and the proportion of tobacco seedlings in each period. When more than 50% of the individual plants in the holes enter the preset period, it is determined that the entire tray has entered that period.

[0109] According to an embodiment of the present invention, preferably, the growth period of the whole tray and the proportion of tobacco seedlings in each period are determined, and when more than 50% of the single plants in the holes enter the preset period, it is determined that the whole tray has entered the period, including:

[0110] The counting results of the 200 sub-images are accumulated to obtain the seedling data. When more than one target object is detected in the sub-image, the count is 1, otherwise it is 0, and the count is not repeated. Whether there is a seedling in each hole is output. If no seedling is detected in a single hole grid, it represents an empty hole;

[0111] The prediction results of 200 models were summarized and counted, and the number of seedlings in the seedling emergence stage, small cross stage, large cross stage and holes were represented by a, b, c and d respectively. The proportion of tobacco seedlings in each stage was represented by Count and summarize the emergence rate of the whole tray;

[0112] When the percentage of tobacco seedlings in the preset period exceeds 50%, the growth period of the entire tray is defined as that period.

[0113] Specifically, the present application provides a model for distinguishing the growth and development status of tobacco seedlings during the seedling period based on a deep learning algorithm, and a flow chart of an algorithm for evaluating the growth status of tobacco seedling trays, such as Figure 4 As shown in the figure. The sub-graph in the hole is used as input, the seedling stage category is used as output, and the seedling emergence rate is summarized and output. An end-to-end model for distinguishing the growth and development status of tobacco seedlings in the seedling stage is designed. The improved YOLOV5 algorithm is used as the basic structure for the task of distinguishing the growth and development status of tobacco seedlings in the seedling stage.

[0114] The overall process is summarized as follows Figure 5 The specific steps are as follows:

[0115] Quantify the growth progress index of the whole tray. In the seedling stage, due to the curling and covering of tobacco leaves, a large number of black shadow areas are formed, which makes the leaf area rate in the seedling stage show a slight downward trend. As a result, the leaf area rate intervals of the seedling stage and the big cross stage, and the big cross to the seedling stage overlap. Therefore, the leaf area rate cannot be directly used as the growth progress. Based on the seedling cycle image data set of the same batch collected in the mountainous area of ​​Chongqing, the daily mean leaf area rate of multiple seedling tray images is calculated, and the proportion of the actual seedling days to the total seedling cycle days is taken as the seedling progress of the whole tray, and a linear mapping relationship between the leaf area rate mean and the seedling progress of the whole tray is established. When the seedling stage of the whole tray is before the big cross stage, the leaf area rate interval is [0,0.75], and the actual seedling progress interval converted by days is [0%,75%]. When the seedling stage of the whole tray is in the big cross stage and after, the leaf area rate interval overlaps, but the actual seedling progress interval increases. Combining the seedling stage of the whole tray, the leaf area ratio interval and the seedling progress interval, the growth progress of the whole tray of seedlings can be roughly calculated based on the linear mapping relationship.

[0116] Quantify growth consistency. Because the growth progress of tobacco seedlings in each cell of the whole seedling tray is inconsistent, for example, the tobacco seedlings in some cells are in the seedling stage, and the other cells are in the small cross stage. Even if they are all in the seedling stage, there are cases where the tobacco seedlings are of different sizes. Statistically, the coefficient of variation (CV) is used to measure the degree of dispersion of data. The Simpson coefficient measures the uniformity of data, and the more uniform the distribution, the higher the value. The Gini coefficient is used to measure the degree of inequality in a set of data or distribution, which is between 0 and 1. The more equal it is, the closer it is to 0, and the more unequal it is, the closer it is to 1. The present invention quantifies the growth consistency of tobacco seedlings in each cell based on the Gini coefficient. In extreme cases, when the leaf area in the 200 cells of the whole tray is completely equal, the Gini coefficient approaches 0 infinitely, indicating that the growth consistency of the whole tray is high.

[0117] A number of improvement strategies are introduced to the YOLOv5s algorithm model structure. The improved model structure is as follows Figure 6As shown, a block detection strategy is introduced, a SimAM attention module is added, and a downsampling strategy is optimized. First, since the original seedling tray image resolution is 1600*800, YOLOv5s has three detection layers to detect small, medium, and large targets respectively. The minimum detectable target is 1 / 20 of the input resolution. The single-hole grid occupies 1 / 200 of the size of the entire seedling tray, and the size of each type of tobacco seedling occupies about 1 / 10 to 1 of the single-hole grid size. Therefore, the tobacco seedlings occupy about 1 / 2000 to 1 / 200 of the size of the entire seedling tray. Therefore, they are all small targets, which will make it difficult to locate and detect tobacco seedlings. This application divides the entire seedling tray into 200 single-hole sub-images in a block manner, so that the proportion of tobacco seedlings to the input hole grid sub-image is adjusted to not less than 1 / 10. When the input resolution is scaled to 640*640, the minimum resolution of the tobacco seedling size in the seedling stage is about 64*64, which is within the detection range of the YOLOv5s model. By batch-feeding 200 images after segmentation into the YOLOv5s detection model for parallel reasoning and prediction, and then merging the detection results, the difficulty of small target detection is reduced, and the problem of easy sampling loss of small targets in large image detection is solved. Secondly, adding an attention mechanism can help the model focus on the key areas of the image and improve the model performance. It can be embedded in multiple modules such as Backbone, Neck, and Head of YOLOv5s. At present, the mainstream attention modules include the global attention mechanism GAM (Global Attention Mechanism), the dual-channel attention mechanism CBAM (Convolutional Block Attention Module), and the SimAM attention mechanism (Simple Attention Mechanism). SimAM is a lightweight, parameter-free attention mechanism. Unlike channel-based and space-based attention modules, SimAM works by inferring three-dimensional attention weights in feature maps without adding parameters to the original network. Quantitative evaluation of various visual tasks shows that the SimAM attention mechanism module is flexible and effective, and can improve the representation ability of many ConvNets. Finally, since the resolution of the cut single hole grid image is 80*80, and the input resolution of YOLOv5s is 640*640. When processing low-resolution images, the pooling layer in the convolution architecture will lead to the loss of fine-grained information and less efficient feature representation. SPD-Conv consists of a space-to-depth (SPD) layer and a non-cross-row convolution (Conv) layer. SPD-Conv downsamples the feature map without losing learnable information, completely abandoning the currently widely used cross-row convolution and pooling operations, and has significant performance improvements on small objects and low-resolution images. The present invention uses the SPD-Conv module to replace all cross-row convolution layers and pooling layers in YOLOv5s.In order to verify whether each optimization module can improve the accuracy of the YOLOv5s model, each improved module is combined with the original YOLOv5s model separately, and the ablation experiment results of the YOLOv5s model after introducing each improved module are compared on the test set.

[0118] Determine the growth period of the whole plate and the proportion of tobacco seedlings in each period. When more than 50% of the single plants in the holes enter a certain period, the whole plate is considered to have entered that period. For example, if the proportion of tobacco seedling holes in the seedling stage is less than 50%, and the empty holes exceed 50%, the whole seedling plate is still in the sowing period. If the proportion of tobacco seedling holes in the seedling stage exceeds 50%, the whole seedling plate has entered the seedling stage from the sowing period, and so on. The classification of the growth period of the whole plate includes sowing period, seedling period, small cross period, large cross period, and seedling period. During the model training process, first modify the model configuration file and modify the number of categories to be identified in the model yaml file. Including the seedling period, small cross period, large cross period, and seedling period, set the parameter nc to 4. If it is a seedling rate detection model, only the seedling period and small cross period appear in the early seedling plate sample, set the parameter nc to 2. Modify the training set and test set data path, number of categories, and category names in the data yaml file. According to the improvement strategy of the model structure, the model structure file was modified. Since the scale of tobacco seedlings changes during their growth, the parameters of the YOLOV5 model architecture were fine-tuned again, the anchors scale was modified, and a small target detection layer was added, so that tobacco seedlings of different sizes can be accurately identified and located. In the reasoning process, firstly, the whole seedling tray was used as a unit, and 200 single-hole sub-images were divided and input into the YOLOv5s model in a single batch for parallel reasoning, and the emergence ratio of the entire seedling tray was calculated in units of holes.

[0119] According to an embodiment of the present invention, preferably, after determining the growth period of the whole tray and the proportion of tobacco seedlings in each period, when more than 50% of the single plants in the holes enter the preset period, it is determined that the whole tray has entered the period, it also includes:

[0120] Visualization of evaluation indicators of the model training process and visualization of test set output results.

[0121] Specifically, the model training and output visualization includes the visualization of the evaluation indicators of the model training process and the visualization of the test set output results. The test image can show the growth and development status of the seedlings in the hole that the model identifies, as well as the confidence of the model in the output results.

[0122] like Figure 7As shown in the figure, a comparison of the detection results after the optimization of the YOLOv5s model is given. Before the improvement, the confidence was set to 0.1, allowing the model to detect as many target objects as possible, but YOLOv5s still had a large number of missed tobacco seedling targets, and the confidence of the detection frame was low. It was difficult for the original YOLOv5s algorithm to effectively detect small target tobacco seedlings in the cells. After the improvement, the confidence was set to 0.3. By comparison, it was found that except for the missed detection of the cell in the upper right corner, the other cells were correctly detected with a higher confidence. The detection results of the improved YOLOv5s model were better than the original algorithm. This shows that the improvement of the YOLOv5s model in this study has certain advantages in the detection of small target tobacco seedlings in the seedling tray. Comprehensive indicators of tobacco seedling growth and development status are as follows: Figure 8 As shown, the blue frame E represents the seedling stage, the orange frame X represents the small cross stage, and D represents the large cross stage.

[0123] In the present application, the tobacco seedling tray is a special container for the seedling stage, which has many advantages such as providing a uniform growth environment for tobacco seedlings, facilitating monitoring and management, saving space, and being easy to transplant. Continuous monitoring of the growth and development status of tobacco seedling trays can provide data support for large-scale, digital, unmanned seedling factories to control the greenhouse environment, water and fertilizer usage, and seedling transplanting and other decisions in real time. In view of the high technical management requirements for seedling management personnel in the identification of the growth and development status of tobacco seedlings in the seedling stage, and the uncontrollable transplanting losses caused by human factors in the manual operation mode, the present invention provides a method for constructing a model for identifying the growth and development status of tobacco seedlings in the seedling stage based on a deep learning image recognition algorithm according to the strategy of detecting crop groups from single crop detection. First, the whole tray of tobacco leaf area is extracted based on the normalized super green feature method (ExG) and the maximum inter-class variance method (Otsu), and the linear mapping relationship between the tobacco seedling leaf area rate and the whole tray seedling progress is analyzed to achieve the derivation of the whole tray seedling progress based on the leaf area rate. Secondly, the tobacco leaf area is extracted from the single hole after block division, and the distribution of leaf area ratio is analyzed by Gini coefficient to infer the growth consistency of the whole tray. Finally, by introducing the block strategy, SimAM attention mechanism and SPD-conv downsampling strategy in the YOLOv5s model, the problem of easy sampling loss of small targets in large image detection, the aggregation ability between image features, the loss of fine-grained information and the feature representation of low-resolution images are solved respectively. The improved YOLOv5s target detection model is used for the classification and positioning of the growth status of single-hole tobacco seedlings, and finally the calculation of the growth status index of the whole tray of tobacco seedlings is derived, where the growth status index of the whole tray of tobacco seedlings includes the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period. The target detection task is to accurately and efficiently identify and locate a large number of predefined categories of object instances from the image. This project will predefine 4 seedling growth and development status categories from a large number of manually collected seedling tray images, including the emergence period, small cross period, large cross period and seedling period. In the model for distinguishing the growth and development status of tobacco seedlings in the seedling stage, the labelimg tool is used to label the tobacco seedlings in the holes at the seedling stage, small cross stage, large cross stage and seedling stage from a large number of sample images. The above-mentioned improved deep learning YOLOV5 target detection technology is used to detect the growth and development status of the tobacco seedlings in each hole from the seedling tray image. The position of the tobacco seedling in the hole is located in the form of a target frame. In the seedling emergence rate detection model, it is used to identify and locate whether the seedlings in the seedling tray have emerged or not. Taking the hole as the unit, one or more seedlings located in the same hole are counted as one emergence, and the emergence rate of the entire seedling tray is calculated. Then a reference for the next transplanting strategy is given.

[0124] From the above description, it can be seen that the present application achieves the following technical effects:

[0125] In the embodiment of the present application, a method based on deep learning image recognition is adopted, by collecting images of seedling trays at various seedling ages during the seedling raising process; preprocessing the seedling tray images and segmenting them to generate a number of seedling hole grid maps; using the Labelimg tool to annotate the seedling hole sub-maps and divide the data set; based on the deep learning algorithm, model training is performed on the annotated seedling hole sub-map data set to train a target discrimination and seedling emergence rate detection model for the tobacco seedling raising period, thereby achieving the purpose of training and generating a target discrimination and seedling emergence rate detection model for the tobacco seedling raising period, thereby realizing accurate and efficient identification and positioning of a large number of predefined categories. The technical effect of other object instances further solves the problem that the current group growth detection of seedling crops is mostly limited to a fixed seedling age state. Although the existing technology can realize the growth status classification of tobacco seedlings in different cell grids, it is limited to a fixed seedling age and cannot be extended to growth monitoring of tobacco seedlings throughout the growth period, or is limited to the growth detection of a single adult crop. It lacks or ignores the continuous growth monitoring of group crops at each seedling stage and the single crop morphology detection and monitoring technology based on machine vision is relatively mature, while the extraction of key growth parameters of seedling crop groups and the growth monitoring of crop groups throughout the growth period are less studied technical problems.

[0126] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a device for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate is provided. Fig.10 As shown, the tobacco seedling growth state identification and emergence rate detection device during the seedling raising period comprises:

[0127] The seedling tray image acquisition module 1001 is used to collect images of seedling trays at different seedling ages during the seedling raising process;

[0128] By collecting a large number of seedling tray images, accurate image collection can be achieved, thus providing a data basis for subsequent analysis and processing.

[0129] The preprocessing module 1002 is used to preprocess the seedling tray image and segment it to generate a plurality of seedling hole grid images;

[0130] By preprocessing the seedling tray image and generating several seedling hole sub-images, the accuracy of the image information can be ensured, and the image can be segmented into multiple required image styles, thereby providing guarantee for subsequent image processing.

[0131] The image annotation and data set segmentation module 1003 is used to use the Labelimg tool to perform seedling category annotation and positioning frame annotation on a plurality of seedling hole sub-images and randomly segment the data set;

[0132] By adopting the preset algorithm to label all the seedling hole sub-images and dividing the data set at the same time, a good image labeling effect can be achieved. At the same time, the images are divided into corresponding data sets according to the preset requirements, thereby providing accurate image data for the subsequent model training.

[0133] The model training module 1004 is used to perform model training on the labeled seedling hole sub-image data set based on a deep learning algorithm, and to train a target discrimination model and a seedling emergence rate detection model for the tobacco seedling raising period.

[0134] The tobacco seedling target detection model is trained by inputting the seedling hole sub-map to be labeled and the relevant data set information into the deep learning model for model training.

[0135] From the above description, it can be seen that the present application achieves the following technical effects:

[0136] In the embodiment of the present application, a method based on deep learning image recognition is adopted, by collecting images of seedling trays at various seedling ages during the seedling raising process; preprocessing the seedling tray images and segmenting them to generate a number of seedling hole grid maps; using the Labelimg tool to annotate the seedling hole sub-maps and divide the data set; based on the deep learning algorithm, model training is performed on the annotated seedling hole sub-map data set to train a target discrimination and seedling emergence rate detection model for the tobacco seedling raising period, thereby achieving the purpose of training and generating a target discrimination and seedling emergence rate detection model for the tobacco seedling raising period, thereby realizing accurate and efficient identification and positioning of a large number of predefined categories. The technical effect of other object instances further solves the problem that the current group growth detection of seedling crops is mostly limited to a fixed seedling age state. Although the existing technology can realize the growth status classification of tobacco seedlings in different cell grids, it is limited to a fixed seedling age and cannot be extended to growth monitoring of tobacco seedlings throughout the growth period, or is limited to the growth detection of a single adult crop. It lacks or ignores the continuous growth monitoring of group crops at each seedling stage and the single crop morphology detection and monitoring technology based on machine vision is relatively mature, while the extraction of key growth parameters of seedling crop groups and the growth monitoring of crop groups throughout the growth period are less studied technical problems.

[0137] The present invention also has the following beneficial effects:

[0138] Key point 1: When collecting images of seedling trays, currently manually collected images have serious skew and background clutter problems. After the method of the present invention, the seedling tray edge detection and affine transformation technology are used to calibrate the skew of the original image and crop the seedling tray. Obtaining a standard horizontal square seedling tray area is a prerequisite for subsequent sub-image cutting. The seedling tray correction and image segmentation operations are very simple.

[0139] Key point 2: In the process of identifying the growth and development status of tobacco seedlings, considering that the growth speed of tobacco seedlings in each hole in the seedling tray is inconsistent, it is more accurate to identify a small hole in the seedling tray than to identify the status of the whole tray, and it is convenient to calculate the emergence rate of the whole tray, which also provides the prerequisite for subsequent transplanting of seedlings.

[0140] Key point 3: In the seedling rate detection model, the YOLOV5 target detection and recognition algorithm is used. The network structure diagram is divided into four parts: input, Backbone, Neck, and Prediction. The input includes Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling technology. Backbone adopts Focus structure and CSP structure; Neck adopts FPN+PAN structure, and Yolov5 uses CIOU_Loss as the loss function of Bounding box on the Prediction side. Since the scale of tobacco seedlings changes during their growth, the YOLOV5 model architecture is fine-tuned, the anchors scale is modified, and a small target detection layer is added, so that tobacco seedlings of different sizes, even those with extremely small morphology, can be accurately identified and located. After the model is trained, it can accurately identify whether the seedlings have emerged in the hole and locate the location of the seedlings, and then count and summarize the emergence rate of the entire plate. The model performed well, with a classification accuracy of 93.8% and mAP@0.5 of 97.6%. Even seedlings photographed with very low pixels by mobile phones could accurately identify their growth and development status (seedling stage), and precisely locate whether there were seedlings in each hole, and then calculate the seedling emergence rate of the entire tray.

[0141] Key point 4: In the model for distinguishing the growth and development status of tobacco seedlings during the seedling period, based on the strategy of detecting individual crops and crop groups, a seedling tray growth status evaluation algorithm based on the improved YOLOv5s target detection model is proposed, which can evaluate the growth status of the entire tray of tobacco seedlings from the growth status of a single-hole tobacco seedling, and complete the prediction of growth status indicators including the progress of the entire tray, the consistency of the growth of the entire tray, the growth period, and the proportion of tobacco seedlings in each period. First, the whole tray of tobacco leaf area is extracted based on the normalized super green feature method (ExG) and the maximum inter-class variance method (Otsu), and the linear mapping relationship between the tobacco seedling leaf area rate and the progress of the entire tray of seedlings is analyzed; secondly, the tobacco leaf area is extracted from the single hole after block division, and the distribution of the leaf area rate is analyzed by the Gini coefficient to infer the growth consistency of the entire tray. By introducing the block strategy, SimAM attention mechanism and SPD-conv downsampling strategy in the YOLOv5s model, the problem of easy sampling loss of small targets in large image detection is solved, the aggregation ability between image features is enhanced, the loss of fine-grained information is reduced, and the feature representation of low-resolution images is enhanced. The improved YOLOv5s target detection model is used to classify and locate the growth status of single-hole tobacco seedlings, and finally completes the calculation of the growth status indicators of the entire tray of tobacco seedlings. The results show that the mAP@0.5:0.95 of single-hole tobacco seedling detection reaches 80.7%, which is 67% higher than the original YOLOv5s model. The improved YOLOv5s algorithm can effectively monitor the growth and development status of tobacco seedlings in the seedling tray, providing technical support for the realization of intelligent and unmanned tobacco seedling cultivation.

[0142] Key point 5: In view of the fact that existing research is limited to the detection of empty holes or multiple plants in one hole, and only focuses on a single hole and a single plant, and its application scenario is limited to tobacco seedling transplanting, the present invention follows the direction of automated detection of single-plant crop morphology to automated detection of crop groups, and from the perspective of "individual to group", explores the feasibility of evaluating the growth status of the entire tray of tobacco seedlings based on the growth status of tobacco seedlings in a single hole, and proposes a method for automatically judging the growth status of a seedling tray based on an improved YOLOv5s target detection model combined with leaf area ratio. Not only can it realize the detection of empty holes and multiple plants in a single hole, but it can also calculate the distribution of different seedling stages in each hole and the proportion of different seedling stages, further realizing a comprehensive multi-indicator evaluation of the growth status of the entire tray group. The application scenarios are richer, covering multiple scenarios such as tobacco seedling transplanting, growth trend detection, and development progress estimation.

[0143] Key point 6: Existing research is limited to fixed seedling age, especially seedling age in the seedling stage, and cannot be extended to growth monitoring of tobacco seedlings throughout their entire growth period. The present invention can achieve growth monitoring of the entire tray group during the entire growth period from sowing to seedling stage.

[0144] like Fig.11As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0145] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0146] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the growth state discrimination and emergence rate detection method of tobacco seedlings during the seedling period and related devices.

[0147] In some embodiments, the growth state discrimination and emergence rate detection method and related apparatus of the tobacco seedlings during the seedling raising period can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the growth state discrimination and emergence rate detection method and related apparatus of the tobacco seedlings during the seedling raising period described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the growth state discrimination and emergence rate detection method and related apparatus of the tobacco seedlings during the seedling raising period by any other appropriate means (for example, by means of firmware).

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

Claims

1. A method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate, characterized in that: include: S1, collecting images of seedling trays at different seedling ages during the seedling raising process; S2, preprocessing the seedling tray image and segmenting it to generate a number of seedling hole grid images; S3, using Labelimg tool to label the seedling categories and positioning frames of several seedling hole sub-maps and randomly split the data set; S4. Based on the deep learning algorithm, the model training is carried out on the labeled seedling hole sub-image dataset to train the tobacco seedling target discrimination and emergence rate detection model during the seedling raising period.

2. The method for determining the growth status of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 1, characterized in that: Collect images of seedling trays at different seedling ages during the seedling raising process, including: Through on-site monitoring of the development process of the seedling trays at the target base, original seedling tray images with preset pixels are regularly collected to obtain several original images of the seedling trays during the entire growth period from seedling emergence to seedling maturity; According to the definition of seedling stage, the original images are classified into four seedling stage categories; among them, the original images include: seedling stage, small cross stage, large cross stage and seedling stage.

3. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 1, characterized in that: The seedling tray image is preprocessed and segmented to generate several seedling hole grid images, including: The original image is skew-calibrated and cropped by using seedling tray edge detection and affine transformation algorithm to obtain a standard square seedling tray. Each seedling tray has 10 rows and 20 columns, with a total of 200 seedling holes, which are cut into 200 hole grid images according to the resolution. The resolution of the hole grid image is 80*80. The seedling trays were processed using an image cutting algorithm, and each cropped seedling tray image was cut into 200 seedling hole sub-images; Based on the normalized super-green feature method and the maximum inter-class variance method, the green tobacco seedling leaf area and seedling tray border area of ​​the whole tray were extracted, and the linear mapping relationship between the tobacco seedling leaf area rate and the whole tray seedling progress was analyzed, and the whole tray seedling progress based on the leaf area rate was derived.

4. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 1, characterized in that: The Labelimg tool was used to label the seedling categories and positioning frames of several seedling hole sub-graphs and randomly split the data set, including: The Labelimg tool was used to label the seedling stage category and position of the seedlings in each small hole sub-graph, and an XML format labeling file of each sub-graph was obtained; the labeling categories included: seedling stage, small cross stage, and large cross stage; The labeled seedling hole sub-maps are randomly divided into training sets and validation sets according to the preset ratio.

5. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 1, characterized in that: Based on the deep learning algorithm, the model training is carried out on the labeled seedling hole sub-image dataset to train the tobacco seedling target discrimination and emergence rate detection model during the seedling raising period, including: Based on the collected image data sets of the same batch of seedling raising cycle, the daily average leaf area rate of multiple seedling tray images was calculated, and the proportion of actual seedling raising days to the total actual seedling raising cycle days was taken as the seedling raising progress of the whole tray, and a linear mapping relationship between the leaf area rate average and the seedling raising progress of the whole tray was established. For the tobacco leaf area extracted from a single hole after block division, the Gini coefficient was used to analyze the distribution of leaf area ratio to estimate the growth consistency of the whole tray; Based on the YOLOv5s algorithm model, the block detection strategy, SimAM attention mechanism module and SPD-conv downsampling strategy were introduced to classify and locate the growth status of single-hole tobacco seedlings, so as to derive the growth status indicators of the whole tray of tobacco seedlings; among them, the growth status indicators of the whole tray of tobacco seedlings include: the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period.

6. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 1, characterized in that: Based on the YOLOv5s algorithm model, a block detection strategy is introduced, including: The entire seedling tray is divided into 200 single-hole sub-images by blocking, so that the proportion of tobacco seedlings in the input hole grid sub-image is adjusted to no less than 1 / 10. When the input resolution is scaled to 640*640, the minimum resolution of the tobacco seedling size in the seedling stage is about 64*64, which is within the detection range of the YOLOv5s model.

7. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 6, characterized in that: Based on the YOLOv5s algorithm model, the block detection strategy, SimAM attention mechanism module and SPD-conv downsampling strategy are introduced to classify and locate the growth status of single-hole tobacco seedlings, so as to derive the growth status indicators of the whole tray of tobacco seedlings; among them, the growth status indicators of the whole tray of tobacco seedlings include: the growth progress of the whole tray, the growth consistency of the whole tray, the development period and the proportion of tobacco seedlings in each period, and also include: Determine the growth period of the entire tray and the proportion of tobacco seedlings in each period. When more than 50% of the individual plants in the holes enter the preset period, it is determined that the entire tray has entered that period.

8. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 7, characterized in that: Determine the growth period of the whole tray and the proportion of tobacco seedlings in each period. When more than 50% of the individual plants in the holes enter the preset period, the whole tray is considered to have entered that period, including: The counting results of the 200 sub-images are accumulated to obtain the seedling data. When more than one target object is detected in the sub-image, the count is 1, otherwise it is 0, and the count is not repeated. Whether there is a seedling in each hole is output. If no seedling is detected in a single hole grid, it represents an empty hole; The prediction results of 200 models were summarized and counted, and the number of seedlings in the seedling emergence stage, small cross stage, large cross stage and holes were represented by a, b, c and d respectively. The proportion of tobacco seedlings in each stage was represented by Count and summarize the emergence rate of the whole tray; When the percentage of tobacco seedlings in the preset period exceeds 50%, the growth period of the entire tray is defined as that period.

9. The method for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate according to claim 7, characterized in that: Determine the growth period of the whole tray and the proportion of tobacco seedlings in each period. When more than 50% of the single plants in the hole enter the preset period, it is determined that the whole tray has entered the period. It also includes: Visualization of evaluation indicators of the model training process and visualization of test set output results.

10. A device for determining the growth state of tobacco seedlings during the seedling raising period and detecting the emergence rate, characterized in that: include: The seedling tray image acquisition module is used to collect images of seedling trays at various seedling ages during the seedling raising process; A preprocessing module is used to preprocess the seedling tray image and segment it to generate a number of seedling hole grid images; The model training module is used to perform model training on the labeled seedling hole sub-map dataset based on the deep learning algorithm, and to train the tobacco seedling target discrimination and emergence rate detection model during the seedling raising period.

Images