Deep learning-based intelligent identification method for growth state of traditional Chinese medicinal materials in seedling growing period

Abstract

The invention discloses a deep learning-based traditional Chinese medicinal material seedling stage growth state intelligent identification method. The method comprises the following steps of S1, obtaining original image data of a traditional Chinese medicinal material seedling region; s2, an improved Faster R-CNN traditional Chinese medicinal material recognition model is constructed; s3, predicting a candidate region; s4, outputting a candidate region feature vector set by adopting a RoIAlign algorithm; s5, introducing a DropBlock regularization method to generate a seedling body detection frame and a growth state recognition result set; s6, taking the improved Faster R-CNN model as a teacher network, and training a lightweight student network through a knowledge distillation mechanism; and S7, outputting a corresponding Chinese herbal medicine seedling growth state identification result. The method achieves the precise recognition and positioning of the growth state of the seedling body in the traditional Chinese medicinal material seedling raising period image, is high in recognition precision, is light in model weight, and is suitable for an intelligent seedling raising monitoring scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and agricultural intelligent monitoring, and particularly relates to a method for intelligently identifying growth state of traditional Chinese medicinal materials in seedling stage based on deep learning. BACKGROUND

[0002] In modern agricultural production, especially in the process of traditional Chinese medicinal material seedling management, the identification of growth state in seedling stage is of great significance for guiding scientific irrigation, disease warning and subsequent cultivation decision. The traditional identification and growth state judgment of traditional Chinese medicinal material seedlings mainly rely on manual patrol or manual feature extraction method based on simple image processing algorithm. Such methods generally have the problems of low recognition efficiency, high labor cost and poor environmental adaptability. Especially in the scene of complex background, large light change and subtle growth state change of seedlings in seedling area, the traditional method is difficult to effectively extract stable features, resulting in insufficient accuracy of recognition results, and it is difficult to meet the intelligent monitoring needs of large-scale seedling operation.

[0003] In recent years, the target detection and classification method based on deep learning has been widely applied in the field of computer vision. Typical representatives such as FasterR-CNN model combine region proposal network (RPN) with backbone convolutional neural network for end-to-end training, and show high precision in general target detection tasks. However, the existing FasterR-CNN still has the following shortcomings in the scene of agricultural image recognition: first, the original FasterR-CNN relies on Anchor mechanism for candidate frame generation, which will cause insufficient coverage or redundant overlap of anchor frame under the condition of variable target size and complex morphology of traditional Chinese medicinal material seedlings, affecting the quality of candidate region; second, the standard FasterR-CNN usually uses deep network such as ResNet as the backbone structure, which is difficult to realize efficient deployment in embedded device or on-site edge computing environment; in addition, in the model training stage, there is lack of knowledge transfer mechanism for the fine-grained category of traditional Chinese medicinal material seedling state, which leads to insufficient generalization ability and recognition stability of the model in non-ideal environment.

[0004] Therefore, it is urgent to design an improved FasterR-CNN model for traditional Chinese medicinal material seedling stage image, which is lightweight and high-precision, by introducing Anchor-Free candidate region generation mechanism, feature alignment algorithm, regularization structure and knowledge distillation method, to improve the recognition performance and deployment efficiency of the model in complex seedling scene, so as to realize the intelligent and accurate identification of growth state of traditional Chinese medicinal material in seedling stage. SUMMARY

[0005] One purpose of the present application is to propose an intelligent recognition method for the growth state of Chinese herbal medicine seedlings during the seedling stage based on deep learning, which fully integrates a lightweight feature extraction network, an Anchor-Free region proposal mechanism, a feature alignment algorithm, and a knowledge distillation training strategy. The construction and optimization process of the improved FasterR-CNN model in complex seedling images is described in detail, and the precise recognition and output of the seedling position and its growth state are realized. The present application has the advantages of high deployment efficiency, high recognition accuracy, and strong environmental adaptability, and can effectively improve the automation and intelligence level of Chinese herbal medicine seedling management.

[0006] According to an embodiment of the present application, an intelligent recognition method for the growth state of Chinese herbal medicine seedlings during the seedling stage based on deep learning includes the following steps:

[0007] S1, obtaining original image data of the Chinese herbal medicine seedling area;

[0008] S2, constructing an improved FasterR-CNN Chinese herbal medicine recognition model, which uses MobileNetV3 as the backbone network for feature extraction;

[0009] S3, introducing an anchor-free region proposal mechanism on the multi-scale feature map, and jointly predicting the candidate regions through the foreground probability score and the bounding box regression offset;

[0010] S4, using the RoIAlign algorithm to align the features of the candidate regions, and outputting the candidate region feature vector set;

[0011] S5, inputting the candidate region feature vector into the subnetwork, introducing the DropBlock regularization method to suppress overfitting, and generating the seedling detection frame and the growth state recognition result set;

[0012] S6, constructing a joint supervision training of the improved FasterR-CNN model based on the cross-entropy and smooth L1 loss function, using the improved FasterR-CNN model as the teacher network, and training the lightweight student network through the knowledge distillation mechanism;

[0013] S7, inputting the real-time seedling image collected on site into the improved FasterR-CNN model after training and compression, and outputting the corresponding Chinese herbal medicine seedling growth state recognition result.

[0014] Optionally, the original image data is automatically acquired by an image acquisition device deployed in the seedling raising area according to a set period, and the acquisition period is automatically adjusted according to the growth state of the Chinese herbal seedling body: the germination period is set to acquire once every 5 minutes, the rapid growth period is set to acquire once every 10 minutes, and the stable period is set to acquire once every 30 minutes; the adjustment mechanism is used to improve the data update frequency of the key stage of seedling raising and improve the timeliness and response ability of the growth state recognition; the original image data is subjected to enhancement processing, including brightness equalization, resolution normalization and noise suppression, to generate a standardized image data set, and the standardized image data set is divided into a training set and a test set.

[0015] Optionally, the feature extraction is a multi-layer convolution and activation operation on the standardized image data, which extracts texture information, structural boundaries and spatial semantic features in the Chinese herbal seedling image, and outputs a multi-scale feature map;

[0016] Optionally, the S3 specifically comprises:

[0017] S31, introducing an anchor-free region proposal mechanism on the multi-scale feature map output by the MobileNetV3 backbone network, wherein the anchor-free region proposal mechanism refers to performing foreground classification judgment and boundary box offset value prediction on each pixel in the multi-scale feature map;

[0018] S32, the foreground classification judgment is a binary classification prediction for each pixel position:

[0019] A foreground classification branch structure is constructed at the top layer of the feature map, a 1x1 convolution layer with a Sigmoid activation function is used to compress the multi-channel feature map into a single-channel foreground probability map; the value of each pixel position in the probability map represents the confidence degree of the corresponding original image region belonging to the foreground of the Chinese herbal seedling body, and the value range is 0 to 1; in the training process, the region of the Chinese herbal seedling body in the labeled image is used as a positive sample, and the remaining background region is used as a negative sample, a foreground label map is constructed, and a binary cross entropy loss function is used to supervise the training of the foreground classification branch to optimize the discrimination ability of the network to the foreground region.

[0020] S33, the boundary box offset value prediction is to output the distance offset values of the pixel position to the upper, lower, left and right four boundaries of the target boundary box for each pixel position, respectively denoted as upper offset, lower offset, left offset and right offset; taking the pixel point as the center, combining the four direction offset values obtained by prediction, the position of the corresponding rectangular candidate box is determined to generate an initial candidate region set;

[0021] S34, sort the initial candidate region set according to the foreground confidence score from high to low, select the candidate region whose foreground confidence score is higher than the set threshold and whose ranking is in the top N, wherein N is the preset maximum candidate box upper limit, and limit the part of the initial candidate region that exceeds the image boundary coordinate to the image size range;

[0022] S35, filter the candidate box with too small area in the candidate region set, remove the detection box with size lower than the set threshold, and reserve the effective candidate region;

[0023] S36, perform non-maximum suppression operation on the reserved candidate region, remove the repeated region according to the overlapping condition between the candidate regions, and output the candidate region set.

[0024] Optionally, the S4 specifically comprises:

[0025] S41, map the candidate region coordinates from the original image space to the multi-scale feature map space according to the down-sampling ratio of the MobileNetV3 backbone network, and obtain the candidate box coordinates under the feature map scale;

[0026] S42, perform the following RoIAlign operation on each mapped candidate box:

[0027] Divide the candidate box region in the feature map into a fixed number of uniform grid units to form a KxK sampling grid structure;

[0028] In each grid, select four fixed sampling points, and calculate the feature value of each sampling point by using bilinear interpolation method according to the continuous position of the sampling point in the feature map;

[0029] Perform average operation on the four sampling point feature values of each grid unit to generate the final feature representation inside the grid and obtain the candidate region feature vector with uniform size;

[0030] S43, perform RoIAlign operation on all candidate regions respectively to form a set of candidate region feature vector sets with uniform size;

[0031] Optionally, the S5 specifically comprises:

[0032] S51, input the candidate region feature vector into the subnetwork containing two fully connected structures in sequence, the first fully connected layer outputs the intermediate expression vector and connects the ReLU activation function, and the second fully connected layer outputs the prediction results of the classification branch and the regression branch in parallel;

[0033] S52, the classification branch takes the candidate region feature vector as input, sequentially passes through a fully connected layer and a Softmax normalization function, outputs a probability distribution of each growth state category, takes the category index corresponding to the maximum probability value as the growth state classification label of the seedling body, and takes the maximum probability value as the corresponding confidence score;

[0034] S53, the regression branch is used for predicting the bounding box correction of the candidate region feature vector, and outputs boundary offset values in the up, down, left and right four directions relative to the initial candidate box;

[0035] S54, DropBlock regularization is introduced between the two fully connected layers, and continuous channel regions in the candidate region feature vector are randomly shielded according to the set reservation probability to inhibit overfitting;

[0036] S55, the growth state category output by the classification branch and the bounding box correction result output by the regression branch are fused to generate a final set of seedling detection boxes and growth state recognition results.

[0037] Optionally, the S6 specifically comprises:

[0038] S61, the growth state category probability distribution output by the candidate region feature vector in the classification branch is supervised and trained by using a cross-entropy loss function;

[0039] S62, the boundary offset value output by the candidate region feature vector in the regression branch is supervised and trained by using a smooth L1 loss function;

[0040] S63, a total loss function is constructed as a model joint optimization target, which is used to simultaneously optimize the growth state recognition ability of the classification branch and the boundary prediction accuracy of the regression branch;

[0041] S64, the improved FasterR-CNN model after training is taken as a teacher network, a lightweight student network is constructed as a target model, the structure parameters of the student network are simplified, and the main network part is kept as a shallow separable convolution structure;

[0042] S65, the student network is trained by using a knowledge distillation mechanism, and a distillation loss function is introduced to jointly supervise the classification output and regression output of the student network and the teacher network at two levels.

[0043] Optionally, the recognition result includes normal growth, slow growth, leaf lesion, wilting and drying, and corresponding confidence, which is used to drive the seedling management platform to perform automatic state display and abnormal alarm operation.

[0044] Optionally, the improved FasterR-CNN model specifically comprises the following steps:

[0045] Take MobileNetV3 as the backbone network, perform layer-by-layer convolution and activation on the input standardized image data, and output multi-scale feature maps; introduce an anchor-free region proposal mechanism on the multi-scale feature maps, take pixels as the prediction center, and perform foreground classification judgment and boundary box offset value prediction respectively to form an initial candidate region set;

[0046] The initial candidate region set is sorted by confidence and boundary constraint correction, and a candidate region set is output; the coordinates of the screened candidate regions are mapped from the original image space to the multi-scale feature map space according to the MobileNetV3 downsampling ratio, and the RoIAlign operation is performed to obtain a candidate region feature vector set;

[0047] The candidate region feature vectors are sequentially input into a subnetwork containing two fully connected structures, the classification branch outputs the probability distribution of each growth state class through the Softmax function, and determines the class label and confidence according to the maximum probability; the regression branch outputs the boundary offset of the candidate box in the up, down, left and right directions, and DropBlock regularization is introduced between the two fully connected layers;

[0048] Based on the classification output and regression output of the candidate region, the cross-entropy loss function is used to supervise the training of the growth state class, the smooth L1 loss function is used to supervise the training of the boundary offset value, and the joint loss function is constructed for overall optimization;

[0049] The improved FasterR-CNN model trained above is used as a teacher network to construct a chemical network, and a distillation loss function is introduced to jointly supervise the training of the student network in the classification output and regression output.

[0050] The deep learning-based intelligent identification method for the growth state of Chinese herbal medicine seedlings proposed in the application has achieved remarkable beneficial effects in solving the problems of relying on manual observation, insufficient recognition accuracy, high model complexity and difficulty in deployment of existing methods.

[0051] The improved FasterR-CNN model constructed in the application takes MobileNetV3 as the backbone feature extraction network, effectively reduces the computational complexity of the model, and improves the deployment efficiency on edge devices. At the same time, the Anchor-Free region proposal mechanism is introduced, which replaces the traditional anchor structure through a joint prediction method of pixel-level foreground probability and boundary offset, avoids the problems of anchor scale matching difficulty and redundant candidate region generation, and significantly improves the accuracy and efficiency of candidate region extraction.

[0052] The RoIAlign feature alignment algorithm is introduced in the candidate region feature extraction process, the down-sampling ratio mapping and the bilinear interpolation operation are combined, the spatial consistency between the candidate region and the feature map is ensured, and the detection and identification accuracy is improved. The DropBlock regularization method is further introduced in the classification and regression subnetwork, the overfitting phenomenon is effectively inhibited by randomly shielding the continuous channel region, and the robustness of the model in the complex seedling background is enhanced.

[0053] The improved FasterR-CNN model is trained by combining the cross-entropy and the smooth L1 loss function, and the improved FasterR-CNN model is used as a teacher network, and a lightweight student network is constructed through a knowledge distillation mechanism, so that the model parameter size and the inference calculation amount are compressed under the premise of maintaining the identification accuracy, and the real-time identification capability and the practical application value of the model in the Chinese herbal medicine seedling field are enhanced.

[0054] The application optimizes the structure design, algorithm process and training strategy, improves the accuracy of Chinese herbal medicine seedling detection and growth state recognition, effectively meets the actual needs of model lightweight and convenient deployment in seedling management, has good popularization prospect and industrial application value. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings are included to provide a further understanding of the application and constitute a part of the specification, illustrate the application together with the embodiments of the application, and do not constitute a limitation on the application. In the drawings:

[0056] Fig. 1 A whole flow chart of a Chinese herbal medicine seedling growth state intelligent identification method based on deep learning is provided for the application;

[0057] Fig. 2 An anchor-free region suggestion mechanism structure diagram of a Chinese herbal medicine seedling growth state intelligent identification method based on deep learning is provided for the application;

[0058] Fig. 3 An improved FasterR-CNN model structure diagram is provided for the application. DETAILED DESCRIPTION

[0059] The application will now be further described in detail in conjunction with the drawings. These drawings are all simplified schematic diagrams, and only illustrate the basic structure of the application in a schematic manner, and therefore only show the components related to the application.

[0060] REFERENCE Figs. 1-3 A Chinese herbal medicine seedling growth state intelligent identification method based on deep learning, comprising the following steps:

[0061] S1, acquire original image data of a Chinese herbal medicine seedling area;

[0062] S2, an improved FasterR-CNN Chinese herbal medicine recognition model is constructed, and the model uses MobileNetV3 as a backbone network for feature extraction;

[0063] S3, an anchor-free region proposal mechanism is introduced on a multi-scale feature map to perform foreground classification judgment and boundary box offset value prediction;

[0064] S4, a RoIAlign algorithm is used to align the features of the candidate regions, and a candidate region feature vector set is output;

[0065] S5, the candidate region feature vector is input into a subnetwork, a DropBlock regularization method is introduced to suppress overfitting, and a seedling detection frame and a growth state recognition result set are generated;

[0066] S6, an improved FasterR-CNN model is constructed based on cross-entropy and smooth L1 loss function joint supervision training, the improved FasterR-CNN model is used as a teacher network, and a lightweight student network is trained through a knowledge distillation mechanism;

[0067] S7, real-time seedling images collected on site are input into the improved FasterR-CNN model after training and compression, and corresponding Chinese herbal medicine seedling growth state recognition results are output.

[0068] The application provides an intelligent recognition method for the growth state of Chinese herbal medicine seedlings during the seedling stage based on deep learning, an improved FasterR-CNN model with MobileNetV3 as a backbone network is constructed, an anchor-free region proposal mechanism is introduced in a multi-scale feature map to realize efficient extraction of candidate regions of Chinese herbal medicine seedlings, a RoIAlign algorithm is used to align the features of the candidate regions to improve the feature expression accuracy, DropBlock regularization is introduced in the detection subnetwork to suppress overfitting and enhance the model generalization ability, a joint supervision optimization strategy is constructed based on cross-entropy and smooth L1 loss function during the training process, the improved FasterR-CNN model after training is used as a teacher network, a lightweight student network is trained through a knowledge distillation mechanism to realize model compression and deployment optimization, and finally high-precision seedling detection and growth state recognition can be realized on real-time seedling images, and the intelligent and efficient recognition requirements in the Chinese herbal medicine seedling process are met.

[0069] In the embodiment, the original image data is periodically acquired by the acquisition device deployed in the seedling raising area, and the acquisition period is automatically adjusted according to the seedling raising stage. The acquisition period is once every 5 minutes in the germination stage, once every 10 minutes in the rapid growth stage, and once every 30 minutes in the stable stage. Enhancement processing is performed on the original image data, including brightness equalization, resolution normalization and noise suppression, to generate a standardized image data set, which is divided into a training set and a test set.

[0070] In the embodiment, the feature extraction is a multi-layer convolution and activation operation on the standardized image data, which extracts texture information, structural boundaries and spatial semantic features in the medicinal material seedling image, and outputs a multi-scale feature map.

[0071] In the embodiment, the S3 specifically includes:

[0072] S31, introducing an anchor-free region proposal mechanism on the multi-scale feature map output by the MobileNetV3 backbone network, which means performing foreground classification judgment and boundary box offset value prediction on each pixel in the multi-scale feature map;

[0073] S32, the foreground classification judgment is a binary classification prediction for each pixel position, outputting a foreground confidence score for identifying the area where the medicinal material seedling body exists in the image;

[0074] S33, the boundary box offset value prediction is to output the distance offset values of the pixel position to the upper, lower, left and right four boundaries of the target boundary box for each pixel position, respectively denoted as upper offset, lower offset, left offset and right offset. Taking the pixel point as the center, the corresponding rectangular candidate box position is determined by combining the four direction offset values obtained by prediction, and an initial candidate region set is generated;

[0075] S34, the initial candidate region set is sorted according to the foreground confidence score from high to low, and the candidate regions with foreground confidence score higher than a set threshold and ranking in the top N are selected, wherein N is a preset maximum candidate box upper limit, so that the part of the initial candidate region exceeding the image boundary is limited within the image size range;

[0076] S35, filtering the candidate boxes with too small area in the candidate region set, removing the detection boxes with size lower than a set threshold, and retaining the effective candidate regions;

[0077] S36, performing a non-maximum suppression operation on the retained candidate regions, removing the repeated regions according to the overlapping condition between the candidate regions, and outputting the candidate region set.

[0078] The application introduces an anchor-free region proposal mechanism on the multi-scale feature map output by the MobileNetV3 backbone network, generates an initial candidate region set by performing foreground classification judgment and boundary box offset value prediction on each pixel position; wherein the foreground classification is used to output the foreground confidence of each pixel point, and the boundary box offset value prediction calculates the distance from the pixel to the upper, lower, left and right four boundaries of the target boundary box, so as to determine the candidate box position with the pixel as the center. Then, the initial candidate regions are sorted according to the foreground confidence, and the effective candidate boxes with high confidence and located in the preset candidate box number range are screened out, and the boundary coordinate constraint is performed on the candidate boxes exceeding the image boundary; further, the candidate regions with too small area are removed, and finally the overlapping candidate regions are removed through non-maximum suppression to output a candidate region set with low redundancy and accurate positioning, laying a high-quality foundation for subsequent feature alignment and growth state recognition.

[0079] In the embodiment, the S4 specifically comprises:

[0080] S41, mapping the candidate region coordinates from the original image space to the multi-scale feature map space according to the down-sampling ratio of the MobileNetV3 backbone network to obtain the candidate box coordinates under the feature map scale;

[0081] S42, performing the following RoIAlign operation on each mapped candidate box:

[0082] Divide the corresponding candidate box region in the feature map into a fixed number of uniform grid units to form a KxK sampling grid structure;

[0083] In each grid, four fixed sampling points are selected, and the feature values of each sampling point are calculated by using the bilinear interpolation method according to the continuous positions of the sampling points in the feature map;

[0084] Performing an average operation on the four sampling point feature values of each grid unit to generate the final feature representation inside the grid and obtain a uniform size candidate region feature vector;

[0085] S43, performing the RoIAlign operation on all candidate regions to form a set of candidate region feature vector sets with consistent size;

[0086] The application accurately locates the position of the candidate box at the feature map scale by mapping the candidate region coordinates from the original image space to the multi-scale feature map space according to the down-sampling ratio of the MobileNetV3 backbone network; then, the RoIAlign algorithm is used to perform feature alignment operation on each candidate box region, specifically including dividing the corresponding region in the feature map into a fixed number of uniform grids, selecting four sampling points in each grid, calculating the feature values of the sampling points by using the bilinear interpolation method, and averaging the four sampling point feature values to generate a uniform size candidate region feature vector. Through this process, consistent feature alignment is performed on all candidate regions, and finally a set of uniform size and consistent structure candidate region feature vectors is formed, providing standardized input for subsequent classification and regression prediction.

[0087] In the embodiment, the S5 specifically includes:

[0088] S51, sequentially input the candidate region feature vector into the subnetwork containing two full connection structures, the first full connection layer outputs the intermediate expression vector and connects the ReLU activation function, and the second full connection layer outputs the prediction results of the classification branch and the regression branch in parallel;

[0089] S52, the classification branch takes the candidate region feature vector output in the S43 step as input, sequentially passes through two full connection neural networks for feature transformation, accesses the Softmax normalization function, and outputs the probability distribution vector of the Chinese herbal medicine seedling body under each growth state category. Let the vector be p = [p0, p1, p2, p3, p4], where p i represents the predicted probability value of the current candidate region belonging to the i-th growth state. According to the index corresponding to the maximum probability in the probability vector, the growth state classification label of the current seedling body is determined, and the maximum probability value is taken as the confidence score of the classification result.

[0090] In order to realize accurate identification of different physiological stages in the cultivation process of Chinese herbal medicines, the growth state categories are pre-set as five categories, which are: category 0 is the germination period, indicating that the seed just starts to germinate, the leaf has not unfolded, the plant area in the image is small and the color is light; category 1 is the early expansion period of seedling, indicating that the cotyledon and the primary true leaf begin to appear, and the structure outline gradually becomes clear; category 2 is the rapid growth period, indicating that the seedling body enters the high growth rate stage, the leaf density and green saturation rapidly increase; category 3 is the stable period, indicating that the plant morphology and color tend to be stable, the structure is regular, but the growth rate slows down; and category 4 is the abnormal state, indicating that there are abnormal growth characteristics such as yellowing, wilting, insect damage, etc., and the image often shows color abnormalities, morphological distortion or edge damage, etc.

[0091] S53, the regression branch takes the candidate region feature vector as input, sequentially passes through two layers of fully connected network and an output layer, and respectively predicts the boundary correction amount of the candidate frame in the upper boundary, lower boundary, left boundary and right boundary. The boundary correction amount is a coordinate offset value relative to the initial candidate frame boundary position, indicating the adjustment amplitude of the current candidate frame boundary position in the vertical direction and the horizontal direction.

[0092] In the correction process, the upper boundary coordinate of the initial candidate frame is added to the upper boundary correction amount to obtain the corrected upper boundary coordinate; the lower boundary coordinate of the initial candidate frame is added to the lower boundary correction amount to obtain the corrected lower boundary coordinate; the left boundary coordinate of the initial candidate frame is added to the left boundary correction amount to obtain the corrected left boundary coordinate; and the right boundary coordinate of the initial candidate frame is added to the right boundary correction amount to obtain the corrected right boundary coordinate. The four direction corrected boundary coordinates jointly constitute a new rectangular detection frame, which is used for further positioning the real boundary position of the Chinese herbal medicine seedling body.

[0093] S54, set the mask block size of DropBlock, define the continuous span of each mask region in the channel dimension; according to the reserved probability and the number of channels of the candidate region feature vector, calculate the number of continuous channel blocks that need to be shielded; then according to the calculation result, generate multiple starting positions in the channel dimension, and construct DropBlock shielding mask in block size units, the elements corresponding to the shielding block position in the DropBlock shielding mask are set to zero, and the non-shielding region is set to one; the DropBlock shielding mask is applied to the candidate region feature vector, and the corresponding positions are multiplied element by element, thereby shielding the selected continuous channel region, and constructing the intermediate feature representation after DropBlock.

[0094] S55, fuse the growth state category output by the classification branch and the bounding box correction result output by the regression branch to generate the final seedling detection frame and growth state recognition result set.

[0095] The present application realizes the growth state recognition and boundary fine positioning of Chinese herbal medicine seedlings by constructing a subnetwork containing two layers of fully connected structure to deeply decode the candidate region feature vector, relying on the collaborative output of the classification branch and the regression branch. The classification branch accesses the Softmax function to output the probability distribution of each growth state category, combines the maximum probability value to determine the growth state classification label and confidence score of the seedling, and takes five growth states as the recognition target, covering key stages such as germination, initial expansion, growth period, stability and abnormality. The regression branch predicts the boundary correction amount of the candidate frame in the upper, lower, left and right four directions, and obtains more accurate detection boundary through coordinate adjustment. At the same time, the DropBlock regularization mechanism is introduced to randomly shield the continuous region in the feature channel dimension, enhance the generalization ability of the model, and finally fuse the classification result and the boundary correction amount to generate an accurate seedling detection frame and growth state recognition result set.

[0096] In this embodiment, S6 specifically includes:

[0097] S61, the growth state category probability distribution output by the candidate region feature vector in the classification branch is supervised trained by using a cross-entropy loss function, and the real growth state category label is set as y i , and the probability distribution of the predicted growth state category is The cross-entropy loss function L cls is:

[0098]

[0099] where C is the number of growth state categories, is the probability value of the i-th category output by Softmax, y i is the real category label in one-hot encoding form;

[0100] S62, the boundary offset value output by the candidate region feature vector in the regression branch is supervised trained by using a smooth L1 loss function, and the loss function L reg is:

[0101]

[0102] where represents the boundary offset value predicted by the model, δ j is the real boundary offset, j corresponds to the upper t, lower b, left l and right r of the target boundary respectively, and the smooth L1 loss is defined as:

[0103]

[0104] S63, a total loss function is constructed as a joint optimization target of the model:

[0105] L total = L cls + λL reg ;

[0106] where λ is a weight coefficient of the regression loss term, which regulates the loss balance relationship between classification and regression training;

[0107] S64, the improved FasterR-CNN model trained is used as a teacher network, a lightweight student network is constructed as a target model, the structure parameters of the student network are simplified, and the main network part is kept as a shallow separable convolution structure;

[0108] S65, the student network is trained by using a knowledge distillation mechanism, a distillation loss function is introduced to jointly supervise the student network and the teacher network in two aspects of classification output and regression output, and the distillation loss function is defined as:

[0109] L distill = a * L soft + b * L feat ;

[0110] wherein L soft is the KL divergence loss based on temperature smoothing of Softmax output, used to align the prediction of the class probability distribution of the teacher network

[0111] and the student network; L feat is the Euclidean distance loss of the student network and the teacher network at the intermediate candidate region feature vector level, used to align the intermediate layer representation feature; a and b are the weight coefficients of the two distillation loss items, respectively.

[0112] S66, during the entire distillation training process, the teacher network is used as a supervision signal source to guide the student network to gradually approach the performance of the teacher network in classification accuracy and regression accuracy, so that the student network can maintain the accuracy and robustness of the identification of the growth state of the Chinese herbal medicine seedling while reducing the model size and the computational complexity.

[0113] In the model training stage, the joint supervision and knowledge distillation strategy is adopted to enhance the accuracy and deployment efficiency of the identification model. By introducing the cross-entropy loss function and the smooth L1 loss function into the output of the candidate region feature vector in the classification branch and the regression branch, respectively, the simultaneous optimization of the growth state identification and the boundary positioning accuracy is realized, and the total loss function is constructed to coordinate the training weight relationship between the classification and the regression task. On this basis, the improved FasterR-CNN model trained is used as the teacher network, a lightweight student network is constructed, the shallow separable convolution design in the main structure is retained, and the knowledge distillation mechanism is used for training. In the distillation process, the KL divergence loss based on the class distribution and the Euclidean distance loss of the intermediate feature layer are introduced, which are used to align the output prediction and the deep representation, respectively, to ensure that the student network maintains the efficient identification ability and robustness of the growth state of the Chinese herbal medicine seedling while compressing the parameter size and the computational complexity.

[0114] In the embodiment, the identification result includes normal growth, slow growth, leaf lesion, wilting and drying, and corresponding confidence, which is used to drive the seedling management platform to perform automatic state display and abnormal alarm operation.

[0115] In the embodiment, the specific steps of the improved FasterR-CNN model include:

[0116] Take MobileNetV3 as the backbone network, perform layer-by-layer convolution and activation on the input standardized image data, and output multi-scale feature maps; introduce an anchor-free region proposal mechanism on the multi-scale feature maps, take pixels as the prediction center, and perform foreground classification judgment and boundary box offset value prediction respectively to form an initial candidate region set;

[0117] Sort the initial candidate region set by confidence and correct the boundary, and output the candidate region set; map the coordinates of the screened candidate regions from the original image space to the multi-scale feature map space according to the MobileNetV3 downsampling ratio, perform RoIAlign operation, and obtain a candidate region feature vector set;

[0118] Input the candidate region feature vectors into the subnetwork containing two fully connected structures in turn, the classification branch outputs the probability distribution of each growth state class through the Softmax function, and determines the class label and confidence according to the maximum probability; the regression branch outputs the boundary offset in the up, down, left and right directions of the candidate box, and DropBlock regularization is introduced between the two fully connected layers;

[0119] Based on the classification output and regression output of the candidate region, the cross-entropy loss function is used to supervise the training of the growth state class, and the smooth L1 loss function is used to supervise the training of the boundary offset value, and a joint loss function is constructed for overall optimization;

[0120] The improved FasterR-CNN model trained above is used as a teacher network to construct a chemical network, and a distillation loss function is introduced to jointly supervise the training of the student network in the classification output and regression output.

[0121] The present application proposes an improved FasterR-CNN-based intelligent identification method for the growth state of Chinese herbal medicine seedlings, which uses MobileNetV3 as the backbone network to extract multi-scale features from the input image, and introduces an anchor-free region proposal mechanism on the feature map to generate a candidate region set through pixel-level prediction. After combining the confidence sorting and boundary correction of the candidate region, the RoIAlign algorithm is used to extract the candidate region feature vector of the same scale, and is input into the classification branch and the regression branch for growth state identification and boundary adjustment, and DropBlock regularization is introduced in the network to enhance the generalization ability of the model. In the training stage, a joint loss function is constructed to simultaneously supervise the classification and regression tasks, and through the knowledge distillation mechanism, the improved FasterR-CNN model trained is used as a teacher network to guide the lightweight student network to approximate the teacher performance in the classification output and regression output, realizing model compression while maintaining recognition accuracy.

[0122] Example 1:

[0123] To verify the feasibility of the present application in implementation, the present application is applied to the Chinese herbal medicine seedling monitoring task of a certain intelligent agricultural breeding center, and the goal is to continuously intelligently identify the growth state of Chinese herbal medicine seedlings in the seedling stage and give an abnormal state warning. The selected seedling crop in the experiment is a certain common Chinese herbal medicine, which has problems such as low recognition efficiency and state change reaction lag. Traditional methods mostly rely on fixed time interval shooting and manual state annotation, which is low in efficiency and strong in subjectivity, and is difficult to adapt to the rapid monitoring needs in large-scale planting scenarios.

[0124] In the actual deployment process, multiple groups of image acquisition devices are set in the seedling area, and the image acquisition resolution is uniformly set to 1280x720. The acquisition frequency is set to 5 minutes / time (germination period), 10 minutes / time (rapid growth period) and 30 minutes / time (stable period) according to the seedling stage. After the system receives the image, the original image is first standardized and preprocessed, including brightness equalization, resolution unification and noise removal. The processed image is input into the improved FasterR-CNN recognition model.

[0125] The recognition model uses MobileNetV3 as the backbone network to perform feature extraction, combines the anchor-free region proposal mechanism to generate initial candidate boxes, and performs feature alignment operation through RoIAlign. Then the model inputs the candidate regions into the double-branch fully connected subnetwork containing the DropBlock structure, and outputs the growth state recognition label and detection box position coordinates of the seedlings. The final recognition result is directly fed back to the seedling management platform to complete intelligent annotation and warning.

[0126] The experimental period is 45 days, and more than 48,000 Chinese herbal medicine seedling images are collected and processed. The system assists the seedling management personnel in adjusting the seedling plan and performing abnormal intervention operations through the recognition results of the growth state label. By comparing with the manual annotation results, the recognition accuracy index (Accuracy), average detection precision (mAP) and model reasoning time delay are calculated, and the results are shown in Table 1:

[0127] Table 1 Performance evaluation data of Chinese herbal medicine seedling period intelligent recognition system

[0128]

[0129] It can be seen from the statistics of the above experimental results that the seedling growth state recognition model based on deep learning has the following remarkable effects: first, by introducing the anchor-free mechanism, the generation accuracy of the candidate region is effectively improved, and the detection deviation caused by unreasonable anchor frame design is avoided; second, by using MobileNetV3 as the main backbone network and DropBlock regularization, the lightness and robustness of the model are significantly enhanced; in addition, by training the student network through the knowledge distillation mechanism, the model compression is realized while maintaining the recognition accuracy, and the calculation resource consumption is effectively reduced.

[0130] Through comprehensive analysis, after the recognition system of the application is deployed, the seedling growth state recognition efficiency of the seedling management platform is improved by about 4 times, the recognition accuracy is improved to more than 96%, and the artificial patrol video rate and the misjudgment risk are greatly reduced, which has extremely high practical value and popularization potential in the actual traditional Chinese medicine planting and seedling raising scene.

[0131] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A deep learning-based intelligent identification method for growth state of Chinese herbal medicine seedlings in the seedling stage, characterized in that, The method comprises the following steps: S1, acquiring original image data of a Chinese herbal medicine seedling area; S2, constructing an improved FasterR-CNN model, MobileNetV3 as a backbone network for feature extraction; S3, introducing an anchor-free region proposal mechanism on a multi-scale feature map, and jointly outputting a candidate region set through foreground classification judgment and boundary box offset value prediction; S4, using the RoIAlign algorithm to align the features of the candidate regions, and outputting a candidate region feature vector set; S5, inputting the candidate region feature vector into a subnetwork, introducing a DropBlock regularization method to suppress overfitting, generating a seedling detection frame and a growth state recognition result set; S6, constructing a joint supervision training of the improved FasterR-CNN model based on cross-entropy and smooth L1 loss functions, using the improved FasterR-CNN model as a teacher network, and training a lightweight student network through a knowledge distillation mechanism; S7, inputting real-time seedling images collected on site into the trained improved FasterR-CNN model, and outputting corresponding Chinese herbal medicine seedling growth state recognition results.

2. The intelligent identification method of the growth state of traditional Chinese medicinal materials in the seedling stage based on deep learning according to claim 1, characterized in that, The original image data is periodically acquired by a collection device deployed in the seedling area; the original image data is subjected to enhancement processing, including brightness equalization, resolution normalization and noise suppression, to generate a standardized image data set. 3.The intelligent identification method of growth state of Chinese medicinal material in seedling stage based on deep learning according to claim 1, characterized in that, The feature extraction is a multi-layer convolution and activation operation on the standardized image data, which extracts texture information, structural boundaries and spatial semantic features in the Chinese herbal seedling image, and outputs a multi-scale feature map.

4. The intelligent identification method of the growth state of traditional Chinese medicinal materials in the seedling stage based on deep learning according to claim 1, characterized in that, The S3 specifically comprises: S31, introducing an anchor-free region proposal mechanism on the multi-scale feature map output by the MobileNetV3 backbone network, which performs foreground classification judgment and boundary box offset value prediction on each pixel on the multi-scale feature map; S32, foreground classification judgment is a binary classification prediction for each pixel position, which outputs a foreground confidence score for identifying the region of the Chinese herbal seedling in the image; S33, boundary box offset value prediction is to output distance offset values of the pixel position to the upper, lower, left and right four boundaries of the target boundary box for each pixel position; taking the pixel point as the center, the four direction offset values are combined to determine the corresponding rectangular candidate frame position, and an initial candidate region set is generated; S34, sorting the initial candidate region set according to the foreground confidence score from high to low, selecting the candidate regions with foreground confidence scores higher than a set threshold and ranking in the top N, wherein N is a preset maximum candidate frame upper limit, so that the part of the initial candidate region exceeding the image boundary is limited within the image size range; S35, filtering the candidate frames with too small area in the candidate region set, removing the detection frames with size lower than a set threshold, and retaining the effective candidate regions; S36, performing a non-maximum suppression operation on the reserved candidate regions, removing repeated regions according to the overlap between the candidate regions, and outputting a candidate region set.

5. The intelligent identification method of the growth state of traditional Chinese medicinal materials in the seedling stage based on deep learning according to claim 1, characterized in that, The S4 specifically comprises: S41, down-sampling the candidate regions according to a down-sampling ratio of a MobileNetV3 backbone network, the down-sampling ratio being a reduction multiple between a multi-scale feature map size and an original image size, mapping from an original image space to a multi-scale feature map space to obtain a candidate box at a feature map scale; S42, performing feature alignment on each mapped candidate box, and performing the following RoIAlign algorithm operation: Dividing the candidate box region in the feature map into a fixed number of uniform grid units to form a KxK sampling grid structure; In each grid, four fixed sampling points are selected, and the feature values of each sampling point are calculated by bilinear interpolation according to the continuous positions of the sampling points in the feature map; Performing an average operation on the four sampling point feature values of each grid unit to generate a final feature representation inside the grid to obtain a uniform size candidate region feature vector; S43, performing the RoIAlign algorithm operation on all candidate regions to form a set of uniform size candidate region feature vectors. 6.The deep learning-based intelligent identification method for growth state of Chinese medicinal material seedlings in the seedling stage according to claim 1, characterized by, The S5 specifically comprises: S51, sequentially inputting the candidate region feature vectors into a subnetwork comprising two fully connected structures, the first fully connected layer outputting an intermediate expression vector and connecting a ReLU activation function, and the second fully connected layer outputting prediction results of a classification branch and a regression branch in parallel; S52, the classification branch taking the candidate region feature vector as input, sequentially passing through a fully connected layer and a Softmax normalization function, outputting a probability distribution of each growth state category, taking the category index corresponding to the maximum probability value as the growth state classification label of the seedling, and taking the maximum probability value as the corresponding confidence score; S53, the regression branch is used for predicting the bounding box correction of the candidate region feature vector, and outputs the boundary offset value in the up, down, left and right four directions relative to the initial candidate box; S54, introducing DropBlock regularization between the two fully connected layers, randomly shielding the continuous channel region in the candidate region feature vector according to the set retention probability, and inhibiting overfitting; S55, fusing the growth state category output by the classification branch and the bounding box correction result output by the regression branch to generate a set of final seedling detection boxes and growth state recognition results.

7. The intelligent identification method of the growth state of traditional Chinese medicinal materials in the seedling stage based on deep learning according to claim 1, characterized in that, The S6 specifically comprises: S61, using a cross-entropy loss function to supervise the training of the growth state category probability distribution output by the classification branch in the candidate region feature vector; S62, using a smooth L1 loss function to supervise the training of the boundary offset value output by the regression branch in the candidate region feature vector; S63, constructing a total loss function as a joint optimization target of the model, for simultaneously optimizing the growth state recognition ability of the classification branch and the boundary prediction accuracy of the regression branch; S64, taking the trained improved FasterR-CNN model as a teacher network, constructing a lightweight student network as a target model, simplifying the structure parameters of the student network, and keeping the backbone network part as a shallow separable convolution structure; S65, the student network is trained by using a knowledge distillation mechanism, and a distillation loss function is introduced to jointly supervise the student network and the teacher network in two aspects of classification output and regression output. 8.The deep learning-based intelligent identification method for growth state of Chinese medicinal material seedlings in the seedling stage according to claim 1, characterized by, The improved FasterR-CNN model comprises the following specific steps: MobileNetV3 is used as the backbone network to perform layer-by-layer convolution and activation on the input standardized image data, and output multi-scale feature maps; an anchor-free region proposal mechanism is introduced on the multi-scale feature maps, taking pixels as the prediction center to perform foreground classification judgment and boundary box offset value prediction respectively, and an initial candidate region set is formed; The initial candidate region set is sorted by confidence and modified by boundary constraint, and a candidate region set is output; the coordinates of the screened candidate regions are mapped from the original image space to the multi-scale feature map space according to the down-sampling ratio of MobileNetV3, and the RoIAlign operation is performed to obtain a candidate region feature vector set; The candidate region feature vectors are sequentially input into a subnetwork comprising two fully connected structures, the classification branch outputs the probability distribution of each growth state category through the Softmax function, and determines the category label and confidence according to the maximum probability; the regression branch outputs the boundary offset in the up, down, left and right directions of the candidate box, and DropBlock regularization is introduced between the two fully connected layers; Based on the classification output and regression output of the candidate region, the growth state category is supervised and trained by using a cross-entropy loss function, the boundary offset value is supervised and trained by using a smooth L1 loss function, and a joint loss function is constructed for overall optimization; The improved FasterR-CNN model trained above is used as the teacher network to construct a student network, and a distillation loss function is introduced to jointly supervise and train the student network in two aspects of classification output and regression output. 9.The deep learning-based intelligent identification method for growth state of Chinese medicinal material seedlings in the seedling stage according to claim 1, characterized by, The recognition result comprises normal growth, slow growth, leaf lesion, wilting and drying, and corresponding confidence, and is used to drive the seedling management platform to perform automatic state display and abnormal alarm operation.

Images