A recognition monitoring system for tea net bugs in tea tree planting

Abstract

The present application relates to a kind of tea tree planting in the identification monitoring system of tea net bug. It is fused by adopting the main network with the first feature fusion network, second feature fusion network of VGG16 network structure, forms bidirectional feature fusion structure, while fusing shallow feature to deep feature, deep feature is reversely fused to shallow feature, to ensure that the spatial information and detail information of shallow feature and deep feature can be fully expressed, to avoid the problem that tea net bug cannot be quickly identified and accurately judged in heat map due to the small size of tea net bug, aggregation overlap and the influence of its excrement.

Description

[0001] This application is a divisional application of application number 202010967812.9, entitled "Network structure for rapid counting of tea net bugs based on heat map". TECHNICAL FIELD

[0002] The present application relates to the technical field of tea garden pest control, in particular to a recognition and monitoring system for tea net bugs in tea tree planting. BACKGROUND

[0003] In the process of tea tree planting, tea net bugs are one of the pests that need to be prevented and controlled in the spring to affect the yield and quality of high-quality green tea. Tea net bugs, nymphs, like to cluster on the back of tea leaves to suck juice, causing the affected leaves to have white small spots that can accumulate into patches, and a large amount of black sticky excretion of tea net bugs is left on the back of the leaves, which seriously affects the photosynthesis of the leaves; when the damage is serious, the tea leaves fall off, the tree vigor is weakened, the tea buds germinate slowly and are small or germination is stagnant, affecting yield and quality. Tea net bugs occur as early as mid-March, coinciding with the picking period of spring high-quality green tea. However, because tea net bugs and nymphs are densely aggregated on the back of the leaves, the leaves are often overlapped, which greatly affects the accuracy and precision of their identification and counting; at the same time, the black sticky excretion is easy to cause misjudgment in identification and counting by workers, further affecting the speed and accuracy of identification and counting, slowing down the prevention and control of tea net bugs and missing the best opportunity to prevent and control tea net bugs, resulting in a decrease in tea yield and affecting tea quality. Therefore, it is necessary to effectively identify and monitor the occurrence trend of tea net bugs and nymphs, so as to take timely and targeted prevention and control measures. SUMMARY

[0004] The purpose of the present application is to provide a recognition and monitoring system for tea net bugs in tea tree planting, which can quickly and accurately identify and count tea net bugs on the back of the leaves, has high counting efficiency and high accuracy, and can effectively carry out identification and monitoring of tea net bugs and nymphs.

[0005] The purpose of the present application is achieved by the following technical solutions:

[0006] The application discloses a recognition and monitoring system for tea net bugs in tea tree planting, and is characterized by comprising a main network, a first feature fusion network and a second feature fusion network; wherein the main network comprises convolutional layers in a VGG16 network structure; the first feature fusion network is to extract a pooling layer of each layer from a second convolutional layer of the main network, then gradually perform feature fusion on the uppermost pooling layer and the next pooling layer through a MergeTB module, and input the uppermost pooling layer into the next layer, and finally perform feature fusion on the lowermost feature of the first feature fusion network and the lowermost feature of the main network; the second feature fusion network is to extract an up-sampling layer of the lowermost convolutional layer to the second convolutional layer of the main network, then gradually perform feature fusion on the lowermost up-sampling module and the previous up-sampling layer through a MergeBT module, and input the lowermost up-sampling module into the previous layer in the form of the up-sampling module, wherein the up-sampling module is obtained through an UpSample module, and finally perform feature fusion on the pooling layer of the uppermost up-sampling module of the second feature fusion network and the lowermost feature of the main network; the main network, the first feature fusion network and the second feature fusion network are fused through an AttentionFuse module; the up-sampling layer adopts a mode of filling corresponding position output values and zero values in non-corresponding positions; that is, if data with a sampling width of N and a height of M are up-sampled by A and B times, respectively, data with a width of NA and a height of MB are obtained; wherein is a corresponding position in an original image (N, M), and all the remaining points excluding the corresponding position of the point in the original image are filling positions.

[0007] Further optimization is made that the main network comprises convolutional layers in six-layer VGG16 network structures of C1-C6.

[0008] Further optimization is made that an input layer of the recognition and monitoring system is a heat map of the tea net bug, and a training process of the heat map adopts a network structure of MCNN for training and evaluation.

[0009] In the heat map calculation process, the volume of tea bugs is small relative to the tea leaves, and is distributed on the back of the leaves. The deep network loses useful spatial information and position information of the tea bugs. Even for the tea bugs with small volume, the deep network features may ignore the information of the tea bugs due to excessive pooling operations, which greatly affects the description of the heat map of the tea bugs. Therefore, the present application fuses the main network and the first feature fusion network, and fuses the shallow features into the deep features, so as to supplement and enrich the spatial information and detailed information of the deep features, and avoid ignoring the information of the heat map of the tea bugs. At the same time, the first feature fusion network is fused with the main network to form a bidirectional feature fusion structure, so that the deep features are reversely fused into the shallow features while the shallow features are fused into the deep features, so as to avoid the influence of the small volume of the tea bugs, the overlapping phenomenon and the black mucous excretion of the tea bugs on the accuracy and the slow recognition counting.

[0010] Further optimization, the MergeTB module is specifically to perform convolution, normalization processing on the features of the previous layer pooling layer, then perform matrix point multiplication with itself (i.e. the previous layer pooling layer), then perform matrix addition with the current layer pooling layer and normalization processing, and finally output to the next layer; the convolution layer after convolution processing is C 7a , and C 7a is a 3*3 convolution layer, and the input and output dimensions are equal; the UpSample module is to perform convolution on the up-sampling layer to obtain a convolution layer C 14 , wherein C 14 is a 3*3 convolution layer, which is used for filtering and smoothing after the up-sampling layer; the MergeBT module is specifically to perform convolution, normalization processing on the features of the next layer up-sampling module, then perform matrix point multiplication with itself (i.e. the next layer up-sampling module), then perform matrix addition with the current layer up-sampling module and normalization processing, and finally output to the previous layer; the convolution layer after convolution processing is C 7b ; and C 7b is a 3*3 convolution layer, and the input and output dimensions are equal; the AttentionFuse module is specifically to perform convolution on the features output by the main network, the first feature fusion network and the second feature fusion network respectively, sequentially form convolution layers C8, C9, then perform feature fusion, then perform convolution, sequentially form three layers of convolution layers C 10 , C 11 , C 12 , and then pass through C 12The data block with an output channel of 3 generated by the convolution layer is respectively subjected to matrix point multiplication with the C9 convolution layer formed after the convolution of the backbone network, the first feature fusion network and the second feature fusion network, and finally subjected to feature fusion and convolution to obtain a convolution layer C 13 , and the output is determined.

[0011] Further optimization is performed, the C8 is a 3*3 convolution layer, the input and output channel numbers are equal; the C9 is a 1*1 convolution layer, the output channel number is 1; the C 10 is a 1*1*128 convolution layer, the C 11 is a 3*3*128 convolution layer; the C 12 is a 1*1*3 convolution layer, the output channel number is 3; and the C 13 is a 3*3 convolution layer.

[0012] The present application has the following beneficial effects:

[0013] The present application provides a recognition and monitoring system for tea net bugs in tea tree planting, which adopts a backbone network with a VGG16 network structure and a first feature fusion network and a second feature fusion network to form a bidirectional feature fusion structure, fuses shallow features to deep features and reversely fuses deep features to shallow features, so that the spatial information and detail information of the shallow features and the deep features can be fully expressed, thereby avoiding the problem that the tea net bugs cannot be quickly identified and accurately judged due to the small size of the tea net bugs, the overlapping aggregation and the influence of the excrement of the tea net bugs, and making the counting error less than ±2%; and ensuring that the tea net bugs in tea tree planting can be timely, quickly and accurately prevented and controlled, that is, the recognition and monitoring of the tea net bugs and nymphs are realized. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 FIG. 1 is a structural schematic diagram of the recognition and monitoring system for tea net bugs in tea tree planting in the embodiment of the present application.

[0015] Figure 2 FIG. 3 is a structural schematic diagram of the MergeTB module in the embodiment of the present application.

[0016] Figure 3 FIG. 4 is a structural schematic diagram of the MergeBT module in the embodiment of the present application.

[0017] Figure 4 FIG. 5 is a structural schematic diagram of the AttentionFuse module in the embodiment of the present application.

[0018] Figure 5 FIG. 6 is a structural schematic diagram of the UpSample module in the embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions provided by the present application will be clearly and completely described below through examples.

[0020] Example 1

[0021] As Figures 1 to 5 shown, a recognition and monitoring system for tea net bugs in tea tree planting has an input layer of a heat map of tea net bugs (i.e. Figure 1 The Input Image shown in the figure is a heat map), and a training process of the heat map is trained and evaluated using the network structure of MCNN;

[0022] It includes a main network and a first feature fusion network and a second feature fusion network; wherein the main network includes six convolutional layers in the VGG16 network structure, i.e. Figure 1 C1-C6 convolutional layers shown in the figure; the first feature fusion network extracts the pooling layer of each layer starting from the second convolutional layer of the main network (e.g. Figure 1 extracts the pooling layer of C2-C6 layers of the main network), then gradually performs feature fusion on the uppermost pooling layer and the next pooling layer through the MergeTB module, and inputs to the next layer, and finally the lowermost feature of the first feature fusion network is fused with the lowermost feature of the main network; the MergeTB module specifically performs convolution and normalization processing on the feature of the previous pooling layer, then performs matrix point multiplication with itself (i.e. 7a the previous pooling layer), then performs matrix addition with the pooling layer of the current layer and normalization processing, and finally outputs to the next layer; the convolutional layer after the convolutional processing is C 7a , and C 7a is a 3*3 convolutional layer, and the input and output dimensions are equal. As Figure 1 and Figure 2 shown, taking the C3 convolutional layer as an example, the pooling layer Pooling1 of the C2 convolutional layer is formed into a 3*3 convolutional layer C 7a after convolution, and C 7a is normalized (BN) processed and then matrix point multiplied with the pooling layer Pooling1 of the C2 convolutional layer, then the matrix point multiplication result is matrix added with the pooling layer of the C3 convolutional layer, and finally the added result is normalized (BN) processed to obtain the pooling layer Pooling2, which is input to the next layer.

[0023] The second feature fusion network extracts the up-sampling layer of the lowermost convolutional layer to the second convolutional layer of the main network, then gradually performs feature fusion on the lowermost up-sampling module and the previous up-sampling layer through the MergeBT module, and inputs to the previous layer in the form of an up-sampling module, which is obtained through the UpSample module, and finally the pooling layer of the uppermost up-sampling module of the second feature fusion network is fused with the lowermost feature of the main network. The UpSample module is a convolutional layer C14 wherein C 14 is a 3*3 convolution layer for filtering and smoothing after the up-sampling layer; the up-sampling layer adopts a mode of filling in values of corresponding positions and zero values in non-corresponding positions, that is, if the data of a sampling width of N and a height of M are up-sampled by A and B times respectively, then data of a width of NA and a height of MB are obtained; wherein is the corresponding position of (N, M) in the original image, and the remaining points after excluding the corresponding positions of all points in the original image are filling positions. The MergeBT module specifically performs convolution, normalization processing on the features of the next layer up-sampling module, then performs matrix point multiplication with itself (i.e. the next layer up-sampling module), then performs matrix addition with the current layer up-sampling module and normalization processing, and finally outputs to the previous layer; the convolution layer after the convolution processing is C 7b ; and C 7b is a 3*3 convolution layer with equal input and output dimensions. As shown in Figure 1 and Figure 3 , taking the C5 convolution layer as an example, first, the C6 convolution layer below the C5 convolution layer is up-sampled and convolved through the UpSample1 module, and the output of the convolution layer C 14 is obtained, then the convolution layer C 14 is convolved to obtain the 3*3 convolution layer C 7b , C 7b is normalized (BN) processed and then matrix-multiplied with the convolution layer C 14 , then the matrix-multiplication result is matrix-added with the up-sampling layer of the C5 convolution layer, and finally the added result is normalized (BN) processed and convolved through the UpSample2 module to input to the next layer.

[0024] Finally, the backbone network and the first feature fusion network and the second feature fusion network are fused through the AttentionFuse module. As shown in Figure 4 , the AttentionFuse module specifically performs twice convolution on the features output by the backbone network, the first feature fusion network and the second feature fusion network respectively, forms the convolution layers of C8 and C9 in turn, then performs feature fusion, and then performs thrice convolution, forms three convolution layers of C 10 , C 11 and C 12 in turn, then generates a data block (three white rectangular blocks shown in Figure 4 ) with an output channel of 3 through the C 12 convolution layer, and performs matrix point multiplication with the C9 convolution layer formed after convolution of the backbone network, the first feature fusion network and the second feature fusion network, and finally performs feature fusion and convolution to obtain the convolution layer C 13, the output is determined. Among them, C8 is a 3*3 convolution layer, the input and output channel numbers are equal; C9 is a 1*1 convolution layer, and the output channel number is 1; C 10 is a 1*1*128 convolution layer, C 11 is a 3*3*128 convolution layer; C 12 is a 1*1*3 convolution layer, and the output channel number is 3; C 13 is a 3*3 convolution layer.

Claims

1. A system for identifying and monitoring tea lace bugs in tea plant cultivation, characterized in that: The system comprises a backbone network, a first feature fusion network, and a second feature fusion network. The backbone network includes convolutional layers from the VGG16 network structure. The first feature fusion network extracts pooling layers from each layer starting from the second convolutional layer of the backbone network. Then, using the MergeTB module, it progressively fuses the features of the topmost pooling layer with the next pooling layer and inputs them into the next layer. Finally, the bottommost features of the first feature fusion network are fused with the bottommost features of the backbone network. The second feature fusion network extracts upsampling layers from the bottommost convolutional layer to the second convolutional layer of the backbone network, and then uses the MergeBT module to progressively upsample the bottommost layers. The module performs feature fusion with the previous upsampling layer and inputs it to the previous layer through the upsampling module. The upsampling module is obtained through the UpSample module. Finally, the pooling layer of the topmost upsampling module of the second feature fusion network fuses features with the bottommost features of the backbone network. The backbone network is fused with the first feature fusion network and the second feature fusion network through the AttentionFuse module. The upsampling layer fills with the output value at the corresponding position and fills with zero value at the non-corresponding position. That is, if the sampled data has a width of N and a height of M, and is upsampled by A and B times respectively, then the data with a width of NA and a height of MB is obtained; where (NA+ MB+ () represents the corresponding position (N, M) in the original image. All other points, excluding the corresponding positions of all points in the original image, are filled positions. The input layer of the identification and monitoring system is a heatmap of the tea lace bug. The training process of the heatmap uses an MCNN network structure for training and evaluation. The MergeTB module specifically performs convolution and normalization on the features of the previous pooling layer, then performs matrix multiplication with itself (i.e., the previous pooling layer), then performs matrix addition and normalization with the current pooling layer, and finally outputs to the next layer. The convolutional layer after convolution processing is C... 7a And C 7a It is a 3x3 convolutional layer with equal input and output dimensions; the UpSample module is to convolve the upsampling layer to obtain the convolutional layer C. 14 C 14 A 3x3 convolutional layer is used for filtering and smoothing after the upsampling layer. Specifically, the MergeBT module convolves and normalizes the features from the next upsampling module, then performs a matrix multiplication with itself (the next upsampling module), followed by matrix addition and normalization with the current upsampling module, and finally outputs to the previous layer. The convolutional layer after the convolutional processing is C. 7b And C 7b The system consists of 3x3 convolutional layers with equal input and output dimensions. Specifically, the AttentionFuse module convolves the features output from the backbone network, the first feature fusion network, and the second feature fusion network, forming convolutional layers C8 and C9 respectively. After feature fusion, further convolutions are performed, resulting in C... 10 C 11 C 12 Three convolutional layers, then through C 12 The data blocks with 3 output channels generated by the convolutional layer are multiplied by matrix dot products after being convolved with the backbone network, the first feature fusion network, and the second feature fusion network to form a C9 convolutional layer. Finally, feature fusion and convolution are performed to obtain the convolutional layer C. 13 The output is used for judgment.

2. The tea lace bug identification and monitoring system in tea tree cultivation as described in claim 1, characterized in that: The backbone network includes convolutional layers in a six-layer VGG16 network structure, C1 to C6.

3. The tea lace bug identification and monitoring system in tea tree cultivation as described in claim 1 or 2, characterized in that: C8 is a 3x3 convolutional layer with equal input and output channels; C9 is a 1x1 convolutional layer with 1 output channel; C... 10 It is a 1*1*128 convolutional layer, the C 11 It is a 3*3*128 convolutional layer; the C 12 It is a 1*1*3 convolutional layer with 3 output channels; the C 13 It is a 3x3 convolutional layer.

Images