Image positioning identification model and tea tender stem segmentation positioning method

Abstract

The invention provides an image positioning identification model and a tea tender stalk segmentation positioning method, the image positioning identification model comprises a backbone network, a neck network and a head network, the backbone network is used for obtaining initial feature information of an input picture, the backbone network comprises a multipath cavity space pyramid pooling module, and the head network is used for obtaining the initial feature information of the input picture. The multi-path void spatial pyramid pooling module is arranged at the output end of initial feature information of the backbone network, and is used for constructing convolution kernels of different receptive fields through different voidage after obtaining feature information of an efficient aggregation layer network ELAN, obtaining multi-scale object information, and simultaneously adopting fixed block pooling operation to obtain multi-scale object information; the invention further provides a tea tender stem segmentation and positioning method, the tea tender stem segmentation and positioning method can more accurately segment and position the tea tender stems, and the tea tender stem segmentation and positioning method comprises the following steps: extracting the multi-scale object information, extracting the significant feature information, fusing the multi-scale object information and the significant feature information to obtain the significant feature information enhanced initial feature information, and outputting the significant feature information enhanced initial feature information to the neck network.

Description

Technical Field

[0001] The invention relates to the technical field of image recognition, in particular to an image positioning and recognition model based on yolov7, and also to a method for segmenting and positioning young tea stems by using the image positioning and recognition model based on yolov7. Background Art

[0002] As the total output value of dry tea increases year by year, the tea planting area continues to expand in order to meet the market demand; however, the tea picking standards are high, and manual picking has disadvantages such as low efficiency, high cost, and insufficient labor. A method of locating the picking point through image recognition and using mechanical equipment for picking has emerged on the market.

[0003] In a complex natural environment, the region of interest containing young buds may contain interfering information such as old leaves, field ridges, and tea tree branches. In addition, the young stems of tea leaves are thin and have unclear texture features. Traditional image processing methods often perform feature extraction based on a specific color space.

[0004] The team from Zhejiang University of Technology selected tea pictures in RGB color space as the research object. They first used the watershed function to segment the tea leaves, then binarized them to obtain the lowest point of the skeleton contour, and then used the Shi-Tomasi algorithm to obtain the picking point of the tea buds. In the existing literature, the article "Research on the Identification and Positioning Method of Tea Buds by Tea Picking Manipulators" extracted the gb component to grayscale the bud image, used the improved watershed algorithm for segmentation, and finally used the intersection of the minimum matrix and the bud as the picking point. The article "Research on Visual Recognition of Tea Buds and Location of Picking Points" designed super green features (2*Gbr) for segmentation, and used edge detection and skeletonization methods to locate picking points. The article "Research on Identification of High-end and Famous Green Tea Buds and Bionic Picking Finger" performed segmentation based on the two rules of (RB)>80 and (GB)>80 and -1.01*G+0.8*B+0.19*R>99. The article "Research on Identification of Tea Buds Based on k-means Clustering" used k-means clustering method to select a and b categories in the Lab color model to segment the bud part. The article "Research on Tea Bud Image Segmentation Method Based on SLIC Superpixels" used super red, cg, and gb three components to synthesize the color map, and used SLIC superpixel segmentation method to segment the image.

[0005] However, most traditional image processing methods are not very robust and often need to be carried out in a stable experimental environment, requiring relatively high stability of the external environment. In the actual picking process, traditional image processing methods have high limitations.

[0006] In order to more accurately locate the picking point of tender shoots in a natural environment, various image recognition deep learning models have appeared on the market, such as Mask R-CNN (Mask Region-based Convolutional Neural Network) deep learning algorithm, Deeplabv3+, Yolov5m, Yolov7, Yolov8l, etc. The Yolov7 model is one of the commonly used image recognition deep learning models, which is mainly divided into three parts: backbone network Backbone, neck network Neck and head network Head. After being processed by the three major network structures, the final segmentation and positioning results are output.

[0007] However, in the natural environment, the tea tree branches and leaves are lush and overlapping, and the tender buds often block each other. In addition, the color and texture of the old buds and the tender buds are similar, which causes certain difficulties for the segmentation and positioning task. As a result, the existing Yolov7 model has a high missed detection rate and false detection rate when segmenting and positioning the tea bud stems. Therefore, it is necessary to improve the Yolov7 model to achieve more accurate segmentation and positioning of the tea bud stems. Summary of the invention

[0008] In order to solve the problems of thin tender stalks, unclear texture features and difficult positioning in the intelligent tea picking process under complex background in the prior art, the present invention provides an image positioning and recognition model based on YOLOv7, and also provides a tea stalk segmentation and positioning method implemented by the image positioning and recognition model based on YOLOv7, which utilizes the learning ability of the YOLOv7-seg model for big data and optimizes it to more accurately segment the tender stalks.

[0009] The present invention provides an image positioning and recognition model based on yolov7, including a backbone network, a neck network and a head network, wherein:

[0010] Backbone network: used to obtain the initial feature information of the input image, the backbone network includes a multi-path void space pyramid pooling module, the multi-path void space pyramid pooling module is arranged at the output end of the initial feature information of the backbone network, and is used to obtain the feature information of the efficient aggregation layer network ELAN, and then construct convolution kernels with different receptive fields through different void rates to obtain multi-scale object information, and at the same time adopt a fixed block pooling operation to extract significant feature information, and then fuse the multi-scale object information and the significant feature information to obtain the initial feature information with enhanced significant feature information, and output it to the neck network;

[0011] Neck network: set at the output end of the backbone network, used to establish connections between feature maps at different levels to achieve multi-level feature fusion;

[0012] Head network: It is set at the output end of the neck network, and is used to output different results in a cascade manner, and screen out the target area with the highest accuracy through the non-maximum suppression algorithm, and output the final segmentation and positioning result.

[0013] Furthermore, the backbone network also includes an attention mechanism module, the backbone network is provided with N levels of efficient aggregation layer networks ELAN compressed by maximum pooling, N is a positive integer greater than 1, the multi-path void space pyramid pooling module is arranged at the output end of the Nth level of efficient aggregation layer network ELAN, the output ends of the 1st to N-1th level efficient aggregation layer networks ELAN are respectively provided with N-1 output channels corresponding to the neck network step-by-step upsampling feature splicing, the attention mechanism module is arranged on the output channel, and is used to adjust the parameter weights of the segmentation target, and perform channel feature enhancement operations on the feature maps output by the output channels

[0014] Furthermore, the processing method of the attention mechanism module is:

[0015] (1) For the input feature In∈R H*W*C , the output feature is O out ∈R H*W*C , where H is the number of pixels in the vertical direction of the feature map R, W is the number of pixels in the horizontal direction of the feature map R, and C is the number of channels. The input feature In is regarded as a channel combination In = {in0, in1, in2, ..., in C-1}, use the global maximum pooling function to converge the feature map to one point, compress the spatial information, and generate a vector q∈R 1*1*C ;

[0016] q=MAX(in k (i,j))(0<k<C,0<i<H,0<j<W)

[0017] Among them, k (i, j) is the spatial feature in the input feature, k is the variable of the channel, i is the variable of the vertical pixel, and j is the variable of the horizontal pixel;

[0018] (2) compress the vector q and perform ReLU activation function mapping operation, and then restore it to its original length;

[0019] (3) Obtaining the adjusted feature weights limited to the interval [0,1], the calculation formula of the feature weights is:

[0020] α=δ(F(q))

[0021] Among them, F(·) represents compression and expansion as well as Relu function activation operations, δ(·) represents the activation process of sigmoid function, and α represents the spatial feature weight vector;

[0022] (4) Take the spatial feature weight vector α as the excitation and apply it to the input feature In to calculate the new output feature O out =α*In,(In∈R H*W*C ,α∈R 1*1*C )

[0023] Furthermore, the multi-path dilated spatial pyramid pooling module includes a first 1*1 convolution component, more than one dilated convolution components with different expansion rates, and a plurality of pooling pyramid components, wherein the dilated convolution component is used to obtain feature information of different receptive fields, and the pooling pyramid component is used to filter a large number of invalid repeated features in the convolutional neural network and extract feature information with more obvious distinguishing effect;

[0024] The multi-path void space pyramid pooling module also includes a feature splicing component and a second 1*1 convolution component, wherein the first 1*1 convolution component, more than one void convolution component with different expansion rates, and a plurality of pooling pyramid components are arranged in parallel, and their input ends respectively receive the same initial feature information, and their output ends are connected to the input end of the feature splicing component, and the output end of the feature splicing component is connected to the input end of the second 1*1 convolution component, and after being processed by the second 1*1 convolution component, it is output to the neck network.

[0025] The present invention also provides a method for segmenting and locating young tea stems using the image positioning and recognition model based on yolov7, comprising the following steps:

[0026] S1: Take a certain number of tea buds pictures to obtain a tea buds picture dataset;

[0027] S2: Divide the tea bud image dataset into a training set, a test set and a validation set according to a certain ratio, so as to train the image positioning and recognition model based on yolov7 and form a tea bud image positioning and recognition model;

[0028] S3: Input a tea bud picture, call the trained tea bud picture positioning and recognition model to segment and locate the tea bud picture, and output a final segmentation and positioning result.

[0029] Further, in step S3, the method of calling the trained tea bud image positioning and recognition model to segment and locate the tea bud image is:

[0030] S301: The backbone network processes the tea bud picture using convolution operations of different kernel sizes and pooling functions to obtain feature information of an efficient aggregation layer network ELAN of the input tea bud picture;

[0031] S302: Input the acquired feature information to a multi-path hole space pyramid pooling module;

[0032] S303: The multi-path void space pyramid pooling module constructs convolution kernels with different receptive fields through different void rates to obtain multi-scale object information, and adopts a fixed block pooling operation to extract significant feature information, and then fuses the multi-scale object information and the significant feature information to obtain initial feature information with enhanced significant feature information, and outputs it to the neck network;

[0033] S304: The neck network is used to establish connections between feature maps at different levels to achieve multi-level feature fusion, and then output the output results of several layers of efficient aggregation layer networks after the multi-level feature fusion to the head network respectively;

[0034] S305: The head network outputs different results in a cascade manner, and selects the target area with the highest accuracy through a non-maximum suppression algorithm, and outputs the final segmentation and positioning result.

[0035] Furthermore, an attention mechanism module is set in the backbone network, the backbone network is provided with N levels of efficient aggregation layer networks ELAN compressed by maximum pooling, N is a positive integer greater than 1, the multi-path void space pyramid pooling module is arranged at the output end of the Nth level of efficient aggregation layer network ELAN, the output ends of the 1st to N-1th level efficient aggregation layer networks ELAN are respectively provided with N-1 output channels corresponding to the features of the neck network up-sampled step by step, and the attention mechanism module is arranged on the output channel.

[0036] While steps S302 and S303 are being executed, the step of adjusting the parameter weights of the segmentation target using the attention mechanism module is also included, and a channel feature enhancement operation is performed on the feature map output by the output channel.

[0037] Furthermore, the maximum pooling compressed efficient aggregation layer network ELAN has three levels, the attention mechanism module includes a first attention mechanism module and a second attention mechanism module, the first attention mechanism module is arranged at the output end of the first-level efficient aggregation layer network ELAN, and the second attention mechanism module is arranged at the output end of the second-level efficient aggregation layer network ELAN.

[0038] The neck network is provided with a two-level upsampling feature splicing component, the first-level upsampling unit samples the initial feature information output by the multi-path void space pyramid pooling module, and performs feature splicing with the information output by the second attention mechanism module through the feature splicing unit, and outputs it to the first-level efficient aggregation layer network ELAN-H, the second-level upsampling unit samples the feature information output by the first-level efficient aggregation layer network ELAN-H, and performs feature splicing with the information output by the first attention mechanism module through the feature splicing unit, and outputs it to the second-level efficient aggregation layer network ELAN-H;

[0039] The information output by the second-level efficient aggregation layer network ELAN-H is processed by the feature processing component of the neck network, and then outputs multi-level feature information to the head network.

[0040] In step S1, when taking the picture of the tea buds, the camera is set above the tea tree, the height range between the camera and the top layer of the tea tree canopy is 20 to 40 centimeters, and the angle between the camera's shooting angle and the horizontal plane is between 45° and 60°.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: by replacing the MASPP in the backbone network of the yolov7 model with a multi-path void space pyramid pooling module, the observation field of the model can be broadened to obtain multi-scale tender shoot feature information, filter a large number of invalid repeated features in the convolutional neural network, extract feature information with more obvious distinguishing effect, and finally enhance the expression ability of the model by stacking and fusing these features, improve the accuracy of feature recognition, and thus improve the stability of the application.

[0042] By setting an attention mechanism module in the backbone network, the attention mechanism is used to automatically increase the parameter weight of the segmentation target, thereby increasing the model's attention to the segmentation target and thus improving the model's ability to extract detailed features.

[0043] By training the image positioning and recognition model of the present invention and applying it to the segmentation and positioning of tea leaves, compared with the existing image recognition models, the present invention has a stronger feature extraction capability, can adapt to more complex natural light environments, and more accurately segment and locate tea leaves. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the solutions in the present application or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1It is a structural diagram of the existing yolov7 model;

[0046] Figure 2 It is a structural schematic diagram of the image positioning and recognition model based on yolov7 of the present invention;

[0047] Figure 3 This is a schematic diagram of the working principle of an embodiment of a multi-path hole space pyramid pooling module of the present invention;

[0048] Figure 4 This is a schematic diagram of the working principle of the attention mechanism module of the present invention;

[0049] Figure 5 It is a schematic diagram for comparing the accuracy indexes of the yolov7 model and the image positioning recognition model of the present invention;

[0050] Figure 6 It is a schematic diagram comparing the recall rate indicators of the yolov7 model and the image positioning and recognition model of the present invention.

[0051] Figure 7 and Figure 8 They are respectively display diagrams of the segmentation and positioning results of tender tea stems according to the present invention. DETAILED DESCRIPTION

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field of this application; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit the application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0053] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0055] 1. Model Optimization

[0056] like Figure 1As shown in the figure, the yolov7-seg model is mainly divided into three parts: backbone network, neck network and head network. The backbone network intensively uses convolution operations of different kernel sizes and pooling functions to obtain the initial features of the input image, and creatively proposes an efficient layer aggregation network ELAN in the yolov7-seg model. This structure repeatedly uses residual structure, multi-path merging and other methods to enhance the feature extraction ability of the network without destroying the depth of the original network. The neck network uses the path aggregation feature pyramid FPN structure. The FPN structure combines the multi-resolution scale prediction of SSD and the multi-resolution feature fusion network structure of U-Net, and realizes multi-level feature fusion by establishing connections between feature maps at different levels. The head network outputs different results in a cascade manner, and the target area with the highest accuracy is screened out through non-maximum suppression NMS, and the final segmentation and positioning results are output.

[0057] like Figure 2 The present invention optimizes the yolov7-seg model based on the fact that it performs poorly in segmentation and positioning of tender tea stems and has certain difficulties. The specific optimization points are:

[0058] (1) The multi-path atrous spatial pyramid pooling module MASPP is used to replace SPPCSPC to broaden the model's observation field of view, obtain multi-scale sprout stem feature information, filter out a large number of invalid repeated features in the convolutional neural network, and extract feature information with more obvious distinguishing effect. Finally, these features are stacked and fused to enhance the model's expressiveness.

[0059] (2) The CBS module on the backbone network is replaced by the attention mechanism module SE, and the attention mechanism is used to automatically increase the parameter weight of the segmentation target, thereby increasing the model's attention to the segmentation target and improving the model's ability to extract detailed features.

[0060] Specifically, the image positioning and recognition model based on yolov7 of the present invention also includes a backbone network, a neck network and a head network, wherein:

[0061] Backbone network: used to obtain the initial feature information of the input image, the backbone network includes a multi-path void space pyramid pooling module, the multi-path void space pyramid pooling module is arranged at the output end of the initial feature information of the backbone network, and is used to obtain the feature information of the efficient aggregation layer network ELAN, and then construct convolution kernels with different receptive fields through different void rates to obtain multi-scale object information, and at the same time adopt a fixed block pooling operation to extract significant feature information, and then fuse the multi-scale object information and the significant feature information to obtain the initial feature information with enhanced significant feature information, and output it to the neck network;

[0062] Neck network: set at the output end of the backbone network, used to establish connections between feature maps at different levels to achieve multi-level feature fusion;

[0063] Head network: It is set at the output end of the neck network, and is used to output different results in a cascade manner, and screen out the target area with the highest accuracy through the non-maximum suppression algorithm, and output the final segmentation and positioning result.

[0064] Among them, Input is input, Output is output, CBS is convolution + standardization + activation function, ELAN is efficient aggregation layer network, MP is Maxpool (maximum pooling) + CBS, Cat is feature splicing module, Up is upsampling module, SE is attention mechanism module, REP is convolution + BN module, Conv is convolution module.

[0065] like Figure 3 As shown, the multi-path void space pyramid pooling module of the present invention has multiple parallel void convolutions with different sampling rates, and the features extracted for each sampling rate are further processed in a separate branch. The module constructs convolution kernels with different receptive fields through different void rates to obtain multi-scale object information. At the same time, in order to effectively avoid problems such as image distortion caused by cropping and scaling operations of the image area, and to solve the problem of repeated feature extraction in convolutional neural networks, multiple fixed-block pooling operations are used to extract relatively significant feature information for inputs of different sizes. In addition, the fusion of features of different sizes has a stronger ability to extract features such as details and color of the bud stem.

[0066] As an embodiment of the present invention, the multi-path void space pyramid pooling module of the present invention includes a first 1*1 convolution component, three void convolution components with different expansion rates (the expansion rates are 5, 9, and 13, respectively), and four pooling pyramid components (the pooling kernel sizes use global pooling functions of 1, 3, 9, and 19, respectively). The void convolution component is used to obtain feature information of different receptive fields, and the pooling pyramid component is used to filter a large number of invalid repeated features in the convolutional neural network, and extract feature information with more obvious distinguishing effect.

[0067] The multi-path void space pyramid pooling module of this example also includes a feature splicing component and a second 1*1 convolution component, wherein the first 1*1 convolution component, three void convolution components with different expansion rates, and four pooling pyramid components are arranged in parallel, and their input ends respectively receive the same initial feature information, and their output ends are connected to the input end of the feature splicing component, and the output end of the feature splicing component is connected to the input end of the second 1*1 convolution component. The expression ability of the model is enhanced by stacking and fusing these features, and then output to the neck network of the subsequent stage.

[0068] like Figure 4 As shown, in the present invention, an attention mechanism module SE is set in the output channel of the backbone network, which can enhance the channel features of the input feature map, and the final SE module output does not change the size of the input feature map.

[0069] The backbone network is provided with N levels of efficient aggregation layer networks ELAN compressed by maximum pooling, N is a positive integer greater than 1, the multi-path void space pyramid pooling module is arranged at the output end of the Nth level of efficient aggregation layer network ELAN, the output ends of the 1st to N-1th level efficient aggregation layer networks ELAN are respectively provided with N-1 output channels for feature splicing corresponding to the step-by-step upsampling of the neck network, the attention mechanism module is arranged on the output channel, and is used to adjust the parameter weights of the segmentation target and perform channel feature enhancement operations on the feature maps output by the output channels.

[0070] The attention mechanism module and upsampling in this example are both two-level. The processing method of the attention mechanism module in this example is:

[0071] (1) For the input feature In∈R H*W*C , the output feature is O out ∈R H*W*C , where H is the number of pixels in the vertical direction of the feature map R, W is the number of pixels in the horizontal direction of the feature map R, and C is the number of channels. The input feature In is regarded as a channel combination In = {in0, in1, in2, ..., in C-1}, use the global maximum pooling function to converge the feature map to one point, compress the spatial information, and generate a vector q∈R 1*1*C ;

[0072] q=MAX(in k (i,j))(0<k<C,0<i<H,0<j<W)

[0073] Among them, k (i, j) is the spatial feature in the input feature, k is the variable of the channel, i is the variable of the vertical pixel, and j is the variable of the horizontal pixel;

[0074] (2) In order to obtain the recalibrated feature gain, the present invention compresses the vector q and performs a ReLU activation function mapping operation, and then restores it to its original length;

[0075] (3) The sigmoid function is used to obtain the adjusted feature weights limited to the interval [0, 1]. The calculation formula of the feature weights is:

[0076] α=δ(F(q))

[0077] Among them, F(·) represents compression and expansion as well as Relu function activation operations, δ(·) represents the activation process of sigmoid function, and α represents the spatial feature weight vector;

[0078] (3) Take the spatial feature weight vector α as the excitation and apply it to the input feature In to calculate the new output feature O out =α*In,(In∈R H*W*C ,α∈R 1*1*C ),Scale is the size of the output feature map.

[0079] 2. Model Training

[0080] The image positioning recognition model of the present invention is trained and applied to the segmentation and positioning of young tea stems. The method for segmenting and positioning young tea stems implemented by the image positioning recognition model based on yolov7 of the present invention comprises the following steps:

[0081] S1: Take a certain number of tea buds pictures to obtain a tea buds picture dataset;

[0082] S2: Divide the tea bud image dataset into a training set, a test set and a validation set according to a certain ratio, so as to train the image positioning and recognition model based on yolov7 and form a tea bud image positioning and recognition model;

[0083] S3: Input a tea bud picture, call the trained tea bud picture positioning and recognition model to segment and locate the tea bud picture, and output a final segmentation and positioning result.

[0084] The following are examples to illustrate each step.

[0085] In this example, a Nikon D5300 camera was used under natural light conditions. It was placed above the tea tree, and the height from the top of the tea tree canopy was maintained at 20 to 40 cm. The angle between the camera and the horizontal plane was maintained at approximately 45° to 60°.

[0086] Pictures of young buds of Meitan 601 Mingqian tea were taken, and 1217 images were obtained after screening and cropping. A tea bud picture dataset was established, and the dataset was divided into training set, test set and validation set in a ratio of 6:2:2. The training set was then used to train the image positioning and recognition model of the present invention. After the training was completed, the test set and validation set were used for testing and verification, respectively.

[0087] 3. Experimental Verification

[0088] 1. Comparison between the yolov7 model and the model of the present invention

[0089] During the testing and verification process, the model evaluation indicators used include precision (Precision, referred to as P), mean average precision (mAP), and recall (Recall, referred to as R). The calculation method of each indicator is:

[0090]

[0091] Among them, TP represents the number of positive samples recognized by the model, FP represents the number of negative samples recognized, FN represents the number of positive samples misrecognized, AP is the area under the PR curve, and the PR curve is composed of the P vertical axis and the R horizontal axis. The larger the PR area, the higher the AP value, which means the detection model is more accurate. AP(n) is the AP value of each category, mAP is the mean AP of all categories, and m represents the number of categories.

[0092] The existing yolov7 model (experiment number 1) is compared with the improved yolov7 model (experiment number 2-4), and the test results of each index are shown in Table 1.

[0093] Experiment No. SE MASPP P R MAP@0.5 1 0.871 0.895 0.916 2 √ 0.920 0.901 0.909 3 √ 0.923 0.900 0.924 4 √ √ 0.932 0.909 0.935

[0094] Table 1 Comparison table of various index tests for experimental numbers 1-4

[0095] The comparison of indicators in the training process between Experiment No. 1 (Yolov7 model) and Experiment No. 4 (model of the present invention) is as follows: Figure 5 and Figure 6 As shown, the blue curve is the training process of the model of the present invention, and the red curve is the training process of the yolov7 model. It can be clearly observed that with the increase in the number of training times, when the model tends to be stable, the improved yolov7-seg model including SE and MASPP modules has higher precision and recall values ​​than the original yolov7-seg model, indicating that the improved yolov7-seg model is better than the original yolov7-seg model in extracting features such as bud stem details and texture.

[0096] 2. Comparison between the model of the present invention and other models

[0097] The model of the present invention is compared with the mainstream image visual recognition model on the market, and the test results of various indicators are shown in Table 2.

[0098] Model P R MAP@0.5 maskRcnn - - 0.498 Deeplabv3+ 0.8825 0.857 - Yolov5m-seg 0.889 0.868 0.914 Yolov7-seg 0.871 0.895 0.916 Yolov8l-seg 0.886 0.884 0.896 Model of the present invention 0.932 0.909 0.935

[0099] Table 2 Comparison table of various model index tests

[0100] It can be observed from Table 2 that the model of the present invention is superior to the existing models such as Yolov8l-seg, Yolov5m-seg, Deeplabv3+, maskRcnn, Yolov7-seg in terms of evaluation indicators such as P, R, and MAP@0.5, indicating that the improved model has a stronger feature extraction capability, can adapt to more complex natural light environments, and can more accurately segment and locate the tea bud stems.

[0101] 3. Positioning and segmentation effect display

[0102] The present invention inputs a picture of tea buds, calls a trained image positioning and recognition model to segment and locate the picture of tea buds, and outputs the effect diagram as shown in FIG. Figure 7 and Figure 8 As shown, the red marked part represents the segmentation and positioning result of the model for the tender shoot rod, d is the identified category, the present invention represents the tender shoot rod category, and the number after d represents the confidence rate of correct segmentation. It can be seen from the effect diagram that the present invention can accurately identify the tender shoot rod.

[0103] The specific implementation modes described above are preferred implementation modes of the present invention, and are not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to the specific implementation modes, and all equivalent changes made according to the present invention are within the protection scope of the present invention.

Claims

1. An image positioning and recognition model based on yolov7, characterized in that: It includes backbone network, neck network and head network, among which, Backbone network: used to obtain the initial feature information of the input image, the backbone network includes a multi-path void space pyramid pooling module, the multi-path void space pyramid pooling module is arranged at the output end of the initial feature information of the backbone network, and is used to obtain the feature information of the efficient aggregation layer network ELAN, and then construct convolution kernels with different receptive fields through different void rates to obtain multi-scale object information, and at the same time adopt a fixed block pooling operation to extract significant feature information, and then fuse the multi-scale object information and the significant feature information to obtain the initial feature information with enhanced significant feature information, and output it to the neck network; Neck network: set at the output end of the backbone network, used to establish connections between feature maps at different levels to achieve multi-level feature fusion; Head network: It is set at the output end of the neck network, and is used to output different results in a cascade manner, and screen out the target area with the highest accuracy through the non-maximum suppression algorithm, and output the final segmentation and positioning result.

2. The image positioning recognition model based on yolov7 according to claim 1 is characterized in that: The backbone network also includes an attention mechanism module. The backbone network is provided with N levels of efficient aggregation layer networks ELAN compressed by maximum pooling, where N is a positive integer greater than 1. The multi-path void space pyramid pooling module is arranged at the output end of the Nth level of efficient aggregation layer network ELAN. The output ends of the 1st to N-1th level efficient aggregation layer networks ELAN are respectively provided with N-1 output channels for feature splicing corresponding to the step-by-step upsampling of the neck network. The attention mechanism module is arranged on the output channel to adjust the parameter weights of the segmentation target and perform channel feature enhancement operations on the feature maps output by the output channels.

3. The image positioning recognition model based on yolov7 according to claim 2 is characterized in that: The processing method of the attention mechanism module is: (1) For the input feature In∈R H*W*C , the output feature is O out ∈R H*W*C The feature map of R, where H is the number of pixels in the vertical direction of the feature map R, W is the number of pixels in the horizontal direction of the feature map R, and C is the number of channels. The input feature In is regarded as a channel combination In = {in0, in1, in2, ..., in C-1 }, use the global maximum pooling function to converge the feature map to one point, compress the spatial information, and generate a vector q∈R 1*1*C ; q=MAX(in k (i,j))(0<k<C,0<i<H,0<j<W) Among them, k (i, j) is the spatial feature in the input feature, k is the variable of the channel, i is the variable of the vertical pixel, and j is the variable of the horizontal pixel; (2) compress the vector q and perform ReLU activation function mapping operation, and then restore it to its original length; (3) Obtaining the adjusted feature weights limited to the interval [0,1], the calculation formula of the feature weights is: α=δ(F(q)) Among them, F(·) represents compression and expansion as well as Relu function activation operations, δ(·) represents the activation process of sigmoid function, and α represents the spatial feature weight vector; (4) Take the spatial feature weight vector α as the excitation and apply it to the input feature In to calculate the new output feature O out =α*In,(In∈R H*W*C ,α∈R 1*1*C ).

4. The image positioning and recognition model based on yolov7 according to any one of claims 1 to 3, characterized in that: The multi-path dilated spatial pyramid pooling module includes a first 1*1 convolution component, more than one dilated convolution components with different expansion rates, and a plurality of pooling pyramid components, wherein the dilated convolution component is used to obtain feature information of different receptive fields, and the pooling pyramid component is used to filter a large number of invalid repeated features in the convolutional neural network and extract feature information with more obvious distinguishing effect; The multi-path void space pyramid pooling module also includes a feature splicing component and a second 1*1 convolution component, wherein the first 1*1 convolution component, more than one void convolution component with different expansion rates, and a plurality of pooling pyramid components are arranged in parallel, and their input ends respectively receive the same initial feature information, and their output ends are connected to the input end of the feature splicing component, and the output end of the feature splicing component is connected to the input end of the second 1*1 convolution component, and after being processed by the second 1*1 convolution component, it is output to the neck network.

5. A method for segmenting and locating young tea stems, implemented by using the image positioning and recognition model based on YOLOv7 according to any one of claims 1 to 4, characterized in that: The steps include: S1: Take a certain number of tea buds pictures to obtain a tea buds picture dataset; S2: Divide the tea bud image dataset into a training set, a test set and a validation set according to a certain ratio, so as to train the image positioning and recognition model based on yolov7 and form a tea bud image positioning and recognition model; S3: Input a tea bud picture, call the trained tea bud picture positioning and recognition model to segment and locate the tea bud picture, and output a final segmentation and positioning result.

6. The method for segmenting and positioning young tea stems according to claim 5, characterized in that: In step S3, the method of calling the trained tea bud image positioning and recognition model to segment and locate the tea bud image is: S301: The backbone network processes the tea bud picture using convolution operations of different kernel sizes and pooling functions to obtain feature information of an efficient aggregation layer network ELAN of the input tea bud picture; S302: Input the acquired feature information to a multi-path hole space pyramid pooling module; S303: The multi-path void space pyramid pooling module constructs convolution kernels with different receptive fields through different void rates to obtain multi-scale object information, and adopts a fixed block pooling operation to extract significant feature information, and then fuses the multi-scale object information and the significant feature information to obtain initial feature information with enhanced significant feature information, and outputs it to the neck network; S304: The neck network is used to establish connections between feature maps at different levels to achieve multi-level feature fusion, and then output the output results of several layers of efficient aggregation layer networks after the multi-level feature fusion to the head network respectively; S305: The head network outputs different results in a cascade manner, and selects the target area with the highest accuracy through a non-maximum suppression algorithm, and outputs the final segmentation and positioning result.

7. The method for segmenting and positioning young tea stems according to claim 6, characterized in that: An attention mechanism module is set in the backbone network, the backbone network is provided with N levels of efficient aggregation layer network ELAN compressed by maximum pooling, N is a positive integer greater than 1, the multi-path void space pyramid pooling module is set at the output end of the Nth level of efficient aggregation layer network ELAN, the output ends of the 1st to N-1th level efficient aggregation layer networks ELAN are respectively provided with N-1 output channels corresponding to the neck network level-by-level upsampling feature splicing, the attention mechanism module is set on the output channel, While steps S302 and S303 are being executed, the step of adjusting the parameter weights of the segmentation target using the attention mechanism module is also included, and a channel feature enhancement operation is performed on the feature map output by the output channel.

8. The method for segmenting and positioning young tea stems according to claim 7, characterized in that: The maximum pooling compressed efficient aggregation layer network ELAN has three levels, the attention mechanism module includes a first attention mechanism module and a second attention mechanism module, the first attention mechanism module is arranged at the output end of the first level efficient aggregation layer network ELAN, and the second attention mechanism module is arranged at the output end of the second level efficient aggregation layer network ELAN, The neck network is provided with a two-level upsampling feature splicing component, the first-level upsampling unit samples the initial feature information output by the multi-path void space pyramid pooling module, and performs feature splicing with the information output by the second attention mechanism module through the feature splicing unit, and outputs it to the first-level efficient aggregation layer network ELAN-H, the second-level upsampling unit samples the feature information output by the first-level efficient aggregation layer network ELAN-H, and performs feature splicing with the information output by the first attention mechanism module through the feature splicing unit, and outputs it to the second-level efficient aggregation layer network ELAN-H; The information output by the second-level efficient aggregation layer network ELAN-H is processed by the feature processing component of the neck network, and then outputs multi-level feature information to the head network.

9. The method for segmenting and positioning young tea stems according to any one of claims 6 to 8, characterized in that: In step S1, when taking the picture of the tea buds, the camera is set above the tea tree, the height range between the camera and the top layer of the tea tree canopy is 20 to 40 centimeters, and the angle between the camera's shooting angle and the horizontal plane is between 45° and 60°.

Images