A segmentation detection method for pests and diseases of apple trees

Abstract

The present application relates to the technical field of apple tree pest image processing, and specifically to a pest segmentation detection method for apple trees, the method comprising: obtaining an apple tree pest image; obtaining feature points in the apple tree pest image at various scales; obtaining auxiliary directions according to the gradient changes of each feature point in different directions, screening suspected pest feature points according to the number of auxiliary directions of all feature points; obtaining the pest probability of the suspected pest feature point according to the grayscale change difference and local density of each suspected pest feature point in different auxiliary directions; screening pest feature points according to the pest probability of the suspected pest feature point in the apple tree pest image at all scales, and identifying the location of the pest in the apple tree pest image to be detected according to all pest feature points. The present application can improve the accuracy of pest segmentation detection.

Description

Technical Field

[0001] The present application relates to the technical field of apple tree pest and disease image processing, and in particular to a pest and disease segmentation detection method for apple trees. Background Art

[0002] At present, the more common apple tree diseases and pests are brown spot, leaf spot, mosaic, black spot, bubble spot, etc. Each disease and pest has an early, middle and late stage. At different stages, the position, color and shape of the disease and pest will change. Moreover, the size of the disease and pest in the leaves is very small, and the color of some diseases and pests is similar to the background color.

[0003] Usually, after identifying the feature points in the image, a neural network model is used to detect pests and diseases. However, the targets of apple tree pests and diseases are small and similar to the background, resulting in low detection accuracy of pest and disease feature points, making it difficult for the neural network model to detect pests and diseases, and the pest and disease segmentation detection accuracy is low. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a disease and insect pest segmentation detection method for apple trees to solve the existing problems.

[0005] The present invention discloses a method for segmenting and detecting pests and diseases of apple trees using the following technical solutions:

[0006] An embodiment of the present application provides a method for segmenting and detecting pests and diseases of apple trees, the method comprising the following steps:

[0007] S1, obtain images of apple tree diseases and pests;

[0008] S2, dividing the scales of the apple tree pest and disease images to obtain the apple tree pest and disease images at each scale, and extracting feature points from the apple tree pest and disease images at each scale;

[0009] S3, for the apple tree pest and disease images at each scale, the auxiliary directions are obtained according to the gradient changes of each feature point in different directions, and the suspected pest and disease feature points are screened according to the number of auxiliary directions of all feature points;

[0010] S4, according to the grayscale change difference of each suspected pest feature point in different auxiliary directions, the auxiliary direction discrimination degree of each suspected pest feature point is obtained; according to the distance between different suspected pest feature points, all suspected pest feature points are clustered to obtain the local density of each suspected pest feature point; the pest probability of the suspected pest feature point is obtained by combining the auxiliary direction discrimination degree and local density of the suspected pest feature point;

[0011] S5, screening pest feature points according to the pest probability of the suspected pest feature points in the apple tree pest image at all scales, and identifying the location of the pest in the apple tree pest image to be detected according to all the pest feature points.

[0012] Furthermore, the scales of the apple tree pest images are divided to obtain apple tree pest images at various scales, and feature points in the apple tree pest images at various scales are extracted, including:

[0013] The apple tree pest and disease images are input into a Gaussian pyramid with a preset number of layers to obtain the apple tree pest and disease images at various scales; all local extreme points in the apple tree pest and disease images at various scales are obtained; and each local extreme point is recorded as a feature point.

[0014] Furthermore, the auxiliary directions are obtained according to the gradient changes of each feature point in different directions, and the suspected pest feature points are screened according to the number of auxiliary directions of all feature points, including:

[0015] Obtain the gradient direction of all pixels in the preset neighborhood of each feature point; obtain a preset gradient direction interval; evenly divide the gradient direction interval into a preset number of sub-intervals; for each feature point, divide all pixels in the preset neighborhood of the feature point into each sub-interval according to the gradient direction, and count the number of pixels contained in each sub-interval;

[0016] According to the difference in the number of pixel points contained in each sub-interval, the auxiliary directions of the feature points are obtained; the number of auxiliary directions of each feature point is counted, and the number of auxiliary directions of all feature points is thresholded to obtain the direction segmentation threshold; the pixel points whose auxiliary directions are greater than or equal to the direction segmentation threshold are marked as suspected pest and disease feature points.

[0017] Furthermore, the method for acquiring each auxiliary direction of the feature point includes:

[0018] The sub-interval to which the gradient direction of the feature point belongs is recorded as the main direction interval; the direction probability of each sub-interval is obtained by comparing the number of pixels in the main direction interval with those in other sub-intervals;

[0019] A sub-interval other than the main direction interval whose direction probability is greater than a preset threshold is recorded as an auxiliary direction interval; and a gradient direction of each pixel point contained in the auxiliary direction interval is recorded as an auxiliary direction of the feature point.

[0020] Furthermore, the method for obtaining the direction probability of each sub-interval includes:

[0021] The number of pixels in the main direction interval is counted and recorded as the first number; the ratio of the number of pixels in each sub-interval to the first number is recorded as the direction probability of each sub-interval.

[0022] Furthermore, the method for obtaining the auxiliary direction discrimination of each suspected pest feature point includes:

[0023] Obtaining the gradient vector of each suspected pest feature point; recording the component of the gradient vector in each auxiliary direction as the gradient amplitude in each auxiliary direction;

[0024] For any two auxiliary directions of each suspected pest feature point, calculate the direction difference between the two auxiliary directions; the direction difference is the angle value between the two auxiliary directions; obtain the variance of all the direction differences of each suspected pest feature point, recorded as the first variance; calculate the difference between the gradient amplitudes of the two auxiliary directions, recorded as the gradient amplitude difference, and record the variance of all the gradient amplitude differences of each suspected pest feature point as the second variance;

[0025] Obtain the vector composed of all auxiliary directions of each suspected pest and disease feature point, recorded as the auxiliary direction vector; obtain the sum of the distances between each suspected pest and disease feature point and the auxiliary direction vectors of all other suspected pest and disease feature points, recorded as the local auxiliary direction difference of each suspected pest and disease feature point;

[0026] According to the first variance, second variance and local auxiliary direction difference of each suspected pest and disease feature point, the auxiliary direction discrimination of each suspected pest and disease feature point is obtained.

[0027] Furthermore, the auxiliary direction discrimination of each suspected pest feature point is obtained according to the first variance, the second variance and the local auxiliary direction difference of each suspected pest feature point, including:

[0028] Calculate the product of the first variance and the second variance of each suspected pest and disease feature point; obtain the sum of the product and a preset constant; take the ratio of the local auxiliary direction difference of each suspected pest and disease feature point to the sum as the auxiliary direction discrimination of each suspected pest and disease feature point; the value of the preset constant is 0.01.

[0029] Furthermore, the method for obtaining the local density of each suspected pest feature point includes:

[0030] The rounded value of the preset ratio value of the number of suspected pest and disease feature points is used as the cutoff distance, and the coordinates of all suspected pest and disease feature points are density clustered to obtain the local density of each suspected pest and disease feature point.

[0031] Further, the method of combining the auxiliary direction discrimination and local density of the suspected pest feature point to obtain the pest probability of the suspected pest feature point includes: recording the pest probability of the i-th suspected pest feature point as , ; In the formula, represents the probability of pests and diseases of the i-th suspected pest and disease feature point; c and d are the first preset weight coefficient and the second preset weight coefficient, c=0.7, d=0.3; is the auxiliary direction discrimination of the i-th suspected pest and disease feature point; is the local density of the i-th suspected pest feature point.

[0032] Furthermore, the method of screening the pest feature points according to the pest probability of the suspected pest feature points in the apple tree pest image at all scales, and identifying the location of the pest in the apple tree pest image to be detected according to all the pest feature points, includes:

[0033] The pest and disease probability of all suspected pest and disease feature points in the apple tree pest and disease images at all scales is threshold segmented to obtain the pest and disease feature threshold; the suspected pest and disease feature points whose pest and disease probability is greater than or equal to the pest and disease feature threshold are taken as pest and disease feature points;

[0034] The eight neighborhood areas of the pest feature points are evenly divided into a first preset number of sub-areas, and the SIFT descriptors are substituted to obtain the feature vectors;

[0035] A second preset number of apple tree pest and disease images are obtained, and the locations of the pests and diseases in the apple tree pest and disease images are manually marked, and the images are used as a training set to train the neural network model. During the training process, all the feature vectors are concatenated with the feature vectors output by the convolutional layer in the neural network model to obtain a trained neural network model; the trained neural network model is used to detect the apple tree pest and disease images to be detected to obtain the locations of the pests and diseases.

[0036] This application has at least the following beneficial effects:

[0037] This application aims to solve the problem of low accuracy in pest and disease detection caused by smaller pest and disease targets and pest and disease targets that are relatively similar to the background. First, the texture change characteristics of apple tree spotted leaf disease are analyzed, and auxiliary directions are obtained according to the gradient changes of each feature point in different directions, and suspected pest and disease feature points are screened, thereby eliminating the interference of environmental factors or noise on the identification of pest and disease feature points; further, the auxiliary direction discrimination of each suspected pest and disease feature point is obtained according to the grayscale change difference of each suspected pest and disease feature point in different auxiliary directions, and the discrimination between the pest and disease feature point and the leaf vein texture is improved, so that the identification of the pest and disease feature point is more accurate; the pest and disease probability of the suspected pest and disease feature point is obtained by combining the auxiliary direction discrimination of the suspected pest and disease feature point and the position information of the suspected pest and disease feature point, which is used to characterize the probability of each feature point as a feature point in the pest and disease area formed by spotted leaf disease, and the pest and disease feature points are screened out; finally, the feature vector of each pest and disease feature point is spliced ​​with the feature vector in the neural network to obtain a richer feature representation, thereby improving the accuracy of the neural network model for pest and disease segmentation detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of the steps of a method for segmenting and detecting pests and diseases of apple trees provided in this application;

[0040] Figure 2 A screening block diagram for characteristic points of suspected pests and diseases provided for one embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0043] The following is a detailed description of a specific method for segmenting and detecting pests and diseases of apple trees provided by the present application in conjunction with the accompanying drawings.

[0044] An embodiment of the present application provides a method for segmenting and detecting pests and diseases for apple trees. Specifically, the following method for segmenting and detecting pests and diseases for apple trees is provided. Figure 1 , the method comprises the following steps:

[0045] S1, obtain images of apple tree diseases and pests.

[0046] The initial symptoms of apple leaf spot disease are small brown dots on the leaves, which gradually expand to reddish brown and eventually produce small shiny black spots or rust spots on the fruit surface. In order to prevent and control this disease, it is necessary to monitor the disease at the early stage of the disease to prevent the spread and spread of the disease.

[0047] First, an optical camera is used to photograph apple tree pests and diseases to obtain an image of apple tree pests and diseases, and the image is preprocessed and denoised using a median filter method to eliminate noise mixed in the image, improve image quality and retain the features of the original image. The median filter technology is a well-known technology and the specific process is not described in detail.

[0048] S2, dividing the scales of the apple tree pest and disease images to obtain the apple tree pest and disease images at each scale, and extracting feature points from the apple tree pest and disease images at each scale.

[0049] In the images of apple tree pests and diseases, it is difficult for the neural network model to detect pests and diseases due to the small pest and disease targets and the similarity between pests and diseases and the background. In response to the above problems, this application first constructs the auxiliary direction discrimination and calculates the local density of each feature point, and then combines the auxiliary direction discrimination and local density to construct an indicator that can characterize the probability that a feature point belongs to a pest and disease feature point, and screens out N pest and disease feature points; then, the SIFT descriptor is used for the pest and disease feature points to obtain a feature vector with the same dimension as the feature vector output by the 5th convolutional layer of the ResNet50 network, and the feature vector is fused with the feature vector output by the convolutional layer in the neural network, so that the neural network can learn the characteristics of pests and diseases in a targeted manner, thereby achieving the purpose of accurately detecting pests and diseases.

[0050] First, the apple tree pest image is input into a Gaussian pyramid with a preset number of layers to obtain the apple tree pest image at each scale. Specifically, a Gaussian pyramid is constructed, where the number of layers of the pyramid is 6, and images of each scale are obtained by continuously downsampling and Gaussian blurring the image. Further, all local extreme points in the apple tree pest image at each scale are obtained, and each local extreme point is recorded as a feature point.

[0051] In this embodiment, the local extreme value points are obtained by using the DOG scale space pyramid. Specifically, a DOG scale space pyramid is established, and the DOG pyramid is obtained by subtracting adjacent scale space functions in the Gaussian pyramid. In order to detect the maximum and minimum values ​​of the DOG space, in the DOG scale space, each pixel point in the middle layer except the bottom and top layers needs to be compared with 8 adjacent pixels in the same layer and 9 adjacent pixels in the upper and lower layers, a total of 26 pixels, to ensure that the local extreme value is detected in both the scale space and the two-dimensional image space. If the pixel point is larger or smaller than the DOG values ​​of the adjacent 26 pixels, the point will be used as a local extreme value point. As other embodiments of the present application, the Harris corner detection method can be used to obtain local extreme value points. Since the DOG value is sensitive to noise and edges, it is necessary to remove low-contrast feature points and unstable edge response points to enhance matching stability and improve noise resistance. Among them, removing low-contrast feature points and unstable edge response points is a well-known technology, and the specific process will not be repeated.

[0052] S3, for the apple tree pest and disease images at each scale, the auxiliary directions are obtained according to the gradient changes of each feature point in different directions, and the suspected pest and disease feature points are screened according to the number of auxiliary directions of all feature points.

[0053] In order to satisfy the rotation invariance of the SIFT description operator, the gradient direction distribution characteristics of the pixels in the 4 neighborhoods of the feature point are used to specify the direction parameters for each feature point. Specifically, for the apple tree pest images at each scale, the gradient vectors of all pixels in the preset neighborhood of each feature point are obtained; wherein the gradient vector includes the gradient magnitude and the gradient direction.

[0054] The method for obtaining the gradient vector of a pixel point in an image is a well-known technique. For the image below, the Sobel operator is used to calculate the feature point (x, y) as the center. is the gradient vector of each pixel in the region image of radius. As other embodiments of the present application, the canny operator can be selected to obtain the gradient vector, which is not limited in the present application.

[0055] After completing the gradient calculation of the feature point, in order to screen the direction that has a greater impact on the gradient direction of the feature point, the significance of the leaf spot disease feature at the feature point is further judged according to the distinction between the gradient direction and other directions.

[0056] The value range of the gradient direction is , recorded as the gradient direction interval; the gradient direction interval is evenly divided into a preset number of sub-intervals; in this embodiment, every 10° is divided into a sub-interval, with a total of 36 sub-intervals. For each feature point, all pixels in the preset neighborhood of the feature point are divided into sub-intervals according to the gradient direction, and the number of pixels contained in each sub-interval is counted; the sub-interval to which the gradient direction of the feature point belongs is recorded as the main direction interval; the number of pixels in the main direction interval is counted and recorded as the first number; the ratio of the number of pixels contained in each sub-interval except the main direction interval to the first number is recorded as the direction probability of each sub-interval; the sub-interval with a direction probability greater than the preset threshold value of 0.8 is recorded as the auxiliary direction interval; the gradient direction of each pixel contained in the auxiliary direction interval is recorded as an auxiliary direction of the feature point.

[0057] The peak direction of the gradient direction histogram represents the main direction of the neighborhood gradient at the feature point, that is, the gradient direction of the feature point. In the gradient direction histogram, if there is another peak value greater than 80% of the main peak value, this direction is considered to be the auxiliary direction of the feature point. A feature point has only one gradient direction, but may have multiple auxiliary directions.

[0058] For apple leaf spot disease, the initial symptoms are small brown dots on the leaves, which gradually expand to reddish brown, and eventually produce small shiny black spots or rust spots on the fruit surface. When the feature point is a feature point in the pest area formed by leaf spot disease, whether it is a small dot, a small spot or a rust spot, there are gradient changes in multiple directions in the local area where the feature point is located, that is, there are multiple auxiliary directions; and if the feature point is not a feature point in the pest area, but a feature point on the vein texture of the leaf, the feature point will only have a large gradient change along the vein texture direction. Therefore, the auxiliary direction of each feature point can be used to preliminarily distinguish the feature points.

[0059] The number of auxiliary directions of each feature point is counted, and the number of auxiliary directions of all feature points is thresholded to obtain the direction segmentation threshold; the pixel points whose auxiliary directions are greater than or equal to the direction segmentation threshold are marked as suspected pest feature points.

[0060] In this embodiment, the number of auxiliary directions of all feature points is used as input, and the direction segmentation threshold is obtained using the Otsu threshold algorithm. The Otsu threshold algorithm is a well-known technology, and the specific process will not be repeated here.

[0061] Screening diagram of suspected pest and disease feature points, such as Figure 2 shown.

[0062] S4, according to the grayscale change difference of each suspected pest and disease feature point in different auxiliary directions, the auxiliary direction discrimination degree of each suspected pest and disease feature point is obtained; according to the distance between different suspected pest and disease feature points, all suspected pest and disease feature points are clustered to obtain the local density of each suspected pest and disease feature point; the pest and disease probability of the suspected pest and disease feature point is obtained by combining the auxiliary direction discrimination degree and local density of the suspected pest and disease feature point.

[0063] Based on the above analysis, the auxiliary direction discrimination is further constructed to characterize the probability that each suspected pest feature point is a pest area feature formed by leaf spot disease. Specifically, it includes:

[0064] Obtain the components of the gradient vector of each suspected pest feature point in each auxiliary direction, recorded as the gradient amplitude in each auxiliary direction;

[0065] For any two auxiliary directions of each suspected pest feature point, calculate the direction difference between the two auxiliary directions; the direction difference is the angle value between the two auxiliary directions; obtain the variance of all the direction differences of each suspected pest feature point, recorded as the first variance; calculate the difference between the gradient amplitudes of the two auxiliary directions, recorded as the gradient amplitude difference, and record the variance of all the gradient amplitude differences of each suspected pest feature point as the second variance;

[0066] Obtain the vector composed of all auxiliary directions of each suspected pest and disease feature point, recorded as the auxiliary direction vector; obtain the sum of the distances between each suspected pest and disease feature point and the auxiliary direction vectors of all other suspected pest and disease feature points, recorded as the local auxiliary direction difference of each suspected pest and disease feature point;

[0067] According to the first variance, second variance and local auxiliary direction difference of each suspected pest and disease feature point, the auxiliary direction discrimination of each suspected pest and disease feature point is obtained;

[0068] Specifically, the product of the first variance and the second variance of each suspected pest and disease feature point is calculated; the sum of the product and a preset constant is obtained; and the ratio of the local auxiliary direction difference of each suspected pest and disease feature point to the sum is used as the auxiliary direction discrimination of each suspected pest and disease feature point.

[0069] As an embodiment of the present application, the auxiliary direction discrimination degree of the i-th suspected pest feature point ; In the formula, is the auxiliary direction discrimination of the i-th suspected pest feature point, n is the number of suspected pest feature points, is the DTW distance between the auxiliary direction vectors of the i-th suspected pest feature point and the j-th suspected pest feature point, are the first variance and second variance of the i-th suspected pest feature point, It is a preset constant that is not 0 to prevent the denominator from being 0. In this embodiment, the value is 0.01.

[0070] The higher the probability that the i-th suspected pest and disease feature point is a feature point in the pest and disease area formed by spotted leaf litter, no matter whether the i-th suspected pest and disease feature point is located in a small dot, a small spot or a embroidery spot, the i-th suspected pest and disease feature point will have gradient changes in multiple directions, and the gradient changes in each direction are relatively close, because the geometric shape of the above area is relatively regular, and the values ​​of the first variance and the second variance are small at this time; then the gradient direction in the area is more concentrated, and the number of auxiliary directions is small. At this time, there are large differences between the auxiliary direction vectors, and the value of the auxiliary direction discrimination is larger.

[0071] For the feature points in the pest and disease area, since their characteristics are small dots or round spots, the local density of this type of feature points is relatively large; for the feature points in the leaf vein texture area, their characteristics are linear and long strips, so the local density of this type of feature points is relatively small.

[0072] Therefore, the location information of all suspected pests and diseases feature points is counted, and the local density of each feature point is evaluated by density clustering. Specifically, the integer value of the preset ratio value of the number of suspected pests and diseases feature points is used as the cutoff distance, and the coordinates of all suspected pests and diseases feature points are density clustered to obtain the local density of each suspected pests and diseases feature point.

[0073] In this embodiment, the rounded value of 2% of the number of suspected pest feature points is used as the set value of the cutoff distance, and the truncated kernel function is used to calculate the local density of each suspected pest feature point. As other embodiments of the present application, density clustering methods such as density peak clustering can be selected, and this application does not limit it.

[0074] Comprehensive consideration of auxiliary direction distinction and the local density Two factors are used to distinguish the feature points of pests and diseases, and to characterize the probability that a feature point belongs to the feature point of pests and diseases. Among them, the auxiliary direction discrimination is the main feature of pest and disease identification. Therefore, a higher weight is given to the auxiliary direction discrimination in the distinction of pest and disease feature points, and the probability of pest and disease of the suspected pest and disease feature point is obtained, which specifically includes: ; In the formula, represents the probability of pests and diseases of the i-th suspected pest and disease feature point; c and d are the first preset weight coefficient and the second preset weight coefficient, c is 0.7, and d is 0.3; is the auxiliary direction discrimination of the i-th suspected pest and disease feature point; is the local density of the i-th suspected pest feature point.

[0075] The greater the auxiliary direction discrimination of a feature point and the greater the local density, the greater the probability that the feature point belongs to the feature point of pests and diseases.

[0076] S5, screening pest feature points according to the pest probability of the suspected pest feature points in the apple tree pest image at all scales, and identifying the location of the pest in the apple tree pest image to be detected according to all the pest feature points.

[0077] The pest probability of all suspected pest feature points in the apple tree pest image at all scales is threshold segmented to obtain the pest feature threshold; the suspected pest feature points whose pest probability is greater than or equal to the pest feature threshold are used as pest feature points. In this embodiment, the pest probability of all suspected pest feature points is used as input, the pest feature threshold is obtained using the Otsu threshold algorithm, and the pest feature points are distinguished from the vein texture feature points according to the feature segmentation threshold q. , then the suspected pest feature point will be used as the pest feature point and will be retained; if , then the feature point will be regarded as the vein texture feature point and will be discarded.

[0078] In the SIFT algorithm, the gradient information in 8 directions calculated in a 4×4 window in the feature point scale space is used to obtain a 4×4×8=128-dimensional feature vector. In order to make the ResNet50 model more accurate in detecting pests and diseases, this application needs to splice the output feature vector of the 5th group of convolutional layers in the ResNet50 network with the feature point descriptor. The necessary condition for splicing is that the two feature vectors have the same dimension. The output vector of the convolutional layer is a 7×7×2048 feature vector, where 7×7 represents the width and height of the image and 2048 represents the dimension. Therefore, in order to be able to splice the feature vectors, a 2048-dimensional SIFT descriptor is required. This application designs a 16×16×8=2048-dimensional SIFT feature vector. The specific process is as follows:

[0079] First, the 8-neighborhood area of ​​the pest feature point is evenly divided into 16×16 sub-areas. Substituting into the SIFT descriptor, a 2048-dimensional feature vector is obtained; therefore, the local area of ​​a pest feature point consists of 16×16 sub-areas, each sub-area has vector information in 8 directions, and a total of 2048 data of 16×16×8 can be generated, forming a 2048-dimensional feature vector, that is, a 2048-dimensional feature point descriptor.

[0080] A total of N pest and disease feature points are obtained, and finally N 2048-dimensional feature vectors are obtained. The N 2048-dimensional feature vectors are concatenated with the 2048-dimensional feature vector output by the fifth convolutional layer of the ResNet50 network. The concatenation technology is a well-known technology, and the specific process is not repeated here. A richer feature representation is obtained, which can help the model better learn the characteristics of pests and diseases.

[0081] To train the ResNet50 model, we first need to obtain manually labeled images, use the image annotation tool Labelme to annotate 500 images of apple tree pests and diseases, and use the annotated pest and disease images as the input of the ResNet50 model. The concatenated feature vectors will pass through the pooling layer and the fully connected layer in turn, and finally output the model's annotation results for pests and diseases.

[0082] The Adam optimizer is used as the optimization algorithm of the model, and the Smooth L1 loss function and the cross entropy loss function are used as the loss function of the model. After 300 rounds of iterative training, the apple tree pest detection model is obtained. Neural network training is a well-known technology, and the specific process is not repeated here. After obtaining the trained model, the apple tree pest image can be detected, and the output result is the location of the pest.

[0083] Through the above description of the implementation method in combination with the accompanying drawings, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0084] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for segmenting and detecting pests and diseases of apple trees, characterized in that: The method comprises the following steps: S1, obtain images of apple tree diseases and pests; S2, dividing the scales of the apple tree pest and disease images to obtain the apple tree pest and disease images at each scale, and extracting feature points from the apple tree pest and disease images at each scale; S3, for the apple tree pest and disease images at each scale, the auxiliary directions are obtained according to the gradient changes of each feature point in different directions, and the suspected pest and disease feature points are screened according to the number of auxiliary directions of all feature points; S4, according to the grayscale change difference of each suspected pest feature point in different auxiliary directions, the auxiliary direction discrimination degree of each suspected pest feature point is obtained; according to the distance between different suspected pest feature points, all suspected pest feature points are clustered to obtain the local density of each suspected pest feature point; the pest probability of the suspected pest feature point is obtained by combining the auxiliary direction discrimination degree and local density of the suspected pest feature point; S5, screening pest feature points according to the pest probability of the suspected pest feature points in the apple tree pest image at all scales, and identifying the location of the pest in the apple tree pest image to be detected according to all the pest feature points; The auxiliary directions are obtained according to the gradient changes of each feature point in different directions, and the suspected pest feature points are screened according to the number of auxiliary directions of all feature points, including: Obtain the gradient direction of all pixels in the preset neighborhood of each feature point; obtain a preset gradient direction interval; evenly divide the gradient direction interval into a preset number of sub-intervals; for each feature point, divide all pixels in the preset neighborhood of the feature point into each sub-interval according to the gradient direction, and count the number of pixels contained in each sub-interval; According to the difference in the number of pixels contained in each sub-interval, the auxiliary directions of the feature points are obtained; the number of auxiliary directions of each feature point is counted, and the number of auxiliary directions of all feature points is threshold segmented to obtain the direction segmentation threshold; the pixel points whose auxiliary directions are greater than or equal to the direction segmentation threshold are marked as suspected pest feature points; The method for obtaining each auxiliary direction of the feature point includes: The sub-interval to which the gradient direction of the feature point belongs is recorded as the main direction interval; the direction probability of each sub-interval is obtained by comparing the number of pixels in the main direction interval with those in other sub-intervals; The sub-intervals with a direction probability greater than a preset threshold value other than the main direction interval are recorded as auxiliary direction intervals; the gradient direction of each pixel point contained in the auxiliary direction interval is recorded as an auxiliary direction of the feature point; The method for obtaining the auxiliary direction discrimination of each suspected pest feature point includes: Obtaining the gradient vector of each suspected pest feature point; recording the component of the gradient vector in each auxiliary direction as the gradient amplitude in each auxiliary direction; For any two auxiliary directions of each suspected pest feature point, calculate the direction difference between the two auxiliary directions; the direction difference is the angle value between the two auxiliary directions; obtain the variance of all the direction differences of each suspected pest feature point, recorded as the first variance; calculate the difference between the gradient amplitudes of the two auxiliary directions, recorded as the gradient amplitude difference, and record the variance of all the gradient amplitude differences of each suspected pest feature point as the second variance; Obtain the vector composed of all auxiliary directions of each suspected pest and disease feature point, recorded as the auxiliary direction vector; obtain the sum of the distances between each suspected pest and disease feature point and the auxiliary direction vectors of all other suspected pest and disease feature points, recorded as the local auxiliary direction difference of each suspected pest and disease feature point; According to the first variance, second variance and local auxiliary direction difference of each suspected pest and disease feature point, the auxiliary direction discrimination of each suspected pest and disease feature point is obtained.

2. A method for segmenting and detecting pests and diseases for apple trees as claimed in claim 1, characterized in that: The scales of the apple tree pest images are divided to obtain apple tree pest images at various scales, and feature points in the apple tree pest images at various scales are extracted, including: The apple tree pest and disease images are input into a Gaussian pyramid with a preset number of layers to obtain the apple tree pest and disease images at various scales; all local extreme points in the apple tree pest and disease images at various scales are obtained; and each local extreme point is recorded as a feature point.

3. A method for segmenting and detecting pests and diseases for apple trees as claimed in claim 1, characterized in that: The method for obtaining the direction probability of each sub-interval includes: The number of pixels in the main direction interval is counted and recorded as the first number; the ratio of the number of pixels in each sub-interval to the first number is recorded as the direction probability of each sub-interval.

4. A method for segmenting and detecting pests and diseases for apple trees as claimed in claim 1, characterized in that: The auxiliary direction discrimination of each suspected pest feature point is obtained according to the first variance, the second variance and the local auxiliary direction difference of each suspected pest feature point, including: Calculate the product of the first variance and the second variance of each suspected pest and disease feature point; obtain the sum of the product and a preset constant; take the ratio of the local auxiliary direction difference of each suspected pest and disease feature point to the sum as the auxiliary direction discrimination of each suspected pest and disease feature point; the value of the preset constant is 0.

01.

5. A method for segmenting and detecting pests and diseases for apple trees as claimed in claim 1, characterized in that: The method for obtaining the local density of each suspected pest feature point includes: The rounded value of the preset ratio value of the number of suspected pest and disease feature points is used as the cutoff distance, and the coordinates of all suspected pest and disease feature points are density clustered to obtain the local density of each suspected pest and disease feature point.

6. A method for segmenting and detecting pests and diseases for apple trees as claimed in claim 1, characterized in that: The method of combining the auxiliary direction discrimination and local density of the suspected pest feature point to obtain the pest probability of the suspected pest feature point includes: recording the pest probability of the i-th suspected pest feature point as , ; In the formula, represents the probability of pests and diseases of the i-th suspected pest and disease feature point; c and d are the first preset weight coefficient and the second preset weight coefficient, c=0.7, d=0.3; is the auxiliary direction discrimination of the i-th suspected pest and disease feature point; is the local density of the i-th suspected pest feature point.

7. A method for segmenting and detecting pests and diseases for apple trees as claimed in claim 1, characterized in that: The method of screening the pest feature points according to the pest probability of the suspected pest feature points in the apple tree pest image at all scales, and identifying the location of the pest in the apple tree pest image to be detected according to all the pest feature points, includes: The pest and disease probability of all suspected pest and disease feature points in the apple tree pest and disease images at all scales is threshold segmented to obtain the pest and disease feature threshold; the suspected pest and disease feature points whose pest and disease probability is greater than or equal to the pest and disease feature threshold are taken as pest and disease feature points; The eight neighborhood areas of the pest feature points are evenly divided into a first preset number of sub-areas, and the SIFT descriptors are substituted to obtain the feature vectors; A second preset number of apple tree pest and disease images are obtained, and the locations of the pests and diseases in the apple tree pest and disease images are manually marked, and the images are used as a training set to train the neural network model. During the training process, all the feature vectors are concatenated with the feature vectors output by the convolutional layer in the neural network model to obtain a trained neural network model; the trained neural network model is used to detect the apple tree pest and disease images to be detected to obtain the locations of the pests and diseases.

Images