Grape downy mildew spore detection method, system and device based on image analysis

Abstract

The invention relates to the technical field of image analysis, and in particular to a method, system and device for detecting grape downy mildew spores based on image analysis. In order to solve the problem of low accuracy and poor stability of detection results when analyzing spore images in the prior art, the invention first sequentially processes a grape downy mildew spore image to be detected through convolution, multiple feature downsampling and global pooling, so as to obtain feature downsampling output and global pooling output that can reflect middle-level features, appropriate scale features and high-level features of grape downy mildew spores; then multi-layer cross feature fusion processing is performed to obtain multiple cross feature fusion images; then, output quality is controlled based on bounding box distance loss and confidence weight distribution loss, and a classification result corresponding to the maximum value of the weighted average value of multiple outputs is used as a grape downy mildew spore detection result; the method is applied to grape downy mildew spore image analysis, so as to improve the accuracy and stability of the detection and analysis results.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a method, system and device for detecting grape downy mildew spores based on image analysis. Background Art

[0002] Grape downy mildew has serious harm to the growth, development, yield and quality of grapes, and will cause huge economic losses to grape growers. However, the early symptoms of grape downy mildew may not be obvious, and it is difficult to accurately detect grape downy mildew through manual inspections. If the image is analyzed, early grape downy mildew spores can be detected, and the presence of the pathogen can be detected before the disease breaks out on a large scale. It is more timely than manual inspections and helps to take prevention and control measures as soon as possible to reduce losses.

[0003] At present, in the field of spore image analysis, Convolutional Neural Network (CNN) performs excellently, but Convolutional Neural Network limits the ability to express contextual features that reflect long-distance spatial dependencies, and does not fully integrate the global information of the image, resulting in poor image analysis capabilities and low accuracy of spore detection results; Generative Adversarial Networks (GAN) can focus on image detail information according to the design, but the overall GAN ​​generator is prone to generate samples that are too similar, resulting in insufficient diversity, making the detection and analysis quality poor and the detection stability low when processing grape downy mildew spore images with complex backgrounds and unclear features. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system and device for detecting grape downy mildew spores based on image analysis with high detection and analysis accuracy and good stability.

[0005] The technical solution of the present invention is as follows:

[0006] A method for detecting grape downy mildew spores based on image analysis comprises the following operations:

[0007] S1. The grape downy mildew spore image to be detected is processed by convolution, several times of feature downsampling and global pooling in turn, and the intermediate feature downsampling output is extracted, and the feature downsampling atlas is formed with the global pooling output; during the several times of feature downsampling, the width and height of the input image are successively reduced, and the number of channels of the input image is successively increased; the operation of feature downsampling is specifically as follows: the input image is processed by feature convolution to obtain a feature convolution image; it is determined whether the color entropy of the feature convolution image is greater than the color entropy threshold; if it is greater, the feature convolution image is processed by multi-path aggregation based on color channels and multi-scale attention to obtain a feature downsampling output; if it is not greater, the feature convolution image is processed by multi-path aggregation based on image brightness and multi-scale attention to obtain a feature downsampling output; the color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution image;

[0008] S2, in the feature downsampling graph set, all feature downsampling graphs are processed by multi-layer cross feature fusion to obtain several cross feature fusion graphs;

[0009] S3. Several cross-feature fusion images are processed by bounding box regression and classification respectively, and the bounding box distance loss and confidence weight distribution loss are output, which are result images that are respectively less than the bounding box loss threshold and the confidence weight threshold, to obtain several initial fusion detection analysis images; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; several initial fusion detection analysis images are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, to obtain the grape downy mildew spore detection result.

[0010] The specific operation of the color channel-based multi-path aggregation processing in S1 is: respectively obtaining the standard deviations of the red color channel, the green color channel, and the blue color channel in the feature convolution image, performing channel feature enhancement processing on the color channel with the maximum standard deviation, and then fusing it with other color channels to obtain a color channel path aggregation map for performing multi-scale attention processing operations; the specific operation of the channel feature enhancement processing is: performing refocusing feature convolution processing and ordinary convolution processing on the color channel with the maximum standard deviation to obtain a first channel refocusing map; performing refocusing feature convolution processing and ordinary convolution processing on the first channel refocusing map to obtain a second refocusing map; the first channel refocusing map and the second refocusing map are used to perform fusion processing with other color channels.

[0011] The operations of multi-path aggregation processing based on image brightness in S1 are as follows: based on the YUV color space features of the feature convolution map, a brightness map, a blue deviation brightness map, and a red deviation brightness map are obtained; the global features and local features of the brightness map are extracted and fused to obtain a brightness enhancement map; the brightness enhancement map is fused with the blue deviation brightness map and the red deviation brightness map to obtain a brightness path aggregation map, which is used to perform multi-scale attention processing operations.

[0012] The specific operations of multi-scale attention processing in S1 are as follows: the feature convolution map is subjected to multi-path aggregation processing based on color channels or based on image brightness to obtain a path aggregation map; the path aggregation map is subjected to key feature extraction processing, depthwise separable convolution and ordinary convolution processing at different scales, and then weighted fusion and ordinary convolution processing are performed to obtain feature downsampling output; the operation of the current path aggregation map after key feature extraction processing is as follows: the current path aggregation map is obtained, and after the mean brightness of all pixels in each channel in the horizontal and vertical directions, channel dimension splicing and depthwise separable convolution processing are performed to obtain a spatial feature map; the spatial feature map is fused after ordinary convolution and nonlinear processing at different scales to obtain a key feature map for performing depthwise separable convolution operations.

[0013] The operation of multi-layer cross feature fusion processing in S2 is specifically as follows: the feature downsampling graph includes the first feature downsampling output, the second feature downsampling output and the global pooling output; the second feature downsampling output is obtained by performing feature downsampling processing on the first feature downsampling output; the first feature downsampling output is subjected to ordinary convolution processing to obtain the bottom-layer feature map; the bottom-layer feature map is subjected to upsampling processing to obtain the first feature upsampling map; the second feature downsampling output is subjected to ordinary convolution processing and fused with the first feature upsampling map to obtain the middle-layer feature map; the global pooling output is subjected to ordinary convolution processing to obtain the third feature convolution map; the middle-layer feature map is subjected to upsampling processing and fused with the third feature convolution map The high-level feature map is obtained by fusion and ordinary convolution with the middle-level feature map, and the high-level cross-feature fusion map is obtained; the high-level cross-feature fusion map is down-sampled to obtain the high-level down-sampled map; the high-level feature map is fused with the middle-level feature map, the bottom-level feature map, and the high-level down-sampled map and ordinary convolution to obtain the middle-level cross-feature fusion map; the middle-level cross-feature fusion map is down-sampled to obtain the middle-level down-sampled map; the middle-level feature map is fused with the bottom-level feature map and the middle-level down-sampled map and ordinary convolution to obtain the bottom-level cross-feature fusion map; the high-level cross-feature fusion map, the middle-level cross-feature fusion map and the bottom-level cross-feature fusion map form several cross-feature fusion maps.

[0014] The bounding box distance loss in S3 is obtained based on the mean distance difference between all bounding boxes and the corresponding ground-truth boxes.

[0015] The confidence weight distribution loss in S3 is obtained based on the weights, confidences, and cross entropy losses of all bounding boxes whose confidences are between the first confidence and the second confidence; the first confidence is less than the second confidence.

[0016] A grape downy mildew spore detection system based on image analysis, used to implement the above-mentioned grape downy mildew spore detection method based on image analysis, comprising:

[0017] The feature downsampling atlas generation module is used to process the grape downy mildew spore image to be detected through convolution, several feature downsampling and global pooling in sequence, extract the intermediate feature downsampling output, and form a feature downsampling atlas with the global pooling output; during the several feature downsampling processes, the width and height of the input image are successively reduced, and the number of channels of the input image is successively increased; the operation of feature downsampling is specifically as follows: the input image is subjected to feature convolution processing to obtain a feature convolution image; it is determined whether the color entropy of the feature convolution image is greater than the color entropy threshold; if it is greater, the feature convolution image is subjected to multi-path aggregation processing based on color channels and multi-scale attention processing to obtain a feature downsampling output; if it is not greater, the feature convolution image is subjected to multi-path aggregation processing based on image brightness and multi-scale attention processing to obtain a feature downsampling output; the color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution image;

[0018] The cross-feature fusion map generation module is used in the feature downsampling map set. All feature downsampling maps are processed by multi-layer cross-feature fusion to obtain several cross-feature fusion maps.

[0019] The grape downy mildew spore detection result generation module is used for a number of cross-feature fusion images, which are respectively processed by bounding box regression and classification, and output bounding box distance loss and confidence weight distribution loss, which are result images less than the bounding box loss threshold and confidence weight threshold, respectively, to obtain a number of initial fusion detection analysis images; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; a number of initial fusion detection analysis images are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, to obtain the grape downy mildew spore detection result.

[0020] A grape downy mildew spore detection device based on image analysis comprises a processor and a memory, wherein the processor implements the grape downy mildew spore detection method based on image analysis when executing a computer program stored in the memory.

[0021] A computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the above-mentioned grape downy mildew spore detection method based on image analysis is implemented.

[0022] The beneficial effects of the present invention are:

[0023] The present invention provides a grape downy mildew spore detection method based on image analysis. First, a grape downy mildew spore image to be detected is subjected to convolution, several feature downsamplings and global pooling processing in sequence to extract local texture or structural details that are helpful for identifying grape downy mildew spores, feature downsampling outputs in the middle of a rough area that is helpful for locating grape downy mildew spores in an image, and global pooling outputs that can provide information such as the distribution of grape downy mildew spores in the entire image, thereby forming a feature downsampling atlas that can reflect the morphological or aggregated morphological characteristics of grape downy mildew spores at different scales; then, a first feature downsampling output, a second feature downsampling output and a global pooling output in the feature downsampling atlas that can reflect the local texture or structural details, appropriate scale and global feature information of the grape downy mildew spore image to be detected are subjected to multi-layer cross-feature fusion processing to make the feature outputs at different levels The cross-feature fusion images obtained by the method integrate information at all levels and can better distinguish grape downy mildew spores from other similar objects or backgrounds. Then, the cross-feature fusion images are processed by bounding box regression and classification respectively. The quality of the initial fusion detection analysis image is controlled based on the bounding box distance loss and confidence weight distribution loss. The weighted average of the corresponding classification probabilities of the bounding boxes at the corresponding positions of the initial fusion detection analysis images is processed. The information of multiple images is integrated to obtain a comprehensive classification probability result to stabilize the detection result. Finally, the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box to obtain the detection result of grape downy mildew spores. This method is applied to the image analysis of grape downy mildew spores, which can improve the accuracy and stability of the detection and analysis results. DETAILED DESCRIPTION

[0024] This embodiment provides a method for detecting grape downy mildew spores based on image analysis, comprising the following operations:

[0025] S1. The grape downy mildew spore image to be detected is processed by convolution, several times of feature downsampling and global pooling in turn, and the intermediate feature downsampling output is extracted, and the feature downsampling atlas is formed with the global pooling output; during the several times of feature downsampling, the width and height of the input image are successively reduced, and the number of channels of the input image is successively increased; the operation of feature downsampling is specifically as follows: the input image is processed by feature convolution to obtain a feature convolution image; it is determined whether the color entropy of the feature convolution image is greater than the color entropy threshold; if it is greater, the feature convolution image is processed by multi-path aggregation based on color channels and multi-scale attention to obtain a feature downsampling output; if it is not greater, the feature convolution image is processed by multi-path aggregation based on image brightness and multi-scale attention to obtain a feature downsampling output; the color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution image;

[0026] S2, in the feature downsampling graph set, all feature downsampling graphs are processed by multi-layer cross feature fusion to obtain several cross feature fusion graphs;

[0027] S3. Several cross-feature fusion images are processed by bounding box regression and classification respectively, and the bounding box distance loss and confidence weight distribution loss are output, which are result images that are respectively less than the bounding box loss threshold and the confidence weight threshold, to obtain several initial fusion detection analysis images; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; several initial fusion detection analysis images are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, to obtain the grape downy mildew spore detection result.

[0028] S1. The grape downy mildew spore image to be detected is processed by convolution, several times of feature downsampling and global pooling in sequence, and the intermediate feature downsampling output is extracted and combined with the global pooling output to form a feature downsampling atlas.

[0029] The grape downy mildew spore image to be detected is subjected to convolution, several times of feature downsampling and global pooling in sequence to extract local texture or structural details that help identify grape downy mildew spores, as well as feature downsampling outputs in the middle of the approximate area of ​​​​the image that helps locate the grape downy mildew spores in the image. Together with the global pooling output that can provide information such as the distribution of grape downy mildew spores in the entire image, a feature downsampling atlas is formed that can reflect the morphological or aggregation morphological characteristics of grape downy mildew spores at different scales.

[0030] Firstly, the grape downy mildew spore image to be detected is convolved (ordinary convolution processing) to obtain the convolution image to be detected; then the convolution image to be detected is subjected to several feature downsampling processes to highlight the key features and reduce the amount of calculation to obtain the feature downsampling image to be detected; the feature downsampling image to be detected is subjected to global pooling processing to obtain the global feature information and obtain the global pooling output.

[0031] Among them, during several feature downsampling processes, the width and height of the input image decrease successively, and the number of channels of the input image increases successively, that is, the width and height of the input in the previous feature downsampling process are greater than the width and height of the input in the current feature downsampling process, and the number of channels is less than the number of channels of the input in the current feature downsampling process.

[0032] The above-mentioned feature downsampling operation is specifically as follows: the input image is processed by feature convolution to obtain a feature convolution map; and it is determined whether the color entropy of the feature convolution map is greater than the color entropy threshold. The feature convolution processing is achieved by sequentially executing ordinary convolution, normalization layer and activation layer.

[0033] Color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution map. Color entropy can help analyze features such as the complexity of spore color distribution.

[0034] The formula for obtaining color entropy is as follows:

[0035] ,

[0036] H is the color entropy, p(i) For the i The probability of occurrence of a color level in the feature convolution map, p(i)=n(i) / N , n (i) is the number of occurrences of the pixel corresponding to the color level in the feature convolution map, N is the total number of pixels in the feature convolution map, L is the total number of color levels.

[0037] If it is greater than, it means that the background of the grape downy mildew spore image to be detected is complex. Therefore, it is necessary to subject the feature convolution image to multi-path aggregation processing based on the color channel and multi-scale attention processing to enhance the feature expression ability of the grape downy mildew spores from the perspective of the color channel and obtain the feature downsampling output.

[0038] The specific operation of color channel-based multi-path aggregation processing is as follows: respectively obtain the standard deviations of the red color channel, green color channel, and blue color channel in the feature convolution map, perform channel feature enhancement processing on the color channel with the maximum standard deviation to amplify the unique information contained in the channel and improve the contrast-related details of the image, and then perform fusion processing with other color channels (stitching processing in the channel dimension) to obtain a color channel path aggregation map for performing multi-scale attention processing operations.

[0039] The specific operation of channel feature enhancement processing is as follows: the color channel with the maximum standard deviation is subjected to refocusing feature convolution processing and ordinary convolution processing, paying more attention to the texture features related to grape downy mildew spores in the channel or the local structural features that are key to the identification of grape downy mildew spores, enhancing the channel feature expression ability, and obtaining the first channel regrouping map; the first channel regrouping map is subjected to refocusing feature convolution processing and ordinary convolution processing to further enhance the channel feature expression ability and obtain the second regrouping map; the first channel regrouping map and the second regrouping map are used to perform fusion processing operations with other color channels.

[0040] The operation of the refocusing feature convolution processing is as follows: the color channel with the maximum standard deviation is processed by ordinary convolution to obtain the first convolution image; the first convolution image is processed by ordinary convolution, refocusing convolution (which can be achieved through the refConv module) and ordinary convolution, and the second convolution image is obtained by adjusting the focus of the convolution (such as changing the distribution of the convolution kernel weights) to more accurately extract the features related to grape downy mildew spores; the second convolution image and the first convolution image are fused to obtain the refocusing feature convolution image, which is used to perform the above-mentioned ordinary convolution processing operation.

[0041] If it is not greater than, it means that the background of the grape downy mildew spore image to be detected is single. To improve efficiency, the feature convolution map is processed by multi-path aggregation based on image brightness and multi-scale attention processing, focusing on image detail features of different brightness from different paths to obtain feature downsampling output.

[0042] The operations of multi-path aggregation processing based on image brightness are as follows: based on the YUV color space features of the feature convolution map, a brightness map, a blue deviation brightness map (the difference information between the blue component information in the feature convolution map and the brightness map) and a red deviation brightness map (the difference information between the red component information in the feature convolution map and the brightness map) are obtained; the global and local features of the brightness map are extracted and fused to enhance the structural feature expression ability of grape downy mildew spores to obtain a brightness enhancement map; the brightness enhancement map is fused with the blue deviation brightness map and the red deviation brightness map to enhance the expression ability of grape downy mildew spores while retaining the initial detail information of the image, and a brightness path aggregation map is obtained for performing multi-scale attention processing operations.

[0043] The operation of extracting the global features of the brightness map can be achieved by converting the brightness map from the spatial domain to the frequency domain (obtained by performing a two-dimensional Fourier transform on the brightness value of each pixel in the brightness map). This can obtain the overall brightness trend of the brightness map, brightness changes over a large area, etc.

[0044] The operation of extracting local features of the brightness map can be achieved by convolving the Laplace operator template original image with the brightness map. The result obtained represents the second-order derivative information of each pixel point. A positive value indicates that the point is a local minimum, and a negative value indicates a local maximum. These extreme points are often surrounded by areas with rich local detail features, such as isolated bright spots or dark spots and detailed parts of the edges, which is conducive to analyzing the local morphological characteristics of grape downy mildew spores.

[0045] In addition, the operation of the above-mentioned multi-scale attention processing is specifically as follows: the feature convolution map is subjected to multi-path aggregation processing based on color channels or based on image brightness to obtain a path aggregation map (color channel path aggregation map or brightness path aggregation map); the path aggregation map is successively subjected to key feature extraction processing at different scales, depthwise separable convolution at different scales and ordinary convolution processing at different scales, and then weighted fusion and ordinary convolution processing at different scales are performed. In this process, the large scale can pay attention to the distribution position of the grape downy mildew spore group, and the small scale can focus on the position of a single grape downy mildew spore, which is beneficial to enhancing the feature expression ability of the spores and obtaining a feature downsampling output that is beneficial to improving the accuracy of image detection results.

[0046] The specific operation of the key feature extraction processing of the current path aggregation map is as follows: after obtaining the current path aggregation map, the mean brightness of all pixels in each channel in the horizontal and vertical directions are respectively obtained, channel dimension stitching processing and depth-separable convolution processing are performed to obtain a spatial feature map; the spatial feature map is fused after ordinary convolution and nonlinear processing of different scales to obtain a key feature map for performing the depth-separable convolution operation.

[0047] S2. In the feature downsampling graph set, all feature downsampling graphs are processed by multi-layer cross feature fusion to obtain several cross feature fusion graphs.

[0048] The first feature downsampling output, the second feature downsampling output and the global pooling output in the feature downsampling atlas, which can reflect the local texture or structural details, appropriate scale and global feature information of the grape downy mildew spore image to be detected, are subjected to multi-layer cross-feature fusion processing, so that the feature outputs at different levels complement each other, reducing the detection errors caused by incomplete single feature information. The obtained several cross-feature fusion images integrate information at each level, which can better distinguish grape downy mildew spores from other similar objects or backgrounds, and is conducive to improving the accuracy of subsequent detection.

[0049] The specific operations of multi-layer cross feature fusion processing are as follows.

[0050] First, the feature downsampling atlas includes a first feature downsampling output containing relatively local and detailed mid- and high-level features, a second feature downsampling output with a more moderate feature scale, and a global pooling output containing the global feature information of the image; wherein the first feature downsampling output is the input for obtaining the second feature downsampling output, that is, the first feature downsampling output is processed by feature downsampling to obtain the second feature downsampling output.

[0051] Then, the first feature downsampling output is processed by ordinary convolution to obtain the bottom feature map; the bottom feature map is processed by upsampling to obtain the first feature upsampling map; the second feature downsampling output is processed by ordinary convolution and fused with the first feature upsampling map to obtain the middle feature map; the global pooling output is processed by ordinary convolution to obtain the third feature convolution map; the middle feature map is processed by upsampling and fused with the third feature convolution map to obtain the high-level feature map. The first feature downsampling output, the second feature downsampling output and the global pooling output are cross-fused for preliminary feature complementation.

[0052] Next, in order to further perform feature cross-fusion and obtain fusion information of each scale, the high-level feature map and the middle-level feature map are fused and processed by ordinary convolution to obtain a high-level cross-feature fusion map; the high-level cross-feature fusion map is down-sampled to obtain a high-level down-sampled map; the high-level feature map is fused and processed by ordinary convolution with the middle-level feature map, the bottom-level feature map, and the high-level down-sampled map to obtain a middle-level cross-feature fusion map; the middle-level cross-feature fusion map is down-sampled to obtain the middle-level down-sampled map; the middle-level feature map is fused and processed by ordinary convolution with the bottom-level feature map and the middle-level down-sampled map to obtain the bottom-level cross-feature fusion map.

[0053] Finally, the cross-feature fusion results at different scales, namely the high-level cross-feature fusion map, the middle-level cross-feature fusion map and the bottom-level cross-feature fusion map, formed several cross-feature fusion maps containing unique structural information about spores under various image states, which are used for subsequent detection and can ensure the accuracy and stability of the detection results.

[0054] If the number of feature downsampling is an odd number, the output result of the next feature downsampling of the first feature downsampling output as the middle feature downsampling output is used as the second feature downsampling output.

[0055] S3. Several cross-feature fusion images are processed by bounding box regression and classification respectively, and the bounding box distance loss and confidence weight distribution loss are output, which are result images that are respectively less than the bounding box loss threshold and the confidence weight threshold, to obtain several initial fusion detection analysis images; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; several initial fusion detection analysis images are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, to obtain the grape downy mildew spore detection result image.

[0056] First, several cross-feature fusion images are processed by bounding box regression and classification respectively. When the bounding box distance loss and confidence weight allocation loss are less than the bounding box loss threshold and confidence weight threshold respectively, the result image at this time is output to obtain several initial fusion detection analysis images. Among them, in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities.

[0057] The bounding box distance loss is based on the average of the distance differences between all bounding boxes and the corresponding ground-truth boxes. The smaller the distance difference between the bounding box and the corresponding ground-truth box, the better the bounding box regression effect, the smaller the bounding box distance loss, and the more accurate the detection and positioning results of grape downy mildew spores.

[0058] The confidence weighted assignment loss is based on the weights, confidences, and cross entropy loss of all bounding boxes whose confidences are between the first confidence and the second confidence. The first confidence is less than the second confidence.

[0059] Among them, the weight of the bounding box whose confidence is between the first confidence and the second confidence can be obtained by the following formula: w i,t =γ·w i,t-1 ·e (1-μ) , w i,t for t Time i The weight of the bounding box, w i,t-1 for t-1 Time i The weight of the bounding box, m is the second confidence level, μ-0.1 is the first confidence level, c Assign tuning parameters to the weights.

[0060] Then, based on the weights of their respective detection analysis graphs, several initial fusion detection analysis graphs are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions. By integrating the information of multiple graphs, a comprehensive classification probability result is obtained to stabilize the detection results.

[0061] Finally, the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, and the grape downy mildew spore detection result map is obtained.

[0062] This embodiment also provides a grape downy mildew spore detection system based on image analysis, which is used to implement the above-mentioned grape downy mildew spore detection method based on image analysis, including:

[0063] The feature downsampling atlas generation module is used to process the grape downy mildew spore image to be detected through convolution, several feature downsampling and global pooling in sequence, extract the intermediate feature downsampling output, and form a feature downsampling atlas with the global pooling output; during the several feature downsampling processes, the width and height of the input image are successively reduced, and the number of channels of the input image is successively increased; the operation of feature downsampling is specifically as follows: the input image is subjected to feature convolution processing to obtain a feature convolution image; it is determined whether the color entropy of the feature convolution image is greater than the color entropy threshold; if it is greater, the feature convolution image is subjected to multi-path aggregation processing based on color channels and multi-scale attention processing to obtain a feature downsampling output; if it is not greater, the feature convolution image is subjected to multi-path aggregation processing based on image brightness and multi-scale attention processing to obtain a feature downsampling output; the color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution image;

[0064] The cross-feature fusion map generation module is used in the feature downsampling map set. All feature downsampling maps are processed by multi-layer cross-feature fusion to obtain several cross-feature fusion maps.

[0065] The grape downy mildew spore detection result generation module is used for a number of cross-feature fusion images, which are respectively processed by bounding box regression and classification, and output bounding box distance loss and confidence weight distribution loss, which are result images less than the bounding box loss threshold and confidence weight threshold, respectively, to obtain a number of initial fusion detection analysis images; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; a number of initial fusion detection analysis images are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, to obtain the grape downy mildew spore detection result.

[0066] This embodiment also provides a grape downy mildew spore detection device based on image analysis, including a processor and a memory, wherein the processor implements the above-mentioned grape downy mildew spore detection method based on image analysis when executing a computer program stored in the memory.

[0067] This embodiment further provides a computer-readable storage medium for storing a computer program, wherein when the computer program is executed by a processor, the above-mentioned grape downy mildew spore detection method based on image analysis is implemented.

[0068] The present embodiment provides a grape downy mildew spore detection method based on image analysis. First, a grape downy mildew spore image to be detected is subjected to convolution, several times of feature downsampling and global pooling processing in sequence to extract local texture or structural details that are helpful for identifying grape downy mildew spores, feature downsampling outputs in the middle of a rough area that is helpful for locating grape downy mildew spores in an image, and global pooling outputs that can provide information such as the distribution of grape downy mildew spores in the entire image, thereby forming a feature downsampling atlas that can reflect the morphological or aggregated morphological characteristics of grape downy mildew spores at different scales; then, a first feature downsampling output, a second feature downsampling output and a global pooling output in the feature downsampling atlas that can reflect the local texture or structural details, appropriate scale and global feature information of the grape downy mildew spore image to be detected are subjected to multi-layer cross-feature fusion processing to make the feature outputs at different levels interlinked. The cross-feature fusion images obtained by the method integrate information at all levels and can better distinguish grape downy mildew spores from other similar objects or backgrounds. Then, the cross-feature fusion images are processed by bounding box regression and classification respectively. The quality of the initial fusion detection analysis image is controlled based on the bounding box distance loss and confidence weight distribution loss. The weighted average of the corresponding classification probabilities of the bounding boxes at the corresponding positions of the initial fusion detection analysis images is processed. The information of multiple images is integrated to obtain a comprehensive classification probability result to stabilize the detection result. Finally, the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box to obtain the detection result of grape downy mildew spores. This method is applied to the image analysis of grape downy mildew spores, which can improve the accuracy and stability of the detection and analysis results.

Claims

1. A method for detecting grape downy mildew spores based on image analysis, characterized in that: The following operations are included: S1, the grape downy mildew spore image to be detected is processed by convolution, several times of feature downsampling and global pooling in sequence, and the intermediate feature downsampling output is extracted and combined with the global pooling output to form a feature downsampling atlas; During several feature downsampling processes, the width and height of the input image decrease successively, and the number of channels of the input image increases successively; The operation of feature downsampling is as follows: the input image is processed by feature convolution to obtain a feature convolution map; whether the color entropy of the feature convolution map is greater than the color entropy threshold; If it is greater than, the feature convolution map is subjected to multi-path aggregation processing based on the color channel and multi-scale attention processing to obtain a feature down-sampling output; the operation of the multi-path aggregation processing based on the color channel is specifically as follows: the standard deviations of the red color channel, the green color channel, and the blue color channel in the feature convolution map are obtained respectively, and the color channel with the maximum standard deviation is subjected to channel feature enhancement processing, and then fused with other color channels to obtain a color channel path aggregation map for performing the multi-scale attention processing operation; The operation of channel feature enhancement processing is specifically as follows: the color channel with the maximum standard deviation is subjected to refocusing feature convolution processing and ordinary convolution processing to obtain a first channel refocusing map; the first channel refocusing map is subjected to refocusing feature convolution processing and ordinary convolution processing to obtain a second refocusing map; the first channel refocusing map and the second refocusing map are used to perform the operation of fusion processing with other color channels; If it is not greater than, the feature convolution map is processed by multi-path aggregation based on image brightness and multi-scale attention to obtain feature downsampling output; Color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution map; S2, in the feature downsampling graph set, all feature downsampling graphs are processed by multi-layer cross feature fusion to obtain several cross feature fusion graphs; S3, several cross-feature fusion images are processed by bounding box regression and classification respectively, and the bounding box distance loss and confidence weight distribution loss are output, and the result images are respectively less than the bounding box loss threshold and the confidence weight threshold, and several initial fusion detection analysis images are obtained; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; The weighted average processing of the corresponding classification probabilities of the corresponding position bounding boxes is performed on several initial fusion detection analysis images, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box to obtain the grape downy mildew spore detection result.

2. The method for detecting grape downy mildew spores based on image analysis according to claim 1, characterized in that: In S1, the operation of multi-path aggregation processing based on image brightness is specifically as follows: Based on the YUV color space features of the feature convolution map, a brightness map, a blue deviation brightness map, and a red deviation brightness map are obtained; the global features and local features of the brightness map are extracted and fused to obtain a brightness enhancement map; The brightness enhancement map is fused with the blue deviation brightness map and the red deviation brightness map to obtain a brightness path aggregation map, which is used to perform multi-scale attention processing operations.

3. The method for detecting grape downy mildew spores based on image analysis according to claim 1, characterized in that: In S1, the operation of multi-scale attention processing is specifically as follows: The feature convolution graph is subjected to multi-path aggregation processing based on color channels or based on image brightness to obtain a path aggregation graph; the path aggregation graph is subjected to key feature extraction processing at different scales, depthwise separable convolution and ordinary convolution processing, and then weighted fusion and ordinary convolution processing are performed to obtain the feature downsampling output; The specific operation of the key feature extraction processing of the current path aggregation map is as follows: after obtaining the current path aggregation map, the average brightness of all pixels in each channel in the horizontal and vertical directions is calculated, channel dimension splicing and depth-separable convolution are performed to obtain a spatial feature map; the spatial feature map is fused after ordinary convolution and nonlinear processing of different scales to obtain a key feature map for performing depth-separable convolution operations.

4. The method for detecting grape downy mildew spores based on image analysis according to claim 1, characterized in that: In S2, the operation of multi-layer cross feature fusion processing is specifically as follows: The feature downsampling atlas includes a first feature downsampling output, a second feature downsampling output and a global pooling output; the second feature downsampling output is obtained by performing feature downsampling processing on the first feature downsampling output; The first feature downsampling output is processed by ordinary convolution to obtain a bottom feature map; the bottom feature map is processed by upsampling to obtain a first feature upsampling map; The second feature downsampling output is fused with the first feature upsampling map after ordinary convolution processing to obtain a middle-level feature map; The global pooling output is processed by ordinary convolution to obtain a third feature convolution map; the middle-level feature map is fused with the third feature convolution map after upsampling to obtain a high-level feature map; The high-level feature map and the middle-level feature map are fused and processed by ordinary convolution to obtain a high-level cross-feature fusion map; The high-level cross-feature fusion map is downsampled to obtain a high-level downsampled map; The high-level feature map, the middle-level feature map, the bottom-level feature map, and the high-level downsampled map are fused and processed by ordinary convolution to obtain a middle-level cross-feature fusion map; the middle-level cross-feature fusion map is downsampled to obtain a middle-level downsampled map; The middle-layer feature map is fused with the bottom-layer feature map and the middle-layer downsampling map through ordinary convolution to obtain the bottom-layer cross-feature fusion map; The high-level cross-feature fusion map, the middle-level cross-feature fusion map and the bottom-level cross-feature fusion map form the several cross-feature fusion maps.

5. The method for detecting grape downy mildew spores based on image analysis according to claim 1, characterized in that: In S3, the bounding box distance loss is obtained based on the average distance difference between all bounding boxes and the corresponding real boxes.

6. The method for detecting grape downy mildew spores based on image analysis according to claim 1, characterized in that: In S3, the confidence weight distribution loss is obtained based on the weights, confidences and cross entropy losses of all bounding boxes whose confidences are between a first confidence and a second confidence; the first confidence is less than the second confidence.

7. A grape downy mildew spore detection system based on image analysis, characterized in that: The method for detecting grape downy mildew spores based on image analysis according to claim 1 comprises: The feature downsampling atlas generation module is used to process the grape downy mildew spore image to be detected through convolution, several feature downsampling and global pooling in sequence, extract the intermediate feature downsampling output, and form a feature downsampling atlas with the global pooling output; during the several feature downsampling processes, the width and height of the input image are successively reduced, and the number of channels of the input image is successively increased; the operation of feature downsampling is specifically as follows: the input image is subjected to feature convolution processing to obtain a feature convolution image; it is determined whether the color entropy of the feature convolution image is greater than the color entropy threshold; if it is greater, the feature convolution image is subjected to multi-path aggregation processing based on color channels and multi-scale attention processing to obtain a feature downsampling output; if it is not greater, the feature convolution image is subjected to multi-path aggregation processing based on image brightness and multi-scale attention processing to obtain a feature downsampling output; the color entropy is obtained based on the sum of the probabilities of all color levels appearing in the feature convolution image; The cross-feature fusion map generation module is used in the feature downsampling map set. All feature downsampling maps are processed by multi-layer cross-feature fusion to obtain several cross-feature fusion maps. The grape downy mildew spore detection result generation module is used for a number of cross-feature fusion images, which are respectively processed by bounding box regression and classification, and output bounding box distance loss and confidence weight distribution loss, which are result images less than the bounding box loss threshold and confidence weight threshold, respectively, to obtain a number of initial fusion detection analysis images; in each initial fusion detection analysis image, a bounding box contains multiple classification results and corresponding classification probabilities; a number of initial fusion detection analysis images are subjected to weighted average processing of the corresponding classification probabilities of the bounding boxes at corresponding positions, and the classification result corresponding to the maximum value of the weighted average value in the bounding box is used as the detection result of the corresponding bounding box, to obtain the grape downy mildew spore detection result.

8. A grape downy mildew spore detection device based on image analysis, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the method for detecting grape downy mildew spores based on image analysis as claimed in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for detecting grape downy mildew spores based on image analysis according to any one of claims 1 to 6 is implemented.