Grape leaf AI grading processing method and system based on computer vision

Abstract

The invention belongs to the technical field of image processing, and discloses a grape leaf AI grading processing method and system based on computer vision. Comprising the following steps: acquiring image data of grape leaves, and performing variety label marking and image optimization to obtain a clear leaf image data set; sequentially performing feature extraction and feature fusion on the clear leaf image data set to obtain fused disease condition features and fused growth condition features; constructing a leaf evaluation framework capable of being adjusted in real time based on the fused disease condition features and the fused growth condition features; performing image evaluation on the clear leaf image data set by using a leaf evaluation framework to obtain a disease degree evaluation index and a growth condition evaluation index, and performing disease degree grading and growth condition grading on the clear leaf image data set based on the disease degree evaluation index and the growth condition evaluation index; obtaining a grape leaf grading result; accurate grading of the grape leaves is achieved, and the industrial production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a grape leaf AI grading processing method and system based on computer vision. Background Art

[0002] In the industrial production process related to grape leaves, grape leaf grading is a key link that has received widespread attention today. However, the grading method that relies solely on manual visual inspection and experience judgment has problems such as low efficiency, possible missed detection, and high subjectivity that is prone to misjudgment. With the development of computer vision technology, this technology has been widely used in the field of grape leaf grading, but challenges still exist.

[0003] Traditional grading methods that rely on computer vision often do not collect comprehensive data, and may only include grape leaf image data from a single angle. This can easily lead to shadows caused by occlusion between leaves, resulting in problems such as the inability to recognize or incomplete recognition of grape leaf images. Traditional algorithms do not take into account the importance of three-dimensional models, and it is difficult to integrate the three-dimensional model of grape leaves into the grading process. Traditional methods cannot perform appropriate image optimization on the collected image data, the filtering scale is relatively simple, and the brightness and color of the image are not adjusted, resulting in image blur, and the target area may be affected by image background interference. When extracting features, traditional algorithms mostly use fixed convolution scales for feature extraction, and cannot be associated with three-dimensional models, which may result in incomplete or incorrect extracted features. In addition, the traditional evaluation framework cannot adaptively adjust parameters to ensure evaluation effectiveness and efficiency, and is more likely to lead to errors in the evaluation results. When grading, traditional grading standards rely on a unified threshold, and cannot design differentiated grading methods based on the characteristics of grape leaves, resulting in inaccurate grading results.

[0004] In view of this, the present invention proposes a grape leaf AI grading processing method and system based on computer vision to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned objectives, the present invention provides the following technical solution: a grape leaf AI grading method based on computer vision, comprising:

[0006] S1. Acquire grape leaf image data and integrate all image data to obtain a leaf image dataset; label the leaf image dataset with variety labels based on grape leaf varieties to obtain a variety-labeled image dataset; and optimize the variety-labeled image dataset to obtain a clear leaf image dataset;

[0007] S2. Construct a feature extraction model to extract features from the clear leaf image dataset to obtain disease characteristics and growth characteristics; perform cross-level feature fusion on all disease characteristics to obtain fused disease characteristics; and perform hierarchical splicing feature fusion on all growth characteristics to obtain fused growth characteristics.

[0008] S3. Constructing a leaf evaluation framework based on the fused disease characteristics and the fused growth characteristics, and adjusting the framework parameters of the leaf evaluation framework in real time;

[0009] S4. Use the leaf evaluation framework to perform image evaluation on the clear leaf image dataset, and obtain the disease degree evaluation index and growth condition evaluation index respectively. Based on the disease degree evaluation index and growth condition evaluation index, the clear leaf image dataset is graded in terms of disease degree and growth condition respectively to obtain the grape leaf grading results.

[0010] Furthermore, the method of performing image optimization on the variety mark image dataset includes:

[0011] A dual-channel filtering architecture was designed, including a high-density noise filtering branch and a mixed noise filtering branch. These two branches processed any variety mark image in the variety mark image dataset in parallel. A filtering sliding window was constructed. The noise density within the filtering sliding window was calculated in the high-density noise filtering branch, and the size of the filtering sliding window was adjusted based on the noise density. The filtering sliding window was used to traverse any variety mark image and filter the image area within the filtering sliding window.

[0012] In the hybrid noise filtering branch, any variety mark image is gridded based on the filtering sliding window, and the variance of all pixels in any grid is calculated. A variance difference threshold is set, and the variance of any grid is compared with the variance of the adjacent grid. If the pixel variance difference between the two grids is greater than the preset variance difference threshold, the grid with the larger variance is determined to be a noise grid, and noise suppression is performed on all noise grids. After processing with a dual-channel filtering architecture, a denoised mark image dataset is output. Brightness-color correction is performed on the denoised mark image dataset to obtain a corrected leaf image dataset.

[0013] A unified spatial coordinate system is constructed, and spatial coordinate alignment is performed on all corrected leaf images in the corrected leaf image dataset based on the unified spatial coordinate system to obtain a coordinate-aligned image dataset; coordinate-aligned images belonging to the same grape leaf are screened from the coordinate-aligned image dataset, and the coordinate-aligned image dataset is grouped; a three-dimensional model of the corresponding grape leaf is constructed based on each group of coordinate-aligned images in the coordinate-aligned image dataset, and the three-dimensional models corresponding to all grape leaves are integrated to obtain a three-dimensional model dataset; image segmentation is performed on the coordinate-aligned image dataset to obtain a clear leaf image dataset.

[0014] Furthermore, the method of performing brightness-color correction on the denoised labeled image dataset includes:

[0015] Perform color space conversion on any denoised marked image in the denoised marked image dataset to obtain a color converted image; calculate the global brightness mean of any color converted image, and perform gamma correction on the color converted image based on the global brightness mean to obtain a gamma corrected image; detect the pixel value distribution of any gamma corrected image, set a high pixel value threshold, and determine the area in any gamma corrected image with a pixel value greater than the high pixel value threshold as an overexposed area; perform illumination decomposition on the overexposed area in any gamma corrected image to obtain the reflection partition and illumination partition of the overexposed area;

[0016] An attenuation function is constructed and used to adjust the pixel values ​​of the illumination partition; the adjusted illumination partition is recombined with the reflection partition to obtain the normal exposure area, and all overexposed areas of any gamma-corrected image are converted into normal exposure areas to obtain the normal exposure image; a saturation gain function is constructed in the saturation channel of the normal exposure image, and the saturation of the normal exposure image is adjusted based on the saturation gain function to obtain the saturation-adjusted image; a standard color card database is constructed, and any saturation-adjusted image is converted into a color space, and the color difference amplitude between any saturation-adjusted image and the standard image in the standard color card database is calculated, and the saturation-adjusted image is corrected for color deviation based on the color difference amplitude to obtain a color deviation-corrected image; all color deviation-corrected images are integrated to obtain a corrected leaf image dataset.

[0017] Furthermore, the method of performing image segmentation on the coordinate-aligned image dataset includes:

[0018] Extract edge features of any coordinate-aligned image in a coordinate-aligned image dataset, including leaf edge features and vein edge features; determine the leaf edge region and vein region in any coordinate-aligned image based on the leaf edge features and vein edge features; calculate the curvature of any edge point in the leaf edge region, perform concave point detection on all edge points on the leaf edge region based on the curvature, and obtain a concave point set; determine the overlapping region in the coordinate-aligned image based on the concave point set, call the three-dimensional model corresponding to the coordinate-aligned image in the three-dimensional model dataset, and use the three-dimensional model to perform projection matching with the overlapping region; set a projection deviation threshold, detect the projection deviation of the projection matching, and resample the leaf edge region if the projection deviation is greater than the preset projection deviation threshold;

[0019] The coordinate-aligned image is segmented based on the leaf edge area to obtain the leaf region image; a geometric constraint loss function is constructed based on all the coordinate-aligned images belonging to the same grape leaf, and the geometric constraint loss function is used as the constraint condition for region segmentation; the vein area in the leaf region image is enhanced, and a grape leaf morphology database is constructed at the same time, and the image enhancement scale is adjusted based on the grape leaf morphology database; a clear leaf image is obtained after region segmentation and image enhancement, and all clear leaf images are integrated to obtain a clear leaf image dataset.

[0020] Furthermore, the method of constructing a feature extraction model to extract features from the clear leaf image dataset includes:

[0021] A feature extraction model was constructed, using the YOLOv8 network model as the backbone network structure of the feature extraction model. The backbone network structure of the feature extraction model was a three-layer topology, including a shallow convolution layer, a deep network layer, and a group weight adjustment layer. A multi-scale feature extraction architecture was constructed and embedded into the backbone network structure of the feature extraction model. The feature extraction scale was adjusted based on the multi-scale feature extraction architecture. Any clear leaf image in the clear leaf image dataset was used as input data for the feature extraction model. Standard convolution processing was performed on the clear leaf image in the shallow convolution layer to output a preliminary processed feature map. The preliminary processed feature map was linearly transformed to obtain a simplified feature map. Disease feature extraction branches and growth condition feature extraction branches were added to extract features from the simplified feature map.

[0022] In the disease feature extraction branch, a multi-scale feature extraction architecture is used, and a grape leaf lesion morphology database is constructed. Based on this architecture and the grape leaf lesion morphology database, the lesion characteristics in the simplified feature map are identified. Pathology-related parameters in the simplified feature map are quantified, and the disease recognition capability of the disease feature extraction branch is enhanced based on these parameters.

[0023] In the growth feature extraction branch, the Hough transform algorithm is used to process the simplified feature map to obtain the vein direction vector and use this vector as a geometric constraint; the three-dimensional model dataset is called to extract the geometric parameters in the three-dimensional model dataset and normalize them; the two-dimensional image features in the simplified feature map are identified, and the normalized geometric parameters are spliced ​​with the two-dimensional image features to obtain the growth feature; in the deep network layer, the channel weights are output through global average pooling, and the feature extraction sensitivity is adjusted based on the channel weights; in the group weight adjustment layer, the weights of all disease condition features and all growth condition features are adjusted separately; all disease condition features and growth condition features belonging to the same clear leaf image are marked with group labels.

[0024] Furthermore, the method of performing cross-level feature fusion on all disease condition features includes:

[0025] All disease condition features are grouped based on grouping labels, and all disease condition features in each group are aligned to obtain aligned disease condition feature maps. All aligned disease condition feature maps in each group are classified by feature type, including shallow lesion texture feature maps and deep lesion semantic feature maps. A dynamic attention architecture is added to dynamically assign weights to shallow lesion texture feature maps and deep lesion semantic feature maps in the feature dimension based on the dynamic attention architecture, and weighted feature splicing is performed on the shallow lesion texture feature map and the deep lesion semantic feature map to obtain a fused disease condition feature.

[0026] The methods for hierarchical splicing and feature fusion of all growth characteristics include:

[0027] All growth characteristics are grouped based on the grouping labels, and all growth characteristics in each group are aligned to obtain an aligned growth characteristic map; all aligned growth characteristic maps in each group are classified into local growth characteristic maps and global growth characteristic maps; the vein direction vector is spliced ​​with the local growth characteristic map, and the contribution ratio of the vein direction vector to the local growth characteristic map is adjusted by weight to obtain a shallow vein characteristic map; the pre-trained recurrent neural network model is used to fuse the shallow vein characteristic map and the global growth characteristic map, and the fused growth characteristics are output.

[0028] Furthermore, the method of constructing the blade evaluation framework and adjusting the framework parameters of the blade evaluation framework in real time includes:

[0029] A parallel evaluation framework is constructed within the leaf evaluation framework, including a disease condition evaluation framework and a growth condition evaluation framework. The fused disease condition features and fused growth condition features are embedded in the disease condition evaluation framework and the growth condition evaluation framework as evaluation constraints for the two evaluation frameworks, respectively.

[0030] An image attention mechanism is introduced into the disease assessment framework and linked to the grape leaf lesion morphology database to dynamically enhance the response of key disease areas. Diseases are identified and a disease index equation is constructed based on the evaluation constraints of the disease assessment framework. The disease assessment results are calculated based on the disease index equation. A color feature recognition structure is constructed in the growth assessment framework, and a physiological indicator equation is constructed. The grape leaf morphology database is used as prior knowledge, and a geometric morphology indicator equation is constructed based on the evaluation constraints of the growth assessment framework and the grape leaf morphology database. A group of growth indicator equations is constructed based on the physiological indicator equation and the geometric morphology indicator equation, and the growth assessment results are calculated.

[0031] All framework parameters of the leaf evaluation framework are extracted, and the historical framework parameters of the leaf evaluation framework in the historical evaluation records are collected at the same time, and a parameter space is constructed based on all current framework parameters and historical framework parameters; an optimization algorithm is used to search for the optimal solution in the parameter space, and a leaf evaluation loss function is constructed based on the disease assessment results and the growth assessment results, and the leaf evaluation loss function is used as the fitness function of the optimization algorithm; the framework parameter combination with the largest fitness function value is taken as the optimal framework parameter combination, and a parameter combination snapshot library is constructed to store the optimal framework parameter combination and the historical optimal parameter combination; the resource occupancy rate of the leaf evaluation framework is monitored in real time, and a resource occupancy rate threshold is set. If the resource occupancy rate is greater than the preset resource occupancy rate threshold, other optimal framework parameter combinations are called from the parameter combination snapshot library.

[0032] Furthermore, the method of performing image evaluation on the clear leaf image dataset using the leaf evaluation framework includes:

[0033] Any clear leaf image in the clear leaf image dataset is used as input to the leaf assessment framework. The leaf assessment framework calls the fused disease condition features and fused growth condition features belonging to the clear leaf image and performs a disease condition assessment on the clear leaf image based on the disease condition assessment architecture. The clear leaf image is compared with the grape leaf lesion morphology database, focusing on the diseased area in the clear leaf image and quantifying the disease severity. The value of the disease index equation is calculated to obtain the disease condition assessment result, i.e., the disease severity assessment index.

[0034] The growth condition of the clear leaf image is evaluated based on a growth condition evaluation framework; the leaf color distribution in the clear leaf image is identified through the color feature recognition structure in the growth condition evaluation framework, and a standard color card database is called as prior knowledge; the color features of the grape leaves in the clear leaf image are quantified using a physiological index equation, and the physiological index is output; the clear leaf image is compared with a grape leaf morphology database, the growth morphology of the leaves in the clear leaf image is identified, and the growth geometric parameters corresponding to the clear leaf image in the evaluation constraints of the growth condition evaluation framework are combined; the growth degree of the grape leaves in the clear leaf image is quantified by calculating a geometric morphology index equation, and the geometric morphology index is output; the weights of the physiological index equation and the geometric morphology index equation are adjusted based on the growth condition index equation group, and the growth condition evaluation result, i.e., the growth condition evaluation index, is output.

[0035] Furthermore, the method of grading the disease degree and growth condition of the clear leaf image dataset includes:

[0036] Based on the disease degree assessment index and growth condition assessment index of any clear leaf image in the clear leaf image dataset, the clear leaf image is graded for disease degree and growth condition; a disease degree grading data table is constructed, and threshold intervals for different disease degrees are set based on the grape leaf lesion morphology database; the disease degree assessment index of any clear leaf image is compared with the disease degree grading data table, and the disease degree grade of the current clear leaf image is output;

[0037] By querying the preset grape leaf category database, a growth classification data table including all grape leaf varieties is constructed; at the same time, a three-level classification standard is constructed for the growth classification data table, and the classification interval of the three-level classification standard is adjusted based on different grape leaf varieties; the growth evaluation index of any clear leaf image is compared with the growth classification data table, and the growth classification of the current clear leaf image is output; the clear leaf image dataset is grouped based on the variety label, and all clear leaf images in each group are regrouped based on the disease severity classification and growth classification, and all grouped clear leaf images are labeled with the classification labels to obtain a graded grouped image dataset;

[0038] Based on the disease severity and growth status classification of each clear leaf image, a disease-growth status heat map was constructed, and different colors were used to represent the disease severity and growth status. At the same time, the variety type and related parameter information of the corresponding clear leaf image were annotated in the disease-growth status heat map. The disease-growth status heat maps of all clear leaf images were integrated to obtain a disease-growth status visualization set. The disease-growth status visualization set and the graded grouped image dataset were integrated to obtain the grape leaf grading results.

[0039] A computer vision-based AI grading and processing system for grape leaves, which is used to implement a computer vision-based AI grading and processing method for grape leaves, is characterized by comprising:

[0040] The data acquisition module acquires image data of grape leaves and integrates all image data to obtain a leaf image dataset; labels the leaf image dataset with variety labels based on grape leaf varieties to obtain a variety-labeled image dataset; and optimizes the variety-labeled image dataset to obtain a clear leaf image dataset;

[0041] The feature extraction module builds a feature extraction model to extract features from the clear leaf image dataset to obtain disease condition features and growth condition features; performs cross-level feature fusion on all disease condition features to obtain fused disease condition features; and performs hierarchical splicing feature fusion on all growth condition features to obtain fused growth condition features;

[0042] A framework construction module constructs a leaf evaluation framework based on the fused disease condition characteristics and the fused growth condition characteristics, and adjusts the framework parameters of the leaf evaluation framework in real time;

[0043] The image evaluation module uses a leaf evaluation framework to perform image evaluation on a clear leaf image dataset, obtains disease severity evaluation indicators and growth condition evaluation indicators, and performs disease severity classification and growth condition classification on the clear leaf image dataset based on the disease severity evaluation indicators and growth condition evaluation indicators to obtain grape leaf grading results; each module is connected to each other via wired and / or wireless means.

[0044] The technical effects and advantages of the computer vision-based AI grading method for grape leaves of the present invention are as follows:

[0045] By collecting grape leaf image data from multiple angles and classifying them based on varieties; the classified image data are filtered, noise suppressed, brightness-color corrected and segmented in succession to output a clear leaf image dataset, and a three-dimensional model corresponding to each grape leaf is constructed; the clear leaf image dataset is feature extracted with the assistance of the three-dimensional model, and the extracted features are fused to obtain fused disease condition features and fused growth condition features; a leaf evaluation framework is constructed and the framework parameters of the evaluation framework are adjusted in real time; the leaf evaluation framework is used to evaluate the clear leaf image dataset, and the disease degree evaluation index and growth condition evaluation index are output. Finally, the clear leaf images are graded based on the disease degree assessment index and the growth condition assessment index. Compared with the existing experience, in order to ensure that the image can be fully collected, this embodiment adopts multi-angle data collection, and at the same time builds a three-dimensional model to solve the occlusion problem in the image. The image data is precisely optimized to make the obtained image data clearer. Feature extraction is completed by building a feature extraction model, and the extracted features are fused across levels to obtain features that can fully represent the semantics of the image. Image evaluation is carried out by building an evaluation framework that can be optimized in real time, which ensures the evaluation efficiency and effect, and finally achieves accurate grape leaf grading based on the evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of a grape leaf AI grading method based on computer vision according to the present invention;

[0047] Figure 2 Schematic diagram of a grape leaf AI grading processing system based on computer vision according to the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Example 1

[0050] See also Figure 1 As shown, the grape leaf AI grading method based on computer vision described in this embodiment includes:

[0051] S1. Acquire grape leaf image data and integrate all image data to obtain a leaf image dataset; label the leaf image dataset with variety labels based on grape leaf varieties to obtain a variety-labeled image dataset; and optimize the variety-labeled image dataset to obtain a clear leaf image dataset;

[0052] S2. Construct a feature extraction model to extract features from the clear leaf image dataset to obtain disease characteristics and growth characteristics; perform cross-level feature fusion on all disease characteristics to obtain fused disease characteristics; and perform hierarchical splicing feature fusion on all growth characteristics to obtain fused growth characteristics.

[0053] S3. Constructing a leaf evaluation framework based on the fused disease characteristics and the fused growth characteristics, and adjusting the framework parameters of the leaf evaluation framework in real time;

[0054] S4. Use the leaf evaluation framework to perform image evaluation on the clear leaf image dataset, and obtain the disease degree evaluation index and growth condition evaluation index respectively. Based on the disease degree evaluation index and growth condition evaluation index, the clear leaf image dataset is graded in terms of disease degree and growth condition respectively to obtain the grape leaf grading results.

[0055] The image data of grape leaves are images of grape leaves from various angles captured by a preset high-precision camera, covering all angles of each grape leaf to ensure that the image data can include the details and completeness of all areas of the grape leaf, avoiding the problem of missing parts of the image due to leaf occlusion. At the same time, some disease characteristics or growth characteristics on grape leaves can only be seen at specific angles, so multi-angle collection of grape leaf images has become a necessary condition.

[0056] Methods for labeling the grape leaf image dataset with variety labels include:

[0057] Based on the existing grape leaf-related data, a grape leaf category database is constructed, which includes all types of grape leaves and corresponding images. The variety label of each grape leaf is obtained based on the grape leaf category database, and each image in the leaf image dataset is compared with the grape leaf category database. Based on the comparison results, the leaf image dataset is classified into varieties and each type of image in the leaf image dataset is marked with a variety label using the variety label.

[0058] The methods for image optimization of the variety mark image dataset include:

[0059] Design a dual-channel filtering architecture, including a high-density noise filtering branch and a mixed noise filtering branch. The two branches process any variety mark image in the variety mark image dataset in parallel. By designing a parallel processing dual-channel filtering architecture, any variety mark image is efficiently filtered. The high-density noise filtering branch is used to remove the high-density salt and pepper noise in the image, and the mixed noise filtering branch is used to remove the complex mixed noise in the image. At the same time, the image details are preserved to prevent the loss of image detail features. Construct a filtering sliding window, calculate the noise density within the filtering sliding window in the high-density noise filtering branch, adjust the filtering sliding window size based on the noise density, and first slide the filtering sliding window with a larger window size, for example, the window size is The filtering sliding window; when the noise density in the window is detected to be less than When the filter sliding window size is gradually reduced, the detection accuracy is improved; in this embodiment, the window size change interval of the filter sliding window is ;The calculation formula of noise density is: ;in represents the noise density; Indicates the window size of the filtering sliding window; Represents the number of noise points in the filtering sliding window; use the filtering sliding window to traverse any variety mark image, and filter the image area within the filtering sliding window, use the hierarchical median filtering algorithm to filter the image area of ​​any variety mark image within the filtering sliding window, and give priority to removing high-density noise in the variety mark image.

[0060] In the mixed noise filtering branch, any variety mark image is gridded based on the filtering sliding window, the variance of all pixels in any grid is calculated, and the number of grid divisions is adjusted based on the size of the variety mark image. In this embodiment, the Grid division of specifications; detect the pixel value of each pixel in any grid and calculate the variance based on the pixel value; set the variance difference threshold and compare the variance of any grid with the variance of the adjacent grid. If the pixel variance difference of the pixels in the two grids is greater than the preset variance difference threshold, the grid with the larger variance between the two grids is judged as a noise grid, and noise suppression is performed on all noise grids. In the field of image processing, the pixel variance of the noise area in the image is significantly higher than that of the normal image area. The noise grid is judged based on the variance difference threshold and all noise grids are denoised using the soft threshold denoising method in wavelet decomposition to suppress noise; the denoised labeled image dataset is output after processing with a dual-channel filtering architecture; the denoised labeled image dataset is brightness-color corrected to obtain a corrected leaf image dataset. By performing brightness-color correction on any denoised labeled image in the denoised labeled image dataset, the image blur caused by uneven lighting or overexposure is eliminated, ensuring the authenticity of the color of the corresponding grape leaf image, which is convenient for the accuracy of subsequent feature extraction based on color.

[0061] A unified spatial coordinate system is constructed, and spatial coordinate alignment is performed on all the corrected leaf images in the corrected leaf image dataset based on the unified spatial coordinate system to obtain a coordinate-aligned image dataset. Taking the unified spatial coordinate system as a reference, a corrected leaf image is randomly selected as a standard image, and all the corrected leaf images in the corrected leaf dataset are processed using an affine transformation algorithm to achieve global mapping of coordinates. For example, an affine transformation matrix is ​​constructed through affine transformation, and the side view image is aligned with the standard image through operations such as rotation or translation based on the affine matrix; coordinate-aligned images belonging to the same grape leaf are screened from the coordinate-aligned image dataset, and the coordinate-aligned image dataset is grouped. When collecting image data of the same grape leaf, a unique feature code is set for each image and the feature code is associated with the grape leaf. Grape leaf matching, so the above screening process is based on the feature code to screen the coordinate alignment image data set; each group contains coordinate alignment images of the same grape leaf at different angles; based on each group of coordinate alignment images in the coordinate alignment image data set, a three-dimensional model of the corresponding grape leaf is constructed, and the three-dimensional models corresponding to all grape leaves are integrated to obtain a three-dimensional model data set. The SFM algorithm is used to generate a three-dimensional point cloud for any group of coordinate alignment images, and the three-dimensional model corresponding to the grape leaf is constructed based on the three-dimensional point cloud; the coordinate alignment image data set is segmented to obtain a clear leaf image data set. By performing image segmentation on any coordinate alignment image, the main part of the leaf is segmented from the background, thereby improving the clarity of the main part and preventing interference from the background part that causes image blur.

[0062] Methods for performing brightness-color correction on denoised labeled image datasets include:

[0063] Perform color space conversion on any denoised marked image in the denoised marked image dataset to obtain a color converted image, wherein the denoised marked image is an RGB color space image. In this embodiment, all denoised marked images are converted into HSV color space images. Since the HSV color space image includes a hue channel, a saturation channel, and a brightness channel, it can automatically decouple the brightness and color characteristics of the image, providing an independent operation space for subsequent brightness and color correction. Calculate the global brightness mean of any color converted image, and perform gamma correction on the color converted image based on the global brightness mean to obtain a gamma corrected image, wherein the calculation formula of the adaptive gamma value is: ;in, Indicates the gamma value; Represents the global brightness mean; based on the gamma value, nonlinear mapping is performed on the brightness channel of any color conversion image to perform gamma correction. The calculation formula is: ;in Indicates that the brightness channel of the color conversion image is adjusted using the gamma value. The brightness value of the gamma-corrected image obtained by gamma correction of the color conversion image is represented; the gamma value is adjusted by global brightness to give priority to enhancing the details of the dark area in the image and improving the visibility of the dark area; the pixel value distribution of any gamma-corrected image is detected, and a high pixel value threshold is set. The area with a pixel value greater than the high pixel value threshold in any gamma-corrected image is determined as an overexposed area. In this embodiment, the high pixel value threshold is set to ; Mark the over-exposed area whose pixel value is greater than the high pixel value threshold to avoid distortion of normal area caused by subsequent operations; Perform illumination decomposition on the over-exposed area in any gamma-corrected image to obtain the reflection partition and illumination partition of the over-exposed area. It should be noted that, using The algorithm decomposes the overexposed areas and suppresses brightness only for the illumination partitions generated by the illumination decomposition, avoiding the loss of details in the overexposed areas during the gamma correction process.

[0064] Construct an attenuation function and use it to adjust the pixel value of the illumination partition. The calculation formula of the attenuation function is: ;in, Indicates that the coordinates in the adjusted lighting partition are The pixel value of the pixel point; Indicates that the coordinates in the illumination partition before adjustment are The pixel value of the pixel point; Represents a correction coefficient, which is dynamically adjusted based on the brightness variance in the illumination partition. In this embodiment, the initial value is ; Recombine the adjusted illumination partition and reflection partition to obtain the normal exposure area. After converting all overexposed areas of any gamma-corrected image into normal exposure areas, a normal exposure image is obtained. The calculation formula for the recombinant is: ;in, Represents a reflective partition; Represents the adjusted lighting partition; represents the normally exposed area; combining gamma correction with illumination decomposition achieves differentiated processing of overexposed and underexposed areas, adjusting brightness while ensuring the preservation of local details. Compared with traditional brightness adjustment methods, it is more efficient and stable; constructing a saturation gain function in the saturation channel of the normally exposed image, and adjusting the saturation of the normally exposed image based on the saturation gain function to obtain a saturation-adjusted image. The calculation formula of the saturation gain function is: ;in, Indicates the mean saturation value of the normally exposed image; Indicates the saturation standard deviation of the normally exposed image; Represents the brightness of any normally exposed image; express The corresponding average brightness of the normally exposed image; It represents the adjustment coefficient, which changes with the local brightness gradient. In this embodiment, the initial value is ; The saturation gain function dynamically enhances the saturation of relatively low brightness areas, suppresses oversaturation in relatively high brightness areas, and avoids color distortion; a standard color card database is constructed, any saturation-adjusted image is converted into a color space, the color difference amplitude between any saturation-adjusted image and the standard image in the standard color card database is calculated, and the saturation-adjusted image is corrected for color deviation based on the color difference amplitude to obtain a color deviation-corrected image, wherein the color space is the LAB color space, and the image in this color space includes Channel and Channel, by compensating the two channels to achieve color deviation correction; based on existing biological data, a standard color card database is constructed, including the color standard of the corresponding grape leaf variety, which is the standard LAB value of the grape leaf image; the color difference amplitude is calculated based on the color standard and the LAB value of the current image using the CIE2000 color difference formula; the calculation formula for color deviation correction is: ; ;in, and Respectively Channel compensation and The compensation amount of the channel; and They represent the adjustment coefficients obtained based on the standard color card database, for example , focusing on red-green balance; , focusing on the contrast between yellow and blue; Represents the chromatic aberration direction angle obtained through measurement; all color deviation corrected images are integrated to obtain the corrected leaf image dataset.

[0065] Methods for image segmentation on coordinate-aligned image datasets include:

[0066] Extract the edge features of any coordinate-aligned image in the coordinate-aligned image dataset, including leaf edge features and vein edge features, wherein the pre-trained segmentation network model is combined with the edge detection Canny operator to extract the edge features of any coordinate-aligned image, and identify the boundary area in the coordinate-aligned image, namely the leaf edge features and vein edge features; the leaf edge feature refers to the edge corresponding to the outer contour of the leaf, and the vein edge feature refers to the contour of the vein in the leaf; based on the leaf edge feature and the vein edge feature, the leaf edge area and vein area in any coordinate-aligned image are determined respectively; the curvature of any edge point in the leaf edge area is calculated, and concave point detection is performed on all edge points on the leaf edge area based on the curvature to obtain a concave point set, wherein the curvature mathematical calculation formula is used to calculate any one The curvature of the edge points, all local extreme points where the curvature of the edge points mutates are determined as concave points; the overlapping area in the coordinate alignment image is determined based on the concave point set, the three-dimensional model corresponding to the coordinate alignment image in the three-dimensional model data set is called, and the three-dimensional model is used to perform projection matching with the overlapping area, wherein the ray method is used to emit rays to the inside of the blade based on the concave points. If there are three or more intersections in the ray path, the intersection is determined as an overlapping concave point; the area surrounded by all overlapping concave points is regarded as the overlapping area; the three-dimensional model point cloud is projected onto a two-dimensional plane to eliminate the topological distortion caused by a single perspective and ensure the topological consistency of the edges at multiple angles; a projection deviation threshold is set to detect the projection deviation of the projection matching. If the projection deviation is greater than the preset projection deviation threshold, the edge area of ​​the blade is resampled. In this embodiment, the projection deviation threshold is set to Resampling is used to avoid the uncertainty of single sampling and the occlusion caused by shadows, ensuring accurate sampling of the leaf edge area.

[0067] The coordinate-aligned image is segmented based on the leaf edge region to obtain a leaf region image, wherein the region segmentation is performed using a pre-trained segmentation network model. In this embodiment, a U-Net network model is used for region segmentation. A geometric constraint loss function is constructed based on all coordinate-aligned images belonging to the same grape leaf, and the geometric constraint loss function is used as a constraint condition for region segmentation. The calculation formula of the geometric constraint loss function is: ;in, represents the contour symmetry loss function; represents the curvature continuity loss function; and Respectively and The weight, in this embodiment , ; The parameters of the segmentation network model are optimized through back propagation using the geometric constraint loss function to improve the model performance and segmentation accuracy; the vein area in the leaf region image is enhanced, and a grape leaf morphology database is constructed at the same time, and the image enhancement scale is adjusted based on the grape leaf morphology database, wherein a pre-trained convolutional neural network model is used for image enhancement, and morphological prior knowledge such as vein width distribution or bifurcation angle is obtained based on the grape leaf morphology database, and a grape leaf morphology database is constructed based on these morphological prior knowledge to adjust the parameters of the convolutional neural network model; a clear leaf image is obtained after region segmentation and image enhancement, and a clear leaf image dataset is obtained by integrating all clear leaf images.

[0068] The methods for constructing a feature extraction model to extract features from a clear leaf image dataset include:

[0069] Construct a feature extraction model and use the YOLOv8 network model as the backbone network structure of the feature extraction model. It should be noted that the YOLOv8 network model is an efficient real-time target detection model. At the same time, due to the core architecture of the model, that is, The architecture enables the YOLOv8 network model to also have powerful feature extraction capabilities; the backbone network structure of the feature extraction model is a three-layer topology structure, including a shallow convolution layer, a deep network layer and a group weight adjustment layer. The shallow convolution layer of the three-layer topology structure in this embodiment is used to perform feature extraction operations, and the deep network layer and the group weight adjustment layer are used to adjust the feature extraction process and weight the output features; a multi-scale feature extraction architecture is constructed and embedded in the backbone network structure of the feature extraction model, and the feature extraction scale is adjusted based on the multi-scale feature extraction architecture, wherein the multi-scale feature extraction architecture realizes adaptive adjustment of the feature extraction scale by dynamically adjusting the specifications of the convolution kernel; for example, when the convolution kernel specification is When , it is mainly used to capture microscopic features in the image, such as lesion spots and other features; when the convolution kernel specification is When , it is mainly used to extract medium-scale features, such as the entire leaf vein or strip-shaped diseased spots; when the convolution kernel specification is It is mainly used to extract the features of the global scale of the image, such as the distribution density of the lesion area or the growth trend of the leaf; in this embodiment, the initial value of the convolution kernel is , adjust the convolution kernel specifications according to the specific situation; use any clear leaf image in the clear leaf image dataset as the input data of the feature extraction model; perform standard convolution processing on the clear leaf image in the shallow convolution layer, output the preliminary processed feature map, and perform linear transformation on the preliminary processed feature map to obtain a simplified feature map, where standard convolution processing refers to depthwise separable convolution, which has lower computational cost and higher computational efficiency than traditional convolution; add disease feature extraction branches and growth condition feature extraction branches to extract features from the simplified feature map.

[0070] In the disease feature extraction branch, a multi-scale feature extraction architecture is called, and a grape leaf lesion morphology database is constructed at the same time. The lesion characteristics in the simplified feature map are identified based on the multi-scale feature extraction architecture and the grape leaf lesion morphology database. The grape leaf lesion morphology database is constructed based on querying biological prior knowledge, including the status of lesions on grape leaves of each variety. The feature extraction scale is adjusted using the multi-scale feature extraction architecture, and the simplified feature map is compared with the grape leaf lesion morphology database to accurately identify the lesion characteristics in the simplified feature map. The parameters related to the lesion condition in the simplified feature map are quantified, and the disease recognition ability of the disease condition feature extraction branch is enhanced based on the parameters. The gray level co-occurrence matrix is ​​used to quantify the parameters, including contrast, energy or correlation. For example, contrast reflects the grayscale difference between lesions and healthy areas, and energy can represent the uniformity of texture in the simplified feature map. The important disease condition parts in the simplified feature map are weighted based on the quantized parameters, thereby improving the sensitivity of the disease condition feature extraction branch to diseases.

[0071] In the growth feature extraction branch, the Hough transform algorithm is used to process the simplified feature map to obtain the vein direction vector and use the vector as a geometric constraint. It should be noted that the Hough transform algorithm is used to detect geometric shapes in the image. In this embodiment, it is used to perform straight line detection, identify the vein direction, and map the identification result to the vector space to obtain the vein direction vector; based on the vein direction vector, the spatial extraction range of the growth feature extraction branch is constrained, such as the vein bifurcation angle threshold. If the threshold is exceeded, feature extraction will not be performed; it is used to limit the search area, reduce the calculation amount of invalid features, and improve the efficiency of feature extraction; call the three-dimensional model data set, extract the geometric parameters in the three-dimensional model data set and normalize them, where the geometric parameters include parameters such as curvature, surface area and volume ratio. The purpose of normalizing the geometric parameters is to eliminate dimensional differences and ensure the scale consistency of subsequent splicing; identify the two-dimensional image features in the simplified feature map, and splice the normalized geometric parameters with the two-dimensional image features to obtain growth characteristics, where the two-dimensional image features refer to features such as leaf color features, leaf texture features and leaf shape features that are intuitively reflected in the simplified feature map; in the channel dimension, the normalized geometric parameters are spliced ​​with the two-dimensional image features as additional channels, which enhances the output growth characteristics' ability to describe spatial attributes; in the deep network layer, the channel weights are output through global average pooling, and the feature extraction sensitivity is adjusted based on the channel weights Degree, through global average pooling, a statistic is provided for each output channel, namely the channel weight; the channel weight is used to weight the feature extraction channel, for example, the high-frequency texture channel response is enhanced, the low-frequency channel area is suppressed, and the sensitivity of feature extraction to the key channel is adjusted to achieve more accurate feature extraction; in the group weight adjustment layer, the weights of all disease condition features and all growth condition features are adjusted respectively, and the weights of the more important features among all features are increased to facilitate subsequent image evaluation. For example, since the contrast reflects the disease condition, the features with higher contrast parameters in the disease condition features are given greater weights, and the features with lower contrast parameters are given smaller weights; all disease condition features and growth condition features belonging to the same clear leaf image are marked with group labels. In the process of feature extraction for each clear leaf image, the features obtained are marked with group labels to facilitate subsequent grouping and feature fusion within the group.

[0072] The methods for cross-level feature fusion of all disease condition features include:

[0073] All disease condition features are grouped based on grouping labels, and all disease condition features in each group are feature aligned to obtain aligned disease condition feature maps, where feature alignment refers to aligning feature channels to eliminate channel dimension differences for the convenience of subsequent feature splicing; each group of disease condition features represents features belonging to the same clear leaf image, avoiding the semantic inconsistency caused by splicing features from different images; all aligned disease condition feature maps in each group are classified by feature type, including shallow lesion texture feature maps and deep lesion semantic feature maps, and classified using a pre-trained classifier, where the shallow lesion texture feature map represents a clearly visible lesion area, which is used to reflect the local lesion condition; the deep lesion semantic feature map represents a clearly visible lesion area, which is used to reflect the local lesion condition; the deep lesion semantic feature map represents a clearly visible lesion area, which is used to reflect the local lesion condition. The lesion semantic feature map represents semantic information such as the lesion diffusion trend, such as the distribution of lesions, and is used to reflect the global changes in the lesion area. A dynamic attention architecture is added to dynamically allocate weights to the shallow lesion texture feature map and the deep lesion semantic feature map in the feature dimension based on the dynamic attention architecture, and weighted feature splicing is performed on the shallow lesion texture feature map and the deep lesion semantic feature map to obtain a fused disease condition feature. Global average pooling is performed on the shallow lesion texture feature map and the deep lesion semantic feature map respectively, and a channel description vector is output. The fully connected layer based on the dynamic attention architecture learns the dependency relationship between channels and outputs a weight vector. Weighted feature splicing is performed based on the weight vector. The calculation formula for weighted feature splicing is: ;in, Indicates the characteristics of the fusion disease condition; Represents the texture feature map of shallow lesions; Represents the semantic feature map of deep lesions; The weight vector representing the texture feature map of shallow lesions; Represents the weight vector of the deep lesion semantic feature map; in this embodiment , Different disease types have different degrees of dependence on texture or semantic features. For example, downy mildew requires strengthening texture features, while powdery mildew requires strengthening the semantic features of diffusion trends.

[0074] The methods for hierarchical splicing and feature fusion of all growth characteristics include:

[0075] All growth characteristics are grouped based on the grouping labels, and all growth characteristics in each group are feature aligned to obtain an aligned growth characteristic map; all aligned growth characteristic maps in each group are classified into local growth characteristic maps and global growth characteristic maps, where feature alignment of growth characteristics also refers to aligning feature channels; local growth characteristic maps represent feature maps that can express local characteristics such as the area covered by a single leaf vein or the angle of leaf vein bifurcation, which are used to reflect the microscopic growth state; global growth characteristic maps refer to semantic information that can represent global growth trends, such as features such as leaf area, which are used to reflect macroscopic growth trends; the leaf vein direction vector is spliced ​​with the local growth characteristic map, and the contribution ratio of the leaf vein direction vector to the local growth characteristic map is adjusted by weight to obtain a shallow leaf vein feature map, where the calculation formula for splicing the leaf vein direction vector and the local growth characteristic map is: ;in, Represents the characteristic diagram of shallow leaf veins; represents the vein direction vector; Characteristic map showing local growth conditions; and Respectively represent the contribution weights of the vein direction vector and the local growth characteristic map. In this embodiment, and The initial values ​​are set to and Since the contribution of leaf veins to growth varies at different growth stages, and The value of is adjusted; the pre-trained recurrent neural network model is used to fuse the shallow vein feature map and the global growth feature map, and the fused growth feature is output. The gated recurrent unit in the pre-trained recurrent neural network model captures the dynamic correlation between the shallow vein feature map and the global growth feature map. The output fused growth feature reflects the growth trend and local changes of the leaf at a certain growth stage.

[0076] Methods for constructing a blade evaluation framework and adjusting the parameters of the blade evaluation framework in real time include:

[0077] A parallel evaluation architecture is constructed in the leaf evaluation framework, including a disease condition evaluation architecture and a growth condition evaluation architecture. The disease condition evaluation architecture and the growth condition evaluation architecture are in a parallel relationship, and the input data is processed at the same time to improve computational efficiency. The fused disease condition features and the fused growth condition features are embedded in the disease condition evaluation architecture and the growth condition evaluation architecture as evaluation constraints of the two evaluation architectures, respectively. When an image is input, the fused disease condition features and the fused growth condition features become reference conditions for evaluating the image. Each time an evaluation is performed, the features corresponding to the input image are extracted to assist in image evaluation.

[0078] The image attention mechanism is introduced into the disease condition assessment framework and associated with the grape leaf lesion morphology database to dynamically enhance the response of key disease areas. The lesion area in the input image is identified based on the grape leaf lesion morphology database, and the image attention mechanism is used to focus on the lesion area. The Grad-CAM visualization algorithm is used to enhance the recognition speed of key disease areas. Key disease areas refer to image areas such as high-density lesion areas that can clearly reflect the degree of disease. Diseases are identified and a disease index equation is constructed based on the evaluation constraints of the disease condition assessment framework. The disease condition assessment results are calculated based on the disease index equation. The disease index equation is constructed based on the parameters related to the disease condition in the fusion of disease condition features, such as contrast and lesion area, which are used to quantify the degree of disease and output a disease degree assessment index. A color feature recognition structure is constructed in the growth condition assessment framework, and a growth condition evaluation framework is constructed. The physiological indicator equation is used to quantify the information that can be reflected by the color of the grape leaves in the input image, such as color features such as chlorophyll content, which can reflect the current growth status of the grape leaves at the nutrient level; the grape leaf morphology database is used as prior knowledge, and a geometric morphology indicator equation is constructed based on the evaluation constraints of the growth evaluation architecture and the grape leaf morphology database, wherein the geometric morphology indicator equation is used to quantify the growth status of the grape leaves at the morphological level, and at the same time combines the relevant geometric parameters in the fusion growth characteristics to reflect the growth status based on the morphological changes of leaf growth; a growth indicator equation group is constructed based on the physiological indicator equation and the geometric morphology indicator equation, and the growth evaluation result is calculated, wherein the constructed growth indicator equation group is used to adjust the importance ratio of the physiological indicator equation and the geometric morphology indicator equation, and output the growth evaluation index that conforms to the growth stage of the grape leaves in the current clear leaf image.

[0079] All framework parameters of the leaf evaluation framework are extracted, and the historical framework parameters of the leaf evaluation framework in the historical evaluation records are collected. The parameter space is constructed based on all current framework parameters and historical framework parameters, where the framework parameters include parameters such as learning rate, weight and recognition accuracy of the disease evaluation framework and growth evaluation framework; the optimization algorithm is used to search for the optimal solution in the parameter space, and the leaf evaluation loss function is constructed based on the disease evaluation results and the growth evaluation results. The leaf evaluation loss function is used as the fitness function of the optimization algorithm, where the optimization algorithm is the particle swarm algorithm, which is widely used to search for the optimal solution in the solution space; any parameter combination is defined as an individual, and all parameter combinations are defined as a population; the calculation formula of the leaf evaluation loss function is: ;in, Represents the function value of the leaf evaluation loss function; It represents the evaluation result of the disease situation, that is, the value of the disease index equation; It represents the growth evaluation result, that is, the value of the growth indicator equation group; and Respectively represent the weights of the disease assessment results and the growth assessment results. In this embodiment , ; The framework parameter combination with the largest fitness function value is taken as the optimal framework parameter combination, and a parameter combination snapshot library is constructed to store the optimal framework parameter combination and the historical optimal parameter combination. When the preset maximum number of iterations is reached, the solution with the largest fitness function value at this time is taken as the optimal solution. The constructed parameter combination snapshot library stores the historical and current optimal parameter combinations, which is convenient for directly calling the parameter combination from the parameter combination snapshot library in the future, thereby improving the optimization efficiency. The resource occupancy rate of the blade evaluation framework is monitored in real time, and a resource occupancy rate threshold is set. If the resource occupancy rate is greater than the preset resource occupancy rate threshold, other optimal framework parameter combinations are called from the parameter combination snapshot library. When the resource occupancy rate of the blade evaluation framework increases, if it exceeds the preset resource occupancy rate threshold, it means that the computing resources are overloaded at this time. If the framework parameter combination is not adjusted, performance degradation or reduced task processing efficiency may occur.

[0080] The image evaluation methods for the clear leaf image dataset using the leaf evaluation framework include:

[0081] Any clear leaf image in the clear leaf image dataset is used as the input of the leaf evaluation framework; the leaf evaluation framework calls the fused disease condition features and fused growth condition features belonging to the clear leaf image, and performs disease condition evaluation on the clear leaf image based on the disease condition evaluation architecture, wherein the fused disease condition features and fused growth condition features corresponding to the input clear leaf image are called to ensure consistency in the overall image evaluation process, ensuring that there is no mismatch between the input image and the called features; the clear leaf image is compared with the grape leaf lesion morphology database, the diseased area in the clear leaf image is focused on and the degree of disease is quantified, the image of the corresponding lesion morphology recorded in the grape leaf lesion morphology database is compared with the clear leaf image to be evaluated, and the degree of disease is quantified by calculating the value of the disease index equation of the lesion area in the image; the value of the disease index equation is calculated to obtain the disease condition evaluation result, that is, the disease degree evaluation index, wherein the calculation formula of the disease index equation is: ;in, Represents the disease severity assessment index of any clear leaf image; Represents the total area of ​​grape leaves in any clear leaf image; Represents the area of ​​any lesion on a grape leaf in any clear leaf image; Indicates the severity coefficient of the disease spot. The value of this coefficient is adjusted according to the specific disease situation. For example, when When , it indicates that the epidermal cells are damaged; Represents the sharpness coefficient of the lesion edge, which is obtained by extracting the gradient of the lesion edge in the image through the Canny operator. The value range is , used to reflect the spread trend of lesions; The time when the lesion was first discovered is obtained by finding the timestamp of the first appearance of the lesion in the clear leaf image of the same grape leaf as the current clear leaf image; Indicates the timestamp of the current clear leaf image; Represents the disease spread rate, which is obtained by querying the grape leaf lesion morphology database.

[0082] The growth condition of the clear leaf image is evaluated based on the growth condition evaluation framework. The color distribution of the leaf in the clear leaf image is identified by the color feature recognition structure in the growth condition evaluation framework, and the standard color card database is called as prior knowledge. It should be noted that the color distribution in the clear leaf image is identified based on the standard color card database. The color characteristics of the grape leaves in the clear leaf image are quantified using the physiological indicator equation, and the physiological indicators are output. The calculation formula of the physiological indicator equation is: ;in, represents the physiological index of any clear leaf image, which in this embodiment represents the chlorophyll content; represents the brightness component, which in this embodiment refers to the brightness component of the LAB color space and is used to represent the overall brightness; Represents the red and green axis components; Indicates the yellow-blue axis component; in this embodiment and Represents the LAB color space Channel and The color component of the channel; Represents a constant greater than zero, which is used as a compensation term. In this embodiment, the value is ; 、 and Respectively 、 and The weight coefficient is determined by multiple linear regression analysis. In this embodiment , , ; Compare the clear leaf image with the grape leaf morphology database to identify the growth morphology of the leaf in the clear leaf image, and combine the growth geometric parameters corresponding to the clear leaf image in the evaluation constraint conditions of the growth evaluation framework, and identify the grape leaf morphology in the clear leaf image by comparing it with the grape leaf morphology recorded in the grape leaf morphology database. At the same time, combine the geometric parameters such as leaf area or vein bifurcation angle in the fusion growth characteristics to accurately identify the grape leaf morphology in the clear leaf image; quantify the growth degree of the grape leaf in the clear leaf image by calculating the geometric morphology index equation, and output the geometric morphology index, wherein the calculation formula of the geometric morphology index equation is: ;in, Represents the geometric morphological index of any clear leaf image; Indicates the size of any pixel in any clear leaf image; Indicated in size The number of pixels required to cover the target shape is calculated. In this embodiment, each pixel is regarded as a box by the box counting method, and the box is used to cover the grape leaf area in the clear leaf image. The weights of the physiological indicator equation and the geometric morphology indicator equation are adjusted based on the growth indicator equation group, and the growth evaluation result, i.e., the growth evaluation index, is output. In this embodiment, the weights in the growth indicator equation group are adjusted based on the growth stage of the grape leaves in the clear leaf image. In this embodiment, the initial weight value of the physiological indicator equation is set to , the initial weight value of the geometric morphology index equation is set to ; Since the importance of chlorophyll content and growth morphology of grape leaves is different at different growth stages, for example, the morphology does not change much in the early growth stage, but the chlorophyll content is higher at this time, so the weight value of the physiological indicator equation is increased; the values ​​of the physiological indicator equation and the geometric morphology indicator equation are weighted and summed to obtain the growth evaluation index.

[0083] Methods for grading the disease severity and growth status of clear leaf image datasets include:

[0084] Based on the disease degree evaluation index and growth condition evaluation index of any clear leaf image in the clear leaf image data set, the clear leaf image is graded for disease degree and growth condition; a disease degree grading data table is constructed, and threshold intervals of different disease degrees are set based on the grape leaf lesion morphology database. It should be noted that the grape leaf lesion morphology database is used as prior knowledge, and a five-level disease grade is constructed in the disease degree grading data table. In this embodiment, the threshold interval of the disease degree evaluation index corresponding to the first-level disease grade is The threshold interval of the disease severity assessment index corresponding to the second-level disease level is ; The threshold interval of the disease severity assessment index corresponding to the third level of disease is The threshold interval of the disease severity assessment index corresponding to the fourth level of disease is ; The threshold interval of the disease severity assessment index corresponding to the five-level disease level is ; Compare the disease degree assessment index of any clear leaf image with the disease degree grading data table, output the disease degree grading of the current clear leaf image, and judge the disease degree grade of any clear leaf image based on the threshold interval and disease degree assessment index of the disease degree grading data table.

[0085] By querying the preset grape leaf category database, a growth classification data table including all grape leaf varieties is constructed. Since the growth cycle or growth stage of each variety of grape leaves is different, in order to ensure the rigor of the classification, the grape leaf category database is used as a theoretical basis. The database records the biological information of each variety of grape leaves. At the same time, a three-level classification standard is constructed for the growth classification data table. The classification interval of the three-level classification standard is adjusted based on different grape leaf varieties. In this embodiment, the three-level classification standard includes level one, level two and level three. Among them, when the growth evaluation index When the growth evaluation index is When the growth evaluation index is It is judged as grade 3, namely growth retardation; Represents the mean value of the growth evaluation index of all clear leaf images; Represents the standard deviation of the growth evaluation index of all clear leaf images; Represents a variety compensation item, which takes different values ​​for different varieties of grape leaves and is obtained by querying the grape leaf category database. In this embodiment, The initial value is set to ; Compare the growth evaluation index of any clear leaf image with the growth classification data table, and output the growth classification of the current clear leaf image; Group based on the variety labels in the clear leaf image data set, and regroup all clear leaf images in each group based on the disease severity classification and growth classification, and mark all grouped clear leaf images with classification labels, and divide all clear leaf images into large groups based on the variety labels, and further subdivide the large groups based on the disease severity classification and growth classification, so as to facilitate subsequent inquiries on the classification of a certain variety of grape leaves.

[0086] A disease-growth condition heat map was constructed based on the disease severity and growth condition classification of each clear leaf image. Different colors were used to represent the disease severity and growth condition. The disease-growth condition heat map was superimposed on the original clear leaf image. The disease severity of the diseased area was represented by gradient red at different levels, and the change in growth condition was represented by gradient yellow and blue. At the same time, the variety type and related parameter information of the corresponding clear leaf image were annotated in the disease-growth condition heat map. The variety type, disease classification, growth condition classification, geometric parameters and other related parameter information were marked in the disease-growth condition heat map in the form of labels. The disease-growth condition heat maps of all clear leaf images were integrated to obtain a disease-growth condition visualization set. The classification of each clear leaf image was visualized to facilitate direct viewing by technical personnel and improve efficiency in industrial production. The disease-growth condition visualization set and the graded grouped image dataset were integrated to obtain the grape leaf grading results.

[0087] This embodiment collects grape leaf image data from multiple angles and classifies them based on varieties; performs filtering, noise suppression, brightness-color correction, and image segmentation on the classified image data in succession to output a clear leaf image dataset, and simultaneously constructs a three-dimensional model corresponding to each grape leaf; extracts features from the clear leaf image dataset with the assistance of the three-dimensional model, and fuses the extracted features to obtain fused disease condition features and fused growth condition features; constructs a leaf evaluation framework and adjusts the framework parameters of the evaluation framework in real time; uses the leaf evaluation framework to perform image evaluation on the clear leaf image dataset, and outputs disease degree evaluation indicators and growth condition evaluation indicators. evaluation index, and finally graded the clear leaf images based on the disease degree evaluation index and the growth condition evaluation index; compared with existing experience, in order to ensure that the image can be fully collected, this embodiment adopts multi-angle data collection, and at the same time builds a three-dimensional model to solve the occlusion problem in the image; precise image optimization is implemented for the image data to make the obtained image data clearer; feature extraction is completed by constructing a feature extraction model, and the extracted features are cross-level feature fusion to obtain features that can fully represent the image semantics; image evaluation is performed by constructing an evaluation framework that can be optimized in real time, which ensures evaluation efficiency and effect, and finally achieves accurate grape leaf grading based on the evaluation results.

[0088] Example 2

[0089] See also Figure 2 As shown, for the parts not described in detail in this embodiment, please refer to the description of Example 1. A computer vision-based grape leaf AI grading processing system is provided, comprising:

[0090] The data acquisition module acquires image data of grape leaves and integrates all image data to obtain a leaf image dataset; labels the leaf image dataset with variety labels based on grape leaf varieties to obtain a variety-labeled image dataset; and optimizes the variety-labeled image dataset to obtain a clear leaf image dataset;

[0091] The feature extraction module builds a feature extraction model to extract features from the clear leaf image dataset to obtain disease condition features and growth condition features; performs cross-level feature fusion on all disease condition features to obtain fused disease condition features; and performs hierarchical splicing feature fusion on all growth condition features to obtain fused growth condition features;

[0092] A framework construction module constructs a leaf evaluation framework based on the fused disease condition characteristics and the fused growth condition characteristics, and adjusts the framework parameters of the leaf evaluation framework in real time;

[0093] The image evaluation module uses a leaf evaluation framework to perform image evaluation on a clear leaf image dataset, obtains disease severity evaluation indicators and growth condition evaluation indicators, and performs disease severity classification and growth condition classification on the clear leaf image dataset based on the disease severity evaluation indicators and growth condition evaluation indicators to obtain grape leaf grading results; each module is connected to each other via wired and / or wireless means.

[0094] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will be able to modify the technical solutions described in the foregoing embodiments or to substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

[0095] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0096] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0097] In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0098] In the description of the present invention, “several” means one or more, and “a large number” means two or more.

[0099] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0100] The formulas in this manual are all dimensionless and calculated using numerical values. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formulas are set by technicians in this field based on actual conditions.

[0101] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A grape leaf AI grading method based on computer vision, characterized in that: include: S1. Acquire image data of grape leaves and integrate all image data to obtain a leaf image dataset; Based on grape leaf varieties, the leaf image dataset is labeled with variety labels to obtain a variety labeled image dataset; the variety labeled image dataset is optimized to obtain a clear leaf image dataset; S2. Construct a feature extraction model to extract features from the clear leaf image dataset to obtain disease characteristics and growth characteristics; perform cross-level feature fusion on all disease characteristics to obtain fused disease characteristics; and perform hierarchical splicing feature fusion on all growth characteristics to obtain fused growth characteristics. S3. Constructing a leaf evaluation framework based on the fused disease characteristics and the fused growth characteristics, and adjusting the framework parameters of the leaf evaluation framework in real time; S4. Use the leaf evaluation framework to perform image evaluation on the clear leaf image dataset, and obtain the disease degree evaluation index and growth condition evaluation index respectively. Based on the disease degree evaluation index and growth condition evaluation index, the clear leaf image dataset is graded in terms of disease degree and growth condition respectively to obtain the grape leaf grading results.

2. The grape leaf AI grading method based on computer vision according to claim 1, characterized in that: The method of performing image optimization on the variety mark image dataset includes: A dual-channel filtering architecture was designed, including a high-density noise filtering branch and a mixed noise filtering branch. These two branches processed any variety mark image in the variety mark image dataset in parallel. A filtering sliding window was constructed. The noise density within the filtering sliding window was calculated in the high-density noise filtering branch, and the size of the filtering sliding window was adjusted based on the noise density. The filtering sliding window was used to traverse any variety mark image and filter the image area within the filtering sliding window. In the hybrid noise filtering branch, any variety mark image is gridded based on the filtering sliding window, and the variance of all pixels in any grid is calculated. A variance difference threshold is set, and the variance of any grid is compared with the variance of the adjacent grid. If the pixel variance difference between the two grids is greater than the preset variance difference threshold, the grid with the larger variance is determined to be a noise grid, and noise suppression is performed on all noise grids. After processing with a dual-channel filtering architecture, a denoised mark image dataset is output. Brightness-color correction is performed on the denoised mark image dataset to obtain a corrected leaf image dataset. A unified spatial coordinate system is constructed, and spatial coordinate alignment is performed on all corrected leaf images in the corrected leaf image dataset based on the unified spatial coordinate system to obtain a coordinate-aligned image dataset; coordinate-aligned images belonging to the same grape leaf are screened from the coordinate-aligned image dataset, and the coordinate-aligned image dataset is grouped; a three-dimensional model of the corresponding grape leaf is constructed based on each group of coordinate-aligned images in the coordinate-aligned image dataset, and the three-dimensional models corresponding to all grape leaves are integrated to obtain a three-dimensional model dataset; image segmentation is performed on the coordinate-aligned image dataset to obtain a clear leaf image dataset.

3. The grape leaf AI grading method based on computer vision according to claim 2, characterized in that: The method of performing brightness-color correction on the denoised labeled image dataset includes: Perform color space conversion on any denoised marked image in the denoised marked image dataset to obtain a color converted image; calculate the global brightness mean of any color converted image, and perform gamma correction on the color converted image based on the global brightness mean to obtain a gamma corrected image; detect the pixel value distribution of any gamma corrected image, set a high pixel value threshold, and determine the area in any gamma corrected image with a pixel value greater than the high pixel value threshold as an overexposed area; perform illumination decomposition on the overexposed area in any gamma corrected image to obtain the reflection partition and illumination partition of the overexposed area; An attenuation function is constructed and used to adjust the pixel values ​​of the illumination partition; the adjusted illumination partition is recombined with the reflection partition to obtain the normal exposure area, and all overexposed areas of any gamma-corrected image are converted into normal exposure areas to obtain the normal exposure image; a saturation gain function is constructed in the saturation channel of the normal exposure image, and the saturation of the normal exposure image is adjusted based on the saturation gain function to obtain the saturation-adjusted image; a standard color card database is constructed, and any saturation-adjusted image is converted into a color space, and the color difference amplitude between any saturation-adjusted image and the standard image in the standard color card database is calculated, and the saturation-adjusted image is corrected for color deviation based on the color difference amplitude to obtain a color deviation-corrected image; all color deviation-corrected images are integrated to obtain a corrected leaf image dataset.

4. The grape leaf AI grading method based on computer vision according to claim 3 is characterized in that: The method of performing image segmentation on the coordinate aligned image dataset includes: Extract edge features of any coordinate-aligned image in a coordinate-aligned image dataset, including leaf edge features and vein edge features; determine the leaf edge region and vein region in any coordinate-aligned image based on the leaf edge features and vein edge features; calculate the curvature of any edge point in the leaf edge region, perform concave point detection on all edge points on the leaf edge region based on the curvature, and obtain a concave point set; determine the overlapping region in the coordinate-aligned image based on the concave point set, call the three-dimensional model corresponding to the coordinate-aligned image in the three-dimensional model dataset, and use the three-dimensional model to perform projection matching with the overlapping region; set a projection deviation threshold, detect the projection deviation of the projection matching, and resample the leaf edge region if the projection deviation is greater than the preset projection deviation threshold; The coordinate-aligned image is segmented based on the leaf edge area to obtain the leaf region image; a geometric constraint loss function is constructed based on all the coordinate-aligned images belonging to the same grape leaf, and the geometric constraint loss function is used as the constraint condition for region segmentation; the vein area in the leaf region image is enhanced, and a grape leaf morphology database is constructed at the same time, and the image enhancement scale is adjusted based on the grape leaf morphology database; a clear leaf image is obtained after region segmentation and image enhancement, and all clear leaf images are integrated to obtain a clear leaf image dataset.

5. The grape leaf AI grading method based on computer vision according to claim 4 is characterized in that: The method of constructing a feature extraction model to extract features from a clear leaf image dataset includes: A feature extraction model was constructed, using the YOLOv8 network model as the backbone network structure of the feature extraction model. The backbone network structure of the feature extraction model was a three-layer topology, including a shallow convolution layer, a deep network layer, and a group weight adjustment layer. A multi-scale feature extraction architecture was constructed and embedded into the backbone network structure of the feature extraction model. The feature extraction scale was adjusted based on the multi-scale feature extraction architecture. Any clear leaf image in the clear leaf image dataset was used as input data for the feature extraction model. Standard convolution processing was performed on the clear leaf image in the shallow convolution layer to output a preliminary processed feature map. The preliminary processed feature map was linearly transformed to obtain a simplified feature map. Disease feature extraction branches and growth condition feature extraction branches were added to extract features from the simplified feature map. In the disease feature extraction branch, a multi-scale feature extraction architecture is used, and a grape leaf lesion morphology database is constructed. Based on this architecture and the grape leaf lesion morphology database, the lesion characteristics in the simplified feature map are identified. Pathology-related parameters in the simplified feature map are quantified, and the disease recognition capability of the disease feature extraction branch is enhanced based on these parameters. In the growth feature extraction branch, the Hough transform algorithm is used to process the simplified feature map to obtain the vein direction vector and use this vector as a geometric constraint; the three-dimensional model dataset is called to extract the geometric parameters in the three-dimensional model dataset and normalize them; the two-dimensional image features in the simplified feature map are identified, and the normalized geometric parameters are spliced ​​with the two-dimensional image features to obtain the growth feature; in the deep network layer, the channel weights are output through global average pooling, and the feature extraction sensitivity is adjusted based on the channel weights; in the group weight adjustment layer, the weights of all disease condition features and all growth condition features are adjusted separately; all disease condition features and growth condition features belonging to the same clear leaf image are marked with group labels.

6. The grape leaf AI grading method based on computer vision according to claim 5, characterized in that: The method of cross-level feature fusion of all disease condition features includes: All disease condition features are grouped based on grouping labels, and all disease condition features in each group are aligned to obtain aligned disease condition feature maps. All aligned disease condition feature maps in each group are classified by feature type, including shallow lesion texture feature maps and deep lesion semantic feature maps. A dynamic attention architecture is added to dynamically assign weights to shallow lesion texture feature maps and deep lesion semantic feature maps in the feature dimension based on the dynamic attention architecture, and weighted feature splicing is performed on the shallow lesion texture feature map and the deep lesion semantic feature map to obtain a fused disease condition feature. The methods for hierarchical splicing and feature fusion of all growth characteristics include: All growth characteristics are grouped based on the grouping labels, and all growth characteristics in each group are aligned to obtain an aligned growth characteristic map; all aligned growth characteristic maps in each group are classified into local growth characteristic maps and global growth characteristic maps; the vein direction vector is spliced ​​with the local growth characteristic map, and the contribution ratio of the vein direction vector to the local growth characteristic map is adjusted by weight to obtain a shallow vein characteristic map; the pre-trained recurrent neural network model is used to fuse the shallow vein characteristic map and the global growth characteristic map, and the fused growth characteristics are output.

7. The grape leaf AI grading method based on computer vision according to claim 6, characterized in that: The method of constructing the blade evaluation framework and adjusting the framework parameters of the blade evaluation framework in real time includes: A parallel evaluation framework is constructed within the leaf evaluation framework, including a disease condition evaluation framework and a growth condition evaluation framework. The fused disease condition features and fused growth condition features are embedded in the disease condition evaluation framework and the growth condition evaluation framework as evaluation constraints for the two evaluation frameworks, respectively. An image attention mechanism is introduced into the disease assessment framework and linked to the grape leaf lesion morphology database to dynamically enhance the response of key disease areas. Diseases are identified and a disease index equation is constructed based on the evaluation constraints of the disease assessment framework. The disease assessment results are calculated based on the disease index equation. A color feature recognition structure is constructed in the growth assessment framework, and a physiological indicator equation is constructed. The grape leaf morphology database is used as prior knowledge, and a geometric morphology indicator equation is constructed based on the evaluation constraints of the growth assessment framework and the grape leaf morphology database. A group of growth indicator equations is constructed based on the physiological indicator equation and the geometric morphology indicator equation, and the growth assessment results are calculated. All framework parameters of the leaf evaluation framework are extracted, and the historical framework parameters of the leaf evaluation framework in the historical evaluation records are collected at the same time, and a parameter space is constructed based on all current framework parameters and historical framework parameters; an optimization algorithm is used to search for the optimal solution in the parameter space, and a leaf evaluation loss function is constructed based on the disease assessment results and the growth assessment results, and the leaf evaluation loss function is used as the fitness function of the optimization algorithm; the framework parameter combination with the largest fitness function value is taken as the optimal framework parameter combination, and a parameter combination snapshot library is constructed to store the optimal framework parameter combination and the historical optimal parameter combination; the resource occupancy rate of the leaf evaluation framework is monitored in real time, and a resource occupancy rate threshold is set. If the resource occupancy rate is greater than the preset resource occupancy rate threshold, other optimal framework parameter combinations are called from the parameter combination snapshot library.

8. The grape leaf AI grading method based on computer vision according to claim 7, characterized in that: The method of using the leaf evaluation framework to perform image evaluation on the clear leaf image dataset includes: Any clear leaf image in the clear leaf image dataset is used as input to the leaf assessment framework. The leaf assessment framework calls the fused disease condition features and fused growth condition features belonging to the clear leaf image and performs a disease condition assessment on the clear leaf image based on the disease condition assessment architecture. The clear leaf image is compared with the grape leaf lesion morphology database, focusing on the diseased area in the clear leaf image and quantifying the disease severity. The value of the disease index equation is calculated to obtain the disease condition assessment result, i.e., the disease severity assessment index. The growth condition of the clear leaf image is evaluated based on a growth condition evaluation framework; the leaf color distribution in the clear leaf image is identified through the color feature recognition structure in the growth condition evaluation framework, and a standard color card database is called as prior knowledge; the color features of the grape leaves in the clear leaf image are quantified using a physiological index equation, and the physiological index is output; the clear leaf image is compared with a grape leaf morphology database, the growth morphology of the leaves in the clear leaf image is identified, and the growth geometric parameters corresponding to the clear leaf image in the evaluation constraints of the growth condition evaluation framework are combined; the growth degree of the grape leaves in the clear leaf image is quantified by calculating a geometric morphology index equation, and the geometric morphology index is output; the weights of the physiological index equation and the geometric morphology index equation are adjusted based on the growth condition index equation group, and the growth condition evaluation result, i.e., the growth condition evaluation index, is output.

9. The grape leaf AI grading method based on computer vision according to claim 8, characterized in that: The method of respectively grading the disease degree and growth condition of the clear leaf image data set includes: Based on the disease degree assessment index and growth condition assessment index of any clear leaf image in the clear leaf image dataset, the clear leaf image is graded for disease degree and growth condition; a disease degree grading data table is constructed, and threshold intervals for different disease degrees are set based on the grape leaf lesion morphology database; the disease degree assessment index of any clear leaf image is compared with the disease degree grading data table, and the disease degree grade of the current clear leaf image is output; By querying the preset grape leaf category database, a growth classification data table including all grape leaf varieties is constructed; at the same time, a three-level classification standard is constructed for the growth classification data table, and the classification interval of the three-level classification standard is adjusted based on different grape leaf varieties; the growth evaluation index of any clear leaf image is compared with the growth classification data table, and the growth classification of the current clear leaf image is output; the clear leaf image dataset is grouped based on the variety label, and all clear leaf images in each group are regrouped based on the disease severity classification and growth classification, and all grouped clear leaf images are labeled with the classification labels to obtain a graded grouped image dataset; Based on the disease severity and growth status classification of each clear leaf image, a disease-growth status heat map was constructed, and different colors were used to represent the disease severity and growth status. At the same time, the variety type and related parameter information of the corresponding clear leaf image were annotated in the disease-growth status heat map. The disease-growth status heat maps of all clear leaf images were integrated to obtain a disease-growth status visualization set. The disease-growth status visualization set and the graded grouped image dataset were integrated to obtain the grape leaf grading results.

10. A computer vision-based AI grading processing system for grape leaves, which is used to implement the computer vision-based AI grading processing method for grape leaves according to any one of claims 1 to 9, characterized in that: include: The data acquisition module acquires the image data of grape leaves and integrates all the image data to obtain the leaf image dataset; Based on grape leaf varieties, the leaf image dataset is labeled with variety labels to obtain a variety labeled image dataset; the variety labeled image dataset is optimized to obtain a clear leaf image dataset; The feature extraction module builds a feature extraction model to extract features from the clear leaf image dataset to obtain disease condition features and growth condition features; performs cross-level feature fusion on all disease condition features to obtain fused disease condition features; and performs hierarchical splicing feature fusion on all growth condition features to obtain fused growth condition features; A framework construction module constructs a leaf evaluation framework based on the fused disease condition characteristics and the fused growth condition characteristics, and adjusts the framework parameters of the leaf evaluation framework in real time; The image evaluation module uses a leaf evaluation framework to perform image evaluation on a clear leaf image dataset, obtains disease severity evaluation indicators and growth condition evaluation indicators, and performs disease severity classification and growth condition classification on the clear leaf image dataset based on the disease severity evaluation indicators and growth condition evaluation indicators to obtain grape leaf grading results; each module is connected to each other via wired and / or wireless means.

Images