Plant leaf fine instance segmentation method based on mask compression reconstruction
By using a mask compression reconstruction method, local frequency domain representation and three-class region discrimination, the problems of high computational complexity and insufficient boundary detail recovery in existing technologies are solved, and efficient and fine instance segmentation in complex scenes is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2026-04-23
- Publication Date
- 2026-05-29
AI Technical Summary
Existing instance segmentation methods suffer from high computational complexity and insufficient ability to recover boundary details during high-resolution mask prediction. Furthermore, instances tend to stick together in complex scenarios, making it difficult to balance segmentation accuracy and efficiency.
A mask compression reconstruction method is adopted, which uses local frequency domain (DCT) representation for mask compression and combines three-class region discrimination and boundary detail reconstruction. By leveraging the energy concentration characteristics in the frequency domain, the problem is transformed into compact low-frequency coefficient prediction, reducing computational complexity and preserving key structural information.
While reducing computational overhead, it improves the accuracy of leaf boundary segmentation and instance discrimination capability, making it suitable for complex agricultural scenarios such as high-density planting and achieving high-resolution fine segmentation.
Smart Images
Figure CN122116342A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a method for fine instance segmentation of plant leaves. Background Technology
[0002] In recent years, plant phenotyping technology has become a research hotspot in smart agriculture and bioscience. Acquiring plant image data through image acquisition devices in greenhouse or field environments and using computer vision algorithms to automatically identify and analyze plant structures provides crucial information for crop breeding, growth monitoring, and precision agricultural management. Traditional methods for plant phenotyping rely on manual measurement of phenotypic parameters such as leaf number and area, which is time-consuming and labor-intensive. With the rapid development of deep learning technology, automated instance segmentation methods based on computer vision are increasingly used in non-invasive plant phenotyping research. Their aim is to segment different leaf instances of a plant and then accurately calculate phenotypic indicators such as leaf area and number. These methods can significantly reduce labor costs, but require higher segmentation accuracy, boundary detail recovery capabilities, and computational efficiency in complex agricultural scenarios.
[0003] Since plant leaves are usually long and irregular in shape, they are prone to occlusion and adhesion in high-density planting environments. In order to obtain fine boundaries, traditional instance segmentation methods usually rely on high-resolution mask prediction, which significantly increases computational complexity and storage overhead. As a result, existing deep learning-based instance segmentation algorithms cannot achieve both boundary accuracy and inference efficiency in complex scenarios. Summary of the Invention
[0004] The purpose of this invention is to address the problems of high computational complexity, insufficient boundary detail recovery ability, and easy instance adhesion in complex scenes in existing instance segmentation methods during high-resolution mask prediction. This invention proposes a fine instance segmentation method for plant leaves based on mask compression reconstruction, which can improve the boundary accuracy and instance discrimination ability of the segmentation results while reducing computational overhead.
[0005] The specific process of a fine-grained instance segmentation method for plant leaves based on mask compression reconstruction is as follows:
[0006] Step 1: Construct the training set;
[0007] Step 2: Construct a fine-grained instance segmentation network model based on mask compression and reconstruction;
[0008] Step 3: Train the fine instance segmentation network model based on mask compression and reconstruction on the training set until the loss function converges, and obtain the trained fine instance segmentation network model based on mask compression and reconstruction.
[0009] Step 4: Input the plant image into the trained fine instance segmentation network model based on mask compression and reconstruction. The fine instance segmentation network model based on mask compression and reconstruction outputs the classification result, object detection result, and fine instance segmentation result.
[0010] The beneficial effects of this invention are as follows:
[0011] This invention proposes a method for fine-grained instance segmentation of plant leaves in complex scenes based on mask compression reconstruction. The high-resolution mask is compressed using local frequency domain (DCT) representation, and combined with three-class region discrimination and boundary detail reconstruction, which can effectively improve the leaf boundary segmentation accuracy and model inference efficiency.
[0012] The purpose of this invention is to alleviate the problem of insufficient segmentation accuracy caused by the large computational overhead of high-resolution masks and the loss of boundary details in complex scenarios of existing instance segmentation methods. It proposes a mask compression reconstruction method based on local frequency domain representation, which utilizes the energy concentration characteristics of the frequency domain to transform the problem into compact low-frequency coefficient prediction, effectively reducing computational complexity and preserving key structural information.
[0013] Since this method uses the complex characteristics of plant leaf boundaries as a prior for mask modeling and regional differentiation learning, it can significantly reduce model parameters and inference costs while maintaining high-resolution fine segmentation. It is especially suitable for complex agricultural scenarios such as high-density planting. It can achieve leaf-level fine instance segmentation without additional high-resolution full-image processing, which can improve the boundary accuracy and instance discrimination ability of the segmentation results while reducing computational overhead.
[0014] To verify the performance of the algorithm proposed in this invention, simulation experiments were conducted on a real agricultural scenario dataset. The experimental results verified the effectiveness of the fine instance segmentation method for plant leaves based on mask compression reconstruction proposed in this invention. Attached Figure Description
[0015] Figure 1 This is a schematic diagram illustrating the implementation process of the present invention;
[0016] Figure 2 Here is a structural diagram of the vector compression and reconstruction module;
[0017] Figure 3 Here is a structural diagram of the feature extraction module;
[0018] Figure 4 This is a visualization comparison of the instance segmentation results of the method of this invention and traditional methods such as Mask R-CNN on the sugar beet dataset;
[0019] Figure 5 This is a visualization comparison of the instance segmentation results of the method of this invention and traditional methods such as Mask R-CNN on the cauliflower dataset. Detailed Implementation
[0020] Specific Implementation Method 1: The specific process of this implementation method for fine instance segmentation of plant leaves based on mask compression reconstruction is as follows:
[0021] Step 1: Construct the training set;
[0022] Step 2: Construct a fine-grained instance segmentation network model based on mask compression and reconstruction;
[0023] Step 3: Train the fine instance segmentation network model based on mask compression and reconstruction on the training set until the loss function converges, and obtain the trained fine instance segmentation network model based on mask compression and reconstruction.
[0024] Step 4: Input the complex scene plant image into the trained fine instance segmentation network model based on mask compression and reconstruction. The fine instance segmentation network model based on mask compression and reconstruction outputs the classification result, object detection result and fine instance segmentation result.
[0025] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that: in step 1, a training set is constructed; the specific process is as follows:
[0026] The training set is either the publicly available PhenoBench beet dataset or the GrowliFlower cauliflower dataset;
[0027] Both datasets were collected from real farmland environments and have high scene complexity: they include different lighting conditions, soil textures, weed interference, plant scale variations, and severe shading and adhesion between leaves.
[0028] The other steps and parameters are the same as in Specific Implementation Method 1.
[0029] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that: in step 2, a fine-grained instance segmentation network model based on mask compression and reconstruction is constructed; the specific process is as follows:
[0030] The fine-grained instance segmentation network model based on mask compression and reconstruction includes: backbone network, feature pyramid network FPN, RPN network (RPN network in Mask R-CNN), ROI Align (ROI Align in Mask R-CNN), and vector compression and reconstruction module.
[0031] Other steps and parameters are the same as in specific implementation method one or two.
[0032] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the working process of the fine instance segmentation network model based on mask compression reconstruction is as follows:
[0033] 1) The image is input into the backbone network, and the backbone network outputs a feature map. Feature map Feature map Feature map ;
[0034] Image size is The first For the length of the image, the first The width of the image, The number of channels in the image;
[0035] 2) Feature map Feature map Feature map Feature map Input Feature Pyramid Network (FPN), output feature map Feature map Feature map Feature map ;
[0036] Feature map Size is The first The second is the length of the image. For the width of the image, the third The number of channels in the image;
[0037] Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image;
[0038] Feature map Size is The first For the length of the image, the second The width of the image, The number of channels in the image;
[0039] Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image;
[0040] 3) Feature map Feature map Feature map Feature map Input the RPN network, and the RPN network outputs candidate boxes;
[0041] Feature map Size is The first The second is the length of the image. For the width of the image, the third The number of channels in the image;
[0042] Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image;
[0043] Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image;
[0044] Feature map Size is The first For the length of the image, the first The width of the image, The number of channels in the image;
[0045] 4) The RPN network outputs candidate boxes and inputs them into ROI Align. ROI Align aligns the candidate boxes and outputs a fixed-size feature map (many candidate boxes input into ROI Align result in the same number of aligned feature maps). The feature map is input into a fully connected FC network, and the fully connected FC network outputs feature map A.
[0046] Feature map A is input to a fully connected FC, and the fully connected FC outputs the target classification result;
[0047] Feature map A is input to a fully connected FC, and the fully connected FC outputs the bounding box result of the object detection;
[0048] 5) Feature Map Input the vector compression and reconstruction module, and output the instance segmentation result.
[0049] The other steps and parameters are the same as those in one of the specific implementation methods one to three.
[0050] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that: in step 1), the image is input to the backbone network, and the backbone network outputs a feature map. Feature map Feature map Feature map The specific process is as follows:
[0051] The input image passes through convolutional layers and pooling layers in sequence, and the pooling layer outputs a feature map. ;
[0052] For feature maps Perform a 2x downsampling to obtain the feature map. ;
[0053] For feature maps Perform a 2x downsampling to obtain the feature map. ;
[0054] For feature maps Perform a 2x downsampling to obtain the feature map. .
[0055] The other steps and parameters are the same as in any of the specific implementation methods one to four.
[0056] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that: in step 2), the feature map... Feature map Feature map Feature map Input Feature Pyramid Network (FPN), output feature map Feature map Feature map Feature map The specific process is as follows:
[0057] feature map through Convolutional dimensionality reduction process outputs feature maps. ;
[0058] feature map through Convolutional dimensionality reduction process yields feature maps. ; for feature maps Perform upsampling by 2x to obtain the feature map. ; feature map and feature map By adding elements one by one, we obtain the feature map. ;
[0059] feature map through Convolutional dimensionality reduction process yields feature maps. ; for feature maps Perform upsampling by 2x to obtain the feature map. ; feature map and feature map By adding elements one by one, we obtain the feature map. ;
[0060] feature map through Convolutional dimensionality reduction process yields feature maps. ; for feature maps Perform upsampling by 2x to obtain the feature map. ; feature map and feature map By adding elements one by one, we obtain the feature map. .
[0061] The other steps and parameters are the same as those in any of the specific implementation methods one to five.
[0062] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that: the feature map in 5) The input vector compression and reconstruction module outputs the instance segmentation result; the specific process is as follows:
[0063] 51) Feature Map Input: Fully connected layer (FC); Output: Fully connected layer (FC) ;
[0064] 52) Divided into Local block set ;
[0065] 53) Perform DCT transform on each local block to obtain the frequency domain coefficient matrix corresponding to each local block. The size of the frequency domain coefficient matrix corresponding to each local block is... ;
[0066] DCT stands for Two-Dimensional Discrete Cosine Transform;
[0067] 54) Input the frequency domain coefficient matrix corresponding to each local block in 53) into the classifier, and the classifier outputs the classification result of each local block;
[0068] The classification result for each local block is either foreground, background, or boundary;
[0069] If a boundary category exists in a local block, execute 55);
[0070] If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category.
[0071] The image after IDCT transformation is the instance segmentation result;
[0072] The IDCT transform is an inverse two-dimensional discrete cosine transform;
[0073] The specific process is as follows:
[0074] Normalize the frequency domain coefficient matrix corresponding to each local block to obtain the normalized classification result of each local block;
[0075] like Then the normalized classification result is the foreground;
[0076] like Then the normalized classification result is the background;
[0077] like Then the normalized classification result is the boundary;
[0078] 55) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size [size missing]. A set of local blocks;
[0079] The size of the local block corresponding to each boundary category is... The set of local blocks is used to obtain the set of all local blocks corresponding to all boundary categories. All sets of local blocks constitute a blending mask. Figure 1 ;
[0080] 56) Mix the mask from 55) Figure 1 Each local block is input into the classifier, and the classifier outputs the classification result for each local block;
[0081] The classification result for each local block is either foreground, background, or boundary;
[0082] If a boundary category exists in a local block, execute 57);
[0083] If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category.
[0084] The image after IDCT transformation is concatenated with the image after IDCT transformation obtained in step 54) to obtain the instance segmentation result;
[0085] The specific process is as follows:
[0086] For mixing masks Figure 1 The frequency domain coefficient matrix corresponding to each local block is normalized to obtain the normalized classification result of each local block;
[0087] like Then the normalized classification result is the foreground;
[0088] like Then the normalized classification result is the background;
[0089] like Then the normalized classification result is the boundary;
[0090] 57) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size [size missing]. A set of local blocks;
[0091] The size of the local block corresponding to each boundary category is... The set of local blocks is used to obtain the set of all local blocks corresponding to all boundary categories. All sets of local blocks constitute a blending mask. Figure 2 ;
[0092] 58) Mix the mask from 57) Figure 2 Each local block is input into the classifier, and the classifier outputs the classification result for each local block;
[0093] The classification result for each local block is either foreground, background, or boundary;
[0094] If a boundary category exists in a local block, execute 59);
[0095] If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category.
[0096] The image after IDCT transformation is stitched together with the image after IDCT transformation obtained in 54) and 56) to obtain the instance segmentation result;
[0097] The specific process is as follows:
[0098] For mixing masks Figure 2 The frequency domain coefficient matrix corresponding to each local block is normalized to obtain the normalized classification result of each local block;
[0099] like Then the normalized classification result is the foreground;
[0100] like Then the normalized classification result is the background;
[0101] like Then the normalized classification result is the boundary;
[0102] 59) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size [size missing]. A set of local blocks;
[0103] The size of the local block corresponding to each boundary category is... The set of local blocks is used to obtain the set of all local blocks corresponding to all boundary categories. All sets of local blocks constitute a blending mask. Figure 3 ;
[0104] 510), mix the mask in 59). Figure 3 Each local block is input into the classifier, and the classifier outputs the classification result for each local block;
[0105] The classification result for each local block is either foreground, background, or boundary;
[0106] If a boundary category exists in a local block, execute 511);
[0107] If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category.
[0108] The image obtained after IDCT transformation is stitched together with the images obtained after IDCT transformation in steps 54), 56), and 58 to obtain the instance segmentation result;
[0109] The specific process is as follows:
[0110] For mixing masks Figure 3 The frequency domain coefficient matrix corresponding to each local block is normalized to obtain the normalized classification result of each local block;
[0111] like Then the normalized classification result is the foreground;
[0112] like Then the normalized classification result is the background;
[0113] like Then the normalized classification result is the boundary;
[0114] 511) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size . A set of local blocks;
[0115] The size of the local block corresponding to each boundary category is... The set of local blocks is used to obtain the set of all local blocks corresponding to all boundary categories. All sets of local blocks constitute a blending mask. Figure 4 ;
[0116] For mixing masks Figure 4 Perform IDCT transformation to obtain the mask after IDCT transformation. Figure 4 ;
[0117] The mask after IDCT transformation Figure 4 The images obtained from IDCT transformation (54), IDCT transformation (56), IDCT transformation (58), and IDCT transformation (510) are stitched together as the instance segmentation result.
[0118] The other steps and parameters are the same as those in one of the specific implementation methods one to six.
[0119] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that the loss function in step 3 is:
[0120]
[0121] in, This represents the loss of a fine-grained instance segmentation network model based on mask compression reconstruction;
[0122] This represents the classification loss of the ROI Align output classification result, using Softmax cross-entropy loss. ,in To predict the true category The probability of;
[0123] The bounding box regression loss represents the object detection result output by ROI Align, using Smooth L1 loss. ;
[0124] in, For the first The predicted coordinates of each detection box. , The predicted coordinates of the lower left corner of the detection box. The predicted coordinates of the upper right corner of the detection box;
[0125] For the first The true coordinates of each detection box. , The coordinates of the bottom left corner of the detection box are the actual values. The actual coordinates of the upper right corner of the detection box;
[0126] For the first The predicted coordinates of the first detection box and the second detection box The error of the true coordinates of each detection box;
[0127] The horizontal axis error is ;
[0128] The error of the ordinate is ;
[0129] .
[0130] The weighted mask loss represents the instance segmentation result;
[0131] express The weighting coefficients, express The weighting coefficients, express The weighting coefficients.
[0132] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.
[0133] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that the loss of the instance segmentation result... The acquisition process is as follows:
[0134] Loss of instance segmentation results For all instances in the segmentation result Loss of local blocks The sum of;
[0135] Each instance segmentation result Loss of local blocks Represented as:
[0136]
[0137] in, This represents the instance segmentation result of a fine-grained instance segmentation network model based on mask compression reconstruction, where each instance segmentation value represents a specific instance segmentation value. Pixels in local blocks The predicted value;
[0138] Indicates each of the instance segmentation results Pixels in local blocks The true value.
[0139] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.
[0140] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that: the... All are set to 1.0.
[0141] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.
[0142] The beneficial effects of the present invention are verified using the following embodiments:
[0143] Example 1:
[0144] The experimental data used were the publicly available PhenoBench sugar beet dataset and the GrowliFlower cauliflower dataset. Both datasets were collected from real farmland environments and have high scene complexity: they include different lighting conditions, soil textures, weed interference, plant scale variations, and severe occlusion and adhesion between leaves. The PhenoBench dataset mainly contains field images of sugar beet crops, providing pixel-level instance annotations; the GrowliFlower dataset focuses on cauliflower crops, with more complex plant structures, making it suitable for verifying the generalization ability of the method in high-density planting scenarios. The method of this invention was quantitatively compared and visualized with traditional mainstream instance segmentation methods such as Mask R-CNN and DCT-Mask. Experimental results show that the method of this invention... It significantly outperforms traditional methods such as Mask R-CNN in terms of boundary accuracy, verifying the effectiveness and efficiency of the mask compression reconstruction strategy in complex scenarios.
[0145]
[0146]
[0147] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.
Claims
1. A method for fine-grained instance segmentation of plant leaves based on mask compression reconstruction, characterized in that: The specific process of the method is as follows: Step 1: Construct the training set; Step 2: Construct a fine-grained instance segmentation network model based on mask compression and reconstruction; Step 3: Train the fine instance segmentation network model based on mask compression and reconstruction on the training set until the loss function converges, and obtain the trained fine instance segmentation network model based on mask compression and reconstruction. Step 4: Input the plant image into the trained fine instance segmentation network model based on mask compression and reconstruction. The fine instance segmentation network model based on mask compression and reconstruction outputs the classification result, object detection result, and fine instance segmentation result.
2. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 1, characterized in that: The construction of the training set in step 1 is as follows: The training set is either the publicly available PhenoBench beet dataset or the GrowliFlower cauliflower dataset.
3. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 2, characterized in that: In step 2, a fine-grained instance segmentation network model based on mask compression reconstruction is constructed; The specific process is as follows: The fine-grained instance segmentation network model based on mask compression and reconstruction includes: backbone network, feature pyramid network FPN, RPN network, ROI Align, and vector compression and reconstruction module.
4. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 3, characterized in that: The working process of the fine instance segmentation network model based on mask compression reconstruction is as follows: 1) The image is input into the backbone network, and the backbone network outputs a feature map. Feature map Feature map Feature map ; Image size is The first For the length of the image, the first The width of the image, The number of channels in the image; 2) Feature map Feature map Feature map Feature map Input Feature Pyramid Network (FPN), output feature map Feature map Feature map Feature map ; Feature map Size is The first The second is the length of the image. For the width of the image, the third The number of channels in the image; Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image; Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image; Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image; 3) Feature map Feature map Feature map Feature map Input the RPN network, and the RPN network outputs candidate boxes; Feature map Size is The first The second is the length of the image. For the width of the image, the third The number of channels in the image; Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image; Feature map Size is The first The second is the length of the image. The width of the image, The number of channels in the image; Feature map Size is The first For the length of the image, the first The width of the image, The number of channels in the image; 4) The RPN network outputs candidate boxes as input to ROI Align. ROI Align aligns the candidate boxes and outputs a fixed-size feature map. The feature map is input to a fully connected FC network, and the fully connected FC network outputs feature map A. Feature map A is input to a fully connected FC, and the fully connected FC outputs the target classification result; Feature map A is input to a fully connected FC, and the fully connected FC outputs the bounding box results of the object detection; 5) Feature Map Input the vector compression and reconstruction module, and output the instance segmentation result.
5. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 4, characterized in that: In step 1), the image is input into the backbone network, and the backbone network outputs a feature map. Feature map Feature map Feature map The specific process is as follows: The input image passes through convolutional layers and pooling layers in sequence, and the pooling layer outputs a feature map. ; For feature maps Perform a 2x downsampling to obtain the feature map. ; For feature maps Perform a 2x downsampling to obtain the feature map. ; For feature maps Perform a 2x downsampling to obtain the feature map. ; The other steps and parameters are the same as in any of the specific implementation methods one to four.
6. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 5, characterized in that: The feature map in 2) Feature map Feature map Feature map Input Feature Pyramid Network (FPN), output feature map Feature map Feature map Feature map The specific process is as follows: feature map through Convolutional dimensionality reduction process outputs feature maps. ; feature map through Convolutional dimensionality reduction process yields feature maps. ; for feature maps Perform upsampling by 2x to obtain the feature map. ; feature map and feature map By adding elements one by one, we obtain the feature map. ; feature map through Convolutional dimensionality reduction process yields feature maps. ; for feature maps Perform upsampling by 2x to obtain the feature map. ; feature map and feature map By adding elements one by one, we obtain the feature map. ; feature map through Convolutional dimensionality reduction process yields feature maps. ; for feature maps Perform upsampling by 2x to obtain the feature map. ; feature map and feature map By adding elements one by one, we obtain the feature map. .
7. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 6, characterized in that: The feature map in 5) The input vector compression and reconstruction module outputs the instance segmentation result; the specific process is as follows: 51) Feature Map Input: Fully connected layer (FC); Output: Fully connected layer (FC) ; 52) will Divided into Local block set ; 53) Perform DCT transform on each local block to obtain the frequency domain coefficient matrix corresponding to each local block. The size of the frequency domain coefficient matrix corresponding to each local block is... ; DCT stands for Two-Dimensional Discrete Cosine Transform; 54) Input the frequency domain coefficient matrix corresponding to each local block in 53) into the classifier, and the classifier outputs the classification result of each local block; The classification result for each local block is either foreground, background, or boundary; If a boundary category exists in a local block, execute 55); If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category. The image after IDCT transformation is the instance segmentation result; The IDCT transform is an inverse two-dimensional discrete cosine transform; The specific process is as follows: Normalize the frequency domain coefficient matrix corresponding to each local block to obtain the normalized classification result of each local block; like Then the normalized classification result is the foreground; like Then the normalized classification result is the background; like Then the normalized classification result is the boundary; 55) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size [size missing]. A set of local blocks; The size of the local block corresponding to each boundary category is... The local block set is obtained by taking the local block set corresponding to the local block of all boundary categories. The local block set constitutes the hybrid mask image 1. 56) Input each local block in the mixed mask image 1 (55) into the classifier, and the classifier outputs the classification result of each local block; The classification result for each local block is either foreground, background, or boundary; If a boundary category exists in a local block, execute 57); If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category. The image after IDCT transformation is concatenated with the image after IDCT transformation obtained in step 54) to obtain the instance segmentation result; The specific process is as follows: Normalize the frequency domain coefficient matrix corresponding to each local block in the hybrid mask image 1 to obtain the normalized classification result of each local block; like Then the normalized classification result is the foreground; like Then the normalized classification result is the background; like Then the normalized classification result is the boundary; 57) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size [size missing]. A set of local blocks; The size of the local block corresponding to each boundary category is... The local block set is obtained by taking the local block set corresponding to the local block of all boundary categories. The local block set constitutes the hybrid mask image 2. 58) Input each local block in the mixed mask image 2 (57) into the classifier, and the classifier outputs the classification result of each local block; The classification result for each local block is either foreground, background, or boundary; If a boundary category exists in a local block, execute 59); If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category. The image after IDCT transformation is stitched together with the image after IDCT transformation obtained in 54) and 56) to obtain the instance segmentation result; The specific process is as follows: Normalize the frequency domain coefficient matrix corresponding to each local block in the hybrid mask image 2 to obtain the normalized classification result of each local block; like Then the normalized classification result is the foreground; like Then the normalized classification result is the background; like Then the normalized classification result is the boundary; 59) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size [size missing]. A set of local blocks; The size of the local block corresponding to each boundary category is... The local block set is obtained by taking the local block set corresponding to all boundary categories. All local block sets constitute the hybrid mask image 3. 510) Input each local block in the mixed mask image 3 in 59) into the classifier, and the classifier outputs the classification result of each local block; The classification result for each local block is either foreground, background, or boundary; If a boundary category exists in a local block, execute 511); If a local block does not have a boundary category, perform IDCT transformation on the image composed of the foreground and background of all local blocks that do not have a boundary category to obtain the IDCT transformed image of all local blocks that do not have a boundary category. The image obtained after IDCT transformation is stitched together with the images obtained after IDCT transformation in steps 54), 56), and 58 to obtain the instance segmentation result; The specific process is as follows: Normalize the frequency domain coefficient matrix corresponding to each local block in the hybrid mask image 3 to obtain the normalized classification result of each local block; like Then the normalized classification result is the foreground; like Then the normalized classification result is the background; like Then the normalized classification result is the boundary; 511) Input each local block corresponding to the boundary category in the classification result into the regressor. The regressor will process each local block corresponding to each boundary category in the classification result into blocks of size . A set of local blocks; The size of the local block corresponding to each boundary category is... The local block set is obtained by taking the local block set corresponding to all boundary categories. All local block sets constitute the hybrid mask image 4. Perform IDCT transformation on the hybrid mask image 4 to obtain the IDCT transformed mask image 4; The IDCT-transformed mask images 4 and 54) are combined to obtain the IDCT-transformed images, 56) are obtained by stitching together the IDCT-transformed images, 58) are obtained by stitching together the IDCT-transformed images, and 510) are obtained by stitching together to obtain the instance segmentation result.
8. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 7, characterized in that: The loss function in step 3 is: in, This represents the loss of a fine-grained instance segmentation network model based on mask compression reconstruction; This represents the classification loss of the ROI Align output classification result; This represents the bounding box regression loss of the object detection results output by ROI Align; The weighted mask loss represents the instance segmentation result; express The weighting coefficients, express The weighting coefficients, express The weighting coefficients.
9. The method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 8, characterized in that: The loss of the instance segmentation result The acquisition process is as follows: Loss of instance segmentation results For all instances in the segmentation result Loss of local blocks The sum of; Each instance segmentation result Loss of local blocks Represented as: in, This represents the instance segmentation result of a fine-grained instance segmentation network model based on mask compression reconstruction, where each instance segmentation value represents a specific instance segmentation value. Pixels in local blocks The predicted value; Indicates each of the instance segmentation results Pixels in local blocks The true value.
10. A method for fine instance segmentation of plant leaves based on mask compression reconstruction according to claim 9, characterized in that: The All are set to 1.0.