An overhead strawberry recognition method and system for accurately determining the ripeness thereof

Abstract

The application provides a kind of overhead strawberry recognition and its maturity accurate discrimination method, system, including: image acquisition, labeling, enhancement;Using adaptive strawberry feature enhancement network to identify strawberry and generate strawberry mask;Convert the RGB color space in mask into HSV color space;Based on threshold, the red region in HSV color space is segmented and the proportion of the number of pixels in the region to the total number of pixels is calculated;According to the proportion, the maturity of strawberry is judged.The self-attention neural network and multi-scale feature fusion network in the adaptive strawberry feature enhancement network of the application can extract rich edge features and semantic features, and fuse different scale strawberry features, improve the feature expression ability and the accuracy of strawberry pixel classification, and the decoupling head neural network can effectively generate the strawberry mask according to the position of the strawberry in the image, and finally accurately judge the best picking time of the strawberry, providing visual support for the strawberry picking robot.

Description

Technical Field

[0001] This invention belongs to the field of deep learning and strawberry maturity detection technology, and in particular relates to a method and system for identifying elevated strawberries and accurately judging their maturity. Background Technology

[0002] With the transformation of agricultural production methods and technological advancements, elevated cultivation technology has become an important means to improve strawberry yield and quality. However, due to varying maturity levels and susceptibility to external environmental factors, the timeliness and accuracy of strawberry harvesting during the growth process of elevated strawberries have become pressing issues. Currently, automated strawberry harvesting robots have been applied in some facility agriculture projects. These robots primarily employ machine vision-based strawberry recognition technology, utilizing cameras and deep learning algorithms to effectively identify strawberries and categorize them into ripe and unripe varieties, thus enabling automated harvesting. However, existing visual recognition technologies mainly focus on the simple classification of strawberry ripeness, without further precise determination of the specific maturity levels of ripe strawberries. Current recognition technologies typically rely on the color or morphological characteristics of strawberries to determine whether they are ripe or unripe, but they cannot accurately distinguish ripe strawberries based on different maturity levels (such as ripe, overripe, etc.). Furthermore, existing methods have low image segmentation accuracy, making it difficult to accurately distinguish strawberries from the background, thus affecting the accuracy of harvesting and maturity determination.

[0003] To address the aforementioned technical problems, this invention presents a novel, efficient, and accurate method and system for identifying and determining the maturity of strawberries based on deep learning. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for identifying elevated strawberries and accurately determining their maturity, which can ensure that strawberry picking robots can efficiently and accurately complete the tasks of strawberry identification and maturity determination during operation.

[0005] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0006] A method for identifying elevated strawberries and accurately determining their maturity includes the following steps:

[0007] Step 1: Acquiring strawberry images;

[0008] Step 2: Manually determine whether the strawberries in the image are ripe and ready to be picked or unripe and not ready to be picked, and then perform contour instance annotation;

[0009] Step 3: Perform image enhancement processing on the original image and update and adjust its annotation file accordingly;

[0010] Step 4: Use an adaptive strawberry feature enhancement network to identify strawberries in the image and generate a strawberry mask; the adaptive strawberry feature enhancement network processes the input image by: extracting features from the input image through a self-attention neural network, then extracting and fusing multi-scale features through a multi-scale feature fusion network; and generating a strawberry mask by decoupling the head network.

[0011] Step 5: Extract the mask of ripe strawberries and convert the RGB color space within the mask to the HSV color space;

[0012] Step 6: Segment the red region in the HSV color space based on the set threshold, and calculate the ratio of the number of pixels in the red region to the total number of pixels in the mask;

[0013] Step 7: Based on the calculated ratio, determine the ripeness of the strawberries according to the set ripeness judgment criteria. The ripeness judgment result is used to control the picking action of the strawberry picking robot.

[0014] Furthermore, in step 4, the specific structural arrangement of the self-attention neural network is as follows:

[0015] The first layer is a patch partitioning layer, which uses convolutional layers to divide and flatten the image into blocks. The convolutional kernel size is 4×4, the stride is 4, and the number of channels is 48. The output feature map size is (H / 4, W / 4, 48). Following the first layer are four stages. The first stage includes one linear embedding layer and two Swin Transformer blocks. The linear embedding layer uses the convolutional layer to perform a linear transformation on the data along the channel direction, doubling the number of channels in the feature map. The second stage includes one patch merging layer and two Swin Transformer blocks. The patch merging layer is a downsampling layer that performs a linear transformation along the channel direction, doubling the number of channels in the feature map. The third stage includes one patch merging layer and six Swin Transformer blocks. The fourth stage includes one patch merging layer and two Swin Transformer blocks. block; the feature maps and their sizes output by each stage are S1(H / 4, W / 4, 96), S2(H / 8, W / 8, 192), S3(H / 16, W / 16, 384), and S4(H / 32, W / 32, 768);

[0016] The Swin Transformer block includes batch normalization, support vector machine, multi-head self-attention for windows, and sliding window self-attention modules. The calculation formula between two consecutive Swin Transformer blocks is as follows:

[0017]

[0018] in, and z l These represent the output feature maps of the W-MSA / SW-MSA and SVM modules of the l-th Swing Transformer block, respectively.

[0019] The calculation formula for the W-MSA module is:

[0020] / Omega{W-MSA}=4HWC 2 +2M 2 HWC

[0021] Where H and W are the height and width of the input image, respectively, C is the depth of the feature map, and M is the size of each window.

[0022] Furthermore, in step 4, the multi-scale feature fusion network includes a feature pyramid network and a strawberry feature adaptive fusion module. The specific process of extracting and fusing multi-scale features through the multi-scale feature fusion network is as follows:

[0023] Feature extraction is performed on the four feature maps output by the self-attention neural network using convolutional layers. The kernel size is 1×1, the stride is 1, and the number of channels is 256. The resulting output feature maps and their sizes are F1(H / 4, W / 4, 256), F2(H / 8, W / 8, 256), F3(H / 16, W / 16, 256), and F4(H / 32, W / 32, 256), respectively. Then, feature map F4 is downsampled using a convolutional layer with a kernel size of 2×2, a stride of 2, and 256 channels. The resulting output feature map size is (H / 64, W / 64, 256), which is output as output 5. Upsample F4 and add it element-wise to feature map F3 to obtain an output feature map with size (H / 32, W / 32, 256), which is output 4; Upsample output 4 and add it element-wise to feature map F3 to obtain an output feature map with size (H / 16, W / 16, 256), which is output 3; Upsample output 3 and add it element-wise to feature map F2 to obtain an output feature map with size (H / 8, W / 8, 256), which is output 2; Upsample output 2 and add it element-wise to feature map F1 to obtain an output feature map with size (H / 4, W / 4, 256), which is output 1.

[0024] The five feature maps output by the feature pyramid network are input into the strawberry feature adaptive fusion module; for the feature map x of the m-th layer... m (m∈{1, 2, 3, 4, 5}), the feature maps x of other layers n Adjust to be similar to x by upsampling or downsampling. m Using the same size and performing adaptive fusion, the output feature map of the m-th layer is obtained;

[0025] The adaptive fusion strategy formula for the strawberry feature adaptive fusion module is as follows:

[0026]

[0027] in, Let represent the feature vector mapped from layer n to layer m at pixel (i, j). This represents the (i, j)th vector of the output feature map along the channel direction; Determined by the following calculation formula:

[0028]

[0029]

[0030] in, They are respectively based on These are control parameters defined by the softmax function. represents the trainable weights for each feature map layer, and μ is the weight coefficient.

[0031] Furthermore, in step 4, the decoupled head network includes two sub-networks for instance category and mask generation. The specific process of generating the strawberry mask through the decoupled head network is as follows:

[0032] The instance classification subnetwork first predefines a grid size S (S∈{40, 36, 24, 16, 12}), and adjusts the feature map size to (S, S, 256) through one convolutional layer; then, it extracts features through seven convolutional layers, integrates these features, and adjusts the feature map size to (S, S, C). c ), C c The number of categories is denoted by ; the centroid of the strawberry falls in the (i, j)th grid, which is responsible for predicting the category of the strawberry. Each channel outputs the probability that the strawberry belongs to a certain category.

[0033] The mask generation sub-network consists of two branches. Each branch first encodes spatial information by concatenating a coordinate channel onto the original feature map using a convolutional layer, outputting a feature map with dimensions (H, W, 256+1). Then, it extracts features through seven convolutional layers, integrates these features, and adjusts the feature map size to (H, W, S). When a strawberry falls into a grid, that grid is responsible for predicting the classification of the pixel within the mask. Finally, the prediction results from both branches are integrated, and pixels classified as strawberries are remapped back to the original image. This result is then fused with the result from the instance classification sub-network using the following formula to obtain the final strawberry mask and its classification:

[0034]

[0035] Where sigmoid is the activation function, F j and F i Let J and L be the feature maps of the j-th and i-th channels, respectively. K represents the convolution kernel of size (1, 1, S), and "⊙" indicates a dynamic convolution operation between the two feature maps. Perform element-wise multiplication on two feature maps, F output This represents the final output feature map of the mask generator subnetwork.

[0036] Furthermore, the specific process of step 6 is as follows:

[0037] Set a threshold for the red region on the H channel, segment the red region within the converted mask along the H channel and binarize it, then calculate the proportion P of the red region. r :

[0038] P r =N r / N m

[0039] Where, N r N represents the number of pixels within the red area. m This represents the total number of pixels within the mask.

[0040] 6. The method for identifying elevated strawberries and accurately determining their maturity according to claim 1, characterized in that, in step 7, the strawberry maturity determination criterion is:

[0041] P r ≤70% immature

[0042] 70% < P r <90% mature

[0043] 90% ≤ P r Overripe.

[0044] A system for implementing the above-mentioned method for identifying elevated strawberries and accurately determining their maturity includes:

[0045] The image acquisition module uses image acquisition equipment such as depth cameras to collect strawberry image data in the elevated strawberry cultivation environment.

[0046] The dataset construction module preprocesses the images acquired by the image acquisition module to build the dataset that will subsequently serve as input to the adaptive strawberry feature enhancement network. The preprocessing includes strawberry image enhancement processing.

[0047] The network model building module constructs an adaptive strawberry feature enhancement network to identify strawberries in images and generate strawberry masks; the adaptive strawberry feature enhancement network consists of three parts: a self-attention neural network, a multi-scale feature fusion network, and a decoupled head network.

[0048] The strawberry segmentation and maturity determination module extracts the mature strawberry mask generated by the network model building module, converts the RGB color space in the mask to the HSV color space, segments the red area in the HSV color space based on a set threshold, and calculates the ratio of the number of pixels in the red area to the total number of pixels in the mask. Based on the ratio, the maturity of the strawberry is accurately determined, and the determination result is finally transmitted to the strawberry picking robot to control the picking action of the strawberry picking robot.

[0049] The present invention has the following beneficial effects:

[0050] This invention combines machine vision, deep learning, and image processing technologies to achieve automatic identification and maturity assessment of elevated strawberries in facility environments. By designing an adaptive strawberry feature enhancement network based on deep learning instance segmentation, this invention can efficiently and accurately identify strawberries and generate masks, meeting the high requirements of robot vision systems. The self-attention neural network and multi-scale feature fusion network designed in this invention can extract rich edge and semantic features and fuse strawberry features at different scales, improving feature representation capabilities and the accuracy of strawberry pixel classification. The decoupled head neural network designed in this invention can classify pixels based on the location of strawberries in the image, effectively generating strawberry masks. The strawberry maturity assessment method designed in this invention can convert the RGB color space within the mature strawberry mask to the HSV color space, segment the red region based on a set threshold, and calculate the proportion of the red region to achieve accurate maturity assessment. This invention can accurately determine the optimal picking time for strawberries, providing efficient visual support for strawberry picking robots and possessing high application value. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method for identifying elevated strawberries and accurately determining their maturity as described in this invention;

[0052] Figure 2This is a schematic diagram of the adaptive strawberry feature enhancement network described in this invention;

[0053] Figure 3 This is a schematic diagram of the self-attention neural network, feature pyramid, and strawberry feature adaptive fusion module described in this invention;

[0054] Figure 4 This is a schematic diagram of the decoupled header network described in this invention;

[0055] Figure 5 This is a schematic diagram of the strawberry maturity determination method described in this invention. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0057] Reference Figure 1 As shown, the method for identifying elevated strawberries and accurately determining their maturity according to the present invention specifically includes the following process:

[0058] Step 1: Strawberry Image Acquisition; Use image acquisition equipment to acquire strawberry images in the elevated strawberry cultivation environment, including image data under different lighting and environmental conditions. The acquired data is transmitted to the computer, the original image size is normalized, and then various modules on the computer perform subsequent analysis and processing to achieve accurate identification of strawberries and their maturity.

[0059] Step 2: Strawberry Image Labeling; The strawberries in the image are manually identified as ripe and ready for picking or unripe and not ready for picking, and then their outlines are labeled. In this implementation, manual distinction is made first, and then a trained model is used to perform more detailed "ripe" and "overripe" identification and classification on the images of ripe strawberries ready for picking. This eliminates the need for manual screening and sales, saving labor costs.

[0060] Step 3: Strawberry Image Enhancement. Image enhancement includes performing image enhancement processing on the original image and updating and adjusting the generated JSON file after annotation, as detailed below:

[0061] Step 3.1: Modify the original image using methods such as rotation, cropping, flipping, brightness adjustment, noise and color perturbation;

[0062] Step 3.2: Update the data in the annotation file according to the image transformation method;

[0063] Step 3.3: Use the original dataset and the enhanced dataset as inputs to the subsequent adaptive strawberry feature enhancement network.

[0064] Step 4: As Figure 2 , 3As shown in Figure 4, an adaptive strawberry feature enhancement network is used to identify strawberries and generate strawberry masks. The adaptive strawberry feature enhancement network includes a self-attention neural network, a multi-scale feature fusion network, and a decoupled head network. The processing of the input image by the adaptive strawberry feature enhancement network includes two parts: image feature extraction and strawberry mask generation.

[0065] Image feature extraction; features of the input image are extracted using a self-attention neural network. The specific structure of the self-attention neural network is as follows:

[0066] The first layer is a patch partitioning layer, which uses convolutional layers to divide and flatten the image into blocks. The convolutional kernel size is 4×4, the stride is 4, and the number of channels is 48. The output feature map size is (H / 4, W / 4, 48). Following the first layer are four stages. The first stage includes one linear embedding layer and two Swin Transformer blocks. The linear embedding layer uses the convolutional layer to perform a linear transformation on the data along the channel direction, doubling the number of channels in the feature map. The second stage includes one patch merging layer and two Swin Transformer blocks. The patch merging layer is a downsampling layer that performs a linear transformation along the channel direction, doubling the number of channels in the feature map. The third stage includes one patch merging layer and six Swin Transformer blocks. The fourth stage includes one patch merging layer and two Swin Transformer blocks. block; the feature maps and their sizes output by each stage are S1(H / 4, W / 4, 96), S2(H / 8, W / 8, 192), S3(H / 16, W / 16, 384), and S4(H / 32, W / 32, 768);

[0067] The specific process of downsampling and linear transformation in the patch merging layer is as follows: The patch merging layer groups the feature map of each channel into 2×2 adjacent pixels, and concatenates the pixels at the same position in each group to generate 4 new feature maps; then, these feature maps are concatenated in the depth direction and layer normalization is performed; and a fully connected layer performs a linear transformation in the depth direction to halve the depth of the feature map.

[0068] The Swin Transformer block includes batch normalization (BN), support vector machine (SVM), window multi-head self-attention (W-MSA), and sliding window self-attention (SW-MSA) modules. The calculation formula between two consecutive Swin Transformer blocks is as follows:

[0069]

[0070] in, and z l These represent the output feature maps of the W-MSA / SW-MSA and SVM modules of the l-th Swing Transformer block, respectively.

[0071] The calculation formula for the W-MSA module is:

[0072] / Omega{W-MSA}=4HWC 2 +2M 2 HWC

[0073] Where H and W are the height and width of the input image, respectively, C is the depth of the feature map, and M is the size of each window.

[0074] Multi-scale features are extracted and fused using a multi-scale feature fusion network (which includes a feature pyramid network and a strawberry feature adaptive fusion module). The specific process is as follows:

[0075] Convolutional layers were used to extract features from the four feature maps output by the self-attention neural network. The kernel size was 1×1, the stride was 1, and the number of channels was 256. The resulting output feature maps and their sizes were F1(H / 4, W / 4, 256), F2(H / 8, W / 8, 256), F3(H / 16, W / 16, 256), and F4(H / 32, W / 32, 256). Then, a convolutional layer is used to downsample feature map F4 with a kernel size of 2×2, a stride of 2, and 256 channels, resulting in an output feature map size of (H / 64, W / 64, 256), which is output 5. Feature map F4 is then upsampled and added element-wise to feature map F3, resulting in an output feature map size of (H / 32, W / 32, 256), which is output 4. Output 4 is then upsampled and added element-wise to feature map F3, resulting in an output feature map size of (H / 16, W / 16, 256), which is output 3. Output 3 is then upsampled and added element-wise to feature map F2, resulting in an output feature map size of (H / 8, W / 8, 256), which is output 2. Output 2 is then upsampled and added element-wise to feature map F1, resulting in an output feature map size of (H / 4, W / 4, 256), which is output 1.

[0076] The five feature maps output by the feature pyramid network are input into the strawberry feature adaptive fusion module; for the feature map x of the m-th layer... m (m∈{1, 2, 3, 4, 5}), the feature maps x of other layers n Adjust to be similar to x by upsampling or downsampling. mUsing the same size, and performing adaptive fusion, we obtain the output feature map of the m-th layer.

[0077] The adaptive fusion strategy formula for the strawberry feature adaptive fusion module is as follows:

[0078]

[0079] in, Let represent the feature vector mapped from layer n to layer m at pixel (i, j). This represents the (i, j)th vector of the output feature map along the channel direction; Determined by the following calculation formula:

[0080]

[0081] in, They are respectively based on These are control parameters defined by the softmax function. represents the trainable weights for each feature map layer, and μ is the weight coefficient.

[0082] Strawberry Mask Generation: The strawberry mask is generated using a decoupled head network. The decoupled head network consists of two sub-networks for instance category and mask generation, and its specific working process is as follows:

[0083] The instance classification subnetwork first predefines a grid size S (S∈{40, 36, 24, 16, 12}), and adjusts the feature map size to (S, S, 256) through one convolutional layer; then, it extracts features through seven convolutional layers, integrates these features, and adjusts the feature map size to (S, S, C). c ), where C c The number of categories is denoted by ; the centroid of the strawberry falls in the (i, j)th grid, which is responsible for predicting the category of the strawberry. Each channel outputs the probability that the strawberry belongs to a certain category.

[0084] The mask generation subnetwork consists of two branches. Each branch first encodes spatial information by concatenating a coordinate channel onto the original feature map using a convolutional layer, outputting a feature map with dimensions (H, W, 256+1). Then, it extracts features through seven convolutional layers, integrates these features, and adjusts the feature map size to (H, W, S). When a strawberry falls into a grid, that grid is responsible for predicting the classification of the pixel within the mask. Finally, the prediction results from both branches are integrated, and pixels classified as strawberries are remapped back to the original image. This result is then fused with the result from the instance classification subnetwork using the following formula to obtain the final strawberry mask (including ripe and unripe strawberry masks) and its category:

[0085]

[0086] Where sigmoid is the activation function, F j and F i Let J and L be the feature maps of the j-th and i-th channels, respectively. K represents the convolution kernel of size (1, 1, S), and "⊙" indicates a dynamic convolution operation between the two feature maps. Perform element-wise multiplication on two feature maps, F output This represents the final output feature map of the mask generator subnetwork.

[0087] Step 5: As Figure 5 As shown, a mask of ripe strawberries is extracted, and the RGB color space within the mask is converted to the HSV color space;

[0088] The formula for converting RGB color space to HSV color space is as follows:

[0089] C max =max(R, G, B), C min = (R, G, B)

[0090] Δ=C max -C min

[0091]

[0092]

[0093] V = C max

[0094] Where R, G, and B are the normalized values ​​of the three channels of the RGB image, respectively; C max Represents the largest value among R, G, and B; C min Represents the minimum value among R, G, and B; Δ represents the difference between the maximum and minimum values ​​among R, G, and B; H represents the hue value; S v V represents the saturation value; V represents the brightness value.

[0095] Step 6: As Figure 5 As shown, the red region in the HSV color space is segmented based on a set threshold, and the ratio of the number of pixels in the red region to the total number of pixels in the mask is calculated. The specific process is as follows:

[0096] Set a threshold for the red region on the H channel, segment the red region within the converted mask along the H channel and binarize it, then calculate the proportion P of the red region. r :

[0097] P r =N r / N m

[0098] Where, Nr N represents the number of pixels within the red area. m This represents the total number of pixels within the mask.

[0099] The ripeness of strawberries is determined based on a set threshold for the red area, which can be dynamically adjusted according to different strawberry varieties.

[0100] Step 7: Accurately determine the ripeness of strawberries based on the calculated ratio. The criteria for determining strawberry ripeness are as follows:

[0101] P r ≤70% immature

[0102] 70% < P r <90% mature

[0103] 90% ≤ P r Overripe

[0104] The maturity determination result is used to control the picking action of the strawberry picking robot, so that it can divide the strawberries into boxes according to their maturity level while picking them. In addition, since deep learning models can also make mistakes and are not 100% accurate, they may mistakenly classify a small portion of "unripe" strawberries as "ripe" strawberries that can be picked. Therefore, this embodiment further sets the discrimination criteria for unripe strawberries in step 7 and performs another accurate judgment.

[0105] This invention also provides a system for identifying elevated strawberries and accurately determining their maturity, including an image acquisition module, a dataset construction module, a network model building module, and a strawberry segmentation and maturity determination module.

[0106] The image acquisition module uses image acquisition equipment such as depth cameras to collect strawberry image data in the elevated strawberry cultivation environment.

[0107] The dataset construction module preprocesses the images acquired by the image acquisition module to build a dataset that will subsequently serve as input to the adaptive strawberry feature enhancement network. The preprocessing includes strawberry image enhancement.

[0108] The network model building module constructs an adaptive strawberry feature enhancement network to identify strawberries in images and generate strawberry masks; the adaptive strawberry feature enhancement network consists of three parts: a self-attention neural network, a multi-scale feature fusion network, and a decoupled head network.

[0109] The strawberry segmentation and maturity determination module converts the RGB color space within the strawberry mask generated by the network model building module into the HSV color space. Based on a set threshold, it segments the red region in the HSV color space and calculates the ratio of the number of pixels in the red region to the total number of pixels in the mask. Based on this ratio, it accurately determines the ripeness of the strawberries and transmits the determination result to the strawberry picking robot to control the picking action of the strawberry picking robot.

[0110] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for identifying elevated strawberries and accurately determining their maturity, characterized in that, The process includes the following: Step 1: Acquiring strawberry images; Step 2: Manually determine whether the strawberries in the image are ripe and ready to be picked or unripe and not ready to be picked, and then perform contour instance annotation; Step 3: Perform image enhancement processing on the original image and update and adjust its annotation file accordingly; Step 4: Use an adaptive strawberry feature enhancement network to identify strawberries in the image and generate a strawberry mask; The adaptive strawberry feature enhancement network processes the input image by: extracting features from the input image through a self-attention neural network, then extracting and fusing multi-scale features through a multi-scale feature fusion network; and generating a strawberry mask by decoupling the head network. Step 5: Extract the mask of ripe strawberries and convert the RGB color space within the mask to the HSV color space; Step 6: Segment the red region in the HSV color space based on the set threshold, and calculate the ratio of the number of pixels in the red region to the total number of pixels in the mask; Step 7: Based on the calculated ratio, determine the ripeness of the strawberries according to the set ripeness judgment criteria. The ripeness judgment result is used to control the picking action of the strawberry picking robot. In step 4, the decoupled head network includes two sub-networks for instance category and mask generation. The specific process of generating the strawberry mask through the decoupled head network is as follows: The instance classification subnetwork first predefines a grid size S (S∈{40, 36, 24, 16, 12}), and adjusts the feature map size to (S, S, 256) through one convolutional layer; then, it extracts features through seven convolutional layers, integrates these features, and adjusts the feature map size to (S, S, C). c ), C c The number of categories is denoted by ; the centroid of the strawberry falls in the (i, j)th grid, which is responsible for predicting the category of the strawberry. Each channel outputs the probability that the strawberry belongs to a certain category. The mask generation sub-network consists of two branches. Each branch first encodes spatial information by concatenating a coordinate channel onto the original feature map using a convolutional layer, outputting a feature map with dimensions (H, W, 256+1). Then, it extracts features through seven convolutional layers, integrates these features, and adjusts the feature map size to (H, W, S). When a strawberry falls into a grid, that grid is responsible for predicting the classification of the pixel within the mask. Finally, the prediction results from both branches are integrated, and pixels classified as strawberries are remapped back to the original image. This result is then fused with the result from the instance classification sub-network using the following formula to obtain the final strawberry mask and its classification: ； Where sigmoid is the activation function. and Let J and L be the feature maps of the j-th and i-th channels, respectively, and K represent the convolution kernel of size (1, 1, S). "Perform dynamic convolution operations on two feature maps," Perform element-wise multiplication on the two feature maps. This represents the final output feature map of the mask generator subnetwork.

2. The method for identifying elevated strawberries and accurately determining their maturity according to claim 1, characterized in that, In step 4, the specific structural arrangement of the self-attention neural network is as follows: The first layer is a patch partitioning layer, which uses convolutional layers to divide and flatten the image into blocks. The convolutional kernel size is 4×4, the stride is 4, and the number of channels is 48. The output feature map size is (H / 4, W / 4, 48). Following the first layer are four stages. The first stage includes one linear embedding layer and two Swin Transformer blocks. The linear embedding layer uses convolutional layers to perform a linear transformation on the data along the channel direction, doubling the number of channels in the feature map. The second stage includes one patch merging layer and two Swin Transformer blocks. The patch merging layer is a downsampling layer that performs a linear transformation along the channel direction, doubling the number of channels in the feature map. The third stage includes one patch merging layer and six Swin Transformer blocks. The fourth stage includes one patch merging layer and two Swin Transformer blocks. block; the feature maps and their sizes output by each stage are S1(H / 4, W / 4, 96), S2(H / 8, W / 8, 192), S3(H / 16, W / 16, 384), and S4(H / 32, W / 32, 768). The Swin Transformer block includes batch normalization, support vector machine, multi-head self-attention for windows, and sliding window self-attention modules. The calculation formula between two consecutive Swin Transformer blocks is as follows: ； ； ； ； in, and These represent the output feature maps of the W-MSA / SW-MSA and SVM modules of the l-th Swing Transformer block, respectively. The calculation formula for the W-MSA module is: ； Where H and W are the height and width of the input image, respectively, C is the depth of the feature map, and M is the size of each window.

3. The method for identifying elevated strawberries and accurately determining their maturity according to claim 1, characterized in that, In step 4, the multi-scale feature fusion network includes a feature pyramid network and a strawberry feature adaptive fusion module. The specific process of extracting and fusing multi-scale features through the multi-scale feature fusion network is as follows: Feature extraction is performed on the four feature maps output by the self-attention neural network using convolutional layers. The kernel size is 1×1, the stride is 1, and the number of channels is 256. The resulting output feature maps and their sizes are F1 (H / 4, W / 4, 256), F2 (H / 8, W / 8, 256), F3 (H / 16, W / 16, 256), and F4 (H / 32, W / 32, 256), respectively. Then, feature map F4 is downsampled using a convolutional layer with a kernel size of 2×2, a stride of 2, and 256 channels, resulting in an output feature map size of (H / 64, W / 64, 256), which is output 5. Feature map F4 is then upsampled and added element-wise to feature map F3, resulting in an output feature map size of (H / 32, W / 32, 256), which is output 4. Upsample output 4 and add it element-wise to feature map F3, resulting in an output feature map of size (H / 16, W / 16, 256), which is output 3; Upsample output 3 and add it element-wise to feature map F2, resulting in an output feature map of size (H / 8, W / 8, 256), which is output 2; Upsample output 2 and add it element-wise to feature map F1, resulting in an output feature map of size (H / 4, W / 4, 256), which is output 1. The five feature maps output by the feature pyramid network are input into the strawberry feature adaptive fusion module; for the feature map x of the m-th layer... m (m∈{1, 2, 3, 4, 5}), the feature maps x of other layers n Adjust to be similar to x by upsampling or downsampling. m Using the same size and performing adaptive fusion, the output feature map of the m-th layer is obtained; The adaptive fusion strategy formula for the strawberry feature adaptive fusion module is as follows: ； in, Let represent the feature vector mapped from layer n to layer m at pixel (i, j). This represents the (i, j)th vector of the output feature map along the channel direction; , , , , Determined by the following calculation formula: ； ； ； ； ； ； in, , , , , They are respectively based on , , , , These are control parameters defined by the softmax function. , , , , For each layer of feature maps, the trainable weights are... These are the weighting coefficients.

4. The method for identifying elevated strawberries and accurately determining their maturity according to claim 1, characterized in that, The specific process of step 6 is as follows: Set a threshold for the red region on the H channel, segment the red region within the converted mask along the H channel and binarize it, then calculate the proportion P of the red region. r : ； Where, N r N represents the number of pixels within the red area. m This represents the total number of pixels within the mask.

5. The method for identifying elevated strawberries and accurately determining their maturity according to claim 1, characterized in that, In step 7, the criteria for judging the ripeness of strawberries are as follows: Immature; Mature; Overripe.

6. A system for implementing the method for identifying elevated strawberries and accurately determining their maturity as described in claim 1, characterized in that, include: The image acquisition module uses image acquisition equipment such as depth cameras to collect strawberry image data in the elevated strawberry cultivation environment; The dataset construction module preprocesses the images acquired by the image acquisition module to build a dataset that will serve as input to the adaptive strawberry feature enhancement network. The preprocessing includes strawberry image enhancement processing. The network model building module constructs an adaptive strawberry feature enhancement network to identify strawberries in images and generate strawberry masks. The adaptive strawberry feature enhancement network consists of three parts: a self-attention neural network, a multi-scale feature fusion network, and a decoupled head network. The strawberry segmentation and maturity determination module extracts the mature strawberry mask generated by the network model building module, converts the RGB color space in the mask to the HSV color space, segments the red area in the HSV color space based on a set threshold, calculates the ratio of the number of pixels in the red area to the total number of pixels in the mask, accurately determines the maturity of the strawberry based on the ratio, and finally transmits the determination result to the strawberry picking robot to control the picking action of the strawberry picking robot.

Images