Method for recognizing and positioning flower buds and flowers for picking robot to carry out partitioned picking

Abstract

The invention provides a flower bud and flower recognition and positioning method for a picking robot to perform zoned picking, and belongs to the technical field of picking robots. Inputting the segmented mask into a multi-scale feature fusion semantic recognition model based on hierarchical attention aggregation to perform preliminary classification of foreground flowers and background vegetation, performing a graph cut energy minimization segmentation algorithm on the preliminary segmentation mask to construct an undirected graph, and calculating a minimum cut set to obtain an accurate flower segmentation mask; key point heat map prediction is carried out on separated flower individual masks to extract flower center point coordinates and petal tip coordinates, depth image channel information is combined to calculate three-dimensional space position coordinates of each flower, and a flower three-dimensional positioning database is established for picking robot path planning; the problem of insufficient segmentation precision caused by serious overlapping of foreground flowers and background vegetation in a color space in a flower picking scene is solved.

Description

Technical Field

[0001] This invention belongs to the field of harvesting robot technology, and more specifically, it relates to a method for identifying and locating flower buds and blossoms for harvesting robots to perform zoned harvesting. Background Technology

[0002] In the visual recognition system of flower-picking robots, traditional methods mainly employ semantic segmentation networks based on color space thresholding or single-scale feature extraction to achieve flower identification and localization. Color thresholding methods distinguish flowers from the background by setting threshold ranges in the HSV or LAB color space, suitable for scenarios with high color contrast. However, in actual planting environments, the color features of green buds and green leaves are highly similar, making it difficult for traditional color segmentation methods to effectively differentiate them. Semantic segmentation networks based on deep learning extract image features and perform pixel-level classification using convolutional neural networks. However, existing methods mostly employ single-scale feature extraction strategies, relying solely on high-level semantic features for segmentation decisions, ignoring the feature differences of flower targets at different spatial scales. In current automated picking applications in flower planting bases, due to the severe overlap in color space between foreground flowers and background vegetation, especially when both buds and leaves are green, existing single-scale feature extraction methods cannot capture the subtle differences in texture details and local structures between buds and leaves, resulting in unclear segmentation boundaries and low individual recognition accuracy. In other words, existing technologies suffer from the technical problem of insufficient segmentation accuracy caused by severe overlap in color space between foreground flowers and background vegetation in flower-picking scenarios. Summary of the Invention

[0003] In view of this, the present invention provides a method for identifying and locating flower buds and flowers for a picking robot to perform segmented picking, which can solve the technical problem in the prior art where the foreground flowers and background vegetation overlap severely in the color space, resulting in insufficient segmentation accuracy.

[0004] This invention is implemented as follows: This invention provides a method for identifying and locating flower buds and blossoms for a harvesting robot to perform zoned harvesting, comprising the following steps: acquiring RGBD four-channel image data of the target flower planting area, the RGBD four-channel image data including color image channels and depth image channels; inputting the RGBD four-channel image data into a flower semantic recognition model for preliminary classification of foreground flowers and background vegetation, outputting a preliminary segmentation mask and a set of coordinates for candidate flower regions; the flower semantic recognition model employs a multi-scale feature fusion algorithm based on hierarchical attention aggregation; and performing graph cut energy analysis on the candidate flower regions in the preliminary segmentation mask. A minimization segmentation algorithm is used to construct an undirected graph and calculate the minimum cut set to obtain an accurate flower segmentation mask. Morphological gradient and watershed transformation algorithms are then applied to the accurate flower segmentation mask to calculate the morphological gradient map and mark the foreground and background regions. Watershed transformation is used to separate adhered individual flowers. Keypoint heatmap prediction is performed on the separated flower individual masks to extract the coordinates of the flower center point and petal tip. Valid keypoints are selected using non-maximum suppression. Based on the keypoint coordinate set and the flower individual mask, the 3D spatial coordinates of each flower are calculated. Combined with depth image channel information, a 3D flower positioning database is established for path planning by the harvesting robot.

[0005] The acquisition of the RGBD four-channel image data is carried out synchronously using a binocular vision system composed of a color camera and a depth camera. The RGB three-channel color image and the depth image channel are then registered and aligned at the pixel level to form the RGBD four-channel image data.

[0006] The flower semantic recognition model adopts a three-segment design of Backbone-Neck-Head. The Backbone part uses a residual network to extract multi-level image features and outputs low-level feature maps, mid-level feature maps and high-level feature maps.

[0007] The multi-scale feature fusion algorithm based on hierarchical attention aggregation constructs three parallel attention branches to process low-level feature maps, mid-level feature maps and high-level feature maps respectively, and expands the receptive field and generates attention weight maps through dilated convolution.

[0008] The attention weight map is multiplied element-wise with the corresponding feature map to obtain a weighted feature map. The three weighted feature maps are then subjected to bilinear interpolation to unify them to the same spatial resolution before feature alignment.

[0009] Specifically, the aligned feature maps are weighted and fused using learnable scale weight parameters, which include low-level weights, mid-level weights, and high-level weights. The scale weight parameters are initialized to a uniform distribution and then adaptively learned through backpropagation.

[0010] The initial values ​​of the low-level weights, middle-level weights, and high-level weights are determined based on the depth variance of the RGBD four-channel image data, the average size of the flower candidate region coordinate set, and the image gradient intensity.

[0011] The establishment of the training dataset for the flower semantic recognition model includes: collecting RGBD four-channel image data samples of flowers of different varieties, different growth stages, different lighting conditions and different degrees of occlusion, and performing pixel-level semantic annotation using image annotation tools.

[0012] The graph cut energy minimization segmentation algorithm constructs each pixel in the image region corresponding to the precise flower segmentation mask as a node of an undirected graph, connecting spatially adjacent pixels with undirected edges.

[0013] The weight of the undirected edge is determined by the color similarity and spatial distance between adjacent pixels. The color similarity is calculated by Euclidean distance in the RGB color space, and the spatial distance is measured by Manhattan distance in the pixel coordinates.

[0014] In this approach, a source node is added to represent foreground flowers and a sink node to represent background vegetation in the undirected graph. A preliminary segmentation mask is used to provide seed point constraints. The maximum flow minimum cut theorem is used to transform the problem of minimizing the energy function into a maximum flow problem on the undirected graph.

[0015] The morphological gradient and watershed transform algorithm performs morphological dilation and morphological erosion operations on the precise flower segmentation mask, and calculates the pixel value difference between the dilated image and the eroded image to obtain the morphological gradient map.

[0016] Specifically, a region labeling operation is performed on the morphological gradient map. The precise flower segmentation mask is transformed by distance to obtain the distance value from each foreground pixel to the nearest background pixel. Local maximum detection is performed on the distance transformed map to extract the distance peak points as foreground marker points.

[0017] The key point heatmap prediction process involves inputting the image block corresponding to the mask cropping of each individual flower into the key point detection network. The key point detection network adopts an encoder-decoder structure and outputs heatmaps of the flower center point, petal tip points, and sepal connection points.

[0018] The non-maximum suppression method performs a sliding window scan on the heatmaps of the flower center point, the petal tip point, and the calyx connection point, retaining the pixel with the largest response value within the sliding window as a candidate key point.

[0019] The calculation of the three-dimensional spatial position coordinates utilizes the pixel coordinates of the flower's center point in the RGB three-channel color image, queries the depth value of the corresponding pixel position in the depth image channel, converts the pixel coordinates and depth values ​​into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameter matrix, and converts them into three-dimensional spatial position coordinates in the world coordinate system through the camera extrinsic parameter matrix.

[0020] This invention constructs a multi-scale feature fusion algorithm based on hierarchical attention aggregation. It establishes three parallel attention branches with different hole rates within the semantic recognition model, capturing small-scale texture details, medium-scale shape contours, and large-scale spatial distribution contextual features, respectively. Learnable scale weight parameters are used to adaptively weight and fuse features at different levels, enabling the model to simultaneously focus on the fine-grained texture differences of flowers and overall region localization information. For challenging samples where green buds and green leaves severely overlap in color space, the low-level attention weight map learns the fine texture features of the bud surface, the medium-level attention weight map captures the shape difference between the compact circular contour of the bud and the flat elliptical contour of the leaf, and the high-level attention weight map utilizes the spatial distribution pattern where buds are typically located at the top of branches and leaves are distributed on the sides. The complementary fusion of these three scale features significantly enhances the model's ability to distinguish color-similar targets. Combined with a cross-scale feature alignment mechanism, it eliminates semantic biases between features of different resolutions, ensuring that the fused features accurately correspond to the same physical region in spatial location. This solves the technical problem of insufficient segmentation accuracy caused by severe overlap in color space between foreground flowers and background vegetation in flower picking scenarios. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the flower semantic recognition model involved in the present invention.

[0022] Figure 2 This is a front view of the overall structure of the rose-picking robot.

[0023] Figure 3 This is a side view of the overall structure of the rose-picking robot.

[0024] Figure 4 This is a schematic diagram of the field of view of a depth camera.

[0025] Figure 5 This is the model detection result image and the corresponding depth map.

[0026] In the attached drawings, the reference numerals are explained as follows: 1. Double-arm linear module, 2. Tracked self-propelled chassis, 3. High-brightness display screen, 4. Depth camera, 5. Harvesting end, 6. Sunshade mounting frame. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0028] This invention provides a method for identifying and locating flower buds and blossoms for a harvesting robot to perform zoned harvesting, comprising the following steps:

[0029] S01. Acquire RGBD four-channel image data of the target flower planting area, wherein the RGBD four-channel image data includes color image channels and depth image channels;

[0030] S02. Input the RGBD four-channel image data into the flower semantic recognition model to perform preliminary classification of foreground flowers and background vegetation, and output the preliminary segmentation mask and the coordinate set of flower candidate regions.

[0031] S03. Perform the graph cut energy minimization segmentation algorithm on the flower candidate regions in the preliminary segmentation mask, construct an undirected graph and calculate the minimum cut set to obtain the accurate flower segmentation mask;

[0032] S04. Perform morphological gradient and watershed transformation algorithms on the precise flower segmentation mask, calculate the morphological gradient map and mark the foreground and background regions, and separate the sticky flower individuals through watershed transformation.

[0033] S05. Perform key point heatmap prediction on the individual flower masks after separation, extract the coordinates of the flower center point and the petal tip coordinates, and filter effective key points through non-maximum suppression.

[0034] S06. Calculate the three-dimensional spatial coordinates of each flower based on the key point coordinate set and the individual flower mask, and establish a three-dimensional flower positioning database by combining the depth image channel information for path planning of the picking robot.

[0035] The acquisition method for the RGBD four-channel image data is as follows: a binocular vision system consisting of a color camera and a depth camera is used for simultaneous acquisition. The color camera acquires RGB three-channel color images, and the depth camera acquires scene depth information as the fourth channel depth image using structured light or time-of-flight method. The RGB three-channel color images and the depth image channel are then pixel-level registered and aligned to form RGBD four-channel image data. The depth image channel can provide information on the distance difference between the flower and the background in three-dimensional space. Utilizing the spatial distribution characteristic that flowers are usually located in front of background vegetation, it assists in subsequent segmentation decisions and reduces the false detection rate caused by color space overlap.

[0036] like Figure 1As shown, the specific structure of the flower semantic recognition model is as follows: The flower semantic recognition model adopts a three-stage design of Backbone-Neck-Head. The Backbone part uses a residual network to extract multi-level image features, outputting three feature maps of different scales, denoted as low-level feature map, mid-level feature map, and high-level feature map, respectively. The Neck part implements a multi-scale feature fusion algorithm based on hierarchical attention aggregation, constructing three parallel attention branches to process the low-level feature map, mid-level feature map, and high-level feature map, respectively. Within each attention branch, the receptive field is expanded through dilated convolution and an attention weight map is generated. The attention weight map is then element-wise compared with the corresponding feature map. The weighted feature maps are multiplied, and bilinear interpolation is performed on the three weighted feature maps to unify them to the same spatial resolution before feature alignment. The aligned feature maps are then weighted and fused using learnable scale weight parameters. The scale weight parameters are initialized to a uniform distribution and then adaptively learned through backpropagation. The Head part uses a decoder structure to progressively upsample the fused feature maps to restore them to the input image resolution. It outputs the probability distribution map of each pixel belonging to the flower, leaf, branch, and background categories. An initial segmentation mask is obtained through argmax operation, and the bounding box coordinates of high-confidence regions are extracted using non-maximum suppression as the coordinate set of flower candidate regions.

[0037] The steps for establishing the training dataset for the flower semantic recognition model specifically include: collecting RGBD four-channel image data samples of flowers of different varieties, growth stages, light conditions, and occlusion levels from multiple flower planting bases, with a total number of samples not less than 10,000; using image annotation tools to perform pixel-level semantic annotation on each RGB three-channel color image, labeling each pixel in the RGB three-channel color image as one of five categories: flower, bud, leaf, branch, or background, forming a semantic annotation mask image; performing data augmentation operations on some samples, including random cropping, random flipping, color jittering, and scaling transformation, to expand the training dataset to 30,000 sets; randomly dividing the training dataset according to the ratio of training set, validation set, and test set of 8:1:1, with the training set used for updating the parameters of the flower semantic recognition model, the validation set used for hyperparameter adjustment, and the test set used for final performance evaluation.

[0038] The specific steps for training the flower semantic recognition model include: using the cross-entropy loss function to measure the difference between the predicted segmentation mask and the ground truth labeled mask, and setting higher loss weights for flower and bud categories to address the class imbalance problem; updating the parameters of the flower semantic recognition model using a stochastic gradient descent optimizer, with an initial learning rate of 0.01, and gradually reducing the learning rate to 0.0001 using a cosine annealing strategy; the training process is divided into a pre-training phase and a fine-tuning phase, in which the Backbone parameters are frozen and only the Neck and Head parts are trained for 20 epochs, and in the fine-tuning phase, all parameters are unfrozen and end-to-end training is performed for 80 epochs; after each epoch, the performance of the flower semantic recognition model is evaluated on the validation set, the average intersection-over-union ratio (IoU) is recorded, and the weights of the flower semantic recognition model with the highest IoU are saved as the final flower semantic recognition model; during training, a batch normalization layer is used to stabilize gradient propagation, the batch size is set to 16, and four GPUs are used for distributed training to accelerate convergence.

[0039] The implementation of the multi-scale feature fusion algorithm based on hierarchical attention aggregation is as follows: First, dilated convolutions with a dilation rate of 2 are performed on the low-level feature maps to generate low-level receptive field features; dilated convolutions with a dilation rate of 4 are performed on the mid-level feature maps to generate mid-level receptive field features; and dilated convolutions with a dilation rate of 8 are performed on the high-level feature maps to generate high-level receptive field features. These three dilated convolutions with different dilation rates capture small-scale texture details, mid-scale local structure, and large-scale global context information, respectively. The low-level receptive field features are then processed through… Convolution generates query vector, key vector, and value vector. The dot product of the query vector and key vector is calculated and normalized to obtain a low-level attention weight map. The low-level attention weight map is multiplied by the value vector to obtain a low-level weighted feature map. The same operation is performed on the mid-level receptive field features and the high-level receptive field features to obtain mid-level attention weight maps, mid-level weighted feature maps, high-level attention weight maps, and high-level weighted feature maps, respectively. The low-level weighted feature maps, mid-level weighted feature maps, and high-level weighted feature maps are upsampled or downsampled to the spatial size of the mid-level feature map through bilinear interpolation to eliminate spatial alignment bias between features of different resolutions, resulting in aligned low-level feature maps, aligned mid-level feature maps, and aligned high-level feature maps. Three learnable features are introduced. The scale weight parameters are denoted as low-level weight, mid-level weight, and high-level weight, respectively. A weighted summation operation is performed on the aligned low-level feature map, aligned mid-level feature map, and aligned high-level feature map. The weighted summation formula is: the fused feature map equals the low-level weight multiplied by the aligned low-level feature map plus the mid-level weight multiplied by the aligned mid-level feature map plus the high-level weight multiplied by the aligned high-level feature map. The sum of the three scale weight parameters (low-level weight, mid-level weight, and high-level weight) is normalized to 1 to maintain the stability of the feature amplitude. After the low-level weight, mid-level weight, and high-level weight are initialized to a uniform distribution, they are adaptively adjusted through backpropagation during training. The flower semantic recognition model automatically learns the contribution of features at different scales to the flower segmentation task.

[0040] The multi-scale feature fusion algorithm based on hierarchical attention aggregation brings the following technical effects to the flower semantic recognition model: By constructing three parallel attention branches with different void ratios, the flower semantic recognition model can simultaneously capture the fine-grained texture features, medium-scale shape and contour features, and large-scale spatial distribution context features of flowers. The complementary fusion of features from different receptive fields significantly enhances the flower semantic recognition model's ability to distinguish between flowers and the background. The introduction of the attention mechanism enables the flower semantic recognition model to dynamically strengthen the feature expression of the flower region while suppressing interference from irrelevant backgrounds. For difficult samples such as green buds and green leaves with severely overlapping color spaces, the low-level, mid-level, and high-level attention weight maps can learn the subtle differences in texture details and local structures between buds and leaves, thus accurately distinguishing them. The cross-scale feature alignment mechanism unifies the spatial resolution of feature maps at different levels through bilinear interpolation, eliminating the need for direct splicing. The semantic bias generated by multi-scale features is addressed to ensure that the fused feature maps accurately correspond to the same physical region in spatial location. The learnable scale weight parameters enable the flower semantic recognition model to adaptively adjust the contribution of features at different scales. During training, the flower semantic recognition model automatically discovers the importance of low-level feature maps for petal edge details and the importance of high-level feature maps for the overall flower region localization, dynamically balancing multi-scale information fusion to avoid information loss caused by the dominance of a certain scale feature. The multi-scale feature fusion algorithm based on hierarchical attention aggregation brings technical benefits to the entire bud and flower recognition and localization scheme. It significantly improves the accuracy of the initial segmentation mask, provides high-quality seed region constraints for subsequent graph cut energy minimization segmentation algorithms, reduces the number of graph cut iterations, and improves the overall processing efficiency. At the same time, the accurate flower candidate region coordinate set reduces the search space of subsequent morphological gradient and watershed transformation algorithms and key point heatmap prediction, accelerates the individual recognition process, and reduces the false detection rate.

[0041] The graph cut energy minimization segmentation algorithm is implemented as follows: Each pixel in the image region corresponding to the precise flower segmentation mask is constructed as a node in an undirected graph. Spatially adjacent pixels are connected by undirected edges. The weight of each undirected edge is determined by the color similarity and spatial distance between adjacent pixels. Color similarity is measured using Euclidean distance in the RGB color space, and spatial distance is measured using Manhattan distance in pixel coordinates. The formula for calculating the weight of an undirected edge is: weight equals color similarity weight factor multiplied by the negative exponent of color similarity, plus spatial distance weight factor multiplied by the negative exponent of spatial distance. The color similarity weight factor and spatial distance weight factor are determined through validation set tuning. A source point representing foreground flowers and a sink point representing background vegetation are added to the undirected graph. The initial segmentation mask output by the flower semantic recognition model provides seed point constraints. Pixels marked as flower category in the initial segmentation mask are connected to the source point, and pixels marked as background category in the initial segmentation mask are connected to the sink point. The weight of the connecting edges is set to infinity to represent a hard constraint. An energy function is constructed, including a data term and a smoothing term. The data term measures the attribute of each pixel. The log-likelihood probability of foreground flowers or background vegetation is used to train a Gaussian mixture model (GMM) to learn the color distribution of the foreground and background by sampling pixel color values ​​from the seed region. The GMM contains five Gaussian components, and the mean and covariance matrix of each Gaussian component are estimated using the expectation-maximization algorithm. A smoothing term penalizes cases where adjacent pixels are assigned to different categories, and the weight of the smoothing term is inversely proportional to the color gradient between pixels, encouraging the segmentation boundary to follow the edge position with a large image gradient. The energy function minimization problem is transformed into a maximum flow problem on an undirected graph using the maximum flow minimum cut theorem. The push-pull algorithm is used to iteratively find augmenting paths from the source to the sink and update the residual network until no augmenting path exists. At this point, the minimum cut set of the residual network corresponds to the optimal solution of the energy function. The foreground and background labels of the pixels are updated according to the minimum cut set result. The GMM is then retrained using the updated foreground and background labels. The graph cut and GMM update process is iteratively executed 5 times until convergence, finally obtaining an accurate flower segmentation mask. The boundary of the accurate flower segmentation mask has sub-pixel accuracy along the true boundary line between the petals and the background.

[0042] The implementation of the morphological gradient and watershed transform algorithm is as follows: Morphological dilation and morphological erosion operations are performed on the precise flower segmentation mask. The morphological dilation operation uses a circular structuring element with a radius of 3 pixels to expand the foreground region outward, and the morphological erosion operation uses a circular structuring element with a radius of 3 pixels to shrink the foreground region inward. The pixel value difference between the dilated and eroded images is calculated to obtain the morphological gradient map. The morphological gradient map exhibits a high-intensity response at the flower edges and adhesion areas. A region labeling operation is performed on the morphological gradient map. The precise flower segmentation mask is transformed to obtain the distance value from each foreground pixel to the nearest background pixel. Local maxima detection is performed on the distance transformed map to extract the distance peak points as foreground marker points. Each foreground marker point is assigned a unique label value representing a potential flower individual. The background region outside the precise flower segmentation mask is uniformly labeled as the background label. The morphological gradient map is treated as a three-dimensional terrain surface, with the gradient value representing the terrain height. The foreground marker points are treated as water sources, and the watershed transform algorithm is executed to simulate the gradual rise of water level from low to high. As water from different foreground markers rises, it gradually spreads. When two streams of water merge, a watershed line is constructed at the confluence as a segmentation boundary. The watershed line generated by the watershed transform algorithm separates the adhered flower individuals into independent connected regions. Each connected region corresponds to a flower individual mask. Because there is usually a gradient indentation between contacting flowers, the watershed transform algorithm can accurately divide the watershed line at the gradient indentation. To address the oversegmentation problem, a region merging strategy is adopted. The average color feature and average gradient feature of adjacent connected regions are calculated. When the color difference between adjacent connected regions is less than the color difference threshold and the boundary gradient strength is lower than the gradient strength threshold, it is determined to be oversegmented. The two adjacent connected regions are merged into the same flower individual mask. The color difference threshold and gradient strength threshold are determined by statistically analyzing the feature distribution of normally segmented and oversegmented regions in the training set. The combination of distance transform and watershed transform algorithms ensures that each flower individual mask contains at least one distance peak point as a foreground marker point, avoiding missed detections caused by occlusion and improving the reliability of individual recognition in dense scenes.

[0043] The keypoint heatmap prediction is implemented as follows: Image patches corresponding to each individual flower are cropped using a mask and input into a keypoint detection network. This network employs an encoder-decoder structure. The encoder layer progressively reduces the feature map resolution through convolutional and pooling layers to extract high-level semantic information. The decoder layer progressively restores the feature map resolution through deconvolutional layers and skip connections to generate a high-resolution heatmap. The keypoint detection network outputs heatmaps with multiple channels, each corresponding to a keypoint type, including heatmaps of the flower center point, petal tip points, and sepal connection points. The value of each pixel in the heatmap represents the confidence level that a corresponding keypoint exists at that pixel location. During training, the keypoint detection network uses real keypoint coordinates to generate a Gaussian-distributed target heatmap. The peak center of the Gaussian distribution is located at the real keypoint coordinates, the standard deviation of the Gaussian distribution is set to 2 pixels, and the loss function uses mean squared error. The difference between the predicted heatmap and the target heatmap is analyzed. Non-maximum suppression is applied to the heatmaps of the flower center point, petal tips, and calyx connection points to extract local peak points. A confidence threshold is set to filter low-confidence responses, and the coordinates of the retained local peak points are the detected keypoint locations, resulting in a keypoint coordinate set. Based on prior knowledge of the flower's anatomical structure, geometric constraints on keypoints are established. The distance from the flower center point to each petal tip should follow a preset distribution, and the angle between petal tips should be approximately uniformly distributed. False keypoints that do not conform to the anatomical structure are eliminated using these geometric constraints, retaining the keypoint coordinate set with optimal geometric consistency. The complete flower outline is inferred by combining the keypoint coordinate set with individual flower masks. Even when only some petals are visible due to occlusion, the geometric constraints on keypoints can reasonably estimate the location of the occluded parts, improving the integrity of shape features in dense scenes.

[0044] The calculation method for the three-dimensional spatial position coordinates is as follows: Using the pixel coordinates of the flower's center point in the RGB three-channel color image, the depth value of the corresponding pixel position in the depth image channel is retrieved. The depth value represents the vertical distance of the pixel position from the camera's imaging plane. The pixel coordinates and depth values ​​are converted into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameter matrix. The conversion formula is: the camera coordinate system X-axis coordinate equals the pixel's horizontal coordinate minus the image center's horizontal coordinate multiplied by the depth value divided by the focal length; the camera coordinate system Y-axis coordinate equals the pixel's vertical coordinate minus the image center's vertical coordinate multiplied by the depth value divided by the focal length; and the camera coordinate system Z-axis coordinate equals the depth value. The camera coordinate system is then converted into three-dimensional coordinates using the camera extrinsic parameter matrix. The 3D coordinates are converted to 3D spatial coordinates in the world coordinate system. The camera extrinsic matrix contains the rotation matrix and translation vector of the camera relative to the world coordinate system. The camera extrinsic matrix is ​​obtained through camera calibration. The 3D spatial coordinates of the center point of each flower are calculated and stored in the flower 3D positioning database. The flower 3D positioning database records the unique identifier, 3D spatial coordinates, flower category and picking priority of each flower. The picking robot path planning module reads the flower 3D positioning database and calculates the optimal picking path according to the workspace constraints and obstacle avoidance requirements of the robotic arm. It prioritizes picking flowers with high maturity and easily reachable 3D spatial coordinates to improve picking efficiency and success rate.

[0045] The initial values ​​of the low-level, mid-level, and high-level weights in the flower semantic recognition model are determined based on the depth variance of the RGBD four-channel image data, the average size of the flower candidate region coordinate set, and the image gradient intensity. The depth variance reflects the 3D structural complexity of the scene; a large depth variance indicates a significant difference in distance between the foreground and background. In this case, increasing the high-level weights is beneficial for utilizing depth image channel information to assist in segmentation. The average size of the flower candidate region coordinate set reflects the spatial scale of the target; small-sized flowers require more texture detail features provided by low-level feature maps, while large-sized flowers require more global context features provided by high-level feature maps. The initial ratio of low-level and high-level weights is adaptively adjusted according to the average size of the flower candidate region coordinate set. The image gradient intensity reflects the edge contrast between the flower and the background; a high image gradient intensity indicates a clear boundary. In this case, the mid-level weights are appropriately increased to utilize the mid-level feature maps to accurately locate the boundary. The initial values ​​of the low-level, mid-level, and high-level weights are calculated by linearly combining the three parameters—depth variance, average size of the flower candidate region coordinate set, and image gradient intensity—after normalization. During training, these initial values ​​are used as a starting point and fine-tuned to the optimal configuration through backpropagation.

[0046] The non-maximum suppression is implemented by performing a sliding window scan on the heatmaps of the flower center point, the petal tip point, and the calyx connection point, with the sliding window size set to... For each pixel, the pixel with the largest response value within the sliding window is retained as a candidate keypoint, while the responses of other pixels within the sliding window are suppressed. After traversing the entire image, a sparse set of keypoint coordinates is obtained.

[0047] The seed region is the set of pixels labeled as flower category and the set of pixels labeled as background category in the initial segmentation mask. Pixel color values ​​are sampled from the seed region to train the Gaussian mixture model. The Gaussian mixture model learns the color distribution features of the foreground and background.

[0048] The multi-scale feature fusion algorithm based on hierarchical attention aggregation refers to an algorithm that captures multi-level feature dependencies by constructing parallel attention branches with different receptive fields, adaptively weights and fuses the outputs of each attention branch using learnable scale weight parameters, and eliminates semantic bias between features of different resolutions through a cross-scale feature alignment mechanism.

[0049] The dilated convolution refers to a convolution operation that inserts holes into a standard convolution kernel to expand the receptive field while keeping the parameter values ​​constant. The dilation rate is defined as the number of pixels inserted between adjacent convolution kernel elements.

[0050] The distance transformation refers to the transformation operation that calculates the shortest distance from each foreground pixel to the nearest background pixel in the binary image. The region with a larger distance value in the distance transformation image corresponds to the center part of the foreground target.

[0051] Optionally, the present invention also provides a computer-based method for forming a flower bud and flower identification and positioning system for a picking robot to perform zoned picking. The computer is equipped with a readable storage medium that stores program instructions. When the program instructions are run on the computer, they execute the above-mentioned method for flower bud and flower identification and positioning for a picking robot to perform zoned picking.

[0052] The specific implementation methods of the above steps are described in detail below.

[0053] The specific implementation of step S01 is to complete image data acquisition through a binocular vision system. This system consists of a color camera and a depth camera, which are synchronously triggered for acquisition. The color camera is responsible for acquiring color images containing three color channels (red, green, and blue) to extract the color features of the flowers. The depth camera measures the distance from each pixel in the scene to the camera's imaging plane using structured light projection or time-of-flight ranging and forms a depth image channel. The depth image channel records the front-to-back hierarchical relationship between the flowers and the background vegetation in three-dimensional space. The camera's intrinsic parameter matrix is ​​used to perform pixel-level registration and alignment operations on the color image and the depth image to eliminate spatial offset caused by the difference in the two cameras' perspectives. The three channels of the registered color image and the depth image channel are merged to form RGBD four-channel image data. This four-channel data not only contains the color and texture information of the flowers but also contains spatial depth information, providing additional constraints for subsequent segmentation. Since flowers usually grow at the front of the plant and are closer to the camera, while the background branches and leaves are located further away, the depth channel can effectively distinguish between foreground targets and background interference, thereby reducing the probability of misjudgment caused by the similarity in color between flowers and leaves.

[0054] The specific implementation of step S02 involves inputting the acquired RGBD four-channel image data into a pre-trained flower semantic recognition model for pixel-by-pixel classification. This model uses a residual network as its backbone network, extracting multi-level feature representations of the input image by stacking residual blocks. Skip connections within the residual blocks allow gradients to propagate effectively in deep networks, avoiding the gradient vanishing problem. The backbone network outputs three feature maps at different depths, corresponding to low-level texture details, mid-level local structures, and high-level global semantics, respectively. These three feature maps are then fed into a feature fusion module for processing. The feature fusion module constructs three... Parallel attention branches perform weighted operations on feature maps at three scales. Each attention branch first expands the receptive field using dilated convolutions with different dilation rates to capture contextual information at different scales. Dilated convolutions increase the receptive field while keeping the number of parameters constant by inserting holes between elements of the standard convolution kernel. Low-level feature maps use dilated convolutions with a dilation rate of 2 to focus on detailed features such as petal edges; mid-level feature maps use dilated convolutions with a dilation rate of 4 to focus on local structures such as flower outlines; and high-level feature maps use dilated convolutions with a dilation rate of 8 to focus on global context such as the overall flower cluster distribution. Each attention branch then... Convolution generates query vectors (key and value vectors) and calculates an attention weight map. Multiplying the attention weight map with the value vector yields a weighted feature representation that highlights the flower region and suppresses background interference. The three weighted feature maps are unified to the same spatial resolution through bilinear interpolation, and then feature alignment is performed to eliminate the spatial offset of features at different levels. Three learnable scale weight parameters are introduced to weight and sum the aligned feature maps to obtain a fused feature map. These three weight parameters are initialized to a uniform distribution and then adaptively learn the contribution of each scale feature to the segmentation task through backpropagation during training. The fused feature map is fed into the decoder module and gradually restored to the original image resolution through multiple deconvolutions and skip connections. The decoder outputs the probability distribution of each pixel belonging to one of five categories: flower, bud, leaf, branch, or background. A pixel-wise maximum value operation is performed on the probability distribution to obtain a preliminary segmentation mask that marks the flower region in the image. At the same time, non-maximum suppression is performed on the high-confidence regions of the flower category to extract the bounding box coordinates and form a set of flower candidate region coordinates for subsequent refinement processing.

[0055] The specific implementation of step S03 involves constructing an undirected graph within the candidate flower region marked by the initial segmentation mask for energy-optimized segmentation. Each pixel within the candidate region is considered a node in the undirected graph, and undirected edges are connected between spatially adjacent pixels. The weight of the undirected edges is determined by the color similarity and spatial distance between adjacent pixels to reflect the correlation strength between pixels. Color similarity is measured by calculating the Euclidean distance in the RGB three-dimensional color space to measure the proximity of pixel colors, while spatial distance is measured by the Manhattan distance of pixel coordinates to measure the geometric proximity of pixels in the image plane. The edge weights are set as a weighted exponential function of color similarity and spatial distance, so that pixels with similar colors and spatial proximity have greater connection strength. Source nodes are added to the undirected graph to represent foreground flowers and sink nodes to represent background vegetation. The initial segmentation mask output from step S02 provides hard constraints, connecting pixels marked as flowers in the mask to source nodes and pixels marked as background to sink nodes. The weights of these connecting edges are set to infinity to ensure that the labels of seed points remain unchanged during the optimization process. An energy function containing data terms and smoothing terms is constructed as the segmentation objective. The data terms are modeled using a Gaussian mixture model. The color distribution of the foreground and background is analyzed, and the log-likelihood probability of each pixel belonging to the foreground or background is calculated. A Gaussian mixture model containing five Gaussian components is used. The mean and covariance matrices of each component are learned from the pixel color values ​​sampled from the seed region using an expectation-maximization algorithm. A smoothing term penalizes cases where spatially adjacent pixels are assigned to different categories to maintain the smoothness of the segmentation boundary. The weight of the smoothing term is inversely proportional to the color gradient between pixels, causing the segmentation boundary to tend to follow the true edge locations with larger image gradients. Based on the maximum flow minimum cut theorem, the energy function minimization problem is transformed into solving the maximum flow problem of an undirected graph. The push-pull algorithm iteratively searches for augmenting paths from the source node to the sink node on the residual network and updates the remaining capacity of each edge. When no augmenting path exists, the algorithm converges. At this point, the minimum cut set of the residual network corresponds to the optimal solution of the energy function. The minimum cut set divides the undirected graph into two disjoint subsets, corresponding to the foreground flowers and the background vegetation, respectively. After updating the pixel labels based on the minimum cut results, the foreground and background pixels are resampled to train the Gaussian mixture model. The graph cut and model update are iteratively performed 5 times to make the color distribution estimate gradually approach the true distribution. Finally, an accurate flower segmentation mask with the boundary tightly attached to the boundary between the petals and the background is obtained.

[0056] The specific implementation of step S04 involves performing morphological operations and watershed transformation on the precise flower segmentation mask to separate adhered individual flowers. First, a circular structuring element with a radius of 3 pixels is used to perform a morphological dilation operation on the precise segmentation mask to expand the foreground region outward. The dilation operation marks the center pixel as foreground when any pixel within the structuring element's coverage area is foreground, by sliding the structuring element across the mask. Then, a morphological erosion operation is performed using the same structuring element to shrink the foreground region inward. The erosion operation requires all pixels within the structuring element's coverage area to be foreground before retaining the center pixel as foreground. The pixel value difference between the dilated image and the eroded image is calculated to obtain a morphological gradient map. This gradient map exhibits a high-intensity response at the flower edges and adhered areas, reflecting the boundary position and topological changes of the foreground region. A distance transformation is performed on the precise segmentation mask to calculate the Euclidean distance from each foreground pixel to the nearest background pixel. Positions with larger distance values ​​in the distance transformation map correspond to the center of the flower, while positions with smaller distance values ​​correspond to edges or adhered contact points. Local maxima detection is performed on the distance transformation map to extract distance peak points as foreground marker points. Each peak... A point represents the center of a potential flower individual and is assigned a unique label value. The background area outside the precise segmentation mask is uniformly marked as the background label. Using the morphological gradient map as the terrain height map and the foreground marker point as the water source, the watershed transformation algorithm is executed. This algorithm simulates the process of water level gradually rising from low to high, with water from different marker points gradually merging during diffusion. When two water streams meet, a watershed line is constructed at the confluence as the segmentation boundary. The watershed line separates the adhered flower individuals into independent connected regions. Since there are usually indentations between the contacting flowers, the morphological gradient map forms a gradient valley at the adhesion point. The watershed transformation can accurately divide the boundary along the gradient valley to achieve individual separation. To suppress the oversegmentation problem, a region merging strategy is adopted. The average color feature value of adjacent connected regions and the average gradient intensity at the boundary are calculated. When the color difference between adjacent regions is less than 15 and the gradient intensity at the boundary is less than 20, it is judged as oversegmentation and the two regions are merged. This threshold is determined by statistically analyzing the feature distribution of normally segmented and oversegmented samples in the training set. Finally, each connected region corresponds to a flower individual mask, realizing reliable individual recognition in dense scenes.

[0057] The specific implementation of step S05 involves performing keypoint detection on each separated flower individual mask to extract the geometric feature points of the flower. The corresponding image patch is cropped based on the bounding box of the individual mask and input into the keypoint detection network. This network employs an encoder-decoder structure, where the encoder progressively reduces the spatial resolution of the feature map by stacking convolutional and pooling layers while increasing the number of feature channels to extract high-level semantic information. The decoder progressively restores the feature map resolution by deconvolutional layers and skip connections, fusing the high-level semantic information with the detailed features of the corresponding layer of the encoder. The network output includes three types of keypoint heatmaps: flower center point, petal tip point, and sepal connection point. The pixel value in each heatmap represents the confidence level of the presence of a corresponding keypoint at that location. During training, a target heatmap is generated based on the manually labeled real keypoint coordinates. The target heatmap sets a peak at the real coordinate location and decays outwards using a Gaussian distribution with a standard deviation of 2 pixels. The mean squared error loss function is used to measure the difference between the predicted heatmap and the target heatmap, and the network parameters are optimized through backpropagation. Non-maximum suppression is performed on the three predicted heatmaps to extract local peak points. A sliding window scanning heatmap of pixels retains the pixel with the largest response within the window and suppresses other responses. A confidence threshold of 0.5 is set to filter low-confidence responses, and the coordinates of the retained peak points are the detected keypoint locations. Geometric constraints on keypoints are established based on prior knowledge of the flower's anatomical structure. The distance from the flower's center point to the tips of each petal should be within a certain range, and the angles between the petal tips should be approximately uniformly distributed. These constraints are used to eliminate false detections that do not conform to the anatomical structure, retaining the set of keypoints with optimal geometric consistency. By combining keypoints with individual masks, the complete flower outline can be inferred, and the location of the occluded part can be estimated even in the case of occlusion.

[0058] The specific implementation of step S06 involves calculating the flower's position coordinates in three-dimensional space using keypoint coordinates and depth information. First, the pixel coordinates of the center point of each flower in the color image are extracted as the pixel x-coordinate and pixel y-coordinate. The depth value at the corresponding position in the depth image channel is then queried; this value represents the vertical distance from the flower's center point to the camera's imaging plane. Using the focal length value in the camera's intrinsic parameter matrix and the image center coordinates, the pixel coordinates and depth values ​​are converted into three-dimensional coordinates in the camera coordinate system. The conversion formula is: the camera coordinate system X-axis coordinate equals the difference between the pixel x-coordinate and the image center x-coordinate multiplied by the depth value and then divided by the focal length; the camera coordinate system Y-axis coordinate equals the difference between the pixel y-coordinate and the image center y-coordinate multiplied by the depth value and then divided by the focal length. The Z-axis coordinate of the camera coordinate system is directly equal to the depth value. The three-dimensional coordinates in the camera coordinate system are converted into three-dimensional spatial position coordinates in the world coordinate system through the pre-calibrated camera extrinsic parameter matrix. The extrinsic parameter matrix contains the rotation matrix and translation vector of the camera relative to the world coordinate system, reflecting the camera's installation pose in the environment. After calculating the three-dimensional spatial position coordinates of the center point of each identified flower, the flower's unique identifier, flower category, and picking priority are stored in the flower three-dimensional positioning database. The path planning module of the picking robot reads the database and calculates the optimal picking path according to the workspace constraints and obstacle avoidance requirements of the robotic arm. It prioritizes picking flowers with high maturity and spatial location that are easy for the robotic arm to reach, thereby improving work efficiency and success rate.

[0059] Specifically, the principle of this invention is as follows: The invention solves this technical problem by transforming single-scale feature extraction into multi-scale hierarchical feature aggregation. Parallel processing pathways with different receptive fields are constructed through dilated convolution, enabling the semantic recognition model to simultaneously acquire both local detail information and global structural information of the target. When faced with overlapping color spaces of buds and leaves, the high-resolution texture information preserved in the low-level feature maps can capture the difference between the fine waxy texture on the bud surface and the vein texture on the leaf surface. This texture difference becomes a key criterion when color features fail. The local shape features provided by the mid-level feature maps can identify the compact circular outline of the bud and the flat elliptical outline of the leaf; shape prior knowledge compensates for the deficiencies of color features. The global context information contained in the high-level feature maps utilizes the spatial distribution differences between buds and leaves to assist in discrimination, as buds are usually located at the top of branches while leaves are distributed on the sides. The attention mechanism enables the model to dynamically enhance effective features and suppress redundant features. It adaptively adjusts the weight allocation of features at different scales for samples of varying difficulty. The learnable scale weight parameters automatically discover the optimal multi-scale fusion strategy through backpropagation, thus achieving accurate segmentation even when color cues are insufficient, through the comprehensive discrimination of texture, shape, and spatial distribution.

[0060] The following provides a specific embodiment 1 of the present invention. The specific implementation of steps S01-S02 in this embodiment 1 is the same as that described above, and will not be repeated in detail here. The specific implementation of other steps is described in detail below.

[0061] The specific implementation of step S03 involves performing a graph cut energy minimization segmentation algorithm on the flower candidate regions in the initial segmentation mask. The formula for calculating the undirected edge weights is as follows:

[0062] ;

[0063] In the formula, For pixels With pixels The weights of the undirected edges between them are dimensionless. The index of the first pixel; The index of the second pixel; and Pixels With pixels RGB color vector, in units of normalized color values, with each channel ranging from 0 to 1; The Euclidean distance between the color vectors is dimensionless. For pixels With pixels The Manhattan distance, in pixels; This is the color similarity weighting factor, dimensionless, with an empirical value of 0.8; This is the spatial distance weighting factor, dimensionless, with an empirical value of 0.2; This is the standard deviation parameter for color similarity, which is dimensionless and typically takes a value of 0.3. This is the standard deviation parameter for spatial distance, expressed in pixels, and typically has a value of 5.

[0064] The energy function includes a data term and a smoothing term, and is expressed as follows:

[0065] ;

[0066] In the formula, The total energy function is dimensionless. Energy is a dimensionless quantity. The energy of the smooth term is dimensionless; This is the balance coefficient, dimensionless, with an empirical value of 50.

[0067] The formula for calculating the energy of a data item is expressed as follows:

[0068] ;

[0069] In the formula, This represents the total number of pixels in the image. For pixels color Belongs to the label The probability of is dimensionless and is calculated using a Gaussian mixture model. For pixels The category label can be either foreground or background.

[0070] The probability calculation formula for a Gaussian mixture model is expressed as follows:

[0071] ;

[0072] In the formula, This represents the number of Gaussian components, with a value of 5. Index of Gaussian components; For the first The weighting coefficients of the Gaussian components are dimensionless and satisfy the following conditions: ; For the first The mean vector of the Gaussian components is: Column vector, in units of normalized color values; For the first The covariance matrix of the Gaussian components is A matrix is ​​dimensionless. Let be the determinant of the covariance matrix, which is dimensionless; The transpose of the color difference vector is... Row vectors; Let be the inverse of the covariance matrix, and be... A matrix is ​​dimensionless.

[0073] The formula for calculating the energy of the smoothing term is expressed as follows:

[0074] ;

[0075] In the formula, It is a set of adjacent pixel pairs; A pair of adjacent pixels; For pixels With pixels The image gradient between the two points, expressed in units of normalized gradient strength, with values ​​ranging from 0 to 1; This is the maximum gradient value in the image, expressed as normalized gradient intensity, with a value of 1. For indicator functions, dimensionless, when The value is 1 if it is true, and 0 otherwise.

[0076] The specific implementation of step S04 involves performing a morphological gradient and watershed transformation algorithm on the accurate flower segmentation mask. The formula for calculating the morphological gradient map is as follows:

[0077] ;

[0078] In the formula, This is a morphological gradient plot, dimensionless, with values ​​ranging from 0 to 1; A dimensionless mask for precise flower segmentation, with a value of 0 or 1; The dilation result of the accurate flower segmentation mask is dimensionless and takes the value of 0 or 1; The erosion result for accurate flower segmentation mask is dimensionless and takes values ​​of 0 or 1. Distance transform calculates the distance from each foreground pixel to its nearest background pixel, and foreground markers are extracted using local maxima detection. The watershed transform algorithm simulates a rising water level, constructing watershed lines at gradient depressions to separate adhered flower individuals.

[0079] The formula for determining color differences in the region merging strategy is expressed as follows:

[0080] ;

[0081] In the formula, Adjacent connected regions and Normalized color differences, dimensionless; and Connected regions and The average color vector is Column vector, in units of normalized color values; The Euclidean distance between the average color vectors is dimensionless. The maximum Euclidean distance in the normalized space of the RGB three channels is dimensionless and takes the value of ; The color difference threshold is dimensionless and has an empirical value of 0.15.

[0082] The formula for determining gradient strength is as follows:

[0083] ;

[0084] In the formula, The normalized average gradient intensity of the boundary of adjacent connected regions is dimensionless. This refers to the number of boundary pixels. Indexed by boundary pixels; For boundary pixels The morphological gradient value is dimensionless and ranges from 0 to 1. The average gradient intensity across the entire image is dimensionless. This is the gradient strength threshold, which is dimensionless and has an empirical value of 0.2.

[0085] The specific implementation of step S05 involves predicting key point heatmaps from the separated individual flower masks. The key point detection network uses a Gaussian distribution to generate the target heatmap during training. The Gaussian distribution function is described below:

[0086] ;

[0087] In the formula, For the target heatmap in coordinates The value at this point is dimensionless and ranges from 0 to 1. and These are the pixel coordinates in the heatmap, in pixels. and These are the pixel coordinates of the actual key points, in pixels. This represents the standard deviation of the Gaussian distribution, expressed in pixels, and is typically set to 2.

[0088] The formula for achieving nonmaximum suppression is expressed as follows:

[0089] ;

[0090] In the formula, For the first Candidate coordinates of key points within a sliding window, in pixels; For sliding window index; For the first A sliding window area, the window size is... Pixel; To predict heatmaps in coordinates The value at this point is dimensionless and ranges from 0 to 1. This indicates the coordinates corresponding to the maximum value.

[0091] The formula for the distance constraint from the center point of the flower to the tip of the petal in the key point geometric constraints is expressed as follows:

[0092] ;

[0093] In the formula, For the first Normalized radial distance of each petal, dimensionless; For petal index; For the first The coordinates of the petal tip are given as a two-dimensional vector containing horizontal and vertical coordinates, in pixels. The coordinates of the flower's center point are represented by a two-dimensional vector containing horizontal and vertical coordinates, in pixels. This is the Euclidean distance from the tip of the petal to the center point, in pixels. The average radius of flowers of the same type, in pixels, is obtained through a statistical training set.

[0094] The formula for constraining the uniformity of the angle between the tips of the petals is as follows:

[0095] ;

[0096] In the formula, For the first The normalized angle of the tip of each petal relative to the center of the flower is dimensionless. and For the first The horizontal and vertical coordinates of the tip of each petal, in pixels; and The x and y coordinates of the center point of the flower, in pixels; This refers to the number of petals. It is a two-parameter arctangent function, and its output range is... arrive radian.

[0097] The specific implementation of step S06 involves calculating the three-dimensional spatial coordinates of each flower based on the keypoint coordinate set and the individual flower mask. The conversion formula from pixel coordinates to camera coordinates is expressed as follows:

[0098] ;

[0099] ;

[0100] ;

[0101] In the formula, , and These are the three-dimensional coordinates in the camera coordinate system, in millimeters. and These are the pixel coordinates of the flower's center point in the image, in pixels. and These are the pixel coordinates of the principal point of the image, in pixels. and This represents the horizontal and vertical components of the camera's focal length, expressed in pixels. For depth image channels in coordinates The depth value at that location, in millimeters.

[0102] The transformation formula from the camera coordinate system to the world coordinate system is expressed as follows:

[0103] ;

[0104] In the formula, , and These are three-dimensional spatial coordinates in the world coordinate system, in millimeters. Let be the rotation matrix from the camera to the world coordinate system, and let be... Orthogonal matrix, dimensionless; Let be the translation vector from the camera to the world coordinate system, and be... Column vector, in millimeters.

[0105] The formula for calculating the attention weight map in the multi-scale feature fusion algorithm based on hierarchical attention aggregation is as follows:

[0106] ;

[0107] In the formula, This is an attention weight graph for a certain layer, which is dimensionless and each element takes a value ranging from 0 to 1. This is the query vector matrix for this layer, through... Convolution is generated from receptive field features; The key vector matrix of this layer is obtained by... Convolution is generated from receptive field features; This is the transpose of the key vector matrix; The dimension of the feature vector is used for normalization to prevent gradient vanishing; For normalized exponential functions, ,in For the input vector of the th One element, For the summation index, This represents the total number of elements in the vector.

[0108] The formula for calculating the weighted feature map is as follows:

[0109] ;

[0110] In the formula, This is a weighted feature map of a certain layer; For the value vector matrix of this layer, through Convolution is generated from receptive field features.

[0111] The formula for calculating the initial value of the scale weight parameter is as follows:

[0112] ;

[0113] ;

[0114] ;

[0115] In the formula, , and These are the initial values ​​for the low-level weights, mid-level weights, and high-level weights, respectively, and are dimensionless. This is a normalized value of the average size of the candidate flower region. It is dimensionless and is calculated by dividing the average size by the maximum side length of the image. This is a normalized value of the image gradient intensity, dimensionless, and is calculated by dividing the average gradient intensity by the maximum gradient value. This is the normalized value of the depth variance, which is dimensionless and is calculated by dividing the depth variance by the maximum depth difference. The linear combination coefficients related to size are dimensionless and have an empirical value of 0.4. The coefficients for the gradient-dependent linear combination are dimensionless and have an empirical value of 0.3. The coefficients for the deeply correlated linear combination are dimensionless and have an empirical value of 0.3.

[0116] The weighted summation formula for the fused feature maps is expressed as follows:

[0117] ;

[0118] In the formula, To fuse feature maps; , and These are the aligned low-level feature maps, mid-level feature maps, and high-level feature maps, respectively. , and The scale weights are learnable, dimensionless parameters that satisfy normalization constraints. .

[0119] It needs to be explained that the formula for calculating the weight of an undirected edge is... By comprehensively considering both color similarity and spatial distance, a negative exponential function is used to map similarity into weight values. The smaller the color difference or the closer the spatial distance, the greater the weight. This formula can accurately model the affinity relationship between pixels and provide precise energy term constraints for graph cut algorithms.

[0120] Energy function The balance between the middle data term and the smoothing term is achieved through parameters. The adjustment and data terms ensure that the segmentation results conform to the color distribution prior, while the smoothing term promotes the segmentation boundary to follow the true edge with a larger gradient intensity. The synergistic effect of the two significantly improves the boundary accuracy of flower segmentation.

[0121] Probability calculation formula for Gaussian mixture model By capturing the complex color distribution of the foreground and background using multiple Gaussian components, each Gaussian component describes a cluster center in the color space. The weighted combination of these multiple components accurately models the color variations of the flower and background under different lighting conditions, providing a reliable probability estimate of the energy of the data items. The Mahalanobis distance in this formula... Measuring the distance of color vectors in the covariance normalization space makes the model adaptable to the variance of color distribution in different directions.

[0122] Energy formula for smooth term gradient weighting term in The graph cut algorithm applies a smaller penalty to areas with high gradient strength, encouraging segmentation boundaries to follow the true edges of the image, while applying a larger penalty to areas with gentle gradients, suppressing unreasonable segmentation. This mechanism enables the graph cut algorithm to automatically find the optimal segmentation path that conforms to the visual edges.

[0123] Color difference determination formula for region merging strategy Gradient strength determination formula Normalization is used to eliminate the scale effect, and the color difference threshold and gradient intensity threshold are determined based on statistical distribution. This strategy effectively suppresses the oversegmentation problem of watershed transformation. When adjacent regions have similar colors and weak boundary gradients, they are judged as oversegmented and merged, thus improving the robustness of individual recognition.

[0124] Key point geometric constraint formula and By leveraging prior knowledge of flower anatomy, false key points are eliminated through radial distance and angular uniformity constraints. The radial distance constraint ensures that the distance from the tip of the petal to the center is within a reasonable range, while the angular uniformity constraint ensures that the petals are evenly distributed in the circumferential direction. Even in cases of partial occlusion, the complete flower outline can be reasonably inferred, significantly improving the accuracy of key point detection in dense scenes.

[0125] 3D spatial coordinate transformation formula and Based on the pinhole camera imaging model, the two-dimensional pixel coordinates and depth values ​​are mapped to three-dimensional coordinates in the world coordinate system by combining the camera's intrinsic and extrinsic parameters. The pixel coordinates and principal point deviations are normalized by the focal length and multiplied by the depth value to obtain the camera coordinates. Then, the coordinates are transformed to the world coordinate system through rotation matrix and translation vector, providing accurate spatial positioning information for the path planning of the harvesting robot.

[0126] Attention weight map calculation formula The correlation between features is calculated by the dot product of the query vector and the key vector, and then divided by... To prevent the gradient from vanishing due to excessively large dot product values, the attention weight distribution is obtained after normalizing the exponential function. This mechanism enables the model to dynamically enhance the feature representation of the flower region while suppressing interference from irrelevant backgrounds. For difficult samples such as green buds and green leaves with severe color space overlap, the attention weight map can learn the subtle differences in texture details and local structure between buds and leaves. The query vector captures the feature patterns that need attention at the current position, the key vector provides feature information for candidate positions, the dot product of the two measures the feature matching degree, and the value vector carries the actual feature content and is aggregated through attention weights.

[0127] Formula for calculating the initial value of the scale weight parameter The contribution of different feature levels is adaptively adjusted based on the 3D structural complexity of the scene, the target spatial scale, and edge contrast. When the depth variance is large, the weights of higher layers are increased to utilize depth information for segmentation; for small-sized flowers, the weights of lower layers are increased to preserve texture details; and for high gradient strength, the weights of middle layers are increased to accurately locate boundaries. This formula enables the feature fusion strategy to be dynamically optimized for different scene characteristics, significantly improving the semantic recognition model's adaptability to complex scenes. During training, the initial values ​​are used as a starting point and fine-tuned to the optimal configuration through backpropagation. The model automatically discovers the importance of lower-level feature maps for petal edge details and the importance of higher-level feature maps for overall flower region localization. The three linear combination coefficients... , and The influence of size, gradient, and depth factors on the initial weight allocation is controlled separately.

[0128] To better understand and implement this invention, the following is a specific application scenario example 2: A planting base grows four varieties of roses: red, pink, white, and yellow. The flowers grow densely and severely overlap, making it difficult for traditional visual recognition methods to accurately distinguish individual flowers. The technical team built a system... Figures 2-3 The robot shown includes: 1. a dual-arm linear module, 2. a tracked self-propelled chassis, 3. a high-brightness display screen, 4. a depth camera, 5. a picking end, and 6. a sunshade mounting frame; and it forms a rose maturity detection model system by running the method described in this invention through a small computer. The small computer is connected to the depth camera, the picking end, and the high-brightness display screen to transmit data.

[0129] Specifically, the dual-arm linear module 1 is an integrated structure, which is installed on the upper surface of the tracked self-propelled chassis 2. As a component of this integrated structure, the two arms each have independent and controllable three-axis motion freedom: x-axis corresponds to the forward and backward direction, y-axis corresponds to the left and right direction, and z-axis corresponds to the up and down direction. The three-axis drive method uniformly adopts servo motor and ball screw transmission, and the positioning accuracy reaches ±0.5mm. The x-axis extends along the length of the chassis, with a total travel of 90cm. Based on this travel, functional zones are defined: the first 45cm area is the dedicated work area for one arm, and the last 45cm area is the dedicated work area for two arms. The two arms together cover the front and back distribution range of a single row of rose plants. The coordinated operation of the two arms is achieved through an improved Traveling Salesman Algorithm (TSP) integrated into the control unit: after the images acquired by the depth camera 4 are identified by the rose maturity detection model, they are automatically assigned to the corresponding work areas according to the x-axis zoning rules. The TSP algorithm optimizes the picking order for the target points of each arm and monitors the x and y axis coordinates of the two arms in real time. When the distance is less than the 5cm safety threshold, priority scheduling is triggered to ensure that the two arms do not interfere with each other during independent movement, efficiently covering the three-dimensional work space and achieving precise picking of roses.

[0130] Specifically, the tracked self-propelled chassis 2 serves as the foundation for the overall equipment's load-bearing and movement. It employs a chain-track structure with evenly distributed anti-slip protrusions on the chain links, adapting to common terrains such as field furrows and soft soil. The double-arm linear module 1, depth camera 4, and sunshade mounting bracket 6 are bolted to the top. The bottom integrates a drive motor with a reducer, allowing for linear movement along pre-set rose planting rows. The movement speed can be adjusted via a control unit, ensuring the equipment stops smoothly in the work area and providing a stable support platform for harvesting operations.

[0131] Specifically, the high-brightness display screen 3 is installed on the operating side of the sunshade mounting frame 6. The screen resolution is 1366*768, and the brightness is adjustable from 450-1000 cd / m², which can adapt to the strong light environment in the field. The display screen shows the equipment operating status and target recognition results in real time, and also supports manual input of parameters, which facilitates operators to monitor and intervene in the operation process in real time.

[0132] Specifically, the depth camera 4 is fixed to the front end of the tracked self-propelled chassis 2 via a bracket. The field of view of the depth camera 4 can be adjusted by adjusting its position, with the lens facing the rose bush in the direction of the device's movement. It can collect color images and depth data within its field of view in real time, providing raw data for target recognition. Figure 4 As shown.

[0133] Specifically, the picking end 5 is installed at the end of the double-arm linear module 1 and adopts a three-degree-of-freedom structure, including rotational degree of freedom, pitch degree of freedom and opening and closing degree of freedom. All three degrees of freedom are driven by servo motors and operate in coordination to realize the action of grasping, separating and collecting roses.

[0134] Specifically, the sunshade mounting bracket 6 is fixed to the operating side of the upper surface of the tracked self-propelled chassis 2. It adopts a frame structure with a reserved solar panel mounting position at the top and a high-brightness screen fixing support on the side. It can not only shield the high-brightness screen 3 from strong field light to ensure display clarity, but also stably support components such as solar panels. At the same time, it does not occupy the working space of the double-arm linear module 1, ensuring the overall operation coordination of the equipment.

[0135] Specifically, the rose maturity detection model is deployed in the electrical control cabinet of the tracked self-propelled chassis 2. Based on the data collected by the depth camera 4, the acquired RGB images are identified by the rose maturity detection model, maturity levels are distinguished and detection boxes are provided, and the detection boxes are mapped to the depth images to extract the target 3D point cloud. The target position is estimated by the PCA dimensionality reduction algorithm, providing accurate positioning basis for the dual-arm straight module 1 and the picking end 5.

[0136] The technical team collected training datasets under various time periods and lighting conditions, including three typical scenarios: low-angle sidelight from 7:00 AM to 9:00 AM, direct overhead light from 12:00 PM to 2:00 PM, and diffused light from 5:00 PM to 7:00 PM. A total of 12,500 sets of RGBD four-channel image data samples were collected. Image annotation was performed using an image annotation tool. Pixel-level semantic annotation of each RGB three-channel color image took approximately 8 minutes, accurately labeling five categories of targets—flowers, buds, leaves, branches, and background—pixel by pixel. To enhance model robustness, data augmentation operations were performed on the original samples, including random flipping in both the horizontal and vertical directions with a probability of 0.5, randomly selecting crop sizes within the range of 80% to 100% of the original image, adding Gaussian noise with a standard deviation of 0.1 to each of the RGB three channels for color jitter, and randomly scaling the images between 0.8 and 1.2 times. Ultimately, the training dataset was expanded to 32,000 sets. The dataset was divided into a training set of 25,600 groups, a validation set of 3,200 groups, and a test set of 3,200 groups in a ratio of 8:1:1. The training set was used for updating the parameters of the flower semantic recognition model, the validation set was used for hyperparameter tuning and early stopping strategy judgment, and the test set was used for final performance evaluation.

[0137] The flower semantic recognition model adopts a three-stage architecture of Backbone-Neck-Head. The Backbone part uses a ResNet-50 residual network as the feature extractor, outputting feature maps at three different scales. The spatial resolution of the low-level feature maps is [missing information - likely a percentage of the input image's resolution]. The number of channels is 256, and the spatial resolution of the middle layer feature map is [missing value] times that of the input image. The number of channels is 512, and the spatial resolution of the high-level feature map is [missing value] times that of the input image. The number of channels is 1024. The Neck section implements a multi-scale feature fusion algorithm based on hierarchical attention aggregation, constructing three parallel attention branches to process low-level, mid-level, and high-level feature maps respectively. The low-level attention branch uses dilated convolutions with a dilation rate of 2 to expand the receptive field to... For pixels, the mid-layer attention branch uses dilated convolutions with a dilation rate of 4 to expand the receptive field. For pixels, the high-level attention branch uses dilated convolutions with a dilation rate of 8 to expand the receptive field. Pixels. Each attention branch internally passes through... Convolution generates query vectors, key vectors, and value vectors. The dimensions of the query vector and key vector are set to 128. After dot product calculation, softmax normalization is applied to obtain an attention weight map. The attention weight map is multiplied by the value vector to obtain a weighted feature map. The three weighted feature maps are unified to a spatial resolution of 512×512 pixels for the intermediate feature map through bilinear interpolation. Three learnable scale weight parameters are introduced and initialized to a uniform distribution of 0.333. A weighted summation operation is then performed to obtain a fused feature map. The Head part adopts a decoder structure containing four deconvolutional layers. Each deconvolutional layer doubles the resolution of the feature map, ultimately restoring the fused feature map to the input image resolution of 1920×1080 pixels. The output is a 5-channel probability distribution map corresponding to five categories: flower, bud, leaf, branch, and background.

[0138] The training process employs the cross-entropy loss function, setting a loss weight of 2.5 for flower and bud categories, 1.0 for leaf and branch categories, and 0.5 for the background category to address class imbalance caused by a high proportion of background pixels in the samples. A stochastic gradient descent optimizer is used to update model parameters, with a momentum coefficient of 0.9, a weight decay coefficient of 0.0005, and an initial learning rate of 0.01. A cosine annealing strategy is used to gradually reduce the learning rate during training, with a minimum decay to 0.0001. The training process is divided into a pre-training phase and a fine-tuning phase. During the pre-training phase, the parameters of the first three residual blocks of ResNet-50 are frozen, and only the Neck and Head parts are trained. After 20 epochs of training, the average intersection-union ratio (IUU) on the validation set reaches 0.762. During the fine-tuning phase, all parameters were unfrozen for end-to-end training. After 80 more epochs, the average crossover ratio (CWR) improved to 0.847, with the CWR for the flower category reaching 0.891 and the CWR for the bud category reaching 0.835. The training process used a batch size of 16 and was performed in a distributed manner on four NVIDIA Tesla V100 GPUs. The training time per epoch was approximately 42 minutes, and the total training time was approximately 70 hours.

[0139] After training, the technical team deployed the flower semantic recognition model to the edge computing unit of the harvesting robot, enabling it to process a 50-acre plot of land in the planting base. Real-time recognition testing is performed on the test area. For example... Figure 1 As shown, after processing the RGBD four-channel image data by the flower semantic recognition model, a preliminary segmentation mask is output. In the preliminary segmentation mask, the flower region is displayed in white, and the background region is displayed in black. The model successfully identified 87 blooming roses and 23 unopened buds. The coordinate set of the flower candidate region extracted through non-maximum suppression contains 110 bounding boxes. The coordinate format of each bounding box is the x-coordinate of the top left corner, the y-coordinate of the top left corner, the width, and the height. Figure 2 As shown, a graph cut energy minimization segmentation algorithm is performed on each flower candidate region in the initial segmentation mask. Pixels within the region are constructed as undirected graph nodes, with undirected edges connecting spatially adjacent pixels. The weight of each undirected edge is determined by both color similarity and spatial distance. Color similarity is calculated using the Euclidean distance metric in the RGB color space, with a weighting factor of 0.6. Spatial distance is measured using the Manhattan distance metric of pixel coordinates, with a weighting factor of 0.4. A source node representing the foreground flower and a sink node representing the background vegetation are added to the undirected graph. Seed point constraints are provided using the initial segmentation mask, containing 825 foreground pixels and 1134 background pixels.

[0140] The energy function's data term trains a Gaussian mixture model (GMM) to learn the color distribution of the foreground and background by sampling pixel color values ​​from the seed region. The GMM contains five Gaussian components, and the mean and covariance matrix of each component are estimated using an expectation-maximization (EM) algorithm, which converges after 15 iterations. The smoothing term weights are set inversely proportional to the color gradient between pixels. For boundary locations where the color gradient is greater than a threshold of 15, the smoothing term weight is reduced to 0.2, while for regions where the color gradient is less than the threshold of 15, the smoothing term weight remains at 1.0. A push-pull algorithm is used to solve the maximum flow problem, converging after an average of 32 iterations to obtain the accurate flower segmentation mask corresponding to the minimum cut set. The graph cut and GMM update process is performed iteratively five times. The boundary accuracy of the accurate flower segmentation mask is improved by an average of 0.18 pixels compared to the initial segmentation mask, and the boundary contour closely matches the true boundary between the petals and the background.

[0141] Morphological dilation and morphological erosion operations are performed on the precise flower segmentation mask. Both operations use circular structuring elements with a radius of 3 pixels. The morphological gradient map is calculated from the pixel value difference between the dilated and eroded images. The gradient values ​​in the morphological gradient map reach 230 to 255 at the flower edges, are below 30 in the flat areas inside the flower, and drop to around 120 in the gradient depressions where adjacent flowers are joined. A distance transform is performed on the precise flower segmentation mask. The distance value of each foreground pixel in the distance transform map represents the shortest distance to the nearest background pixel. Local maxima detection is performed on the distance transform map to extract distance peak points. The detection window size is set to... A total of 108 foreground markers were extracted from the pixels. The morphological gradient map was treated as a 3D terrain surface, and the foreground markers as water sources. A watershed transform algorithm was executed to simulate the rising water level, with water from different sources converging at gradient depressions to form watershed lines. The watershed lines generated by the watershed transform algorithm separated the individual flowers that were in contact with each other, resulting in 108 independent connected regions. Each connected region corresponds to a flower individual mask.

[0142] To address the oversegmentation issue, the technical team statistically analyzed the feature distributions of normally segmented and oversegmented regions in the training set, setting the color difference threshold to 18 and the gradient strength threshold to 135. The average RGB color value of adjacent connected regions was calculated; when the color difference was less than 18 and the boundary gradient strength was less than 135, it was considered oversegmented, and adjacent connected regions were merged. After this region merging strategy, the number of oversegmented connected regions decreased from 108 to 103, resulting in 7 missed detections compared to the actual number of 110 flowers. These missed detections mainly occurred in cases where severe occlusion led to excessively small visible areas. Figure 3As shown, the image patch corresponding to each individual flower is cropped using a mask and input into the keypoint detection network. The keypoint detection network adopts an encoder-decoder structure. The encoder part contains 5 convolutional layers and pooling layers, and the decoder part contains 5 deconvolutional layers and skip connections. The output heatmaps in 3 channels correspond to the heatmaps of the flower center point, the petal tip point, and the sepal connection point, respectively. When training the keypoint detection network, Gaussian distribution target heatmaps are generated using real keypoint coordinates. The standard deviation of the Gaussian distribution is set to 2 pixels. The loss function uses mean squared error to measure the difference between the predicted heatmap and the target heatmap. After training for 50 epochs, the keypoint detection accuracy on the validation set reaches 0.913.

[0143] Non-maximum suppression was applied to the heatmap of the flower center point, and the sliding window size was set to... The pixel with the largest response value within the sliding window was selected as a candidate keypoint, with a confidence threshold of 0.75. For 103 individual flower masks, 103 flower center points were successfully detected. An average of 5.2 petal tip points were detected per flower, and an average of 1 sepal connection point was detected per flower. Based on prior knowledge of rose anatomy, the distance from the flower center point to each petal tip point should follow a normal distribution with a mean of 32 pixels and a standard deviation of 6 pixels, and the angle between petal tips should be between 60° and 80°. False keypoints that do not conform to the anatomical structure were eliminated using keypoint geometric constraints. The set of keypoint coordinates with optimal geometric consistency was retained for each flower. Ultimately, the effective keypoint count for each flower was 1 flower center point, 5 petal tip points, and 1 sepal connection point.

[0144] Using the pixel coordinates of the flower center point in the RGB three-channel color image, the depth value of the corresponding pixel position in the depth image channel was retrieved. The depth values ​​of the 103 flower center points are distributed in the range of 0.65m to 0.92m, with an average depth value of 0.78m. The camera intrinsic parameter matrix was obtained using the Zhang Zhengyou calibration method, with the focal length parameter set to 1850 pixels and the image center coordinates at 960, 540. The pixel coordinates and depth values ​​were converted into three-dimensional coordinates in the camera coordinate system. The calculation range of the camera coordinate system X-axis coordinate is -0.28m to 0.31m, the calculation range of the camera coordinate system Y-axis coordinate is -0.19m to 0.24m, and the camera coordinate system Z-axis coordinate is equal to the depth value. The camera extrinsic parameter matrix includes a rotation matrix and a translation vector. The Euler angles of the rotation matrix are 45° around the X-axis, 0° around the Y-axis, and 0° around the Z-axis. The translation vectors are 0.15m, 0.08m, and 1.20m. The 3D coordinates in the camera coordinate system are converted into 3D spatial position coordinates in the world coordinate system using the camera extrinsic parameter matrix. The 3D spatial position coordinates of the 103 flowers are stored in the flower 3D positioning database, as shown in Table 1.

[0145] Table 1. Example of a 3D flower localization database

[0146]

[0147] The path planning module of the picking robot reads the 3D positioning database of flowers and calculates the optimal picking path based on the workspace constraints and obstacle avoidance requirements of the robotic arm. The maximum extension distance of the robotic arm is 1.5m, and the gripping range of the end effector is 10mm to 80mm. The path planning algorithm uses a fast expanding random tree algorithm to search for collision-free paths, with an upper limit of 5000 search iterations. Path smoothness is optimized through Bézier curve fitting. Picking priority is comprehensively evaluated based on the maturity of the flower and the accessibility of its 3D spatial coordinates. Maturity is judged by the flower's color saturation and petal unfolding, while accessibility is evaluated by the feasibility of solving the inverse kinematics of the robotic arm and the path length. The system prioritizes picking flowers with high maturity and easily accessible 3D spatial coordinates. After sorting the 103 identified flowers according to picking priority, the picking action is performed sequentially. The average picking time per flower is 8.5 seconds, and the picking operation in the entire test area takes about 15 minutes. The picking success rate reaches 0.932, and the damage rate is reduced to 0.047. The detection result image and corresponding depth map are shown below. Figure 5 As shown.

Claims

1. A method for identifying and locating flower buds and blossoms for a harvesting robot to perform zoned harvesting, characterized in that, Includes the following steps: Collect RGBD four-channel image data of the target flower planting area. The RGBD four-channel image data includes color image channels and depth image channels. RGBD four-channel image data is input into the flower semantic recognition model for preliminary classification of foreground flowers and background vegetation. The model outputs a preliminary segmentation mask and a set of coordinates of candidate flower regions. The flower semantic recognition model adopts a multi-scale feature fusion algorithm based on hierarchical attention aggregation. The graph cut energy minimization segmentation algorithm is executed on the candidate flower regions in the preliminary segmentation mask to construct an undirected graph and calculate the minimum cut set to obtain the accurate flower segmentation mask. The morphological gradient and watershed transformation algorithms are applied to the precise flower segmentation mask to calculate the morphological gradient map and mark the foreground and background regions. Adhesive flower individuals are separated by watershed transformation. Key point heatmap prediction is performed on the individual flower masks after separation to extract the coordinates of the flower center point and the petal tip. Valid key points are screened by nonmaximum suppression. The three-dimensional spatial coordinates of each flower are calculated based on the key point coordinate set and the individual flower mask. A three-dimensional positioning database of flowers is established by combining the depth image channel information for path planning of the picking robot.

2. The method according to claim 1, characterized in that, The acquisition of the RGBD four-channel image data is carried out synchronously using a binocular vision system composed of a color camera and a depth camera. The RGB three-channel color image and the depth image channel are then registered and aligned at the pixel level to form the RGBD four-channel image data.

3. The method according to claim 2, characterized in that, The flower semantic recognition model adopts a three-segment design of Backbone-Neck-Head. The Backbone part uses a residual network to extract multi-level image features and outputs low-level feature maps, mid-level feature maps and high-level feature maps.

4. The method according to claim 3, characterized in that, The multi-scale feature fusion algorithm based on hierarchical attention aggregation constructs three parallel attention branches to process low-level feature maps, mid-level feature maps and high-level feature maps respectively, and expands the receptive field and generates attention weight maps through dilated convolution.

5. The method according to claim 4, characterized in that, The attention weight map is multiplied element-wise with the corresponding feature map to obtain a weighted feature map. After performing bilinear interpolation on the three weighted feature maps to unify them to the same spatial resolution, feature alignment is performed.

6. The method according to claim 5, characterized in that, The aligned feature maps are weighted and fused using learnable scale weight parameters, which include low-level, mid-level, and high-level weights. The scale weight parameters are initialized to a uniform distribution and then adaptively learned through backpropagation.

7. The method according to claim 6, characterized in that, The initial values ​​of the low-level weights, middle-level weights, and high-level weights are determined based on the depth variance of the RGBD four-channel image data, the average size corresponding to the coordinate set of the flower candidate region, and the image gradient intensity.

8. The method according to claim 3, characterized in that, The training dataset for the flower semantic recognition model is established by: collecting RGBD four-channel image data samples of flowers of different varieties, different growth stages, different lighting conditions and different degrees of occlusion, and performing pixel-level semantic annotation using image annotation tools.

9. The method according to claim 1, characterized in that, The graph cut energy minimization segmentation algorithm constructs each pixel in the image region corresponding to the precise flower segmentation mask as a node of an undirected graph, connecting spatially adjacent pixels with undirected edges.

10. The method according to claim 9, characterized in that, The weight of the undirected edge is determined by the color similarity and spatial distance between adjacent pixels.

Images