An ADBnet-based near-color small target fruit instance segmentation method and system
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- YANCHENG INST OF TECH
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-09
Smart Images
Figure CN122176710A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of target recognition and smart agriculture, specifically to a method and system for segmenting small and medium-sized target fruit instances based on ADBnet. Background Technology
[0002] Similar-color small-to-medium-sized target fruits typically refer to fruit types that are visually similar in color to the background, small in size, and with indistinct target features, such as green peppers, strawberries, unripe green-skinned citrus fruits, plums, and kiwis. These fruits place higher demands on machine vision recognition technology. However, in existing instance segmentation techniques, for images of fruits after compression, not only is it impossible to extract effective boundary information of the fruits, but it also introduces interference information, resulting in low accuracy in fruit recognition.
[0003] Furthermore, the loss function for fruit target instance segmentation is crucial for improving the accuracy of deep learning models. Current techniques typically employ the BCEwl loss function (Binary Cross Entropy With Logits Loss), a loss function used for binary classification problems to assess the difference between predicted values and actual labels, often used for target segmentation loss calculation. However, in real-world scenarios, the segmentation boundaries of fruit targets are not always clearly defined. Using the BCEwl loss function to calculate the loss for these less defined boundaries is clearly insufficient. Essentially, the BCEwl loss function performs binary classification for each pixel. When dealing with similarly colored targets like fruit, the segmentation boundaries often exhibit ambiguity, discontinuity, or even overlap. Using this loss function for fruit instance segmentation is not sensitive enough to boundary regions, failing to effectively guide the model to learn fine boundary features, leading to a decrease in target instance segmentation accuracy. Summary of the Invention
[0004] To address the technical problems of existing instance segmentation models' network architecture being unable to adapt to the actual size and distribution of fruit targets, leading to redundant model network architectures, and the BCEwl loss function's significant shortcomings in calculating target objects for images with unclear segmentation boundaries, this invention aims to provide a near-color small-to-medium-sized fruit instance segmentation method based on ADBnet. The specific technical solution adopted is as follows:
[0005] Fruit images were collected, preprocessed, and then divided into training and validation sets.
[0006] We established a deep learning network architecture, a dual-view channel attention block network, and a binary distributed focal length loss function, respectively, and integrated them to construct an ADBnet instance segmentation model, which was then trained using a training set.
[0007] The detection accuracy requirement is preset, and the accuracy of the trained ADBnet instance segmentation model is verified using a validation set. If the detection accuracy requirement is met, the fruit image is segmented to obtain the segmented fruit.
[0008] Preferably, fruit images are acquired, preprocessed, and then divided into training and validation sets, specifically:
[0009] Fruit images are collected and cleaned to retain clear fruit images that cover the target fruit. Normalized images are obtained based on the clear fruit images and divided into training and validation sets.
[0010] Preferably, a deep learning network architecture is established, including:
[0011] Based on the normalized image, the height ratio and length ratio of all fruit targets in the corresponding normalized image are statistically analyzed;
[0012] Obtain a baseline model, analyze the height ratio and length ratio to obtain the corresponding histogram distribution density, adjust the feature extraction layers and number of channels of the baseline model, and construct a deep learning network architecture that is suitable for effective feature map extraction of the target fruit size.
[0013] Preferably, the distribution density of the corresponding histogram is obtained by analyzing the height ratio and length ratio. The feature extraction layers and number of channels of the baseline model are adjusted to construct a deep learning network architecture that is suitable for effective feature map extraction of the target fruit size, including:
[0014] Obtain the feature map compression level, and divide the histogram distribution density based on the corresponding feature map compression level to obtain the segments with height ratio and length ratio;
[0015] The proportion of fruit targets in each segment is statistically analyzed to determine the overall height ratio.
[0016] Design a channel number adjustment rule for the feature map, adjust the feature extraction layer and channel number of the baseline model according to the channel number adjustment rule based on the length-to-height ratio, and establish a deep learning network architecture adapted to the current normalized image.
[0017] Preferably, a dual-view channel attention block network is established, including:
[0018] The corresponding input feature map is obtained from the fruit image and then sequentially transmitted to the dual-view attention branch, the main convolution branch, and the optional residual branch.
[0019] Attention feature maps corresponding to the input feature maps are obtained based on dual-view attention branches. These attention feature maps are then integrated and enhanced to obtain reinforced features. Figure 1 ;
[0020] The convolutional feature map corresponding to the input feature map is obtained based on the main branch of the convolution, and then combined with the enhancement feature map. Figure 1The combined features are enhanced. Figure 2 ;
[0021] Preset selection conditions, analyze and enhance features Figure 2 If the selection criteria are met, the input feature map is concatenated with the input feature map of the optional residual branch and then convolved to obtain the output feature map; if the selection criteria are not met, the feature map is enhanced. Figure 2 This is for outputting feature maps.
[0022] Preferably, attention feature maps corresponding to the viewpoints of the input feature maps are obtained based on dual-view attention branches, and the attention feature maps are integrated and enhanced to obtain enhanced features. Figure 1 ,include:
[0023] The dual-view attention branch includes a pooling mean view and a convolution weight view. The input feature map is subjected to adaptive average pooling and convolution processing respectively to obtain the corresponding attention feature map.
[0024] The linear feature map is obtained by summing the attention feature maps from the two perspectives, and then compressed, transposed, convolved, and normalized sequentially to obtain the enhanced features. Figure 1 .
[0025] Preferably, a binary distributed focal length loss function is established, including:
[0026] Based on the fruit target, determine the target tensor and the corresponding prediction probability, obtain the sub-tensors according to the target tensor, and calculate the loss at the sub-tensors;
[0027] The loss of the combined subtensors is summed to construct a binary distributed focal length loss function, and a multi-population genetic algorithm is used to optimize the weight parameters of the binary distributed focal length loss function.
[0028] Preferably, a multi-population genetic algorithm is used to optimize the weight parameters of the binary distribution focal length loss function, including:
[0029] Define the weight parameters as the optimization objective, set the initialization parameters based on the optimization objective, and generate initial multiple populations;
[0030] Obtain the baseline model of the loss function, and train the baseline model with an initial multi-population model until it fully converges. Obtain the pre-trained weights of the binary distributed focal length loss function, and analyze the pre-trained weights to determine the optimal weight parameters for actual segmentation.
[0031] Preferably, the initialization parameters include decision variables, constraints, decision variable transformation arrays, population encoding methods, and the number of individuals in each population.
[0032] To address the aforementioned problems, this invention also provides: a near-color small-to-medium-sized target fruit instance segmentation system based on ADBnet, the system comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the near-color small-to-medium-sized target fruit instance segmentation method based on any of the preceding claims.
[0033] The present invention has the following beneficial effects:
[0034] 1. This paper proposes an ADBnet-based method for segmenting small-to-medium-sized fruit instances with similar colors. It integrates a deep learning network architecture, a dual-viewpoint channel attention block network, and a binary distributed focal length loss function to construct an ADBnet instance segmentation model. The deep learning network architecture adapts to the actual size and distribution of the fruit targets, reducing model redundancy and significantly minimizing interference from model parameters and redundant architecture. The dual-viewpoint channel attention block network improves learnability, allowing feature map generation to dynamically adjust based on the data characteristics of the fruit image, thus enhancing model performance. Furthermore, the binary distributed focal length loss function addresses the significant shortcomings of BCEwl in calculating the segmentation boundary of the actual target object. Finally, the established ADBnet instance segmentation model is used to segment the fruit image, comprehensively improving the instance segmentation accuracy of the model.
[0035] 2. The near-color small-target fruit instance segmentation system based on ADBnet provided by this invention has the same beneficial effects as the near-color small-target fruit instance segmentation method based on ADBnet provided by this invention, and will not be described in detail here. Attached Figure Description
[0036] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 This is a flowchart illustrating an implementation of a method for segmenting small-to-medium-sized fruit instances based on ADBnet, as provided in one embodiment of the present invention.
[0038] Figure 2 A schematic diagram illustrating a method for segmenting small-to-medium-sized fruit instances based on ADBnet, specifically a chili pepper, provided as an embodiment of the present invention.
[0039] Figure 3A comparison diagram of the baseline model network architecture and the deep learning network architecture of a near-color small target fruit instance segmentation method based on ADBnet provided in an embodiment of the present invention;
[0040] Figure 4 This is a schematic diagram of the structure of a dual-view channel attention block network for a near-color small-target fruit instance segmentation method based on ADBnet, provided in an embodiment of the present invention.
[0041] Figure 5 This is a schematic diagram illustrating the calculation method of the binary distribution focal length loss function in a near-color small-target fruit instance segmentation method based on ADBnet, provided in an embodiment of the present invention.
[0042] Figure 6 The flowchart illustrates the implementation of an ADBnet-based method for segmenting small-to-medium-sized fruit instances with similar colors, specifically the optimization of weight parameters in the binary distribution focal length loss function, as provided in one embodiment of the present invention.
[0043] Figure 7 A comparison curve of the binary distribution focal length loss function and existing loss functions in a near-color small target fruit instance segmentation method based on ADBnet provided in an embodiment of the present invention;
[0044] Figure 8 This is a network structure diagram of the ADBnet instance segmentation model for a near-color small-target fruit instance segmentation method provided in an embodiment of the present invention;
[0045] Figure 9 This is a training comparison diagram between the ADBnet instance segmentation model and existing segmentation models of a method for segmenting small and medium-sized fruit instances based on similar colors, provided in an embodiment of the present invention.
[0046] Figure 10 This is a bar chart showing the parameter quantities of the ADBnet instance segmentation model and existing segmentation models in a near-color small-target fruit instance segmentation method provided in an embodiment of the present invention. Detailed Implementation
[0047] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method and system for segmenting small-to-medium-sized target fruit instances based on ADBnet according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.
[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0049] The following description, in conjunction with the accompanying drawings, details the specific scheme of the method and system for segmenting small-to-medium-sized fruit instances based on ADBnet provided by this invention.
[0050] Please see Figure 1 It illustrates a flowchart of an implementation of a method for segmenting small-to-medium-sized fruit instances based on ADBnet, according to a first embodiment of the present invention. The method includes:
[0051] Step S1: Collect fruit images, preprocess them, and divide them into training and validation sets;
[0052] Step S2: Establish a deep learning network architecture, a dual-view channel attention block network, and a binary distributed focal length loss function respectively, integrate them to construct an ADBnet instance segmentation model, and train it using the training set;
[0053] Step S3: Preset the detection accuracy requirement, and use the validation set to verify the accuracy of the trained ADBnet instance segmentation model. If the detection accuracy requirement is met, perform segmentation operation on the fruit image to obtain the segmented fruit.
[0054] To better illustrate, in the process of building the ADBnet instance segmentation model, the deep learning network architecture, the dual-view channel attention block network, and the binary distributed focal length loss function are independent and run in parallel, and are portable, meaning that the deep learning model can be replaced for different actual application scenarios.
[0055] Please see Figure 2 As an optional implementation method, this application uses green peppers as an example to illustrate small target fruits with similar colors. The specific analysis can be adjusted according to the actual situation. Among them, the width ratio and height ratio of the fruit refer to the proportion of the width and height of the fruit image to the width and height of the entire fruit image, respectively. Taking the horn pepper variety for analysis, its prominent feature is its slender shape, which results in the number of green peppers with a width ratio of less than 0.1 accounting for 83.4% of the total number of green peppers; the number of green peppers with a height ratio of less than 0.1 accounting for 27.8% of the total number of green peppers; and the number of green peppers with both the width ratio and height ratio less than 0.1 accounting for 26.4% of the total number of green peppers. Therefore, these are regarded as small target fruits.
[0056] Furthermore, in step S1, specifically:
[0057] Fruit images are collected and cleaned to retain clear fruit images that cover the target fruit. Normalized images are obtained based on the clear fruit images and divided into training and validation sets.
[0058] Specifically, diverse fruit images are collected from the planting base to cover relevant situations in actual application scenarios. Next, the data is cleaned, and unqualified images that are blurry, overexposed, underexposed, or severely occluded are marked and removed, resulting in clear fruit images that cover the target fruit. Then, the clear fruit images are normalized to any square size to ensure generalization ability for subsequent analysis; optionally, 640×640, 480×480, etc., are commonly used. Finally, the normalized images are divided into training and validation sets according to a ratio. The training set is used for learning model parameters, and the validation set is used to evaluate the model's generalization ability during training and adjust parameters.
[0059] Please see Figure 3 Figure (a) shows the baseline model YOLOv8l-seg network architecture (Y8lNA), and Figure (b) shows the network architecture for effective feature map extraction adapted to target size characteristics (AT-EFNA). Further, in step S2, a deep learning network architecture is established, including:
[0060] Step S201: Based on the normalized image, calculate the height ratio and length ratio of all fruit targets in the corresponding normalized image;
[0061] Step S202: Obtain the baseline model, analyze the height ratio and length ratio to obtain the corresponding histogram distribution density, adjust the feature extraction layers and number of channels of the baseline model, and construct a deep learning network architecture for effective feature map extraction adapted to the target size of the fruit.
[0062] As an optional implementation, in this embodiment, the baseline model is the YOLOv8l network architecture (Y8lNA), that is, based on the baseline model, a deep learning network architecture (AT-EFNA, i.e., Adaptive Target-Size Effective Feature Map Network Architecture) is constructed to extract effective feature maps adapted to the target size of the fruit.
[0063] Further, step S202 includes:
[0064] Step S2021: Obtain the feature map compression level, divide the histogram distribution density based on the feature map compression level, and obtain the segments with height ratio and length ratio.
[0065] Preferably, in this embodiment, the feature map compression level is 5 levels, which can be specifically set according to the actual situation.
[0066] Specifically, the height and length ratios of fruit targets in fruit images are analyzed, as they reflect the morphological characteristics of the fruit targets. The number of fruit targets within different height and length ratio intervals is statistically analyzed and visualized as a bar chart to visually display the distribution density of fruit morphological features, thus obtaining the corresponding bar chart distribution density. The number of segments for the height and length ratios is the same as the feature map compression level, and the two correspond one-to-one. Therefore, a 5-level feature map compression level is used. Determine the segments with height ratio and / or length ratio, denoted as... The correspondence between the two is as follows: .
[0067] Step S2022: Based on the segmented statistics of the proportion of fruit targets to the total, and integrate them to determine the overall growth ratio.
[0068] Specifically, the proportion of fruit targets to the total is statistically analyzed segment by segment based on both height ratio and length ratio, where the height ratio corresponds to... Length ratio corresponding The combined length-to-height ratio is determined by combining the two data points, i.e., Rt = [max(rth1, rtl1), max(rth2, rtl2), max(rth3,rtl3), max(rth4, rtl4), max(rth5, rtl5)] = [rt1, rt2, rt3, rt4, rt5].
[0069] Step S2023: Design the channel number adjustment rules for the feature map. Adjust the feature extraction layer and channel number of the baseline model according to the channel number adjustment rules based on the length-to-height ratio, and establish a deep learning network architecture that adapts to the current normalized image.
[0070] Specifically, when the overall length-to-height ratio If the ratio is less than 0.125, then the current feature map compression level is... The number of channels in the extraction layer is reduced by 50%; when If the proportion is less than 0.05, then the current All channels of the extraction layer are removed, i.e., the entire layer's feature map is removed; when If the ratio is greater than 0.375, the current number of channels increases by 50%; when If the proportion is greater than 0.5, an additional feature extraction layer is added to the current feature map, with the same number of channels. The additional feature extraction layer can be any feature extraction network layer, i.e., a custom or existing feature extraction network layer, such as C2-DVCAB, C2f, etc. In the channel number adjustment rules of the feature map, the values of 0.125, 0.05, 0.375 and 0.5 can be adjusted according to the actual situation.
[0071] It can be explained that if an additional feature extraction layer is added to the C / 4 level feature map, then this additional feature extraction layer is added to the input port of the Neck network module, and the C / 2 level feature map is downsampled and connected to the Neck network module. The Neck network module is a component of the baseline model, and its structure is diverse, including Feature Pyramid Network (FPN), Path Aggregation Network (PANet), and Bidirectional Feature Pyramid Network (BiFPN). If all C / 8, C / 16, or C / 32 level feature maps are removed, then the connection to the feature extraction layer of that level is removed from the input port of the Neck network module. Finally, after adjusting according to the length-to-height ratio and the channel number adjustment rules, the channels of each level feature map change, and the entire level feature map is pruned. The input and output channels are then rematched to establish a deep learning network architecture adapted to the current normalized image.
[0072] To better illustrate and verify the applicability of the proposed deep learning network architecture, we will use the green bell pepper fruit, corresponding to the horn-shaped bell pepper, as an example. Specifically, the 5th (C / 32) feature map compression and feature extraction layer was removed, significantly reducing the parameters of the deep learning network architecture. This is because, for the elongated shape of the green bell pepper, the high-level (C / 32) feature map has very weak representation ability for the pepper's information, resulting in almost no ability to recognize the pepper's segmentation boundary features; therefore, it was directly removed. After the 1st (C / 2) feature map compression, an additional low-level feature map extraction layer was added, focusing more on the extraction of features from small and medium-sized targets. This is because the low-level network has a relatively small receptive field, strong geometric detail representation ability, high resolution, but weak semantic information representation ability; therefore, the addition of the low-level feature map extraction layer enhances the feature extraction of the pepper's shape and boundaries. When the input interface of the Neck network module is the feature extraction layer C / 4, C / 8, and C / 16, an additional low-level feature map extraction layer (C / 4) is added, and the interface of the high-level feature layer C / 32 is removed. At the same time, a feature extraction layer C / 2 is added independently, so that the deep learning network architecture can focus more on the aggregation and output of mid-to-shallow features. The number of output channels of all mid-to-high-level (C / 16 or C / 8) layers is halved, which significantly reduces the parameters and computation of the network architecture. That is, the receptive field of mid-to-high-level layers is relatively large and has sufficient semantic information representation ability. Since the mid-to-high-level layers have a weaker ability to represent green pepper information, their output channels are halved, which reduces unnecessary computation and still achieves semantic information representation.
[0073] Please see Figure 4 Furthermore, in step S2, a dual-view channel attention block network is established, including:
[0074] Step S211: Obtain the corresponding input feature map from the fruit image and transmit it sequentially to the dual-view attention branch, the main convolution branch and the optional residual branch.
[0075] It can be explained that the input feature map for each image is obtained based on the image data after preprocessing the fruit image, and this feature map is used as the input to the dual-view channel attention block network, denoted as . , This indicates the number of channels in the input feature map; , These represent the height and width of the input feature map, respectively.
[0076] Step S212: Obtain attention feature maps corresponding to the viewpoints of the input feature maps based on dual-view attention branches, and integrate the attention feature maps for enhancement processing to obtain enhanced features. Figure 1 .
[0077] To explain, the dual-perspective attention branch, namely... .
[0078] Further, step S212 includes:
[0079] Step S2121: The dual-view attention branch includes the pooling mean view and the convolution weight view. The input feature map is subjected to adaptive average pooling and convolution processing respectively to obtain the corresponding attention feature map.
[0080] Specifically, based on the input feature map From the perspective of pooling mean, adaptive average pooling is performed to obtain the attention feature map from the pooling mean perspective. The corresponding calculation formula is as follows:
[0081]
[0082] in, This represents the attention feature map corresponding to the pooling mean view. ; This indicates average pooling.
[0083] From the perspective of convolution weights, the input feature map After convolution with output channel 1 and kernel 1 The feature map is calculated. Thus, the normalized result is obtained. weight This refers to the attention feature map corresponding to the convolution weights, where the weights are learnable parameters, in order to obtain a learnable attention feature map from the perspective of convolution weights. The corresponding calculation formula is:
[0084]
[0085]
[0086] in, This indicates convolution processing. Indicates that the convolution kernel is , This indicates a step size of 1. This indicates that the output channel is 1.
[0087] Understandably, Indicates normalization processing; based on weights The solution process involves using the convolutional weights in the Dual-View Channel Attention Block Network (DVCAB) with different perspectives. The weights of the convolution are used as attention feature maps, and the weights... These are learnable parameters, while existing attention mechanisms SE (Squeeze-and-Excitation) or CBAM (Convolutional Block Attention Module) use the mean or maximum value of the channel feature maps as attention feature maps, resulting in different methods for obtaining these attention feature maps. Furthermore, during training, the weights... Updates are performed using the backpropagation algorithm to minimize the loss function, which has learnability. Convolution can be dynamically adjusted according to the characteristics of the data, thereby improving the performance of DVCAB.
[0088] Step S2122: Summing the attention feature maps based on the dual-viewpoints yields a linear feature map, which is then compressed, transposed, convolved, and normalized sequentially to obtain enhanced features. Figure 1 .
[0089] Specifically, the attention feature maps corresponding to the two perspectives and Summation yields the linear feature map. ,Right now Then compress it to linear and transpose it to obtain... ,Right now , Indicates in Dimensional compression tensors This represents the feature map tensor transpose operation; then, it is passed through an adaptive convolution kernel with an output channel of C. ,Right now One-dimensional convolution The enhanced features are obtained after calculation and normalization using the sigmoid activation function. Figure 1 ,Right now .
[0090] Step S213: Obtain the convolutional feature map corresponding to the input feature map based on the main convolutional branch, and combine it with the enhancement feature map. Figure 1 The combined features are enhanced. Figure 2 .
[0091] Specifically, the main branch of convolution, i.e. Input feature map After convolution with two output channels of C and a kernel size of 3... Calculate the convolutional feature map ,Right now ; will enhance features Figure 1 Transpose and expand the dimension to generate a feature map. ,Right now , Indicates in Dimensionally expand the tensor; then combine it with the convolutional feature map obtained from the main convolution branch. Element-wise multiplication yields enhanced features Figure 2 ,Right now .
[0092] Step S214: Set preset selection conditions and analyze enhancement features. Figure 2 If the selection criteria are met, the input feature map is concatenated with the input feature map of the optional residual branch and then convolved to obtain the output feature map; if the selection criteria are not met, the feature map is enhanced. Figure 2 This is for outputting feature maps.
[0093] Specifically, based on reinforcement features Figure 2 Further analysis will be conducted, and the selected conditions will be denoted as follows: It is set in a dual-view channel attention block network, if Set to True to enhance features. Figure 2 With optional residual branches Input feature map Concatenated along the channel dimension, then convolved with C output channels and a kernel of 1. Calculate and obtain the output feature map. Conversely, if If the value is False, the feature will be enhanced. Figure 2 Directly passed to the output as the output feature map The corresponding calculation formula is:
[0094]
[0095] in, This indicates that the splicing is performed along the channel dimension, and the number of output channels is 2C.
[0096] Understandably, the advantage of DFL (Distribution Focal Loss) lies in transforming the bounding box regression problem into a bounding box probability distribution problem. It uses the cross-entropy function to optimize the probability of the two positions to the left and right of the label value, so that the network distribution focuses on the vicinity of the label value. Therefore, based on the idea of probability distribution, a binary distribution focal loss function (BDFL, or Binary Distribution Focal Loss) is proposed to optimize the probability of the two positions to the left and right of the binary segmentation label, so that the segmentation position focuses on the vicinity of the label value. That is, the left and right positions of the target tensor are calculated to determine the flexible integral target.
[0097] Please combine Figure 5 and Figure 6 Furthermore, in step S2, a binary distributed focal length loss function is established, including:
[0098] Step S221: Determine the target tensor and the corresponding prediction probability based on the fruit target, obtain the sub-tensors according to the target tensor, and calculate the loss at the sub-tensors.
[0099] Specifically, the category label corresponding to the fruit target in the fruit image is defined as the target tensor. That is, the target tensor is the label value of the segment (target). , Represents the target tensor Total number of elements in the middle; Indicates the element index; Represent the binary target value; and determine the prediction probability corresponding to the target tensor, i.e. , This represents the binary predicted value.
[0100] Further analysis is conducted using the target tensor and the corresponding predicted probabilities, where, It is the target tensor median element 0 gather, It is the target tensor median element 0 Predicted probability The set of elements, and the corresponding calculation formula is:
[0101]
[0102]
[0103] in, Represents the target tensor The value in the middle represents the total number of zero elements; It is represented as the 0th element index.
[0104] Similarly, the target tensor contains elements of 1, i.e. It is the target tensor Median element is 1 gather, It is the target tensor Median element is 1 Predicted probability The set of elements, and the corresponding calculation formula is:
[0105]
[0106]
[0107] in, Represents the target tensor The middle element represents the total number of 1s. It is represented as an element index of 1.
[0108] Based on the element correspondences in the target tensor, two sub-tensors are constructed, denoted as follows: and First, compute the subtensor. The loss at that location is calculated using the following formula:
[0109]
[0110]
[0111]
[0112] Secondly, compute the subtensor The loss at that location is calculated using the following formula:
[0113]
[0114]
[0115]
[0116] in, , This represents the flexible integral weight coefficient, which is the weight parameter used to subsequently integrate and establish the binary distributed focal length loss function. , .
[0117] Step S222: The loss of the combined subtensors is summed to construct a binary distributed focal length loss function, and the weight parameters of the binary distributed focal length loss function are optimized using a multi-population genetic algorithm.
[0118] Specifically, a binary distributed focal length loss function is constructed based on the loss at the two sub-tensors in step S221, which is a sub-tensor. Losses at the location Harmony tensor Losses at the location The formula for summing is:
[0119]
[0120] Understandably, a multi-population genetic algorithm is used to optimize the weight parameters of the binary distribution focal length loss function (BFDL), i.e. and To improve the optimal accuracy of instance segmentation in subsequent segmentation model construction, the Multi-Population Genetic Algorithm (MPGA) is an improved scheme based on genetic algorithms. It divides the problem solution space into multiple independent or semi-independent subpopulations and performs independent genetic operations on each subpopulation. At the same time, it introduces appropriate migration mechanisms between subpopulations to share information and promote diversity, so as to avoid problems such as getting trapped in local optima, slow convergence speed, and premature convergence.
[0121] Further, in step S222, a multi-population genetic algorithm is used to optimize the weight parameters of the binary distribution focal length loss function, including:
[0122] Step S2221: Define the weight parameters as the optimization objective, set the initialization parameters based on the optimization objective, and generate initial multiple populations.
[0123] Specifically, the weight parameter is the optimization objective, i.e., max{(Box (model{BDFL(α, β)})+Mask (model{BDFL(α, β)})) / 2}, in this embodiment, the training epoch of the model is set to 15 to obtain the optimal weight parameters, and the target value {(Box +Mask ) / 2}.
[0124] Furthermore, the initialization parameters include decision variables, constraints, decision variable transformation arrays, population encoding methods, and the number of individuals in each population.
[0125] Specifically, the decision variables are , This represents the number of individuals in a population; therefore, different populations have different numbers of individuals. Different; constraints are , Indicates the population individual ID index. , Representing arrays respectively and All elements are discrete variables; the decision variable transformation array is... Based on the actual weight parameters, depending on the specific circumstances. and To retain three decimal places, the array interval is 0.001; population settings are performed, with the population encoding method being RI (mixed real-integer encoding), and the population count matrix is as follows. .
[0126] Step S2222: Obtain the baseline model of the loss function, and train the baseline model with the initial multi-group training until it fully converges to obtain the pre-trained weights of the binary distributed focal length loss function. Analyze the pre-trained weights to determine the optimal weight parameters for actual segmentation.
[0127] Specifically, the baseline model of the loss function is... The training proceeds until full convergence (epoch=200). The resulting weights are then used as pre-training weights for the binary distributed focal length loss function. This effectively reduces the number of training iterations for the binary distributed focal length loss function, allowing it to reach the convergence criterion with fewer training iterations, thus distinguishing different weight parameters. and The real impact on model accuracy, i.e., solving the optimization objective problem based on multi-population genetic algorithms to obtain the optimal solution. This is then transformed into the actual optimal solution, i.e., the actual optimal weight parameters for segmentation. We set it in the binary distributed focal length loss function, retrained it, and set the epoch to 200 to obtain the instance segmentation accuracy of the model without pre-trained weights.
[0128] Please see Figure 7 To better illustrate and verify the reliability of the binary distributed focal length loss function in the model, the binary distributed focal length loss function, along with existing BCEwl (Binary Cross-Entropy with Weighted Loss), FL (Focal Loss), and VFL (Variational Focal Loss) segmentation losses, were applied to the baseline model YOLOv8l-seg to verify four core accuracy metrics, namely Box... Box Mask and Mask It can be seen that the binary distribution focal length loss function performs best on the validation set, and its performance is superior.
[0129] Please see Figure 8 It can be explained that in step S2, based on the sequentially established deep learning network architecture, dual-view channel attention block network, and binary distributed focal length loss function, an ADBnet (Adaptive Dilated Bottleneck Network) instance segmentation model is constructed and trained using a training set. The corresponding total loss function is established by fusing the binary distributed focal length loss function, and the corresponding calculation formula is as follows:
[0130]
[0131] in, Represents the total loss function; The baseline model representing the loss function; This represents the loss function operator for bounding box overlap. This indicates Distribution Focal Loss; This represents the focal length loss function for a binary distribution. , These represent the feature map of the predicted classification value and the feature map of the target value, respectively. , These represent the feature map of the predicted value of the detection box and the feature map of the target value, respectively. , These represent the feature map of the predicted segment and the feature map of the target segment, respectively.
[0132] It is explained that the segmentation loss adopts a binary distributed focal length loss function instead of the original baseline model. Optimize the segmentation boundary position to improve the performance of instance segmentation in the entire ADBnet instance segmentation model.
[0133] Next, in step S3, the training ADBnet instance segmentation model is used to verify the segmentation accuracy of the fruit target using a validation set. If the detection accuracy requirement is met, the ADBnet instance segmentation model is deployed on the mobile terminal system of the corresponding fruit-picking robot to achieve the corresponding instance segmentation and obtain the segmented fruit. Otherwise, if the detection accuracy requirement is not met, the process returns to step S2, and the analysis is re-expanded based on the fruit target to construct an ADBnet instance segmentation model that can meet the detection accuracy requirement.
[0134] Understandably, this paper proposes an ADBnet-based method for segmenting small-to-medium-sized fruit instances with similar colors. This method integrates a deep learning network architecture, a dual-viewpoint channel attention block network, and a binary distributed focal length loss function to construct an ADBnet instance segmentation model. The deep learning network architecture adapts to the actual size and distribution of the fruit targets, reducing model redundancy and significantly minimizing interference from model parameters and redundant architecture. The dual-viewpoint channel attention block network improves learnability, allowing feature map generation to dynamically adjust based on the data characteristics of the fruit image, thus enhancing model performance. Furthermore, the binary distributed focal length loss function addresses the significant shortcomings of BCEwl in calculating the segmentation boundary of the actual target object. Finally, the established ADBnet instance segmentation model is used to segment the fruit image, comprehensively improving the instance segmentation accuracy of the model.
[0135] Please combine Figure 9 and Figure 10 To better illustrate and verify the reliability of the ADBnet instance segmentation model, it is compared and analyzed with existing technical models, including BoxInst (Box Instance Segmentation), SOLOv2 (Segmenting Objects by Locations v2), Mask R-CNN (Mask Region-based Convolutional Neural Networks), RTMDet (Real-Time and Memory-Efficient Detection), and YOLOv8l-seg to YOLOv13l-seg, totaling 10 models. The Box Inst model is also compared and analyzed. Box Mask and Mask Based on the four core accuracy metrics, it can be seen that the ADBnet instance segmentation model performs best on the validation set, has superior instance segmentation performance, and has the fewest parameters.
[0136] The second embodiment of the present invention provides a near-color small-to-medium-sized target fruit instance segmentation system based on ADBnet. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the near-color small-to-medium-sized target fruit instance segmentation method based on ADBnet as described in any embodiment of the present invention.
[0137] When it is in operation, it needs to use a near-color small target fruit instance segmentation system based on ADBnet. Therefore, whether the system and program data are integrated or different hardware is configured to produce a function with similar effect to that achieved by the present invention, it is within the protection scope of the present invention. The system has the same beneficial effect as the near-color small target fruit instance segmentation system based on ADBnet provided above, and will not be described in detail here.
[0138] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0139] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
Claims
1. A method for segmenting small-to-medium-sized fruit instances based on ADBnet, characterized in that, The method includes: Fruit images were collected, preprocessed, and then divided into training and validation sets. We established a deep learning network architecture, a dual-view channel attention block network, and a binary distributed focal length loss function, respectively, and integrated them to construct an ADBnet instance segmentation model, which was then trained using a training set. The detection accuracy requirement is preset, and the accuracy of the trained ADBnet instance segmentation model is verified using a validation set. If the detection accuracy requirement is met, the fruit image is segmented to obtain the segmented fruit.
2. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 1, characterized in that, Fruit images were collected, preprocessed, and then divided into training and validation sets, as follows: Fruit images are collected and cleaned to retain clear fruit images that cover the target fruit. Normalized images are obtained based on the clear fruit images and divided into training and validation sets.
3. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 2, characterized in that, Establishing a deep learning network architecture includes: Based on the normalized image, the height ratio and length ratio of all fruit targets in the corresponding normalized image are statistically analyzed; Obtain a baseline model, analyze the height ratio and length ratio to obtain the corresponding histogram distribution density, adjust the feature extraction layers and number of channels of the baseline model, and construct a deep learning network architecture that is suitable for effective feature map extraction of the target fruit size.
4. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 3, characterized in that, Analyzing the height and length ratios yields the corresponding histogram distribution density. The feature extraction layers and number of channels in the baseline model are adjusted, and a deep learning network architecture suitable for effective feature map extraction based on the target fruit size is constructed, including: Obtain the feature map compression level, and divide the histogram distribution density based on the corresponding feature map compression level to obtain the segments with height ratio and length ratio; The proportion of fruit targets in each segment is statistically analyzed to determine the overall height ratio. Design a channel number adjustment rule for the feature map, adjust the feature extraction layer and channel number of the baseline model according to the channel number adjustment rule based on the length-to-height ratio, and establish a deep learning network architecture adapted to the current normalized image.
5. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 2, characterized in that, Establish a dual-view channel attention block network, including: The corresponding input feature map is obtained from the fruit image and then sequentially transmitted to the dual-view attention branch, the main convolution branch, and the optional residual branch. Based on the dual-view attention branch, attention feature maps corresponding to the viewpoints of the input feature maps are obtained. The attention feature maps are then integrated and enhanced to obtain enhanced feature map one. Based on the main branch of convolution, the convolutional feature map corresponding to the input feature map is obtained, and then combined with the first enhanced feature map to obtain the second enhanced feature map. The selection criteria are preset, and the enhanced feature map 2 is analyzed. If the selection criteria are met, it is concatenated with the input feature map of the optional residual branch and convolved to obtain the output feature map; if the selection criteria are not met, the enhanced feature map 2 is the output feature map.
6. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 5, characterized in that, Based on the dual-view attention branch, attention feature maps corresponding to the viewpoints of the input feature map are obtained. These attention feature maps are then integrated and enhanced to obtain Enhanced Feature Map 1, which includes: The dual-view attention branch includes a pooling mean view and a convolution weight view. The input feature map is subjected to adaptive average pooling and convolution processing respectively to obtain the corresponding attention feature map. The linear feature map is obtained by summing the attention feature maps from the dual perspectives, and then compressed, transposed, convolved and normalized in sequence to obtain the enhanced feature map one.
7. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 2, characterized in that, Establish a binary distributed focal length loss function, including: Based on the fruit target, determine the target tensor and the corresponding prediction probability, obtain the sub-tensors according to the target tensor, and calculate the loss at the sub-tensors; The loss of the combined subtensors is summed to construct a binary distributed focal length loss function, and a multi-population genetic algorithm is used to optimize the weight parameters of the binary distributed focal length loss function.
8. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 7, characterized in that, A multi-population genetic algorithm is used to optimize the weight parameters of the binary distribution focal length loss function, including: Define the weight parameters as the optimization objective, set the initialization parameters based on the optimization objective, and generate initial multiple populations; Obtain the baseline model of the loss function, and train the baseline model with an initial multi-population model until it fully converges. Obtain the pre-trained weights of the binary distributed focal length loss function, and analyze the pre-trained weights to determine the optimal weight parameters for actual segmentation.
9. The method for segmenting small-to-medium color target fruit instances based on ADBnet according to claim 8, characterized in that, The initialization parameters include decision variables, constraints, decision variable transformation arrays, population encoding methods, and the number of individuals in each population.
10. A near-color small-to-medium-sized target fruit instance segmentation system based on ADBnet, characterized in that, The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the ADBnet-based method for segmenting small and medium-sized fruit instances with similar colors as described in any one of claims 1 to 9.