A dense stack of deep learning-based green ball diameter distribution visual measurement method
By using the GPDNet raw particle target detection network to detect the bounding boxes of raw particle instances and construct spatial cue information, the segmentation model is guided to extract the segmentation mask and perform circle fitting. This solves the real-time and accuracy problems of raw particle size distribution measurement in existing technologies and realizes high-precision automated particle size distribution measurement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUXI UNIV
- Filing Date
- 2026-04-21
- Publication Date
- 2026-08-04
AI Technical Summary
Existing techniques for measuring the size distribution of green pellets suffer from poor real-time performance and high reliance on manual intervention. Traditional image segmentation methods lack robustness in densely stacked or occluded scenarios, while deep learning methods often fail to accurately locate the bounding boxes of green pellet instances and miss small-diameter targets in complex industrial settings, all of which affect the accuracy of the particle size distribution measurement results.
The GPDNet network for detecting raw objects is used to detect raw objects, output bounding boxes and construct spatial cue information to guide the instance segmentation model to extract segmentation masks, and combine edge contour point sets to perform circle fitting to generate particle size distribution results.
It improves the accuracy and robustness of measuring the particle size distribution of green pellets in densely stacked, adhered, and occluded scenarios, and realizes efficient automated visual measurement of particle size distribution.
Smart Images
Figure CN122089814B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of machine vision inspection and intelligent inspection technology in the metallurgical industry, specifically to a deep learning-based visual measurement method for the particle size distribution of densely stacked green pellets. Background Technology
[0002] Green pellets are an important raw material in modern blast furnace ironmaking processes. Their particle size distribution directly affects the subsequent roasting quality, permeability, and energy utilization efficiency, making them a crucial indicator for evaluating green pellet quality and guiding process control. Therefore, accurate and rapid measurement of green pellet particle size distribution is of great significance for improving product quality and optimizing the production process.
[0003] Currently, the measurement of green pellet size distribution in industrial settings still primarily relies on offline methods such as manual sieving. This involves taking samples periodically during production, sieving them through standard sieves of different aperture sizes, and then statistically analyzing the distribution across each particle size range. This method suffers from low measurement efficiency, high reliance on manual labor, long detection cycles, and delayed feedback, making it difficult to meet the demands of modern metallurgical industries for online, automated, and real-time detection.
[0004] With the development of machine vision and deep learning technologies, image-based methods for measuring green pellet size have gradually attracted attention. Current techniques typically involve deploying industrial cameras at stations such as the stabilization zone, conveyor belt, and discharge port of a disc pelletizer to acquire images of the green pellets. These images are then used to identify the green pellet regions based on image segmentation, target detection, or instance segmentation methods, followed by geometric fitting or size conversion to estimate the particle size. Compared to traditional manual screening methods, this approach offers advantages such as non-contact operation, automation, and high detection speed.
[0005] However, in actual industrial production environments, green pellets often exhibit dense stacking, mutual adhesion, and partial occlusion, especially in the stable zone of disc pelletizing machines and conveyor belt stations. In such scenarios, the boundaries of individual green pellets in the image are prone to overlap or blurring, making it difficult for traditional image segmentation algorithms to accurately separate individual green pellet instances. Existing traditional methods such as watershed and morphological segmentation are quite sensitive to changes in illumination, background noise, and boundary adhesion, easily leading to oversegmentation or undersegmentation. Although some deep learning-based instance segmentation methods can achieve good segmentation results in scenarios with relatively sparse targets, in densely stacked scenarios, there are still problems such as inaccurate localization of green pellet instance boundaries, difficulty in separating occluded areas, and missed detection of small-diameter targets, making it difficult to guarantee the accuracy of subsequent particle size calculation results.
[0006] Furthermore, raw particle images from different workstations in industrial settings exhibit significant differences in imaging distance, stacking density, and particle size distribution, resulting in substantial scale variations in raw particle targets. Existing methods often struggle to balance detection accuracy, segmentation accuracy, and computational efficiency when handling multi-workstation, multi-scale mixed scenarios, thereby affecting the stability and reliability of raw particle size distribution measurement results.
[0007] In summary, existing green pellet size distribution measurement technologies suffer from at least the following shortcomings: First, offline measurement methods such as manual sieving are inefficient and lack real-time performance, failing to meet online detection requirements. Second, traditional image segmentation methods lack robustness in densely stacked or occluded scenarios, making it difficult to accurately separate individual green pellet instances. Third, existing deep learning methods lack sufficient accuracy in defining bounding boxes for densely stacked green pellets in complex industrial scenarios, particularly prone to inaccurate boundary localization and missed detection of small-diameter targets under conditions of occlusion and multi-scale mixing, thus affecting the accuracy of subsequent instance segmentation and particle size measurement results. Therefore, there is an urgent need to provide a method suitable for densely stacked green pellet scenarios that can achieve high-precision visual measurement of particle size distribution. Summary of the Invention
[0008] To address the shortcomings of existing technologies in measuring the particle size distribution of raw balls, such as poor real-time performance, high reliance on manual intervention, inaccurate bounding box localization of raw ball instances in densely stacked or occluded scenarios, missed detection of small-diameter targets, and insufficient particle size measurement accuracy, this invention provides a deep learning-based visual measurement method for the particle size distribution of raw balls in densely stacked scenarios. This method employs the GPDNet raw ball detection network to detect raw ball instances in densely stacked scenarios and outputs the bounding boxes of each raw ball instance. Then, spatial cue information is constructed based on the bounding boxes to guide the instance segmentation model to obtain the corresponding segmentation mask. Further, edge contour point sets are extracted based on the segmentation mask and circle fitting is performed to obtain the diameter of each raw ball instance, ultimately generating the raw ball particle size distribution result. This method can improve the accuracy and robustness of visual measurement of raw ball particle size distribution in densely stacked or occluded scenarios.
[0009] In a first aspect, embodiments of the present invention provide a deep learning-based visual measurement method for the particle size distribution of densely stacked green pellets, the method comprising:
[0010] Obtain the raw ball image of the workstation to be tested;
[0011] The raw ball image is input into a trained raw ball target detection network to obtain the bounding boxes of each raw ball instance in the raw ball image. The raw ball target detection network is GPDNet, which includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network includes a C3K2_MAB module, the neck network includes an HPFA module and an EG-SDI module, and an LSKA module is set in front of the detection head. The backbone network is used to extract multi-scale features from the raw ball image, the neck network is used to align, fuse, and enhance the multi-scale features, and the detection head is used to output the bounding boxes of each raw ball instance based on the enhanced features.
[0012] The bounding boxes of each raw ball instance are used to construct spatial cue information, and the spatial cue information and the raw ball image are input into a trained instance segmentation model to obtain the segmentation mask corresponding to each raw ball instance.
[0013] The edge contour point set is extracted based on the segmentation mask of each raw ball instance, and a circle is fitted based on the edge contour point set to obtain the diameter of each raw ball instance.
[0014] Statistical analysis was performed on the diameters of all green pellet instances to generate green pellet size distribution results.
[0015] In one possible implementation, the raw sphere target detection network GPDNet includes a backbone network, a neck network, and a detection head connected in sequence; the backbone network is used to extract multi-scale features from the raw sphere image, the neck network is used to align, fuse, and enhance the multi-scale features, and the detection head is used to output bounding boxes for each raw sphere instance based on the enhanced features.
[0016] In one possible implementation, the backbone network includes a C3K2_MAB module, which includes a grouped multi-scale large kernel attention unit and a spatially gated attention unit; wherein the grouped multi-scale large kernel attention unit is used to capture long-range dependencies, and the spatially gated attention unit is used to enhance the edge features of live objects and suppress background interference to output enhanced multi-scale features.
[0017] In one possible implementation, the neck network includes a hierarchical parameterless feature alignment module (HPFA). The HPFA module uses a mesoscale feature layer as a spatial anchor point and aligns features at different levels to the same spatial resolution through a parallel alignment strategy. Specifically, high-resolution features are downsampled to the resolution corresponding to the spatial anchor point through adaptive pooling, low-resolution features are upsampled to the resolution corresponding to the spatial anchor point through nearest neighbor interpolation, mesoscale features maintain an identity mapping, and the aligned features of each layer are spliced and fused in the channel dimension.
[0018] In one possible implementation, the neck network includes a semantic and detail interaction module (EG-SDI), which is used to adaptively fuse aligned shallow and deep features. The EG-SDI module generates fusion weights based on the information entropy of the features and performs weighted fusion of features at different levels according to the fusion weights to enhance the edge detail features of the spherical body and suppress background interference.
[0019] In one possible implementation, a Large Kernel Separable Attention Module (LSKA) is provided in front of the detection head. The LSKA module includes a depth convolution decomposition unit, a dilated convolution decomposition unit, and an attention fusion unit. The depth convolution decomposition unit is used to extract local texture and edge features through depth convolution in the horizontal and vertical directions. The dilated convolution decomposition unit is used to extract global context features through dilated depth convolution in the horizontal and vertical directions. The attention fusion unit is used to generate a spatial attention map and perform weighted enhancement on the input features based on the spatial attention map.
[0020] In one possible implementation, the spatial cue information is constructed from the bounding boxes output by the raw sphere target detection network. The spatial cue information and the raw sphere image are jointly input into the instance segmentation model to guide the instance segmentation model to perform pixel-level segmentation on the corresponding raw sphere instances and output the segmentation mask corresponding to each raw sphere instance.
[0021] In one possible implementation, the step of constructing corresponding spatial cue information based on each of the bounding boxes, and inputting the spatial cue information and the raw ball image into an instance segmentation model to obtain a segmentation mask corresponding to each raw ball instance includes: inputting the bounding boxes into a cue encoder to generate cue features; inputting the raw ball image into an image encoder to generate image features; and inputting the cue features and the image features into a mask decoder to output a segmentation mask corresponding to each raw ball instance.
[0022] In one possible implementation, the step of extracting edge contour point sets based on the segmentation mask of each raw sphere instance, and performing circle fitting based on the edge contour point sets to obtain the diameter of each raw sphere instance includes:
[0023] Extract the edge pixels of the segmentation mask of each raw ball instance to form an edge contour point set;
[0024] With the objective of minimizing the sum of squared distances from each point in the edge contour point set to the fitted circle, the center and radius of the fitted circle are solved.
[0025] The diameter of each green bulb instance is calculated based on the radius.
[0026] Secondly, embodiments of the present invention provide a deep learning-based visual measurement device for the particle size distribution of densely stacked green pellets, the device comprising:
[0027] The image acquisition unit is used to acquire images of the raw balls at the workstation under test.
[0028] The target detection unit is used to input the raw ball image into a trained raw ball target detection network to obtain the bounding box of each raw ball instance in the raw ball image;
[0029] The cue construction and segmentation unit is used to construct spatial cue information based on the bounding box, and input the spatial cue information and the raw ball image into a trained instance segmentation model to obtain the segmentation mask corresponding to each raw ball instance;
[0030] The contour fitting diameter measurement unit is used to extract the edge contour point set according to the segmentation mask of each raw ball instance, and to fit the diameter of each raw ball instance based on the edge contour point set.
[0031] The particle size distribution generation unit is used to statistically analyze the diameter of each green pellet instance and generate the green pellet particle size distribution result.
[0032] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the deep learning-based visual measurement method for the particle size distribution of densely stacked green balls as described in any of the above embodiments.
[0033] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the deep learning-based visual measurement method for densely stacked green pellet size distribution described in any of the above embodiments.
[0034] Beneficial effects:
[0035] Compared with the prior art, the present invention has at least the following beneficial effects: (1) By setting the C3K2_MAB module, HPFA module, EG-SDI module and LSKA module in the raw ball target detection network GPDNet, the present invention enhances the representation and fusion capabilities of multi-scale targets, edge detail information and global context information, which is conducive to improving the accuracy of raw ball instance bounding box detection in densely stacked and occluded scenes; (2) By constructing the bounding box output by the target detection network as spatial cue information, the present invention guides the instance segmentation model to focus on the corresponding raw ball instance region, reduces the segmentation interference between adjacent occluded targets, and improves the boundary accuracy and instance integrity of the segmentation mask; (3) The present invention forms a complete visual measurement process from target detection, instance segmentation to particle size measurement and statistical analysis, which can improve the automation level and robustness of particle size distribution measurement in densely stacked raw ball scenes. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 The overall flowchart of the deep learning-based visual measurement method for the particle size distribution of densely stacked green balls provided in the embodiments of the present invention is shown below.
[0038] Figure 2 This is a diagram showing the overall structure of the GPDNet network for detecting live targets provided in an embodiment of the present invention.
[0039] Figure 3 This is a structural diagram of the C3K2_MAB module provided in an embodiment of the present invention.
[0040] Figure 4 This is a structural diagram of the hierarchical parameterless feature alignment module HPFA provided in an embodiment of the present invention.
[0041] Figure 5 The diagram shows the structure of the Entropy-Guided Semantic Detail Interaction (EG-SDI) module provided in this embodiment of the invention.
[0042] Figure 6 This is a structural diagram of the Large Core Separable Attention Module (LSKA) provided in an embodiment of the present invention.
[0043] Figure 7 This is a schematic diagram showing the target detection, instance segmentation, and particle size distribution results of densely stacked sphere images in an embodiment of the present invention.
[0044] Figure 8 This is a schematic diagram of the structure of a deep learning-based visual measurement device for the particle size distribution of densely stacked green balls, provided in an embodiment of the present invention. Detailed Implementation
[0045] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Other embodiments obtained by those skilled in the art based on the disclosure of this invention without inventive effort should all fall within the scope of protection of this invention.
[0046] In the description of this invention, the terms "first," "second," etc., are used only to distinguish different objects and should not be construed as indicating or implying relative importance. Unless otherwise expressly specified and limited, the term "a plurality of" refers to two or more. The term "and / or" describes the relationship between related objects, indicating that three relationships may exist, for example, A and / or B, which can mean that A exists alone, B exists alone, or A and B exist simultaneously. Furthermore, the terms "comprising," "including," and any variations thereof are intended to cover non-exclusive inclusion.
[0047] As mentioned earlier, green pellet size distribution is a crucial indicator affecting the quality of blast furnace ironmaking products, energy consumption levels, and process stability. Currently, green pellet size distribution measurement in industrial settings primarily relies on offline methods such as manual sieving, which suffers from low efficiency, high subjectivity, and delayed feedback, making it difficult to meet the demands of online real-time detection. Although machine vision-based particle size measurement methods have been gradually applied in industrial scenarios, in areas such as the stable zone of the disc pelletizer, conveyor belt, and discharge port, green pellets are often densely stacked, adhered to each other, and partially occluded. This easily leads to blurred boundaries and weakened textures of individual green pellets in the image, thus affecting instance separation accuracy. Traditional image segmentation methods are prone to oversegmentation or undersegmentation in these scenarios, while existing deep learning vision detection methods still struggle to simultaneously achieve positioning accuracy, segmentation accuracy, and processing efficiency in complex stacked scenarios, resulting in significant errors in particle size distribution measurement results.
[0048] To address the aforementioned problems, this invention proposes a deep learning-based visual measurement method for the particle size distribution of densely stacked green spheres. This method first utilizes the Green Sphere Detection Network (GPDNet) to detect green sphere instances in the test image, obtaining the bounding boxes of each instance. Then, spatial cue information is constructed based on these bounding boxes and combined with the original image input instance segmentation model to obtain the segmentation mask corresponding to each green sphere instance. Finally, edge contour point sets are extracted based on the segmentation mask and circle fitting is performed to obtain the diameter of each green sphere instance, thereby generating the green sphere particle size distribution result. This technical solution helps to improve the boundary blurring and segmentation difficulties caused by adhesion and occlusion of densely stacked green spheres, improving the accuracy and robustness of visual particle size distribution measurement.
[0049] like Figure 1 As shown, the visual measurement method for the particle size distribution of densely stacked green pellets based on deep learning provided in this embodiment of the invention includes at least the following steps:
[0050] S101. Obtain the raw ball image of the workstation to be tested.
[0051] Specifically, in industrial production sites, image acquisition devices can be deployed at key stations on the green pellet production line to acquire images of the green pellets at the stations under test. The stations under test may include at least one of the following: the stabilization zone of the disc pelletizing machine, the conveyor belt, and the discharge port. By acquiring images at different stations, the imaging states of green pellets under different scenarios, such as dense stacking, adhesion and occlusion, and relatively sparse distribution, can be covered. In one embodiment, the image acquisition device is an industrial camera, mounted above the area to be tested, with the lens optical axis pointing towards the area of the green pellets being tested. To reduce the impact of shadows, reflections, and ambient light fluctuations on image quality, supplementary lighting sources can be further configured to improve the clarity and grayscale consistency of the green pellet boundaries in the image. The acquired images can be RGB color images, or grayscale images or other data formats suitable for subsequent processing can be selected according to actual needs. In some embodiments, the acquired raw images can also be preprocessed, including but not limited to noise reduction, contrast enhancement, brightness equalization, and image cropping, to improve the accuracy and stability of subsequent detection and segmentation processing. This invention does not impose specific limitations on image acquisition resolution, triggering method, and preprocessing algorithm, as long as they can meet the requirements of subsequent sphere detection and instance segmentation.
[0052] S102. Input the raw ball image into the trained raw ball target detection network to obtain the bounding box of each raw ball instance in the raw ball image.
[0053] During the training phase, the raw ball target detection network GPDNet is trained under supervised supervision based on raw ball image data with bounding box annotations to learn the location features, scale features, and target representation features in densely stacked scenes. During the inference phase, raw ball images acquired at the workstation to be tested are input into the raw ball target detection network GPDNet, which outputs the bounding box information corresponding to each raw ball instance in the image. The bounding box information may include the target's location coordinates, scale information, and corresponding confidence score.
[0054] In one implementation, please refer to Figure 2 The raw sphere target detection network GPDNet comprises a backbone network, a neck network, and a detection head connected in sequence. The backbone network is used to extract multi-scale features from the raw sphere image; the neck network is used to align, fuse, and enhance features at different levels; and the detection head is used to output bounding boxes for each raw sphere instance based on the enhanced features.
[0055] Furthermore, to improve detection accuracy in complex industrial scenarios with dense stacking, adhesion and occlusion, and large scale variations, GPDNet sets up a C3K2_MAB module in the backbone network, an HPFA module and an EG-SDI module in the neck network, and an LSKA module in front of the detection head to enhance the network's ability to model multi-scale targets, edge detail information and global context information, thereby improving the accuracy of raw target detection and localization.
[0056] In one implementation, the backbone network includes a C3K2_MAB module; please refer to [link / reference]. Figure 3The C3K2_MAB module includes a grouped multi-scale large kernel attention unit and a spatially gated attention unit. The C3K2_MAB module enhances the network's ability to represent raw targets at different scales, edge details, and contextual relationships during backbone feature extraction, thereby improving target detection accuracy in complex industrial scenarios. Specifically, the grouped multi-scale large kernel attention unit performs multi-scale modeling of input features through grouped parallel processing, combined with large kernel convolution and dilated convolution, to capture richer contextual information and long-range dependencies, thus enhancing the network's ability to perceive raw targets at different scales and their spatial distribution relationships. Compared to traditional local convolution operations, the grouped multi-scale large kernel attention unit can better balance local details and global semantic information, improving adaptability to densely stacked, partially occluded, and scale-variable scenarios. The spatially gated attention unit adaptively adjusts the spatial response in the feature map, highlighting salient regions related to raw targets, enhancing target edge contours and local structural features, while suppressing irrelevant interference information such as background texture, equipment structure, and lighting changes. In scenarios involving dense stacking and overlapping occlusion, the spatially gated attention unit further enhances the network's ability to identify adjacent raw object boundary regions, thereby improving the discrimination effect of raw object targets. Through the above configuration, the C3K2_MAB module can achieve multi-scale contextual information modeling and spatial saliency enhancement in the backbone network stage, thus improving the effectiveness of raw object target feature extraction in complex industrial scenarios and providing a better feature foundation for subsequent feature fusion and bounding box prediction.
[0057] In one implementation, the neck network includes a hierarchical, parameter-free feature alignment module (HPFA). See [link to relevant documentation]. Figure 4 Let the shallow, mid-level, and deep features output by the backbone network be respectively... X P3 , X P4 and X P5 Among them, the middle layer feature X P4 As a spatial anchor point, the fused feature after alignment by the HPFA module can be expressed as:
[0058]
[0059] Here, Pool represents adaptive pooling, Up represents nearest neighbor interpolation upsampling, and Concat represents channel-dimensional concatenation. The HPFA module aligns features from different levels to the same spatial resolution before fusion. Shallow features contain rich edge textures and local details, deep features contain strong semantic representations, and mid-level features offer a good balance between spatial resolution and semantic expression; therefore, mid-level features are selected. X P4 As an alignment reference, it is beneficial to balance the preservation of feature details with high-level semantic fusion. Furthermore, the HPFA module does not introduce additional learnable parameters; instead, it achieves unified scale alignment of cross-layer features through pooling and upsampling operations, thereby reducing the computational overhead and feature shift risk caused by complex parameter mapping. Through this approach, while maintaining consistency in the feature space, it is possible to effectively aggregate shallow detail information, mid-level structural information, and deep semantic information, improving the multi-scale feature fusion effect and providing a more complete and stable feature representation for subsequent bounding box prediction by the detection head.
[0060] Furthermore, the neck network also includes an entropy-guided semantic detail interaction module (EG-SDI), see [link to relevant documentation]. Figure 5 The EG-SDI module is used for adaptive fusion of aligned shallow and deep features. Let the two aligned features be... X 1′ and X 2′, then the fusion weights can be generated based on the information entropy of the features, and weighted fusion can be performed. The output features can be expressed as:
[0061]
[0062] in, w k This represents the fusion weights generated based on feature information entropy. Shallow features typically contain richer edge texture information and local detail information, while deep features typically contain stronger semantic representation information and overall target structure information. Through the EG-SDI module, the contribution of each branch in the fusion process can be adaptively adjusted according to the differences in the amount of information carried by different feature branches, thereby achieving synergistic enhancement of detail information and semantic information. Furthermore, the EG-SDI module utilizes feature information entropy to characterize the effective information distribution of different branch features, enabling the fusion weights to dynamically change with the input feature content, rather than simply being superimposed at a fixed ratio. Through this approach, the feature flow rich in edge detail information can be enhanced, the interference of background noise and invalid responses on the feature fusion process can be suppressed, and the information interaction capability between shallow and deep features can be improved, thereby enhancing the feature representation effect and subsequent detection accuracy of spherical targets in complex scenes.
[0063] In one embodiment, a large-nucleus separable attention module (LSKA) is disposed in front of the detection head; see [link to relevant documentation]. Figure 6 Let its input features be... F in The LSKA module generates a spatial attention map through depthwise convolution and dilated convolution, and then weights and enhances the input features. Its output features can be represented as:
[0064]
[0065]
[0066] Where DWConv represents a depthwise convolution operation, DilatedDWConv represents a dilated depthwise convolution operation, and Conv represents a convolution mapping operation. This represents element-wise multiplication. The LSKA module enhances the representation of features for large-scale stacking patterns and global contextual information. Furthermore, the deep convolution operation extracts local spatial information from the input features, and the dilated deep convolution operation expands the receptive field while maintaining relatively controllable parameter levels, thereby enhancing the network's representation of large-scale stacking patterns, relative positional relationships between targets, and overall regional distribution features. The spatial attention map generated after convolution mapping highlights key response regions in the input features and suppresses irrelevant background regions, improving the effectiveness of feature representation. Through these methods, the LSKA module adaptively enhances the features of the input detection head, allowing features to retain local details while also considering richer global contextual information, thus improving the detection accuracy of spherical targets in complex industrial scenarios, especially beneficial for improving target localization in situations of dense stacking, partial occlusion, and boundary interference.
[0067] Through the above structural design, the raw sphere target detection network GPDNet can perform high-precision detection and localization of raw sphere targets in complex industrial scenarios with dense stacking, adhesion and occlusion, and large scale changes, thereby outputting the bounding boxes of each raw sphere instance, providing a foundation for subsequent spatial cue construction and instance segmentation.
[0068] S103. Construct spatial cue information from the bounding boxes of each raw ball instance, and input the spatial cue information and the raw ball image into the trained instance segmentation model to obtain the segmentation mask corresponding to each raw ball instance.
[0069] Specifically, after obtaining the bounding boxes of each live ball instance, these bounding boxes are used to construct spatial cue information, which provides spatial position constraints for the corresponding live ball instances in subsequent instance segmentation models. Due to issues such as mutual adhesion, occlusion, and blurred boundaries among live balls in densely stacked scenes, directly performing instance segmentation on the entire image can easily lead to insufficient separation of adjacent live balls or missegmentation of the background region. Therefore, in this embodiment, the bounding boxes output by the object detection network are used as spatial cue information to first coarsely locate the region where a single live ball instance is located, and then guide the instance segmentation model to perform fine segmentation on the corresponding region, thereby improving the segmentation accuracy of a single live ball instance.
[0070] In one implementation, the spatial cue information is constructed from the coordinate information of the bounding boxes of each raw sphere instance, and is input together with the raw sphere image into an instance segmentation model to guide the instance segmentation model to perform pixel-level segmentation on the corresponding raw sphere instances and output a segmentation mask for each raw sphere instance. The segmentation mask is used to characterize the pixel-level region range of each raw sphere instance in the image, providing a basis for subsequent edge contour extraction and diameter fitting.
[0071] In one implementation, the step of constructing corresponding spatial cue information based on each bounding box and inputting the spatial cue information and the raw ball image into an instance segmentation model to obtain a segmentation mask corresponding to each raw ball instance includes: inputting the bounding box into a cue encoder to generate cue features; inputting the raw ball image into an image encoder to generate image features; and inputting the cue features and the image features into a mask decoder to output a segmentation mask corresponding to each raw ball instance.
[0072] Furthermore, let the bounding box hints for each live ball instance be... B Input raw ball image is I The corresponding segmentation mask M It can be represented as:
[0073]
[0074] in, E p This indicates a prompt from the encoder. E i Indicates an image encoder. D This represents the mask decoder. Using the above method, the instance segmentation model can focus on the corresponding raw sphere instance region under bounding box cue constraints, achieving fine segmentation of individual raw sphere instances in densely stacked scenes. Compared to directly segmenting the entire image, this method effectively reduces segmentation interference between adjacent, adhered raw spheres, improving the boundary accuracy and instance integrity of the segmentation mask.
[0075] S104. Extract the edge contour point set based on the segmentation mask of each raw ball instance, and perform circle fitting based on the edge contour point set to obtain the diameter of each raw ball instance.
[0076] Specifically, after obtaining the segmentation mask corresponding to each green ball instance, the edge contour point set of each green ball instance can be extracted based on the segmentation mask. Since the green ball is approximately circular overall, circle fitting can be performed based on the edge contour point set to obtain the geometric dimension parameters of each green ball instance, and its diameter can be further calculated.
[0077] In one implementation, the step of extracting edge contour point sets based on the segmentation mask of each raw ball instance and performing circle fitting based on the edge contour point sets to obtain the diameter of each raw ball instance includes: extracting edge pixels of the segmentation mask of each raw ball instance to form an edge contour point set; solving for the center and radius of the fitted circle with the objective of minimizing the sum of squared distances from each point in the edge contour point set to the fitted circle; and calculating the diameter of each raw ball instance based on the radius.
[0078] Specifically, let the edge contour corresponding to a certain raw ball instance be:
[0079]
[0080] The fitted circle has its center at (a, b) and its radius is... r Then, the objective function can be established using the least squares criterion as follows:
[0081]
[0082] By solving the above objective function, the center and radius of the fitted circle can be obtained. Furthermore, the diameter of each raw ball instance... d It can be represented as:
[0083] d=2r
[0084] In practical applications, the diameter in the image coordinate system can also be converted to the actual physical size based on the image calibration parameters; this invention does not limit this. By extracting the edge contour of the segmentation mask and performing circle fitting, the near-circular geometric features of green pellets can be better utilized, reducing the influence of local boundary noise on the particle size measurement results, thereby improving the accuracy and stability of diameter measurement for individual green pellet instances.
[0085] S105. Perform statistical analysis based on the diameter of all green pellet instances to generate green pellet size distribution results.
[0086] Specifically, after obtaining the diameter of each green pellet instance, the diameter data can be further statistically analyzed to form the green pellet particle size distribution result corresponding to the test station. The particle size distribution result can be used to reflect the particle size composition of the green pellet product at the current station, providing data support for process adjustment, quality assessment, and online monitoring.
[0087] In one embodiment, the diameters of each green pellet instance can be grouped and statistically analyzed according to a preset particle size range to obtain the quantity distribution, proportion distribution, or frequency distribution of green pellet instances within each particle size range. The preset particle size range can be set according to actual production needs, and the present invention does not specifically limit it.
[0088] In some implementations, the particle size distribution results can be further output to a display terminal, storage device, or production control system to achieve visualization of the particle size distribution results, storage of historical data, and optimization and adjustment of process parameters.
[0089] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating the target detection, instance segmentation, and particle size distribution results of a densely stacked sphere image in an embodiment of the present invention. Figure 7 As shown, target detection is performed on the densely stacked green ball image to obtain the bounding box of each green ball instance; spatial cue information is constructed based on the bounding box, and the spatial cue information is input into the instance segmentation model to obtain the segmentation mask corresponding to each green ball instance; then, based on the segmentation results, particle size measurement and statistical analysis are performed to generate the green ball particle size distribution results.
[0090] Through the above steps, the embodiments of the present invention complete the entire processing flow from raw ball image acquisition, raw ball instance detection, prompt-guided segmentation, contour fitting diameter measurement to particle size distribution statistics. It can better adapt to industrial scenarios with dense stacking, adhesion and occlusion, and large scale changes, and achieve high-precision visual measurement of raw ball particle size distribution.
[0091] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a deep learning-based visual measurement device for the particle size distribution of densely stacked green balls, provided in an embodiment of the present invention.
[0092] like Figure 8 As shown, the deep learning-based visual measurement device 800 for densely stacked spherical particle size distribution includes: an image acquisition unit 801, a target detection unit 802, a cue construction and segmentation unit 803, a contour fitting diameter measurement unit 804, and a particle size distribution generation unit 805.
[0093] The image acquisition unit 801 is used to acquire images of green pellets at the workstation to be tested. The workstation to be tested can be the feed inlet, conveyor belt, pelletizing tray, or other workstations requiring green pellet size distribution detection during the pellet production process; this invention does not specifically limit this. The green pellet image acquired by the image acquisition unit 801 can be a single frame image or an image frame from a video stream.
[0094] The target detection unit 802 is connected to the image acquisition unit 801 and is used to input the raw ball image into a trained raw ball target detection network to obtain the bounding boxes of each raw ball instance in the raw ball image. In one embodiment, the target detection unit 802 may use the raw ball target detection network GPDNet in the aforementioned embodiments to achieve effective localization of each raw ball instance in a densely stacked scene.
[0095] The cue construction and segmentation unit 803 is connected to the target detection unit 802 and is used to construct spatial cue information based on the bounding box. The spatial cue information and the raw object image are then input into a trained instance segmentation model to obtain a segmentation mask corresponding to each raw object instance. Specifically, the cue construction and segmentation unit 803 can construct spatial cue information based on bounding box coordinate information and use this information to guide the instance segmentation model to perform pixel-level segmentation of the corresponding raw object instances, thereby improving the accuracy of instance segmentation in densely stacked and occluded scenes.
[0096] The contour fitting diameter measurement unit 804 is connected to the prompt construction and segmentation unit 803, and is used to extract the edge contour point set according to the segmentation mask of each raw ball instance, and perform circle fitting based on the edge contour point set to obtain the diameter of each raw ball instance.
[0097] The particle size distribution generation unit 805 is connected to the contour fitting diameter measurement unit 804 and is used to perform statistical analysis on the diameter of each green pellet instance to generate a green pellet particle size distribution result. In one embodiment, the particle size distribution generation unit 805 can group and statistically analyze the diameter of each green pellet instance according to a preset particle size interval to obtain the quantity distribution, proportion distribution, or frequency distribution of green pellet instances within each particle size interval, and can output the particle size distribution result to a display terminal, storage device, or production control system.
[0098] In this embodiment of the invention, the image acquisition unit 801, the target detection unit 802, the prompt construction and segmentation unit 803, the contour fitting and diameter measurement unit 804, and the particle size distribution generation unit 805 cooperate sequentially to realize a complete processing flow from raw sphere image acquisition, raw sphere target detection, prompt-guided segmentation, contour fitting and diameter measurement to particle size distribution generation. The specific implementation methods and technical effects of the above units can be referred to the relevant descriptions in the foregoing method embodiments, and will not be repeated here.
[0099] In one embodiment, the present invention also provides a computer-readable storage medium.
[0100] The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the deep learning-based visual measurement method for the size distribution of densely stacked green balls as described in any of the foregoing embodiments. The computer-readable storage medium may be a read-only memory, random access memory, disk, optical disk, flash memory, solid-state drive, or other tangible storage medium capable of storing program code; the present invention does not specifically limit this.
[0101] In one embodiment, the present invention also provides an electronic device.
[0102] The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the deep learning-based visual measurement method for the size distribution of densely stacked green balls as described in any of the foregoing embodiments.
[0103] In one embodiment, the electronic device may further include an image input interface, a display interface, and a communication interface. The image input interface is used to receive images of green pellets from the workstation under test; the display interface is used to output detection results, segmentation results, and particle size distribution results; and the communication interface is used to interact with external storage devices, display terminals, or production control systems.
[0104] It should be understood that the aforementioned electronic devices are merely illustrative examples and do not constitute a limitation on the scope of protection of this invention.
Claims
1. A deep learning-based visual measurement method for the particle size distribution of densely stacked green balls, characterized in that, include: Obtain the raw ball image of the workstation to be tested; The raw ball image is input into a trained raw ball target detection network to obtain the bounding boxes of each raw ball instance in the raw ball image. The raw ball target detection network is GPDNet, which includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network includes a C3K2_MAB module, the neck network includes an HPFA module and an EG-SDI module, and an LSKA module is set in front of the detection head. The backbone network is used to extract multi-scale features from the raw ball image, the neck network is used to align, fuse, and enhance the multi-scale features, and the detection head is used to output the bounding boxes of each raw ball instance based on the enhanced features. The backbone network of GPDNet includes a C3K2_MAB module, which includes a grouped multi-scale large kernel attention unit and a spatially gated attention unit. The grouped multi-scale large kernel attention unit is used to extract multi-scale features and establish long-distance dependencies through grouped parallel processing, combined with large kernel convolution and dilated convolution; The spatial gated attention unit is used to perform gated weighting on features, enhance the edge features of the live sphere and suppress background interference, so as to output enhanced multi-scale features; The neck network includes a hierarchical parameterless feature alignment module (HPFA). The HPFA module uses a mesoscale feature layer as a spatial anchor point and aligns features at different levels to the same spatial resolution through a parallel alignment strategy. Specifically, high-resolution features are downsampled to the resolution corresponding to the spatial anchor point through adaptive pooling, low-resolution features are upsampled to the resolution corresponding to the spatial anchor point through nearest neighbor interpolation, mesoscale features maintain an identity mapping, and the aligned features of each layer are spliced and fused in the channel dimension. The neck network includes a semantic and detail interaction module (EG-SDI), which is used to adaptively fuse aligned shallow and deep features. The EG-SDI module generates fusion weights based on the information entropy of the features and performs weighted fusion of features at different levels according to the fusion weights to enhance the edge detail features of the spherical body and suppress background interference. The detection head is equipped with a large kernel separable attention module (LSKA). The LSKA module includes a depth convolution decomposition unit, a dilated convolution decomposition unit, and an attention fusion unit. The depth convolution decomposition unit is used to extract local texture and edge features through depth convolution in the horizontal and vertical directions. The dilated convolution decomposition unit is used to extract global context features through dilated depth convolution in the horizontal and vertical directions. The attention fusion unit is used to generate a spatial attention map and perform weighted enhancement on the input features based on the spatial attention map. Based on each bounding box, construct corresponding spatial cue information, and input the spatial cue information and the raw sphere image into the instance segmentation model to obtain the segmentation mask corresponding to each raw sphere instance; The edge contour point set is extracted based on the segmentation mask of each raw ball instance, and a circle is fitted based on the edge contour point set to obtain the diameter of each raw ball instance. The diameter of each green bulb instance is statistically analyzed to generate the green bulb size distribution results.
2. The deep learning-based visual measurement method for the particle size distribution of densely stacked green balls according to claim 1, characterized in that, The spatial cue information is a bounding box cue information constructed based on the bounding boxes of each raw sphere instance. The bounding box cue information and the raw sphere image are input together into the instance segmentation model to guide the instance segmentation model to perform pixel-level segmentation on the corresponding raw sphere instance and output the segmentation mask corresponding to each raw sphere instance.
3. The deep learning-based visual measurement method for the particle size distribution of densely stacked green balls according to claim 2, characterized in that, The step of constructing corresponding spatial cue information based on each bounding box and inputting the spatial cue information and the raw sphere image into an instance segmentation model to obtain a segmentation mask corresponding to each raw sphere instance includes: inputting the bounding box into a cue encoder to generate cue features; The raw ball image is input into an image encoder to generate image features; The prompt features and the image features are input into the mask decoder to output the segmentation mask corresponding to each raw ball instance.
4. The deep learning-based visual measurement method for the particle size distribution of densely stacked green balls according to claim 1, characterized in that, The process of extracting edge contour point sets based on the segmentation mask of each raw ball instance and performing circle fitting based on the edge contour point sets to obtain the diameter of each raw ball instance includes: extracting edge pixels of the segmentation mask of each raw ball instance to form an edge contour point set; using the least squares circle fitting method to solve for the center and radius of the fitted circle based on the edge contour point set; and calculating the diameter of each raw ball instance based on the radius.
5. A deep learning-based visual measurement device for the particle size distribution of densely stacked green pellets, characterized in that, include: The image acquisition unit is used to acquire images of the raw balls at the workstation under test. An object detection unit is used to input the raw ball image into a trained raw ball object detection network to obtain the bounding boxes of each raw ball instance in the raw ball image. The raw ball object detection network is GPDNet, which includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network includes a C3K2_MAB module, the neck network includes an HPFA module and an EG-SDI module, and an LSKA module is set in front of the detection head. The backbone network is used to extract multi-scale features from the raw ball image, the neck network is used to align, fuse, and enhance the multi-scale features, and the detection head is used to output the bounding boxes of each raw ball instance based on the enhanced features. The backbone network of GPDNet includes a C3K2_MAB module, which includes a grouped multi-scale large kernel attention unit and a spatially gated attention unit. The grouped multi-scale large kernel attention unit is used to extract multi-scale features and establish long-distance dependencies through grouped parallel processing, combined with large kernel convolution and dilated convolution; The spatial gated attention unit is used to perform gated weighting on features, enhance the edge features of the live sphere and suppress background interference, so as to output enhanced multi-scale features; The neck network includes a hierarchical parameterless feature alignment module (HPFA). The HPFA module uses a mesoscale feature layer as a spatial anchor point and aligns features at different levels to the same spatial resolution through a parallel alignment strategy. Specifically, high-resolution features are downsampled to the resolution corresponding to the spatial anchor point through adaptive pooling, low-resolution features are upsampled to the resolution corresponding to the spatial anchor point through nearest neighbor interpolation, mesoscale features maintain an identity mapping, and the aligned features of each layer are spliced and fused in the channel dimension. The neck network includes a semantic and detail interaction module (EG-SDI), which is used to adaptively fuse aligned shallow and deep features. The EG-SDI module generates fusion weights based on the information entropy of the features and performs weighted fusion of features at different levels according to the fusion weights to enhance the edge detail features of the spherical body and suppress background interference. The detection head is equipped with a large kernel separable attention module (LSKA). The LSKA module includes a depth convolution decomposition unit, a dilated convolution decomposition unit, and an attention fusion unit. The depth convolution decomposition unit is used to extract local texture and edge features through depth convolution in the horizontal and vertical directions. The dilated convolution decomposition unit is used to extract global context features through dilated depth convolution in the horizontal and vertical directions. The attention fusion unit is used to generate a spatial attention map and perform weighted enhancement on the input features based on the spatial attention map. The cue construction and segmentation unit is used to construct spatial cue information based on the bounding box, and input the spatial cue information and the raw ball image into a trained instance segmentation model to obtain the segmentation mask corresponding to each raw ball instance; The contour fitting diameter measurement unit is used to extract the edge contour point set according to the segmentation mask of each raw ball instance, and to fit the diameter of each raw ball instance based on the edge contour point set. The particle size distribution generation unit is used to generate the particle size distribution results of green pellets based on the diameter of each green pellet instance.