Industrial particle detection and particle size measurement method, system, electronic device and readable medium

Abstract

The industrial particle detection and particle size measurement method, system, electronic equipment and readable medium belong to the technical fields of industrial machine vision, target detection, feature recovery and non-contact size measurement. The first detection can obtain high recall candidates through adaptive lightweight pre-enhancement, the local target density state is generated through the first detection result and the original dense target response, the visible reliability state is generated through candidate overlap, contour support and shape consistency, the low light and blur degradation state is generated through the original image brightness and definition; the states are fused into a spatial control chart for modulating low light and deblurring feature recovery residual, thereby forming a closed loop of "initial detection - explicit state calculation - selective feature recovery - re-detection - reliable measurement".

Description

Technical Field

[0001] This invention relates to the fields of industrial machine vision, target detection, feature recovery, and non-contact size measurement technology. Specifically, it relates to a method and system for generating spatial control information based on explicit target state and image degradation state, selectively modulating the visual feature recovery process using the spatial control information, and further completing industrial particle detection and particle size measurement. The industrial particles include, but are not limited to, pellets, green pellets, ore particles, coal lumps, tablets, food particles, and other discrete targets that can be acquired and sized by an industrial camera. Background Technology

[0002] Particle detection and size measurement on industrial conveyor belts are crucial steps in mineral processing, pellet production, food sorting, and pharmaceutical manufacturing. Particle quantity, size distribution, and stacking status reflect the operating status of upstream processing equipment and can be used to adjust feed rate, roller pressing parameters, screening parameters, and subsequent process parameters. Compared to offline screening and manual measurement, non-contact vision measurement based on industrial cameras can continuously acquire production line information, offering advantages such as fast response time, ease of deployment, and no disruption to the production process.

[0003] In existing technologies, one approach uses traditional image processing methods such as grayscale conversion, median filtering, binarization, Canny edge detection, distance transformation, morphological processing, and watershed segmentation to obtain particle boundaries, and then calculates the particle size based on the particle's projected area, perimeter, fitted circle, or fitted ellipse. This type of method can obtain relatively accurate edges under conditions of stable illumination, clear target boundaries, and good particle separation. However, it is prone to threshold failure, oversegmentation, and undersegmentation under conditions of low illumination, motion blur, dust interference, and dense particle adhesion.

[0004] Another approach employs deep learning networks such as YOLO, Faster R-CNN, Mask R-CNN, and U-Net for object detection, semantic segmentation, or instance segmentation. YOLO networks are characterized by their fast single-stage inference speed, Mask R-CNN can simultaneously output candidate boxes and instance masks, and U-Net preserves spatial details through skip connections between the encoder and decoder. While deep learning methods are robust to background changes, their detection or segmentation results may still experience missed detections, merging of adjacent objects, and contour shifts under conditions of extremely low light, motion blur, and severe occlusion.

[0005] To improve detection performance under low-light or blurred conditions, existing methods typically add Gamma correction, CLAHE, Retinex, deblurring networks, or low-light enhancement networks before the detection network to uniformly enhance the entire image before inputting the enhanced image into the detection network. This approach has at least the following drawbacks: First, there is a lack of feedback between the enhancement process and the target detection results; the background and non-target areas are also enhanced simultaneously, potentially amplifying conveyor belt textures, dust, and noise. Second, the same enhancement strategy cannot simultaneously adapt to two types of degradation with different physical causes: low light and motion blur. Third, improvements in image visual quality do not necessarily correspond to improvements in target detection discrimination features. Fourth, even if a severely occluded target is detected, reliable particle size measurement conditions are not guaranteed.

[0006] There are also schemes that use attention mechanisms to weight feature maps, but ordinary channel attention or spatial attention is usually directly generated by neural network features, making it difficult to clearly explain the relationship between control weights and target density, visibility, low light level and blur level; at the same time, ordinary attention mechanisms often only perform one detection, without forming a processing link that reverses the initial detection result to determine the recovery region and detect it again.

[0007] Therefore, there is a need for an industrial vision method that can reliably generate initial candidates under low illumination and motion blur conditions, explicitly calculate the target spatial state and image degradation state based on the initial candidates and the original image, restore the position and restoration intensity using the explicit state control features, and re-detect and judge measurement eligibility after restoration. Summary of the Invention

[0008] The purpose of this invention is to overcome the problems in the prior art, such as easy failure of initial detection in low light and blurred scenes, easy amplification of background noise by uniform enhancement of the whole image, difficulty in interpreting black box attention, mutual interference between different degradation types, and unreliable measurement of target particle size due to severe occlusion. The invention provides an industrial particle detection and particle size measurement method and system based on explicit state control and selective feature recovery.

[0009] This invention ensures high recall candidates in the first detection by adaptive lightweight pre-enhancement, generates a local target density state based on the first detection result and the original dense target response, generates a visible reliability state based on candidate overlap, contour support and shape consistency, and generates a low-light and blurred degradation state based on the original image brightness and sharpness. The states are then fused into a spatial control map to modulate the low-light and deblurred feature recovery residuals, thus forming a closed loop of "initial detection - explicit state calculation - selective feature recovery - re-detection - reliable measurement".

[0010] To achieve the above objectives, the present invention adopts the following technical solution: Industrial particle detection and particle size measurement methods include the following steps: S1: Image Acquisition and Adaptive Lightweight Pre-Augmentation: Acquiring raw images of industrial particles on a conveyor belt using an industrial camera. According to the original image The local average brightness is used to calculate low-light trigger information, and the original image is processed according to the low-light trigger information. Adaptive Gamma correction and contrast-limited adaptive histogram equalization (CLAHE) are performed on the image. The results are then spatially soft-fused with the original image to obtain the pre-enhanced image. ; S2: Multi-scale feature extraction and first coarse detection: pre-enhanced image... The input feature extraction backbone network is used to obtain at least two basic visual features at different resolutions. These basic visual features are then input into a shared object detection network for the first detection, yielding initial detection results including candidate bounding boxes, candidate confidence scores, and candidate center locations. And retain the original dense target response output by the shared target detection network before candidate selection and nonmaximum suppression. ; S3: Calculation of target spatial state and image degradation state: based on initial detection results Original dense target response and the original image The target spatial state and image degradation state are explicitly calculated, wherein the target spatial state includes at least the local target density state. and target visible reliability status The image degradation state includes at least a low-light degradation state. and fuzzy degenerate state ; S4: Spatial Control Map Generation: Based on Local Target Density Status Target visibility reliability status and low light degradation state Generate low-light recovery spatial control map Based on the local target density state Target visibility reliability status and fuzzy degenerate state Generate deblurred recovery spatial control map and low-light recovery spatial control map Deblurring and recovery of spatial control chart Each feature is resampled to the spatial resolution corresponding to its basic visual features. S5: Dual-path selective feature recovery: Each basic visual feature is input into the low-light feature recovery branch and the deblurring feature recovery branch respectively to obtain the low-light recovery feature residual and the deblurring recovery feature residual; using the low-light recovery spatial control map of the corresponding spatial resolution... Deblurring and recovery of spatial control chart Spatially modulate the low-light recovery feature residual and the deblurred recovery feature residual respectively, and inject the modulated recovery feature residual into the corresponding basic visual feature in a bounded addition manner to obtain the recovered visual feature; S6: Second fine-tuning detection: The recovered visual features are input again into the shared target detection network to obtain the final detection result of industrial particles. The final detection result At least the location of the particle target and the detection confidence level should be included; S7: Based on the final test results In the original image The visible contours of the particle targets are extracted, and the pixel size and contour fitting residuals of the particle targets are obtained by using circle fitting or ellipse fitting. The measurement qualification score is calculated based on the detection confidence, final visibility reliability, contour fitting residuals and the blurring degree of the target area. When the measurement qualification score reaches the preset threshold, the pixel size is converted into the actual particle size according to the camera calibration parameters. When the measurement qualification score is lower than the preset threshold, the target detection result is retained and the corresponding particle target is marked as unreliable for measurement.

[0011] Further, step S1 includes the following sub-steps: S1.1: Convert the original image Convert to a color space with a luminance component, and calculate the local average luminance within a neighborhood window for each spatial location. ; S1.2: According to Generate low-light trigger map ,in To preset the brightness threshold, To prevent positive numbers with a denominator of zero, ; S1.3: Based on the low-light trigger map The global or partition mean is used to determine the Gamma correction parameters, as well as the CLAHE limiting parameters and grid size, for the original image. Perform Gamma correction and CLAHE processing to obtain candidate enhanced images. ; S1.4: According to Generate pre-enhanced image This allows the original image information to be preserved in areas of normal brightness, while providing limited enhancement to low-light areas; the same pre-enhancement rules are used in both the training and inference phases.

[0012] The Gamma correction and CLAHE are preferred implementation methods. Adaptive pre-enhancement can also employ Retinex, homomorphic filtering, local contrast stretching, or a combination thereof.

[0013] Furthermore, in step S2, the feature extraction backbone network preferably adopts MobileNetV4, ResNet, EfficientNet, ConvNeXt, Swin Transformer, or other networks with multi-scale feature output. When MobileNetV4 is adopted, MobileNetV4 outputs downsampled features of 8x, 16x, and 32x. The features of different downsampled factors are upsampled, downsampled, concatenated, or additively fused through the feature pyramid network FPN and the path aggregation network PAN to obtain fused features. The fused features are then input into the subsequent shared target detection network. The shared object detection network preferably uses the decoupled detection head of YOLOv8 or YOLO11, Faster R-CNN or other object detection networks. When using the decoupled detection head of YOLOv8 or YOLO11, it outputs the class confidence, object presence response and bounding box regression, respectively.

[0014] Furthermore, the first detection in step S2 employs a high-recall candidate generation method, including: using a candidate confidence threshold lower than that of the final detection stage, and employing Soft-NMS (Soft Non-Maximum Suppression), DIoU-NMS (Distance Intersection over Union Non-Maximum Suppression), or a non-maximum suppression method with a high overlap threshold to retain densely adjacent targets, and retaining the original dense target response before candidate screening. When no candidate bounding boxes meet the candidate confidence threshold, the original dense target response is still used. Generate local target density state To prevent low-light recovery spatial control charts from being empty due to initial detection results. Deblurring and recovery of spatial control chart All are zero.

[0015] Furthermore, the local target density state in step S3 The calculations include: With the first Candidate Centers Determine the Gaussian kernel standard deviation centered on the candidate target scale. And based on candidate confidence As weights, the candidate box density state is obtained by superposition. ; Response to the original dense target Normalization and spatial resampling are performed to obtain the response density state. ; according to Fusion candidate box density state and response density state ,in The fusion coefficient is between 0 and 1.

[0016] Furthermore, in step S3, the target's visible reliability status... The calculations include: Calculate the first Asymmetric coverage ratio where each candidate bounding box is covered by other candidate bounding boxes in, Indicates the first The nth candidate bounding box, which is the nth output from the coarse detection stage. One target detection bounding box, Indicates the first The candidate bounding box is used to compare with the first candidate bounding box. Compare and calculate the candidate boxes; With the first The candidate region is divided into multiple angular intervals using candidate centers as poles. The proportion of angular intervals with effective contour support is determined based on the results of Canny edge detection, Sobel edge detection, or learned edge detection. ; Perform RANSAC circle or ellipse fitting on the candidate regions and obtain shape consistency based on the coarse fitting residuals. ; according to The target-level visible reliability is obtained, and the target-level visible reliability is mapped to the target visible reliability state diagram through a spatial kernel centered on the candidate center. ,in , and These are the weighting coefficients.

[0017] Furthermore, in step S3, the low-light degradation state Based on the original image The local average brightness calculation is preferably based on Obtain; among them Brightness threshold for low-light detection; blurring and degradation state Based on the original image The local sharpness is calculated using at least one of Laplacian response variance, Tenengrad gradient, frequency domain high-frequency energy, or edge spread width, preferably according to... Obtain, among which This represents the local sharpness value. The threshold for determining sharpness.

[0018] Furthermore, in step S4, first follow For local target density state Modulate, and then according to and Generate low-light recovery spatial control maps respectively Deblurring and recovery of spatial control chart ,in The basic restoration coefficient for sparse targets. and To control the gain coefficient; the fusion method of the spatial state is not limited to element-wise multiplication, but also includes weighted sum, logical combination, lookup table mapping or preset function mapping, as long as the generated spatial control information is related to both the target spatial state and the corresponding image degradation state.

[0019] Further, in step S5, the low-light feature recovery branch and the deblurring feature recovery branch adopt at least one of the following structures: lightweight U-Net structure, residual convolution structure, depthwise separable convolution structure, or Transformer recovery structure; preferably, the two recovery branches share the previous feature transformation layer and are respectively set with low-light recovery output head and deblurring recovery output head; the original output of the recovery branch is constrained by the tanh function to restore the residual amplitude, and the recovery injection coefficients are... Set as Then, the spatial control map modulates the corresponding recovery residuals and injects them into the basic features in an additive manner, so that the recovered visual features satisfy... = + ⊙ + ⊙ Where s represents the feature scale, For scale The corresponding basic feature map, For scale Learnable injection intensity coefficient of the lower light recovery branch. For scale The recovery residual output of the low-light recovery branch. For scale Learnable injection strength coefficients for the fuzzy recovery branch. scale The recovery residual of the output of the fuzzy recovery branch. and Broadcast along the channel dimension.

[0020] Furthermore, the training of the feature extraction backbone network, the shared object detection network, and the low-light feature recovery branch and the deblurring feature recovery branch includes: In the first stage, the feature extraction backbone network and the shared object detection network are pre-trained using training images that have undergone the same adaptive pre-enhancement processing as those in the inference stage. In the second stage, the feature extraction backbone network and the shared target detection network are frozen, and the low-light recovery branch and the deblurring recovery branch are trained using pairs of clear, low-light and blurred images; the recovery branch can also be trained using the second detection loss. In the third stage, the first detection is performed, and the initial detection results are used to calculate after a stopping gradient operation. , , , , and The system performs feature recovery and a second detection, and jointly trains the feature extraction backbone network, the shared object detection network, and the two recovery branches based on the first detection loss, the second detection loss, and the optional feature recovery consistency loss.

[0021] Furthermore, the total loss function is ,in The classification loss and bounding box regression loss are used for the first detection. The classification loss and bounding box regression loss are used for the second detection. To restore the relationship between visual features and clear image teacher features Loss, mean squared error loss, cosine consistency loss, or perception loss, at least one of these. , , These are the weight coefficients of each loss term; in the early stage of training, the density state and reliability state generated by the ground truth label are weighted and mixed with the density state and reliability state generated by the first detection result, and the weight of the ground truth state is gradually reduced to zero as training progresses.

[0022] Further, step S7 includes: determining the region of interest in the original image based on the final detection box; extracting a set of visible contour points using the Canny or Sobel operator; fitting a circular or elliptical geometric model using RANSAC, least squares, or Hough transform to obtain the center and radius of the circle, or the center, major axis, minor axis, and orientation angle of the ellipse; calculating the contour fitting residual based on the distance between the fitted model and the visible contour points; and calculating the measurement qualification score based on the final detection confidence, final visibility reliability, contour fitting residual, and average blurriness of the target region. ,in For the first The final detection confidence of each target, For the first The contour fitting residuals of each target This is the tolerance coefficient for the fitted residuals. The larger the value, the higher the tolerance for residuals. For the first The average blur level of each target region The attenuation coefficient is the factor for the degree of blur. The larger the value, the higher the tolerance for ambiguity.

[0023] Furthermore, the camera intrinsic parameters, distortion coefficients, rotation vectors, and translation vectors are calculated using the checkerboard calibration method or the Zhang Zhengyou camera calibration method. A scale mapping between image position and actual length is established using the conveyor belt planar homography or partitioned pixel equivalents. For round particles, according to Calculate the actual particle size, and for elliptical particles, calculate the equivalent circle diameter based on the major and minor axes; and count the number and proportion of particles in each preset particle size range to form the particle size distribution.

[0024] An industrial particle detection and particle size measurement system includes an image acquisition module, an adaptive pre-enhancement module, a feature extraction module, a shared target detection module, an explicit state calculation module, a spatial control map generation module, a dual-path feature recovery module, a contour fitting module, and a measurement qualification judgment module. Each module is configured to execute the above-mentioned methods. Among them, the shared target detection module performs a first detection on the basic visual features and a second detection on the recovered visual features output by the dual-path feature recovery module.

[0025] An electronic device includes an industrial camera interface, a processor, and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the electronic device to perform the method described above.

[0026] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0027] The beneficial effects of this invention are: 1. The present invention sets up a light pre-enhancement of Gamma and CLAHE that is adaptively triggered according to the low light level before the first detection, which can improve the availability of initial candidates under extremely low illumination conditions, while reducing unnecessary enhancement of normal areas through soft fusion.

[0028] 2. Based on the initial detection results, dense target response and original image, the present invention explicitly calculates density, visibility reliability, low light and blur state. The spatial control information has a clear physical meaning and is easier to interpret and adjust than the black box attention weights generated entirely by the network.

[0029] 3. This invention sets up independent spatial control maps and independent recovery residuals for low light and motion blur, avoiding the mixing of different physical degradations by the same enhancement branch.

[0030] 4. This invention only utilizes spatial control chart modulation to add recovery residuals, while preserving the basic features. It also limits the control chart, recovery residuals, and injection coefficients, thereby reducing the damage to the original discriminative features caused by erroneous recovery.

[0031] 5. This invention forms a functional closed loop of first detection, explicit state calculation, selective feature recovery and second detection, which is different from the traditional single detection after full image enhancement.

[0032] 6. This invention uses dense target responses as a supplement to candidate box density. When no candidate boxes reach the threshold, it can still provide weak guidance for subsequent recovery, reducing the risk that the system cannot start due to empty initial detection.

[0033] 7. This invention separates target detection and particle size measurement for quality control, outputting particle size only for targets that meet the requirements for visible reliability and geometric fitting quality, thereby reducing erroneous sizes caused by severely occluded targets.

[0034] 8. This invention can be implemented using MobileNetV4, YOLOv8, YOLO11, and lightweight U-Net, and is suitable for edge computing devices; the specific networks mentioned above are only examples, and this invention can also be implemented using other feature extraction, object detection, and feature recovery networks with the same functions. Attached Figure Description

[0035] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a system model structure diagram of the present invention; Figure 3 This is a flowchart of the adaptive lightweight pre-enhancement process of the present invention; Figure 4 This is a flowchart of the explicit state calculation process of the present invention; Figure 5 This is a schematic diagram of the dual-path selective feature recovery and bounded injection of the present invention; Figure 6 This is a schematic diagram of the joint training and gradient backpropagation of the present invention; Figure 7 This is a schematic diagram illustrating the visible contour fitting and measurement eligibility determination of the present invention; Figure 8 This is a schematic diagram of the installation of the industrial camera, conveyor belt, and edge computing device of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0037] It should be noted that, unless otherwise specified, the embodiments and technical features described in the embodiments of this application can be combined with each other. The names MobileNetV4, FPN, PAN, YOLOv8, YOLO11, U-Net, CLAHE, Canny, RANSAC, etc., used in the following embodiments are for clearly describing preferred implementations; those skilled in the art can use networks, algorithms, or equivalent substitutions with the same functionality, and the scope of protection of this invention is not limited by the specific network versions, number of layers, number of channels, thresholds, or parameter values ​​described below.

[0038] In this embodiment, pellets or green pellets on a conveyor belt are used as the detection object. An industrial camera is fixedly installed above the conveyor belt, and an edge computing device receives the camera image and outputs the target position, particle size, and measurement reliability. Figure 8 This illustrates one installation method for an industrial camera, conveyor belt, light source, edge computing device, and display terminal.

[0039] Example 1: Dataset Construction and Basic Detection Network Step 1.1: Acquire raw images of industrial particles under different illumination levels, conveyor belt speeds, particle densities, and degrees of motion blur. For continuous videos, group them according to acquisition date, production batch, and continuous time period to avoid adjacent video frames from simultaneously entering the training and test sets.

[0040] Step 1.2: Label the identifiable particle targets using LabelImg, CVAT, or Labelme. For particles with complete outlines or whose center positions can be stably estimated, label and estimate the complete bounding box; for severely occluded regions with only extremely short arcs and indeterminate independent target centers, do not consider them as complete particle positive samples. The visibility ratio, measurability, and true particle size can be recorded in an attached JSON or CSV file.

[0041] Step 1.3: Divide the data acquisition group into training, validation, and test sets. Perform random data augmentation on the training set, including brightness, saturation, slight scaling, translation, mosaic stitching, and slight motion blur. It should be noted that the adaptive Gamma and CLAHE pre-augmentation modules used during training are always consistent with those used during inference, and the random data augmentation occurs before the adaptive pre-augmentation.

[0042] Step 1.4: Initialize the network using YOLOv8n or YOLO11n weights pre-trained with COCO. The backbone network can directly use the default YOLO backbone, or it can be replaced with MobileNetV4. In this embodiment, MobileNetV4 outputs P3, P4, and P5 features, and uses the Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) for multi-scale fusion. YOLO is used to decouple the detection head outputting the category, target presence response, and bounding box regression.

[0043] Step 1.5: In the first stage, only the backbone, feature fusion network, and shared detection head are trained to achieve a high recall rate for the first detection under low threshold conditions. The final particle size is not required in the first detection stage, nor is spatially selective recovery performed.

[0044] Example 2: Adaptive Lightweight Pre-Enhancement like Figure 3 As shown, the adaptive lightweight pre-enhancement includes the following steps: Step 2.1: Convert the original RGB image to Lab, HSV, or YCbCr color space, and select... , or Component as luminance channel .

[0045] Step 2.2: Calculate the local average brightness using moving average, integral image, or average pooling. The window size can be determined based on the smallest pixel size of the grain.

[0046] Step 2.3: Generate a low-light trigger map based on the local average brightness. : For areas where the brightness exceeds the threshold, It is zero or close to zero; for areas with lower brightness, The closer it is to 1.

[0047] Step 2.4, according to The mean, quantiles, or partition mean determine the Gamma parameter. For 8-bit images, the Gamma parameter can vary between 0.5 and 1.0; the limiting parameter of CLAHE, the limiting coefficient, can vary between 1 and 4, and the block grid size can be 8×8 or 16×16. The above values ​​are for illustrative purposes only.

[0048] Step 2.5, processing results of Gamma and CLAHE With the original image Soft fusion is performed to obtain the pre-enhanced image. : =( - ) + ⊙ Compared to hard switching, which selects "enhance" or "not enhance" based on a single threshold, soft fusion can reduce abrupt changes in the input distribution near the threshold.

[0049] Step 2.6: Pre-enhancement is only used to improve initial target perception; the pre-enhanced brightness is not used to recalculate the true low-light degradation state, nor is it used for subsequent low-light degradation states. Still from the original image calculate.

[0050] Example 3: First Detection and Explicit State Calculation like Figure 2 and Figure 4 As shown, the first detection and explicit state calculation include the following steps: Step 3.1, will Input MobileNetV4. The general inverted bottleneck module in MobileNetV4 has a configurable structure and can be instantiated into one of the following four forms according to deployment requirements: inverted bottleneck form, extra depth convolution form, ConvNeXt form, and feedforward network form. This module continuously downsamples to obtain feature maps at three scales: P3, P4, and P5. In step 3.2, FPN upsamples the high-level features and fuses them with the low-level features, and PAN then passes the shallow localization information down. The fused features are fed into the YOLOv8 or YOLO11 decoupled detection head to generate classification response, target presence response, and bounding box regression values, respectively.

[0051] Step 3.3: Apply a lower candidate threshold to the detector head output and employ Soft Non-Maximum Suppression (Soft-NMS) or Distance Intersection over Union (DIoU-NMS) to retain densely adjacent particles, thus obtaining... Simultaneously, the target's existence is preserved before NMS. .

[0052] Step 3.4: Adaptive Gaussian kernel formation for each candidate center stacking scale. : ,in For the first Confidence level of each candidate target For the first candidate target center coordinate, For the first candidate target center coordinate, These represent the horizontal and vertical coordinates of the pixel location in the image space, respectively. Indicates the first The variance of the Gaussian kernel corresponding to each candidate target controls the width of the response region. The standard deviation of the Gaussian kernel is related to the width, height, or fitted radius of the candidate box, causing larger particles to produce a wider response and smaller particles to produce a narrower response.

[0053] Step 3.5, will Upsampling is performed at state diagram resolution, through... The minimum-maximum normalization yields Blend according to preset coefficients and ,get .

[0054] Step 3.6, calculate the asymmetric coverage ratio of each candidate box relative to other candidate boxes: The asymmetric coverage ratio uses the area of ​​the evaluated target itself as the denominator, which can distinguish the different effects of a large frame covering a small frame and a small frame covering a large frame.

[0055] Step 3.7: Perform Canny edge detection within the candidate bounding box and divide the polar angle relative to the candidate center into... For each interval, the number of intervals with continuous edge support is counted. For low-light regions, contours can also be extracted from both the original image and the pre-enhanced image, and the larger support value can be taken.

[0056] Step 3.8: Calculate coarse shape residuals using RANSAC circle fitting or ellipse fitting, and weight and fuse coverage ratio, contour support rate, and shape consistency to obtain target-level reliability. : This is then mapped to a spatial reliability map R.

[0057] Step 3.9, Calculate the low-light state from the original image and fuzzy state The fuzzy state can be obtained from the Laplacian variance, Tenengrad, or high-frequency energy in the frequency domain. The specific algorithm can be calibrated based on camera noise and conveyor belt speed.

[0058] Example 4: Spatial Control Chart and Dual-Path Feature Recovery like Figure 5 As shown, the spatial control chart and dual-path recovery include the following steps: Step 4.1, density pass Convert to density priority factor. When β equals 0, only regions with density response are recovered; when β is greater than 0, the density state is used to increase the recovery priority of dense regions, without completely prohibiting the recovery of sparse targets, allowing sparse but significantly degraded reliable targets to obtain basic recovery. The spatial state can also be fused using weighted summation, logical mapping, lookup tables, or other preset functions, as long as the spatial control information is related to both the target spatial state and the corresponding degraded state.

[0059] Step 4.2, fuse them separately , , and , , ,get and : , .for Areas with severe occlusion close to zero, even and At a higher level, the spatial control map is still suppressed, thus avoiding the restoration of branches to fill in unverifiable target boundaries out of thin air.

[0060] Step 4.3, will and By converting the data to the same spatial dimensions as P3, P4, and P5 through area interpolation or average pooling, a single-channel control chart is formed and broadcast along the channel dimension when multiplied with multi-channel features.

[0061] Step 4.4: The low-light feature recovery branch and the deblurring feature recovery branch adopt a lightweight U-Net-like structure. The encoding end uses 3×3 depthwise separable convolution and SiLU activation, while the decoding end uses upsampling, skip connections, and 1×1 channel hybrid convolution; the two branches can share the encoding layer, but have independent output heads.

[0062] Step 4.5: The original output of the recovery branch is first tanh-limited and then multiplied by a bounded learnable scale. The low-light control map modulates only the low-light recovery residual, and the deblurring control map modulates only the deblurring recovery residual.

[0063] Step 4.6: Add the controlled residual to the basic feature to obtain... : = + ⊙ + ⊙ ,in This process does not directly multiply the control chart by the fundamental characteristics, so even if the control chart is estimated incorrectly, the fundamental characteristics can still be propagated along the identity path.

[0064] Example 5: Joint Training and Gradient Backpropagation like Figure 6 As shown, the joint training adopts a phased training approach. In the first phase, basic detection pre-training is performed on the MobileNetV4 backbone, FPN / PAN feature fusion network, and shared YOLO detection head. The training input uses the same adaptive pre-enhancement processing as the deployment phase to avoid inconsistencies between training and inference input distributions.

[0065] The second stage trains the low-light and deblurring feature recovery branches. Alternatively, the backbone network and shared detection head can be frozen, utilizing paired data consisting of clear images and synthetic or real low-light / blurred images to make the features of the recovered degraded image approximate the teacher features of the clear image; alternatively, a clear image teacher can be omitted, and the recovery branches can be directly trained using the second detection loss, making the recovery process oriented towards the detection task.

[0066] The third stage performs full joint training. The first detection result undergoes a gradient cessation operation before entering the rule-based state calculation module. The rule module does not contain trainable parameters, and the final detection loss does not modify the first detection box through candidate filtering, box overlap, and edge calculation. However, the final detection loss can be backpropagated to the two recovery branches, the shared detection head, and the backbone network through recovered features. The spatial control graph, acting as fixed spatial weights, determines the gradient intensity obtained by the two recovery branches at each spatial location.

[0067] In the early stages of training, density states generated by truth labels, reliability states, and rule states generated by the first detection results can be mixed and used, and the weight of the truth states can be gradually decayed to zero, so that a consistent rule state generation method can be used in the later stages of training and inference.

[0068] Joint training specifically includes the following steps: Step 5.1, Output of the first test A stopping gradient operation is performed before proceeding to candidate box filtering, asymmetric coverage, Canny edges, and RANSAC computation. The rule state module forms a forward function bridge, but differentiability of all rule operations is not required.

[0069] Step 5.2, First loss detection Directly update the shared YOLO detection head and MobileNetV4 backbone. Second detection loss. pass The basic feature path updates the backbone and shared detection head, and the two recovery branches are updated through the recovery residual path modulated by the spatial control map. The detection loss can be the classification loss, distributed focal loss (DFL), and CIoU loss used in YOLOv8 or YOLO11, or other target detection losses such as Focal Loss, Smooth L1, GIoU, and DIoU.

[0070] Step 5.3, for any spatial location, when When the gradient is zero, the detection gradient obtained by the low-light recovery branch at that location is zero; when When the gradient increases, the low-light recovery branch at that location receives a stronger gradient. The spatial gradient of the deblurring recovery branch is controlled in the same way.

[0071] Step 5.4: When there are sharp-degraded paired images, the exponential moving average copy of the backbone network is used as the teacher network, and the teacher features of the sharp images are used as the recovery target; when there are no paired sharp images, the following can be used: Set to zero, and rely solely on the final detection loss to train the recovery branch.

[0072] Step 5.5: In the early stages of training, ground truth object centers and visibility annotations can be used to calculate... and and with and Hybrid; as the number of training rounds increases, the proportion of truth states is gradually reduced, and eventually the rule states from the inference phase are used entirely.

[0073] Example 6: Fine detection, profile fitting and particle size measurement like Figure 7 As shown, fine inspection and particle size measurement include the following steps: Step 6.1, will Re-input the shared YOLO detection head, using the final detection confidence threshold and Soft-NMS output. The shared detection head uses the same parameters in both detections to reduce the number of model parameters and unify the target definition in both detections, but the first detection uses a high-recall candidate configuration and the second uses the final output configuration.

[0074] Step 6.2, according to The target bounding box in the image extracts the region of interest from the original image, and the candidate overlap, contour support, and shape consistency are recalculated to obtain the final visibility reliability. .

[0075] Step 6.3: Obtain the set of visible contour points using a Canny, Sobel, or learned edge detector. Use RANSAC to fit circles or ellipses, and remove outlier edge points caused by dust, textures, and occlusion boundaries.

[0076] Step 6.4 calculates the product-type measurement qualification score of detection confidence, final reliability, fitting residual, and local ambiguity. The product structure causes a significant drop in the overall score when any key quality factor is too low; a weighted sum or logical rule can also be used.

[0077] Step 6.5: Obtain camera intrinsic parameters and distortion coefficients using the checkerboard pattern and Zhang Zhengyou calibration method: Distortion correction is performed on the image; position-related scales are obtained based on the homography of the conveyor belt plane or the pixel equivalent of the partitions. Convert pixel diameter to actual grain size: .

[0078] Step 6.6: Divide the actual particle size into multiple particle size intervals, count the number and proportion of particles in each interval, and output the data to the display terminal or PLC control system.

[0079] Example 7: System Implementation and Alternative Methods The system in this embodiment includes an industrial camera, a supplementary lighting source, a conveyor belt, an edge computing device, and a display terminal. The industrial camera is preferably a global shutter camera, and the supplementary lighting source can be a ring light, a strip light, or a strobe light; the edge computing device can be an industrial computer equipped with a GPU, NPU, or other neural network accelerators.

[0080] In one implementation, MobileNetV4, FPN / PAN, and YOLO11n are deployed in FP16 or INT8 mode, the rule state computation is implemented using OpenCV, and the low-light and deblurring recovery branches are performed using depthwise separable convolutions on P3 and P4 features.

[0081] In another implementation, the backbone uses ResNet50 or ConvNeXt, the detector uses Faster R-CNN or RT-DETR (Real-Time Detection Transformer), and the feature recovery branch uses Restormer or a lightweight Transformer. It still generates explicit states and modulates the recovery residuals based on the initial detection and the original image. This alternative approach still falls under the technical concept of "initial detection—explicit state—spatial control—feature recovery—re-detection" of this invention.

[0082] In another implementation, the feature map is divided into several blocks based on the control map, and the recovery branch is only executed on active blocks whose control map mean or maximum value reaches a threshold, thereby further reducing the actual computational cost of the recovery module. When sparse block execution is not used, the spatial control map is mainly used to reduce invalid feature modifications and suppress erroneous recovery, rather than necessarily reducing the convolutional computation cost.

[0083] The above description is merely a preferred embodiment of the present invention. For those skilled in the art, any equivalent substitutions or modifications made to the network type, state calculation formula, control chart fusion method, number of recovery branches, loss function, and measurement model without departing from the core concept of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for industrial particle detection and particle size measurement, characterized in that, Includes the following steps: S1: Image Acquisition and Adaptive Lightweight Pre-Augmentation: Acquiring raw images of industrial particles on a conveyor belt using an industrial camera. According to the original image The local average brightness is used to calculate low-light trigger information, and the original image is processed according to the low-light trigger information. Adaptive Gamma correction and contrast-limited adaptive histogram equalization (CLAHE) are performed on the image. The results are then spatially soft-fused with the original image to obtain the pre-enhanced image. ; S2: Multi-scale feature extraction and first coarse detection: pre-enhanced image... Input the feature extraction backbone network to obtain at least two basic visual features at different resolutions; The basic visual features are input into the shared object detection network for the first detection, obtaining initial detection results including candidate bounding boxes, candidate confidence scores, and candidate center locations. And retain the original dense target response output by the shared target detection network before candidate selection and nonmaximum suppression. ; S3: Calculation of target spatial state and image degradation state: based on initial detection results Original dense target response and the original image The target spatial state and image degradation state are explicitly calculated, wherein the target spatial state includes at least the local target density state. and target visible reliability status The image degradation state includes at least a low-light degradation state. and fuzzy degenerate state ; S4: Spatial Control Map Generation: Based on Local Target Density Status Target visibility reliability status and low light degradation state Generate low-light recovery spatial control map Based on the local target density state Target visibility reliability status and fuzzy degenerate state Generate deblurred recovery spatial control map and low-light recovery spatial control map Deblurring and recovery of spatial control chart Each feature is resampled to the spatial resolution corresponding to its basic visual features. S5: Dual-path selective feature recovery: Each basic visual feature is input into the low-light feature recovery branch and the deblurring feature recovery branch respectively to obtain the low-light recovery feature residual and the deblurring recovery feature residual; using the low-light recovery spatial control map of the corresponding spatial resolution... Deblurring and recovery of spatial control chart Spatially modulate the low-light recovery feature residual and the deblurred recovery feature residual respectively, and inject the modulated recovery feature residual into the corresponding basic visual feature in a bounded addition manner to obtain the recovered visual feature; S6: Second fine-tuning detection: The recovered visual features are input again into the shared target detection network to obtain the final detection result of industrial particles. The final detection result At least the location of the particle target and the detection confidence level should be included; S7: Based on the final test results In the original image The visible contours of the particle targets are extracted, and the pixel size and contour fitting residuals of the particle targets are obtained by using circle fitting or ellipse fitting. The measurement qualification score is calculated based on the detection confidence, final visibility reliability, contour fitting residuals and the blurring degree of the target area. When the measurement qualification score reaches the preset threshold, the pixel size is converted into the actual particle size according to the camera calibration parameters. When the measurement qualification score is lower than the preset threshold, the target detection result is retained and the corresponding particle target is marked as unreliable for measurement.

2. The method according to claim 1, characterized in that, Step S1 includes the following sub-steps: S1.1: Convert the original image Convert to a color space with a luminance component, and calculate the local average luminance within a neighborhood window for each spatial location. ; S1.2: According to Generate low-light trigger map ,in To preset the brightness threshold, To prevent positive numbers with a denominator of zero, ; S1.3: Based on the low-light trigger map The global or partition mean is used to determine the Gamma correction parameters, as well as the CLAHE limiting parameters and grid size, for the original image. Perform Gamma correction and CLAHE processing to obtain candidate enhanced images. ; S1.4: According to Generate pre-enhanced image This ensures that areas with normal brightness retain their original image information, while providing limited enhancement to low-light areas; the same pre-enhancement rules are used in both the training and inference phases. Adaptive pre-enhancement can also employ Retinex, homomorphic filtering, local contrast stretching, or combinations thereof.

3. The method according to claim 2, characterized in that, The feature extraction backbone network in step S2 uses MobileNetV4, ResNet, EfficientNet, ConvNeXt, Swin Transformer, or other networks with multi-scale feature output. When using MobileNetV4, MobileNetV4 outputs downsampled features at 8x, 16x, and 32x. The features at different downsampled ratios are upsampled, downsampled, concatenated, or fused by addition through the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN) to obtain fused features. The fused features are then input into the subsequent shared target detection network. The shared object detection network uses a decoupled detection head of YOLOv8 or YOLO11, Faster R-CNN or other object detection networks. When using a decoupled detection head of YOLOv8 or YOLO11, it outputs the class confidence, object presence response and bounding box regression, respectively. The first detection in step S2 employs a high-recall candidate generation method, including: using a candidate confidence threshold lower than that of the final detection stage; using Soft-NMS, DIoU-NMS, or a non-maximum suppression method with a high overlap threshold to retain densely adjacent targets; and retaining the original dense target responses before candidate screening. When no candidate bounding boxes meet the candidate confidence threshold, the original dense target response is still used. Generate local target density state To prevent low-light recovery spatial control charts from being empty due to initial detection results. Deblurring and recovery of spatial control chart All are zero.

4. The method according to claim 3, characterized in that, Local target density state in step S3 The calculations include: With the first Candidate Centers Determine the Gaussian kernel standard deviation centered on the candidate target scale. And based on candidate confidence As weights, the candidate box density state is obtained by superposition. ; Response to the original dense target Normalization and spatial resampling are performed to obtain the response density state. ; according to Fusion candidate box density state and response density state ,in The fusion coefficient is between 0 and 1; Target visible reliability status in step S3 The calculations include: Calculate the first Asymmetric coverage ratio where each candidate bounding box is covered by other candidate bounding boxes in, Indicates the first The nth candidate bounding box, i.e., the nth output from the coarse detection stage. One target detection bounding box, Indicates the first The candidate bounding box is used to compare with the first candidate bounding box. Compare and calculate the candidate boxes; With the first The candidate region is divided into multiple angular intervals using candidate centers as poles. The proportion of angular intervals with effective contour support is determined based on the results of Canny edge detection, Sobel edge detection, or learned edge detection. ; Perform RANSAC circle or ellipse fitting on the candidate regions and obtain shape consistency based on the coarse fitting residuals. ; according to The target-level visible reliability is obtained, and the target-level visible reliability is mapped to the target visible reliability state diagram through a spatial kernel centered on the candidate center. ,in , and These are the weighting coefficients; Low light degradation state in step S3 Based on the original image The calculation of local average brightness is based on Obtain; among them Brightness threshold for low-light detection; blurring and degradation state Based on the original image The local sharpness is calculated using at least one of Laplacian response variance, Tenengrad gradient, frequency domain high-frequency energy, or edge spread width, according to... Obtain, among which This represents the local sharpness value. The threshold for determining sharpness.

5. The method according to claim 4, characterized in that, In step S4, first follow For local target density state Modulate, and then according to and Generate low-light recovery spatial control maps respectively Deblurring and recovery of spatial control chart ,in The basic restoration coefficient for sparse targets. and To control the gain coefficient; the fusion method of the spatial state is not limited to element-wise multiplication, but also includes weighted sum, logical combination, lookup table mapping or preset function mapping, as long as the generated spatial control information is related to both the target spatial state and the corresponding image degradation state. In step S5, the low-light feature recovery branch and the deblurring feature recovery branch adopt at least one of the following structures: lightweight U-Net structure, residual convolution structure, depthwise separable convolution structure, or Transformer recovery structure; the two recovery branches share the previous feature transformation layer and are respectively set with low-light recovery output head and deblurring recovery output head; the original output of the recovery branch is constrained by the tanh function to restore the residual amplitude, and the recovery injection coefficients are... Set as Then, the spatial control map modulates the corresponding recovery residuals and injects them into the basic features in an additive manner, so that the recovered visual features satisfy... = + ⊙ + ⊙ Where s represents the feature scale, For scale The corresponding basic feature map, For scale The learnable injection intensity coefficient of the lower light recovery branch. For scale The recovery residual output of the low-light recovery branch. For scale The learnable injection strength coefficient of the fuzzy recovery branch. scale The recovery residual output of the fuzzy recovery branch; and Broadcast along the channel dimension.

6. The method according to claim 5, characterized in that, In step S6, the training of the feature extraction backbone network, the shared target detection network, and the low-light feature recovery branch and the deblurring feature recovery branch includes: In the first stage, the feature extraction backbone network and the shared object detection network are pre-trained using training images that have undergone the same adaptive pre-enhancement processing as those in the inference stage. In the second stage, the feature extraction backbone network and the shared target detection network are frozen, and the low-light recovery branch and the deblurring recovery branch are trained using pairs of clear, low-light and blurred images; the recovery branch can also be trained using the second detection loss. In the third stage, the first detection is performed, and the initial detection results are used to calculate after a stopping gradient operation. , , , , and The feature extraction backbone network, the shared object detection network, and the two recovery branches are jointly trained based on the first detection loss, the second detection loss, and the optional feature recovery consistency loss. The total loss function is ,in The classification loss and bounding box regression loss are used for the first detection. The classification loss and bounding box regression loss are used for the second detection. To restore the relationship between visual features and clear image teacher features Loss, mean squared error loss, cosine consistency loss, or perception loss, at least one of the following: , , These are the weight coefficients of each loss term; in the early stage of training, the density state and reliability state generated by the ground truth label are weighted and mixed with the density state and reliability state generated by the first detection result, and the weight of the ground truth state is gradually reduced to zero as training progresses.

7. The method according to claim 6, characterized in that, Step S7 includes: determining the region of interest in the original image based on the final detection box; extracting the set of visible contour points using the Canny or Sobel operator; fitting a circular or elliptical geometric model using RANSAC, least squares, or Hough transform to obtain the center and radius of the circle, or the center, major axis, minor axis, and orientation angle of the ellipse; calculating the contour fitting residual based on the distance between the fitted model and the visible contour points; and calculating the measurement qualification score based on the final detection confidence, final visibility reliability, contour fitting residual, and average blurriness of the target region. ,in For the first The final detection confidence of each target, For the first The contour fitting residuals of each target This is the tolerance coefficient for the fitted residuals. The larger the value, the higher the tolerance for residuals. For the first The average blur level of each target region The attenuation coefficient is the factor for the degree of blur. The larger the value, the higher the tolerance for ambiguity; The camera intrinsic parameters, distortion coefficients, rotation vectors, and translation vectors are calculated using either the checkerboard calibration method or the Zhang Zhengyou camera calibration method. Furthermore, the scale mapping between image position and actual length is established using the conveyor belt planar homography or partitioned pixel equivalents. For round particles, according to Calculate the actual particle size, and for elliptical particles, calculate the equivalent circle diameter based on the major and minor axes; and count the number and proportion of particles in each preset particle size range to form the particle size distribution.

8. An industrial particle detection and particle size measurement system, characterized in that, The method includes an image acquisition module, an adaptive pre-enhancement module, a feature extraction module, a shared target detection module, an explicit state calculation module, a spatial control map generation module, a dual-path feature recovery module, a contour fitting module, and a measurement eligibility judgment module; each module is configured to perform the method described in any one of claims 1 to 7; wherein the shared target detection module performs a first detection on the basic visual features and a second detection on the recovered visual features output by the dual-path feature recovery module.

9. An electronic device comprising an industrial camera interface, a processor, and a memory, characterized in that, The memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images