Multi-algorithm fusion detection method and device for dynamic target and medium

Abstract

The invention discloses a multi-algorithm fusion detection method and device for a dynamic target and a medium, and relates to the technical field of image processing, and the method comprises the steps: collecting continuous video frames of a moving target in a backlight imaging environment; adopting a three-frame difference method to extract dynamic target candidate areas in continuous video frames; on the basis of the structure-enhanced YOLOv8-P2 model, target detection result extraction is carried out on the candidate region under the condition that low-layer feature information is reserved; performing grid slicing on an original video frame in the continuous video frames at a preset overlapping rate, and inputting the original video frame into the RT-DETR model to perform target detection result extraction under background noise suppression; and two target detection results with different emphasis are fused based on spatial position alignment, and coordinates and confidence information of the dynamic target are output. According to the method, advantage complementation is formed through a double-model parallel architecture, and the adaptability of the system in harsh industrial environments such as backlight and dynamic impurities is remarkably improved on the premise that the real-time processing capacity is maintained.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a multi-algorithm fusion detection method, device and medium for dynamic targets. Background Art

[0002] In the field of industrial visual inspection, real-time detection of small dynamic objects (such as bubbles in fluids and semiconductor wafer defects) has long faced three technical bottlenecks. First, traditional image processing methods lack adaptability. While inter-frame differencing can detect moving objects, two-frame differencing is sensitive to background fluctuations (such as lighting changes or mechanical vibrations), easily misclassifying tiny impurities as objects. Three-frame differencing, while improving robustness, struggles to distinguish densely packed small objects from noise. Morphological methods such as Hough transform and blob detection significantly reduce the detection rate for non-standard circular objects (such as elliptical bubbles) and are almost ineffective for objects smaller than 10 pixels. Second, single deep learning models have inherent flaws. Standard YOLO models improve detection speed by fusing high-level features, but multiple downsampling results in the loss of object details smaller than 10 pixels (such as bubble texture), leading to a sharp drop in confidence in backlit or complex backgrounds. Transformer architecture models (such as RT-DETR) can suppress background interference, but their ability to capture spatial details of small objects is inferior to CNNs, resulting in high miss detection rates when using direct whole-image detection. Third, achieving both real-time performance and precision is difficult. Traditional methods require a cascade of multiple processing steps (such as differential analysis, morphological filtering, and feature matching), resulting in computational latency that cannot meet the demands of online detection. Pursuing high precision in end-to-end deep learning models (e.g., by increasing input resolution) will cause inference speeds to drop below 1 fps, making them incapable of keeping pace with industrial assembly lines. Summary of the Invention

[0003] In order to match the target detection rhythm on the industrial assembly line, the present invention proposes a multi-algorithm fusion detection method for dynamic targets, including the following steps: S1: Capturing continuous video frames of a moving target in a backlit imaging environment; A1: Use the three-frame difference method to extract dynamic target candidate regions in consecutive video frames; A2: Based on the structure-enhanced YOLOv8-P2 model, target detection results are extracted from the candidate regions while preserving low-level feature information; B1: After grid slicing the original video frames in the continuous video frames with a preset overlap rate, the frames are input into the RT-DETR model to extract the target detection results under background noise suppression; S2: Based on spatial position alignment, the target detection results of steps A2 and B1 are fused to output the coordinates and confidence information of the dynamic target.

[0004] This invention utilizes a dual-model parallel architecture to complement each other: YOLOv8-P2 specializes in fast and precise detection of differential regions, while RT-DETR focuses on enhancing global background interference. Ultimately, these two models are combined through spatially aligned confidence-weighted fusion to produce a unified detection result. This design significantly improves the system's adaptability in harsh industrial environments, such as backlighting and dynamic impurities, while maintaining real-time processing capabilities.

[0005] Furthermore, in step A1, the specific implementation of the three-frame difference method includes: Calculating a first difference map between the current video frame and the previous video frame; Calculating a second difference map between the current video frame and the next video frame; A logical AND operation is performed on the first difference map and the second difference map to generate a dynamic target candidate region.

[0006] Furthermore, the step A1 further includes: performing binarization processing on the dynamic target candidate area, and filtering out impurity pixels through morphological filtering.

[0007] Furthermore, the morphological filtering is an opening operation, which includes first performing an erosion operation to remove small-sized impurities, and then performing a dilation operation to restore the original size of the target area.

[0008] Furthermore, in step A2, the structure-enhanced YOLOv8-P2 model retains low-level feature information through the following structure: The backbone network retains the P2 feature layer with a second downsampling output resolution of 1 / 4 the size of the input image; The head network upsamples the P5 layer features by a factor of 2 and concatenates them with the P2 feature layer, which are then fused through the C2f module to generate the P2 / 4-tiny features. The detection head simultaneously receives the feature outputs of the four feature layers P2, P3, P4, and P5.

[0009] Furthermore, in the step B1, the preset overlap rate is 50%, and a grid division with a 50% overlap rate is adopted during grid slicing.

[0010] Furthermore, in step B1, the RT-DETR model includes the following processing steps for the input grid slice: Transformer-based object detection is performed independently on each grid slice, and duplicate detection boxes are eliminated by merging the object detection results of all grid slices through the non-maximum suppression algorithm.

[0011] Furthermore, the fusion operation in step S2 includes: Map the target detection results of step B1 to the image coordinate system of the original video frame; Perform confidence-weighted fusion on the detection boxes at the same spatial position in step A2 and step B1.

[0012] The present invention also includes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a multi-algorithm fusion detection method for dynamic targets.

[0013] The present invention also includes a device for processing data, comprising: a memory having a computer program stored thereon; The processor is used to execute the computer program in the memory to implement the steps of a multi-algorithm fusion detection method for dynamic targets.

[0014] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention proposes a multi-algorithm fusion detection method for dynamic targets. It uses a dynamic region extraction method based on the three-frame difference method combined with a logic and operation mechanism to effectively suppress false triggering caused by illumination fluctuations. The improved YOLOv8-P2 model enhances the ability to capture small target textures in low-level feature fusion by retaining the P2 feature layer with 1 / 4 resolution, and uses morphological filtering to accurately eliminate pixel-level impurity interference, thus breaking through the detection limit of traditional methods for small-sized targets. (2) A grid slicing strategy with a 50% overlap rate is used to partition the original frame, so that the Transformer attention mechanism of the RT-DETR model focuses on suppressing local background noise. The non-maximum suppression algorithm is used to eliminate redundant detection frames, and the target in the image boundary area can still be fully captured in interference scenes such as metal reflections and water ripples, thus making up for the spatial blind spot defect of the frame difference method. (3) The dual-model parallel architecture complements each other's strengths: YOLOv8-P2 specializes in fast and precise detection of differential regions, while RT-DETR focuses on enhancing interference resistance against global backgrounds. Ultimately, unified detection results are output through spatially aligned confidence-weighted fusion. This design significantly improves the system's adaptability in harsh industrial environments such as backlighting and dynamic impurities while maintaining real-time processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a step diagram of a multi-algorithm fusion detection method for dynamic targets; Figure 2 Schematic diagram of the structural improvement YOLOv8-P2; Figure 3 A side view of the experimental environment; Figure 4 A top-down diagram of the experimental environment. DETAILED DESCRIPTION

[0016] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0017] Example 1 In the fields of industrial inspection and fluid analysis, accurate identification and real-time monitoring of dynamic targets (such as bubbles and tiny particles) are critical technologies for ensuring system reliability. However, traditional detection methods face numerous challenges. Preprocessing based on inter-frame differencing (ID) can quickly extract dynamic regions, but it is susceptible to environmental interference (such as water ripples and impurities), resulting in excessive noise in candidate regions. Furthermore, single deep learning models (such as the YOLO series or DETR) often suffer from missed and false detections when detecting complex backgrounds or small targets, and their performance is particularly limited in backlit imaging conditions. Furthermore, existing technologies lack compatibility with multi-scale targets, making it difficult to strike a balance between detection accuracy and computational efficiency.

[0018] In response to the above problems, the present invention proposes a multi-algorithm fusion detection method for dynamic targets, which achieves breakthrough improvements through innovative technical architecture and model optimization, such as Figure 1 As shown, the method includes the following steps: S1: Capturing continuous video frames of a moving target in a backlit imaging environment; A1: Use the three-frame difference method to extract dynamic target candidate regions in consecutive video frames; A2: Based on the structure-enhanced YOLOv8-P2 model, target detection results are extracted from the candidate regions while preserving low-level feature information; B1: After grid slicing the original video frames in the continuous video frames with a preset overlap rate, the frames are input into the RT-DETR model to extract the target detection results under background noise suppression; S2: Based on spatial position alignment, the target detection results of steps A2 and B1 are fused to output the coordinates and confidence information of the dynamic target.

[0019] Specifically, the present invention first needs to construct a high-contrast moving target capture scene in a backlit imaging environment through the synergy of a forward-arranged image acquisition device and a rearward uniform light source. Here, backlit imaging adopts a penetrating lighting strategy to make the moving target appear as a dark field outline on the imaging plane, effectively enhancing the grayscale difference between the target edge and the background; continuous video frames are collected at a predetermined frame rate to ensure that the target displacement between adjacent frames meets the motion perception requirements of the inter-frame difference method, while avoiding motion blur of high-speed targets. This imaging environment is suitable for dynamic target detection in transparent or semi-transparent media, and suppresses ambient light interference through a constant light source.

[0020] For the captured continuous video frames, the present invention uses a three-frame differencing method in a subsequent branch processing step (steps A1 to A2) to extract candidate dynamic target regions. By calculating the pixel-level differences between the current frame and the adjacent frames, a robust motion region identifier is generated using a logical AND operation. Specifically, a grayscale difference is first performed between the current frame and the previous frame to generate a first difference map. A second difference map is then generated between the current frame and the next frame. These two difference maps are then binarized, an adaptive threshold is set to isolate pixels with significant motion, and a final dynamic region mask is synthesized using a logical AND operation. This process effectively suppresses transient noise interference caused by illumination fluctuations or mechanical vibrations while preserving the complete motion trajectory of small targets such as bubbles. Furthermore, morphological filtering is used to optimize the candidate regions, removing non-target impurities at the 2-10 pixel level while avoiding excessive erosion of the target morphology. This provides high signal-to-noise ratio input features for the subsequent deep learning model. For example, an opening operation is used to first perform an erosion operation to remove small impurities, followed by a dilation operation to restore the target region to its original size.

[0021] The present invention then uses a unique feature pyramid network to achieve high-precision capture of tiny objects when performing object detection in candidate regions based on the structurally enhanced YOLOv8-P2 model. This model retains the P2 feature layer at 1 / 4 the resolution of the input image in the backbone network, allowing the edge texture and morphological features of tiny objects such as bubbles to be continuously transmitted throughout the deep network, effectively alleviating the problem of small object feature loss caused by multiple downsampling in traditional YOLO models. During the feature fusion stage, the model combines high-level semantic features with detailed features from the P2 layer across scales. After optimizing the cross-stage connectivity structure of the C2f module, a multi-scale feature representation (P2 / 4-tiny) is generated that combines both spatial detail and semantic information. This feature fusion path effectively combines high-level semantic information with low-level detailed information, further enhancing the model's ability to detect small objects in complex backgrounds. Finally, the detection head simultaneously receives feature inputs at four scales, P2 to P5. By leveraging the complementary features of different receptive fields, it achieves precise localization and classification of 2-10 pixel objects within the candidate region.

[0022] This structural improvement maintains a high-resolution feature map of 160×160 (e.g. Figure 2As shown in the figure (using 640×640 input as an example), key detail features such as bubble edges and textures are retained, thereby solving the feature loss problem of traditional YOLOv8 in small target detection. The use of upsampling splicing and the C2f module to achieve cross-scale feature fusion enables the model to simultaneously capture local details and understand global semantics. Finally, through the collaborative prediction of multi-level features from P2 to P5, adaptive detection of targets of different sizes is achieved, especially improving the detection rate of tiny bubbles. This design, which maintains a lightweight network while adding the P2 feature layer, not only meets the detection speed requirements of industrial real-time requirements, but also significantly improves the system's sensitivity to tiny dynamic targets, providing reliable support for real-time detection in complex industrial scenarios.

[0023] Compared to steps A1 and A2, in another subsequent branch step (step B1, running synchronously with steps A1 and A2), the present invention utilizes the relatively pure background of dynamic regions (such as bubble motion trajectories) output by the three-frame differencing method. This allows it to maximize the advantages of high-precision detection of small targets after being assigned to the structurally improved YOLOv8-P2. However, because the differencing method cannot filter out dynamic background interference, such as metal reflections and water ripples, it may miss small targets with blurred boundaries. Therefore, in branch step B1, the present invention introduces the RT-DETR model to perform target detection with enhanced anti-interference in non-differential regions.

[0024] Specifically, the present invention first performs grid slicing processing with an overlap rate of 50% on the original video frame to ensure that the moving target is completely covered by at least two adjacent slices in the boundary transition area, so as to overcome the inherent defects of boundary target truncation and missed detection in traditional whole-image detection. This slicing strategy can not only achieve the best balance between detection effect and computational efficiency, but is also particularly suitable for processing small-sized targets, such as tiny dynamic objects such as bubbles. Specifically, an overlap rate of less than 50% will cause the bubble detection rate in the boundary area to drop significantly, especially for those small bubbles with a diameter of less than 5 pixels; while an overlap rate of more than 50% will increase unnecessary computational burden and have limited improvement in detection accuracy. Therefore, choosing an overlap rate of 50% means that each pixel will be covered by 2 to 4 slices at most, which not only ensures the effectiveness of detection but also controls computational redundancy.

[0025] Each slice obtained is independently input into the RT-DETR model for parallel processing. The global attention mechanism of the Transformer architecture is used to suppress background noise interference such as metal reflections and water ripples in local areas, enhancing the feature focus on small-scale targets. The final slice detection results are spatially consistent and fused using the non-maximum suppression algorithm, thereby eliminating redundant detection frames caused by regional overlap and ultimately generating target positioning information with strong background robustness. Here, the present invention introduces the RT-DETR model (focusing on complex background interference resistance) to complement the structurally improved YOLOv8-P2 (focusing on small target details).

[0026] It is worth noting that during the training phase, to enhance the RT-DETR model's ability to perceive small objects, the present invention also uses data augmentation strategies specifically for small objects, including random scaling, brightness and contrast adjustment, and mosaic enhancement. These enhancements help improve the model's generalization and robustness, enabling it to operate stably under different lighting conditions and backgrounds.

[0027] After the branching step, the RT-DETR slice detection results are converted to the image coordinate system of the original video frame according to the grid coordinate mapping relationship to eliminate the spatial deviation caused by local processing, and then the overlapping detection frames are confidence-weighted fused based on the target spatial position matching. Among them, the detection results of YOLOv8-P2 in the pure area pre-screened by the differential method should be given a higher weight, while the detection results of RT-DETR in the complex background area need to be conservatively weighted to participate in the supplementary decision. This fusion mechanism reconciles the scene adaptability differences of the two models: when the target is in a high-contrast dynamic area, the high-precision recognition of YOLOv8-P2 is preferred, while in interference areas such as lighting transitions or metal reflections, the anti-interference detection of RT-DETR is enhanced, and finally the global target positioning information with consistent spatial position and confidence calibration is output, achieving a system-level leap in detection robustness and accuracy in complex industrial environments.

[0028] In summary, the multi-algorithm fusion detection method for dynamic targets proposed in the present invention effectively suppresses false triggering caused by illumination fluctuations by extracting dynamic regions based on the three-frame difference method and combining it with a logical AND operation mechanism. The improved YOLOv8-P2 model enhances the ability to capture the texture of small targets in low-level feature fusion by retaining the P2 feature layer with 1 / 4 resolution, and uses morphological filtering to accurately eliminate pixel-level impurity interference, thus breaking through the detection limit of traditional methods for small-sized targets.

[0029] A grid slicing strategy with a 50% overlap rate is used to partition the original frame, allowing the Transformer attention mechanism of the RT-DETR model to focus on suppressing local background noise. Combined with the non-maximum suppression algorithm to eliminate redundant detection frames, it can still fully capture targets in the image boundary area in interference scenarios such as metal reflections and water ripples, making up for the spatial blind spot defects of the frame difference method.

[0030] The dual-model parallel architecture complements each other's strengths: YOLOv8-P2 specializes in fast and precise detection of differential regions, while RT-DETR focuses on enhancing global background interference. Ultimately, these models are combined through spatially aligned confidence-weighted fusion to produce a unified detection result. This design significantly improves the system's adaptability to harsh industrial environments, such as backlighting and dynamic impurities, while maintaining real-time processing capabilities.

[0031] Example 2 In order to better understand the technical content of the present invention, this embodiment further designs an extended imaging field of view based on the principle of mirror reflection in a specific experimental environment on the basis of the theoretical technical content to verify the technical solution of the present invention. Figure 3 、 Figure 4 As shown in the figure, the experimental environment is a water tank. Highly reflective plane mirrors with a 45° inclination are deployed on the front and rear sides of the water tank detection scene, each with a silver-gray metal plate as the bottom. The camera faces the front glass of the water tank (an isolation shield is installed on the side facing away from the front light source when looking down), and the front light source faces the rear glass of the water tank. The left and right mirrors reflect bubbles in the lateral areas into the imaging optical path. This structure enables a single frame image to simultaneously include the main field of view transmission imaging and the bilateral reflected field of view. Mirror reflection maintains the backlit imaging characteristics, and bubbles still appear as dark field outlines in the reflected field of view. The full field of view resolution remains ≥4 pixels / mm, with no resolution degradation at the edges.

[0032] To address the problem of reflective artifacts introduced by mirrors, the three-frame difference method automatically filters out abnormal directional trajectories through motion vector analysis: the movement direction of bubbles in the main field of view is mirror-symmetric with the reflected field of view, and artifact removal is performed when a target moving in the same position and in the opposite direction is detected. When processing synthetic image slices, the RT-DETR model's Transformer attention mechanism adaptively learns the reflective characteristics of the mirrored metal frame, locking in the true bubble shape amidst complex background noise. Actual measurements have shown that this solution reduces the missed detection rate of bubbles at group boundaries to below 5%, and the processing time for a single frame is stabilized within 900ms, meeting the real-time and integrity requirements of industrial online inspection.

[0033] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0034] In addition, in the present invention, descriptions such as "first," "second," and "one" are for descriptive purposes only and should not be understood to indicate or imply their relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0035] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two elements or interaction between two elements, unless otherwise specified. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0036] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A multi-algorithm fusion detection method for dynamic targets, characterized by: Including steps: S1: Capturing continuous video frames of a moving target in a backlit imaging environment; A1: Use the three-frame difference method to extract dynamic target candidate regions in consecutive video frames; A2: Based on the structure-enhanced YOLOv8-P2 model, target detection results are extracted from the candidate regions while preserving low-level feature information; B1: After grid slicing the original video frames in the continuous video frames with a preset overlap rate, the frames are input into the RT-DETR model to extract the target detection results under background noise suppression; S2: Based on spatial position alignment, the target detection results of steps A2 and B1 are fused to output the coordinates and confidence information of the dynamic target.

2. A multi-algorithm fusion detection method for dynamic targets according to claim 1, characterized in that: In step A1, the specific implementation of the three-frame difference method includes: Calculating a first difference map between the current video frame and the previous video frame; Calculating a second difference map between the current video frame and the next video frame; A logical AND operation is performed on the first difference map and the second difference map to generate a dynamic target candidate region.

3. The multi-algorithm fusion detection method for dynamic targets according to claim 1, characterized in that: The step A1 further includes: performing binarization processing on the dynamic target candidate area and filtering out impurity pixels through morphological filtering.

4. The multi-algorithm fusion detection method for dynamic targets according to claim 3, characterized in that: The morphological filtering is an opening operation, which includes first performing an erosion operation to remove small-sized impurities, and then performing a dilation operation to restore the original size of the target area.

5. The multi-algorithm fusion detection method for dynamic targets according to claim 1, characterized in that: In step A2, the structure-enhanced YOLOv8-P2 model retains low-level feature information through the following structure: The backbone network retains the P2 feature layer with a second downsampling output resolution of 1 / 4 the size of the input image; The head network upsamples the P5 layer features by a factor of 2 and concatenates them with the P2 feature layer, which are then fused through the C2f module to generate the P2 / 4-tiny features. The detection head simultaneously receives the feature outputs of the four feature layers P2, P3, P4, and P5.

6. The multi-algorithm fusion detection method for dynamic targets according to claim 1, characterized in that: In the step B1, the preset overlap rate is 50%, and a grid division with a 50% overlap rate is used during grid slicing.

7. The multi-algorithm fusion detection method for dynamic targets according to claim 1, characterized in that: In step B1, the RT-DETR model includes the following processing steps for the input grid slice: Transformer-based object detection is performed independently on each grid slice, and duplicate detection boxes are eliminated by merging the object detection results of all grid slices through the non-maximum suppression algorithm.

8. The multi-algorithm fusion detection method for dynamic targets according to claim 1, characterized in that: The fusion operation of step S2 includes: Map the target detection results of step B1 to the image coordinate system of the original video frame; Perform confidence-weighted fusion on the detection boxes at the same spatial position in step A2 and step B1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-algorithm fusion detection method for dynamic targets described in any one of claims 1 to 8 are implemented.

10. A device for processing data, characterized in that: include: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the multi-algorithm fusion detection method for dynamic targets as described in any one of claims 1 to 8.

Images