Intelligent detection and prediction method and system for filter tip defects

Abstract

The invention belongs to the technical field of intelligent manufacturing and production line quality control, and particularly relates to a filter tip defect intelligent detection and prediction method and system. A lightweight YOLOv8 network is taken as a core, a convolution block attention module and depth separable convolution are fused in a cross-stage partial residual error backbone network, and an improved backbone structure is constructed; meanwhile, multi-branch texture / edge feature extraction and a Transform-based neck feature fusion structure are introduced, so that global modeling of cross-scale features is realized; a local amplification secondary detection mechanism is designed for a suspicious region with relatively low confidence coefficient, so that the recognition precision of extremely small flaws is improved; and dynamic evolution of the defects in the filter tip movement process is predicted and traced in combination with long and short term memory network time sequence modeling and a multi-target tracking algorithm. The method has the advantages of low delay, high robustness and good expansibility while giving consideration to the detection precision and efficiency, and can be widely applied to quality detection, defect analysis and intelligent production line monitoring in the tobacco production process.

Description

Technical Field

[0001] This invention belongs to the field of intelligent manufacturing and production line quality control technology, specifically relating to an intelligent detection and prediction method and system for filter tip defects. Background Technology

[0002] With the continuous expansion of production scale and the ongoing improvement of automation levels in the tobacco manufacturing industry, the operating speed of filter tip production lines has significantly increased. Traditional manual inspection and machine vision inspection methods based on simple image processing are no longer sufficient to meet the real-time and accuracy requirements of high-speed production lines for defect detection. During the production process, filter tips may develop minor defects such as cracks, abnormal surface textures, and localized collapses. These problems often seriously affect the appearance quality and performance of the product, and even adversely affect the stability of the entire batch. Due to the small size and complex shape of defects, and the accompanying jitter and blurring during movement, traditional inspection methods are prone to missed detections and false detections, and cannot perform dynamic evolution tracking and trend prediction. Therefore, there is an urgent need for an intelligent inspection method that can balance inspection speed and accuracy to achieve real-time identification, cross-frame tracking, and evolution prediction of filter tip defects, thereby meeting the industrial production demand for efficient, stable, and low-latency inspection. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and propose an intelligent detection and prediction method and system for filter tip defects. This method and system can solve the problems of missed detection, false detection and unstable detection that easily occur when filter tip defects such as micro-cracks, abnormal textures and local collapses are detected on high-speed production lines under complex lighting, rapid movement and occlusion interference conditions. This improves the accuracy and robustness of filter tip defect identification.

[0004] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0005] S1. Preprocess the image data collected from the filter production line, and perform noise suppression and image enhancement on the video frames;

[0006] S2. Image feature extraction is performed based on the improved YOLOv8 network. The improvement includes introducing a convolutional block attention module (CBAM) and depthwise separable convolutions into the backbone network CSP-Backbone.

[0007] S3. Construct a dual-branch feature extraction structure that includes texture branches and edge branches, and fuse the two types of extracted features;

[0008] S4. Introduce the Transformer module into the neck network to perform global interactive modeling of the fused features at different scales;

[0009] S5. For suspicious areas with confidence levels below the set threshold in the initial detection, perform local magnification and secondary detection;

[0010] S6. Combining Long Short-Term Memory (LSTM) network temporal modeling with multi-target tracking algorithm, the dynamic evolution of detected defects during the filter movement process is tracked and predicted.

[0011] Preferably, step S1 preprocesses the image data collected from the filter production line and performs noise suppression and image enhancement on the video frames, specifically including:

[0012] The raw image data collected from the filter tip production line was normalized using mean-standard deviation.

[0013] Image data is subjected to distortion correction, and barrel and pincushion distortion caused by wide-angle lenses, fisheye lenses and production line vibrations are corrected using camera calibration parameters;

[0014] The scale and resolution of the image are adjusted by using an affine transformation matrix to adapt to the standard input size of the network model and to achieve alignment of images from different cameras or different production batches.

[0015] Gaussian filtering is used to suppress noise in consecutive video frames, and affine rotation is used to enhance the image.

[0016] Preferably, the improvement in step S2 specifically includes:

[0017] (1) A convolutional block attention module (CBAM) is introduced into the YOLOv8 backbone network CSP-Backbone. CBAM adaptively weights the importance of different channels through the channel attention mechanism and enhances the salient regions of the feature map in the spatial dimension using the spatial attention mechanism.

[0018] (2) Use depthwise separable convolution to replace some of the standard convolution operations in CSP-Backbone.

[0019] Preferably, step S3 constructs a dual-branch feature extraction structure including texture branches and edge branches, and fuses the extracted two types of features, specifically including:

[0020] (1) Construct texture branches to extract fine texture information on the surface of the filter tip;

[0021] (2) Construct edge branches to extract edge features of cracks, gaps, and geometric anomalies;

[0022] (3) Features are extracted simultaneously at different levels and different modalities through texture branching and edge branching, and then the two types of features are organically integrated through the feature fusion module.

[0023] Preferably, step S4 introduces a Transformer module into the neck network to perform global interactive modeling of the fused features at different scales, specifically including:

[0024] (1) Align the channels and scales of feature maps from different scales and serialize them into a token sequence containing position encoding;

[0025] (2) The token sequence is mapped to a query matrix Q, a key matrix K, and a value matrix V through a linear transformation;

[0026] (3) Based on the Q, K, and V matrices, multi-head self-attention calculation is performed in windowing or grouping mode, and combined with residual connection, layer normalization and lightweight feedforward network to obtain a stable output enhanced feature representation;

[0027] (4) The enhanced feature representation is rearranged back into the spatial layout and then fused across scales in both directions before being output to the detection head.

[0028] Preferably, step S5 involves local magnification and secondary detection of suspicious areas with confidence levels below a set threshold in the initial detection, specifically including:

[0029] (1) Perform preliminary detection on the input image to obtain candidate boxes and their confidence scores, and obtain preliminary detection results through non-maximum suppression;

[0030] (2) From the preliminary detection results, select the suspected regions with confidence levels between the preset low threshold and high threshold;

[0031] (3) The selected suspected regions are scaled up and affine cropped, and then resampled into ROI image blocks of uniform resolution;

[0032] (4) Perform secondary detection on each ROI image block to obtain refined detection results;

[0033] (5) The refined detection results obtained from the secondary detection are mapped back to the original image coordinate system through inverse affine transformation;

[0034] (6) The mapped secondary detection results are fused with the preliminary detection results, and the detection results of the same target are integrated using a score weighting strategy. Finally, a stable final detection result is output after global non-maximum suppression.

[0035] Preferably, step S6 combines Long Short-Term Memory (LSTM) network temporal modeling with a multi-target tracking algorithm to track and predict the dynamic evolution of detected defects during the filter nozzle's movement, specifically including:

[0036] (1) Extract the appearance feature vectors corresponding to the candidate defect regions in each frame of the image to form an observation sequence;

[0037] (2) Kalman filtering is used to predict and update the motion state of the defect, and the cost matrix is ​​constructed by fusing the position observation information and the appearance feature vector;

[0038] (3) The cost matrix is ​​optimized by the Hungarian matching algorithm to complete cross-frame data association in order to maintain the trajectory continuity of the same defect;

[0039] (4) Input the spatial state sequence and appearance feature sequence of the trajectory that has been associated with the data into the Long Short-Term Memory (LSTM) network;

[0040] (5) Use the LSTM to capture the dynamic evolution trend of defects in consecutive frames and output the state prediction results and defect risk scores for several future frames;

[0041] (6) By jointly optimizing the loss function, constraints are imposed on location regression, evolution probability and identity consistency, forming an integrated modeling framework for detection, tracking and prediction.

[0042] Meanwhile, this invention proposes an intelligent detection and prediction system for filter tip defects, comprising:

[0043] The image preprocessing module is configured to perform the following processes: preprocessing image data acquired from the filter production line and performing noise suppression and image enhancement on video frames;

[0044] The feature extraction and enhancement module is configured to perform the following process: image feature extraction based on an improved YOLOv8 network, the improvement including the introduction of a convolutional block attention module CBAM and depthwise separable convolutions in the backbone network CSP-Backbone;

[0045] The multi-feature fusion module is configured to perform the following process: construct a two-branch feature extraction structure that includes texture branches and edge branches, and fuse the extracted two types of features;

[0046] The global feature modeling module is configured to perform the following process: introduce the Transformer module into the neck network to perform global interactive modeling of the fused features at different scales;

[0047] The refined detection module is configured to perform the following process: for suspicious areas with a confidence level below a set threshold in the initial detection, local magnification and secondary detection are performed;

[0048] The timing prediction module is configured to perform the following process: combining LSTM timing modeling with a multi-target tracking algorithm to track and predict the dynamic evolution of detected defects during the movement of the filter tip.

[0049] Furthermore, the present invention proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed, it implements the steps of the method described in the present invention.

[0050] Finally, the present invention provides a computer-readable storage medium having a computer program stored thereon, the computer program being configured to implement the steps of the method described in the present invention when invoked by a processor.

[0051] The intelligent detection and prediction method and system for filter tip defects described in this invention, compared with the prior art, has the following technical advantages:

[0052] (1) This invention significantly improves the ability to perceive and detect minute defects on the filter surface by introducing the Convolutional Block Attention Module (CBAM) and depthwise separable convolution. CBAM adaptively enhances defect-related features and suppresses background interference through a dual attention mechanism, while depthwise separable convolution greatly reduces computational complexity while maintaining accuracy, enabling the system to achieve a balance between high accuracy and low latency in a high-speed pipeline environment.

[0053] (2) This invention effectively enhances the robustness of identifying defects in complex shapes by constructing a dual-branch feature extraction structure of texture and edge and combining it with the Transformer module for multi-scale feature fusion. The texture branch captures subtle differences on the surface, the edge branch strengthens the perception of geometric anomalies, and the Transformer achieves cross-scale feature interaction through global context modeling, enabling the system to maintain stable performance in complex scenes such as weak texture and low contrast.

[0054] (3) The present invention adopts a ROI-based local magnification and secondary detection mechanism, which successfully solves the problem of identifying extremely subtle defects. This strategy performs scale magnification and fine re-inspection on low-confidence suspected areas. Without significantly increasing the computational load, it significantly improves the ability to capture tiny defects through "local close-up" analysis, achieving the best balance between detection accuracy and efficiency.

[0055] (4) This invention achieves a technological leap from static detection to dynamic prediction by integrating LSTM time-series modeling and multi-target tracking algorithms. The system maintains the continuity of defect trajectories through Kalman filtering and Hungarian algorithm, captures dynamic evolution trends using LSTM network, and outputs future state predictions and risk scores, providing key technical support for production line quality trend analysis and predictive maintenance. Attached Figure Description

[0056] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0057] Figure 1 This invention relates to a flowchart of an intelligent detection and prediction method for filter defects.

[0058] Figure 2 This invention relates to an integrated process for detecting, tracking, and predicting filter defects. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] Example 1: This example illustrates the specific implementation process of an intelligent detection and prediction method for filter tip defects. (Refer to...) Figure 1 The steps are as follows:

[0061] S1. Perform mean-standard deviation normalization on the raw image data collected from the filter production line; perform distortion correction on the image data, using camera calibration parameters to correct barrel and pincushion distortion caused by wide-angle lenses, fisheye lenses, and production line vibrations; adjust the scale and resolution of the images through affine transformation matrices to adapt to the standard input size of the network model and achieve alignment of images from different cameras or different production batches; use Gaussian filtering to suppress noise in consecutive video frames and enhance the images through affine rotation.

[0062] Specifically, the raw image data collected from the filter tip production line is first subjected to mean-standard deviation normalization:

[0063]

[0064]

[0065] in, Indicates the input image in pixels Location, passage The original grayscale value, Represents the set of pixels in an image. Indicates channel The average brightness, Indicates channel standard deviation This represents the normalized pixel value. Next, distortion correction is performed on the image, using the intrinsic parameter matrix. Remapping image coordinates using distortion parameters:

[0066]

[0067] in, and These are the focal lengths in pixel coordinates; and These are the coordinates of the optical center. Subsequently, the image is scaled and its resolution is adjusted using an affine transformation matrix:

[0068]

[0069] ,

[0070] in, Represents the pixel coordinates after affine transformation. Indicates the horizontal and vertical scaling factors. Indicates the translation component. This represents the shearing factor in the affine matrix. Meanwhile, Gaussian filtering is applied to consecutive video frames to suppress noise, and affine rotation is used to enhance the image.

[0071]

[0072]

[0073] ,

[0074] in, After noise reduction processing, in pixels Pixel value at that location, Represents the relative coordinates of the convolution kernel. This represents the squared Euclidean distance between neighboring pixels and the center pixel. The standard deviation of the Gaussian kernel is represented. This represents the new pixel coordinates after the rotation transformation. Indicates the rotation angle. Represents a two-dimensional rotation matrix. This represents the translation matrix.

[0075] S2. Image feature extraction is performed based on an improved YOLOv8 network. The improvement includes introducing a convolutional block attention module (CBAM) and depthwise separable convolutions into the backbone network CSP-Backbone. Specifically:

[0076]

[0077]

[0078] in, Indicates the spatial location of the input feature map With channel The value of , Represents the global average feature and global maximum feature of the channel. Indicates channel attention weights. For learnable parameters, Represents the ReLU activation function. This represents the Sigmoid function. This represents the feature map after channel attention adjustment. Indicates spatial location The channel average characteristics Indicates spatial location The most prominent feature of the channel, Represents a spatial attention mask. This represents the feature enhancement map after SVA. Simultaneously, depthwise separable convolution is used to replace some standard convolution operations:

[0079]

[0080]

[0081] in, For depthwise convolution kernels, For pointwise convolution kernels, Indicates the output channel index. This represents intermediate features after depthwise convolution. This indicates the characteristics of the final output.

[0082] S3. Construct a two-branch feature extraction structure containing texture and edge branches, and fuse the extracted two types of features, specifically including:

[0083] (1) Construct texture branches to extract fine texture information on the surface of the filter tip;

[0084] (2) Construct edge branches to extract edge features of cracks, gaps, and geometric anomalies;

[0085] (3) Features are extracted simultaneously at different levels and different modalities through texture branching and edge branching, and then the two types of features are organically integrated through the feature fusion module.

[0086] Specifically, in the texture branch, local texture features are extracted through multi-scale convolution:

[0087]

[0088]

[0089] in, Represents a set of different void ratios. This represents the activation function. Indicates the kernel size as The void ratio is Convolution operation, Indicates the void ratio Texture features are extracted through convolution. In the edge branches, edge information is extracted using gradient operators.

[0090]

[0091]

[0092]

[0093] in, These represent the Sobel convolution kernels in the horizontal and vertical directions, respectively. These represent the edge responses in the horizontal and vertical directions, respectively. Indicates gradient magnitude characteristics, This represents the output of the edge branch. Subsequently, the two types of features are organically integrated through the feature fusion module:

[0094]

[0095] in, This represents the feature map obtained after fusion.

[0096] S4. Introduce a Transformer module into the neck network to perform global interaction modeling of fused features at different scales. This includes:

[0097] (1) Align the channels and scales of feature maps from different scales and serialize them into a token sequence containing position encoding;

[0098] (2) The token sequence is mapped to a query matrix Q, a key matrix K, and a value matrix V through a linear transformation;

[0099] (3) Based on the Q, K, and V matrices, multi-head self-attention calculation is performed in windowing or grouping mode, and combined with residual connection, layer normalization and lightweight feedforward network to obtain a stable output enhanced feature representation;

[0100] (4) The enhanced feature representation is rearranged back into the spatial layout and then fused across scales in both directions before being output to the detection head.

[0101] Specifically, a lightweight Transformer module is introduced in the Neck stage. It first performs channel alignment on feature maps from different scales through linear projection, unfolds them into a serialized token sequence, and then adds position encoding.

[0102]

[0103] in, This represents an input feature sequence with positional encoding. This represents the positional encoding. Subsequently, a linear transformation is used to map the input features into a query matrix, a key matrix, and a value matrix:

[0104]

[0105] in, Represents the learnable weight matrix. These represent the query, key, and value matrices, respectively. Multi-head self-attention is performed in windowed or grouped modes, and a stable output is obtained by combining residual connections, normalization, and a lightweight feedforward network.

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] in, Represents the correlation matrix. The dimension of the attention subspace is represented. Indicates attention weights. Represents the normalization function. This indicates single-head attention output. This indicates multi-head attention output. Represents the learnable weight matrix. These represent the intermediate and final feature representations of the Transformer module, respectively. Represents the regularization function. This represents the feedforward network. Finally, the special diagnosis representations are rearranged back into the spatial layout and fused bidirectionally across scales:

[0112]

[0113] in, This represents a cross-scale bidirectional fusion operator. This represents the final enhanced feature map after passing through the Transformer module and cross-scale bidirectional fusion.

[0114] S5. For suspicious areas with confidence levels below the set threshold in the initial detection, perform local magnification and secondary detection:

[0115] (1) Perform preliminary detection on the input image to obtain candidate boxes and their confidence scores, and obtain preliminary detection results through non-maximum suppression;

[0116] (2) From the preliminary detection results, select the suspected regions with confidence levels between the preset low threshold and high threshold;

[0117] (3) The selected suspected regions are scaled up and affine cropped, and then resampled into ROI image blocks of uniform resolution;

[0118] (4) Perform secondary detection on each ROI image block to obtain refined detection results;

[0119] (5) The refined detection results obtained from the secondary detection are mapped back to the original image coordinate system through inverse affine transformation;

[0120] (6) The mapped secondary detection results are fused with the preliminary detection results, and the detection results of the same target are integrated using a score weighting strategy. Finally, a stable final detection result is output after global non-maximum suppression.

[0121] Specifically, reasoning is first performed on the input image to obtain candidate boxes and their confidence scores:

[0122]

[0123] in, Indicates the input image. As the first-stage detector, These represent the candidate bounding box, confidence level, and category, respectively. This represents the set of candidate boxes output by the first-stage detector. Preliminary detection results are obtained through non-maximum suppression.

[0124]

[0125] in, Denotes the candidate set after nonmaximum suppression. This represents the non-maximum suppression operator. Based on this, suspected regions with confidence levels within a set threshold range are selected:

[0126]

[0127] in, Indicates the trigger threshold. Indicates the upper limit of candidates, This represents the set of all candidate boxes whose confidence scores fall within a specified threshold range. This represents the K most likely suspected regions selected from the pool. Subsequently, these regions are scaled up and affine cropped.

[0128]

[0129]

[0130] in, Indicates the magnification factor. Candidate boxes The coordinates of the top left corner Candidate boxes Width and height, This represents the width and height of the scaled candidate region. The coordinates of the top-left corner of the scaled candidate region. Next, a second detection is performed on these regions:

[0131]

[0132] in, This indicates the second-stage detector. Indicates the first Candidate ROI image patches, This represents the set of secondary detection results. This indicates the number of candidate targets output by the second detector. The results of the second detection are then mapped back to the original image coordinate system using an inverse affine transformation:

[0133]

[0134] in, This represents the affine matrix from the original image to the ROI. The inverse affine back mapping operator of the bounding box. After inverse affine transformation, the candidate bounding boxes are mapped back to the original image coordinate system and then fused with the preliminary detection results.

[0135]

[0136] in, This represents the fused candidate bounding box set. Detection results for the same target are integrated using a score-weighted approach.

[0137]

[0138]

[0139] in, Indicates the score fusion weight, This represents the final confidence score after fusion. This represents the final set of detection boxes after NMS is executed.

[0140] S6. Combining Long Short-Term Memory (LSTM) network temporal modeling with multi-target tracking algorithms, the dynamic evolution of detected defects during filter movement is tracked and predicted, including:

[0141] (1) Extract the appearance feature vectors corresponding to the candidate defect regions in each frame of the image to form an observation sequence;

[0142] (2) Kalman filtering is used to predict and update the motion state of the defect, and the cost matrix is ​​constructed by fusing the position observation information and the appearance feature vector;

[0143] (3) The cost matrix is ​​optimized by the Hungarian matching algorithm to complete cross-frame data association in order to maintain the trajectory continuity of the same defect;

[0144] (4) Input the spatial state sequence and appearance feature sequence of the trajectory that has been associated with the data into the Long Short-Term Memory (LSTM) network;

[0145] (5) Use the LSTM to capture the dynamic evolution trend of defects in consecutive frames and output the state prediction results and defect risk scores for several future frames;

[0146] (6) By jointly optimizing the loss function, constraints are imposed on location regression, evolution probability and identity consistency, forming an integrated modeling framework for detection, tracking and prediction.

[0147] Specifically, the appearance feature vector corresponding to the candidate defect region is first extracted from each frame of the image:

[0148]

[0149] in, No. Frame number The appearance embedding vector of each detected target This represents the appearance feature extraction function. Indicates the first Frame image. Subsequently, Kalman filtering is used to predict and update the motion state of the defect:

[0150]

[0151]

[0152] in, This indicates the number of the same defect in cross-frame tracking. Indicates the first Frame trajectory The state of motion, Represents the state transition matrix. Represents the observation matrix. This represents process noise and observation noise. This represents the process noise covariance and the observation noise covariance. This represents the prior state predicted from the state of the previous frame to the state of the current frame. Let these represent the prior and posterior state covariance matrices, respectively. The gain matrix represents the injection of the observed residuals into the prior state. Represents the identity matrix. The cost matrix is ​​constructed by combining location observations and appearance embeddings:

[0153]

[0154] in, Indicates by The predicted bounding box obtained by projection, Representing the trajectory Predicted appearance features Represents the cost matrix, Represented as a trajectory With detection The matching cost, This represents the weighting factor between location and appearance costs. This represents the intersection-union ratio of the two boxes. Cosine similarity is used to represent appearance features. Matching is achieved by minimizing this cosine similarity.

[0155]

[0156] Next, the spatial state and appearance feature sequences of the matched trajectories are input into the LSTM:

[0157]

[0158]

[0159] in, These represent the input gate, forget gate, and output gate of the LSTM, respectively. These represent the candidate cell state and the cell state, respectively. Indicates a hidden state. This represents the weights and biases of each gate in the LSTM. This represents the Sigmoid activation function. Indicates hyperbolic tangent. Indicates will Linear layer parameters mapped to future states. Indicates the future Frame state prediction Indicates will The linear layer parameters are mapped to risk scores. This represents the defect evolution risk / probability score. Finally, joint optimization of position regression, evolution probability, and identity consistency is achieved through loss function constraints.

[0160]

[0161] in, Represents the total loss function. These represent the weighting coefficients of the three losses: location regression, risk classification, and identity consistency, respectively. Represents the binary cross-entropy loss. This indicates an optional appearance redesign.

[0162] Finally, Precision, Recall, And evaluate the detection model based on real-time inference rate:

[0163]

[0164]

[0165] in, Indicates accuracy. Indicates recall rate, Indicates a true positive result. This indicates a false positive. This indicates a false negative. The cumulative reliability curve (PR curve) is set from high to low as follows: ,but:

[0166]

[0167] in, Recall is the independent variable. Indicates in category Threshold The precision function varies with recall. Indicate category At the threshold The average accuracy is as follows.

[0168]

[0169] in, express The average accuracy value.

[0170]

[0171] in, This indicates the total number of frames used in the evaluation. Indicates the total reasoning time. Indicates frame rate, This indicates the delay of a single frame.

[0172]

[0173] in, It represents the harmonic average of precision and recall, used for comprehensive comparison under a single threshold.

[0174] The integrated filter defect detection, tracking, and prediction process in this embodiment is as follows: Figure 2 As shown.

[0175] Example 2: This example proposes an intelligent detection and prediction system for filter defects, including:

[0176] The image preprocessing module is configured to perform the following processes: preprocessing image data acquired from the filter production line and performing noise suppression and image enhancement on video frames;

[0177] The feature extraction and enhancement module is configured to perform the following process: image feature extraction based on an improved YOLOv8 network, the improvement including the introduction of a convolutional block attention module CBAM and depthwise separable convolutions in the backbone network CSP-Backbone;

[0178] The multi-feature fusion module is configured to perform the following process: construct a two-branch feature extraction structure that includes texture branches and edge branches, and fuse the extracted two types of features;

[0179] The global feature modeling module is configured to perform the following process: introduce the Transformer module into the neck network to perform global interactive modeling of the fused features at different scales;

[0180] The refined detection module is configured to perform the following process: for suspicious areas with a confidence level below a set threshold in the initial detection, local magnification and secondary detection are performed;

[0181] The timing prediction module is configured to perform the following process: combining LSTM timing modeling with a multi-target tracking algorithm to track and predict the dynamic evolution of detected defects during the movement of the filter tip.

[0182] Example 3: This example proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the steps of the method described in this invention.

[0183] Example 4: This example proposes a computer-readable storage medium storing a computer program configured to implement the steps of the method described in this invention when invoked by a processor.

[0184] It should be noted that the processing flow of embodiments 2-4 corresponds to the specific steps of the method provided in embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the method provided in embodiment 1 of the present invention.

[0185] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0186] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for intelligent detection and prediction of filter tip defects, characterized in that, Includes the following steps: S1. Preprocess the image data collected from the filter production line, and perform noise suppression and image enhancement on the video frames; S2. Image feature extraction is performed based on the improved YOLOv8 network. The improvement includes introducing a convolutional block attention module (CBAM) and depthwise separable convolutions into the backbone network CSP-Backbone. S3. Construct a dual-branch feature extraction structure that includes texture branches and edge branches, and fuse the two types of extracted features; S4. Introduce the Transformer module into the neck network to perform global interactive modeling of the fused features at different scales; S5. For suspicious areas with confidence levels below the set threshold in the initial detection, perform local magnification and secondary detection; S6. Combining Long Short-Term Memory (LSTM) network temporal modeling with multi-target tracking algorithm, the dynamic evolution of detected defects during the filter movement process is tracked and predicted.

2. The intelligent detection and prediction method for filter tip defects according to claim 1, characterized in that, Step S1 preprocesses the image data acquired from the filter production line, and performs noise suppression and image enhancement on the video frames, specifically including: (1) Perform mean-standard deviation normalization on the raw image data collected from the filter production line; (2) Perform distortion correction on the image data and use camera calibration parameters to correct barrel distortion and pincushion distortion caused by wide-angle lens, fisheye lens and production line vibration; (3) Adjust the scale and resolution of the image through the affine transformation matrix to adapt to the standard input size of the network model and achieve alignment of images from different cameras or different production batches; (4) Gaussian filtering is used to suppress noise in consecutive video frames, and affine rotation is used to enhance the image.

3. The intelligent detection and prediction method for filter tip defects according to claim 1, characterized in that, The improvements described in step S2 specifically include: (1) A convolutional block attention module (CBAM) is introduced into the YOLOv8 backbone network CSP-Backbone. CBAM adaptively weights the importance of different channels through the channel attention mechanism and enhances the salient regions of the feature map in the spatial dimension using the spatial attention mechanism. (2) Use depthwise separable convolution to replace some of the standard convolution operations in CSP-Backbone.

4. The intelligent detection and prediction method for filter tip defects according to claim 1, characterized in that, Step S3 constructs a dual-branch feature extraction structure containing texture and edge branches, and fuses the extracted two types of features, specifically including: (1) Construct texture branches to extract fine texture information on the surface of the filter tip; (2) Construct edge branches to extract edge features of cracks, gaps, and geometric anomalies; (3) Features are extracted simultaneously at different levels and different modalities through texture branching and edge branching, and then the two types of features are organically integrated through the feature fusion module.

5. The intelligent detection and prediction method for filter tip defects according to claim 1, characterized in that, Step S4 introduces a Transformer module into the neck network to perform global interactive modeling of the fused features at different scales, specifically including: (1) Align the channels and scales of feature maps from different scales and serialize them into a token sequence containing position encoding; (2) The token sequence is mapped to a query matrix Q, a key matrix K, and a value matrix V through a linear transformation; (3) Based on the Q, K, and V matrices, multi-head self-attention calculation is performed in windowing or grouping mode, and combined with residual connection, layer normalization and lightweight feedforward network to obtain a stable output enhanced feature representation; (4) The enhanced feature representation is rearranged back into the spatial layout and then fused across scales in both directions before being output to the detection head.

6. The intelligent detection and prediction method for filter tip defects according to claim 1, characterized in that, Step S5 involves local magnification and secondary detection of suspicious areas with confidence levels below a set threshold in the initial detection. Specifically, this includes: (1) Perform preliminary detection on the input image to obtain candidate boxes and their confidence scores, and obtain preliminary detection results through non-maximum suppression; (2) From the preliminary detection results, select the suspected regions with confidence levels between the preset low threshold and high threshold; (3) The selected suspected regions are scaled up and affine cropped, and then resampled into ROI image blocks of uniform resolution; (4) Perform secondary detection on each ROI image block to obtain refined detection results; (5) The refined detection results obtained from the secondary detection are mapped back to the original image coordinate system through inverse affine transformation; (6) The mapped secondary detection results are fused with the preliminary detection results, and the detection results of the same target are integrated using a score weighting strategy. Finally, a stable final detection result is output after global nonmaximum suppression.

7. The intelligent detection and prediction method for filter tip defects according to claim 1, characterized in that, Step S6 combines Long Short-Term Memory (LSTM) network temporal modeling with a multi-target tracking algorithm to track and predict the dynamic evolution of detected defects during the filter nozzle's movement. Specifically, this includes: (1) Extract the appearance feature vectors corresponding to the candidate defect regions in each frame of the image to form an observation sequence; (2) Kalman filtering is used to predict and update the motion state of the defect, and the cost matrix is ​​constructed by fusing the position observation information and the appearance feature vector; (3) The cost matrix is ​​optimized by the Hungarian matching algorithm to complete cross-frame data association in order to maintain the trajectory continuity of the same defect; (4) Input the spatial state sequence and appearance feature sequence of the trajectory that has been associated with the data into the Long Short-Term Memory (LSTM) network; (5) Use the LSTM to capture the dynamic evolution trend of defects in consecutive frames and output the state prediction results and defect risk scores for several future frames; (6) By jointly optimizing the loss function, constraints are imposed on location regression, evolution probability and identity consistency, forming an integrated modeling framework for detection, tracking and prediction.

8. A smart detection and prediction system for filter tip defects, characterized in that, The system includes: The image preprocessing module is configured to perform the following processes: preprocessing image data acquired from the filter production line and performing noise suppression and image enhancement on video frames; The feature extraction and enhancement module is configured to perform the following process: image feature extraction based on an improved YOLOv8 network, the improvement including the introduction of a convolutional block attention module CBAM and depthwise separable convolutions in the backbone network CSP-Backbone; The multi-feature fusion module is configured to perform the following process: construct a two-branch feature extraction structure that includes texture branches and edge branches, and fuse the extracted two types of features; The global feature modeling module is configured to perform the following process: introduce the Transformer module into the neck network to perform global interactive modeling of the fused features at different scales; The refined detection module is configured to perform the following process: for suspicious areas with a confidence level below a set threshold in the initial detection, local magnification and secondary detection are performed; The timing prediction module is configured to perform the following process: combining LSTM timing modeling with a multi-target tracking algorithm to track and predict the dynamic evolution of detected defects during the movement of the filter tip.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is configured to implement the steps of the method according to any one of claims 1 to 7 when invoked by a processor.

Images