Electronic cigarette middle frame surface defect detection method based on improved RT-DETR

Abstract

The invention discloses an electronic cigarette middle frame surface defect detection method based on improved RT-DETR, and the method comprises the steps: S1, obtaining an electronic cigarette middle frame surface defect image, and carrying out the preprocessing of the image; s2, constructing an improved RT-DETR target detection model, and introducing a partial convolution and gating feature fusion module PCGF in a backbone network of the RT-DETR model to replace an original convolution feature extraction unit; in an encoder structure of the RT-DETR model, a multi-scale sequence fusion enhancement module ESSF3D is introduced; in a target regression stage of the RT-DETR model, a target regression loss function perceived by a difficult sample is introduced; s3, model training; and S4, detecting the surface defect image of the middle frame of the electronic cigarette by using the trained improved RT-DETR target detection model to obtain a defect detection result. Through the improvement of a backbone network structure, the optimization of a feature fusion module and the design of a loss function, the high-precision detection of tiny defects such as scratches, pits, dust and gaps is realized, and the light weight and the real-time performance of the model are considered at the same time.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a method for detecting surface defects in the middle frame of electronic cigarettes based on an improved RT-DETR. Background Technology

[0002] The appearance quality of e-cigarette products is a key factor for consumers. However, the e-cigarette frame is prone to minor defects such as scratches, dents, dust, and gaps during production and assembly. If these defects are not detected in time, they will directly affect product quality and user experience. Traditional manual inspection methods are inefficient and susceptible to subjective factors, making them unsuitable for the high-efficiency inspection requirements of industrial production lines.

[0003] In recent years, with the rapid development of computer vision and deep learning technologies, object detection algorithms have been widely used in industrial surface defect detection. Algorithms such as YOLO, Faster-RCNN, and DETR can achieve automated detection, significantly improving detection efficiency. However, existing algorithms still have several shortcomings when dealing with tiny defects in the frame of e-cigarettes:

[0004] 1. Insufficient small target detection capability: Tiny scratches and dents account for a very small proportion in the original image and are easily ignored by the backbone network;

[0005] 2. High model complexity and insufficient real-time performance: The standard RT-DETR (Real-Time DetectionTransformer) and DETR series models have a large number of parameters, which are not suitable for the real-time detection requirements of the production line;

[0006] 3. Significant impact from changes in lighting and materials: The surface of the e-cigarette frame is reflective, and the metal material and different lighting conditions cause traditional algorithms to lose detection accuracy in complex environments;

[0007] 4. Imbalanced defect sample problem: The large difference in the number of different defect categories can easily cause model training bias.

[0008] Therefore, there is an urgent need for a small-target, lightweight, and real-time method for detecting defects on the surface of the e-cigarette frame, in order to meet the requirements of efficient and automated inspection on the production line. Summary of the Invention

[0009] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a method for detecting surface defects in the middle frame of electronic cigarettes based on an improved RT-DETR.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A method for detecting surface defects in the middle frame of an electronic cigarette based on an improved RT-DETR includes the following steps:

[0012] S1. Obtain images of surface defects on the inner frame of the electronic cigarette and preprocess the images;

[0013] S2. Construct an improved RT-DETR target detection model. In the backbone network of the RT-DETR model, introduce a partial convolutional and gated feature fusion module PCGF to replace the original convolutional feature extraction unit. In the encoder structure of the RT-DETR model, introduce a multi-scale sequence fusion enhancement module ESSF3D. In the target regression stage of the RT-DETR model, introduce a target regression loss function that is sensitive to difficult samples.

[0014] S3. The improved RT-DETR target detection model is trained using surface defect images of the electronic cigarette frame.

[0015] S4. Using the improved RT-DETR target detection model that has been trained, the surface defect image of the electronic cigarette frame is detected to obtain the defect detection results.

[0016] The present invention also includes an electronic cigarette frame surface defect detection system based on improved RT-DETR. The system adopts the electronic cigarette frame surface defect detection method provided by the present invention. The system includes an image acquisition and preprocessing module, a target detection model construction module, a target detection model training module, and a defect detection output module.

[0017] The image acquisition and preprocessing module is used to acquire images of surface defects in the frame of the electronic cigarette and to preprocess the images.

[0018] The target detection model building module is used to construct an improved RT-DETR target detection model. In the backbone network of the RT-DETR model, a partial convolutional and gated feature fusion module PCGF is introduced to replace the original convolutional feature extraction unit. In the encoder structure of the RT-DETR model, a multi-scale sequence fusion enhancement module ESSF3D is introduced. In the target regression stage of the RT-DETR model, a target regression loss function that is aware of difficult samples is introduced.

[0019] The target detection model training module is used to train the improved RT-DETR target detection model using surface defect images of the electronic cigarette frame.

[0020] The defect detection output module is used to detect surface defects in the frame of an electronic cigarette using a trained and improved RT-DETR target detection model, and output the defect detection results.

[0021] The present invention also includes a computer-readable storage medium storing a program that, when executed by a processor, implements the method for detecting surface defects in the middle frame of an electronic cigarette provided by the present invention.

[0022] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0023] 1. To enhance the perception of minute defect features, this invention replaces the original basic convolutional module of RT-DETR with a partial convolutional and gated fusion module, PCGF (Partial Convolution Gated Fusion). The PCGF module combines partial convolution (PConv) and gated activation (Gated Linear Unit, GLU) to achieve key feature enhancement and cost-effective channel bypass. The PCGF module can highlight the local details of minute defects on the surface of the e-cigarette frame while maintaining a lightweight design, improving the backbone network's ability to represent small targets.

[0024] 2. To further enhance the representation of minute defects on the surface of the electronic cigarette frame in a multi-scale feature space, this invention introduces a multi-scale sequence fusion enhancement module ESSF3D in the feature fusion stage of the RT-DETR model. This module takes the multi-scale features output by the backbone network as input, and achieves efficient interaction and joint modeling between features of different scales through scale-related feature enhancement, cross-scale serialization representation and three-dimensional convolution operation, thereby enhancing the model's ability to identify small target defects and complex differential features.

[0025] 3. By introducing the Focaler concept into the GIoU loss function, this invention effectively alleviates the problem of insufficient regression accuracy in the detection of surface defects in electronic cigarette frames due to the small size of defects and the high difficulty of samples. It enables the model to pay more attention to the localization error of small defect targets during training, thereby significantly improving the bounding box prediction accuracy and overall detection performance. Attached Figure Description

[0026] Figure 1 This is a flowchart of the method of the present invention.

[0027] Figure 2 This is a schematic diagram of the overall network structure of the RT-DETR baseline model in the embodiment.

[0028] Figure 3 This is a schematic diagram of the structure of the partial convolution and gating fusion module proposed in this invention.

[0029] Figure 4 This is a schematic diagram of the structure of the multi-scale sequence fusion enhancement module proposed in this invention.

[0030] Figure 5 This is a schematic diagram of the network structure of the improved RT-DETR model in the embodiment.

[0031] Figure 6 This is an example image of a dataset of surface defects in the frame of an electronic cigarette.

[0032] Figure 7 This is a schematic diagram comparing the detection results of the improved model and the benchmark model in the electronic cigarette frame surface defect detection task.

[0033] Figure 8 This is a block diagram of the system according to an embodiment of the present invention. Detailed Implementation

[0034] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0035] Explanation of relevant terms:

[0036] RT-DETR: RT-DETR (Real-Time Detection Transformer) is a real-time object detection model based on the Transformer architecture. It performs object location regression and category prediction in an end-to-end manner. This model adopts a query-driven mechanism, avoiding the complex candidate box generation and post-processing processes in traditional object detection. It can meet the real-time inference requirements while maintaining detection accuracy, making it suitable for applications with high requirements for real-time performance and accuracy, such as industrial vision.

[0037] Focaler concept: The Focaler concept is an adaptive optimization idea for difficult samples. Its core lies in dynamically adjusting the contribution of the loss based on the localization error or prediction difficulty of the sample. This makes the model pay more attention to samples with large regression errors or inaccurate predictions during training, thereby suppressing the dominant role of easy samples in the overall training and improving the localization accuracy and robustness of the model in scenarios with small targets, ambiguous boundaries, and complex defects.

[0038] PCGF: The PCGF module is a feature fusion structure that combines partial convolution and gating mechanisms. By performing channel-wise convolution on the input features and introducing gating weights to adaptively adjust different feature branches, it enhances effective information and suppresses redundant information, thereby improving feature representation capabilities without significantly increasing computational load. It is especially suitable for fine-grained feature extraction tasks such as surface defects.

[0039] ESSF3D: The ESSF3D module is a multi-scale sequential feature fusion enhancement structure. By sequentially modeling and fusing features at different levels in spatial and scale dimensions, it achieves efficient interaction and context enhancement of cross-scale information. This module can improve the model's ability to perceive defective targets of different sizes, enhance feature consistency and discriminability, thereby improving detection stability and overall recognition performance in complex backgrounds.

[0040] Examples; such as Figure 1As shown, a method for detecting surface defects in the middle frame of an electronic cigarette based on an improved RT-DETR includes the following steps:

[0041] S1. Obtain images of surface defects on the inner frame of the electronic cigarette and preprocess the images;

[0042] like Figure 6 As shown in this embodiment, the surface defect images of the e-cigarette frame are obtained from an actual e-cigarette production line and are acquired by an industrial camera in a natural production environment. The acquisition process fully considers complex factors such as different lighting conditions, different shooting angles, and metal surface reflection, ensuring that the image dataset accurately reflects the appearance characteristics of the e-cigarette frame in an industrial inspection scenario.

[0043] The surface defects of the electronic cigarette frame mainly include scratches, dents, cracks, burrs, foreign matter adhesion defects, dust, and assembly gaps. These defects are generally characterized by small size, irregular shape, blurred edges, and low contrast with the background metal material, which makes automatic detection very difficult.

[0044] To improve the generalization ability of the detection model and its ability to perceive minute defects, data augmentation is performed on the original image data, including but not limited to: random cropping, random brightness adjustment, random rotation, mirror flipping, Gaussian noise perturbation (random Gaussian noise superposition), image stitching, and scale scaling. These data augmentation operations effectively expand the distribution space of defect samples and alleviate the problem of sample imbalance between different defect categories.

[0045] In addition, in step S1, all images are uniformly scaled to a preset size, such as 640×640 pixels, and the pixel values ​​are normalized to meet the input requirements of the subsequent network model.

[0046] Finally, the training set, validation set, and test set were divided in an 8:1:1 ratio.

[0047] S2. Construct an improved RT-DETR target detection model;

[0048] like Figure 5 The diagram shows the overall network structure of the improved RT-DETR object detection model. For comparison, the network structure of the baseline RT-DETR-ResNet18 model is shown below. Figure 2 As shown.

[0049] The baseline RT-DETR model uses ResNet18 as the backbone network and combines a Transformer encoder, a perceptual query selection module, and a decoder to achieve end-to-end object detection. Although this model achieves a good balance between real-time performance and accuracy in general object detection tasks, it still suffers from problems such as insufficient response to small defects, inadequate feature fusion, and limited accuracy of bounding box regression in the task of detecting surface defects in e-cigarette frames.

[0050] To address the aforementioned issues, this embodiment improves the model from three aspects—backbone feature extraction structure, multi-scale feature fusion method, and loss function design—while maintaining the overall RT-DETR detection framework, thereby enhancing the model's ability to detect minute defects on the surface of the e-cigarette frame.

[0051] S21. Improved backbone network feature extraction based on partial convolution and gating fusion module;

[0052] The RT-DETR-ResNet18 backbone network is improved by introducing a partial convolutional and gated fusion module (PCGF) into the original basic residual structure. Its structure is illustrated below. Figure 3 As shown, the PCGF module combines partial convolution (PConv) and gated linear unit (GLU) mechanisms to achieve key feature enhancement and inexpensive channel bypass.

[0053] For PCGF, the input feature map is split into two paths in the cross-stage local network: one path goes directly to the end of the module to retain low-cost, basic features; the other path enters the PCGF bottleneck module for deep feature processing, dynamically selecting important channels through partial convolutions and achieving feature enhancement through GLU gating; the specific process is as follows:

[0054] Let the input feature map be:

[0055] First, the channel dimension is divided into a subset of key channels. With the cheap channel subset ,in:

[0056]

[0057] Convolution is performed only on a subset of key channels, and part of the convolution process can be represented as follows:

[0058]

[0059] The number of channels is then aligned using 1×1 convolutions, and convolutionally gated linear units are introduced for feature modulation. The gate expression is as follows:

[0060]

[0061] in, This represents the Sigmoid activation function. This represents element-wise multiplication;

[0062] Finally, the gated enhanced features are concatenated with the features from the low-cost channel to obtain the output of the PCGF module:

[0063]

[0064] By introducing the PCGF module, the backbone network can adaptively enhance key feature channels related to defect regions without significantly increasing the number of parameters, effectively improving the representation ability of fine scratches and weak texture defects.

[0065] S22. Multi-scale feature fusion enhancement based on ESSF3D;

[0066] The original feature fusion structure of RT-DETR is improved by introducing a multi-scale sequence fusion enhancement module (ESSF3D, Enhanced Scale-Sequence Feature Fusion with 3D Convolution), as shown in the schematic diagram below. Figure 4 As shown.

[0067] The ESSF3D module takes multi-scale features output from the backbone network as input and achieves efficient interaction and joint modeling between features of different scales through scale-related feature enhancement, cross-scale sequential representation, and 3D convolution operations. This enhances the model's ability to identify defects and complex differences in small targets. The multi-scale sequence fusion enhancement module ESSF3D includes the following steps:

[0068] First, feature maps of different scales are selected as inputs from the backbone network. Let the feature maps from layers P2, P3, and P4 be denoted as follows: , ;

[0069] To enhance the discriminative power of features at different scales along the scale dimension, scale-dependent Gaussian convolution operations are introduced into the aforementioned feature maps to smooth them and adjust their response range. For any scale feature, the feature representation after Gaussian convolution is as follows:

[0070]

[0071] in, Represents the input feature mapping. The two-dimensional Gaussian kernel function is defined as follows:

[0072]

[0073] Among them, parameters As the feature scale increases progressively, it is used to characterize the smoothness of features at different scales. In this way, features at different scales obtain response characteristics with scale differences before entering the subsequent fusion stage, providing a more stable and discriminative input representation for cross-scale feature modeling.

[0074] Subsequently, to achieve sequential fusion of multi-scale features, the Nearest Neighbor Interpolation (NNI) method was used to uniformly adjust the multi-scale features after Gaussian convolution to the same spatial resolution as layer P2. After resolution alignment, the features at each scale were stacked in the scale dimension to construct a three-dimensional feature tensor.

[0075]

[0076] in, Let represent the k-th scale feature map after resolution alignment, and Stack represent the feature stacking operation in the scale dimension. For the constructed 3D feature tensor, 3D convolution is introduced to extract features, so as to explicitly model the correlation and difference between features at different scales. The calculation process is expressed as follows:

[0077]

[0078] in, Represents a three-dimensional convolution operation. This represents a feature mapping function composed of batch normalization and nonlinear activation functions (such as LeakyReLU); through three-dimensional convolution operations, the model can simultaneously perform feature modeling in both spatial and scale dimensions, thereby effectively learning the correlation between cross-scale defect features;

[0079] After completing the 3D convolution, the 3D max pooling operation is used to compress and integrate the features, compressing the scale dimension information back into the 2D feature representation, and outputting the fused 2D feature map as the input for the subsequent feature pyramid and decoder stages.

[0080] At the network structure level, the ESSF3D module is used to perform cross-scale fusion of multi-layer features, and its output is concatenated or added with the features obtained by the conventional upsampling path to enrich the feature hierarchy expression. Subsequently, the shallow features and fused features are fused again based on the ESSF3D module to generate feature representations that contain both deep semantic information and shallow detail information, which are used for target prediction in the decoder stage.

[0081] For ESSF3D, let the multi-scale feature map output by the backbone network be:

[0082]

[0083] in, It has a high resolution and contains rich local detail information. It contains stronger high-level semantic information;

[0084] First, the nearest neighbor interpolation method is used to align the resolution of features at different scales:

[0085]

[0086] Subsequently, the aligned features are stacked along the newly added scale dimension to form a three-dimensional feature representation:

[0087]

[0088] Perform 3D convolution operations on the above 3D features to model the correlation between features across scales:

[0089]

[0090] After batch normalization and nonlinear activation function processing, 3D max pooling is used to compress the scale dimension to obtain the final fused features:

[0091]

[0092] The ESSF3D module enables sequential modeling and joint representation of features at different scales, effectively compensating for the problem of insufficient cross-scale information utilization in small target detection by the traditional FPN structure, and significantly improving the model's response to small defects on the surface of the e-cigarette frame.

[0093] S23. Improvement of the loss function based on Focaler-GIoU;

[0094] The original bounding box regression loss function of RT-DETR is improved by introducing the Focaler idea to construct the Focaler-GIoU loss function.

[0095] GIoU loss, a commonly used bounding box regression loss in object detection, is defined as follows:

[0096]

[0097] The calculation method for GIoU is as follows:

[0098] ;

[0099] Where A represents the predicted bounding box, B represents the ground truth bounding box, and C represents the smallest bounding rectangle that simultaneously encloses A and B. This represents the area of ​​the corresponding region. Compared to traditional IoU, GIoU can still provide effective gradient information when the predicted bounding box and the ground truth bounding box do not overlap. However, during the training process, it still has the problem of paying too much attention to large target samples and not contributing enough to small defect samples.

[0100] To address the aforementioned issues, this invention introduces the Focaler concept, weighting the GIoU loss with hard samples, thus enabling the model to focus more on locating defects that are more difficult to locate and smaller in scale during training. Specifically, the Focaler-GIoU loss can be expressed as:

[0101]

[0102] in, These are weighting coefficients used to balance the proportion of regression loss in the total loss function; , is the Focaler exponent parameter, used to adjust the degree of attention given to difficult samples, amplifying the contribution of difficult samples with large localization errors to the total loss;

[0103] When the overlap between the predicted bounding box and the ground truth bounding box is high, i.e. the GIoU value is large, the loss weight is relatively reduced; when the overlap between the predicted bounding box and the ground truth bounding box is low and the regression is difficult, the loss weight is significantly amplified, thereby prompting the model to optimize for this type of sample more fully during the training process.

[0104] During the overall training of the model, the Focaler-GIoU loss, as the bounding box regression loss term, together with the classification loss, constitutes the total loss function of the detection model, which is expressed as follows:

[0105]

[0106] in, The classification loss represents the defect category. These are the weighting coefficients used to balance classification loss and regression loss.

[0107] By introducing the Focaler concept into the GIoU loss function, this invention effectively alleviates the problem of insufficient regression accuracy in the detection of surface defects in electronic cigarette frames caused by small defect scale and high sample difficulty. It enables the model to pay more attention to the localization error of small defect targets during training, thereby significantly improving the bounding box prediction accuracy and overall detection performance.

[0108] By designing this loss function, the model can pay more attention to small-sized defect targets that are difficult to locate during training, thereby effectively improving the accuracy of bounding box regression.

[0109] S3, Model Training;

[0110] An improved RT-DETR target detection model was trained using a dataset of surface defects in the frames of electronic cigarettes. The AdamW optimizer was used during training, with a learning rate of 0.0001, a batch size of 4, and 500 training epochs to allow the model to gradually converge.

[0111] During the training phase, the backbone features enhanced by the PCGF module and the multi-scale features fused with the ESSF3D module are input into the detection head, and the target regression and category classification are optimized by Focaler-GIoU loss.

[0112] After training, the test set images are input into the model, and the output results of surface defects in the electronic cigarette frame are displayed, including the defect category and bounding box coordinates.

[0113] S4, Defect detection output;

[0114] After training, the surface image of the e-cigarette frame to be detected is input into the trained target detection model. The model outputs the defect category information and corresponding bounding box coordinates through the Transformer decoder, realizing the automatic detection of surface defects in the e-cigarette frame. The ablation experiment results of each module and the defect detection effect of the model before and after improvement are compared in Table 1 below. Figure 7 As shown.

[0115] Table 1 Ablation experimental results of different improved modules

[0116]

[0117] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0118] Based on the same idea as the improved RT-DETR-based method for detecting surface defects in the electronic cigarette frame described in the above embodiments, this invention also provides an improved RT-DETR-based system for detecting surface defects in the electronic cigarette frame. This system can be used to perform the above-described method for detecting surface defects in the electronic cigarette frame. For ease of explanation, the schematic diagram of the embodiment of the electronic cigarette frame surface defect detection system only shows the parts related to the embodiments of this invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0119] like Figure 8As shown, the electronic cigarette frame surface defect detection system 100 based on improved RT-DETR includes an image acquisition and preprocessing module 101, a target detection model construction module 102, a target detection model training module 103, and a defect detection output module 104.

[0120] The image acquisition and preprocessing module 101 is used to acquire images of surface defects in the middle frame of an electronic cigarette and to preprocess the images.

[0121] The target detection model construction module 102 is used to construct an improved RT-DETR target detection model. In the backbone network of the RT-DETR model, a partial convolutional and gated feature fusion module PCGF is introduced to replace the original convolutional feature extraction unit. In the encoder structure of the RT-DETR model, a multi-scale sequence fusion enhancement module ESSF3D is introduced. In the target regression stage of the RT-DETR model, a target regression loss function that is aware of difficult samples is introduced.

[0122] The target detection model training module 103 is used to train the improved RT-DETR target detection model using surface defect images of the electronic cigarette frame.

[0123] The defect detection output module 104 is used to detect surface defects in the frame of an electronic cigarette using a trained and improved RT-DETR target detection model, and output the defect detection results.

[0124] It should be noted that the electronic cigarette frame surface defect detection system of the present invention corresponds one-to-one with the electronic cigarette frame surface defect detection method of the present invention. The technical features and beneficial effects described in the embodiments of the electronic cigarette frame surface defect detection method described above are applicable to the embodiments of the electronic cigarette frame surface defect detection system. For details, please refer to the description in the embodiments of the method of the present invention, which will not be repeated here.

[0125] Furthermore, in the above-described implementation of the electronic cigarette frame surface defect detection system, the logical division of each program module is merely an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or the convenience of software implementation. That is, the internal structure of the electronic cigarette frame surface defect detection system can be divided into different program modules to complete all or part of the functions described above.

[0126] In another embodiment, a computer-readable storage medium is provided storing a program that, when executed by a processor, implements the electronic cigarette frame surface defect detection method of the present invention, specifically:

[0127] Acquire images of surface defects on the inner frame of an electronic cigarette and preprocess the images;

[0128] An improved RT-DETR target detection model is constructed. In the backbone network of the RT-DETR model, a partial convolutional and gated feature fusion module PCGF is introduced to replace the original convolutional feature extraction unit. In the encoder structure of the RT-DETR model, a multi-scale sequence fusion enhancement module ESSF3D is introduced. In the target regression stage of the RT-DETR model, a target regression loss function that is sensitive to difficult samples is introduced.

[0129] An improved RT-DETR target detection model was trained using surface defect images of the frame of an electronic cigarette.

[0130] An improved RT-DETR target detection model, after training, was used to detect surface defects in the frame of an electronic cigarette, and the defect detection results were obtained.

[0131] The computer-readable storage medium may be transient or non-transient. Exemplary examples include, but are not limited to, various media capable of storing computer program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] For example, the processor may be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0133] It should also be noted that, in this specification, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0134] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting surface defects in the middle frame of an electronic cigarette based on an improved RT-DETR, characterized in that, Includes the following steps: S1. Obtain images of surface defects on the inner frame of the electronic cigarette and preprocess the images; S2. Construct an improved RT-DETR target detection model. In the backbone network of the RT-DETR model, introduce a partial convolutional and gated feature fusion module PCGF to replace the original convolutional feature extraction unit. In the encoder structure of the RT-DETR model, introduce a multi-scale sequence fusion enhancement module ESSF3D. In the target regression stage of the RT-DETR model, introduce a target regression loss function that is sensitive to difficult samples. S3. The improved RT-DETR target detection model is trained using surface defect images of the electronic cigarette frame. S4. Using the improved RT-DETR target detection model that has been trained, the surface defect image of the electronic cigarette frame is detected to obtain the defect detection results.

2. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 1, characterized in that, In step S1, image preprocessing includes image size unification, data augmentation, and dataset partitioning; Image size unification specifically involves normalizing the pixel values ​​of all images and unifying their sizes. Data augmentation includes, but is not limited to, random cropping, brightness adjustment, rotation, horizontal and vertical flipping, random Gaussian noise superposition, and image stitching; Specifically, the dataset was divided into training, validation, and test sets by dividing the surface defect images of the electronic cigarette frame in a ratio of 8:1:

1.

3. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 1, characterized in that, In step S2, a partial convolutional and gated feature fusion module PCGF is introduced to replace the original convolutional feature extraction unit, specifically: The original basic convolutional module of RT-DETR is replaced with a partial convolution and gated feature fusion module PCGF. PCGF combines partial convolution and gated activation mechanism to achieve key feature enhancement and low-cost channel bypass. PCGF includes the following steps: The input feature map is split into two paths in the cross-stage local network: one path goes directly to the end of the module to retain low-cost, basic features; the other path enters the PCGF bottleneck module for deep feature processing, dynamically selecting important channels through partial convolutions and achieving feature enhancement through GLU gating; the specific process is as follows: Let the input feature map be: First, the channel dimension is divided into a subset of key channels. With the cheap channel subset ,in: Convolution is performed only on a subset of key channels, and part of the convolution process can be represented as follows: The number of channels is then aligned using 1×1 convolutions, and convolutionally gated linear units are introduced for feature modulation. The gate expression is as follows: in, This represents the Sigmoid activation function. This represents element-wise multiplication; Finally, the gated enhanced features are concatenated with the low-cost channel features to obtain the output of the PCGF module, represented as:

4. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 1, characterized in that, In step S2, the multi-scale sequence fusion enhancement module ESSF3D is introduced, specifically as follows: In the feature fusion stage of the RT-DETR model, a multi-scale sequence fusion enhancement module, ESSF3D, is introduced. This module takes the multi-scale features output from the backbone network as input and achieves efficient interaction and joint modeling between features of different scales through scale-related feature enhancement, cross-scale serialization representation, and 3D convolution operations. This enhances the model's ability to identify defects and complex differences in small targets. The multi-scale sequence fusion enhancement module ESSF3D includes the following steps: First, feature maps of different scales are selected as inputs from the backbone network. Let the feature maps from layers P2, P3, and P4 be denoted as follows: , ; To enhance the discriminative power of features at different scales along the scale dimension, scale-dependent Gaussian convolution operations are introduced into the aforementioned feature maps to smooth them and adjust their response range. For any scale feature, the feature representation after Gaussian convolution is as follows: in, Represents the input feature mapping. The two-dimensional Gaussian kernel function is defined as follows: Among them, parameters As the feature scale increases progressively, it is used to characterize the smoothness of features at different scales. In this way, features at different scales obtain response characteristics with scale differences before entering the subsequent fusion stage, providing a more stable and discriminative input representation for cross-scale feature modeling. Subsequently, to achieve sequential fusion of multi-scale features, the nearest neighbor interpolation method was used to uniformly adjust the multi-scale features after Gaussian convolution to the same spatial resolution as the P2 layer. After resolution alignment, the features at each scale were stacked in the scale dimension to construct a three-dimensional feature tensor. in, Let represent the k-th scale feature map after resolution alignment, and Stack represent the feature stacking operation in the scale dimension. For the constructed 3D feature tensor, 3D convolution is introduced to extract features, so as to explicitly model the correlation and difference between features at different scales. The calculation process is expressed as follows: in, Represents a three-dimensional convolution operation. This represents a feature mapping function composed of batch normalization and nonlinear activation functions; through three-dimensional convolution operations, the model can simultaneously perform feature modeling in both spatial and scale dimensions, thereby effectively learning the correlation between cross-scale defect features; After completing the 3D convolution, the 3D max pooling operation is used to compress and integrate the features, compressing the scale dimension information back into the 2D feature representation, and outputting the fused 2D feature map as the input for the subsequent feature pyramid and decoder stages. At the network structure level, the ESSF3D module is used to perform cross-scale fusion of multi-layer features, and its output is concatenated or added with the features obtained by the conventional upsampling path to enrich the feature hierarchy expression. Subsequently, the shallow features and fused features are fused again based on the ESSF3D module to generate feature representations that contain both deep semantic information and shallow detail information, which are used for target prediction in the decoder stage.

5. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 4, characterized in that, Suppose the multi-scale feature map output by the backbone network is: in, It has a high resolution and contains rich local detail information. It contains stronger high-level semantic information; First, the nearest neighbor interpolation method is used to align the resolution of features at different scales: Subsequently, the aligned features are stacked along the newly added scale dimension to form a three-dimensional feature representation: Perform 3D convolution operations on the above 3D features to model the correlation between features across scales: After batch normalization and nonlinear activation function processing, 3D max pooling is used to compress the scale dimension to obtain the final fused features:

6. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 1, characterized in that, In step S2, a target regression loss function for hard sample perception is introduced, specifically: Based on the original regression loss function of RT-DETR, the Focaler concept is introduced to improve the generalized intersection-union (GIoU) ​​loss by weighting it, resulting in the Focaler-GIoU loss function. GIoU loss, as a commonly used bounding box regression loss in object detection, is defined as follows: The calculation method for GIoU is as follows: ； Where A represents the predicted bounding box, B represents the ground truth bounding box, and C represents the smallest bounding rectangle that simultaneously encloses A and B. Indicates the area of ​​the corresponding region; By introducing the Focaler concept, the GIoU loss is weighted with hard samples, making the model focus more on locating defects that are more difficult to locate and smaller in scale during training. The Focaler-GIoU loss is expressed as: in, These are weighting coefficients used to balance the proportion of regression loss in the total loss function; This is the Focaler index parameter, used to adjust the degree of attention given to difficult samples; When the overlap between the predicted bounding box and the ground truth bounding box is high, i.e. the GIoU value is large, the loss weight is relatively reduced; when the overlap between the predicted bounding box and the ground truth bounding box is low and the regression is difficult, the loss weight is significantly amplified, thereby prompting the model to optimize for this type of sample more fully during the training process. During the overall training of the model, the Focaler-GIoU loss, as the bounding box regression loss term, together with the classification loss, constitutes the total loss function of the detection model, which is expressed as follows: in, The classification loss represents the defect category. These are the weighting coefficients used to balance classification loss and regression loss.

7. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 1, characterized in that, In step S3, the model training uses the AdamW optimizer, with a learning rate of 0.0001, a batch size of 4, and 500 training epochs. During the training phase, the backbone features enhanced by the PCGF module and the multi-scale features fused with the ESSF3D module are input into the detection head, and the target regression and category classification are optimized by Focaler-GIoU loss. After training, the test set images are input into the model, and the output results of surface defects in the electronic cigarette frame are displayed, including the defect category and bounding box coordinates.

8. The method for detecting surface defects in the middle frame of an electronic cigarette based on improved RT-DETR according to claim 1, characterized in that, Surface defects of the electronic cigarette frame include, but are not limited to, scratches, dents, cracks, burrs, and foreign matter adhesion defects. Images of surface defects in the electronic cigarette frame come from both real-time production line photography and manually annotated defect samples.

9. A surface defect detection system for the middle frame of an electronic cigarette based on an improved RT-DETR, characterized in that, The system employs the electronic cigarette frame surface defect detection method according to any one of claims 1-8. The system includes an image acquisition and preprocessing module, a target detection model construction module, a target detection model training module, and a defect detection output module. The image acquisition and preprocessing module is used to acquire images of surface defects in the frame of the electronic cigarette and to preprocess the images. The target detection model building module is used to construct an improved RT-DETR target detection model. In the backbone network of the RT-DETR model, a partial convolutional and gated feature fusion module PCGF is introduced to replace the original convolutional feature extraction unit. In the encoder structure of the RT-DETR model, a multi-scale sequence fusion enhancement module ESSF3D is introduced. In the target regression stage of the RT-DETR model, a target regression loss function that is aware of difficult samples is introduced. The target detection model training module is used to train the improved RT-DETR target detection model using surface defect images of the electronic cigarette frame. The defect detection output module is used to detect surface defects in the frame of an electronic cigarette using a trained and improved RT-DETR target detection model, and output the defect detection results.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the method for detecting surface defects in the middle frame of an electronic cigarette as described in any one of claims 1-8.

Images