Package defect detection method and system

Abstract

The invention relates to a packaging defect detection method and system, and the method comprises the steps: S1, obtaining the size information of packaging bags and the interval information of adjacent packaging bags through an area-array camera and a photoelectric sensor in combination with a multi-scale template matching method; s2, delay time of a conveying belt is calculated, and it is ensured that the distance between adjacent packaging bags is larger than a set value; s3, triggering a line scanning camera to shoot an image of the packaging bag, and collecting the vibration frequency; s31, performing motion compensation correction on the image by adopting an improved optical flow method; s32, inputting the corrected image into a forward propagation network based on an improved OfficientViT model, wherein the model comprises a multi-scale linear attention module and an SPP-ELan module; s33, the images are classified, if the classification result is that the bags are not packaged, the unpackaged bags are grabbed through a grabbing robot, and if it is determined that the bags are not packaged based on a classification confidence coefficient threshold value, the unpackaged bags are removed; and S34, inputting the defect image into an improved ACGAN network for data enhancement, wherein the network comprises a CMFNet denoising module based on a U-Net architecture. The method and the system can monitor and identify the packaging defects on the production line in real time, improve the accuracy and the efficiency of inspection, and reduce the labor cost.

Description

Technical Field

[0001] The present invention belongs to the technical field of packaging defect detection, and particularly relates to a packaging defect detection method and system. Background Art

[0002] In modern manufacturing, packaging defect detection is a crucial step in ensuring product quality and consumer safety. With increasing market competition, companies are placing increasingly high demands on product packaging. Packaging must not only protect the product but also appeal to consumers. Therefore, the timely detection and resolution of packaging defects is crucial.

[0003] Traditional packaging defect inspection methods rely heavily on manual inspection, which is not only inefficient but also susceptible to human error, leading to missed or false detections. Furthermore, as production scale expands, the cost of manual inspection continues to rise. Therefore, there is an urgent need for an efficient and accurate packaging defect inspection method and system to improve production efficiency, reduce costs, and ensure product quality. Summary of the Invention

[0004] The purpose of the present invention is to provide a packaging defect detection method and system, which can monitor and identify packaging defects in real time on the production line, improve the accuracy and efficiency of inspection, and reduce labor costs.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a packaging defect detection method, comprising the following steps:

[0006] S1. Using an area array camera and a photoelectric sensor in combination with a multi-scale template matching method, the size information of the packaging bag containing the target material and the spacing information of adjacent packaging bags are obtained;

[0007] S2. Calculate the conveyor belt delay time based on the size information of the packaging bags and the spacing information of adjacent packaging bags to ensure that the spacing between adjacent packaging bags is greater than the set value;

[0008] S3, triggering the line scan camera to capture the image of the packaging bag based on the conveyor belt speed and delay time, and synchronously collecting the vibration frequency through the vibration sensor;

[0009] S31. Use the improved optical flow method to perform motion compensation correction on the image by combining the vibration frequency and amplitude;

[0010] S32. Input the corrected image into a forward propagation network based on an improved EfficientViT model, wherein the improved EfficientViT model includes a multi-scale linear attention module and an SPP-Elan module;

[0011] S33, classifying the image into properly packaged bags and unpackaged bags. If the classification result is an unpackaged bag, the unpackaged bag is grasped by a grasping robot and a classification confidence threshold is determined to be greater than a set value. If so, the bag is removed; otherwise, a manual judgment is required;

[0012] S34. Input the defect image into the improved ACGAN network for data enhancement to increase the network generalization ability. The improved ACGAN network includes a CMFNet denoising module based on the U-Net architecture.

[0013] Furthermore, in step S1, the implementation process of the multi-scale template matching method includes:

[0014] Gaussian Blur:

[0015] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0016] Where G(x,y,σ) is a Gaussian kernel with standard deviation σ, and L(x,y,σ) is an image in scale space;

[0017] Difference Gaussian:

[0018] DoG(x,y,σ)=L(x,y,kσ)-L(x,y,σ)

[0019] Where k is a constant;

[0020] Key point precise positioning, using Taylor series expansion to perform sub-pixel precision on key point positions; the Taylor series expansion formula is:

[0021]

[0022] Among them, D is the DoG function value, x is the offset of the key point position, and by solving the derivative Get the precise location of key points;

[0023] Direction assignment:

[0024]

[0025] Descriptor vector:

[0026] D=[h1,h2,…,h 128 ]

[0027] Among them, h i is the gradient weight of each direction in each sub-region;

[0028] By comparing the Euclidean distance between descriptor vectors, key points in different images are matched; the calculation formula of Euclidean distance is:

[0029]

[0030] Matching rules:

[0031]

[0032] Among them, D2 and D3 are the nearest neighbor and the second nearest neighbor respectively, and τ is the threshold.

[0033] Furthermore, in step S2, the conveyor belt delay time t delay The calculation formula is:

[0034]

[0035] Among them, t delay is the conveyor belt delay time, d current is the current spacing between adjacent bags, v is the speed of the conveyor belt, 15 is the minimum spacing between adjacent bags, and max(0,·) is the function that takes the larger value to ensure that the delay time is not negative. If the current spacing has met the requirements, then t delay =0.

[0036] Furthermore, in step S3, the camera starts shooting at:

[0037]

[0038] Among them, t triggerstart The time when the camera starts shooting, t sensor is the time point when the photoelectric sensor detects the packaging bag, d sensor is the distance from the sensor to the camera;

[0039] The camera stops recording when:

[0040]

[0041] Among them, t triggerend The time when the camera stops shooting, L bag is the length of the packaging bag.

[0042] Furthermore, in step S3, the vibration frequency is collected by a MEMS accelerometer, and the implementation method is as follows:

[0043] The main sources of vibration are the conveyor belt and the motor. The acceleration values in the xyz directions are obtained by MEMS. The amplitude of the vibration is expressed by the RMS acceleration, and the calculation formula is:

[0044]

[0045] Where N is the number of sampling points; a iis the acceleration value of the i-th sampling point;

[0046] The amplitude after attenuation is:

[0047] A f =A i ·e -αx

[0048] Among them, A f is the amplitude after attenuation, A i is the initial amplitude, α is the attenuation coefficient, and x is the propagation distance;

[0049] The attenuation coefficient α is calculated by the following formula:

[0050]

[0051] Where f is the vibration frequency, c is the speed of sound propagation in the material, E is the Young's modulus of the material, and ρ is the density of the material;

[0052] Perform FFT on the acceleration signal to convert it to the frequency domain:

[0053] A(f)=FFT(a(t))

[0054] Where A(f) is the frequency domain signal;

[0055] The frequency component f passes through the sampling frequency f s and the index of the FFT result to calculate:

[0056]

[0057] Where k is the index of the FFT result, from 0 to N-1, N is the total number of sampling points, and f s is the sampling frequency;

[0058] Damping frequency f d With the natural frequency f n The relationship is:

[0059]

[0060] Among them, f d is the damping frequency, f n is the natural frequency, ζ is the damping ratio;

[0061] Natural frequency f n The calculation formula is:

[0062]

[0063] Where k is the stiffness of the system and m is the mass of the system;

[0064] The calculation formula of damping ratio ζ is:

[0065]

[0066] Where c is the damping coefficient;

[0067] In step S31, motion estimation is performed on the image based on the improved optical flow method, and a total energy function is constructed by combining the brightness constancy constraint and the motion smoothness constraint;

[0068]

[0069] Among them, u and v represent the displacement vector field of the image pixel in the x direction and y direction, that is, the optical flow field; I x [i,j]、I y [i,j] and I t [i, j] represents the partial derivatives of the image at point (i, j) with respect to x, y, and time t respectively; λ is the weight coefficient of the smoothing term, which is used to balance the brightness constrain and motion smoothness constraint; E brightness (u, v) represents the brightness constancy constraint, assuming that the brightness of the same object remains unchanged in consecutive frames; E smooth (u, v) represents the motion smoothness constraint, assuming that the motion of adjacent pixels is smooth and continuous; E(u, v) is the total energy function, which combines the two constraints of brightness constancy and motion smoothness;

[0070] Then adjust u and v according to the amplitude and frequency:

[0071] u′=u+A·cos(2πft)

[0072] v′=v+A·sin(2πft)

[0073] Where u' and v' are the displacement vectors after adjustment of u and v, A is the amplitude, f is the frequency, and t is the time;

[0074] Based on the estimated camera motion u, motion compensation is applied in reverse; for each pixel I, its motion vector u is used to adjust the pixel position back to its original position:

[0075]

[0076] Among them, I corrected [i] is the rectified image, I(t) is the original image at time t, is the estimated mean displacement vector;

[0077] The above process will adjust the pixel positions back to their original positions based on the estimated motion, thereby compensating for the image distortion caused by motion.

[0078] Furthermore, in step S32, the dataset is trained based on the improved EfficientVit model, which introduces the SPP-Elan module. The SPP-Elan module includes:

[0079] 1) Front-end feature extraction: The input image is preprocessed with standard convolution and depthwise separable convolution to extract low-order texture features. The feature expression capability is gradually enhanced through multiple layers of moving inverted bottleneck convolution.

[0080] 2) Path processing:

[0081] Left path (global semantics): Combined with EfficientViT's window attention mechanism, it captures the global context of the image and performs two-loop processing to strengthen long-range dependency modeling.

[0082] Right path (local details): uses an 11-layer repeated bneck module to enhance subtle defect features through lightweight deep convolution, and compresses redundant information through average pooling and fully connected layers to improve computational efficiency.

[0083] 3) Dynamic mask fusion:

[0084] The Mask module receives the left and right features and generates a binary mask based on the local complexity. The high-complexity area prioritizes the local detail features of the right-path bneck, and the low-complexity area uses the global features of the left-path EfficientViT. The feature fusion formula is:

[0085] F fused =M⊙F 右路 +(1-M)⊙F 左路

[0086] 4) Multi-scale optimization and classification:

[0087] The fused features are input into the SPP-Elan module, which aggregates the multi-granular features of the defects through multi-scale pooling, and finally outputs the detection results through the classification head.

[0088] Furthermore, the implementation method of the bneck module is:

[0089] After entering the module, it first undergoes a 1×1 point-by-point dimensionality increase convolution, and the activation function is Hard-Swish, which is expressed as:

[0090]

[0091] Perform depth-wise separable convolution, which uses a 3×3 convolution kernel to process each channel;

[0092] Perform 1×1 point-by-point dimensionality reduction convolution;

[0093] Perform attention mechanism and residual connection.

[0094] Furthermore, the improved ACGan network used in step S33 introduces a CMFNET module after the generator, and the CMFNET module includes:

[0095] 1) Multi-branch architecture: It consists of three branches, each of which processes different types of image features and uses a different attention mechanism: pixel attention, channel attention, and spatial attention;

[0096] 2) U-Net Backbone: Each branch is based on the U-Net architecture, allowing efficient encoding and decoding of features at multiple scales;

[0097] 3) Residual Attention Module (RAM): The output of each branch is processed through RAM to further optimize the features;

[0098] 4) Hybrid Skip Connection (MSC): The output of each branch is integrated through MSC, which uses learnable parameters to dynamically adjust the weight of each branch output;

[0099] 5) Loss function: The perceptual indicators SSIM and PSNR are combined with the edge loss.

[0100] The present invention also provides a packaging defect detection system, comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method can be implemented.

[0101] Compared with the prior art, the present invention has the following beneficial effects:

[0102] 1. By combining advanced image processing technology and intelligent algorithms, real-time monitoring and identification of packaging defects on the production line are achieved, which improves the accuracy and efficiency of inspection and reduces labor costs.

[0103] 2. By introducing the SPP-Elan module into the improved EfficientVit model, the dynamic mask fusion mechanism is used to effectively balance global information and local details, thereby improving the recognition ability of various defect types.

[0104] 3. By introducing the CMFNET module into the improved ACGan network, the quality of generated images is effectively improved, especially in the restoration of fine textures and microscopic details, ensuring that the generated images are smoother, clearer, and more natural, making the confrontation between the generator and the discriminator more intense, further promoting the improvement of the generation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1This is a block diagram of the implementation principle of the packaging defect detection method according to an embodiment of the present invention;

[0106] Figure 2 2 is a schematic diagram of the structure of the improved EfficientVit model in an embodiment of the present invention;

[0107] Figure 3 Schematic diagram of the structure of the SPP-Elan module in an embodiment of the present invention;

[0108] Figure 4 is a schematic diagram of the structure of an improved ACGan network in an embodiment of the present invention;

[0109] Figure 5 This is a defect image generated based on the improved ACGan network in an embodiment of the present invention;

[0110] Figure 6 Schematic diagram of the structure of the CMFNET module in an embodiment of the present invention. DETAILED DESCRIPTION

[0111] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0112] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0113] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0114] like Figure 1 As shown, this embodiment provides a packaging defect detection method, comprising the following steps:

[0115] S1. Using an area array camera and a photoelectric sensor in combination with a multi-scale template matching method, the size information of the packaging bag containing the target material and the spacing information of adjacent packaging bags are obtained;

[0116] S2. Calculate the conveyor belt delay time based on the size information of the packaging bags and the spacing information of adjacent packaging bags to ensure that the spacing between adjacent packaging bags is greater than a set value (set to 15 cm in this embodiment);

[0117] S3, triggering the line scan camera to capture the image of the packaging bag based on the conveyor belt speed and delay time, and synchronously collecting the vibration frequency through the vibration sensor;

[0118] S31. Use the improved optical flow method to perform motion compensation correction on the image by combining the vibration frequency and amplitude;

[0119] S32. Input the corrected image into a forward propagation network based on an improved EfficientViT model, wherein the improved EfficientViT model includes a multi-scale linear attention module and an SPP-Elan module;

[0120] S33. Classify the image into normal and unpackaged bags. If the classification result is an unpackaged bag, the grabbing robot grabs the unpackaged bag and determines whether the classification confidence threshold is greater than a set value (set to 0.8 in this embodiment). If so, the bag is removed; otherwise, it waits for manual judgment.

[0121] S34. Input the defect image into the improved ACGAN network for data enhancement to increase the network generalization ability. The improved ACGAN network includes a CMFNet denoising module based on the U-Net architecture.

[0122] In the field of solid particulate packaging, a common method is to first fill a fixed quantity of solid particles into vacuum bags, typically in sizes ranging from 5, 10, and 25 kg. The vacuum bags are then fed into a bagging machine, where outer bags of various manufacturers and designs are placed over the vacuum bags to create the bags containing the target material. This method of separating the inside and outside of the bag is highly universal.

[0123] In step S1, the implementation process of the multi-scale template matching method includes:

[0124] Gaussian Blur:

[0125] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0126] Here, G(x,y,σ) is a Gaussian kernel with standard deviation σ, and L(x,y,σ) is an image in scale space.

[0127] Difference Gaussian:

[0128] DoG(x,y,σ)=L(x,y,kσ)-L(x,y,σ)

[0129] Where k is a constant, usually

[0130] Key point precise positioning, using Taylor series expansion to perform sub-pixel precision on key point positions; the Taylor series expansion formula is:

[0131]

[0132] Among them, D is the DoG function value, x is the offset of the key point position, and by solving the derivative Get the exact location of the key points.

[0133] Direction assignment:

[0134]

[0135] Descriptor vector:

[0136] D=[h1,h2,…,h 128 ]

[0137] Among them, h i is the gradient weight of each direction in each sub-region.

[0138] By comparing the Euclidean distance between descriptor vectors, key points in different images are matched; the calculation formula of Euclidean distance is:

[0139]

[0140] Matching rules:

[0141]

[0142] Among them, D2 and D3 are the nearest neighbor and the second nearest neighbor respectively, and τ is the threshold (usually τ = 0.8).

[0143] In step S2, the conveyor belt is delayed for a time t delay The calculation formula is:

[0144]

[0145] Among them, t delay is the conveyor belt delay time (unit: seconds, s), d current is the current spacing between adjacent bags (unit: centimeters, cm), v is the speed of the conveyor belt (unit: centimeters / second, cm / s), 15 is the minimum spacing between adjacent bags (unit: centimeters, cm), max(0, ) is the function that takes the larger value to ensure that the delay time is not negative. If the current spacing has met the requirements, then t delay =0.

[0146] In step S3, the camera starts shooting at:

[0147]

[0148] Among them, t triggerstart The time when the camera starts shooting, t sensor is the time point when the photoelectric sensor detects the packaging bag, d sensor is the distance from the sensor to the camera.

[0149] The camera stops recording when:

[0150]

[0151] Among them, t triggerend The time when the camera stops shooting, L bag is the length of the packaging bag.

[0152] In step S3, the vibration frequency is collected by a MEMS accelerometer, and the implementation method is as follows:

[0153] The main sources of vibration are the conveyor belt and the motor. The acceleration values in the xyz directions are obtained by MEMS. The amplitude of the vibration is expressed by the RMS acceleration, and the calculation formula is:

[0154]

[0155] Where N is the number of sampling points; a i is the acceleration value of the i-th sampling point.

[0156] The amplitude after attenuation is:

[0157] A f =A i ·e -αx

[0158] Among them, A f is the amplitude after attenuation, A i is the initial amplitude, α is the attenuation coefficient, and x is the propagation distance.

[0159] The attenuation coefficient α is calculated by the following formula:

[0160]

[0161] Where f is the vibration frequency (Hz), c is the speed of sound in the material (m / s), E is the Young's modulus of the material (Pa), and ρ is the density of the material (kg / m).

[0162] Perform FFT on the acceleration signal to convert it to the frequency domain:

[0163] A(f)=FFT(a(t))

[0164] Where A(f) is the frequency domain signal.

[0165] The frequency component f is obtained by sampling frequency f s and the index of the FFT result to calculate:

[0166]

[0167] Where k is the index of the FFT result, from 0 to N-1, N is the total number of sampling points, and f s is the sampling frequency (in Hz).

[0168] Damping frequency f d With the natural frequency f n The relationship is:

[0169]

[0170] Among them, f d is the damping frequency, f n is the natural frequency and ζ is the damping ratio.

[0171] Natural frequency f n The calculation formula is:

[0172]

[0173] Where k is the stiffness of the system (N / m) and m is the mass of the system (kg).

[0174] The calculation formula of damping ratio ζ is:

[0175]

[0176] Where c is the damping coefficient (N·s / m).

[0177] In step S31, motion estimation is performed on the image based on the improved optical flow method, and a total energy function is constructed by combining the brightness constancy constraint and the motion smoothness constraint;

[0178]

[0179] Among them, u and v represent the displacement vector field of the image pixel in the x direction and y direction, that is, the optical flow field; I x [i,j]、I y [i,j] and I t [i, j] represents the partial derivatives of the image at point (i, j) with respect to x, y, and time t respectively; λ is the weight coefficient of the smoothing term, which is used to balance the brightness constrain and motion smoothness constraint; E brightness (u, v) represents the brightness constancy constraint, assuming that the brightness of the same object remains unchanged in consecutive frames; E smooth(u, v) represents the motion smoothness constraint, assuming that the motion of adjacent pixels is smooth and continuous; E(u, v) is the total energy function, which combines the two constraints of brightness constancy and motion smoothness.

[0180] Then, according to the amplitude and frequency, adjust u and v (in the optical flow method, u and v represent the optical flow (motion) components of each pixel in the image in the x and y directions respectively):

[0181] u′=u+A·cos(2πft)

[0182] v′=v+A·sin(2πft)

[0183] In the amplitude and frequency adjustment formula, u' and v' are the displacement vectors after u and v are adjusted, A is the amplitude, f is the frequency, and t is the time.

[0184] Based on the estimated camera motion u, motion compensation is applied in reverse; for each pixel I, its motion vector u is used to adjust the pixel position back to its original position:

[0185]

[0186] In the image correction formula, I corrected [i] is the rectified image, I(t) is the original image at time t, is the estimated mean displacement vector;

[0187] The above process will adjust the pixel positions back to their original positions based on the estimated motion, thereby compensating for the image distortion caused by motion.

[0188] In step S32, the dataset is trained based on the improved EfficientVit model, which introduces the SPP-Elan module to form the SPP-EfficientVIT model, whose structure is as follows: Figure 2 shown.

[0189] like Figure 3 As shown, the SPP-Elan module includes:

[0190] 1) Front-end feature extraction: The input image is preprocessed by Conv (standard convolution) and DsConv (depthwise separable convolution) to extract low-order texture features, and the feature expression capability is gradually enhanced through multiple layers of MBConv (moving inverted bottleneck convolution).

[0191] 2) Path processing:

[0192] Left path (global semantics): Combined with EfficientViT's window attention mechanism, it captures the global context of the image (such as the overall packaging structure), and two loop processes (MBConv + EfficientViT) strengthen long-range dependency modeling;

[0193] Right path (local details): uses an 11-layer repeated bneck module to enhance subtle defect features (such as sealing wrinkles) through lightweight deep convolution, and compresses redundant information through AVG_pool (average pooling) and FC (fully connected layer) to improve computational efficiency.

[0194] Among them, the implementation method of the bneck module is:

[0195] After entering the module, it first undergoes a 1×1 point-by-point dimensionality increase convolution, and the activation function is Hard-Swish, which is expressed as:

[0196]

[0197] Perform depth-wise separable convolution, which uses a 3×3 convolution kernel to process each channel;

[0198] Perform 1×1 point-by-point dimensionality reduction convolution;

[0199] Perform attention mechanism (SE module) and residual connection.

[0200] 3) Dynamic mask fusion (core innovation):

[0201] The Mask module (hexagonal node) receives the left and right features and generates a binary mask based on the local complexity. The high-complexity area (such as the defect edge) gives priority to the local detail features of the right-path bneck, and the low-complexity area (such as the smooth background) uses the global features of the left-path EfficientViT. The feature fusion formula is:

[0202] F fused =M⊙F 右路 +(1-M)⊙F 左路

[0203] 4) Multi-scale optimization and classification:

[0204] The fused features are input into the SPP-Elan module, which aggregates the multi-granularity features of the defects through multi-scale pooling (4×4, 8×8, 16×16), and finally outputs the detection results through the Classify module (classification head).

[0205] The improved ACGan network used in step S33 introduces the CMFNET module after the generator, and its structure is as follows: Figure 4 As shown. The defect image generated based on the improved ACGan network is shown Figure 5shown.

[0206] like Figure 6 As shown, the CMFNET module includes:

[0207] 1) Multi-branch architecture: It consists of three branches, each of which is inspired by different types of retinal ganglion cells (RGCs), namely P cells, M cells, and K cells, which process different types of image features, and each branch uses a different attention mechanism (pixel attention, channel attention, and spatial attention).

[0208] 2) U-Net Backbone: Each branch is based on the U-Net architecture, allowing efficient encoding and decoding of features at multiple scales.

[0209] 3) Residual Attention Module (RAM): The output of each branch is processed through RAM to further optimize the features.

[0210] 4) Hybrid Skip Connection (MSC): MSC is used to integrate the output of each branch. Unlike traditional residual connections, MSC uses learnable parameters to dynamically adjust the weight of each branch output.

[0211] 5) Loss function: Combining perceptual metrics (SSIM and PSNR) with edge loss.

[0212] This embodiment also provides a packaging defect detection system, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0213] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0214] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0215] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0216] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0217] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A packaging defect detection method, characterized in that: The following steps are involved: S1. Using an area array camera and a photoelectric sensor in combination with a multi-scale template matching method, the size information of the packaging bag containing the target material and the spacing information between adjacent packaging bags are obtained; S2. Calculate the conveyor belt delay time based on the size information of the packaging bags and the spacing information of adjacent packaging bags to ensure that the spacing between adjacent packaging bags is greater than the set value; S3, triggering the line scan camera to capture the image of the packaging bag based on the conveyor belt speed and delay time, and synchronously collecting the vibration frequency through the vibration sensor; S31. Use the improved optical flow method to perform motion compensation correction on the image by combining the vibration frequency and amplitude; S32. Input the corrected image into a forward propagation network based on an improved EfficientViT model, wherein the improved EfficientViT model includes a multi-scale linear attention module and an SPP-Elan module; S33, classifying the image into properly packaged bags and unpackaged bags. If the classification result is an unpackaged bag, the unpackaged bag is grasped by a grasping robot and a classification confidence threshold is determined to be greater than a set value. If so, the bag is removed; otherwise, a manual judgment is required; S34. Input the defect image into the improved ACGAN network for data enhancement to increase the network generalization ability. The improved ACGAN network includes a CMFNet denoising module based on the U-Net architecture.

2. The packaging defect detection method according to claim 1, characterized in that: In step S1, the implementation process of the multi-scale template matching method includes: Gaussian Blur: L(x,y,σ)=G(x,y,σ)*I(x,y) Where G(x,y,σ) is a Gaussian kernel with standard deviation σ, and L(x,y,σ) is an image in scale space; Difference Gaussian: DoG(x,y,σ)=L(x,y,kσ)-L(x,y,σ) Where k is a constant; Key point precise positioning, using Taylor series expansion to perform sub-pixel precision on key point positions; the Taylor series expansion formula is: Among them, D is the DoG function value, x is the offset of the key point position, and by solving the derivative Get the precise location of key points; Direction assignment: Descriptor vector: D=[h1,h2,…,h 128 ] Among them, h i is the gradient weight of each direction in each sub-region; By comparing the Euclidean distance between descriptor vectors, key points in different images are matched; the calculation formula of Euclidean distance is: Matching rules: Among them, D2 and D3 are the nearest neighbor and the second nearest neighbor respectively, and τ is the threshold.

3. The packaging defect detection method according to claim 1, characterized in that: In step S2, the conveyor belt is delayed for a time t delay The calculation formula is: Among them, t delay is the conveyor belt delay time, d current is the current spacing between adjacent bags, v is the speed of the conveyor belt, 15 is the minimum spacing between adjacent bags, and max(0,·) is the function that takes the larger value to ensure that the delay time is not negative. If the current spacing has met the requirements, then t delay =0.

4. The packaging defect detection method according to claim 1, characterized in that: In step S3, the camera starts shooting at: Among them, t triggerstart The time when the camera starts shooting, t sensor is the time point when the photoelectric sensor detects the packaging bag, d sensor is the distance from the sensor to the camera; The camera stops recording when: Among them, t triggerend The time when the camera stops shooting, L bag is the length of the packaging bag.

5. The packaging defect detection method according to claim 1, characterized in that: In step S3, the vibration frequency is collected by a MEMS accelerometer, and the implementation method is as follows: The main sources of vibration are the conveyor belt and the motor. The acceleration values in the xyz directions are obtained by MEMS. The amplitude of the vibration is expressed by the RMS acceleration, and the calculation formula is: Where N is the number of sampling points; a i is the acceleration value of the i-th sampling point; The amplitude after attenuation is: THE f =A i ·and -αx Among them, A f is the amplitude after attenuation, A i is the initial amplitude, α is the attenuation coefficient, and x is the propagation distance; The attenuation coefficient α is calculated by the following formula: Where f is the vibration frequency, c is the speed of sound propagation in the material, E is the Young's modulus of the material, and ρ is the density of the material; Perform FFT on the acceleration signal to convert it to the frequency domain: A(f)=FFT(a(t)) Where A(f) is the frequency domain signal; The frequency component f is obtained by sampling frequency f s and the index of the FFT result to calculate: Where k is the index of the FFT result, from 0 to N-1, N is the total number of sampling points, and f s is the sampling frequency; Damping frequency f d With the natural frequency f n The relationship is: Among them, f d is the damping frequency, f n is the natural frequency, ζ is the damping ratio; Natural frequency f n The calculation formula is: Where k is the stiffness of the system and m is the mass of the system; The calculation formula of damping ratio ζ is: Where c is the damping coefficient; In step S31, motion estimation is performed on the image based on the improved optical flow method, and a total energy function is constructed by combining the brightness constancy constraint and the motion smoothness constraint; E(u,v)=E brightness (u,v)+E smooth (u,v) Among them, u and v represent the displacement vector field of the image pixel in the x direction and y direction, that is, the optical flow field; I x [i,j]、I y [i,j] and I t [i, j] represents the partial derivatives of the image at point (i, j) with respect to x, y, and time t respectively; λ is the weight coefficient of the smoothing term, which is used to balance the brightness constrain and motion smoothness constraint; E brightness (u, v) represents the brightness constancy constraint, assuming that the brightness of the same object remains unchanged in consecutive frames; E smooth (u, v) represents the motion smoothness constraint, assuming that the motion of adjacent pixels is smooth and continuous; E(u, v) is the total energy function, which combines the two constraints of brightness constancy and motion smoothness; Then adjust u and v according to the amplitude and frequency: u′=u+A·cos(2πft) v′=v+A·sin(2πft) Where u' and v' are the displacement vectors after adjustment of u and v, A is the amplitude, f is the frequency, and t is the time; Based on the estimated camera motion u, motion compensation is applied in reverse; for each pixel I, its motion vector u is used to adjust the pixel position back to its original position: Among them, I corrected [i] is the rectified image, I(t) is the original image at time t, is the estimated mean displacement vector; The above process will adjust the pixel positions back to their original positions based on the estimated motion, thereby compensating for the image distortion caused by motion.

6. The packaging defect detection method according to claim 1, characterized in that: In step S32, the data set is trained based on the improved EfficientVit model, which introduces the SPP-Elan module. The SPP-Elan module includes: 1) Front-end feature extraction: The input image is preprocessed with standard convolution and depthwise separable convolution to extract low-order texture features. The feature expression capability is gradually enhanced through multiple layers of moving inverted bottleneck convolution. 2) Path processing: Left path (global semantics): Combined with EfficientViT's window attention mechanism, it captures the global context of the image and performs two-loop processing to strengthen long-range dependency modeling. Right path (local details): uses an 11-layer repeated bneck module to enhance subtle defect features through lightweight deep convolution, and compresses redundant information through average pooling and fully connected layers to improve computational efficiency. 3) Dynamic mask fusion: The Mask module receives the left and right features and generates a binary mask based on the local complexity. The high-complexity area prioritizes the local detail features of the right-path bneck, and the low-complexity area uses the global features of the left-path EfficientViT. The feature fusion formula is: F fused =M⊙F 右路 +(1-M)⊙F 左路 4) Multi-scale optimization and classification: The fused features are input into the SPP-Elan module, which aggregates the multi-granular features of the defects through multi-scale pooling, and finally outputs the detection results through the classification head.

7. The packaging defect detection method according to claim 6, characterized in that: The implementation method of the bneck module is: After entering the module, it first undergoes a 1×1 point-by-point dimensionality increase convolution, and the activation function is Hard-Swish, which is expressed as: Perform depth-wise separable convolution, which uses a 3×3 convolution kernel to process each channel; Perform 1×1 point-by-point dimensionality reduction convolution; Perform attention mechanism and residual connection.

8. The packaging defect detection method according to claim 1, characterized in that: The improved ACGan network used in step S33 introduces a CMFNET module after the generator, and the CMFNET module includes: 1) Multi-branch architecture: It consists of three branches, each of which processes different types of image features and uses a different attention mechanism: pixel attention, channel attention, and spatial attention; 2) U-Net Backbone: Each branch is based on the U-Net architecture, allowing efficient encoding and decoding of features at multiple scales; 3) Residual Attention Module (RAM): The output of each branch is processed through RAM to further optimize the features; 4) Hybrid Skip Connection (MSC): The output of each branch is integrated through MSC, which uses learnable parameters to dynamically adjust the weight of each branch output; 5) Loss function: The perceptual indicators SSIM and PSNR are combined with the edge loss.

9. A packaging defect detection system, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method according to any one of claims 1 to 8 can be implemented.

Images