A smoke box detection system and method integrating radar and vision multi-mode fusion

By integrating radar and vision into a multi-modal cigarette box inspection system, and combining deep learning and convolutional neural networks, the efficiency and accuracy issues in cigarette box inspection have been resolved. This enables non-destructive and comprehensive inspection of cigarette boxes, improving production management efficiency and product quality.

CN118279275BActive Publication Date: 2026-11-10THE 41ST INST OF CHINA ELECTRONICS TECH GRP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410446051.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-15
Publication Date
2026-11-10
Estimated Expiration
2044-04-15

AI Technical Summary

Technical Problem

Existing cigarette box inspection technologies are insufficient in terms of efficiency and accuracy, and cannot effectively detect internal defects, damage, and appearance quality issues in cigarette boxes. This results in defective cigarette boxes entering the retailer's supply chain, causing losses.

Method used

The detection system employs a multi-modal fusion of radar and vision, including pre-sealing visual inspection, radar imaging inspection, appearance quality inspection, and a multi-modal fusion system. It uses deep learning algorithms and convolutional neural networks to fuse information and identify defects such as missing strips, flaws, arrangement patterns, and appearance defects in cigarette boxes.

Benefits of technology

It enables non-destructive and comprehensive testing of cigarette boxes, improves testing efficiency and accuracy, reduces missed and false detections, lowers the cost of manual intervention, and meets the high-efficiency requirements of tobacco production.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118279275B_ABST
    Figure CN118279275B_ABST
Patent Text Reader

Abstract

This invention discloses a multi-modal fusion system and method for inspecting cigarette boxes using radar and vision, belonging to the field of product inspection technology. The invention employs radar imaging technology to acquire internal structural information of the cigarette box, and uses an AI vision system to acquire detailed information about the product's appearance. After data preprocessing and standardization, feature extraction technology is used to extract the appearance and features such as barcodes and QR codes of the cigarette box. A multi-modal fusion model is constructed using AI vision technology to automatically identify quality problems such as internal defects, flaws, abnormal arrangement patterns, and external defects such as wrinkles, stains, mislabeling, and tape sealing defects. Combining radar and vision imaging modalities, through data preprocessing, feature extraction and recognition, and fusion decision-making, this invention achieves an online cigarette box quality inspection system that integrates the detection of various defect signals inside and outside the cigarette box, as well as barcode and QR code related information, and enables precise traceability, thus improving the efficiency and accuracy of cigarette box quality inspection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of product testing technology, specifically relating to a smoke box testing system and method that integrates radar and vision multi-mode fusion. Background Technology

[0002] In cigarette box packaging production lines, after the sealing machine seals the boxes with tape, the boxes are conveyed to the finished product warehouse via a conveyor line. During the process of packing cigarette boxes by the sealing machine and shipping them from the finished product warehouse to retailers and consumers, various factors such as improper tape sealing, collisions, and squeezing may occasionally lead to issues like missing strips, damage, or abnormal arrangement inside the cigarette boxes. These are considered serious quality defects, causing considerable trouble and losses for tobacco companies. Furthermore, issues on the cigarette box's exterior, such as mislabeling, dampness, soiling, breakage, or incorrect barcodes or QR codes, can also severely affect the quality of the cigarette boxes. If these defects are not detected, they can cause further losses at later stages, such as at the retail level. Damaged sealing tape buckles, loose tape, or missing tape can result in sealing failures, causing the cigarettes inside the box to leak out when it leaves the finished product warehouse, seriously affecting cigarette box production. These quality defects are increasingly receiving high attention from tobacco manufacturers.

[0003] Currently, the main methods for detecting missing or damaged cigarette packs include machine vision inspection, dynamic weighing inspection, and photonics inspection. Machine vision inspection is performed before sealing the pack, when the pack is unpackaged. Because image acquisition takes place in a non-sealed environment with many interference factors, and the camera, mounted on a push rod, moves back and forth, resulting in poor imaging quality due to vibration, dust, and light sources, and the presence of laser-coated packaging on some cigarette packs further affects image quality, leading to numerous false positives and missed positives. Weighing inspection devices are affected by the upper and lower limits of weight deviation during production, resulting in significant cumulative deviations. The recycling of some boxes also affects the overall weight of the pack, leading to multiple influencing factors and false positives and missed positives. Photonics inspection essentially uses real-time X-ray scanning and density measurement for comparison. Its radiation dose is equivalent to that of a chest X-ray during a human medical examination, which can have a significant impact on people and the environment with long-term use. Traditional manual methods cannot guarantee the efficiency and accuracy of inspection, making it difficult to meet the current timeliness requirements of cigarette packaging workshops for warehousing. Furthermore, in testing environments with short production lines and limited space, it is impossible to install weighing and testing devices after the packaging process. Only radar imaging defect detection devices can be installed. The lack of process inspection for the internal cigarette packs, their internal arrangement, and their appearance can lead to problems such as missed detection of internal damage and external breakage, mixed packaging of incorrect labels, and barcode errors. The market urgently needs a system that can solve these problems.

[0004] In summary, existing technologies and equipment still suffer from low detection efficiency and poor detection accuracy in detecting missing strips, damage, abnormal arrangement, and appearance quality of cigarette boxes. They cannot prevent cigarette boxes with missing strips or appearance defects from entering the retail or distribution process. Therefore, in order to meet market demand, save installation space, and facilitate unified management, a multi-mode fusion system and method for detecting cigarette boxes using radar and vision is proposed. Summary of the Invention

[0005] In view of the above-mentioned technical problems in the prior art, the present invention proposes a smoke box detection system and method that integrates radar and vision multi-mode fusion. The system and method are reasonably designed, overcome the shortcomings of the prior art, and have good performance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A multi-modal fusion radar and vision system for inspecting cigarette boxes includes a pre-sealing visual inspection system, a radar imaging inspection system, an appearance quality inspection system, a multi-modal fusion system, and a control and processing system.

[0008] The pre-sealing visual inspection system is configured to acquire and process images of the end face of the cigarette boxes before sealing.

[0009] The radar imaging detection system is configured to acquire radar echo information and perform imaging detection and identification of the internal state of the cigarette box after it is sealed, including the extraction of feature information such as missing strips, internal flaw detection, and arrangement pattern.

[0010] The appearance quality inspection system is configured to collect information on the surface images of the cigarette box, including the detection of wrinkles, dirt, and defects in the sealing tape, as well as barcode and QR code information.

[0011] The multimodal fusion system employs a hybrid fusion method and is configured to fuse several modalities, including visual imaging before sealing of cigarette boxes, radar imaging appearance, and visual imaging, through model training and decision output.

[0012] The control and processing system is configured to summarize, statistically analyze, and classify information on missing strips, flaw detection, arrangement, appearance defects, barcodes, and QR codes in cigarette boxes.

[0013] Preferably, the pre-sealing visual inspection system includes an image acquisition module and an analysis and processing module;

[0014] The image acquisition module is configured to acquire images of the end face of the cigarette box before it is sealed.

[0015] The analysis and processing module is configured to use deep learning algorithms to extract feature information of the end face of the carton in the image.

[0016] Preferably, the image acquisition module includes an industrial camera and a stroboscopic modulated light source. The industrial camera sets the exposure value based on the light intensity and is equipped with a polarizing lens, enabling multi-angle and multi-directional adjustment. Under the illumination of the stroboscopic modulated light source, it acquires the end face image of the cigarette box before sealing.

[0017] Preferably, the radar imaging detection system includes a radio frequency module, a data acquisition module, and an imaging processing module;

[0018] The radio frequency module is configured for generating, modulating, transmitting, receiving, and demodulating terahertz / millimeter-wave signals and echo signals.

[0019] The data acquisition module is configured to acquire demodulated data of the echo signal reflected from the smoke box;

[0020] The imaging processing module is configured to generate reconstructed images from the data acquired by the data acquisition module using imaging algorithms, and then use deep learning algorithms to identify information such as missing bars, flaw detection, and arrangement patterns of the cigarette box in the reconstructed images.

[0021] Preferably, the appearance quality inspection system includes an image acquisition module and an AI large model recognition and classification module;

[0022] The AI ​​large-scale model recognition and classification module adopts a generative adversarial network (GAN). While training an image recognition model from cigarette box image samples, a deep neural network generator is trained to generate AI image samples. The deep neural network discriminator then identifies various defect features in the cigarette box images.

[0023] Preferably, the multimodal fusion system acquires all identification data of the same cigarette box before sealing using both visual imaging and radar imaging modalities, achieving time-series synchronization of identification decisions for missing strips, flaw detection, and arrangement pattern information.

[0024] The multimodal fusion system uses a convolutional neural network to extract contour feature information from the regions of interest in visual and radar imaging before the cigarette box is sealed. The system groups and associates these features according to their type, and uses an attention mechanism to achieve deep fusion and recognition of the two types of contour feature information from vision and radar to obtain the fusion output. The output is then input into the model before the decision-level fusion.

[0025] The multimodal fusion system first performs feature-level fusion of visual imaging and radar imaging features before the cigarette box is sealed, and uses the fused output as input data to train a model; then, it inputs the feature information of the appearance visual imaging modality into another model for training; the output results of the two model training are further fused at the decision level through an attention mechanism fusion algorithm to output the defect classification decision and transmit it to the control processing system.

[0026] Preferably, the attention mechanism involves introducing an attention weight vector and then using a neural network structure to learn a function to calculate the weights. This function dynamically calculates the attention weights based on the input of modal features to reflect the importance of different modalities in the fusion process.

[0027] Preferably, the control processing system includes a human-computer interaction unit and a database unit;

[0028] The human-computer interaction unit is configured for operators to view and analyze various data and image information of the system and interact with the machine.

[0029] The database unit is configured to store statistical correlation data on defects in all inspected cigarette boxes.

[0030] Preferably, the control and processing system can statistically and classify the status of each cigarette box, including whether it has missing strips, visual and radar images, appearance defects, and their association information with barcodes and QR codes. The control and processing system can classify and statistically store different appearance defect types such as wrinkles, dirt, and sealing tape defects. Operators can view each defect type by year, month, day, shift, or batch through the human-machine interface. The control and processing system can operate independently through the system's human-machine interaction unit or be integrated into the box-sealing machine system interface.

[0031] Furthermore, this invention also mentions a smoke box detection method based on radar and vision multi-mode fusion. This method employs the radar and vision multi-mode fusion smoke box detection system described above and includes the following steps:

[0032] Step 1: Before the carton sealing machine seals the carton, the light source of the pre-sealing visual inspection system adopts a strobe modulation control method to illuminate the end face of the cigarette box before sealing.

[0033] Step 2: Adjust the lighting to a suitable intensity that allows for clear imaging of the cigarette box, and then use an industrial camera to capture an image of the end face of the cigarette box before sealing.

[0034] Step 3: The analysis and processing module of the pre-sealing visual inspection system performs deep learning algorithm processing on the collected images of the inner side of the cigarette box before sealing, identifies missing strip information and transmits it along with the image data to the multimodal fusion system;

[0035] Step 4: After the carton sealing machine seals the carton, the radio frequency module of the radar imaging detection system emits terahertz / millimeter waves to both ends of the cigarette box on the conveyor line and receives the reflected echo signals. The radar point cloud data is obtained through data processing by the data acquisition module.

[0036] Step 5: The imaging processing module of the radar imaging detection system reconstructs the image of the cigarette box end face based on the radar point cloud data;

[0037] Step 6: Using the deep learning algorithm of the multimodal fusion system, the contour feature information of the region of interest of visual imaging and radar imaging before the cigarette box is sealed is extracted by the convolutional neural network. The attention mechanism is used to achieve deep fusion of the two types of contour feature information, namely visual and radar, and to make decisions on missing strips, flaw detection and arrangement pattern information.

[0038] Step 7: Under the operation of multiple image acquisition modules, the appearance quality inspection system acquires images of various surfaces of the cigarette box. The AI ​​large model recognition and classification module uses a convolutional neural network to extract surface feature information from the images of each surface of the cigarette box and intelligently identifies defect information, including wrinkles, dirt, and sealing tape, and transmits it to the multimodal fusion system. At the same time, image processing is used to obtain the barcode and QR code information of the cigarette box and send it to the control processing system.

[0039] Step 8: After receiving the identification information transmitted from each system, the multimodal fusion system will further integrate the identification information of defects, flaws, and arrangement patterns from visual imaging and radar imaging before sealing with the appearance feature identification information from the appearance quality inspection system. Through an attention mechanism fusion algorithm, the system will further fuse various features, output the defect classification decision, and transmit it to the control processing system.

[0040] Step 9: The control and processing system classifies, statistically summarizes, and stores the defects, flaws, arrangement, wrinkles, dirt, and sealing tape defects of each cigarette box, and associates them with the visual imaging, industrial radar imaging images, barcodes, and QR codes of each cigarette box to determine the qualified status of the cigarette box.

[0041] The beneficial technical effects of this invention are as follows:

[0042] The present invention provides a smoke box detection scheme that integrates radar and vision multi-modal fusion. It utilizes convolutional neural network technology to combine industrial radar imaging and visual imaging, and makes decisions on various defects through multi-modal fusion algorithms. This improves the efficiency and accuracy of information transmission and expression. The system's human-computer interaction unit can form an integrated information system, enabling unified operation and management and improving work efficiency.

[0043] The present invention provides a multi-mode fusion radar and vision cigarette box detection solution that can operate the overall control system as an independent human-machine interaction system, or it can be integrated with the cigarette factory's box-sealing machine. The human-machine interaction interface is integrated into the box-sealing machine system, reducing the number of on-site human-machine interaction screens, lowering costs, and improving the convenience and efficiency of production management through flexible deployment.

[0044] This invention provides a multi-mode fusion radar and vision method for cigarette box inspection. It can associate information on missing strips, damage, and arrangement of cigarette boxes with information on appearance wrinkles, dirt, tape sealing defects, and barcode / QR code information, and store the images as historical traceability basis to ultimately determine whether the tested packaging box is qualified. It enables a single system to perform comprehensive, non-destructive, and non-contact inspection of the inside and outside of the box, making up for the shortcomings of current market methods for inspecting sealed cigarette boxes. The inspection is efficient and the method is safe and reliable. Attached Figure Description

[0045] Figure 1 This is a functional block diagram of a smoke box detection system that integrates radar and vision multi-mode fusion according to the present invention.

[0046] Figure 2 System block diagram;

[0047] Figure 3 This is a schematic diagram of the system structure distribution for an example.

[0048] Figure 4 This is a picture of the actual cigarette box being tested.

[0049] Figure 5 A qualified industrial radar image;

[0050] Figure 6 This is a substandard industrial radar image.

[0051] Figure 7 Diagram showing damage to the inner packaging of cigarette packs;

[0052] Figure 8 The images and radar images are substandard.

[0053] Figure 9 This is an inspection image of the cigarette box. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0055] like Figure 1-2 As shown, the present invention provides a multi-modal fusion system for cigarette box inspection using radar and vision, including a pre-sealing visual inspection system, a radar imaging inspection system, an appearance quality inspection system (post-sealing visual inspection system), a multi-modal fusion system, and a control and processing system.

[0056] The pre-sealing visual inspection system is used to acquire and process images of the end face of the cigarette boxes before sealing.

[0057] The radar imaging inspection system is used to collect radar echo information and perform imaging inspection and identification of the internal state of the cigarette box after it is sealed, including the extraction of feature information such as missing strips, internal flaw detection, and arrangement pattern.

[0058] The appearance quality inspection system is used to collect information on the surface of cigarette boxes, including the detection of wrinkles, dirt, and defects in the sealing tape, as well as barcode and QR code information.

[0059] The multimodal fusion system employs a hybrid fusion method to train and fuse several modalities, including visual imaging before sealing the cigarette box, radar imaging appearance, and visual imaging, and then makes a decision output.

[0060] The control and processing system is used to summarize, classify, and store information on missing strips, flaw detection, arrangement, appearance defects, barcodes, and QR codes in cigarette boxes.

[0061] like Figure 3 As shown, the system of this invention is applied to the comprehensive quality inspection of cigarette cartons during the conveyor belt transport process at the rear of the carton sealing machine or the finished product warehouse, focusing on missing parts, internal flaw detection, and the inspection of their arrangement and appearance. The tested item is a cigarette carton, as shown in the supporting documents. Figure 4 As shown. The pre-sealing visual inspection system is used for visual inspection of missing strips in the cigarette boxes before sealing; the radar imaging inspection system is used for detecting missing strips, internal flaws, and arrangement patterns in the cigarette boxes; the appearance quality inspection system is used for detecting defects such as wrinkles, dirt, and tape seals on the appearance of the cigarette boxes, as well as barcodes and QR codes; the multimodal fusion system is used to achieve the fusion of recognition information from pre-sealing visual imaging, radar imaging, and appearance visual imaging modalities, and to output decision data; the control processing system associates and stores the barcode and QR code information of each cigarette box with flaw detection defect information and appearance defect information.

[0062] When the cigarette box is in the tape sealing position, the pre-sealing vision inspection system uses an industrial camera to capture images of the end faces of 25 cigarette packs arranged in a 5×5 pattern inside the cigarette box under appropriate lighting conditions. Since each cigarette box contains 50 cigarette packs, and the push rod loads every 25 packs, images are captured twice during the loading process. The analysis and processing module of the pre-sealing vision inspection system processes the captured images using deep learning algorithms, extracts the contour recognition information of the region of interest, transmits it to the multimodal fusion system, and transmits the image data to the control processing system.

[0063] When the cigarette box is transported to the radar imaging detection system on the production line, a radio frequency front-end module transmits and receives terahertz / millimeter-wave signals. The terahertz / millimeter-waves illuminate the cigarette box under test, and then a receiving antenna receives the reflected signals from the cigarette pack ends inside the box. The reflected signals undergo relevant signal processing, and a radar imaging algorithm reconstructs the shape image of the cigarette pack arrangement inside the box. A qualified reconstructed image is shown in the certification document. Figure 5 As shown, the image of the end face of the cigarette packs arranged in a 5*5 pattern inside the cigarette box was reconstructed.

[0064] After obtaining the reconstructed image, the multimodal fusion system uses a deep learning algorithm of multi-frame convolutional neural network to identify and extract contour feature information in the regions of interest of visual imaging and radar imaging before the cigarette box is sealed. It uses an attention mechanism to achieve deep fusion of the two types of contour feature recognition information, namely visual and radar, and inputs them into the model for training to obtain the model output.

[0065] The appearance quality inspection system operates independently, featuring multiple industrial cameras and several lighting sources. Based on the actual site conditions, the industrial cameras are installed with multi-angle adjustable mounting and equipped with polarizing lenses. After setting and adjusting parameters such as exposure and gain, the stroboscopic modulated light source illuminates the cigarette box, capturing images from six sides: top, bottom, left, right, front, and back. The images undergo preprocessing such as grayscale conversion, binarization, and smoothing / denoising. Feature regions are then segmented. Simple features are segmented using thresholding or region growing methods, while complex backgrounds are segmented using a semantic segmentation algorithm trained on a neural network. Defect information such as the presence, damage, and wrinkles of sealing tape on the front and back sides is extracted and identified. Barcodes and QR codes on the left and right sides are decoded using an image processing library, and OCR recognition and verification of digital characters are performed. Brand patterns on the top and bottom sides are identified using features extracted using a deep learning algorithm, as shown in the supporting documents. Figure 9 As shown in the figure. The final appearance quality inspection system transmits the identification information of the cigarette box's appearance features to the multimodal fusion system, and transmits the barcode and QR code information to the control and processing system.

[0066] The multimodal fusion system fuses the feature information from the regions of interest of visual imaging and radar imaging before sealing, and then trains the model output to obtain the fused model output. This model output is then combined with the feature information from the appearance quality imaging, and further fused using an attention mechanism for decision-level fusion. The resulting decisions address issues such as missing strips, flaw detection, arrangement, wrinkles and dirt on the cigarette box, mislabeling, and tape sealing defects. Defects such as damage and missing strips are detailed in the documentation. Figure 6 As shown in the proof document Figure 6 The "2" in the document indicates the missing carton, proving that... Figure 6 The "1" in the text indicates damage to the carton; the actual damage to the carton is as shown in the documentation. Figure 7 As shown in the document. A comparison is made between the physical photograph and the radar image of the defective item (e.g., the one with missing parts). Figure 8 As shown. Finally, the system sends the results to the control processing system.

[0067] After obtaining information on internal flaw detection defects, external defects, and barcode and QR code information from the cigarette box, the control and processing system uses a database unit to store statistical correlation data of defects in all inspected cigarette boxes, and to determine and store whether the cigarette box is qualified. The human-machine interaction unit is used by the operator to view and statistically analyze various data and image information of the system and interact with the machine.

[0068] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A smoke box detection system that integrates radar and vision multi-mode fusion, characterized in that: It includes a pre-sealing visual inspection system, a radar imaging inspection system, an appearance quality inspection system, a multi-modal fusion system, and a control and processing system; The pre-sealing visual inspection system is configured to acquire and process images of the end face of the cigarette boxes before sealing. The radar imaging detection system is configured to acquire radar echo information and perform imaging detection and identification of the internal state of the cigarette box after it is sealed, including the extraction of feature information such as missing strips, internal flaw detection, and arrangement pattern. The appearance quality inspection system is configured to collect information on the surface images of the cigarette box, including the detection of wrinkles, dirt, and defects in the sealing tape, as well as barcode and QR code information. The multimodal fusion system employs a hybrid fusion method and is configured to fuse several modalities, including visual imaging before sealing of cigarette boxes, radar imaging appearance, and visual imaging, through model training and decision output. The multimodal fusion system acquires all identification data of the same cigarette box before sealing using both visual imaging and radar imaging modalities, achieving time-synchronous decision-making for missing strips, flaw detection, and arrangement pattern information. The multimodal fusion system uses a convolutional neural network to extract contour feature information from the regions of interest in visual and radar imaging before the cigarette box is sealed. The system groups and associates these features according to their type, and uses an attention mechanism to achieve deep fusion and recognition of the two types of contour feature information from vision and radar to obtain the fusion output. The output is then input into the model before the decision-level fusion. The multimodal fusion system first performs feature-level fusion of visual imaging and radar imaging features before the cigarette box is sealed, and uses the fused output as input data to train a model; then, it inputs the feature information of the appearance visual imaging modality into another model for training; the output results of the two model training are further processed by an attention mechanism fusion algorithm to perform decision-level fusion of various features, output the defect classification decision, and transmit it to the control processing system; The control and processing system is configured to summarize, statistically analyze, and classify information on missing strips, flaw detection, arrangement, appearance defects, barcodes, and QR codes in cigarette boxes.

2. The radar and vision multi-mode fusion smoke box detection system according to claim 1, characterized in that: The pre-sealing visual inspection system includes an image acquisition module and an analysis and processing module; The image acquisition module is configured to acquire images of the end face of the cigarette box before it is sealed. The analysis and processing module is configured to use deep learning algorithms to extract feature information of the end face of the carton in the image.

3. The radar and vision multi-mode fusion smoke box detection system according to claim 2, characterized in that: The image acquisition module includes an industrial camera and a stroboscopic modulated light source. The industrial camera sets the exposure value based on the light intensity and is equipped with a polarizing lens, enabling multi-angle and multi-directional adjustment. Under the illumination of the stroboscopic modulated light source, it acquires the end face image of the cigarette box before sealing.

4. The smoke box detection system based on radar and vision multi-mode fusion according to claim 1, characterized in that: The radar imaging detection system includes a radio frequency module, a data acquisition module, and an imaging processing module; The radio frequency module is configured for generating, modulating, transmitting, receiving, and demodulating terahertz / millimeter-wave signals and echo signals. The data acquisition module is configured to acquire demodulated data of the echo signal reflected from the smoke box; The imaging processing module is configured to generate reconstructed images from the data acquired by the data acquisition module using imaging algorithms, and then use deep learning algorithms to identify information such as missing bars, flaw detection, and arrangement patterns of the cigarette box in the reconstructed images.

5. The smoke box detection system based on radar and vision multi-mode fusion according to claim 1, characterized in that: The appearance quality inspection system includes an image acquisition module and an AI large model recognition and classification module; The AI ​​large-scale model recognition and classification module adopts a generative adversarial network (GAN). While training an image recognition model from cigarette box image samples, a deep neural network generator is trained to generate AI image samples. The deep neural network discriminator then identifies various defect features in the cigarette box images.

6. The smoke box detection system based on radar and vision multi-mode fusion according to claim 1, characterized in that: The attention mechanism introduces an attention weight vector and then uses a neural network structure to learn a function to calculate the weights. This function dynamically calculates the attention weights based on the input of modal features to reflect the importance of different modalities in the fusion process.

7. The smoke box detection system based on radar and vision multi-mode fusion according to claim 1, characterized in that: The control and processing system includes a human-computer interaction unit and a database unit; The human-computer interaction unit is configured for operators to view and analyze various data and image information of the system and interact with the machine. The database unit is configured to store statistical correlation data on defects in all inspected cigarette boxes.

8. The radar and vision multi-mode fusion smoke box detection system according to claim 7, characterized in that: The control and processing system can statistically and classify the status of each cigarette box, including whether it has missing strips, visual and radar images, appearance defects, and their association information with barcodes and QR codes. The control and processing system can classify and statistically store different appearance defect types according to wrinkles, dirt, and sealing tape defects. Operators can view each defect type by year, month, day, shift, or batch through the human-machine interface. The control and processing system can operate independently through the system's human-machine interaction unit or be integrated into the box sealing machine system interface.

9. A method for detecting smoke boxes using radar and vision multi-mode fusion, characterized in that: The smoke box detection system using radar and vision multi-mode fusion as described in claim 3 includes the following steps: Step 1: Before the carton sealing machine seals the carton, the light source of the pre-sealing visual inspection system adopts a strobe modulation control method to illuminate the end face of the cigarette box before sealing. Step 2: Adjust the lighting to a suitable intensity that allows for clear imaging of the cigarette box, and then use an industrial camera to capture an image of the end face of the cigarette box before sealing. Step 3: The analysis and processing module of the pre-sealing visual inspection system performs deep learning algorithm processing on the collected images of the inner side of the cigarette box before sealing, identifies missing strip information and transmits it along with the image data to the multimodal fusion system; Step 4: After the carton sealing machine seals the carton, the radio frequency module of the radar imaging detection system emits terahertz / millimeter waves to both ends of the cigarette box on the conveyor line and receives the reflected echo signals. The radar point cloud data is obtained through data processing by the data acquisition module. Step 5: The imaging processing module of the radar imaging detection system reconstructs the image of the cigarette box end face based on the radar point cloud data; Step 6: Using the deep learning algorithm of the multimodal fusion system, the contour feature information of the region of interest of visual imaging and radar imaging before the cigarette box is sealed is extracted by the convolutional neural network. The attention mechanism is used to achieve deep fusion of the two types of contour feature information, namely visual and radar, and to make decisions on missing strips, flaw detection and arrangement pattern information. Step 7: Under the operation of multiple image acquisition modules, the appearance quality inspection system acquires images of various surfaces of the cigarette box. The AI ​​large model recognition and classification module uses a convolutional neural network to extract surface feature information from the images of each surface of the cigarette box and intelligently identifies defect information, including wrinkles, dirt, and sealing tape, and transmits it to the multimodal fusion system. At the same time, image processing is used to obtain the barcode and QR code information of the cigarette box and send it to the control processing system. Step 8: After receiving the identification information transmitted from each system, the multimodal fusion system will further integrate the identification information of defects, flaws, and arrangement patterns from visual imaging and radar imaging before sealing with the appearance feature identification information from the appearance quality inspection system. Through an attention mechanism fusion algorithm, the system will further fuse various features, output the defect classification decision, and transmit it to the control processing system. Step 9: The control and processing system classifies, statistically summarizes, and stores the defects, flaws, arrangement, wrinkles, dirt, and sealing tape defects of each cigarette box, and associates them with the visual imaging, industrial radar imaging images, barcodes, and QR codes of each cigarette box to determine the qualified status of the cigarette box.