A multi-modal electronic cigarette pressure relief safety detection system and method based on visual perception
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA NAT TOBACCO QUALITY SUPERVISION & TEST CENT
- Filing Date
- 2026-04-09
- Publication Date
- 2026-07-14
Smart Images
Figure CN122391802A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical fields of pattern recognition and deep learning, and more specifically, to a multimodal electronic cigarette pressure relief safety detection method and system based on visual perception. Background Technology
[0002] With the rapid development of the e-cigarette industry, its product forms are becoming increasingly diversified, from traditional stick-type to irregularly shaped structures, and from fixed mouthpieces to replaceable cartridge designs, leading to a continuous increase in the complexity of manufacturing and quality control. Among these advancements, safety testing during the pressure relief process is a core aspect of ensuring product quality and user safety. Pressure relief failure can lead to risks such as abnormal vapor emission, sparks, casing explosions, and even mouthpiece separation, directly threatening consumer safety and operational safety on the production line. Therefore, e-cigarette pressure relief safety testing technology has become one of the key research directions in the field of intelligent tobacco manufacturing.
[0003] Currently, safety testing for e-cigarette pressure relief mainly relies on manual sampling, single-modal sensor detection, or simple machine vision recognition. Existing testing solutions mostly focus on the identification of single physical features, such as detecting smoke concentration through smoke sensors or monitoring abnormal temperatures through infrared sensors. They lack the ability to conduct collaborative detection of multiple risk scenarios, including smoke-flame-explosion-mouthpiece separation.
[0004] Secondly, most mainstream detection systems adopt a single-module independent working mode, and the data from modules such as visual detection, audio recognition, and motion tracking are not effectively integrated. For example, the visual module only outputs the coordinates of the smoke area, but does not combine it with the explosion spectrum characteristics of the audio module to verify the risk level; motion analysis only focuses on the static judgment of the mouthpiece position, without dynamically linking it with the posture of the e-cigarette, resulting in fragmented detection logic and difficulty in building a complete risk assessment model. At the same time, traditional machine vision algorithms have poor adaptability to dynamic scenes, and are prone to feature extraction lag in rapidly changing scenarios such as instantaneous smoke diffusion and flickering flames during the depressurization process of e-cigarettes.
[0005] With the continuous improvement of intelligent manufacturing and safe production requirements, the industry has put forward new demands for multi-dimensional integration, high versatility and reliability in the safety testing of e-cigarette pressure relief: it needs to cover multiple risk characteristics such as smoke, fire, explosion and mouthpiece separation, adapt to different product specifications and complex working conditions, and realize the automation and lightweighting of the testing process.
[0006] Therefore, how to overcome the bottlenecks of existing technologies, such as single detection dimensions, poor adaptability, weak data collaboration, and cumbersome calibration, and to build an electronic cigarette pressure relief safety detection system that is free from complex calibration and can be quickly deployed, has become a technical problem that the industry needs to solve.
[0007] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention
[0008] Therefore, it is necessary to provide a multimodal electronic cigarette pressure relief safety detection method and system based on visual perception to address the above-mentioned technical problems. This method integrates image visual and audio features, covers four types of risks: smoke emission, flame, explosion, and mouthpiece separation, and forms a complete risk assessment system. This will overcome the shortcomings of existing electronic cigarette pressure relief safety detection technologies, such as single detection dimension, weak data collaboration, poor dynamic adaptability, and cumbersome calibration.
[0009] To achieve the above objectives, a first aspect of the present invention provides a multimodal electronic cigarette pressure relief safety detection method based on visual perception, comprising:
[0010] A dataset containing e-cigarette rods and mouthpieces was constructed, and the YOLO model was trained and optimized to obtain a YOLO object detection model; a dataset containing e-cigarette depressurization explosion sounds and secondary background sounds was constructed, and the SVM model was trained and optimized to obtain an SVM audio classification model.
[0011] The system acquires video and audio streams during the decompression process of an electronic cigarette in real time, and synchronizes the timestamps of the image data in the video stream with the audio data in the audio stream.
[0012] Image data is processed using the YOLO target detection model to obtain the mouthpiece and stem regions. Feature extraction is performed on the mouthpiece and stem regions to obtain mouthpiece separation features. The image data is converted to the HSV color space, and the flame region is initially located in the image data based on the HSV color threshold. Feature extraction is performed on the flame region to obtain flame features. The image data is then processed using a fusion algorithm of inter-frame difference and optical flow to obtain smoke emission features.
[0013] The temporal, frequency, and time-frequency features of the audio data corresponding to the image data are extracted and fed into the SVM audio classification model to obtain the explosion features.
[0014] The safety of e-cigarettes is judged by comprehensively considering the characteristics of mouthpiece separation, flame, smoke emission, and explosion.
[0015] This solution, through deep learning and fully automated process design, completely solves the four major bottlenecks of traditional testing: single dimension, poor adaptability, weak collaboration, and cumbersome calibration. It constructs a high-precision, robust, barrier-free, and fast-deployment method for testing the pressure relief safety of e-cigarettes.
[0016] Specifically, the system simultaneously utilizes a YOLO deep learning model to process video streams to identify cigarette holder separation, flames, and smoke, and an SVM model to process audio streams to identify explosion sounds. This multi-dimensional approach integrates changes in physical structure, combustion state, smoke dynamics, and sound events to form a comprehensive safety assessment. Even when one dimension is compromised, such as visual obstruction or excessive environmental noise, other dimensions can still provide valuable information, significantly reducing the probability of false alarms and missed alarms.
[0017] Based on YOLO, a deep convolutional neural network automatically learns the high-dimensional semantic features of cigarette holders and stems, accurately locating them even when the target is deformed, rotated, or under poor lighting conditions, demonstrating strong generalization ability. It integrates inter-frame difference and optical flow methods to capture the spatiotemporal motion features of smoke during depressurization, upgrading static image analysis to a dynamic understanding of the depressurization process. Cigarette holder separation features, flame features, and smoke features are all automatically extracted by the algorithm, eliminating the need for manually defined feature rules. Adaptive thresholding significantly improves the scientific rigor and objectivity of the detection.
[0018] Furthermore, the entire process, from audio and video stream access, timestamp alignment, model inference, feature extraction to final security decision-making, is automated without manual intervention in any intermediate steps. The automatic audio and video alignment technology solves the time synchronization problem for multimodal data, ensuring a strict logical correspondence between visual and auditory events, eliminating the need for manual verification. It supports real-time acquisition and synchronous processing of video and audio streams, meeting the needs of industrial-grade real-time detection.
[0019] To achieve the above objectives, a second aspect of the present invention provides a multimodal electronic cigarette pressure relief safety detection system based on visual perception, comprising:
[0020] Explosion-proof enclosure, used to provide a safe and isolated environment for pressure relief testing of electronic cigarettes;
[0021] The explosion-proof cabinet is equipped with an explosion-proof fan on the top, and module mounting brackets on the side and front walls for fixing the electronic cigarette image acquisition module and the electronic cigarette audio detection module, respectively; an electronic cigarette fixing fixture and a host computer are set at the bottom, and a display is installed on the outer wall to display various data during the electronic cigarette depressurization process;
[0022] The electronic cigarette image acquisition module is used to acquire video streams during the depressurization process of electronic cigarettes.
[0023] The electronic cigarette audio detection module is used to collect the audio stream during the depressurization process of the electronic cigarette;
[0024] The data receiving unit is used to receive video streams and audio data streams;
[0025] The host computer is configured to run the visual perception-based electronic cigarette pressure relief safety detection method described in the first aspect and output the safety judgment result.
[0026] To achieve the above objectives, a third aspect of the present invention provides a computer device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to execute the program stored in the memory to implement the steps of the visual perception-based electronic cigarette pressure relief safety detection method described in the first aspect.
[0027] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the visual perception-based electronic cigarette pressure relief safety detection method described in the first aspect.
[0028] To achieve the above objectives, the fifth aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the visual perception-based electronic cigarette pressure relief safety detection method as described in the first aspect.
[0029] The beneficial effects of this invention are as follows:
[0030] This invention employs a video vision + audio fusion architecture, simultaneously acquiring video and audio streams of the e-cigarette depressurization process to achieve collaborative monitoring of four types of risks: smoke, flame, explosion, and mouthpiece separation, breaking through the limitations of traditional single-modal detection. By optimizing the detection frame size of the mouthpiece and the e-cigarette body, and using data augmentation and precise loss functions, the confidence level of e-cigarette body and mouthpiece detection is improved. Simultaneously, it integrates technologies such as color segmentation, morphological processing, inter-frame differencing and optical flow methods, dynamic coordinate system analysis, and multi-dimensional audio feature extraction to form a complete feature extraction system, maintaining stable risk identification capabilities even when facing complex e-cigarette shapes such as irregular mouthpieces and irregularly shaped e-cigarette bodies.
[0031] The multimodal electronic cigarette pressure relief safety detection system is highly adaptable and covers a variety of detection needs. It is especially suitable for electronic cigarette pressure relief safety detection scenarios and can flexibly handle the detection needs of different specifications of electronic cigarettes such as stick type and replaceable cartridge type, as well as different cigarette postures from 0° to 90°. It does not require frequent adjustment of equipment parameters for different scenarios, thus solving the problem of poor adaptability of traditional detection solutions to product specifications. Attached Figure Description
[0032] Figure 1 This is an overall structural diagram of a visual perception-based electronic cigarette pressure relief safety detection method according to the present invention.
[0033] Figure 2 The images shown are of the physical object and model in one embodiment of the present invention.
[0034] Figure 3 This is a camera installation diagram according to one embodiment of the present invention.
[0035] Figure 4 This is an example diagram of the interface in one embodiment of the present invention.
[0036] Figure 5 This is an explosion detection diagram in one embodiment of the present invention.
[0037] Figure 6 This is a flame detection diagram in one embodiment of the present invention.
[0038] Figure 7 This is a coordinate establishment diagram in one embodiment of the present invention.
[0039] Figure 2 In the diagram, 1 is the explosion-proof cabinet, 2 is the exhaust and purification assembly, 3 is the display screen, 4 is the front observation window, 5 is the mechanical door, 6 is the host computer, and 7 is the voltage detection module.
[0040] Figure 3 In the diagram, A is the top image acquisition module, B and C are the side image acquisition modules, D is the front image acquisition module, and E is the high-sensitivity microphone. Detailed Implementation
[0041] The technical solution of the present invention will be further described in detail below through specific embodiments.
[0042] Example 1
[0043] This embodiment provides a multimodal electronic cigarette pressure relief safety detection method based on visual perception, such as... Figure 1 As shown, it includes:
[0044] A YOLO object detection model was trained based on an e-cigarette image dataset, and an SVM audio classification model was trained based on an e-cigarette depressurization audio dataset.
[0045] The system acquires video and audio streams during the decompression process of an electronic cigarette in real time, and synchronizes the timestamps of the image data in the video stream with the audio data in the audio stream.
[0046] In one embodiment, after acquiring the video stream during the depressurization process of the electronic cigarette, the original image data in the video stream is first subjected to Gaussian filtering for noise reduction and brightness normalization to retain the edge features of the cigarette rod and mouthpiece.
[0047] After acquiring the audio stream during the depressurization process of an electronic cigarette, the raw audio data in the audio stream is converted into a NumPy array, and the NumPy array is processed using a first-order differential filter. A short-time Fourier transform is then performed on the NumPy array after the first-order differential filter to generate a two-dimensional time-frequency matrix. Each vector in the two-dimensional time-frequency matrix is then normalized to eliminate interference caused by volume differences.
[0048] Specifically, the normalization of the two-dimensional time-frequency matrix is as follows:
[0049] ;
[0050] In the formula, S norm S is the normalized audio array, and S is the processed audio data array. max S is the global maximum value of the array. min It is the global minimum value of the array.
[0051] Image data is processed using the YOLO target detection model to obtain the mouthpiece and stem regions. Feature extraction is performed on the mouthpiece and stem regions to obtain mouthpiece separation features. The image data is converted to the HSV color space, and the flame region is initially located in the image data based on the HSV color threshold. Feature extraction is performed on the flame region to obtain flame features. The image data is then processed using a fusion algorithm of inter-frame difference and optical flow to obtain smoke emission features.
[0052] The temporal, frequency, and time-frequency features of the audio data corresponding to the image data are extracted and fed into the SVM audio classification model to obtain the explosion features.
[0053] The safety of e-cigarettes is judged by comprehensively considering the characteristics of mouthpiece separation, flame, smoke emission, and explosion.
[0054] In one specific embodiment, the training steps of the YOLO object detection model are as follows:
[0055] Collect e-cigarette samples of different sizes, take pictures in strong light, low light and side light scenes, label the two types of targets, e-cigarette rod and mouthpiece, and form an e-cigarette image set;
[0056] Based on the YOLOv5s model, the detection box size of the cigarette holder and the cigarette stem is optimized by Anchor clustering to solve the problem of insufficient detection accuracy of small targets such as the cigarette holder; Mosaic data augmentation and CIoU loss function are used for iterative training to obtain the YOLO object detection model.
[0057] Specifically, the Mosaic data augmentation technique is used to randomly scale and stitch together four training images to improve the model's adaptability to different cigarette postures; the CIoU loss function is used to consider bounding box overlap, center point distance and aspect ratio to improve target localization accuracy.
[0058] The training steps of the SVM audio classification model are as follows: Collect the electronic cigarette depressurization explosion sound and secondary background sounds (fan noise, ambient sound, etc.) to form an electronic cigarette depressurization audio dataset, extract 52-dimensional audio feature vectors, train the SVM model based on the 52-dimensional audio feature vectors to obtain the SVM explosion classification model, and store it through the pickle library.
[0059] Specifically, the 52-dimensional audio feature vector includes time-domain features, frequency-domain features, and time-frequency-domain features. The time-domain features include average amplitude, standard deviation, kurtosis, skewness, maximum amplitude, minimum amplitude, median amplitude, and total energy. The frequency-domain features include spectral centroid, spectral bandwidth, spectral roll-off, spectral flux, spectral entropy, and energy percentage of 10 frequency bands. The time-frequency-domain features include 13-dimensional Mel-frequency cepstral coefficients, 13-dimensional MFCC first-order difference, and 6-dimensional MFCC second-order difference.
[0060] It is understandable that the 52-dimensional audio feature vector in the training phase has the same dimension as the 52-dimensional audio feature vector fed into the SVM audio classification model in the application phase.
[0061] In one embodiment, image data is processed based on the YOLO object detection model to obtain a mouthpiece region and a cigarette stem region. Feature extraction is performed on the mouthpiece region and the cigarette stem region to obtain mouthpiece separation features, including:
[0062] The YOLO object detection model is used to process the image data, and the bounding box coordinates and confidence scores of the mouthpiece region and the cigarette stem region are output.
[0063] Calculate the center point of the mouthpiece and the center point of the rod based on the bounding box coordinates of the mouthpiece region and the rod region, respectively.
[0064] A two-dimensional local coordinate system is established with the center point of the cigarette stem as the origin and the principal axis of the cigarette stem as the positive X-axis. The principal axis of the cigarette stem is determined by calculating the minimum bounding rectangle angle of the cigarette stem's outline pixels. (See [link to relevant documentation]). Figure 7 ;
[0065] In a two-dimensional local coordinate system, the Euclidean distance between the center point of the cigarette holder and the origin is calculated, as well as the offset angle of the center point of the cigarette holder relative to the X-axis, which serves as the cigarette holder separation feature.
[0066] The Euclidean distance calculation formula is as follows:
[0067]
[0068] In the formula, D is the Euclidean distance, (x tip ,y tip (x) is the center point of the cigarette holder bounding box. rod ,y rod () is the origin.
[0069] The formula for calculating the offset angle is:
[0070]
[0071] In the formula, θ is the offset angle of the center point of the cigarette holder relative to the X-axis.
[0072] In one embodiment, the image data is converted to the HSV color space, the fire area is initially located in the image data based on the HSV color threshold, and features are extracted from the fire area, including:
[0073] The image data is converted to a color space to obtain an HSV format image;
[0074] Based on the preset HSV color range of the explosion flame, the flame area is initially located in the HSV format image to obtain the initial mask of the flame area; preferably, the HSV color range of the explosion flame is defined by the red-yellow flame characteristic of the flame.
[0075] Opening operations are used to remove noise from the initial mask of the flame area, and closing operations are used to fill the holes in the initial mask of the flame area. Then, continuous region contours are found in the initial mask of the flame area after opening and closing operations, the area of each continuous region contour is calculated, and valid flame contours are filtered according to the area threshold.
[0076] Calculate the centroid coordinates of the effective flame profile, and calculate the angle formed by the flame centroid with the center of the pipe and the center of the mouthpiece as the flame light characteristic. See [link to relevant documentation]. Figure 6 .
[0077] In one embodiment, an inter-frame difference and optical flow fusion algorithm is used to process image data to obtain smoke emission features, including:
[0078] The image data is converted to grayscale, and a historical frame list is constructed. When the number of accumulated grayscale images meets the preset frame number threshold, the inter-frame difference image of adjacent frames is obtained by the inter-frame difference method. The inter-frame difference image is subjected to Gaussian blur smoothing and threshold segmentation to generate a motion mask. The motion mask is then subjected to opening, closing and dilation operations in sequence to form a coherent motion region contour. The area, aspect ratio and contour direction of the motion region contour are calculated.
[0079] Among them, the pixel changes between two frames;
[0080]
[0081] In the formula, D(x,y) represents the pixel change value, It n (x,y) represents the pixel value of the current frame. n-1 (x,y) represents the pixel values of the previous frame.
[0082] Specifically, when calculating pixel differences between adjacent frames after grayscale conversion, the grayscale changes caused by subtle smoke are highlighted by lowering the difference threshold. Then, a Gaussian blur kernel is used to smooth the noise, and finally, an adaptive threshold segmentation is used to generate a motion mask to initially locate dynamic regions.
[0083] Optical flow algorithms are used to calculate the optical flow field between adjacent frames. The optical flow is decomposed into amplitude and angle to create an optical flow mask, which filters out weak motion interference. After performing morphological opening, closing, and dilation operations on the optical flow mask, the contour of the optical flow motion region is obtained. For each optical flow motion region contour, the standard deviation of the direction and the average amplitude of the optical flow in that region are calculated. Feature points with amplitude > 1.5 are retained to form an optical flow mask, focusing on the obvious motion region of the smoke. Preferably, the optical flow calculation is the Lucas-Kanade optical flow algorithm, which selects Shi-Tomasi corner points as feature points and calculates the motion vectors of feature points between adjacent frames, i.e., the standard deviation of the direction and the average amplitude.
[0084] Morphological opening operations are performed on the contours of the motion region and the optical flow motion region to remove small noise, and then dilation operations are performed to connect adjacent regions to obtain a fusion mask; the overlap between the fusion mask and the initial mask of the firelight region is calculated.
[0085] The overlap, along with the area, aspect ratio, contour directionality, optical flow standard deviation, and amplitude average of the optical flow of the motion region contour, are collectively used as smoke emission characteristics.
[0086] Specifically, if the overlap is less than the threshold and the contour area and aspect ratio meet the set threshold range, then the overlap, contour area, aspect ratio, etc. will be used as smoke characteristics.
[0087] In one embodiment, the temporal, frequency, and time-frequency features of the audio data corresponding to the image data are extracted and fed into an SVM audio classification model to obtain explosion features, including:
[0088] Calculate the spectral representation of the two-dimensional time-frequency matrix, and extract time-domain features, frequency-domain features, and time-frequency-domain features based on the spectral representation to form a 52-dimensional audio feature vector;
[0089] The extracted 52-dimensional audio feature vector is fed into an SVM audio classification model, and the output explosion confidence score is used as the explosion feature. See [link / reference needed]. Figure 5 .
[0090] In one embodiment, the safety of an electronic cigarette is determined by comprehensively considering the characteristics of the mouthpiece separation, the flame, the smoke, and the explosion, including: determining whether an explosion event has occurred based on the explosion, the flame, and the smoke.
[0091] Determine whether the pressure relief direction meets the preset standard based on the characteristics of the cigarette holder separation, the flame characteristics, and the smoke emission characteristics.
[0092] The safety assessment result is output based on whether an explosion event has occurred and whether the direction of pressure relief meets the preset standards.
[0093] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0094] Example 2
[0095] Based on the same inventive concept, this application also provides a vision-based multimodal electronic cigarette pressure relief safety detection system for implementing the aforementioned vision-based multimodal electronic cigarette pressure relief safety detection method. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more vision-based multimodal electronic cigarette pressure relief safety detection system embodiments provided below can be found in the limitations of the vision-based multimodal electronic cigarette pressure relief safety detection method described above, and will not be repeated here.
[0096] Specifically, the multimodal electronic cigarette pressure relief safety detection system, such as Figures 2-3 As shown, it includes:
[0097] Explosion-proof cabinet 1 is used to provide a safe and isolated environment for electronic cigarette pressure relief testing.
[0098] Specifically, the explosion-proof cabinet 1 is integrally welded from a 3mm thick cold-rolled steel plate. A mechanical door 5 is provided on the front of the cabinet 1, equipped with a high- and low-temperature resistant explosion-proof sealing ring and a mechanical interlocking device. A φ100mm explosion-proof pressure relief port is opened on the left side wall of the cabinet 1 to release internal explosion pressure in a directional manner, preventing the cabinet from cracking. A side observation window 4 is also provided on the side wall of the cabinet 1, made of thick explosion-proof glass, facilitating operators to observe the internal testing status while isolating risks. The explosion-proof cabinet adopts a top-integrated exhaust and purification assembly 2, including an explosion-proof exhaust fan, which is linked to the host computer 6 via a control line.
[0099] The bottom of the explosion-proof cabinet 1 is equipped with an electronic cigarette fixing fixture. Four image acquisition module mounting grooves are reserved on the side of the electronic cigarette fixing fixture. The inner wall of the groove is lined with a 2mm thick flame-retardant silicone pad to ensure that the module is installed in a sealed and dustproof manner.
[0100] The electronic cigarette image acquisition module is used to acquire video streams of the electronic cigarette depressurization process.
[0101] Specifically, the electronic cigarette image acquisition module includes four sets of image acquisition modules. One set of image acquisition modules is installed on the front wall of the explosion-proof cabinet to capture images of the front of the electronic cigarette. Two sets of image acquisition modules are installed on the left side wall of the explosion-proof cabinet to capture images of the side and upper diagonal areas of the electronic cigarette. The last set of image acquisition modules is installed on the top of the explosion-proof cabinet to capture images of the smoke area at the top of the electronic cigarette.
[0102] In one embodiment, four recesses are respectively formed on the front wall, side wall, and top of the metal cabinet, with inner diameters adapted to the housing of the acquisition module. Each recess integrates an adjustable angle bracket. The adjustable angle brackets in the front and side wall recesses are damped rotating structures, supporting bidirectional adjustment around the horizontal and vertical axes. The angle is locked via a knob, ensuring that each acquisition module can accurately align with the core pressure relief area of the e-cigarette. The adjustable angle bracket in the upper recess is a screw adjustment bracket, with a limit block at the end of the screw. The operator can adjust the screw according to the position of the e-cigarette to adapt to the top field of view coverage requirements of different e-cigarettes, maintaining a suitable angle between the upper image acquisition module and the top of the e-cigarette. Furthermore, an explosion-proof sealing sleeve is provided at the connection between the screw and the recess to prevent smoke leakage from the screw gaps and reduce vibration transmission during screw adjustment, ensuring stable imaging by the upper image acquisition module.
[0103] In one embodiment, the front image acquisition module D is installed in the groove on the front wall of the explosion-proof cabinet 1. It is aligned with the front of the electronic cigarette via a damped rotating bracket to ensure that the cigarette holder and mouthpiece occupy a significant portion of the image. The side image acquisition module is installed in the grooves (B, C) on the left side wall via a damped rotating bracket to cover the side and obliquely upward field of view. The top image acquisition module A is installed in the top groove via a screw-adjustable bracket to achieve adjustable angle of the acquisition module, adapting to the top field of view of different electronic cigarettes and focusing on the top smoke emission area.
[0104] It is understood that the four image acquisition modules are all cameras with a resolution of 1920×1080 and a frame rate of 30fps, and their lenses are equipped with anti-fog lenses.
[0105] The electronic cigarette audio detection module is used to collect the audio stream during the depressurization process of the electronic cigarette.
[0106] In one embodiment, the electronic cigarette audio detection module is disposed on the inner side wall of the explosion-proof cabinet to collect the audio stream during the pressure relief process and identify sound characteristics such as explosions and abnormal exhaust. Specifically, the electronic cigarette audio detection module includes a high-sensitivity microphone E, an audio preprocessing unit, and a data transmission unit. The high-sensitivity microphone E is installed on the side of the top recess and can be adjusted vertically to achieve rapid audio acquisition while avoiding interference from internal echoes. The data transmission unit is connected to the host computer 6 via a USB interface, and each frame of audio data is assigned a unified timestamp for synchronization with image data.
[0107] The host computer 6 is configured to run the electronic cigarette pressure relief safety detection method based on visual perception as described in Example 1, and output the safety judgment result.
[0108] Specifically, the host computer 6 serves as the core processing unit of the system, equipped with a CPU and a dedicated GPU, and has three main functional units built-in:
[0109] Data receiving unit: Receives four sets of image data via Gigabit Ethernet, audio data via USB, and voltage data, and stores them in association with timestamps.
[0110] Model processing unit: Runs a dedicated YOLO object detection model to identify visual objects and loads a pre-trained SVM audio classification model to output explosive confidence scores.
[0111] Display screen 3: A high-definition touchscreen connected to the host computer 6 via HDMI, used for visually displaying the testing process and results. In one embodiment, its interface is divided into functional zones, see [link to documentation]. Figure 4 It is divided into a real-time monitoring area, which synchronously displays the video stream from the image acquisition module and the audio waveform from the audio module. It supports touch operation and can retrieve historical detection logs.
[0112] Preferably, the host computer uses a multi-dimensional data association storage mechanism in the detection log storage. Specifically, in addition to the detection timestamp, security level, and risk type, the log file also stores the following data: keyframes of the e-cigarette image acquisition module, and displacement and angle of the mouthpiece separation. The log is stored in DOCX format and supports multi-condition retrieval by detection date, e-cigarette model, and risk level, which facilitates subsequent traceability analysis.
[0113] Preferably, the display screen adopts a synchronous display and interactive design, displaying the video stream of the acquisition module in real time, and overlaying the outline of the flame, the outline of the smoke, and the boundary frame of the smoke rod and the mouthpiece; the real-time spectrum diagram of the audio module is displayed at the bottom of the screen, and the total risk level and key parameters are displayed on the left side. It supports touch zooming in on any sub-window image, and clicking on the risk feature box can view the detailed parameters of the feature, which meets the interactive needs of industrial testing.
[0114] Furthermore, the multimodal electronic cigarette pressure relief safety detection system also includes a voltage detection module 7, which is set next to the electronic cigarette fixture. While the DC regulated power supply is connected to the electronic cigarette power supply interface to overcharge the electronic cigarette, it collects voltage and current data in real time during the pressure relief process.
[0115] Example 3
[0116] Based on the above embodiments, this embodiment provides a computer device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to execute the program stored in the memory to implement the steps of the visual perception-based electronic cigarette pressure relief safety detection method as described in Embodiment 1.
[0117] Example 4
[0118] Based on the above embodiments, this embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the visual perception-based electronic cigarette pressure relief safety detection method as described in Embodiment 1.
[0119] Example 5
[0120] Based on the above embodiments, this embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the visual perception-based electronic cigarette pressure relief safety detection method as described in Embodiment 1.
[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.
Claims
1. A multimodal electronic cigarette pressure relief safety detection method based on visual perception, characterized in that, include: A YOLO object detection model was trained based on an e-cigarette image dataset, and an SVM audio classification model was trained based on an e-cigarette depressurization audio dataset. The system acquires video and audio streams during the decompression process of an electronic cigarette in real time, and synchronizes the timestamps of the image data in the video stream with the audio data in the audio stream. Image data is processed using the YOLO object detection model to obtain the mouthpiece region and the stem region. Feature extraction is performed on the mouthpiece region and the stem region to obtain mouthpiece separation features. The image data is converted to the HSV color space. Based on the HSV color threshold, the flame region is initially located in the image data, and feature extraction is performed on the flame region to obtain flame features. An inter-frame difference and optical flow fusion algorithm is used to process image data to obtain smoke emission features; The temporal, frequency, and time-frequency features of the audio data corresponding to the image data are extracted and fed into the SVM audio classification model to obtain the explosion features. The safety of e-cigarettes is judged by comprehensively considering the characteristics of mouthpiece separation, flame, smoke emission, and explosion.
2. The method for safety detection of electronic cigarette pressure relief based on visual perception according to claim 1, characterized in that, After acquiring the video stream during the depressurization process of the electronic cigarette, the original image data in the video stream is first subjected to Gaussian filtering for noise reduction and brightness normalization. After acquiring the audio stream during the depressurization process of an electronic cigarette, the raw audio data in the audio stream is converted into a NumPy array, and the NumPy array is processed using a first-order differential filter. A short-time Fourier transform is performed on the NumPy array after the first-order differential filter to generate a two-dimensional time-frequency matrix. Each vector in the two-dimensional time-frequency matrix is normalized to eliminate interference caused by volume differences.
3. The electronic cigarette pressure relief safety detection method based on visual perception according to claim 1 or 2, characterized in that, Image data is processed using the YOLO object detection model to obtain the mouthpiece region and the stem region. Feature extraction is then performed on the mouthpiece and stem regions to obtain mouthpiece separation features, including: The YOLO object detection model is used to process the image data, and the bounding box coordinates and confidence scores of the mouthpiece region and the cigarette stem region are output. Calculate the center point of the mouthpiece and the center point of the rod based on the bounding box coordinates of the mouthpiece region and the rod region, respectively. A two-dimensional local coordinate system is established with the center point of the cigarette rod as the origin and the main axis direction of the cigarette rod as the positive X-axis. The main axis direction of the cigarette rod is determined by calculating the minimum bounding rectangle angle of the cigarette rod contour pixels. In a two-dimensional local coordinate system, the Euclidean distance between the center point of the cigarette holder and the origin is calculated, as well as the offset angle of the center point of the cigarette holder relative to the X-axis, which serves as the cigarette holder separation feature.
4. The electronic cigarette pressure relief safety detection method based on visual perception according to claim 3, characterized in that, The training steps for the YOLO object detection model are as follows: Collect electronic cigarette samples of different specifications, take pictures in strong light, low light and side light scenes, label two types of targets, namely the cigarette rod and the mouthpiece, and form a labeled image training set; Based on the YOLOv5s model, the detection box size of the mouthpiece and the cigarette holder is optimized by AnChor clustering, and the YOLO object detection model is obtained by iterative training using MosAiC data augmentation and CIoU loss function. The training steps of the SVM audio classification model are as follows: collect the depressurization explosion sound of an electronic cigarette and the secondary background sound, extract a 52-dimensional audio feature vector, train the SVM model based on the 52-dimensional audio feature vector, and obtain the SVM explosion classification model.
5. The electronic cigarette pressure relief safety detection method based on visual perception according to claim 2, characterized in that, The image data is converted to the HSV color space. Based on the HSV color threshold, the firelight area is initially located in the image data, and features are extracted from the firelight area, including: The image data is converted to a color space to obtain an HSV format image; Based on the preset HSV color range of the explosion light, the light area is initially located in the HSV format image to obtain the initial mask of the light area. Opening operations are used to remove noise from the initial mask of the flame area, and closing operations are used to fill the holes in the initial mask of the flame area. Then, continuous region contours are found in the initial mask of the flame area after opening and closing operations, the area of each continuous region contour is calculated, and valid flame contours are filtered according to the area threshold. Calculate the centroid coordinates of the effective flame profile, and calculate the angle formed by the flame centroid with the center of the smoke rod and the center of the mouthpiece as the flame light feature.
6. The electronic cigarette pressure relief safety detection method based on visual perception according to claim 5, characterized in that, Image data is processed using a fusion algorithm combining inter-frame difference and optical flow to obtain smoke emission features, including: The image data is converted to grayscale, and a historical frame list is constructed. When the number of accumulated grayscale images meets a preset frame threshold, the inter-frame difference image of adjacent frames is obtained through the inter-frame difference method. The inter-frame difference image is then subjected to Gaussian blur smoothing and threshold segmentation to generate a motion mask. The motion mask is then subjected to opening, closing, and dilation operations in sequence to form a coherent motion region contour. The area, aspect ratio, and contour directionality of the motion region contour are calculated. The optical flow algorithm is used to calculate the optical flow field between adjacent frames. The optical flow is decomposed into amplitude and angle, and an optical flow mask is created to filter out weak motion interference. After performing morphological opening, closing and dilation operations on the optical flow mask, the contour of the optical flow motion region is obtained. For each optical flow motion region contour, the standard deviation of the direction and the average amplitude of the optical flow in that region are calculated. Morphological opening operations are performed on the contours of the motion region and the optical flow motion region to remove small noise, and then dilation operations are performed to connect adjacent regions to obtain a fusion mask; the overlap between the fusion mask and the initial mask of the firelight region is calculated. The overlap, along with the area, aspect ratio, contour directionality, and standard deviation and average amplitude of the optical flow of the motion region contour, are combined to form the smoke emission characteristics.
7. The electronic cigarette pressure relief safety detection method based on visual perception according to claim 3, characterized in that, The temporal, frequency, and time-frequency features of the audio data corresponding to the image data are extracted and fed into the SVM audio classification model to obtain explosion features, including: The spectral representation of the two-dimensional time-frequency matrix is calculated, and time-domain features, frequency-domain features, and time-frequency-domain features are extracted based on the spectral representation to form a 52-dimensional audio feature vector. The time-domain features include mean amplitude, standard deviation, kurtosis, skewness, maximum amplitude, minimum amplitude, median amplitude, and total energy. The frequency-domain features include spectral centroid, spectral bandwidth, spectral roll-off, spectral flux, spectral entropy, and energy percentage of 10 frequency bands. The time-frequency-domain features include 13-dimensional Mel-frequency cepstral coefficients, 13-dimensional MFCC first-order difference, and 6-dimensional MFCC second-order difference. The extracted 52-dimensional audio feature vector is fed into the SVM audio classification model, and the exploded confidence score is output as the exploded feature.
8. The visual perception-based electronic cigarette pressure relief safety detection method according to claim 1, 2, 4, 5 or 6, characterized in that, The safety of e-cigarettes is judged comprehensively based on the characteristics of mouthpiece separation, flame, smoke, and explosion. This includes determining whether an explosion has occurred based on the characteristics of explosion, flame, and smoke. Determine whether the pressure relief direction meets the preset standard based on the characteristics of the cigarette holder separation, the flame characteristics, and the smoke emission characteristics. The safety assessment result is output based on whether an explosion event has occurred and whether the direction of pressure relief meets the preset standards.
9. A multimodal electronic cigarette pressure relief safety detection system based on visual perception, characterized in that, include: Explosion-proof enclosure, used to provide a safe and isolated environment for pressure relief testing of electronic cigarettes; The side walls and front walls of the explosion-proof cabinet are respectively equipped with an electronic cigarette image acquisition module and an electronic cigarette audio detection module. The bottom is equipped with an electronic cigarette fixing fixture and a host computer, and the outer wall is equipped with a display to display various data during the electronic cigarette depressurization process; The electronic cigarette image acquisition module is used to acquire video streams during the depressurization process of electronic cigarettes. The electronic cigarette audio detection module is used to collect the audio stream during the depressurization process of the electronic cigarette; The data receiving unit is used to receive video streams and audio data streams; The host computer is configured to run the visual perception-based electronic cigarette pressure relief safety detection method according to any one of claims 1-8 and output the safety judgment result.
10. A multimodal electronic cigarette pressure relief safety detection system based on visual perception according to claim 9, characterized in that, The electronic cigarette image acquisition module includes four sets of image acquisition modules. One set of image acquisition modules is installed on the front wall of the explosion-proof cabinet to capture images of the front of the electronic cigarette. Two sets of image acquisition modules are installed on the left side wall of the explosion-proof cabinet to capture images of the side and upper diagonal areas of the electronic cigarette. The last set of image acquisition modules is installed on the top of the explosion-proof cabinet to capture images of the smoke area at the top of the electronic cigarette.
11. The multimodal electronic cigarette pressure relief safety detection system based on visual perception according to claim 9, characterized in that, All four image acquisition modules are mounted on the explosion-proof cabinet via adjustable angle brackets. The adjustable angle brackets mounted on the top of the explosion-proof cabinet are also equipped with screws at their ends, and limit blocks are provided at the ends of the screws.
12. A multimodal electronic cigarette pressure relief safety detection system based on visual perception according to claim 8 or 9, characterized in that, It also includes a voltage detection module, which is set next to the electronic cigarette fixture. While the DC regulated power supply is connected to the electronic cigarette power supply interface to overcharge the electronic cigarette, it collects voltage and current data in real time during the depressurization process.
13. A computer device, characterized in that: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in a memory, it implements the steps of the visual perception-based electronic cigarette pressure relief safety detection method as described in any one of claims 1 to 8.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the electronic cigarette pressure relief safety detection method based on visual perception as described in any one of claims 1 to 8.
15. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the visual perception-based electronic cigarette pressure relief safety detection method as described in any one of claims 1 to 8.