Tobacco smoldering monitoring method based on binocular vision sensing

Abstract

A binocular vision sensing tobacco smoldering monitoring method comprises the following steps: first, firstly, using vision to conduct a smoldering experiment to obtain a large amount of data of smoke images of a large number of small targets such as tobacco leaves and tobacco shreds in a warehouse, and image data of black edges of objects after smoldering; secondly, respectively constructing deep learning networks for smoldering smoke and edges of objects after smoldering suitable for early small targets, and establishing an improved YOLOV7 deep learning network model based on an attention mechanism; thirdly, using the improved YOLOV7 to identify the features of objects after smoldering for coarse classification and recognition, combined with color image processing; fourthly, obtaining the probability of smoldering through an optimized smoldering recognition image algorithm; fifthly, determining the three-dimensional position where smoldering smoke and smoldering objects occur; locating the smoldering spatial position through binocular vision to obtain the position and area of ​​smoldering; sixthly, obtaining its big data through real-time monitoring of CO concentration, and using the network YOLOV7 to identify smoldering data and the probability of its occurrence, so as to determine whether it is in a smoldering state.

Description

Technical Field

[0001] The invention relates to a novel method for monitoring tobacco smoldering by binocular vision sensing. Background Art

[0002] At present, smoke detection mainly focuses on flame detection and infrared detection, and no in-depth research has been conducted on the extraction method of early suspected smoke areas (Jia Yang. Research on video fire detection method based on saliency detection and smoke spatiotemporal characteristics [D]. University of Science and Technology of China, 2016.), lack of visual methods for early non-flame small target smoldering smoke and the visualization of the edge features of smoldering objects, and AI intelligent monitoring and decision-making methods that fuse vision and CO sensor data. To this end, we developed a tobacco smoldering monitoring method using binocular visual sensing.

[0003] Invention content

[0004] The technical problem to be solved by the new invention is to provide a binocular vision sensing tobacco smoldering monitoring method, which is based on a network-based AI smoldering recognition decision system, and designs visual-based feature-level and decision-level submodules; the fusion decision module of visual sensing and CO sensing uses DS evidence theory to make decisions, judge normal and abnormal smoldering, and obtain smoldering decisions. If abnormal, an early warning is issued, and the server or mobile phone can get an early warning prompt so that the technician can make a decision.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is to provide a binocular vision sensing tobacco smoldering monitoring method, which comprises the following steps:

[0006] First, we use vision to conduct smoldering experiments to obtain a large amount of smoke image data of a large number of small targets such as tobacco leaves and tobacco shreds in the warehouse, as well as image data of the black edges of objects after smoldering;

[0007] Second, we construct deep learning networks for the smoldering smoke and the edge of objects after smoldering, which are suitable for early small targets, and establish an improved YOLOV7 deep learning network model based on the attention mechanism, so as to be suitable for the recognition of small targets in smoldering smoke;

[0008] 3. Use the model to detect smoke or suspected smoke areas and identify smoldering smoke; then use the improved YOLOV7 to identify the features of smoldering objects for rough classification and recognition, combine color image processing, use color space algorithm to calculate the edges of smoldering objects, and find the optimized smoldering recognition image algorithm;

[0009] Fourth, through the optimized smoldering recognition image algorithm, the edges of smoldering smoke and smoldering objects are identified respectively. Visual smoldering detection is designed to fuse image feature attributes, find out the internal relationship between image feature attributes and attributes, weight the smoldering feature attributes and edge features, and finally obtain the probability of smoldering;

[0010] 5. Determine the three-dimensional location of the smoldering smoke and the objects after smoldering; locate the smoldering space position through binocular vision to obtain the location and area of ​​smoldering;

[0011] 6. Obtain big data through real-time monitoring of CO concentration, and use the deep learning network YOLOV7 to identify smoldering data and the probability of its occurrence to determine whether it is in a smoldering state.

[0012] In one or more embodiments of the present invention, in step one, a network-controlled binocular vision and a bracket are installed above the bundled cigarette lines or tobacco leaves (referred to as cigarette packs). In order to expand the field of view, the binocular vision direction is adjustable by 270 degrees, and there is a bionic hood and a device for automatically wiping the visual lens regularly to prevent smoke and ash from affecting the detection accuracy. It is designed to automatically adjust 90 degrees every hour; at the same time, a network-controlled CO sensor is placed between every 2 bundles of cigarette packs in the bundled stacking area, and the number of sensors is selected according to the number of cigarette packs.

[0013] In one or more embodiments of the present invention, in step five, in order to make smoldering identification more accurate and reduce the probability of misidentification, a CO sensor may be added on the basis of visual detection.

[0014] In one or more embodiments of the present invention, in step six, if there is an abnormality, an early warning is issued, and the server or mobile phone can receive an early warning prompt to facilitate technical personnel to make decisions.

[0015] Compared with the background technology, the new invention has the following effects: adopting the above technical solution, the network-based AI smoldering recognition decision system designs visual feature-level and decision-level submodules; the fusion decision module of visual sensing and CO sensing uses DS evidence theory to make decisions, judge normal and abnormal smoldering, and obtain smoldering decisions. If abnormal, an early warning is issued, and the server or mobile phone can get an early warning prompt to facilitate technical personnel to make decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the positions of the temperature sensor and the CO sensor in one embodiment of the present invention;

[0017] Those skilled in the art can understand the shapes, structures and schemes obtained from the figures shown in the drawings. The various components of the drawings are not necessarily to scale, and the sizes of the various components and elements of the drawings can be enlarged or reduced to more clearly illustrate the new implementation scheme of the invention described in this article. DETAILED DESCRIPTION

[0018] The following references Figure 1To describe specific embodiments, however, those skilled in the art will easily understand that the detailed description given herein with respect to these drawings is for illustrative purposes only and should not be understood as limiting. This embodiment provides a binocular vision sensing tobacco smoldering monitoring method of the present invention, which includes the following steps:

[0019] First, we use vision to conduct smoldering experiments to obtain a large amount of smoke image data of a large number of small targets such as tobacco leaves and tobacco shreds in the warehouse, as well as image data of the black edges of objects after smoldering;

[0020] Second, we construct deep learning networks for the smoldering smoke and the edge of objects after smoldering, which are suitable for early small targets, and establish an improved YOLOV7 deep learning network model based on the attention mechanism, so as to be suitable for the recognition of small targets in smoldering smoke;

[0021] 3. Use the model to detect smoke or suspected smoke areas and identify smoldering smoke; then use the improved YOLOV7 to identify the features of smoldering objects for rough classification and recognition, combine color image processing, use color space algorithm to calculate the edges of smoldering objects, and find the optimized smoldering recognition image algorithm;

[0022] Fourth, through the optimized smoldering recognition image algorithm, the edges of smoldering smoke and smoldering objects are identified respectively. Visual smoldering detection is designed to fuse image feature attributes, find out the internal relationship between image feature attributes and attributes, weight the smoldering feature attributes and edge features, and finally obtain the probability of smoldering;

[0023] 5. Determine the three-dimensional location of the smoldering smoke and the objects after smoldering; locate the smoldering space position through binocular vision to obtain the location and area of ​​smoldering;

[0024] 6. Obtain big data through real-time monitoring of CO concentration, and use the deep learning network YOLOV7 to identify smoldering data and the probability of its occurrence to determine whether it is in a smoldering state.

[0025] In step 1, a network-controlled binocular vision and bracket are installed above the bundled cigarette lines or tobacco leaves (referred to as cigarette packs). In order to expand the field of view, the binocular vision direction is adjustable by 270 degrees, and there is a bionic cover and a device for automatically wiping the visual lens regularly to prevent smoke and ash from affecting the detection accuracy. It is designed to automatically adjust 90 degrees every hour; at the same time, a network-controlled CO sensor is placed between every 2 bundles of cigarette packs in the bundled stacking area, and the number of sensors is selected according to the number of cigarette packs.

[0026] In step five, to make smoldering identification more accurate and reduce the probability of misidentification, a CO sensor can be added on the basis of visual detection.

[0027] In step six, if there is an abnormality, an early warning will be issued, and the server or mobile phone can receive the early warning prompt to facilitate technical personnel to make decisions.

[0028] In summary of the above contents and in combination with all the figures, the present invention mainly monitors the phenomenon of black edges of objects produced by smoldering of tobacco leaves, tobacco shreds, etc. stacked in bundles in tobacco warehouses. However, sometimes the smoldering starts inside the bundled objects, with only a small amount of smoke, so it is also necessary to perform image recognition on the initial smoldering smoke. The present invention installs binocular vision and bracket based on network control above the tobacco lines or tobacco leaves (referred to as cigarette packs) stacked in bundles. In order to expand the field of view, the binocular vision direction is adjustable by 270 degrees, and there is a bionic cover and a device for automatically wiping the visual lens regularly to prevent smoke and ash from affecting the detection accuracy. The design automatically adjusts 90 degrees every hour. At the same time, a network-controlled CO sensor 1, 2, 3 is placed between every 2 bundles of cigarette packs in the bundled stacking place, and the number of sensors is selected according to the number of cigarette packs. First, a smoldering experiment is performed using vision to obtain a large amount of data of smoke images of a large number of small targets such as tobacco leaves and tobacco shreds in the warehouse, and image data of black edges of objects after smoldering (the edges of objects after smoldering are usually black).

[0029] The deep learning networks for the smoldering smoke and the edge of the object after smoldering are respectively constructed, which are suitable for the recognition of the small target of smoldering smoke. The improved YOLOV7 deep learning network model based on the attention mechanism is established to be suitable for the recognition of the small target of smoldering smoke. The model is first used to detect the smoke or suspected smoke area to identify the smoldering smoke; then the improved YOLOV7 is used to identify the features of the object after smoldering for rough classification and recognition. Combined with color image processing, the edge of the object after smoldering is calculated using the color space algorithm to find the optimized smoldering recognition image algorithm. Through the above algorithm, the edges of smoldering smoke and objects after smoldering are respectively identified. The visual smoldering detection is designed to perform image feature attribute fusion, find out the internal relationship between the image feature attributes and the attributes, weight the smoldering feature attributes and edge features, and finally obtain the probability of smoldering.

[0030] Determine the three-dimensional location of the smoldering smoke and the objects after smoldering. Use binocular vision to locate the smoldering space and obtain the location and area of ​​smoldering.

[0031] In the early stage of smoldering, there is less smoke, the target is small and white, which may cause misidentification. However, the black edge of the object after smoldering occurs inside the cigarette pack, so the phenomenon cannot be detected. It can only be detected when the smoldering reaches a certain degree and spreads to the outer surface of the package. In order to make smoldering identification more accurate and reduce the probability of misidentification, on the basis of visual detection, a CO sensor is added to obtain big data through real-time monitoring of CO concentration, and the deep learning network YOLOV7 is used to identify smoldering data and its probability of occurrence to determine whether it is in a smoldering state.

[0032] Design a network-based AI smoldering identification decision system, design visual-based feature-level and decision-level submodules; the fusion decision module of visual sensing and CO sensing uses DS evidence theory to make decisions, judge normal and abnormal smoldering, and obtain smoldering decisions. If abnormal, an early warning will be issued, and the server or mobile phone can get an early warning prompt to facilitate technical personnel to make decisions.

[0033] For the purpose of illustration, the foregoing description has been thoroughly understood by those skilled in the art of the embodiments. However, it will be apparent to those skilled in the art that specific details are not required in order to practice the embodiments. Those skilled in the art should understand that the present invention is not limited to the specific embodiments described above, and improvements and substitutions based on the present invention using the known techniques in the art fall within the scope of protection of the present invention and should be defined by the claims.

Claims

1. A method for monitoring tobacco smoldering by binocular vision sensing, characterized in that: It includes the following steps: First, use vision to conduct a smoldering experiment to obtain the data of smoke images of small targets of tobacco leaves and tobacco shreds in the warehouse, and the image data of the black edges of objects after smoldering; stack the tobacco strands or tobacco leaves into bundles, and install a network-controlled binocular vision device and bracket above the cigarette bales. In order to expand the field of view, the binocular vision device is adjustable in 270 degrees. The binocular vision device is equipped with a bionic cover and a device for automatically wiping the visual lens regularly to prevent smoke and ash from affecting the detection accuracy. It is designed to automatically adjust 90 degrees every hour; at the same time, a network-controlled CO sensor is placed between every two cigarette bales in the bundled stacking area, and the number of sensors is selected according to the number of cigarette bales; Second, we construct deep learning networks for the smoldering smoke and the edge of objects after smoldering, which are suitable for early small targets, and establish an improved YOLOV7 deep learning network model based on the attention mechanism, so as to be suitable for the recognition of small targets in smoldering smoke; 3. Use the model to detect smoke or suspected smoke areas and identify smoldering smoke; then use the improved YOLOV7 to identify the features of smoldering objects for rough classification and recognition, combine color image processing, use color space algorithm to calculate the edges of smoldering objects, and find the optimized smoldering recognition image algorithm; Fourth, through the optimized smoldering recognition image algorithm, the edges of smoldering smoke and smoldering objects are identified respectively; visual smoldering detection is designed to fuse image feature attributes, find out the internal relationship between image feature attributes and attributes, weight the smoldering feature attributes and edge features, and finally obtain the probability of smoldering; 5. Determine the three-dimensional location of smoldering smoke and smoldering objects; locate the smoldering space through binocular vision devices to obtain the smoldering location and area; make smoldering identification more accurate and reduce the probability of misidentification. On the basis of visual detection, add a CO sensor; 6. Obtain big data through real-time monitoring of CO concentration, and use the deep learning network YOLOV7 to identify smoldering data and the probability of its occurrence to determine whether it is in a smoldering state. If it is abnormal, an early warning will be issued. The server or mobile phone can receive early warning prompts to facilitate technical personnel to make decisions.

Images