A method and system for monitoring material status in a cabinet

Abstract

The present invention provides a method and system for monitoring the status of materials in a cabinet, which belongs to the technical field of tobacco material processing monitoring. The method for monitoring the status of materials in a cabinet comprises the following steps: obtaining a first video image in the cabinet and generating a frame-by-frame standard background picture set; judging whether there are foreign objects in the tobacco according to the frame-by-frame standard background picture set; if not, executing the next step; pre-processing the frame-by-frame standard background picture set and judging whether the tobacco cabinet is in a feeding state; if not, executing the next step; obtaining a second video image in the cabinet to obtain a frame-by-frame supplementary picture set; judging the operating state of the tobacco cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set. Using a computer system to identify foreign objects not only ensures accuracy but also does not cause fatigue to the human eyes. The identification and timely alarm of foreign objects in the production and processing of the tobacco cabinet greatly improve the quality of tobacco material processing and ensure the safety of workers.

Description

Technical Field

[0001] The present invention belongs to the technical field of tobacco material processing monitoring, and in particular relates to a method and system for monitoring the status of materials in a cabinet. Background Art

[0002] In recent years, with the advancement of science and technology, artificial intelligence technologies, represented by deep learning, have been increasingly applied in all aspects of scientific research and engineering. Autonomous driving, human-computer interaction, content-based image retrieval, and facial recognition, among other AI technologies, have all gained widespread application in these fields. Deep learning-based foreign object detection technology is widely used in various industrial scenarios. The tobacco industry has always valued automated control of production processes. Since the beginning of the new century, with the rapid development of the computer field, deep learning and artificial intelligence have been increasingly applied in the industrial field. Using deep learning methods and computer vision to improve production efficiency and product quality is a requirement and inevitable choice for modern enterprises.

[0003] At present, the tobacco material processing process of cigarette cabinets mainly relies on manual visual identification of foreign objects. This method not only has low accuracy, but also easily causes fatigue and affects the quality of cigarettes. In addition, the operating status of the cigarette cabinet cannot be known in a timely manner and often requires manual confirmation. However, once the cigarette cabinet is in a stagnant state and manual inspection is not discovered in time, it is easy to affect production efficiency. Summary of the Invention

[0004] The embodiment of the present invention provides a method and system for monitoring the status of materials in a cabinet, which aims to solve the problem that the tobacco material processing process of the existing cigarette cabinet mainly relies on manual visual recognition of foreign objects and the operating status cannot be known in a timely manner.

[0005] In view of the above problems, the technical solution proposed by the present invention is:

[0006] A method for monitoring material status in a cabinet, comprising:

[0007] Obtaining a first cabinet interior video image and generating a frame-by-frame standard background image set;

[0008] Determining whether there is foreign matter in the tobacco according to the frame-by-frame standard background image set; if not, executing the next step;

[0009] Pre-processing the frame-by-frame standard background image set and determining whether the smoke cabinet is in a charging state; if not, executing the next step;

[0010] Obtaining the video images in the second cabinet to obtain a frame-by-frame supplementary picture set;

[0011] The operating status of the smoke cabinet is determined according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set.

[0012] As a preferred technical solution of the present invention, the process of determining whether there is foreign matter in tobacco based on the frame-by-frame standard background image set specifically includes:

[0013] Assigning a training label to each picture in the frame-by-frame standard background picture set to obtain a first training sample;

[0014] Using the first training sample to train a foreign body judgment model;

[0015] Inputting the frame-by-frame standard background picture set into the foreign body judgment model to identify the position information and area information of the foreign body;

[0016] Determine whether there is any foreign matter in the smoke cabinet based on the position information and area information; if not, proceed to the next step.

[0017] As a preferred technical solution of the present invention, the pre-processing of the frame-by-frame standard background image set and judging whether the smoke cabinet is in a charging state specifically includes:

[0018] Performing edge detection on each picture in the frame-by-frame standard background picture set to obtain a pixel threshold;

[0019] Comparing the pixel threshold with the preset threshold to obtain a comparison result;

[0020] According to the comparison result, it is determined whether the smoke cabinet is in a charging state; if it is not in a charging state, the next step is executed.

[0021] As a preferred technical solution of the present invention, the determining of the operating status of the smoke cabinet based on the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set specifically includes:

[0022] Rotating each image in the frame-by-frame supplementary image set;

[0023] Applying the operating state classification to each picture in the frame-by-frame standard background picture set and the frame-by-frame supplementary image set to obtain a classified picture set;

[0024] Assigning a label to each picture in the classified picture set to obtain a second training sample;

[0025] Using the second training sample to perform training to obtain a state judgment model;

[0026] The frame-by-frame supplementary picture set and the frame-by-frame standard background picture set are input into the state judgment model to judge the operating state of the smoke cabinet.

[0027] As a preferred technical solution of the present invention, the operating state includes three states: empty, full and feeding.

[0028] On the other hand, an embodiment of the present invention further provides a system for monitoring the status of materials in a cabinet, comprising:

[0029] A first acquisition module, configured to acquire a first in-cabinet video image and generate a frame-by-frame standard background image set;

[0030] a first judging module, configured to judge whether there is foreign matter in the tobacco according to the frame-by-frame standard background image set; if not, executing the next step;

[0031] A second judgment module is used to pre-process the frame-by-frame standard background image set and judge whether the smoke cabinet is in a charging state; if not, executing the next step;

[0032] A second obtaining module, the second obtaining module is used to obtain a second video image in the cabinet to obtain a frame-by-frame supplementary picture set;

[0033] The third judgment module is used to judge the operating status of the smoke cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set.

[0034] As a preferred technical solution of the present invention, the first judgment module specifically includes:

[0035] a first configuration unit, configured to configure a training label for each picture in the frame-by-frame standard background picture set to obtain a first training sample;

[0036] A first training unit, configured to obtain a foreign body judgment model by training using a first training sample;

[0037] an identification unit, the identification unit being configured to input the frame-by-frame standard background picture set into the foreign body judgment model to identify position information and area information of foreign bodies;

[0038] The first judgment unit is used to judge whether there is a foreign object in the smoke cabinet according to the position information and the area information; if not, the next step is executed.

[0039] As a preferred technical solution of the present invention, the second judgment module specifically includes:

[0040] A detection unit, configured to perform edge detection on each picture in the frame-by-frame standard background picture set to obtain a pixel threshold;

[0041] A comparison unit, configured to compare a pixel threshold with a preset threshold to obtain a comparison result;

[0042] The second judgment unit is used to judge whether the smoke cabinet is in a charging state according to the comparison result; if it is not in a charging state, the next step is executed.

[0043] As a preferred technical solution of the present invention, the third judgment module specifically includes:

[0044] a rotation unit, configured to rotate each image in the frame-by-frame supplementary image set;

[0045] A classification unit is used to classify each picture in the frame-by-frame standard background picture set and the frame-by-frame supplementary image set according to the running state to obtain a classified picture set.

[0046] a second configuration unit, configured to configure a label for each image in the classified image set to obtain a second training sample;

[0047] A second training unit, the second training unit is used to train a state judgment model using a second training sample;

[0048] The third judgment unit is used to input the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set into the state judgment model to judge the operating state of the smoke cabinet.

[0049] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least: filling the gap in the efficiency of manual foreign body identification and smoke cabinet status judgment during the smoke material processing process, using a computer system to identify foreign bodies not only ensures accuracy but also does not cause fatigue to the human eye. The identification and timely alarm of foreign bodies in the production and processing of smoke cabinets greatly improves the quality of smoke material processing and ensures the safety of workers.

[0050] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a method for monitoring the status of materials in a cabinet disclosed in the present invention;

[0052] Figure 2 This is a flow chart of step S2 of a method for monitoring the status of materials in a cabinet disclosed in the present invention;

[0053] Figure 3 This is a flow chart of step S3 of a method for monitoring the status of materials in a cabinet disclosed in the present invention;

[0054] Figure 4 This is a flow chart of step S5 of a method for monitoring the status of materials in a cabinet disclosed in the present invention;

[0055] Figure 5 It is a structural diagram of a cabinet material status monitoring system disclosed in the present invention.

[0056] Description of reference numerals: 100, first obtaining module; 200, first judging module; 300, second judging module; 400, second obtaining module; 500, third judging module. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0058] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

[0059] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0060] Example 1

[0061] Refer to the attached Figures 1-2 As shown, the present invention provides a technical solution: a method for monitoring the status of materials in a cabinet, comprising the following steps:

[0062] S1, obtaining a first video image in the cabinet and generating a frame-by-frame standard background image set.

[0063] Specifically, the first video image inside the cabinet is captured by an industrial camera installed in the smoke cabinet.

[0064] S2, judging whether there are foreign objects in the tobacco according to the frame-by-frame standard background image set; if not, executing the next step.

[0065] Specifically, judging whether there is foreign matter in the tobacco according to the frame-by-frame standard background picture set specifically includes:

[0066] S21 , configuring a training label for each picture in the frame-by-frame standard background picture set to obtain a first training sample.

[0067] S22: Using the first training sample to perform training to obtain a foreign body judgment model.

[0068] It should be noted that the first training sample is trained using the Yolo target detection algorithm.

[0069] S23, inputting the frame-by-frame standard background picture set into the foreign object judgment model to identify the position information and area information of the foreign object.

[0070] S24, judging whether there is any foreign matter in the smoke cabinet according to the position information and area information; if not, executing the next step.

[0071] If yes, a foreign object alarm signal will be output.

[0072] S3, pre-processing the frame-by-frame standard background picture set and judging whether the smoke cabinet is in a state of adding fuel; if it is not in a state of adding fuel, executing the next step.

[0073] Specifically, the pre-processing of the frame-by-frame standard background image set and determining whether the smoke cabinet is in a charging state specifically includes:

[0074] S31, performing edge detection processing on each picture in the frame-by-frame standard background picture set to obtain a pixel threshold.

[0075] It should be noted that edge detection is performed using the Sobel edge detection algorithm.

[0076] S32: Compare the pixel threshold with a preset threshold to obtain a comparison result.

[0077] It should be noted that the comparison results are two results: the pixel threshold is greater than or equal to the preset threshold and the pixel threshold is less than or equal to the preset threshold.

[0078] S33, judging whether the smoke cabinet is in a state of adding fuel according to the comparison result; if it is not in a state of adding fuel, executing the next step.

[0079] If the pixel threshold is greater than or equal to the preset threshold, it is in the feeding state; if the pixel threshold is less than or equal to the preset threshold, it is not in the feeding state.

[0080] S4, obtaining a second video image inside the cabinet to obtain a frame-by-frame supplementary picture set.

[0081] S5, judging the operating status of the smoke cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set.

[0082] In a preferred embodiment of the present invention, the operating state includes three states: empty, full and feeding.

[0083] Specifically, judging the operating status of the smoke cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set specifically includes:

[0084] S51 , rotating each image in the frame-by-frame supplementary image set.

[0085] It is worth noting that the rotation angle of each image ranges from 0 to 5°.

[0086] S52, each picture in the frame-by-frame standard background picture set and the frame-by-frame supplementary image set is classified according to the operating status to obtain a classified picture set.

[0087] S53: Assign a label to each picture in the classified picture set to obtain a second training sample.

[0088] S54: Use the second training sample to perform training to obtain a state judgment model.

[0089] It should be noted that the second training sample is trained through the VGG network.

[0090] S55: Input the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set into the state judgment model to judge the operating state of the smoke cabinet.

[0091] Example 2

[0092] The embodiment of the present invention also discloses a system for monitoring the status of materials in a cabinet. Figure 3 Shown, including:

[0093] The first obtaining module 100 is used to obtain a first in-cabinet video image and generate a frame-by-frame standard background picture set.

[0094] The first judgment module 200 is used to judge whether there is foreign matter in the tobacco according to the frame-by-frame standard background image set; if not, the next step is executed.

[0095] Specifically, the first judgment module 200 specifically includes:

[0096] a first configuration unit, configured to configure a training label for each picture in the frame-by-frame standard background picture set to obtain a first training sample;

[0097] A first training unit, configured to obtain a foreign body judgment model by training using a first training sample;

[0098] an identification unit, the identification unit being configured to input the frame-by-frame standard background picture set into the foreign body judgment model to identify position information and area information of foreign bodies;

[0099] The first judgment unit is used to judge whether there is a foreign object in the smoke cabinet according to the position information and the area information; if not, the next step is executed.

[0100] The second judgment module 300 is used to pre-process the frame-by-frame standard background image set and judge whether the smoke cabinet is in the charging state; if it is not in the charging state, the next step is executed.

[0101] Specifically, the second judgment module 300 includes:

[0102] A detection unit, configured to perform edge detection on each picture in the frame-by-frame standard background picture set to obtain a pixel threshold;

[0103] A comparison unit, configured to compare a pixel threshold with a preset threshold to obtain a comparison result;

[0104] The second judgment unit is used to judge whether the smoke cabinet is in a charging state according to the comparison result; if it is not in a charging state, the next step is executed.

[0105] A second obtaining module 400 is configured to obtain a second video image in the cabinet to obtain a frame-by-frame supplementary picture set;

[0106] The third judgment module 500 is used to judge the operating status of the smoke cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set.

[0107] Specifically, the third judgment module 500 includes:

[0108] a rotation unit, configured to rotate each image in the frame-by-frame supplementary image set;

[0109] A classification unit is used to classify each picture in the frame-by-frame standard background picture set and the frame-by-frame supplementary image set according to the running state to obtain a classified picture set.

[0110] a second configuration unit, configured to configure a label for each image in the classified image set to obtain a second training sample;

[0111] A second training unit, the second training unit is used to train a state judgment model using a second training sample;

[0112] The third judgment unit is used to input the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set into the state judgment model to judge the operating state of the smoke cabinet.

[0113] In the foregoing detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. On the contrary, as reflected in the appended claims, the invention comprises less than all the features of any individual disclosed embodiment. The appended claims are therefore hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0114] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein may be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described around their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. A skilled person may implement the described functions in an adaptable manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of this disclosure.

[0115] The steps of the methods or algorithms described in conjunction with the embodiments herein may be directly embodied as hardware, software modules executed by a processor, or a combination thereof. The software module may be located in a RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and storage medium may be located in an ASIC. The ASIC may be located in a user terminal. Of course, the processor and storage medium may also be present in a user terminal as discrete components.

[0116] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented within the processor or external to the processor. In the latter case, it is communicatively coupled to the processor via various means, which are well known in the art.

[0117] The foregoing description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purposes of describing the above embodiments, but one of ordinary skill in the art will recognize that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to encompass all such changes, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent the term "comprising" is used in the specification or claims, the term is intended to be encompassed in a manner similar to the term "including," as explained in terms of "including," used as a transitional word in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or."

Claims

1. A method for monitoring the status of materials in a cabinet, characterized in that: include: Obtaining a first cabinet interior video image and generating a frame-by-frame standard background image set; Judging whether there is foreign matter in the tobacco according to the frame-by-frame standard background image set specifically includes: Assigning a training label to each picture in the frame-by-frame standard background picture set to obtain a first training sample, wherein the first training sample is trained using a Yolo target detection algorithm; Using the first training sample to train a foreign body judgment model; Inputting the frame-by-frame standard background picture set into the foreign body judgment model to identify the position information and area information of the foreign body; Determine whether there is any foreign matter in the smoke cabinet according to the position information and area information; if not, proceed to the next step; Preprocessing the frame-by-frame standard background image set and determining whether the smoke cabinet is in a charging state specifically includes: Performing edge detection on each picture in the frame-by-frame standard background picture set to obtain a pixel threshold, wherein the edge detection is performed using a Sobel edge detection algorithm; Comparing the pixel threshold with the preset threshold to obtain a comparison result; According to the comparison result, it is determined whether the smoke cabinet is in a charging state; if it is not in a charging state, the next step is executed; Obtaining the video images in the second cabinet to obtain a frame-by-frame supplementary picture set; Determining the operating status of the smoke cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set specifically includes: Rotating each image in the frame-by-frame supplementary image set; Applying the operating state classification to each picture in the frame-by-frame standard background picture set and the frame-by-frame supplementary image set to obtain a classified picture set; Configuring a label for each image in the classified image set to obtain a second training sample, and training the second training sample through a VGG network; Using the second training sample to perform training to obtain a state judgment model; The frame-by-frame supplementary picture set and the frame-by-frame standard background picture set are input into the state judgment model to judge the operating state of the smoke cabinet.

2. The method for monitoring the status of materials in a cabinet according to claim 1, characterized in that: The operating status includes three states: empty, full and feeding.

3. A system for monitoring the status of materials in a cabinet, applied to the method for monitoring the status of materials in a cabinet according to any one of claims 1 to 2, comprising: A first acquisition module, configured to acquire a first in-cabinet video image and generate a frame-by-frame standard background image set; a first judging module, configured to judge whether there is foreign matter in the tobacco according to the frame-by-frame standard background image set; if not, executing the next step; A second judgment module is used to pre-process the frame-by-frame standard background image set and judge whether the smoke cabinet is in a charging state; if not, executing the next step; A second obtaining module, the second obtaining module is used to obtain a second video image in the cabinet to obtain a frame-by-frame supplementary picture set; The third judgment module is used to judge the operating status of the smoke cabinet according to the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set.

4. The system for monitoring the status of materials in a cabinet according to claim 3, characterized in that: The first judgment module specifically includes: a first configuration unit, configured to configure a training label for each picture in the frame-by-frame standard background picture set to obtain a first training sample; A first training unit, configured to obtain a foreign body judgment model by training using a first training sample; an identification unit, the identification unit being configured to input the frame-by-frame standard background picture set into the foreign body judgment model to identify position information and area information of foreign bodies; The first judgment unit is used to judge whether there is a foreign object in the smoke cabinet according to the position information and the area information; if not, the next step is executed.

5. The system for monitoring the status of materials in a cabinet according to claim 3, characterized in that: The second judgment module specifically includes: A detection unit, configured to perform edge detection on each picture in the frame-by-frame standard background picture set to obtain a pixel threshold; A comparison unit, configured to compare a pixel threshold with a preset threshold to obtain a comparison result; The second judgment unit is used to judge whether the smoke cabinet is in a charging state according to the comparison result; if it is not in a charging state, the next step is executed.

6. The system for monitoring the status of materials in a cabinet according to claim 3, characterized in that: The third judgment module specifically includes: a rotation unit, configured to rotate each image in the frame-by-frame supplementary image set; A classification unit is used to classify each picture in the frame-by-frame standard background picture set and the frame-by-frame supplementary image set according to the running state to obtain a classified picture set. a second configuration unit, configured to configure a label for each image in the classified image set to obtain a second training sample; A second training unit, the second training unit is used to train a state judgment model using a second training sample; The third judgment unit is used to input the frame-by-frame supplementary picture set and the frame-by-frame standard background picture set into the state judgment model to judge the operating state of the smoke cabinet.

Images