Tobacco raw material packaging material detection method, device, equipment and medium

Abstract

The invention discloses a tobacco raw material packaging material detection method, device and equipment and a medium. The method comprises the following steps: reconstructing a lightweight target detection model by using probability distribution distillation to obtain a package detection pre-training model; acquiring a historical tobacco packaging material detection image acquired by the camera in the view of the waste collection area, and determining a tobacco packaging material detection model based on the historical tobacco packaging material detection image and the packaging material detection pre-training model; when it is determined that the position of the tobacco package is abnormal according to the current waste collection area view image and the tobacco package detection model, early warning is conducted. According to the technical scheme, the detection precision of the tobacco raw material packaging material can be greatly improved, so that the strong dependence on manual inspection is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, equipment and medium for detecting tobacco raw material packaging. Background Art

[0002] Unpacking tobacco boxes is the first step in silk production, and its level of automation impacts the overall automation level of the silk production line. Currently, robots are primarily used for automated unpacking of tobacco boxes, resolving the labor-intensive, poor working environment, and direct exposure of workers to tobacco dust associated with manual unpacking and semi-automated unpacking machines.

[0003] Due to the different light intensities at different times on site and the different packaging materials of cigarette boxes from different origins, the existing system may miss or misidentify the packaging. There are situations where the packaging is not taken out and flows into the next process, or it is actually taken out but cannot be identified, causing the production line to stop, requiring manual quality control. Summary of the Invention

[0004] The present invention provides a method, device, equipment and medium for detecting tobacco raw material packaging, which can solve the problem that the detection effect of existing tobacco raw material packaging is poor and has a strong dependence on manual inspection.

[0005] According to one aspect of the present invention, a method for detecting tobacco raw material packaging is provided, comprising:

[0006] The lightweight object detection model is reconstructed using probability distribution distillation to obtain a pre-trained model for packaging detection.

[0007] Obtain historical tobacco packaging detection images captured by the camera in the field of view of the waste collection area, and determine a tobacco packaging detection model based on the historical tobacco packaging detection images and a pre-trained packaging detection model;

[0008] When it is determined that the tobacco packaging is in an abnormal position based on the current waste collection area field of view image and the tobacco packaging detection model, an early warning is issued.

[0009] According to another aspect of the present invention, there is provided a device for detecting tobacco raw material packages, comprising:

[0010] The model distillation module is used to reconstruct the lightweight object detection model using probability distribution distillation to obtain a pre-trained model for packaging detection;

[0011] a tobacco packaging detection model determination module, configured to obtain historical tobacco packaging detection images captured by the camera in the field of view of the waste collection area, and determine a tobacco packaging detection model based on the historical tobacco packaging detection images and a pre-trained packaging detection model;

[0012] The early warning module is used to issue an early warning when it is determined that the position of the tobacco packaging is abnormal based on the current waste collection area field of view image and the tobacco packaging detection model.

[0013] According to another aspect of the present invention, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the tobacco raw material packaging detection method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tobacco raw material packaging detection method according to any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention reconstructs the lightweight target detection model using probability distribution distillation to obtain a packaging detection pre-training model, thereby obtaining historical tobacco packaging detection images collected by the camera in the waste collection area field of view, and determining the tobacco packaging detection model based on the historical tobacco packaging detection images and the packaging detection pre-training model. Then, when it is determined that the tobacco packaging has an abnormal position based on the current waste collection area field of view image and the tobacco packaging detection model, an early warning is issued. In this solution, the accuracy of the lightweight target detection model is improved through probability distribution distillation, so that the packaging detection pre-training model takes into account both computing speed and accuracy. Then, after the packaging detection pre-training model is trained, the position of the tobacco packaging in the current waste collection area field of view image is accurately identified, which greatly reduces the missed and incorrect identification of tobacco packaging, effectively reduces the dependence on manpower, and solves the problem that the existing tobacco raw material packaging detection effect is poor and has a strong dependence on manual inspection. It can greatly improve the detection accuracy of tobacco raw material packaging and reduce the strong dependence on manual inspection.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a method for detecting tobacco raw material packaging provided in Example 1 of the present invention;

[0022] Figure 2 A flow chart of a method for detecting tobacco raw material packaging provided in Example 2 of the present invention;

[0023] Figure 3 A schematic diagram of a camera installation position provided in Example 3 of the present invention;

[0024] Figure 4 A schematic structural diagram of a tobacco raw material packaging detection device provided in a fourth embodiment of the present invention;

[0025] Figure 5 A schematic structural diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Example 1

[0029] Figure 1This is a flow chart of a method for detecting tobacco raw material packaging provided in the first embodiment of the present invention. This embodiment is applicable to the case of high-precision identification of tobacco raw material packaging. The method can be executed by a tobacco raw material packaging detection device. The tobacco raw material packaging detection device can be implemented in the form of hardware and / or software. The tobacco raw material packaging detection device can be configured in an electronic device. The electronic device can be a computer or a server, etc. Figure 1 As shown, the method includes:

[0030] Step 110: Use probability distribution distillation to reconstruct the lightweight object detection model to obtain a pre-trained model for packaging detection.

[0031] The lightweight object detection model can be a lightweight object detection model. The pre-trained packaging detection model can be a lightweight object detection model after distillation. Optionally, the lightweight object detection model can be an existing model or a model obtained by performing model lightweight processing on an existing object detection model.

[0032] In an embodiment of the present invention, a lightweight target detection model may be determined first, and then the lightweight target detection model may be reconstructed based on probability distribution distillation, and the model after the distillation process may be used as a pre-training model for packaging detection.

[0033] Step 120: Obtain historical tobacco package detection images captured by the camera in the field of view of the waste collection area, and determine a tobacco package detection model based on the historical tobacco package detection images and a package detection pre-trained model.

[0034] The waste collection area field of view may be the field of view of a camera monitoring the tobacco raw material packaging collection process. The historical tobacco packaging detection images may be images of the waste collection area field of view over a period of time. The tobacco packaging detection model may be a model obtained by training a pre-trained packaging detection model using historical tobacco packaging detection images.

[0035] In an embodiment of the present invention, images under the field of view of the waste collection area can be collected by a camera, and the images collected by the camera over a period of time in the past can be used as historical tobacco packaging detection images. The packaging detection pre-training model can then be trained based on the historical tobacco packaging detection images to obtain a tobacco packaging detection model.

[0036] Step 130: When it is determined that the tobacco package is in an abnormal position based on the current waste collection area field of view image and the tobacco package detection model, an early warning is issued.

[0037] In an embodiment of the present invention, the field of view image of the current waste collection area can be cleaned, and the cleaned image can be input into the tobacco packaging detection model to identify the position of the tobacco raw material packaging through the tobacco packaging detection model, and when the tobacco packaging does not fall completely into the waste collection area, it is determined that there is an abnormal position of the tobacco packaging and an early warning is issued.

[0038] The technical solution of the embodiment of the present invention reconstructs the lightweight target detection model using probability distribution distillation to obtain a packaging detection pre-training model, thereby obtaining historical tobacco packaging detection images collected by the camera in the waste collection area field of view, and determining the tobacco packaging detection model based on the historical tobacco packaging detection images and the packaging detection pre-training model. Then, when it is determined that the tobacco packaging has an abnormal position based on the current waste collection area field of view image and the tobacco packaging detection model, an early warning is issued. In this solution, the accuracy of the lightweight target detection model is improved through probability distribution distillation, so that the packaging detection pre-training model takes into account both computing speed and accuracy. Then, after the packaging detection pre-training model is trained, the position of the tobacco packaging in the current waste collection area field of view image is accurately identified, which greatly reduces the missed and incorrect identification of tobacco packaging, effectively reduces the dependence on manpower, and solves the problem that the existing tobacco raw material packaging detection effect is poor and has a strong dependence on manual inspection. It can greatly improve the detection accuracy of tobacco raw material packaging and reduce the strong dependence on manual inspection.

[0039] Example 2

[0040] Figure 2 The flowchart of a method for detecting tobacco raw material packaging provided in the second embodiment of the present invention is a refinement of the above embodiment. In this embodiment, the tobacco packaging detection model is specifically refined based on historical tobacco packaging detection images and packaging detection pre-training models. Specifically, it may include: cleaning historical tobacco packaging detection images, and marking target detection objects in historical tobacco packaging detection images to obtain tobacco packaging detection sample sets; determining tobacco packaging detection models based on tobacco packaging detection sample sets and packaging detection pre-training models. Figure 2 As shown, the method includes:

[0041] Step 210: Use probability distribution distillation to reconstruct the lightweight object detection model to obtain a pre-trained model for packaging detection.

[0042] Step 220: Obtain historical tobacco packaging detection images collected by the camera in the field of view of the waste collection area.

[0043] Step 230 : clean the historical tobacco package detection image, and mark the target detection objects in the cleaned historical tobacco package detection image to obtain a tobacco package detection sample set.

[0044] The target detection object may be a key object to be identified in a historical tobacco packaging detection image. The target detection object may include different types of tobacco raw material packaging, a robotic arm, and a waste collection conveyor belt. The tobacco packaging detection sample set may be a sample set obtained by annotating the historical tobacco packaging detection images.

[0045] In an embodiment of the present invention, the historical tobacco packaging detection image can be cleaned to remove images with poor image quality, duplicates, and invalid images in the historical tobacco packaging detection image, and the target detection objects in the cleaned historical tobacco packaging detection image can be marked to obtain the final tobacco packaging detection sample set.

[0046] Step 240: Determine a tobacco package detection model based on the tobacco package detection sample set and the package detection pre-training model.

[0047] In an embodiment of the present invention, a package detection pre-training model or an optimized package detection pre-training model may be trained based on a tobacco package detection sample set to obtain a tobacco package detection model.

[0048] In an optional embodiment of the present invention, determining a tobacco packaging detection model based on a tobacco packaging detection sample set and a packaging detection pre-trained model may include: pruning the packaging detection pre-trained model according to a preset model accuracy condition to obtain a target packaging detection pre-trained model; training the target packaging detection pre-trained model based on the tobacco packaging detection sample set to obtain a tobacco packaging detection model.

[0049] The preset model accuracy condition may be a preset accuracy condition that needs to be met by the model optimization. The target packaging detection pre-trained model may be a model that meets the preset model accuracy condition after pruning the packaging detection pre-trained model.

[0050] In an embodiment of the present invention, the redundant layers of the packaging detection pre-training model can be pruned, and the pruned packaging detection pre-training model that meets the preset model accuracy conditions can be used as the target packaging detection pre-training model. The target packaging detection pre-training model can then be trained using the tobacco packaging detection sample set to obtain a tobacco packaging detection model.

[0051] Step 250: When it is determined that the tobacco package is in an abnormal position based on the current waste collection area field of view image and the tobacco package detection model, an early warning is issued.

[0052] In an optional embodiment of the present invention, when it is determined that the tobacco packaging is in an abnormal position based on the current waste collection area field of view image and the tobacco packaging detection model, an early warning is issued, which can include: inputting the current waste collection area field of view image into the tobacco packaging detection model to obtain tobacco packaging identification data, waste collection area identification data and robotic arm area identification data; when it is determined that the tobacco packaging is in an abnormal position based on the tobacco packaging identification data, waste collection area identification data and robotic arm area identification data, an early warning is issued.

[0053] The tobacco packaging identification data may be tobacco packaging-related data identified by the tobacco packaging detection model from the current waste collection area field of view image. The tobacco packaging identification data may include, but is not limited to, the category of the tobacco packaging and the regional location of the tobacco packaging in the field of view image of the previous waste collection area. The waste collection area identification data may be used to characterize the regional location of the waste collection conveyor belt in the field of view image of the current waste collection area. The robotic arm area identification data may be used to characterize the regional location of the robotic arm that unpacks tobacco raw materials in the field of view image of the current waste collection area.

[0054] In an embodiment of the present invention, the current waste collection area field of view image can be input into the tobacco packaging detection model to identify the key identification objects of the current waste collection area field of view image based on the tobacco packaging detection model, and obtain tobacco packaging identification data, waste collection area identification data and robotic arm area identification data. Then, based on the tobacco packaging identification data, waste collection area identification data and robotic arm area identification data, it is determined whether the tobacco packaging is completely within the waste collection conveyor belt, so as to issue an early warning when an abnormal position of the tobacco packaging is detected.

[0055] In an optional embodiment of the present invention, when it is determined that the tobacco package has an abnormal position based on the tobacco package identification data, the waste collection area identification data and the robotic arm area identification data, an early warning is issued, which may include: determining a first identification area based on the tobacco package identification data, and determining a second identification area matching the first identification area based on the waste collection area identification data; determining a third identification area based on the robotic arm area identification data; when the first identification area partially overlaps with the second identification area, determining that the tobacco package has an abnormal position, and issuing a first type of early warning; when the first identification area overlaps with the third identification area, determining that the tobacco package has an abnormal position, and issuing a second type of early warning.

[0056] Among them, the first identification area can be the area of ​​the tobacco packaging in the current waste collection area field of view image after the tobacco raw material box is unpacked. Exemplarily, the tobacco raw material packaging can include cardboard, kraft paper and packaging bags, so the first identification area corresponding to each tobacco raw material box can include a cardboard identification sub-area, a kraft paper identification sub-area and a packaging bag identification sub-area. The second identification area can be the waste collection conveyor belt carrying the tobacco raw material packaging corresponding to the first identification area, and the area in the current waste collection area field of view image. The third identification area can be the area of ​​the robotic arm under the waste collection area field of view, and the area in the current waste collection area field of view image. The first type of warning and the second type of warning can be different warning signals, which are used to provide differentiated warnings for different abnormal positions of the tobacco raw material packaging.

[0057] In an embodiment of the present invention, tobacco packaging identification data can be parsed to determine the first identification area corresponding to each tobacco raw material box after unpacking. Then, based on the waste collection area identification data, a second identification area that matches the first identification area corresponding to each tobacco raw material box is determined. A third identification area is determined from the robotic arm area identification data. Then, the first identification area corresponding to each tobacco raw material box after unpacking is overlapped with the corresponding second identification area for identification. When the first identification area partially overlaps with the corresponding second identification area, it is determined that the tobacco packaging has a positional abnormality, and a first type of warning is issued. When the first identification area overlaps with the third identification area, it is determined that the tobacco packaging has a positional abnormality, and a second type of warning is issued.

[0058] In an optional embodiment of the present invention, after inputting the current waste collection area field of view image into the tobacco packaging detection model to obtain tobacco packaging identification data, waste collection area identification data and robotic arm area identification data, it can also include: counting the total number of tobacco packaging identifications based on the tobacco packaging identification data; obtaining the current number of unpacking operations, and generating tobacco packaging collection comparison data based on the current number of unpacking operations and the total number of tobacco packaging identifications.

[0059] The total number of tobacco packaging identified may be the total number of tobacco packaging identified from the current waste collection area field of view image, and the current number of unpacking operations may be the number of unpacking operations of tobacco raw material boxes corresponding to the current waste collection area field of view image.

[0060] In an embodiment of the present invention, based on the tobacco packaging identification data, the number of tobacco packaging in the current waste collection area field of view can be counted to obtain the total number of tobacco packaging identifications, and then the current number of unpacking operations can be obtained. Based on the current number of unpacking operations and the number of tobacco packaging generated each time unpacking (determined by the packaging process of the manufacturer of the tobacco raw material cigarette box), the total number of various types of tobacco packaging that need to be identified is determined, and the total number of various types of tobacco packaging that need to be identified and the total number of tobacco packaging identifications are used as tobacco packaging collection comparison data.

[0061] In an optional embodiment of the present invention, after generating tobacco packaging collection and comparison data based on the current number of unpacking operations and the total number of tobacco packaging identifications, it may also include: parsing the tobacco packaging collection and comparison data, determining missing packaging-related data, and displaying the missing packaging-related data on a graphical display interface.

[0062] The missing package-related data may be used to describe data related to tobacco raw material packages that did not normally enter the waste collection conveyor belt in the current waste collection area field of view image. The missing package-related data may include the missing package type and the missing package quantity.

[0063] Specifically, while counting the total number of tobacco packaging identifications, the identification numbers of different types of tobacco packaging can also be classified and summarized in a targeted manner to obtain the identification numbers of different types of tobacco packaging. If the total number of various types of tobacco packaging in the tobacco packaging collection and comparison data is different from the identification number, it can be determined that some tobacco packaging has abnormal positions, and thus missing packaging-related data is generated based on the tobacco packaging collection and comparison data and the identification number of different types of tobacco packaging, and the missing packaging-related data is displayed on a graphical display interface.

[0064] The technical solution of the embodiment of the present invention reconstructs the lightweight target detection model by using probability distribution distillation to obtain a pre-trained model for packaging detection, and then obtains historical tobacco packaging detection images captured by the camera in the field of view of the waste collection area, thereby cleaning the historical tobacco packaging detection images and annotating the target detection objects in the cleaned historical tobacco packaging detection images to obtain a tobacco packaging detection sample set. In this solution, the accuracy of the lightweight target detection model is improved through probability distribution distillation, so that the packaging detection pre-trained model takes into account both computing speed and accuracy. After the packaging detection pre-trained model is trained, the position of tobacco packaging in the current field of view image of the waste collection area is accurately identified, which greatly reduces the missed and wrong identification of tobacco packaging, effectively reduces the dependence on manpower, and solves the problem that the existing tobacco raw material packaging detection effect is poor and has a strong dependence on manual inspection. It can greatly improve the detection accuracy of tobacco raw material packaging and reduce the strong dependence on manual inspection.

[0065] Example 3

[0066] The third embodiment of the present invention provides an optional embodiment of a method for detecting tobacco raw material packaging, and its specific implementation can be found in the following embodiments. Among them, the technical terms that are the same as or corresponding to the above embodiments are not repeated here.

[0067] Install a high-definition surveillance camera above the waste collection conveyor belt to collect tobacco packaging inspection images. Ensure that the camera can capture the entire conveyor belt and the process of the robotic arm entering the conveyor belt. The camera installation location can be found in Figure 3 Using an installed camera, tobacco packaging inspection images are captured and pre-processed, including noise reduction. These images are then fed into a pre-trained packaging detection model with guaranteed accuracy and real-time performance. This model then outputs object location and category information, supplemented by an alarm control system and a graphical display interface. When waste materials such as cardboard and plastic bags are detected in unusual locations, the alarm control system promptly issues an early warning. This information, based on the category and location of the robotic arm, cardboard, plastic bag, and conveyor belt, infers whether the robotic arm has successfully grasped the object and issues an alarm, allowing production staff to promptly address the unusual situation. The alarm control system is connected to the silk production centralized control system and can be an audible and visual alarm. The alarm control logic configured in the alarm control system includes an alarm signal and an alarm duration. Staff can clear alarms using a graphical display interface, which displays alarm information (such as missing cardboard and / or missing packaging bags) for easy confirmation. It also displays cumulative unpacking information, which can be cleared manually or automatically at a fixed time each day.

[0068] For example, the alarm control system's warning logic can use the robot's grasping success rate to infer whether there are any remaining packaging materials on the cigarette pack. If the cardboard, kraft paper, and packaging bag are all detected, then all packaging materials have been removed and the cigarette pack can pass normally. Conversely, if only one type of packaging material is detected, or none of them are detected, then there are still some packaging materials on the cigarette pack. The alarm control system will issue a warning message so that production staff can promptly address the abnormal situation.

[0069] Specifically, cameras capture images of historical waste collection areas, which are then cleaned to eliminate poor-quality, duplicate, and irrelevant data to ensure dataset quality. The data is then labeled (for tobacco packaging, tobacco leaf bags and cardboard are labeled, as well as robotic arms and conveyor belts). The pre-trained model for packaging detection can be the ShuffYOLOX algorithm, a lightweight YOLOX algorithm.

[0070] For example, the lightweight structure ShuffDet (Shuff Single Shot MultiBoxDetector, lightweight forward transfer) is used to reconstruct the target detection model, resulting in a lightweight target detection model. This model is then reconstructed using probability distribution distillation to improve model accuracy and meet implementation standards. The lightweight model is deployed on edge computing software and, after inference acceleration, achieves an inference speed of 31 FPS, bringing the model detection speed up to industrial standards.

[0071] Example 4

[0072] Figure 4 This is a schematic diagram of a detection device for tobacco raw material packaging provided by the fourth embodiment of the present invention. Figure 4 As shown, the device includes:

[0073] A model distillation module 310 is used to reconstruct the lightweight object detection model using probability distribution distillation to obtain a pre-trained model for packaging detection;

[0074] a tobacco package detection model determination module 320 for obtaining historical tobacco package detection images captured by the camera in the field of view of the waste collection area and determining a tobacco package detection model based on the historical tobacco package detection images and a pre-trained package detection model;

[0075] The early warning module 330 is used to issue an early warning when it is determined that there is an abnormal position of the tobacco package based on the current waste collection area field image and the tobacco package detection model.

[0076] The technical solution of the embodiment of the present invention reconstructs the lightweight target detection model using probability distribution distillation to obtain a packaging detection pre-training model, thereby obtaining historical tobacco packaging detection images collected by the camera in the waste collection area field of view, and determining the tobacco packaging detection model based on the historical tobacco packaging detection images and the packaging detection pre-training model. Then, when it is determined that the tobacco packaging has an abnormal position based on the current waste collection area field of view image and the tobacco packaging detection model, an early warning is issued. In this solution, the accuracy of the lightweight target detection model is improved through probability distribution distillation, so that the packaging detection pre-training model takes into account both computing speed and accuracy. Then, after the packaging detection pre-training model is trained, the position of the tobacco packaging in the current waste collection area field of view image is accurately identified, which greatly reduces the missed and incorrect identification of tobacco packaging, effectively reduces the dependence on manpower, and solves the problem that the existing tobacco raw material packaging detection effect is poor and has a strong dependence on manual inspection. It can greatly improve the detection accuracy of tobacco raw material packaging and reduce the strong dependence on manual inspection.

[0077] Optionally, the tobacco packaging detection model determination module 320 includes a sample labeling unit and a tobacco packaging detection model determination unit. The sample labeling unit is configured to clean the historical tobacco packaging detection image and label the target detection objects in the cleaned historical tobacco packaging detection image to obtain a tobacco packaging detection sample set. The tobacco packaging detection model determination unit is configured to determine a tobacco packaging detection model based on the tobacco packaging detection sample set and the pre-trained packaging detection model.

[0078] Optionally, the tobacco packaging detection model determination unit is specifically used to prune the packaging detection pre-trained model according to a preset model accuracy condition to obtain a target packaging detection pre-trained model; and train the target packaging detection pre-trained model based on the tobacco packaging detection sample set to obtain the tobacco packaging detection model.

[0079] Optionally, the early warning module 330 is specifically used to input the current waste collection area field of view image into the tobacco packaging detection model to obtain tobacco packaging identification data, waste collection area identification data and robotic arm area identification data; when it is determined that there is an abnormal position of the tobacco packaging based on the tobacco packaging identification data, the waste collection area identification data and the robotic arm area identification data, an early warning is issued.

[0080] Optionally, the early warning module 330 is specifically used to determine a first identification area based on the tobacco package identification data, and determine a second identification area matching the first identification area based on the waste collection area identification data; determine a third identification area based on the robotic arm area identification data; when the first identification area partially overlaps with the second identification area, determine that there is a positional abnormality in the tobacco package, and issue a first type of early warning; when the first identification area overlaps with the third identification area, determine that there is a positional abnormality in the tobacco package, and issue a second type of early warning.

[0081] Optionally, the tobacco raw material packaging detection device further includes a data aggregation unit configured to count the total number of tobacco packaging identifications based on the tobacco packaging identification data; obtain the current number of unpacking operations, and generate tobacco packaging collection and comparison data based on the current number of unpacking operations and the total number of tobacco packaging identifications.

[0082] Optionally, the detection device for tobacco raw material packaging also includes a data display module for parsing the tobacco packaging collection and comparison data, determining missing packaging-related data, and displaying the missing packaging-related data on a graphical display interface; wherein the missing packaging-related data includes the missing packaging type and the missing packaging quantity.

[0083] The tobacco raw material packaging detection device provided in the embodiment of the present invention can execute the tobacco raw material packaging detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0084] Example 5

[0085] Figure 5 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present inventions described and / or claimed herein.

[0086] like Figure 5 As shown, electronic device 10 includes at least one processor 11 and memory, such as ROM 12 and RAM 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor, and processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for programming controller 10 operations. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An I / O interface 15 is also connected to bus 14. ROM 12 is a read-only memory, RAM 13 is a random access memory, and I / O interface 15 is an input / output interface.

[0087] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0088] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for detecting tobacco raw material packaging.

[0089] In some embodiments, the detection method of tobacco raw material package can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the detection method of tobacco raw material package described above can be executed. Alternatively, in other embodiments, processor 11 can be configured to perform the detection method of tobacco raw material package by any other appropriate means (for example, by means of firmware).

[0090] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0092] In the context of the present invention, computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0095] A computing system may include clients and servers. The clients and servers are generally remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS servers.

[0096] The present application also discloses a computer program product comprising a computer program that, when executed by a processor, implements the tobacco raw material packaging detection method provided in any of the embodiments of the present application. This program product and the tobacco raw material packaging detection method disclosed in each embodiment of the present application share the same inventive concept and are therefore not further described here.

[0097] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0098] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for detecting tobacco raw material packaging, characterized in that: include: The lightweight object detection model is reconstructed using probability distribution distillation to obtain a pre-trained model for packaging detection. Obtaining historical tobacco package detection images captured by a camera in a field of view of a waste collection area, and determining a tobacco package detection model based on the historical tobacco package detection images and the package detection pre-trained model; When it is determined that there is an abnormal position of the tobacco package based on the current waste collection area field of view image and the tobacco package detection model, an early warning is issued.

2. The method according to claim 1, characterized in that Determining a tobacco package detection model based on the historical tobacco package detection image and the package detection pre-trained model includes: Cleaning the historical tobacco packaging detection image, and marking the target detection objects in the cleaned historical tobacco packaging detection image to obtain a tobacco packaging detection sample set; A tobacco package detection model is determined based on the tobacco package detection sample set and the package detection pre-training model.

3. The method according to claim 2, characterized in that Determining a tobacco package detection model based on the tobacco package detection sample set and the package detection pre-trained model includes: Pruning the package detection pre-trained model according to a preset model accuracy condition to obtain a target package detection pre-trained model; Based on the tobacco package detection sample set, the target package detection pre-training model is trained to obtain the tobacco package detection model.

4. The method according to claim 1, wherein When it is determined that the tobacco package is in an abnormal position according to the current waste collection area field of view image and the tobacco package detection model, an early warning is issued, including: Inputting the current waste collection area field image into the tobacco package detection model to obtain tobacco package identification data, waste collection area identification data, and robotic arm area identification data; When it is determined that the tobacco package has an abnormal position based on the tobacco package identification data, the waste collection area identification data and the robot arm area identification data, an early warning is issued.

5. The method according to claim 4, characterized in that When it is determined that the tobacco package is in an abnormal position according to the tobacco package identification data, the waste collection area identification data, and the robotic arm area identification data, an early warning is issued, including: Determining a first identification area based on the tobacco package identification data, and determining a second identification area matching the first identification area based on the waste collection area identification data; determining a third identification area according to the robotic arm area identification data; When the first recognition area and the second recognition area partially overlap, determining that there is an abnormal position of the tobacco package and issuing a first type of warning; When the first recognition area and the third recognition area overlap, it is determined that there is a positional abnormality in the tobacco package, and a second type of warning is issued.

6. The method according to claim 4, characterized in that After inputting the current waste collection area field image into the tobacco package detection model to obtain tobacco package identification data, waste collection area identification data, and robotic arm area identification data, the method further includes: Counting the total number of tobacco package identifications based on the tobacco package identification data; The current number of unpacking operations is obtained, and tobacco package collection and comparison data is generated according to the current number of unpacking operations and the total number of tobacco package identifications.

7. The method according to claim 6, characterized in that After generating tobacco package collection and comparison data according to the current number of unpacking operations and the total number of tobacco package identifications, the method further includes: parsing the tobacco packaging collection and comparison data, determining missing package-related data, and displaying the missing package-related data on a graphical display interface; The missing package-related data includes the missing package type and the missing package quantity.

8. A device for detecting tobacco raw material packaging, characterized in that: include: The model distillation module is used to reconstruct the lightweight object detection model using probability distribution distillation to obtain a pre-trained model for packaging detection; a tobacco package detection model determination module, configured to obtain historical tobacco package detection images captured by a camera in a field of view of a waste collection area, and determine a tobacco package detection model based on the historical tobacco package detection images and the package detection pre-trained model; The early warning module is used to issue an early warning when it is determined that the position of the tobacco package is abnormal based on the current waste collection area field image and the tobacco package detection model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the tobacco raw material packaging detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tobacco raw material packaging detection method according to any one of claims 1 to 7 when executed.

Images