Management and control system and method for operation of cigarette making and tipping machine combination

Abstract

The invention discloses a control system and method for operation of a cigarette making and tipping machine combination. The method comprises the following steps: receiving an operation state of equipment; when the self-triggering condition is met or the running state of the equipment meets the triggering condition, receiving a first data set of a first production operation behavior in preset time, performing operation behavior recognition based on the first data set, and forming a key point operation data set of the equipment according to an operation behavior recognition result in the preset time; and sending the key point operation data set to a local computer, so that the local computer determines whether to send alarm push information to the first production operation behavior or not according to the key point operation data set. According to the method, data acquisition is performed on the production operation behaviors, whether the production operation behaviors conform to the standard operation program or not is judged by analyzing the data, and alarm information is sent out for non-standard behaviors in time.

Description

Technical Field

[0001] The present application relates to the technical field of quality control, and more specifically, to a control system and method for the operation of a rolling and splicing machine group. Background Art

[0002] The packaging process is the most concentrated and prominent part of cigarette product quality feedback. The standard operating procedure (SOP) of packaging equipment is an effective means to guide personnel to operate correctly and ensure the effective implementation of safety, quality and equipment management requirements. However, the existing SOP can only be displayed, trained and coached through means such as graphics, and cannot identify and feedback the actual behavior of operators. The role of standard operating procedures cannot be truly played. Summary of the invention

[0003] The present application provides a control system and method for the operation of a winding and splicing machine group. It collects data on production operation behaviors, and determines whether the production operation behaviors comply with standard operating procedures through data analysis, and promptly issues alarm information for non-standard behaviors, so that the SOP can truly play a role.

[0004] The present application provides a control method for a rolling machine group operation, comprising:

[0005] The operating status of the receiving device;

[0006] When the self-triggering condition is met or the operating state of the equipment meets the triggering condition, a first data set of the first production operation behavior within a preset time is received, and operation behavior recognition is performed based on the first data set, and the operation behavior recognition results within the preset time form a key point operation data set of the equipment;

[0007] The key point operation data set is sent to the local computer, so that the local computer determines whether to issue an alarm push information for the first production operation behavior according to the key point operation data set.

[0008] Preferably, the first data set of the first production operation behavior is a matrix data set obtained by preprocessing the collected first video of the first production operation behavior.

[0009] Preferably, when the key point operation data set is obtained, the first data set is input into the behavior recognition model to perform operation behavior recognition to obtain the first behavior type to which the first production operation behavior belongs.

[0010] Preferably, training the behavior recognition model includes:

[0011] When performing production based on a standard operating procedure of the equipment, receiving a second data set consisting of a second video of a second production operation behavior;

[0012] Extracting a human body region from each frame of the second video, and identifying human body joints in the human body region to obtain a corresponding skeleton model;

[0013] Extracting action features from the skeleton model of each frame, and determining a second behavior type corresponding to the second video based on the action features;

[0014] Annotate each frame and each video in the second data set to form a training set;

[0015] The behavior recognition model is iteratively trained based on the training set.

[0016] Preferably, the local computer provides quality traceability function and operation and maintenance report generation function.

[0017] The present application also provides a control system for the operation of a roll-to-roll machine group, comprising a first receiving module, an operation data set obtaining module and a sending module;

[0018] The first receiving module is used to receive the operating status of the device;

[0019] The operation data set acquisition module is used to receive a first data set of a first production operation behavior within a preset time when a self-trigger condition is met or the operation state of the equipment meets the trigger condition, and perform operation behavior recognition based on the first data set, and the operation behavior recognition result within the preset time forms a key point operation data set of the equipment;

[0020] The sending module is used to send the key point operation data set to the local computer, so that the local computer determines whether to send an alarm push information for the first production operation behavior according to the key point operation data set.

[0021] Preferably, the first data set of the first production operation behavior is a matrix data set obtained by preprocessing the collected first video of the first production operation behavior.

[0022] Preferably, the operation data set acquisition module is used to input the first data set into the behavior recognition model to perform operation behavior recognition, and obtain the first behavior type to which the first production operation behavior belongs.

[0023] Preferably, the control system for the operation of the rolling machine group further includes a training module, which includes a second receiving module, a skeleton model obtaining module, a feature extraction and type determination module, a training set obtaining module and an iteration module;

[0024] The second receiving module is used to receive a second data set consisting of a second video of a second production operation behavior when performing production based on a standard operating procedure of the equipment;

[0025] The skeleton model acquisition module is used to extract a human body region from each frame of the second video, and identify human body joints in the human body region to obtain a corresponding skeleton model;

[0026] The feature extraction and type determination module is used to extract action features from the skeleton model of each frame, and determine the second behavior type corresponding to the second video based on the action features;

[0027] The training set acquisition module is used to annotate each frame and each video in the second data set to form a training set;

[0028] The iteration module is used to iteratively train the behavior recognition model based on the training set.

[0029] Preferably, the local computer provides quality traceability function and operation and maintenance report generation function.

[0030] Other features and advantages of the present application will become apparent from the following detailed description of exemplary embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0032] Figure 1 A structural diagram of a control device for the operation of a rolling and splicing unit provided in this application;

[0033] Figure 2 A flow chart of a control method for a roll-to-roll unit operation provided in this application;

[0034] Figure 3 The training flow chart of the behavior recognition model provided for this application;

[0035] Figure 4 A structural diagram of the control system for the operation of a rolling and splicing unit provided in this application. DETAILED DESCRIPTION

[0036] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0037] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0038] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0039] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0040] The present application provides a control system and method for the operation of a winding and splicing machine group. It collects data on production operation behaviors, and determines whether the production operation behaviors comply with standard operating procedures through data analysis, and promptly issues alarm information for non-standard behaviors, so that the SOP can truly play a role.

[0041] like Figure 1 As shown, the control device for the operation of the winding and splicing machine provided in the present application includes a video acquisition device 110, an equipment status data acquisition device 120, a server 130 and a local computer 140.

[0042] The video acquisition device 110 includes a depth camera for quality inspection and operation and maintenance and a first embedded platform in the depth camera. The depth camera is used to collect operation and maintenance video data and quality inspection video data based on the embedded platform. The first embedded platform stores the video data for a short period of time and performs corresponding preprocessing on the video data. The preprocessing results are uploaded to the server 130.

[0043] The equipment status data acquisition device 120 includes a sensor component and a second embedded platform arranged on the cigarette making machine group. The second embedded platform determines the equipment operation status based on the information of the sensor module, realizes short-term storage of the equipment operation status data, and serves as a trigger source for the event trigger unit in the server 130, and at the same time responds to abnormal equipment operation with an alarm.

[0044] The core controller of the embedded platform is suitable for a variety of cross-domain application scenarios such as advanced graphics, advanced imaging, advanced machine vision, high-end audio, high-end voice, high-end video and high-security embedded applications.

[0045] The video acquisition device 110 and the equipment status data acquisition device 120 realize communication and data transmission with the server 130 through wireless (5G) and wired (Ethernet) methods respectively. The server 130 is mainly used to receive and regularly store the preprocessed data sets of operation and maintenance and quality inspection videos, and receive the operation status data of the equipment, so as to identify the operation behavior of the received preprocessed data sets when the trigger conditions are met, and determine whether the production operation behavior complies with the SOP. The recognition results constitute the key point operation data set of the equipment, and realize the packaging and regular storage of key point operation and maintenance data. In this application, the server usually triggers the operation behavior recognition process according to the equipment status (such as the key links in the operation of the equipment), and can also periodically self-trigger the operation behavior recognition process.

[0046] The local computer 140 realizes communication and data transmission with the server 130 by wireless (5G) and wired (Ethernet) methods. The monitoring software in the local computer 140 receives the key point operation and maintenance data transmitted by the server 130. If a certain production operation behavior does not comply with the SOP, the production operation behavior is a non-standard behavior. The local computer 140 sends an alarm push message on the display screen. At the same time, the local computer 140 also pushes the alarm information to the production operation and maintenance management system, and the corresponding alarm information is pushed through the production operation and maintenance management system. The local computer 140 also realizes quality traceability query based on the pre-processed data set and key point operation and maintenance data transmitted by the server 130, and generates an operation and maintenance report, including the analysis results of the production operation behavior of the operator, the analysis results of the responsiveness of the operator, and the analysis results of the control level of the operator. The local computer 140 is also used for monitoring equipment parameters, real-time display of operation and maintenance and quality inspection videos (with human joints), and historical trend display of monitoring data.

[0047] like Figure 2 As shown, the control method for the operation of the roll-to-roll unit provided in the present application is applicable to the server, and the control method includes:

[0048] S210: The server receives the operating status of the device.

[0049] Specifically, the server receives the running status of the equipment from the equipment status data collection device.

[0050] S220: When the self-trigger condition is met (for example, the preset monitoring time is reached) or the operating status of the equipment meets the trigger condition (for example, the tobacco processing program is started), the server receives a first data set of the first production operation behavior obtained by the video acquisition device within the preset time, and performs operation behavior recognition based on the first data set to determine whether each operation behavior in the first production operation behavior is a non-standard behavior, and combines the recognition results of all operation behaviors within the preset time to form a key point operation data set of the equipment.

[0051] It should be noted that the first production operation behavior within the preset time may include one operation behavior or multiple operation behaviors, so the key point operation data set may include the recognition result of one operation behavior or multiple operation behaviors. The operation behavior includes the operation behavior of the operator and the operation behavior of the equipment.

[0052] Specifically, when the trigger condition is met, the server receives the first data set from the video acquisition device.

[0053] As an embodiment, the first data set of the first production operation behavior is a matrix data set obtained by preprocessing the first video of the first production operation behavior. The depth camera can capture the position and depth information of the object, so by preprocessing each frame of the image captured by the depth camera, the joint matrix formed by the human joints of the operator and the three-dimensional information of the equipment, materials, tools, etc. around the operator can be obtained. The information after preprocessing all frames of the first video constitutes a matrix data set.

[0054] As an embodiment, when obtaining a key point operation data set, firstly, all operation behaviors within the first production operation behavior are classified based on the first data set to form a first behavior type. Secondly, the various parts and movements of the operator and the equipment in each operation behavior within the first production operation behavior are compared with the standard range of the various parts and movements of the operator and the equipment corresponding to the corresponding behavior type. If they are within the range, the recognition result is that the first production operation behavior is a standard behavior; if they are not within the range, the recognition result is that the first production operation behavior is a non-standard behavior. The recognition results of all operation behaviors within the first production operation behavior form the key point operation data set of the equipment.

[0055] Preferably, the first data set is input into a behavior recognition model to perform operation behavior recognition, and a key point operation data set is output.

[0056] As an example, Figure 3 As shown, training the behavior recognition model includes:

[0057] S310: When performing production based on a standard operating procedure of the equipment, receiving a second data set consisting of a second video of a second production operation behavior.

[0058] Specifically, in this step, when the operator performs production according to the requirements of the standard operating procedure (SOP), the video acquisition device acquires the production operation behavior of the operator to obtain a second video, and obtains a second data set after preprocessing the second video. The video acquisition device transmits the second data set to the server.

[0059] Preprocessing involves extracting images from the video frame by frame to obtain multiple images. The images are then enhanced, including adjusting brightness, contrast, and applying filters to remove noise to improve the accuracy of subsequent processing.

[0060] S320: extracting a human body region from each frame of the second video, and identifying human body joints in the human body region to obtain a corresponding skeleton model.

[0061] As an embodiment, when extracting the human body region, a background subtraction algorithm may be used to compare the difference between the current frame and the background frame, detect the motion region, and thus locate the human body.

[0062] As another embodiment, when extracting the human body region, a deep learning model (such as a Faster R-CNN target detection model) may be used to identify and locate the human body in the video.

[0063] When human joints are identified within the human body region, they can be input into a deep learning model (e.g., a PoseNet model) to identify human joints such as the head, shoulders, elbows, wrists, hips, knees, and ankles.

[0064] Preferably, after obtaining the human body joint points, missing value processing is also performed on the human body joint points so that the joint points constituting the skeleton model are complete.

[0065] When dealing with missing values, first perform missing value detection to detect whether the human joint data identified in each frame is missing. For example, if the coordinates corresponding to the human joints are invalid values ​​(such as zero values ​​or specific marker values), the joints are missing. If missing values ​​are detected, interpolation is performed. Interpolation can be performed using the joint coordinate values ​​of the previous or next frame, or using time series analysis techniques such as linear interpolation, polynomial interpolation, or model-based interpolation methods. After interpolation, the data needs to be smoothed to reduce the discontinuity caused by interpolation, which can be achieved by applying a low-pass filter or a smoothing algorithm. The data is then normalized to ensure that the effects of features of different scales on the model are balanced to avoid the negative impact of outliers introduced by interpolation on the normalized results. Finally, the data is input into the recurrent neural network for training and validation. At this stage, it is necessary to ensure that the model can adapt to the interpolated data and test it on the validation set to evaluate the generalization ability of the model. Missing value processing can improve the accuracy and reliability of behavior recognition.

[0066] When obtaining complete joint points and forming a skeleton model, the coordinates of the detected joint points are first converted from the image coordinate system to the actual three-dimensional space coordinate system. Since the video is shot using a depth camera, the depth information of each joint point is calculated using the principle of stereo vision. The joint points in the three-dimensional space are connected according to the human anatomical structure to form a skeleton diagram, and the specific coordinates of each joint point in the three-dimensional space are recorded to form the specific coordinate data of the skeleton diagram.

[0067] S330: Extracting action features from the skeleton model of each frame, and determining a second behavior type corresponding to the second video based on the action features.

[0068] Specifically, the positions of various human body parts in the skeleton model determine a human body action, forming a first action feature of the skeleton model, and the first action feature is classified by comparing the first action feature with a pre-stored action template. The action changes corresponding to the continuous frames of the second video form the action sequence and action mode of the video, forming a second action feature, which is compared with the preset action sequence and action mode, and the second action feature is classified to obtain a second behavior type.

[0069] Preferably, after obtaining the action sequence of the second video, the second video is converted into skeleton time-space matrix data using the skeleton model of each frame and its time series information, and the second action feature is classified using the skeleton time-space matrix data to obtain the second behavior type. The skeleton model can represent the joints of the human body and their connection relationships, while the time-space matrix can capture the spatial position information of these joints that change over time, that is, convert the human action information in the video into a structured form. Through this conversion, key action features can be extracted more effectively, and the accuracy and efficiency of action recognition can be improved.

[0070] S340: Label each frame and each video in the second data set to form a training set.

[0071] Preferably, before annotation, first select the key frames in the video, that is, the frames containing important action information, to reduce the annotation workload. During the annotation process, the action boundaries are marked, and the boundary boxes or key points are drawn in the video frames to accurately define the area or body part where the action occurs. During the annotation process, in addition to marking the human joints, action types, action attributes (such as speed, direction, intensity, etc.) in each frame of the image, the skeleton time-space matrix data and behavior type corresponding to each video, the key facilities of the equipment in each frame of the image (such as the location of the workstation equipment, etc.), the size, position, posture of the material, the location and quantity of the measuring equipment, etc. are marked according to the needs of different application scenarios to fully reflect the correspondence between the operator, the equipment and the operating status.

[0072] S350: Iteratively train the behavior recognition model based on the training set.

[0073] As an embodiment, the trained behavior recognition model provides the range of human joints of the operator corresponding to each operating state of the equipment or each production process, the location range of key facilities of the equipment, the size, position and posture range of the material, etc.

[0074] S230: The server sends the key point operation data set to the local computer, so that the local computer determines whether to issue an alarm push information for the first production operation behavior according to the key point operation data set.

[0075] If the operation behavior identification result in the key point operation data set is a non-standard behavior, the local computer sends an alarm push message for the first production operation behavior on the display screen, and sends an alarm message to the operator to remind the operator to correct the production behavior.

[0076] Based on the above, the present application also provides a control system for the operation of a roll-to-roll unit. Figure 4 As shown, the management and control system includes a first receiving module 410 , a job data set obtaining module 420 and a sending module 430 .

[0077] The first receiving module 410 is used to receive the operating status of the device.

[0078] The operation data set acquisition module 420 is used to receive a first data set of the first production operation behavior within a preset time when the self-trigger condition is met or the operating status of the equipment meets the trigger condition, and perform operation behavior identification based on the first data set. The operation behavior identification results within the preset time form a key point operation data set of the equipment.

[0079] The sending module 430 is used to send the key point operation data set to the local computer, so that the local computer determines whether to send an alarm push information for the first production operation behavior according to the key point operation data set.

[0080] Preferably, the first data set of the first production operation behavior is a matrix data set obtained by preprocessing the collected first video of the first production operation behavior.

[0081] Preferably, the operation data set acquisition module 420 (as an operation behavior recognition device) includes a data set receiving module 4201 and a recognition module 4202 .

[0082] The data set receiving module 4201 is used to receive a first data set of a first production operation behavior of a cigarette making machine group.

[0083] The identification module 4202 is used to identify the operation behavior based on the first data set, and determine whether each operation behavior in the first production operation behavior is a non-standard behavior.

[0084] Preferably, the identification module 4202 includes a classification module 42021 , a comparison module 42022 and a judgment module 42023 .

[0085] The classification module 42021 is used to classify all operation behaviors within the first production operation behavior based on the first data set to form a first behavior type.

[0086] The comparison module 42022 is used to compare the various parts and movements of the operator and equipment in each operation behavior within the first production operation behavior with the standard range of the various parts and movements of the operator and equipment corresponding to the corresponding behavior type.

[0087] The judgment module 42023 is used to judge that the first production operation behavior is a non-standard behavior when the various parts and movements of the operator and equipment in a certain operation behavior within the first production operation behavior are not within the standard range of the various parts and movements of the operator and equipment corresponding to the corresponding behavior type; and to judge that the first production operation behavior is a standard behavior when the various parts and movements of the operator and equipment in a certain operation behavior within the first production operation behavior are within the standard range of the various parts and movements of the operator and equipment corresponding to the corresponding behavior type.

[0088] Preferably, the recognition module 4202 is a behavior recognition model, that is, the first data set is input into the behavior recognition model to perform operation behavior recognition to obtain a key point operation data set.

[0089] Preferably, in the control system for the rolling machine operation, the operation data set acquisition module 420 also includes a training module 4203, and the training module 4203 includes a second receiving module 42031, a skeleton model acquisition module 42032, a feature extraction and type determination module 42033, a training set acquisition module 42034 and an iteration module 42035.

[0090] The second receiving module 42031 is used to receive a second data set consisting of a second video of a second production operation behavior when production is performed based on the standard operating procedure of the equipment.

[0091] The skeleton model acquisition module 42032 is used to extract the human body region from each frame of the second video, and identify the human body joints in the human body region to obtain the corresponding skeleton model.

[0092] The feature extraction and type determination module 42033 is used to extract action features from the skeleton model of each frame, and determine the second behavior type corresponding to the second video based on the action features.

[0093] The training set acquisition module 42034 is used to annotate each frame and each video in the second data set to form a training set.

[0094] The iteration module 42035 is used to iteratively train the behavior recognition model based on the training set.

[0095] This application combines video recognition technology with SOP to provide timely reminders for key links in quality inspection and provide instant feedback on the operator's performance, so that SOP can be effectively implemented, effectively standardize operating behaviors, and prevent and control quality risks from the source. In addition, this application is committed to the data-based research of operation and maintenance behaviors, converting video data into skeleton time-series-space matrix data, which can accurately restore operation and maintenance behaviors while compressing the data volume. It has data storage and query capabilities with a span of years, so that quality traceability can be integrated into assessment management, and assessment pressure can be converted into a working motivation for operators to compare with standard operating procedures. SOP is used as a prerequisite for other assessment items, so that operators can work together and continue to focus their work on benchmarking, laying the foundation for high-quality development.

[0096] This application uses real-time video recognition technology to estimate the posture of cigarette machine operators' operation and maintenance behaviors and quality inspection actions, uses the operating status of the cigarette machine as the time baseline to trigger standard operating procedures, trains and optimizes deep learning models, and achieves instant warnings and regular records of irregular behaviors, thereby improving equipment operation and maintenance management and product quality traceability. It is necessary to standardize operating behaviors and further effectively prevent and reduce equipment operation and maintenance and quality control risks.

[0097] Although some specific embodiments of the present application have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are only for illustration, not for limiting the scope of the present application. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.

Claims

1. A control method for the operation of a rolling and splicing unit, characterized in that: include: The operating status of the receiving device; When a self-triggering condition is met or the operating state of the device meets a triggering condition, a first data set of a first production operation behavior within a preset time is received, and operation behavior recognition is performed based on the first data set, and the operation behavior recognition result within the preset time forms a key point operation data set of the device; The key point operation data set is sent to a local computer, so that the local computer determines whether to issue an alarm push information for the first production operation behavior according to the key point operation data set.

2. The control method for the operation of a rolling and splicing unit according to claim 1, characterized in that: The first data set of the first production operation behavior is a matrix data set obtained by preprocessing the collected first video of the first production operation behavior.

3. The control method for the operation of a rolling and splicing unit according to claim 1, characterized in that: When the key point operation data set is obtained, the first data set is input into the behavior recognition model to perform operation behavior recognition to obtain the key point operation data set.

4. The control method for the operation of a rolling and splicing unit according to claim 3, characterized in that: Training the behavior recognition model includes: When performing production based on a standard operating procedure of the equipment, receiving a second data set consisting of a second video of a second production operation behavior; Extracting a human body region from each frame of the second video, and identifying human body joints in the human body region to obtain a corresponding skeleton model; Extracting action features from the skeleton model of each frame, and determining a second behavior type corresponding to the second video based on the action features; Annotating each frame and each video in the second data set to form a training set; The behavior recognition model is iteratively trained based on the training set.

5. The control method for the operation of a rolling and splicing unit according to claim 1, characterized in that: The local computer provides quality tracing functions and operation and maintenance report generation functions.

6. A control system for the operation of a roll-to-roll unit, characterized in that: It includes a first receiving module, a job data set obtaining module and a sending module; The first receiving module is used to receive the operating status of the device; The operation data set acquisition module is used to receive a first data set of a first production operation behavior within a preset time when a self-trigger condition is met or the operating state of the device meets the trigger condition, and perform operation behavior recognition based on the first data set, and the operation behavior recognition result within the preset time forms a key point operation data set of the device; The sending module is used to send the key point operation data set to a local computer, so that the local computer determines whether to issue an alarm push information for the first production operation behavior according to the key point operation data set.

7. The control system for the operation of the rolling and splicing unit according to claim 6, characterized in that: The first data set of the first production operation behavior is a matrix data set obtained by preprocessing the collected first video of the first production operation behavior.

8. The control system for the operation of the rolling and splicing unit according to claim 6, characterized in that: The operation data set acquisition module is used to input the first data set into a behavior recognition model to perform operation behavior recognition and obtain the key point operation data set.

9. The control system for the operation of the rolling and splicing unit according to claim 8, characterized in that: It also includes a training module, which includes a second receiving module, a skeleton model obtaining module, a feature extraction and type determination module, a training set obtaining module and an iteration module; The second receiving module is used to receive a second data set consisting of a second video of a second production operation behavior when performing production based on a standard operating procedure of the equipment; The skeleton model acquisition module is used to extract a human body region from each frame of the second video, and identify human body joints in the human body region to obtain a corresponding skeleton model; The feature extraction and type determination module is used to extract action features from the skeleton model of each frame, and determine the second behavior type corresponding to the second video based on the action features; The training set acquisition module is used to annotate each frame and each video in the second data set to form a training set; The iteration module is used to iteratively train the behavior recognition model based on the training set.

10. The control system for the operation of the rolling and splicing unit according to claim 6, characterized in that: The local computer provides quality tracing function and operation and maintenance report generation function.

Images