Paperboard state monitoring method and system for wooden supporting board of papermaking wet pulp packaging machine

Abstract

The invention provides a paperboard state monitoring method and system for a wooden supporting plate of a papermaking wet pulp packaging machine, and belongs to the technical field of automatic monitoring. The method comprises the following steps: acquiring an original image of a field network camera through an RTSP protocol, and carrying out standardized preprocessing; the method comprises the following steps: collecting preprocessed images in a paper / paper-free state under different illumination, constructing a training set and training a convolutional neural network model; during production operation, images are acquired and preprocessed in real time, a CNN model is loaded for state inference, and papery or paperless labels are output; and finally, a control signal is sent to the programmable controller through a TCP protocol according to the label, and switching value output is driven to be linked with a packaging machine system. According to the method, non-contact visual identification is adopted, the problems of dependence on the physical structure of the supporting plate and misjudgment of indirect judgment logic in a traditional method are solved, stable and accurate state monitoring under the complex illumination environment is achieved, and the automation level and operation reliability of a production line are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of automated monitoring technology, and more specifically relates to a method and system for monitoring the status of cardboard on wooden pallets of a papermaking wet pulp packaging machine. Background Technology

[0002] In the automated process of a paper pulp packaging production line, accurately determining whether there are packaging paperboard (wet pulp bales) on wooden pallets is a crucial step in ensuring continuous and stable equipment operation. This determination directly impacts subsequent control logic such as paper picking, packaging, automatic switching between empty and full pallets, and system start-up and shutdown. Traditional monitoring methods primarily rely on physical sensors or indirect inference based on equipment action feedback, which have significant limitations in practical applications.

[0003] A common approach relies on machining a specific central circular perforation into the pallet and using a laser or photoelectric sensor to detect the occlusion status of this perforation. When cardboard is present, it blocks the perforation, and the sensor status changes; when no cardboard is present, the perforation is exposed, and the status returns to normal. However, this method requires the pallet to have this precision perforation pre-fabricated. In practice, many wooden pallets are not designed with this structure due to strength, cost, or standardization considerations, resulting in a lack of versatility and making this technology unsuitable for scenarios with pallets lacking the perforation.

[0004] Another widely adopted strategy is indirect logical judgment based on the number of "paper retrieval failures." The system typically only determines that the tray is "out of paper" after multiple consecutive failed paper retrieval attempts. The fundamental flaw of this method lies in the non-uniqueness of its causal logic. Paper retrieval failures can stem from various factors, such as insufficient vacuum suction, abnormal moisture on the cardboard surface, or mechanical positioning errors. Therefore, even under normal circumstances where there is paper on the tray, multiple paper retrieval failures can occur due to other malfunctions, leading the system to misjudge "out of paper," generate incorrect alarms, or execute harmful subsequent actions.

[0005] In summary, existing technologies either lack versatility due to their reliance on specific pallet physical structures or suffer from poor reliability due to their use of indirect judgment logic, thus failing to achieve direct, reliable, and universal online monitoring of the status of wooden pallets and cardboard. This technological bottleneck directly prevents packaging machines from intelligently identifying empty / full pallets for automatic switching, and also prevents them from accurately triggering automatic shutdown when paper is actually out of stock, severely hindering the improvement of automation and intelligence levels in production lines. Summary of the Invention

[0006] To address the above problems, the present invention aims to provide a method and system for monitoring the cardboard status of wooden pallets in a papermaking wet pulp packaging machine. Through non-contact visual recognition based on convolutional neural networks, the present invention achieves direct, accurate, and real-time determination of the cardboard status of wooden pallets that is adaptable to complex working conditions. This effectively solves the unreliability problem caused by traditional methods relying on specific physical structures and indirect logical judgments, thereby significantly improving the accuracy and continuity of automated control of the packaging machine production line.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a method for monitoring the condition of cardboard on wooden pallets of a papermaking wet pulp packaging machine, comprising: The original images of the pallets are acquired using network cameras on the paper wet pulp packaging production line via the RTSP streaming media protocol. The original images are then subjected to standardized preprocessing to generate standardized preprocessed images. Acquire multiple standardized preprocessed images of the tray with and without paper under different ambient lighting conditions, construct a training image set and label the cardboard state, use the labeled training image set to train a convolutional neural network model, and generate a trained CNN model. During the operation of the papermaking wet pulp packaging production line, real-time images of the pallets are captured by a webcam and processed to generate real-time standardized pre-processed images; a CNN model is loaded to infer the real-time standardized pre-processed images and output cardboard status labels. Based on the cardboard status label, the corresponding control signal is written to the programmable controller via TCP protocol; the programmable controller drives the switch output according to the received signal to link the packaging machine system.

[0008] In one optional implementation, the step of acquiring raw images of pallets using a network camera on the paper pulp packaging production line via the RTSP streaming media protocol, and then performing standardized preprocessing on the raw images to generate standardized preprocessed images includes: Using the integrated preset SDK function library via the RTSP streaming media protocol, log in to the specified webcam and capture a single frame of the tray, saving it as an original BMP image; The original BMP image is read, its RGB channels are separated, histogram equalization is performed on each channel, and the processed channels are merged to generate a merged image; the histogram equalization process includes calculating the cumulative distribution function of pixel values ​​and mapping according to the calculation results; The merged images are converted to preset formats and specifications to generate standardized preprocessed images.

[0009] In an optional implementation, the step of acquiring multiple standardized preprocessed images of the tray in a paper-containing and paperless state under different ambient lighting conditions, constructing a training image set and labeling the cardboard state, and using the labeled training image set to train a convolutional neural network model to generate a trained CNN model includes: Within a predetermined period, the network camera is controlled to collect standardized pre-processed images of the tray with and without paper at different times to form the original training image set. Using an auxiliary annotation tool, each image in the original training image set is labeled with either a "havepaper" tag or a "nohavepaper" tag, generating an annotated image set. Image enhancement operations are then performed on the annotated image set, and the training image set is output. Using the training image set, the prediction results are obtained through forward propagation of a convolutional neural network; the cross-entropy loss between the prediction results and the true labels is calculated, and an adaptive moment estimation optimizer is used to iteratively update the convolutional kernel weights and bias term parameters in the network through the backpropagation algorithm to minimize the loss function; after reaching the preset number of iterations or performance convergence, the optimized parameters and network structure are fixed and saved to generate a CNN model file.

[0010] In an optional implementation, during the operation of the paper pulp packaging production line, real-time images of the pallets are acquired via a webcam and processed to generate real-time standardized preprocessed images; a CNN model is then loaded to infer the real-time standardized preprocessed images and output cardboard status labels, including: During the operation of the paper wet pulp packaging production line, real-time images of the pallets are acquired using network cameras on the production line via the RTSP streaming media protocol. The real-time images are then preprocessed in a standardized manner to generate real-time standardized preprocessed images. Load the CNN model file and input the real-time standardized preprocessed image into the CNN model; call the forward computation engine of the CNN model to execute the convolution, activation and pooling operation sequences stored in the CNN model in sequence, and output a classification probability distribution vector containing the probability values ​​of each preset category through the fully connected layer and the output layer; From the classification probability distribution vector, extract the first probability value P1 corresponding to the havepaper category and the second probability value P0 corresponding to the nohavepaper category; compare the magnitudes of P1 and P0, and determine the label corresponding to the larger probability value as the final cardboard status label and output it.

[0011] In an optional implementation, the step of writing a corresponding control signal to the programmable controller via TCP protocol based on the cardboard status label, and then having the programmable controller drive a switching output based on the received signal to coordinate the packaging machine system, includes: In a programmable controller, the address of a status flag register that indicates whether there is paper or not is predefined and mapped to a physical output point. Convert the cardboard status label into the corresponding logical value; assemble a write instruction message containing the status flag register address and logical value according to the industrial communication protocol format supported by the programmable controller, and send it through the TCP / IP network; After successfully receiving and executing the write instruction, the programmable controller updates the state of its internal registers and drives the corresponding switch output points accordingly, thereby achieving linkage with the main control system of the packaging machine.

[0012] In one optional implementation, the programmable controller is an S7-200Smart series PLC, and the industrial communication protocol is ISO-on-TCP & S7 protocol; the write instruction message includes a TPKT header, a COTP layer, an S7 communication header, and a parameter data block containing the target address and data.

[0013] In an optional implementation, the process of acquiring and processing real-time images of the pallets via a webcam during the operation of the paper pulp packaging production line to generate real-time standardized pre-processed images, and loading a CNN model to infer the real-time standardized pre-processed images and output cardboard status labels, further includes: When the larger of the first probability value P1 and the second probability value P0 is lower than a preset confidence threshold, the confidence of this inference is determined to be insufficient; at least one image acquisition parameter of the network camera is automatically adjusted, and the real-time image of the tray is re-acquired and processed after adjustment to generate a real-time standardized pre-processed image, which is then loaded into a CNN model for inference and outputs a cardboard status label.

[0014] Secondly, embodiments of this application also provide a paperboard condition monitoring system for wooden pallets of a papermaking wet pulp packaging machine, comprising: The image acquisition and processing module is used to acquire the original image of the pallet using a network camera on the paper wet pulp packaging production line via the RTSP streaming media protocol, and to perform standardized preprocessing on the original image to generate a standardized preprocessed image. The model training module is used to acquire multiple standardized preprocessed images of the tray with and without paper under different ambient lighting conditions, construct a training image set and label the cardboard state, and use the labeled training image set to train a convolutional neural network model to generate a trained CNN model. The real-time inference module is used to acquire and process real-time images of the pallets via a webcam during the operation of the paper wet pulp packaging production line, generating real-time standardized pre-processed images; loading a CNN model to infer the real-time standardized pre-processed images and output cardboard status labels; The linkage control module is used to write corresponding control signals to the programmable controller via TCP protocol according to the cardboard status label; the programmable controller drives the switch output according to the received signals to link the packaging machine system.

[0015] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the paperboard state monitoring method for wooden pallets of a papermaking wet pulp packaging machine as described in any of the above claims.

[0016] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the paperboard state monitoring method for wooden pallets of a papermaking wet pulp packaging machine as described in any of the above claims.

[0017] As can be seen from the above technical solutions, the present invention has the following advantages: The paperboard condition monitoring method for wooden pallets in a papermaking wet pulp packaging machine provided in this application directly determines the state of the paperboard on the surface of the wooden pallet through non-contact visual inspection and convolutional neural network (CNN) intelligent recognition technology. This effectively overcomes the dependence on the physical structure of the pallet and the unreliability of indirect logical judgments in traditional methods. This method can achieve high-precision and robust real-time online monitoring, significantly reducing the false and false judgment rates. This ensures that the packaging machine system can accurately and timely execute control logic such as automatic switching between empty / full pallets and process start / stop, greatly improving the automation level, operating efficiency, and reliability of the production line.

[0018] This application achieves a fundamental innovation and improved versatility in its detection method by employing a non-contact detection principle based on computer vision, eliminating the reliance on specific physical structures of the pallet (such as a central perforation). By directly analyzing the visual features of the pallet surface, this method is applicable to various standard wooden pallets without pre-drilled holes, overcoming the limitations of traditional solutions due to hardware dependence.

[0019] This application significantly enhances the system's environmental adaptability and robustness by constructing a training image set covering multiple time periods and lighting conditions, combined with adaptive image preprocessing techniques. The trained convolutional neural network model can effectively compensate for interference from lighting changes, thereby maintaining stable and high-precision recognition performance under complex industrial lighting conditions, ensuring the reliability of monitoring.

[0020] This application utilizes a pre-trained convolutional neural network model to perform end-to-end direct analysis and state determination on pre-processed images, providing deterministic "paper present" or "paperless" labels and constructing a direct and reliable decision-making logic. This fundamentally avoids the chain of misjudgments that may occur when inferring based on indirect signals such as "multiple paper-taking failures," significantly improving the accuracy of state determination.

[0021] This application achieves seamless and efficient integration with existing industrial automation control systems by converting visual recognition results into communication messages conforming to industrial standards and communicating with programmable controllers using the TCP / IP protocol and specific industrial control protocols. This method allows the monitoring system to be easily integrated as an intelligent sensing module, driving subsequent linkage control with switch signals, thus reducing the complexity and cost of system modification.

[0022] This application provides a reliable basis for the core control system to execute high-level automation logic by providing real-time, accurate, and direct cardboard status signals. This enables the packaging machine to intelligently and promptly complete operations such as automatic switching between empty and full pallets and precise start and stop of packaging cycles, thereby reducing production interruptions and material waste, and ultimately improving the automation level, operating efficiency, and economic benefits of the entire production line. Attached Figure Description

[0023] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the method for monitoring the condition of wooden pallets in a papermaking wet pulp packaging machine provided in this application.

[0025] Figure 2 This is a schematic diagram of the paperboard condition monitoring system for the wooden pallet of the papermaking wet pulp packaging machine provided in this application.

[0026] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0027] The various embodiments of this disclosure will be described more fully in the following detailed description of the specific steps of the method for monitoring the condition of wooden pallets in a papermaking wet pulp packaging machine. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0028] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 The diagram shows a flowchart of a method for monitoring the condition of a wooden pallet in a papermaking wet pulp packaging machine, according to a specific embodiment. The method includes: S1: The original image of the pallet is obtained by using a network camera on the paper wet pulp packaging production line via the RTSP streaming media protocol. The original image is then subjected to standardized preprocessing to generate a standardized preprocessed image.

[0031] In a specific implementation, real-time photos from the on-site network camera are acquired via the RTSP streaming media protocol, original images are captured under multiple lighting conditions, and histogram equalization is performed on the photos to enhance image contrast and compensate for the color deviation caused by external lighting.

[0032] Specifically, this step includes the following sub-steps: S101: Integrates Hikvision SDK function library, logs into a specified network camera and captures a single frame, saving it as a raw BMP image.

[0033] For example, the RTSP streaming media protocol is used to read each frame of photo data from the on-site network camera. For easy integration, the SDK (which can call function libraries DLLs) provided by Hikvision is integrated using the Winform platform. The library functions are called to the network camera according to the actual situation, which is divided into: initializing the SDK -> logging into the network camera via IP / camera account / camera password -> browsing the network camera image -> image capture -> saving the image to the local machine in BMP format.

[0034] S102: Read the original BMP image and separate its RGB channels; perform histogram equalization processing on each channel, the processing including calculating the cumulative distribution function of pixel values ​​and mapping it accordingly; then merge the processed channels.

[0035] For example, a local BMP format photo is extracted into the program cache, the RGB channel of the image is split into three individual R, G, and B channels, and histogram equalization is performed on each channel before merging them into an RGB image.

[0036] The specific principles of image processing are as follows: RGB image histogram equalization is a key preprocessing technique for improving the color contrast of "white paper / yellow wood board" (especially suitable for scenes where white / yellow features are blurred due to uneven lighting).

[0037] First, calculate the single-channel histogram using the following formula:

[0038] Where W is the image width, H is the image height, and K is a 250 or 180 pixel value. For example: for the R channel, count the number of pixels for "white cardboard R value = 250" and the number of pixels for "yellow wooden pallet R value = 180".

[0039] Then, the single-channel pixel probability is calculated using the following formula:

[0040] Single-channel pixel probability represents the frequency of pixel value k in that channel (e.g., in the R channel, the probability of 250 is 0.3, and the probability of 180 is 0.2).

[0041] Furthermore, the cumulative distribution function is calculated using the following formula:

[0042] Finally, the equalization mapping is performed using the following formula:

[0043] Wherein, if IR(x,y)=200 in the R channel represents a matte white paper, and its CDFr(200)=0.8, then This indicates an increase in brightness. S103: Output the merged image as the standardized preprocessed image.

[0044] S2: Obtain multiple standardized preprocessed images of the tray in paper-containing and paperless states under different ambient lighting conditions, construct a training image set and label the cardboard state, use the labeled training image set to train a convolutional neural network model, and generate a trained CNN model.

[0045] In a specific implementation, real-time images of the scene are acquired through a streaming media protocol, and photos of these images under different ambient light conditions / states (with or without paperboard on the tray: the image is white when there is paper on the tray and yellow when there is no paper) are stored locally. These images are then merged into a unified file set. Each image is labeled (havepaper / nohavepaper) using auxiliary annotation software, and image enhancement learning is performed to obtain key parameters such as convolution kernels, normalization methods, and feature equation baseline rules under simulated different lighting conditions, thereby generating a trained CNN model.

[0046] Specifically, this step includes the following sub-steps: S201: Within a predetermined period, control the camera to repeatedly execute step S1 at different time periods to collect the standardized preprocessed images in two states: paper on the tray and paperless, to form the original training image set.

[0047] For example, using the WinForms platform, a timed photo capture and local saving function was created. At 6:00 AM, 6:30 AM, 7:00 AM... 11:30 PM, 10 images were captured every half hour for a total of 24 hours, one image per minute, capturing images separately for both paper-containing and paperless states on the tray. The total time was 48 hours.

[0048] S202: Using an auxiliary annotation tool, label each image in the original training image set with the tags "havepaper" or "nohavepaper"; perform image enhancement operations on the labeled set to generate the final training image set.

[0049] For example, the MVTec Deep Learning Tool software was used to classify the collected images of paper-containing and paperless states, respectively. The labels were in English ASCII encoding format: havepaper and nohavepaper.

[0050] S203: Use the training image set to train the preset convolutional neural network. After training, save the model parameters and structure as the trained CNN model file.

[0051] For example, the Compact preprocessing model is used to train a model for classifying photos. The specific training parameters are as follows: Image width: 256, image height: 256, number of epochs: 2000, number of iterations: 100, image enhancement: enabled.

[0052] S3: During the operation of the papermaking wet pulp packaging production line, real-time images of the pallets are acquired through a network camera and processed to generate real-time standardized pre-processed images; a CNN model is loaded to infer the real-time standardized pre-processed images and output cardboard status labels.

[0053] In a specific implementation, real-time photos from the on-site network camera are acquired via the Rtsp streaming media protocol, and images of the on-site operation are collected. Based on the pre-learning rules during "learning", the images are processed and inferred using INT8 quantization. The corresponding feature vectors are obtained by performing 3×3 convolution operations based on the convolution kernels, and the corresponding labels (havepaper / nohavepaper) are output based on the vectors. The stage goal is color binary classification.

[0054] Specifically, this step includes the following sub-steps: S301: When the papermaking wet pulp packaging production line is running, repeat steps S101 to S103 to obtain the real-time pre-processed image at the current moment.

[0055] S302: Load the trained CNN model file from the storage medium, input the pixel data of the real-time preprocessed image in the format required by the model; call the model forward computation engine, and execute the convolution, activation and pooling operation sequences stored in the model in sequence, and finally output a classification probability distribution vector containing the probability values ​​of each preset category by the fully connected layer and the output layer.

[0056] For example, the convolutional layers in the model are used to capture the distribution features of white and yellow pixels, and the specific formula is as follows:

[0057] Where F is the input feature map (e.g., H-channel tensor, white area value ≈ 1, yellow area value ≈ 0.7). K represents the kernel (3×3 kernel for compact models) extracted from the convolution kernel model in the HDL file. I and j are the coordinates of the output feature map, m and n are the local coordinates of the convolution kernel, and b is the bias term (compensating for the overall offset after convolution).

[0058] Taking the first convolutional kernel (mean kernel) of Halcon as an example:

[0059] Taking the first layer convolution kernel (mean kernel) as an example: With paper (white): F[i+m,j+n]≈1; Convolution result≈9×1×(1 / 9)+b=1+b; Paperless (yellow): F[i+m,j+n]≈0.7; Convolution result≈9×0.7×(1 / 9)+b=0.7+b.

[0060] S303: Extract the first probability value P1 corresponding to the "havepaper" category and the second probability value P0 corresponding to the "nohavepaper" category from the classification probability distribution vector; compare the magnitudes of P1 and P0, determine the label corresponding to the larger probability value as the final cardboard status label and output it.

[0061] In addition, step S303 also includes a confidence verification and re-examination mechanism: when the value of the larger of the first probability value P1 and the second probability value P0 is lower than a preset confidence threshold, it is determined that the confidence of this inference is insufficient; then at least one image acquisition parameter of the network camera is automatically adjusted, and after adjustment, steps S301 to S303 are re-executed to obtain a new inference result; the results of multiple inferences are fused or the final result is selected according to a preset strategy to generate a more reliable cardboard status label.

[0062] S4: Based on the cardboard status label, write the corresponding control signal to the programmable controller via TCP protocol; the programmable controller drives the switch output according to the received signal to link the packaging machine system. In a specific implementation, after obtaining the label, a string comparison is performed based on the label name. For example, when the text matches the "paper present" state, the program will set the corresponding address of the programmable controller to True via the TCP protocol to indicate that there is paper on the tray. Otherwise, there is no paper. The programmable controller then outputs the obtained signal as a switch input signal through address mapping. At this time, if there is no paper on the tray, it means that the DO switch input signal is turned on. Finally, it is connected to the existing system switch input signal through relays and cables for linkage operation.

[0063] Specifically, this step includes the following sub-steps: S401: In a programmable controller, the address of a status flag register used to indicate the presence or absence of paper is predefined and mapped to a physical output point.

[0064] Taking Siemens S7-200Smart as an example, the register addresses are labeled and annotated, and the relationships are mapped according to the digital output points (if a point is true, the digital output DO point is engaged). The DO points are: system power-on status, paper present / no-paper status, and fault signal.

[0065] S402: Convert the cardboard status label generated in step S303 into the corresponding logical value; assemble a write instruction message containing the status flag register address and logical value according to the industrial communication protocol format supported by the programmable controller, and send it through the TCP / IP network.

[0066] For example, parse the Siemens ISO-on-TCP & S7 Comm protocol, dividing its protocol message format into the TPKT layer (identifying the ISO-on-TCP message length), the COTP layer (type, destination / establishing a logical connection), the S7 Header (function code, data length), and the Data (address, data content), and write a communication program according to the Siemens ISO-on-TCP & S7 Comm protocol format.

[0067] Taking "Read DB1.DBW20" as an example (the actual address corresponding to 200Smart is VW20), the complete hexadecimal byte stream of the request message is as follows: / / Total length: 37 bytes (0x25) 03 00 00 25 / / TPKT layer (4 bytes) 02 00 00 01 / / COTP layer (4 bytes) 32 01 00 00 / / S7 Header (7 bytes, first 4 bytes) 04 00 00 00 / / S7 Header (last 3) + Function Code 00 00 04 01 / / S7 parameter block (13 bytes, first 4 bytes) 12 0A 10 02 / / Parameter block (middle 4) 00 01 00 14 / / Parameter block (last 4) 00 02 / / Parameter block (last 1) 00 00 / / S7 data block (2 bytes, empty) S403: After successfully receiving and executing the write instruction, the programmable controller updates the state of its internal registers and drives the corresponding switch output points accordingly, thereby achieving linkage with the main control system of the packaging machine.

[0068] In this embodiment, by integrating non-contact visual acquisition, adaptive image preprocessing, convolutional neural network intelligent recognition, and industrial protocol communication, direct, accurate, and stable real-time monitoring of the pallet cardboard status is achieved. This effectively solves the problems of dependence on the physical structure of the pallet and misjudgment by indirect judgment logic in traditional methods, thereby significantly improving the automation level and operational reliability of the packaging machine in terms of automatic switching between empty and full pallets and precise process control.

[0069] like Figure 2 As shown, the following is an embodiment of the paperboard condition monitoring system for wooden pallets of a papermaking wet pulp packaging machine provided in this disclosure. This system and the paperboard condition monitoring methods for wooden pallets of papermaking wet pulp packaging machines in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the paperboard condition monitoring system for wooden pallets of papermaking wet pulp packaging machines, please refer to the embodiments of the paperboard condition monitoring methods for wooden pallets of papermaking wet pulp packaging machines described above.

[0070] A paperboard condition monitoring system for wooden pallets in a papermaking wet pulp packaging machine includes: The image acquisition and processing module is used to acquire the original image of the pallet using a network camera on the paper wet pulp packaging production line via the RTSP streaming media protocol, and to perform standardized preprocessing on the original image to generate a standardized preprocessed image.

[0071] The model training module is used to acquire multiple standardized preprocessed images of the tray with and without paper under different ambient lighting conditions, construct a training image set and label the paperboard state, and use the labeled training image set to train a convolutional neural network model to generate a trained CNN model.

[0072] The real-time inference module is used to acquire and process real-time images of the pallets via a webcam during the operation of the paper wet pulp packaging production line, generating real-time standardized pre-processed images; loading a CNN model to infer the real-time standardized pre-processed images and output cardboard status labels.

[0073] The linkage control module is used to write corresponding control signals to the programmable controller via TCP protocol according to the cardboard status label; the programmable controller drives the switch output according to the received signals to link the packaging machine system.

[0074] The paperboard status monitoring system for wooden pallets of the papermaking wet pulp packaging machine provided in this embodiment directly determines the presence status of the paperboard on the wooden pallet through non-contact visual recognition and adaptive image processing based on convolutional neural networks. It effectively overcomes the dependence on the physical structure of the pallet and the unreliability of indirect logical inference in traditional methods, and realizes stable and accurate real-time monitoring under complex lighting conditions. This significantly improves the intelligence level and overall operating efficiency of the packaging machine production line in terms of automatic pallet changing and precise start-stop.

[0075] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0076] The paperboard condition monitoring method for wooden pallets in a papermaking wet pulp packaging machine provided in this application embodiment can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0077] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0078] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0079] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0080] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0081] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0082] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0083] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0084] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0085] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0086] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0087] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0088] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0089] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0090] The aforementioned electronic device realizes the paperboard state monitoring method of the wooden pallet of the papermaking wet pulp packaging machine of this application. By adopting a non-contact visual detection scheme based on convolutional neural network, it directly performs intelligent analysis and recognition of the surface state of the pallet and achieves seamless integration with the industrial control system. It achieves high-precision and robust real-time monitoring of the paperboard's state without modifying the physical structure of the pallet or eliminating interference from equipment operation, thereby significantly improving the automation level and operating efficiency of the packaging production line.

[0091] The storage medium provided in this application stores a program product capable of implementing a method for monitoring the condition of paperboard on wooden pallets of a papermaking wet pulp packaging machine.

[0092] Methods for monitoring the condition of wooden pallets in papermaking wet pulp packaging machines include: The original images of the pallets are acquired using network cameras on the paper wet pulp packaging production line via the RTSP streaming media protocol. The original images are then subjected to standardized preprocessing to generate standardized preprocessed images. Acquire multiple standardized preprocessed images of the tray with and without paper under different ambient lighting conditions, construct a training image set and label the cardboard state, use the labeled training image set to train a convolutional neural network model, and generate a trained CNN model. During the operation of the papermaking wet pulp packaging production line, real-time images of the pallets are captured by a webcam and processed to generate real-time standardized pre-processed images; a CNN model is loaded to infer the real-time standardized pre-processed images and output cardboard status labels. Based on the cardboard status label, the corresponding control signal is written to the programmable controller via TCP protocol; the programmable controller drives the switch output according to the received signal to link the packaging machine system.

[0093] In some possible implementations, the paperboard condition monitoring method for wooden pallets of a papermaking wet pulp packaging machine disclosed herein can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0094] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0095] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring the condition of cardboard on wooden pallets of a papermaking wet pulp packaging machine, characterized in that, include: The original images of the pallets are acquired using network cameras on the paper wet pulp packaging production line via the RTSP streaming media protocol. The original images are then subjected to standardized preprocessing to generate standardized preprocessed images. Acquire multiple standardized preprocessed images of the tray with and without paper under different ambient lighting conditions, construct a training image set and label the cardboard state, use the labeled training image set to train a convolutional neural network model, and generate a trained CNN model. During the operation of the papermaking wet pulp packaging production line, real-time images of the pallets are captured by a webcam and processed to generate real-time standardized pre-processed images; a CNN model is loaded to infer the real-time standardized pre-processed images and output cardboard status labels. Based on the cardboard status label, the corresponding control signal is written to the programmable controller via TCP protocol; the programmable controller drives the switch output according to the received signal to link the packaging machine system.

2. The method for monitoring the cardboard condition of a wooden pallet in a papermaking wet pulp packaging machine according to claim 1, characterized in that, The process involves acquiring raw images of pallets using network cameras on the paper pulp packaging production line via the RTSP streaming media protocol, and then performing standardized preprocessing on these raw images to generate standardized preprocessed images, including: Using the integrated preset SDK function library via the RTSP streaming media protocol, log in to the specified webcam and capture a single frame of the tray, saving it as an original BMP image; The original BMP image is read, its RGB channels are separated, histogram equalization is performed on each channel, and the processed channels are merged to generate a merged image; the histogram equalization process includes calculating the cumulative distribution function of pixel values ​​and mapping according to the calculation results; The merged images are converted to preset formats and specifications to generate standardized preprocessed images.

3. The method for monitoring the paperboard condition of a wooden pallet in a papermaking wet pulp packaging machine according to claim 2, characterized in that, The process involves acquiring multiple standardized preprocessed images of the tray in both paper-containing and paperless states under different ambient lighting conditions, constructing a training image set and labeling the tray with cardboard state tags, and then using the labeled training image set to train a convolutional neural network model to generate a trained CNN model. Within a predetermined period, the network camera is controlled to collect standardized pre-processed images of the tray with and without paper at different times to form the original training image set. Using an auxiliary annotation tool, each image in the original training image set is labeled with either a "havepaper" or "nohavepaper" tag, generating an annotated image set. Image enhancement operations are then performed on the annotated image set, and the training image set is output. Using the training image set, the prediction results are obtained through forward propagation of a convolutional neural network; the cross-entropy loss between the prediction results and the true labels is calculated, and an adaptive moment estimation optimizer is used to iteratively update the convolutional kernel weights and bias term parameters in the network through the backpropagation algorithm to minimize the loss function; after reaching the preset number of iterations or performance convergence, the optimized parameters and network structure are fixed and saved to generate a CNN model file.

4. The method for monitoring the paperboard condition of a wooden pallet in a papermaking wet pulp packaging machine according to claim 3, characterized in that, During the operation of the papermaking wet pulp packaging production line, real-time images of the pallets are acquired through a network camera and processed to generate real-time standardized pre-processed images. The CNN model is loaded to infer the real-time normalized preprocessed image and output cardboard status labels, including: During the operation of the paper wet pulp packaging production line, real-time images of the pallets are acquired using network cameras on the production line via the RTSP streaming media protocol. The real-time images are then preprocessed in a standardized manner to generate real-time standardized preprocessed images. Load the CNN model file and input the real-time standardized preprocessed image into the CNN model; call the forward computation engine of the CNN model to execute the convolution, activation and pooling operation sequences stored in the CNN model in sequence, and output a classification probability distribution vector containing the probability values ​​of each preset category through the fully connected layer and the output layer; From the classification probability distribution vector, extract the first probability value P1 corresponding to the havepaper category and the second probability value P0 corresponding to the nohavepaper category; compare the magnitudes of P1 and P0, and determine the label corresponding to the larger probability value as the final cardboard status label and output it.

5. The method for monitoring the cardboard condition of a wooden pallet in a papermaking wet pulp packaging machine according to claim 4, characterized in that, The corresponding control signal is written to the programmable controller via TCP protocol based on the cardboard status label. The programmable logic controller (PLC) drives a switching output based on the received signal to coordinate with the packaging machine system, including: In a programmable controller, the address of a status flag register representing the presence or absence of paper is predefined and mapped to a physical output point. Convert the cardboard status label into the corresponding logical value; assemble a write instruction message containing the status flag register address and logical value according to the industrial communication protocol format supported by the programmable controller, and send it through the TCP / IP network; After successfully receiving and executing the write instruction, the programmable controller updates the state of its internal registers and drives the corresponding switch output points accordingly, thereby achieving linkage with the main control system of the packaging machine.

6. The method for monitoring the cardboard condition of a wooden pallet in a papermaking wet pulp packaging machine according to claim 5, characterized in that, The programmable controller adopts the S7-200Smart series PLC, and the industrial communication protocol is ISO-on-TCP & S7 protocol; the write instruction message includes a TPKT header, a COTP layer, an S7 communication header, and a parameter data block containing the target address and data.

7. The method for monitoring the cardboard condition of a wooden pallet in a papermaking wet pulp packaging machine according to claim 4, characterized in that, During the operation of the papermaking wet pulp packaging production line, real-time images of the pallets are acquired through a network camera and processed to generate real-time standardized pre-processed images. Loading a CNN model to infer the real-time standardized preprocessed image and output cardboard status labels, also includes: When the larger of the first probability value P1 and the second probability value P0 is lower than the preset confidence threshold, the confidence of this inference is determined to be insufficient. The system automatically adjusts at least one image acquisition parameter of the network camera, and after adjustment, re-acquires and processes the real-time image of the tray to generate a real-time standardized pre-processed image. It then loads a CNN model for inference and outputs a cardboard status label.

8. A paperboard condition monitoring system for wooden pallets in a papermaking wet pulp packaging machine, characterized in that, The system employs the paperboard condition monitoring method for wooden pallets of a papermaking wet pulp packaging machine as described in any one of claims 1 to 7; The system includes: The image acquisition and processing module is used to acquire the original image of the pallet using a network camera on the paper wet pulp packaging production line via the RTSP streaming media protocol, and to perform standardized preprocessing on the original image to generate a standardized preprocessed image. The model training module is used to acquire multiple standardized preprocessed images of the tray with and without paper under different ambient lighting conditions, construct a training image set and label the cardboard state, and use the labeled training image set to train a convolutional neural network model to generate a trained CNN model. The real-time inference module is used to acquire and process real-time images of the pallets via a webcam during the operation of the paper wet pulp packaging production line, generating real-time standardized pre-processed images; loading a CNN model to infer the real-time standardized pre-processed images and output cardboard status labels; The linkage control module is used to write corresponding control signals to the programmable controller via TCP protocol according to the cardboard status label; the programmable controller drives the switch output according to the received signals to link the packaging machine system.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the paperboard condition monitoring method for wooden pallets of a papermaking wet pulp packaging machine as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the paperboard condition monitoring method for wooden pallets of a papermaking wet pulp packaging machine as described in any one of claims 1 to 7.

Images