Carton posture correction and packing collaborative control method and system based on visual feedback

Abstract

The application provides a carton posture correction and boxing collaborative control method and system based on visual feedback, comprising: constructing a hardware architecture comprising a twisting conveying device, a visual detection device, a correction execution device and a boxing manipulator, designing a four-layer software system architecture to realize closed-loop control; identifying the carton corner point coordinates through a deep learning model, calculating the posture parameters, dividing multiple levels of states according to the verticality deviation angle and the horizontal offset, and executing corresponding response strategies, including the compensation of the grabbing point, the start of the correction device and the abnormal handling. The system realizes torque compensation by differential suction control, and adjusts the grabbing strategy in real time through sensors; at the same time, the adaptive control technology based on the polynomial-Fourier-spline hybrid optimizes the system operation parameters, and continuously improves by recording and analyzing system data. The application solves the posture deviation problem in the twisting process of the carton, and improves the boxing efficiency and reliability.

Description

Technical Field

[0001] This invention relates to the field of packaging automation, and in particular to a visual feedback-based paper box posture correction and packing collaborative control method and system, which is used to solve the posture deviation problem generated by the paper box during the twisting and standing process, and improve the efficiency and reliability of the subsequent packing process. Background Technology

[0002] In the modern packaging industry, the automation level of the cardboard box packaging process is constantly improving, especially in industries such as food, pharmaceuticals, and daily chemicals. Highly efficient and stable cardboard box forming, conveying, and packing systems have a decisive impact on production efficiency. The process of the cardboard box entering the packing station after being twisted and upright is a crucial step in the packaging production line; the stability of this step directly affects subsequent packing efficiency and product quality.

[0003] Traditional cardboard box torsion conveyor systems typically combine mechanical limit guides with belt conveyors, such as the common combination of a 90-degree deflector and a vertical conveyor belt. These systems use fixed mechanical structures to transform a horizontally lying cardboard box into an upright position, and adjust the conveyor belt speed and limiter positions to accommodate different box sizes. Another common solution uses a photoelectric switch array to perform simple detection of the cardboard box position; when a deviation is detected, it triggers a stop or alarm.

[0004] Currently, the more advanced carton twisting and conveying systems mainly rely on mechanical guiding and limiting devices to control the carton's posture. This involves adjusting parameters such as the distance between the guide plates on both sides and the angle of the limiting baffles to accommodate different carton sizes. In the packing process, robotic arms typically employ a fixed-position gripping strategy, with pre-set gripping point coordinates, relying on the high repeatability of the carton's position to ensure a high success rate.

[0005] However, this type of passive mechanical control method has obvious technical defects: First, the cardboard box is prone to tilting and displacement during the twisting and standing process, especially for lightweight or smooth-surfaced cardboard boxes, resulting in a high failure rate of gripping in the downstream packing process; Second, the traditional system's response to abnormal cardboard box posture is too simple, either continuing production and causing quality problems, or shutting down the entire line and affecting efficiency; In addition, cardboard boxes of different specifications and materials require repeated manual adjustment of equipment parameters, which depends on the operator's experience, and the adjustment time is long and the consistency is poor. Summary of the Invention

[0006] The purpose of this invention is to provide a paper box posture correction-packing collaborative control method and system based on visual feedback. By constructing a closed-loop control system that links visual detection with a robotic arm, the invention achieves real-time monitoring, graded correction, and adaptive gripping of paper box posture, solving the packing problem caused by abnormal paper box posture in traditional systems and improving the stability and adaptability of the production line.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] This invention provides a visual feedback-based cardboard box posture correction and packing collaborative control method, comprising:

[0009] A hardware architecture including a torsion conveyor, a vision inspection device, a correction execution device, a packing robot, and a central control system is constructed. A four-layer software system architecture including an equipment layer, a communication layer, an algorithm layer, and an application layer is designed. An information flow diagram from image acquisition to control command issuance is established to obtain a closed-loop control system architecture.

[0010] Based on the closed-loop control system architecture, after acquiring the cardboard box image and completing the calibration, a deep learning model is used to identify the corner coordinates of the cardboard box, calculate the vertical deviation angle and horizontal offset of the cardboard box, and obtain the cardboard box attitude parameters.

[0011] Based on the paper box posture parameters, the system divides the state into multiple levels according to the magnitude of the vertical deviation angle and the horizontal offset, and executes corresponding response strategies, including gripping point compensation, start-up of the correction execution device and abnormal marking processing, to obtain hierarchical linkage control commands.

[0012] Based on the carton posture parameters and the hierarchical linkage control command, the gripping position of the packing robot is calculated and the suction cup force distribution is adjusted to achieve torque compensation. The robot is monitored in real time by the sensor and compensation adjustment is performed to obtain the robot's adaptive gripping control command.

[0013] Based on the closed-loop control system architecture, the system operating parameters and abnormal data are recorded. The attitude anomaly rate in the system operating parameters and the abnormal data is extracted. A correlation model between the system operating parameters and the attitude anomaly rate is established. The optimal parameter combination is searched through an optimization algorithm and the equipment parameters are automatically adjusted to obtain the parameter optimization result.

[0014] Furthermore, the hardware architecture includes a torsion conveying device, a vision inspection device, a correction execution device, a packing robot, and a central control system. A four-layer software system architecture, comprising a device layer, a communication layer, an algorithm layer, and an application layer, is designed. An information flow diagram from image acquisition to control command issuance is established, resulting in a closed-loop control system architecture, including:

[0015] Based on the existing cardboard box twisting conveyor line and packing station, an industrial camera, a pneumatic push rod correction device, a controller, and a human-machine interface are added to construct a hardware architecture that includes the twisting conveyor, the vision inspection device, the correction execution device, the packing robot, and the central control system, thus obtaining the physical implementation basis of the system.

[0016] Based on the physical implementation of the system, a four-layer software system architecture is designed, including a device layer driver, a communication layer OPC UA protocol data interaction network, an algorithm layer attitude recognition algorithm and response strategy decision algorithm, and an application layer operation interface and data analysis platform, to obtain software function support.

[0017] Based on the physical implementation of the system and the software functionality, the industrial camera captures images of the cardboard box and transmits them to the attitude recognition algorithm in the algorithm layer to obtain attitude data. The attitude data is then transmitted to the response strategy decision algorithm in the algorithm layer via the OPC UA protocol data interaction network in the communication layer to obtain control commands. The control commands are then sent to the correction execution device and the packing robot in the device layer to complete the execution actions and obtain execution data. Finally, the data analysis platform in the application layer collects the execution data to optimize the parameters, resulting in the closed-loop control system architecture.

[0018] Furthermore, the step of acquiring the cardboard box image and completing the calibration based on the closed-loop control system architecture includes:

[0019] Based on the vision inspection device in the closed-loop control system architecture, an industrial camera with a resolution of 2048×1536 is installed above the conveyor belt outlet of the torsional conveyor, equipped with a polarizing filter to eliminate reflective interference, and a ring LED is used to provide uniform illumination, thus obtaining the vision inspection hardware configuration.

[0020] Based on the aforementioned visual inspection hardware configuration, the camera's intrinsic and extrinsic parameters are calibrated using the checkerboard calibration method, establishing a mapping relationship between pixel coordinates and actual physical coordinates to obtain the calibrated camera system.

[0021] Based on the calibrated camera system, when the cardboard box is conveyed to the detection area by the conveyor belt of the twisting conveyor, the industrial camera captures an image of the cardboard box and transmits it to the algorithm layer in the closed-loop control system architecture to obtain the image of the cardboard box to be identified.

[0022] Furthermore, the step of using a deep learning model to identify the corner coordinates of the cardboard box includes:

[0023] Based on the algorithm layer in the closed-loop control system architecture, images of cardboard boxes of various specifications and materials in different states are collected, and the coordinates of the four corner points are marked as key feature points. The dataset is expanded to a preset number of samples by methods such as rotation, scaling, brightness adjustment, and adding noise to obtain the training dataset.

[0024] Based on the training dataset, the output layer of the YOLOv8-Pose model was modified to adapt to the requirements of cardboard box four-corner point detection. MobileNetV3 was used as the backbone network, and the model parameters of the YOLOv8-Pose model were initialized using the transfer learning method. Then, the model was trained iteratively using the Adam optimizer with an adaptive learning rate to obtain a lightweight pose detection model with a recognition accuracy greater than 98% and a single frame processing time of less than 50ms.

[0025] Furthermore, based on the paper box posture parameters, the system divides the paper box into multiple states according to the magnitude of the vertical deviation angle and the horizontal offset, and executes corresponding response strategies, including gripping point compensation, activation of the correction execution device, and anomaly marking processing, to obtain hierarchical linkage control commands, including:

[0026] Based on the carton posture parameters, when the vertical deviation angle is detected to be less than the first threshold, the compensation amount for the packing robot's gripping position is calculated, wherein the compensation amount is related to the vertical deviation angle and the carton height, and is transmitted to the robot controller in real time through the communication layer using a communication protocol to obtain the first-level control command;

[0027] Based on the carton posture parameters, when the vertical deviation angle is detected to be between the first threshold and the second threshold, the thrust application point height and push rod extension time are calculated based on the first-level control command. The thrust application point height is related to the vertical deviation angle and the carton height, and the push rod extension time is related to the vertical deviation angle and the conveyor belt speed. The thrust magnitude is adjusted through the equipment layer control signal to obtain the second-level control command.

[0028] According to the paper box posture parameters, when the vertical deviation angle is detected to be greater than the second threshold or when multiple corrections fail, based on the first-level control command and the second-level control command, the central control system controls the conveyor belt of the twisting conveyor to decelerate, the controller of the central control system controls the alarm light to light up and emits an audible prompt, the abnormal location and image are displayed in real time on the human-machine interface, the abnormality type and frequency are automatically recorded, and a third-level control command is obtained.

[0029] The hierarchical linkage control command is obtained based on the first-level control command, the second-level control command, and the third-level control command.

[0030] Furthermore, the execution of the corresponding response strategy also includes security response control based on a random polynomial time delay, including:

[0031] Based on the verticality deviation angle in the paper box posture parameters, a time delay function containing a deterministic polynomial part and a controlled random disturbance term is designed, wherein the controlled random disturbance term follows a normal distribution, thus obtaining a delay control model;

[0032] Based on the aforementioned delay control model, a dynamic delay is added to the calculation of the compensation amount of the packing robot's gripping position in the first-level control command, so that the compensation action is delayed. The delay time changes nonlinearly with the verticality deviation angle, resulting in a first-level response command with random delay.

[0033] Based on the delay control model and the first-level response command, the push rod extension time in the second-level control command is adjusted, and the delay amount related to the verticality deviation angle is increased to obtain a second-level response command with random delay;

[0034] Based on the vertical deviation angle and horizontal offset in the paper box posture parameters, a polynomial boundary function is introduced to dynamically adjust the first threshold and the second threshold, and a safety status assessment model is developed. When the safety status assessment value is lower than the safety threshold, the model parameters of the delay control model are automatically adjusted. Based on the first-level response command, the second-level response command, and the third-level control command, a safety-enhanced hierarchical linkage control command is obtained.

[0035] Furthermore, the method of using a deep learning model to identify the corner coordinates of the cardboard box also includes robust pose recognition with residual noise enhancement, comprising:

[0036] Based on the algorithm layer in the closed-loop control system architecture, a residual noise linear layer is inserted into the feature extraction network of the deep learning model. The residual noise linear layer enhances the feature expression capability by superimposing a noise function on the linear transformation result, thus obtaining a feature extraction network with noise enhancement capability.

[0037] Based on the feature extraction network with noise enhancement capability, a strategy for dynamically adjusting noise intensity with training progress is designed. When the number of training iterations is less than 30% of the total number of iterations, high-intensity noise with a noise standard deviation of 0.1 is used; when the number of training iterations is greater than 70% of the total number of iterations, low-intensity noise with a noise standard deviation of 0.01 is used; and when the number of training iterations is between 30% and 70% of the total number of iterations, a linearly decaying noise standard deviation is used, thus obtaining an adaptive noise scheduling mechanism.

[0038] The model is trained based on the aforementioned adaptive noise scheduling mechanism to obtain a lightweight attitude detection model with enhanced robustness.

[0039] Furthermore, the model training based on the adaptive noise scheduling mechanism further includes:

[0040] Based on the adaptive noise scheduling mechanism, the corner detection loss function of the deep learning model is improved into a weighted combined loss function that includes the original loss function and a smooth loss function that introduces Gaussian blur, thus obtaining the smooth loss function.

[0041] Based on the smoothing loss function, the model parameters of the deep learning model are updated using an exponential moving average method, so that the model parameters at the current time are determined by a weighted combination of the parameters at the previous time and the optimized parameters of the current batch, thus obtaining the soft target network update mechanism.

[0042] Model training is completed based on the soft target network update mechanism. A noise perception evaluation index is developed to measure the performance difference of the model under different noise levels. The noise injection strategy in the adaptive noise scheduling mechanism is dynamically adjusted to obtain the lightweight attitude detection model with enhanced robustness.

[0043] Furthermore, the method of recording system operating parameters and abnormal data based on the closed-loop control system architecture, extracting the attitude anomaly rate from the system operating parameters and the abnormal data, establishing a correlation model between the system operating parameters and the attitude anomaly rate, searching for the optimal parameter combination through an optimization algorithm, and automatically adjusting the equipment parameters also includes parameter adaptive control based on a polynomial-Fourier-spline hybrid approach, including:

[0044] Based on the closed-loop control system architecture, a pseudo-linear input-output model of the carton torsion conveying system is established. The pseudo-linear input-output model represents the output attitude anomaly rate as the product of the feature matrix constructed based on historical data and the parameter vector to be estimated, plus the error term, to obtain the pseudo-linear control model.

[0045] Based on the pseudolinear control model, a hybrid basis function space containing polynomial basis functions, Fourier basis functions, and cubic spline basis functions is constructed. The torsional conveying speed, limit rod spacing, guide plate angle, carton height, carton width, carton thickness, and carton material type in the system operating parameters are mapped to the hybrid basis function space to obtain the feature matrix.

[0046] Based on the feature matrix and the parameter vector to be estimated in the pseudo-linear control model, the recursive least squares algorithm with a forgetting factor is used to update the parameter vector in real time to obtain adaptive parameter estimates.

[0047] Based on the adaptive parameter estimation and the pseudo-linear control model, the observables of the system are defined to represent the anomaly rate and the trigger frequency of each level of response per unit time. The observables of the system under different parameter combinations are predicted. An adaptive control law is designed so that the adjustment amount of the torsional conveying speed, the limit rod spacing and the guide plate angle are gradient-dependent with the difference between the current value and the target value of the observables of the system. The torsional conveying parameters are adjusted in real time through the equipment layer to obtain the parameter optimization results.

[0048] Further, calculating the gripping position of the packing robot based on the carton posture parameters and the hierarchical linkage control command includes:

[0049] Based on the visual inspection device and the packing robot in the closed-loop control system architecture, the transformation relationship between the camera coordinate system and the robot coordinate system is established through hand-eye calibration technology. The suction force of the suction cup and the reading of the vacuum pressure sensor are calibrated, a suction force estimation model is established, and the calibrated robot gripping system is obtained.

[0050] Based on the calibrated robotic gripping system and the cardboard box posture parameters, a center of gravity gripping strategy is used to calculate the center of gravity position of the cardboard box. The center point of the suction cup is aligned with the center of gravity position of the cardboard box, and the distance between the two suction cups is automatically adjusted according to the height and width of the cardboard box to obtain the optimal gripping position planning.

[0051] Furthermore, the adjustment of the suction cup force distribution to achieve torque compensation includes:

[0052] Based on the cardboard box posture parameters and the dynamic adjustment command for the gripping point in the graded linkage control command, for cardboard boxes whose vertical deviation angle is less than the first threshold, a dual suction cup differential suction force control is designed. The suction force of the tilting side suction cup is set as the product of the standard suction force and the first correlation coefficient of the vertical deviation angle, and the suction force of the stable side suction cup is set as the product of the standard suction force and the second correlation coefficient of the vertical deviation angle. The standard suction force is pre-calibrated and determined according to the weight and material type of the cardboard box. The vacuum pressure of each suction cup is adjusted in real time through a proportional adjustment valve and a PID control algorithm to obtain a torque compensation control command.

[0053] According to the torque compensation control command, the adsorption state of the dual suction cups is monitored in real time by a vacuum pressure sensor. When the vacuum pressure difference between the tilted suction cup and the stable suction cup exceeds a preset pressure threshold or the adsorption stability evaluation index is lower than the stability threshold, compensation control is activated. The pitch angle compensation amount of the packing robot is adjusted by the servo motor of the packing robot, or the vacuum pressure increment of the tilted suction cup is increased by the proportional adjustment valve to obtain a dynamic compensation adjustment command.

[0054] Based on the dynamic compensation adjustment command, the grasping process data is recorded through the closed-loop control system architecture. The grasping process data includes the vertical deviation angle, the vacuum pressure of the tilt side suction cup, the vacuum pressure of the stable side suction cup, the pitch angle compensation amount, the vacuum pressure increment, and the grasping success rate. The grasping process data is analyzed through the data analysis platform of the application layer to optimize the first correlation coefficient and the second correlation coefficient, thereby obtaining the adaptive grasping control command of the robot arm.

[0055] This invention also provides a visual feedback-based cardboard box posture correction-packing collaborative control system, comprising:

[0056] The system architecture building unit is used to construct the hardware architecture including a torsion conveyor, a vision inspection device, a correction execution device, a packing robot and a central control system. It designs a four-layer software system architecture including an equipment layer, a communication layer, an algorithm layer and an application layer, and establishes an information flow diagram from image acquisition to control command issuance to obtain a closed-loop control system architecture.

[0057] The paper box posture recognition unit is used to acquire paper box images and complete calibration based on the closed-loop control system architecture, and then use a deep learning model to identify the corner coordinates of the paper box, calculate the vertical deviation angle and horizontal offset of the paper box, and obtain the paper box posture parameters.

[0058] The hierarchical linkage control unit is used to divide the paper box posture parameters into multiple states according to the magnitude of the vertical deviation angle and the horizontal offset, and execute the corresponding response strategies, including gripping point compensation, start-up of the correction execution device and abnormal marking processing, to obtain hierarchical linkage control commands.

[0059] The adaptive gripping unit of the robotic arm is used to calculate the gripping position of the packing robot and adjust the suction force distribution of the suction cup to achieve torque compensation based on the posture parameters of the carton and the hierarchical linkage control command. The unit monitors and compensates for the torque compensation in real time through sensors to obtain the adaptive gripping control command of the robotic arm.

[0060] The parameter optimization unit is used to record system operating parameters and abnormal data based on the closed-loop control system architecture, extract the attitude anomaly rate from the system operating parameters and the abnormal data, establish a correlation model between the system operating parameters and the attitude anomaly rate, search for the optimal parameter combination through an optimization algorithm, and automatically adjust the equipment parameters to obtain the parameter optimization result.

[0061] The beneficial effects of this invention are:

[0062] 1. By constructing a closed-loop control system architecture that links visual inspection and robotic arms, the originally independent paper box twisting and conveying and packing processes are integrated into a collaborative system, realizing intelligent control of the entire process and improving the overall collaborative efficiency of the system;

[0063] 2. A lightweight cardboard box posture recognition technology based on deep learning is adopted. By detecting the coordinates of four corner points, the vertical deviation angle and horizontal offset are calculated to achieve quantitative and high-precision detection of the cardboard box posture, providing accurate data support for subsequent control.

[0064] 3. A hierarchical linkage control method with a three-level response strategy is designed. Different response measures are taken according to the degree of deviation of the carton posture, which avoids unnecessary downtime, ensures the packing quality, and improves the stability and continuity of the production line.

[0065] 4. Implement a force-position hybrid control method for adaptive gripping of the robotic arm, simultaneously adjusting the gripping position and the suction force distribution of the dual suction cups to form a "torque compensation" mechanism, preventing the cardboard box from tipping over again during the gripping process and improving the gripping success rate;

[0066] 5. Develop a self-learning optimization method for torsional transport parameters based on a multiple regression model. By analyzing the relationship between the anomaly rate and each parameter, the method automatically recommends the optimal parameter combination, thereby shifting from "manual debugging" to "data-driven" and improving system adaptability. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A flowchart illustrating the visual feedback-based paper box posture correction and packing collaborative control method provided for this invention;

[0069] Figure 2 This invention provides a schematic diagram of the structure of a cardboard box posture correction-packing collaborative control system based on visual feedback. Detailed Implementation

[0070] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0071] Example 1:

[0072] like Figure 1 As shown, this invention provides a visual feedback-based cardboard box posture correction and packing collaborative control method, comprising:

[0073] Step S1: Construct a hardware architecture that includes a torsion conveyor, a vision inspection device, a correction execution device, a packing robot, and a central control system. Design a four-layer software system architecture that includes an equipment layer, a communication layer, an algorithm layer, and an application layer. Establish an information flow diagram from image acquisition to control command issuance to obtain a closed-loop control system architecture.

[0074] Step S2: Based on the closed-loop control system architecture, after acquiring the cardboard box image and completing the calibration, a deep learning model is used to identify the corner coordinates of the cardboard box, calculate the vertical deviation angle and horizontal offset of the cardboard box, and obtain the cardboard box attitude parameters.

[0075] Step S3: Based on the paper box posture parameters, divide the state into multiple levels according to the magnitude of the vertical deviation angle and the horizontal offset, and execute the corresponding response strategy, including gripping point compensation, start of the correction execution device and abnormal marking processing, to obtain hierarchical linkage control instructions;

[0076] Step S4: Calculate the gripping position of the packing robot based on the carton posture parameters and the graded linkage control command, and adjust the suction force distribution of the suction cup to achieve torque compensation. Monitor and compensate in real time through sensors to obtain the robot's adaptive gripping control command.

[0077] Step S5: Based on the closed-loop control system architecture, record the system operating parameters and abnormal data, extract the attitude anomaly rate from the system operating parameters and the abnormal data, establish a correlation model between the system operating parameters and the attitude anomaly rate, search for the optimal parameter combination through an optimization algorithm and automatically adjust the equipment parameters to obtain the parameter optimization result.

[0078] Specifically, firstly, in the hardware architecture construction process, the arrangement of the twisting conveyor device is completed. This device mainly consists of a turning conveyor belt, guide plates, and limiting rods, used to transform the carton from a lying position to an upright position. The turning conveyor belt adopts a segmented design: the front section is a horizontal conveying section, the middle section is a gradually changing turning section, and the rear section is a vertical conveying section. The speed of each section is controlled in coordination by a frequency converter. The guide plates are installed on both sides of the conveyor belt in a V-shape, and the angle is adjustable, used to guide the carton to complete a 90-degree turn. The limiting rods are located at the end of the conveyor belt, and their spacing is adjustable, used to prevent the carton from tipping over excessively.

[0079] Secondly, a vision inspection device is installed, consisting of an industrial camera, a lighting system, and an image acquisition card. The industrial camera is a model with a resolution of 2048×1536 and a frame rate of no less than 30fps, equipped with a 16mm fixed-focus lens, and mounted approximately 60-80cm above the exit of the torsion conveyor, covering the entire inspection area. The lighting system uses a ring-shaped LED light source to provide uniform illumination and is equipped with a polarizing filter to effectively eliminate glare interference from the cardboard box surface. The image acquisition card connects to the central control system via a gigabit network, supporting real-time image transmission and preprocessing.

[0080] Next, the correction actuator is installed, mainly consisting of a pneumatic pusher array, an air circuit control system, and a support frame. The pneumatic pusher array contains 3-5 vertically arranged pushers, with an installation height covering 30%-80% of the carton height range. Each pusher is controlled by an independent proportional valve, allowing adjustment of the thrust (50-200N adjustable) and action time (50-300ms adjustable). The air circuit control system consists of an air source processing unit, a solenoid valve assembly, and a proportional pressure valve, providing stable air pressure input and achieving precise control. The support frame, made of aluminum profile, is installed on the side of the conveyor belt, and its position and height are adjustable to ensure the pushers accurately act on the side of the carton.

[0081] Next, the packing robot system is configured, including a six-axis robotic arm, a dual-suction cup end effector, and a controller. The six-axis robotic arm is selected with a load capacity of at least 10kg, and its working range covers the entire area from the conveyor belt exit to the packing position. The dual-suction cup end effector consists of two independently controlled vacuum suction cups, each with a diameter of 60-80mm and an adjustable spacing of 100-300mm to accommodate different sized cartons. The suction cups are controlled by independent proportional vacuum generators, and each suction cup is equipped with a vacuum pressure sensor to monitor the adsorption status in real time. The robot controller adopts an open architecture, supports multiple communication interfaces, and can receive external position compensation commands and adjust gripping parameters in real time.

[0082] Finally, a central control system is built, consisting of an industrial computer, a real-time controller, and a human-machine interface (HMI). The industrial computer uses a high-performance processor (such as an Intel i7 or equivalent), equipped with at least 16GB of RAM and a 500GB SSD, and runs deep learning frameworks and image processing software. The real-time controller uses a high-performance PLC or motion controller, with multiple I / O interfaces and high-speed communication capabilities. The HMI includes a 22-inch or larger touchscreen display and an industrial keyboard, providing an intuitive operating interface and real-time status display.

[0083] In the software system architecture design, the first step is to build the device layer, which includes drivers and low-level control programs for each hardware device. The camera driver is responsible for image acquisition and basic image processing; the pneumatic component driver controls the extension and retraction of the push rod and the adjustment of thrust; the robot arm driver implements the motion control and path planning of the robotic arm; and the conveyor belt driver controls the motor speed and synchronization. The device layer programs are developed using C++ or custom firmware and run on their respective controllers to ensure real-time responsiveness.

[0084] Secondly, a communication layer is established, employing the OPC UA protocol to construct a data interaction network between devices. The OPC UA server runs on the central control system, providing a unified data access interface; each device acts as an OPC UA client, connecting to the server via Ethernet. The communication layer implements a data publish / subscribe mechanism, supporting both periodic data transmission and event-triggered transmission modes, ensuring that critical data (such as paper tray posture parameters and control commands) can be reliably transmitted within the system in real time. The latency of the communication layer is controlled within 10ms, meeting the real-time control requirements of the system.

[0085] Next, the algorithm layer was developed, including a pose recognition algorithm and a response policy decision algorithm. The pose recognition algorithm, developed based on a deep learning framework and running on an industrial computer, is responsible for processing images and extracting the cardboard box's pose parameters. The response policy decision algorithm uses a hybrid rule-based and model-based approach to generate hierarchical control commands based on the pose parameters. The algorithm layer programs were developed using Python or C++, utilizing acceleration technologies such as CUDA or OpenVINO to ensure computational efficiency, with a single frame processing time controlled within 50ms.

[0086] Finally, the application layer is constructed, including the user interface and the data analysis platform. The user interface adopts a web-based responsive design, displaying real-time system status, cardboard box images, and key parameters, supporting parameter adjustment and manual control functions. The data analysis platform integrates data storage, statistical analysis, and visualization functions, recording system operating data and generating reports to support optimization decisions. The application layer is developed using technologies such as JavaScript and React and deployed on the industrial computer of the central control system.

[0087] Through the integration of the aforementioned hardware and software systems, a complete information flow diagram is established. The information flow begins with image acquisition by the vision inspection device, which processes the data through the algorithm layer to generate carton posture parameters. The decision algorithm then generates control commands, which are distributed to each execution device (correction execution device and packing robot) via the communication layer. The execution devices complete the corresponding actions and return to the execution status, forming a complete closed-loop control system. The total latency of acquisition-processing-execution-feedback is controlled within 200ms, ensuring that the system can respond promptly to changes in carton posture.

[0088] Secondly, the image acquisition process begins with a photoelectric sensor detecting the cardboard box entering the field of view, triggering the industrial camera to capture the image. The camera employs a region scanning mode with an exposure time set to 1-5ms, complemented by ring LED illumination to provide stable imaging conditions. To reduce ambient light interference, a polarizing filter is installed between the camera and the light source, along with a light shield to ensure stable image quality. The image is transmitted to the image processing unit at a resolution of 1920×1440 via a gigabit network, with transmission latency controlled within 5ms.

[0089] Camera calibration employs Zhang's calibration method, using a 9×6 standard checkerboard calibration board, and at least 20 calibration images are captured from different positions and angles. The calibration process calculates the camera's intrinsic parameter matrix (including focal length, principal point coordinates, and radial / tangential distortion coefficients) and extrinsic parameter matrix (camera position and attitude relative to the world coordinate system). The reprojection error of the calibration results is controlled within 0.5 pixels to ensure measurement accuracy. The system also establishes a transformation relationship between the camera coordinate system and the conveyor belt coordinate system, enabling accurate conversion of pixel coordinates in the images to actual physical coordinates.

[0090] Image preprocessing includes three steps: distortion correction, noise removal, and contrast enhancement. Distortion correction, based on the calibrated intrinsic parameter matrix, eliminates radial and tangential distortions caused by the lens. Noise removal employs a combination of Gaussian filtering and median filtering to reduce image noise while preserving edge information. Contrast enhancement is achieved through adaptive histogram equalization, enhancing the contrast between the cardboard box edges and the background, providing clearer features for corner detection.

[0091] The deep learning model training process begins with constructing a training dataset. Images of cardboard boxes of various sizes (height 200-500mm, width 150-350mm) and materials (corrugated paper, coated paper, white cardboard) in different poses are collected, with at least 100 original images for each combination. The coordinates of the four corner points of the cardboard box in each image are labeled using professional annotation tools. Data augmentation techniques (rotation ±15°, scaling 80%-120%, brightness adjustment ±30%, adding noise, etc.) are used to expand the dataset to over 10,000 samples. The dataset is then divided into training, validation, and test sets in an 8:1:1 ratio.

[0092] The model architecture utilizes an improved YOLOv8-Pose network, replacing the original human keypoint detection with cardboard box corner detection. The backbone network employs the lightweight MobileNetV3, reducing computational cost while maintaining feature extraction capabilities. To enhance model robustness, a residual noise layer is added to the feature extraction network, injecting controlled noise during training to improve the model's resistance to interference. An improved smoothing L1 loss function is used to reduce the impact of annotation errors on training.

[0093] The model training employed a transfer learning strategy, initializing network parameters using weights pre-trained on the COCO dataset. The Adam optimizer was used during training, with an initial learning rate of 0.001, dynamically adjusted using a cosine annealing scheduling strategy. Batch size was set to 32, training lasted 100 epochs, and early stopping was employed to avoid overfitting. Training was performed on a workstation equipped with a GPU, with each training session taking approximately 8 hours. After training, the model achieved a corner detection accuracy of 98.5% on the test set (corner position error <5 pixels was considered a correct detection), with a single frame processing time of less than 40ms.

[0094] The cardboard box attitude parameters are calculated based on the coordinates of the four corner points identified by a deep learning model. First, the pixel coordinates are converted to physical coordinates (in millimeters). Then, key parameters are calculated: the vertical deviation angle θ is calculated as the angle between the line connecting the midpoints of the left and right edges of the cardboard box and the vertical line; a positive value indicates a rightward tilt, and a negative value indicates a leftward tilt. The horizontal offset Δx is the lateral distance of the cardboard box's center point relative to the conveyor belt's center line; a positive value indicates a rightward offset, and a negative value indicates a leftward offset. The cardboard box's height H and width W are also calculated as auxiliary parameters for subsequent control strategy adjustments. The attitude parameter calculation uses a geometric method, achieving an accuracy better than 0.1° (angle) and 0.5mm (displacement).

[0095] To improve system reliability, temporal filtering of attitude parameters and anomaly detection were also implemented. Temporal filtering uses the Kalman filter method, which comprehensively considers the detection results of multiple consecutive frames, smooths parameter fluctuations, and suppresses occasional false detections. Anomaly detection identifies possible erroneous detections by abrupt changes in detection parameters and physical constraints (such as the reasonable range of cardboard box size). When an anomaly is detected, the valid result of the previous frame is temporarily used or a re-detection is triggered.

[0096] Third, a three-level response strategy is adopted, with different measures taken based on the severity of the cardboard box's posture. The thresholds for state classification were determined through extensive experimental data, while also considering the influence of cardboard box dimensions on the thresholds. Under standard conditions, the first threshold is set at 5°, and the second threshold is set at 15°. These thresholds can be dynamically adjusted according to the cardboard box's height-to-width ratio. For example, for slender cardboard boxes with a height / width ratio greater than 2, the first threshold can be reduced to 3° to improve system sensitivity.

[0097] For slight tilting (vertical deviation angle θ < first threshold), the first-level response—grip point compensation—is executed. The compensation amount for the packing robot's gripping position is calculated. This compensation amount is related to the deviation angle and the carton height, and the calculation formula is ΔX = H·sin(θ)·k1, where H is the carton height and k1 is the compensation coefficient (typically 0.6-0.8). This compensation allows the robot to accurately align with the actual center of gravity of the tilted carton, rather than its geometric center, reducing unbalanced torque during gripping. The system transmits the compensation parameters to the robot controller in real time via the OPC UA protocol, with a delay of less than 15ms, ensuring the accuracy of the gripping action.

[0098] For moderate tilt (first threshold ≤ θ < second threshold), a second-level response is executed—the correction actuator is activated while retaining the gripping point compensation. First, the thrust application point height h and the push rod extension time t are calculated. The thrust application point height is related to the carton height and the deviation angle, and is typically set as h = H·(0.5 + 0.1·θ / θ1), where θ1 is the first threshold. The push rod extension time is related to the deviation angle and the conveyor belt speed, and is typically set as t = t0·(1 + 0.3·(θ-θ1) / (θ2-θ1)), where t0 is the basic push rod time (typically 100ms), and θ2 is the second threshold. The pneumatic push rod at the corresponding height is activated via a PLC control signal, and the extension time and thrust magnitude are precisely controlled. The thrust magnitude is controlled by a proportional valve, proportional to the deviation angle, and ranges from 50-200N. After the correction is completed, a second test is immediately performed to evaluate the correction effect. If the deviation angle decreases to below the first threshold, it is downgraded to the first level response; if it is still above the first threshold, a second correction will be attempted, with a maximum of three consecutive attempts.

[0099] For severe tilting (θ ≥ second threshold) or three consecutive failed corrections, a third-level response—anomaly marking—is implemented. In this case, the conveyor belt of the twisting conveyor is first decelerated to 40% of its normal speed via a frequency converter to reduce the generation of new abnormal cartons and buy time for manual intervention. An alarm light flashes (1Hz) and emits an intermittent audible alert (approximately 80 dB) to attract the operator's attention. An anomaly alert window pops up on the human-machine interface, displaying real-time images and carton position information, along with brief handling suggestions. Detailed information about the anomaly is automatically recorded, including a timestamp, carton parameters (specifications, material, deviation angle, etc.), attempted corrections, and their effects. This data is stored in a database for subsequent analysis and system optimization.

[0100] To improve system security, a random delay mechanism is introduced into the response strategy. A time delay function containing both deterministic and random components is designed, giving the execution time of the response action a small degree of uncertainty. The delay function has the form T. delay = Tbase ·(1+R·ε), where T base R is the base delay time (related to the deviation angle), R is the control parameter (usually 0.1-0.2), and ε is a random variable following a standard normal distribution. This design increases the unpredictability of system behavior, improves the system's resistance to external disturbances, and, because the random delay amplitude is small, does not significantly affect the normal function and efficiency of the system.

[0101] The hierarchical linkage control commands are comprehensive control signals generated by the system based on a three-level response strategy, comprising multiple command sets. The first-level command set includes parameters for manipulator gripping position compensation, such as target position offset and gripping posture adjustment parameters. The second-level command set includes control parameters for the correction actuator, such as push rod selection, thrust magnitude, and action time. The third-level command set includes anomaly handling parameters, such as conveyor belt deceleration ratio, alarm type, and anomaly recording commands. These commands are distributed to each execution unit via a unified control bus, ensuring the coordination and timeliness of the system response. Command priority management is also implemented, ensuring that higher-level commands can promptly interrupt and override the execution of lower-level commands, guaranteeing system safety in abnormal situations.

[0102] Fourth, the packing robot's gripping system first establishes a mapping relationship between the visual coordinate system and the robot's coordinate system through hand-eye calibration. The calibration process uses a calibration plate marked with a precise checkerboard pattern. The robot moves the calibration plate to multiple different positions (at least eight non-coplanar positions), capturing an image at each position. By calculating the position of the calibration plate in the image and the actual position of the robot, a transformation matrix between the two coordinate systems is established. The transformation accuracy is better than ±0.5mm, ensuring that the position of the cardboard box detected by vision can be accurately converted into the target position in the robot's coordinate system.

[0103] The gripping position is calculated using a center-of-gravity gripping strategy. First, the center-of-gravity position is calculated based on the cardboard box's posture parameters. For a tilted cardboard box, the center-of-gravity position will deviate from the geometric center. The center-of-gravity coordinate calculation considers the effects of the vertical deviation angle and horizontal offset, as well as the height and width parameters of the cardboard box. For a standard (non-tilted) cardboard box, the center-of-gravity position is located at the center of the box; for a tilted cardboard box, the center-of-gravity position will shift in the tilt direction, with the offset amount related to the tilt angle and the height of the cardboard box. The spacing between the two suction cups is also automatically adjusted according to the cardboard box width, typically set to 60% of the box width, ensuring the suction cups are located in a stable area of ​​the cardboard box and avoiding proximity to the edges.

[0104] Suction cup force distribution is a key technology used to achieve torque compensation. First, a standard suction force is determined through testing. This standard suction force is related to the weight, material, and surface condition of the cardboard box and is typically set to 2.5 times the box weight to ensure safe gripping. For tilted cardboard boxes (deviation angle less than a first threshold), a differential suction force control strategy is employed: the suction force of the tilted side suction cup (lower side) is set to P. low = P std ·(1+k1·θ), the suction force of the stable side suction cup (high side) is set to P. high = P std ·(1-k2·θ), where P std This is the standard suction force. k1 and k2 are correlation coefficients, determined experimentally, typically k1=0.05 and k2=0.03. This differential suction force distribution generates a corrective torque, making the cardboard box tend to be vertical during lifting, reducing the risk of tipping. Suction control is achieved through a proportional vacuum generator, with each suction cup controlled by an independent proportional valve. The response time is less than 50ms, and the adjustment accuracy is ±2kPa.

[0105] The sensor monitoring system consists of a vacuum pressure sensor, an adsorption status sensor, and a position sensor. The vacuum pressure sensor is installed in the air path of each suction cup to monitor the vacuum level within the cup in real time, with a sampling frequency of 100Hz and a measurement accuracy of ±0.5kPa. The adsorption status sensor is a specially designed flow detection device that detects the seal between the suction cup and the cardboard box surface. When leakage is detected exceeding a threshold, a poor seal alarm is triggered. The position sensor includes the robot arm's encoder and an additionally installed proximity switch to detect the robot arm's actual position and movement status.

[0106] The compensation and adjustment mechanism is a key function of the system to handle abnormal situations during the data capture process. Stability assessment indicators are calculated periodically. , where σ p It is the standard deviation of pressure. This is the average pressure; p1 and p2 are the pressures of the two suction cups. max This is the maximum permissible pressure difference. When S falls below the threshold (typically 0.8) or a pressure difference between the two suction cups exceeds 10 kPa, compensation adjustment is initiated. Compensation methods include: 1) Robotic arm posture adjustment: by controlling the pitch joint of the robotic arm's end effector, adjusting the angle within ±3° to make the suction cups fit the cardboard box surface more closely; 2) Suction force adjustment: by increasing the vacuum level of the tilted suction cup (increasing it by 10%-30%) or decreasing the vacuum level of the stable side, the suction force distribution is rebalanced. The most suitable compensation strategy is automatically selected based on real-time monitoring data, and stability is checked again after compensation to ensure the reliability of the gripping.

[0107] The adaptive gripping control command for the robotic arm is a comprehensive command generated based on the cardboard box's posture parameters, hierarchical linkage control commands, and sensor feedback. It includes the following core components: 1) Target position parameters, specifying the position and orientation of the robotic arm's end effector; 2) Suction cup control parameters, including the target vacuum level, suction timing, and monitoring thresholds for the two suction cups; 3) Compensation control parameters, specifying the compensation strategy and parameter range when unstable states are detected; and 4) Motion control parameters, including the speed, acceleration, and path planning during gripping and placement. These parameters are transmitted through the robotic arm controller's real-time communication interface, with an update frequency of 100Hz, ensuring that the robotic arm can adjust its gripping strategy in real time according to the current situation.

[0108] Fifth, continuously collect and store key operating parameters, including equipment parameters (torsional conveyor speed v, limit bar spacing d, guide plate angle α, etc.), cardboard box parameters (height H, width W, thickness D, material type m, etc.), and system status data (response trigger frequency at each level, anomaly rate, grasping success rate, etc.). The data collection frequency varies depending on the parameter type; equipment parameters are sampled once per minute, and status data is statistically analyzed by production batch. All data is preprocessed (outlier removal, standardization, etc.) and then stored in a time-series database, retaining at least 30 days of historical records to provide a data foundation for subsequent analysis.

[0109] Attitude anomaly rate is a core indicator for evaluating system performance, defined as the proportion of cardboard boxes exhibiting attitude anomalies (vertical deviation angle exceeding the first threshold) per unit time. The anomaly rate is calculated at both hourly and shift levels, and the distribution of anomalies of different severity (mild, moderate, and severe) is recorded. The success rate of the second-level response (the proportion of corrected deviation angles less than the first threshold) and the trigger rate of the third-level response are also calculated. These indicators collectively reflect the system's stability and adaptability.

[0110] The correlation model is established using a pseudo-linear model approach, representing the output (attitude anomaly rate) as the product of the feature vector and the parameter vector plus an error term. The key lies in the construction of the feature matrix, employing a hybrid basis function space method to map the original input parameters to a high-dimensional feature space through multinomial basis functions, Fourier basis functions, and cubic spline basis functions. For example, for the transport speed v, v0 and v1 are calculated simultaneously. 2 v 3 , sin(ωv), cos(ωv) and max(0, (v-ξ) 3 Features such as (e.g., the distance between limit rods and the angle of the guide plate) are mapped similarly. This hybrid basis function can capture the complex nonlinear relationships between parameters, improving the model's expressive power and prediction accuracy.

[0111] The parameter estimation employs a recursive least squares (RLS) algorithm with a forgetting factor, enabling real-time updates to model parameters as new data arrives, eliminating the need to store and process all historical data. The forgetting factor (typically set to 0.95-0.99) makes the algorithm more attentive to new data, adapting to the dynamic changes in the system. The update process includes three steps: calculating the gain vector, updating the parameter vector, and updating the covariance matrix. An update is performed whenever new data points (input parameters and corresponding anomalies) are collected. This adaptive parameter estimation method allows the model to automatically adjust as the system state changes, improving prediction accuracy.

[0112] The optimization algorithm searches for the optimal parameter combination based on the updated association model. First, the objective function is defined as J(v, d, α) = w1r + w2(f2+3f3) + w3c, where r is the anomaly rate, f2 and f3 are the trigger frequencies of second- and third-level responses respectively, c is the cost of parameter changes, and w1, w2, and w3 are weighting coefficients. The goal is to minimize this function, i.e., to find a combination that results in a low anomaly rate, few high-level response triggers, and minimal parameter changes. The optimization employs a modified gradient descent method, considering both physical constraints on the parameters (such as upper and lower limits for velocity and spacing) and rate-of-change constraints (to prevent drastic parameter changes). The algorithm performs an optimization calculation after each production batch, generating recommended parameter settings for the next batch.

[0113] The automatic parameter adjustment mechanism transforms the output of the optimization algorithm into actual equipment adjustment commands. An adaptive control law is designed, where the adjustment amount is proportional to the gradient of the difference between the current and target values, while also considering rate-of-change constraints. For example, the formula for adjusting the conveyor speed is Δv = -η1·∂J / ∂v·clip(v -v, -Δv max Δv max ), where η1 is the learning rate, v It is the target velocity, Δv max This is the maximum permissible variation. Similarly, the adjustment amounts for the limit rod spacing and guide plate angle are also calculated. These adjustments are automatically achieved through actuators at the equipment level (such as frequency converters, stepper motors, etc.), and operators can also view and confirm these adjustments through a human-machine interface.

[0114] The evaluation of parameter optimization results employs a controlled experimental approach, comparing system performance before and after optimization. Key evaluation metrics include: reduction in anomaly rate (typically 30%-50%), reduction in advanced response trigger frequency (typically 40%-60%), and improvement in production efficiency (typically 5%-15%). The stability of parameter optimization, i.e., whether the parameters converge to relatively stable values, and their adaptability to different cardboard box types are also recorded. These evaluation results are provided to management in report form and used to guide further system improvements.

[0115] In summary, a complete closed-loop control system was constructed. Through visual detection, intelligent decision-making, and adaptive control, the posture deviation problem during the cardboard box twisting and standing process was effectively solved, improving the efficiency and reliability of the packing process. The innovation of this method lies in achieving precise detection of cardboard box posture, a hierarchical linkage response strategy, force-position hybrid adaptive grasping control, and data-driven parameter self-optimization. This gives the system high efficiency, high reliability, and strong adaptability, providing a new technical solution for the field of packaging automation.

[0116] Example 2:

[0117] In this embodiment, the hardware architecture includes a torsion conveying device, a vision inspection device, a correction execution device, a packing robot, and a central control system. A four-layer software system architecture comprising a device layer, a communication layer, an algorithm layer, and an application layer is designed. An information flow diagram from image acquisition to control command issuance is established, resulting in a closed-loop control system architecture, including:

[0118] Based on the existing cardboard box twisting conveyor line and packing station, an industrial camera, a pneumatic push rod correction device, a controller, and a human-machine interface are added to construct a hardware architecture that includes the twisting conveyor, the vision inspection device, the correction execution device, the packing robot, and the central control system, thus obtaining the physical implementation basis of the system.

[0119] Based on the physical implementation of the system, a four-layer software system architecture is designed, including a device layer driver, a communication layer OPC UA protocol data interaction network, an algorithm layer attitude recognition algorithm and response strategy decision algorithm, and an application layer operation interface and data analysis platform, to obtain software function support.

[0120] Based on the physical implementation of the system and the software functionality, the industrial camera captures images of the cardboard box and transmits them to the attitude recognition algorithm in the algorithm layer to obtain attitude data. The attitude data is then transmitted to the response strategy decision algorithm in the algorithm layer via the OPC UA protocol data interaction network in the communication layer to obtain control commands. The control commands are then sent to the correction execution device and the packing robot in the device layer to complete the execution actions and obtain execution data. Finally, the data analysis platform in the application layer collects the execution data to optimize the parameters, resulting in the closed-loop control system architecture.

[0121] Specifically, the first step involves assessing the existing production line equipment, meticulously recording key parameters of the twisting conveyor line, including belt specifications (width typically 400-600mm), drive motor power (typically 1.5-3kW), linear speed range (typically 0.2-0.5m / s), conveyor belt structure (usually divided into horizontal, transition, and vertical sections), and the location and spacing of existing limit devices. Simultaneously, the layout of the packing station, the type and specifications of the robotic arm, the control system model and interface type are assessed to determine compatibility with the new equipment. Based on this assessment, a hardware modification plan is designed, determining the installation location, electrical interfaces, and mechanical interfaces of the new equipment.

[0122] The industrial camera is mounted on an adjustable aluminum alloy bracket, fixed 600-800mm above the conveyor belt exit. The bracket supports three-dimensional angle and height fine-tuning to ensure the camera's field of view completely covers the inspection area. The selected industrial camera features a 2048×1536 pixel resolution, a frame rate of over 30fps, a global shutter, and a C-mount interface. Combined with a 16mm low-distortion industrial lens, it achieves a field of view of approximately 600×450mm. The camera connects to the central control system via Gigabit Ethernet or a USB 3.0 interface, with transmission latency controlled within 5ms. A ring-shaped LED light source, approximately 120mm in inner diameter, provides over 120W of illumination power, emits natural white light with a color temperature of 5500K, and is equipped with a polarizing filter to effectively eliminate reflective interference from the cardboard box surface.

[0123] The pneumatic push rod straightening device is a custom-developed carton posture straightening equipment, comprising 3-5 vertically arranged pneumatic push rod modules. Each push rod module consists of a cylinder, a guide device, and a buffer end. The cylinder is a double-acting type with an inner diameter of 40-63mm and a stroke of 50-100mm, generating a thrust of 100-300N. The push rod array is installed on the side of the conveyor belt, covering 30%-80% of the carton height range, supporting overall height and spacing adjustments. Each push rod is controlled by an independent proportional flow valve, achieving precise thrust and speed regulation. The push rod end is equipped with a specially designed rubber buffer head with a smooth surface and appropriate elasticity to reduce damage to the carton and improve the straightening effect. The entire pneumatic system consists of an air source processing unit (filter, pressure regulating valve, oil mist lubricator), a pneumatic integrated control box, and an air distribution system, with the working air pressure stable within the range of 0.4-0.6MPa.

[0124] The controller system adopts a hierarchical distributed architecture, comprising a main controller and multiple sub-controllers. The main controller uses an industrial-grade PC, equipped with an Intel i7 or equivalent processor, 16GB DDR4 memory, 512GB SSD storage, and multiple high-speed communication interfaces (such as Gigabit Ethernet, USB 3.0, RS485, etc.). The sub-controllers include a vision processing controller, a pneumatic system controller, and the existing robotic arm controller. The vision processing controller focuses on image acquisition and processing, equipped with a dedicated GPU to improve the execution efficiency of deep learning algorithms. The pneumatic system controller uses a high-performance PLC, providing more than 16 digital inputs / outputs and more than 8 analog inputs / outputs, with a response time of less than 10ms. The controller system also includes a power module, a communication gateway, and I / O expansion modules, supporting standard industrial communication protocols (such as Profinet, EtherCAT, Modbus TCP, etc.).

[0125] The human-machine interface comprises two parts: hardware and software. The hardware uses an industrial-grade touchscreen display of 22 inches or larger, with a resolution of 1920×1080 and a brightness of at least 350 cd / m². It is equipped with a dustproof and waterproof front panel (IP65 rating) and installed in a convenient location within the operating area. It also includes an industrial keyboard, indicator lights, and an emergency stop button to meet the needs of daily operation and emergency handling. The software interface adopts a partitioned layout, including a system status area, a real-time image area, a parameter setting area, and an alarm information area. The status area displays the real-time operating status and key parameters of each component; the image area displays real-time images captured by the camera and detection result markers; the parameter setting area provides an interface for user-adjustable system parameters; and the alarm information area displays system anomalies and warning messages. The interface design follows ergonomic principles, using intuitive graphical representations combined with concise text descriptions to reduce operational difficulty.

[0126] Through the integration and arrangement of the above hardware, a complete physical system architecture is formed. The components are connected organically through mechanical, electrical, and communication connections, forming the physical foundation of the system. The system layout has been optimized to ensure both functionality and performance requirements while also considering installation space limitations and ease of maintenance, providing a reliable hardware platform for the implementation of the software system.

[0127] Second, the device layer driver is the foundation of the software system, interacting directly with the hardware device and providing standardized interfaces for upper-layer software to call. First, a camera driver was developed, based on the SDK provided by the device manufacturer, encapsulating functions such as image acquisition, parameter setting, and trigger control, supporting both continuous acquisition and external triggering modes. The image acquisition frame rate is set to 30fps, supporting the setting of ROI (Region of Interest) to reduce data volume, while implementing basic image preprocessing functions (such as noise filtering and contrast enhancement). Second, a pneumatic actuator driver was developed to achieve precise control of multi-way proportional valves, supporting the setting of thrust magnitude (by adjusting air pressure), action time (by controlling valve opening time), and actuator selection (by switching different valves). The driver uses an interrupt response mechanism to ensure real-time execution of control commands, with a response latency controlled within 5ms. Then, a robot driver was integrated, using a communication interface with the existing controller to control the robot's position, speed, and acceleration, and to obtain the robot's status feedback. The driver supports multiple motion modes, including point-to-point motion, continuous trajectory motion, and force-controlled motion. Finally, a conveyor belt driver program was developed. This program controls the motor speed via a frequency converter interface, enabling basic functions such as start, stop, acceleration, and deceleration. The actual speed is monitored through encoder feedback. All drivers provide a unified error handling mechanism and logging functionality to facilitate fault diagnosis and maintenance.

[0128] The communication layer employs the OPC UA (OPC Unified Architecture) protocol to construct a data interaction network. This is an open standard communication protocol for industrial automation, characterized by platform independence, security, and scalability. The system is designed with a three-layer communication architecture: the bottom layer uses fieldbuses (such as Profinet and EtherCAT) for internal communication within the device layer; the middle layer uses OPC UA for data exchange between subsystems; and the top layer uses HTTP / WebSocket for integration with the enterprise management system. The OPC UA server is deployed on the main controller, providing services such as information model definition, data access, method calls, and event notifications. The information model is divided into functional modules: a vision subsystem, a correction subsystem, a robotic arm subsystem, and a production management subsystem. Each subsystem contains multiple nodes with referencing relationships, forming a tree structure. Data exchange adopts a publish / subscribe model. The update frequency of key data (such as cardboard box posture parameters and control commands) is set to 100Hz to ensure system real-time performance; the update frequency of status information and statistical data is lower, typically 1-10Hz. The communication layer also implements a data caching mechanism to save data when the network is temporarily interrupted and automatically synchronize it after recovery, thereby improving the reliability of the system.

[0129] The algorithm layer is the core of the system, comprising two main parts: the attitude recognition algorithm and the response strategy decision algorithm. The attitude recognition algorithm is based on deep learning methods, and its specific implementation will be detailed in Example 4. This algorithm receives image data from the device layer, outputs the coordinates of the four corner points of the cardboard box, and then calculates attitude parameters such as the vertical deviation angle and horizontal offset. The algorithm implements various optimization techniques, such as model quantization, CUDA acceleration, and batch processing mechanisms, keeping the single-frame processing time within 40ms to meet real-time processing requirements. The response strategy decision algorithm adopts a rule-based and model-based hybrid approach, designing a three-level response mechanism. The algorithm first determines the severity of the cardboard box's state based on the attitude parameters, then selects the corresponding response level according to preset rules, generating specific control commands. For minor deviations, a gripping point compensation command is generated; for moderate deviations, push rod correction and gripping compensation commands are generated; for severe deviations, deceleration, alarm, and recording commands are generated. The decision-making process considers the current system state, historical data, and environmental constraints, and uses fuzzy logic to handle boundary conditions, improving the robustness of the decision. The algorithm also implements a self-learning mechanism, continuously optimizing decision rules and parameters by recording and analyzing historical decision results to adapt to different working conditions and cardboard box types.

[0130] The application layer, comprising the user interface and data analysis platform, serves as the window for system-user interaction. The user interface, developed using web technologies, supports both PC and mobile access, featuring a responsive design that automatically adapts to different display sizes. Functionally, the interface is divided into multiple modules: a monitoring module displays real-time system status and key parameters; a configuration module provides system parameter settings and access control; an alarm module displays and manages system anomalies; and a reporting module generates various statistical reports and trend charts. The interface operation logic is simplified to a three-step process of "task-operation-result," improving efficiency and reducing errors. The data analysis platform is a system integrating data acquisition, storage, processing, and visualization. It stores system operation data using a time-series database (such as InfluxDB), supporting efficient data writing and querying. The platform implements various data analysis functions: status monitoring analysis detects real-time system operation status and identifies potential problems; performance trend analysis tracks changes in key indicators and predicts system performance; anomaly cause analysis uses association rules to identify the causes and influencing factors of anomalies; and optimization suggestion generation provides parameter optimization suggestions based on historical data analysis. The platform supports custom reports and dashboards, allowing users to view data and indicators from different dimensions according to their needs.

[0131] Through the design and implementation of the above four-layer software architecture, a complete software functional system is formed. The layers interact through standard interfaces, achieving modular functionality and loose coupling, facilitating later maintenance and upgrades. This architectural design fully considers the requirements of real-time performance, reliability, and scalability, providing strong support for the stable operation of the system—this is the aforementioned software functional support.

[0132] Third, the information flow of the entire system begins with image acquisition. When the cardboard box enters the field of view, a photoelectric sensor or a timed trigger signal activates the camera to continuously acquire images at a rate of 30fps. Image data is transmitted to the main controller via a gigabit network, with transmission time controlled within 5ms to ensure real-time data transmission. After the image arrives, it first undergoes noise filtering and quality enhancement through an image preprocessing module, with preprocessing time of approximately 10ms. The preprocessed image then enters the attitude recognition algorithm for processing, identifying the coordinates of the four corner points of the cardboard box and calculating attitude parameters such as the vertical deviation angle θ and the horizontal offset Δx, with processing time of approximately 40ms. After the attitude parameters are calculated, they are immediately published to the communication network through the data change notification mechanism of the OPC UA service, with the data item marked as high priority and transmission delay controlled within 5ms.

[0133] Attitude data is transmitted to the response strategy decision module via the OPC UA network. This module subscribes to attitude data in real time and initiates the decision-making process immediately upon receiving new data. The decision-making process first determines the severity of the cardboard box's condition and then generates corresponding control commands based on the three-level response mechanism. The control commands consist of several parts: correction commands specify the push rod selection, push force magnitude, and duration; gripping commands specify the target position, attitude adjustment, and suction distribution; and system control commands specify conveyor belt speed adjustment and alarm triggering, etc. The decision-making process executes in approximately 15ms, ensuring that the decision is completed within one image frame cycle (33ms). The generated control commands are sent to each execution unit via OPC UA method calls. The method calls employ a synchronous mode to ensure that the commands are correctly received.

[0134] Control commands are distributed to the controllers of each actuator via the OPC UA network. Each controller receives the commands through an OPC UA client and then invokes the corresponding device-level driver to execute the specific action. Upon receiving the command, the controller of the correction actuator activates the designated push rod via the pneumatic driver, controlling the magnitude and duration of the thrust to perform carton posture correction. Upon receiving the command, the controller of the packing robot adjusts the target position, motion parameters, and suction cup control parameters via the robot driver to perform carton gripping and packing actions. The controller of the conveyor system adjusts the speed or triggers a stop operation based on the commands. After all actions are completed, each controller records the execution results and status data, including execution time, actual parameters, sensor readings, and abnormal information; this data constitutes the execution data. The execution data is written back to the communication network via the OPC UA data access service for access by other components.

[0135] The data analysis platform continuously collects operational data from various parts of the system via the OPC UA client, including attitude parameters, control commands, and execution data. The data undergoes preprocessing, including data cleaning, outlier filtering, and data standardization. It is then aggregated and statistically analyzed at different time granularities (e.g., per minute, per hour, per shift) to generate various performance and status indicators. The platform implements multiple analytical functions: performance analysis calculates indicators such as the system's anomaly rate, response level distribution, and processing efficiency; correlation analysis identifies relationships between system parameters and performance indicators; predictive analysis predicts future performance trends based on historical data; and optimization analysis uses simulation models and optimization algorithms to find the optimal parameter combination. The analysis results are displayed intuitively on the dashboard, generating optimization suggestions to guide operators in adjusting system parameters or directly adjusting equipment parameters through the automatic control interface, completing closed-loop optimization.

[0136] Through the design and implementation of the above information flow and control flow, a complete closed-loop control architecture is formed: from perception (image acquisition and posture recognition) to decision-making (response strategy formulation) to execution (correction and grasping actions), and finally through feedback (execution data and analysis results) to guide the system to self-adjust and optimize. This closed-loop design enables the system to operate continuously and stably, and to constantly adapt to changing working conditions and cardboard box types, achieving efficient and reliable cardboard box posture correction and packing operations.

[0137] Example 3:

[0138] In this embodiment, acquiring the cardboard box image and completing the calibration based on the closed-loop control system architecture includes:

[0139] Based on the vision inspection device in the closed-loop control system architecture, an industrial camera with a resolution of 2048×1536 is installed above the conveyor belt outlet of the torsional conveyor, equipped with a polarizing filter to eliminate reflective interference, and a ring LED is used to provide uniform illumination, thus obtaining the vision inspection hardware configuration.

[0140] Based on the aforementioned visual inspection hardware configuration, the camera's intrinsic and extrinsic parameters are calibrated using the checkerboard calibration method, establishing a mapping relationship between pixel coordinates and actual physical coordinates to obtain the calibrated camera system.

[0141] Based on the calibrated camera system, when the cardboard box is conveyed to the detection area by the conveyor belt of the twisting conveyor, the industrial camera captures an image of the cardboard box and transmits it to the algorithm layer in the closed-loop control system architecture to obtain the image of the cardboard box to be identified.

[0142] Specifically, the first step involves detailed planning of the industrial camera's installation location. By measuring the geometry of the twisting conveyor, the conveyor belt exit area is determined as the optimal detection location, as the cardboard boxes here have completed a 90-degree rotation, are in a relatively stable vertical state, and are not obstructed by subsequent devices. Installation height is a critical parameter; experiments have determined that 600-800mm above the conveyor belt surface is optimal. This height ensures that the field of view covers the entire cardboard box (including the largest size, reaching 500mm in height) while maintaining sufficient depth of field for image clarity. The camera is mounted using a dedicated bracket system, consisting of an aluminum profile main frame and a precision adjustment mechanism. This system supports X, Y, and Z-axis position adjustment (accuracy ±1mm) and rotation adjustment around the three axes (accuracy ±0.5°), ensuring the camera's optical axis is perpendicular to the conveyor belt surface and reducing perspective distortion. The bracket design incorporates vibration resistance, using reinforcing ribs and dampers to reduce vibration transmission during equipment operation and ensure image stability.

[0143] The selected industrial camera features a 2048×1536 pixel CMOS sensor with a 1 / 1.8-inch sensor size and 3.45μm×3.45μm pixel size, ensuring high resolution and good light sensitivity. The camera supports a global shutter mode with an adjustable exposure time range of 10μs-1s to adapt to different lighting conditions. The frame rate exceeds 30fps, meeting real-time inspection requirements. The camera uses a C-mount and is paired with a 16mm low-distortion industrial lens with an aperture range of F1.4-F16, supporting manual aperture and focus adjustment. The lens has a field of view of approximately 28°×21°, and at a working distance of 600-800mm, the field of view is approximately 600mm×450mm, sufficient to cover even the largest cardboard boxes (up to 400mm wide) with some margin. The entire imaging system achieves a resolution of 0.3mm / pixel, sufficient for accurate detection of cardboard box edges and corners.

[0144] Polarizing filters are a key component in addressing glare interference from cardboard box surfaces. Cardboard box surfaces, especially laminated or glossy printed boxes, produce strong reflections under intense light, affecting image quality and recognition accuracy. The system uses linearly polarizing filters to solve this problem. Specifically, a horizontally polarizing filter is installed in front of the light source, and a vertically polarizing filter is installed in front of the camera lens. This cross-polarization configuration effectively filters out specular reflections from the cardboard box surface (maintaining the polarization direction), allowing only diffuse reflections (whose polarization direction is changed by scattering) to pass through, significantly reducing strong reflections and improving image contrast. The polarizing filters use optical-grade materials with high transmittance (>90%) and low color cast, without significantly reducing image brightness. The filters are mounted on a rotating filter ring at the front of the lens, allowing for easy adjustment of the polarization angle to achieve optimal results.

[0145] The lighting system employs a ring-shaped LED design, a circular light source surrounding the camera lens that provides uniform, shadow-free illumination, particularly suitable for inspecting planar objects. Specifically, it features a ring-shaped LED light source with an inner diameter of 120mm and an outer diameter of 200mm, mounted coaxially with the front of the camera lens. The light source consists of 144 high-brightness LEDs, with a total power of 120W, providing natural white light at a color temperature of 5500K, offering a highly uniform (>85%) illumination area. The light source brightness can be adjusted via a PWM controller (256 levels of adjustment within the 0-100% range) to adapt to different colors and materials of cardboard boxes. To reduce ambient light interference, a light-shielding plate is also installed in the inspection area, creating a semi-enclosed inspection environment and improving image quality stability. The lighting system design also considers heat dissipation, employing an aluminum heat sink and a miniature fan to ensure stable operation over extended periods without temperature fluctuations affecting light intensity.

[0146] The aforementioned visual inspection hardware configuration forms an organic whole. Through precise calibration and testing, it ensures the stable acquisition of high-quality cardboard box images under various production conditions. Configuration parameters (such as camera height, focal length, aperture, exposure time, and light source brightness) are recorded in the system configuration file as standard operating settings, while also retaining a certain adjustment margin to meet the needs of different cardboard box types and environmental changes.

[0147] Secondly, camera calibration is a crucial step in vision measurement systems, directly impacting measurement accuracy. Zhang's calibration method is employed, a camera calibration technique based on a planar calibration plate, characterized by its ease of operation and high precision. The calibration process begins with the selection of a high-precision calibration plate. This plate uses a 9×6 black and white checkerboard pattern, with each grid measuring 25mm×25mm, printed on a 3mm thick aluminum composite plate. The surface is matte to reduce glare. The calibration plate's manufacturing precision is controlled within ±0.05mm to ensure the accuracy of the calibration reference.

[0148] Calibration data acquisition requires capturing images of the calibration board at multiple different positions and orientations. The specific procedure is as follows: Place the calibration board within the detection area, covering the entire field of view, and acquire at least 20 sets of images in different orientations by rotating and tilting the calibration board. Orientation changes include: translation of the calibration board in the XY plane (at least 9 different positions, covering the four corners and the center of the field of view), rotation around the X-axis (at least 3 angles within ±30°), rotation around the Y-axis (at least 3 angles within ±30°), and rotation around the Z-axis (at least 3 angles within ±45°). Maintain stable lighting conditions during the shooting process and use fixed camera parameter settings (aperture, focal length, exposure time) to ensure consistent imaging conditions for all images.

[0149] Image processing and corner detection are preprocessing steps in the calibration calculation. The acquired calibration images are processed as follows: first, Gaussian filtering is applied to reduce noise; then, an adaptive thresholding algorithm is used to binarize the image; next, a sub-pixel corner detection algorithm (such as an improved version of the Harris corner detector) is applied to accurately locate the position of each checkerboard intersection. The detection accuracy reaches the 0.1 pixel level, far exceeding the accuracy of general edge detection. The system automatically detects the checkerboard pattern, determines the number of rows and columns, and sorts the corners to form an ordered set of corners. For each calibration image, the system extracts the pixel coordinates of all 54 (9×6) corners and records the corresponding physical coordinates (based on the known checkerboard size).

[0150] Intrinsic parameter calibration calculates the camera's internal parameters, including focal length, principal point coordinates, and distortion coefficients. The calculation process is based on a perspective projection model, obtaining optimal parameter estimates by solving a system of nonlinear equations. The intrinsic parameter matrix contains the focal length (fx, fy) and principal point coordinates (cx, cy), describing the ideal pinhole camera model; the distortion coefficients contain radial distortion coefficients (k1, k2, k3) and tangential distortion coefficients (p1, p2), describing the distortion introduced by the actual lens. The system uses the Levenberg-Marquardt optimization algorithm to minimize the reprojection error (the difference between the actual corner position and the position reprojected according to the model), obtaining the optimal solution through multiple iterations. The quality of the calibration results is evaluated by the reprojection error; the reprojection error of this system is controlled within 0.3 pixels, indicating good calibration accuracy.

[0151] Extrinsic parameter calibration establishes the transformation relationship between the camera coordinate system and the world coordinate system (usually defined as the conveyor belt surface coordinate system). The extrinsic parameters, including the rotation matrix R and translation vector t, describe the camera's position and orientation in the world coordinate system. To obtain extrinsic parameters consistent with practical applications, specific markers (such as calibration plates of known dimensions) are placed on the conveyor belt surface. Images are captured and the markers are identified; the extrinsic parameters are then calculated in conjunction with intrinsic parameters. The accuracy of the extrinsic parameter calibration directly affects the accuracy of physical dimension measurements; therefore, a multi-point verification method is employed. Multiple markers at known locations are placed within the working area to verify the accuracy of the transformation, ensuring an average error of less than 1 mm.

[0152] The mapping between pixel coordinates and physical coordinates is the ultimate goal of calibration. With intrinsic and extrinsic parameters, three coordinate transformations can be achieved: 1) transformation from pixel coordinates to camera coordinates, achieved through the intrinsic parameter matrix and distortion correction; 2) transformation from camera coordinates to world coordinates, achieved through the extrinsic parameter matrix; 3) inverse transformation from world coordinates to pixel coordinates. For carton posture detection, the most crucial aspect is planar measurement, which converts the pixel positions in the image into the actual physical positions (in millimeters) on the conveyor belt surface. A simplified mapping relationship was established, directly mapping the corrected pixel coordinates to the XY plane of world coordinates (i.e., the conveyor belt surface), forming a mapping function f(u, v) → (x, y), where (u, v) are pixel coordinates and (x, y) are physical coordinates. The accuracy of this mapping function was verified by measuring an object of known size; the measurement error was controlled within ±0.5 mm, meeting the accuracy requirements for carton posture detection.

[0153] Saving and applying the calibration results is the final step. The calculated intrinsic parameter matrix, distortion coefficients, extrinsic parameter matrix, and coordinate mapping relationships are saved in a configuration file as the system's calibration data. These parameters are automatically loaded each time the system starts for real-time image correction and coordinate transformation. A calibration verification function is also implemented, periodically (e.g., once per shift) to check the validity of the calibration by measuring a standard object. If the deviation exceeds a threshold, the operator is alerted to recalibrate. This mechanism ensures the long-term validity of the calibration parameters, maintaining measurement accuracy even with slight changes in camera position or environmental conditions.

[0154] Finally, the carton detection process first requires precise determination of the detection area and triggering timing. The detection area is located at the output end of the twisting conveyor, where the carton has completed a 90-degree rotation and remains relatively stable on the conveyor belt. To accurately capture the carton, a dual triggering mechanism is employed: the primary trigger is based on a photoelectric sensor, and the secondary trigger is based on a timer. The photoelectric sensor (through-beam type) is installed on the side of the conveyor belt, with the sensor beam height approximately one-third of the carton's height. When the carton passes by, the beam is interrupted, generating a trigger signal. To avoid the instability of a single sensor, an image analysis-based confirmation mechanism is added. Detection is considered valid only when the sensor triggers and an object is indeed detected in the image. Anti-vibration processing is also implemented, ignoring trigger signals shorter than a set time (e.g., 50ms) to prevent false triggering caused by carton shaking or conveyor belt vibration.

[0155] Timing control of image acquisition is crucial to ensuring the acquisition of optimal images. After a trigger signal is confirmed to be valid, an image is not acquired immediately. Instead, a delay period (typically 100-200ms, adjusted according to conveyor belt speed) is waited to ensure the cardboard box is fully within the center of the field of view. Acquisition employs a "multi-frame mode," continuously acquiring 3-5 frames at 30-50ms intervals, selecting the best-quality image for subsequent processing. This method effectively handles potential slight cardboard box vibrations and lighting fluctuations, improving image quality stability. A maximum waiting time (e.g., 500ms) is set. If an image meeting quality requirements is not acquired within this time, a warning is triggered, and the best-quality image already acquired is used for further processing, preventing system crashes.

[0156] Image preprocessing is a crucial step in improving recognition accuracy. The original image undergoes a series of processing steps: First, lens distortion correction is applied, eliminating radial and tangential distortion based on calibrated distortion coefficients; then, illumination equalization is performed, using adaptive histogram equalization to reduce shadows and highlights, improving image contrast; next, noise removal is applied, employing a bilateral filter to reduce noise while preserving edge details; finally, geometric correction is performed, transforming the image to a standard viewpoint (top view) to eliminate perspective distortion caused by the camera's mounting angle. The preprocessed image exhibits higher quality and consistency, providing a solid foundation for subsequent deep learning-based recognition.

[0157] Image quality assessment is a crucial mechanism for ensuring reliable recognition. Each acquired image undergoes a quality assessment, including sharpness evaluation (based on image gradient statistics), contrast evaluation (based on histogram distribution), integrity evaluation (checking if the cardboard box is completely within the field of view), and illumination evaluation (detecting overexposed or underexposed areas). Each metric is scored between 0 and 100. Images with a combined score below a threshold (typically 70) are marked as low-quality, triggering re-acquisition or a warning. Statistical data on image quality metrics are recorded to monitor camera and lighting system performance and promptly identify potential problems (such as lens contamination, light source attenuation, etc.).

[0158] Image transmission employs an efficient and reliable mechanism to ensure data integrity and real-time performance. Industrial cameras support multiple transmission interfaces; this system prioritizes Gigabit Ethernet (GigE Vision) or USB 3.0 interfaces. The former is suitable for long-distance transmission (up to 100m), while the latter is suitable for short-distance, high-bandwidth transmission. Image data is transmitted in lossless compression formats (such as PNG) or the original format to avoid image quality degradation due to compression. The system implements a data flow control mechanism; when the processing unit load is too high, the image resolution or frame rate can be temporarily reduced to ensure the system's real-time responsiveness. To improve transmission efficiency, a Region of Interest (ROI) transmission mode is supported, transmitting only the image area containing the cardboard box, reducing data volume by approximately 40-60%. The transmission process includes a verification mechanism to ensure data integrity; automatic retransmission occurs when a transmission error is detected, and the current frame is only abandoned after the maximum number of retries has been exceeded.

[0159] Data management is the final link in the image acquisition system. A tiered storage strategy is implemented for acquired images: all images are compressed at low resolution and stored for 7 days for routine viewing and statistics; original images of abnormal situations (such as recognition failures or severe tilting) are stored separately for 30 days for problem analysis and algorithm optimization; a certain proportion of original images are randomly saved periodically (e.g., hourly) for long-term quality monitoring and algorithm verification. A distributed file system is used for storage to ensure data security and fast access. Image retrieval functionality is also implemented, supporting quick searching of historical images by time, cardboard box type, and anomaly type, facilitating problem tracing and analysis.

[0160] Through the above steps, high-quality images of cardboard boxes can be stably acquired and necessary preprocessing can be completed to form images of cardboard boxes to be recognized, laying a solid foundation for subsequent deep learning recognition. The design fully considers the complexity and stability requirements of the industrial environment, and ensures the reliability and accuracy of the image acquisition process through multiple safeguard mechanisms.

[0161] Example 4:

[0162] In this embodiment, the step of using a deep learning model to identify the corner coordinates of the cardboard box includes:

[0163] Based on the algorithm layer in the closed-loop control system architecture, images of cardboard boxes of various specifications and materials in different states are collected, and the coordinates of the four corner points are marked as key feature points. The dataset is expanded to a preset number of samples by methods such as rotation, scaling, brightness adjustment, and adding noise to obtain the training dataset.

[0164] Based on the training dataset, the output layer of the YOLOv8-Pose model was modified to adapt to the requirements of cardboard box four-corner point detection. MobileNetV3 was used as the backbone network, and the model parameters of the YOLOv8-Pose model were initialized using the transfer learning method. Then, the model was trained iteratively using the Adam optimizer with an adaptive learning rate to obtain a lightweight pose detection model with a recognition accuracy greater than 98% and a single frame processing time of less than 50ms.

[0165] Specifically, firstly, before training the deep learning model, the primary task is to construct a high-quality, diverse training dataset. The data acquisition strategy of this system is based on the "full coverage" principle, ensuring that the training data represents various situations that may be encountered in actual production. Firstly, the coverage range of the cardboard box samples is determined: the specification dimension includes 5 different size combinations (height range 200-500mm, width range 150-350mm), covering more than 90% of the cardboard box specifications on the production line; the material dimension includes 3 common materials (corrugated paper, white cardboard, coated paper), and surface treatment variations of these materials (such as lamination, printing, embossing, etc.); the color dimension includes monochrome (white, brown, gray, etc.) and multi-color printing (various brand packaging); the status dimension includes normal verticality, slight tilt (±5°), moderate tilt (±5° to ±15°), and severe tilt (±15° or more).

[0166] Sample collection employed a combination of methods: actual production line sampling and simulated scenario sampling. Actual production line sampling was conducted during normal production, using the same hardware configuration as the actual system to capture images of cardboard boxes in a real production environment. During continuous production, an image was randomly captured every 50-100 boxes, for 3-5 days, covering different shifts and production batches to ensure data diversity and representativeness. Simulated scenario sampling was conducted on a specially constructed test bench, using the same camera and lighting system, but by manually controlling the position and orientation of the cardboard boxes, targeted samples were collected in specific states (especially various tilt states), compensating for the lack of samples in certain states during actual production.

[0167] The annotation process is a crucial step in ensuring data quality. A semi-automated annotation method is employed: first, preliminary annotations are generated using edge detection and corner detection algorithms based on traditional computer vision; then, professional annotators review and correct these annotations using a custom annotation tool. The annotation tool provides multi-level scaling, contrast adjustment, and precise positioning functions, supporting sub-pixel-level position adjustments to ensure annotation accuracy. Each image is annotated with the coordinates of four corner points, arranged in the order of top left, top right, bottom right, and bottom left, forming an ordered set of points. Annotation quality control employs a dual verification mechanism: first, each image is independently annotated by two annotators; if the difference between the two annotations exceeds a preset threshold (e.g., 5 pixels), a third annotator makes the final decision; second, 10% of the annotated samples are randomly selected for manual review to ensure annotation consistency. This rigorous annotation process ensures high-quality training data, providing a reliable foundation for model training.

[0168] The number of samples collected initially is often insufficient to train a robust deep learning model, so data augmentation techniques are used to expand the training set. Specific augmentation methods include: geometric transformations such as random rotation (within ±15°), random translation (within ±10% of the image size), random scaling (within 80%-120%), random cropping (preserving at least 90% of the cardboard box area), and random perspective transformations (simulating different viewpoints); color adjustments such as random brightness variations (within ±30%), random contrast variations (within ±20%), random hue variations, and random saturation variations; noise additions such as Gaussian noise (standard deviation 0.01-0.05), salt-and-pepper noise (noise ratio 0.5%-2%), and blurring (simulating slight camera shake or out-of-focus); and occlusion simulation, randomly adding small occlusion patches (covering less than 5% of the cardboard box area) to simulate partial occlusion in a real-world environment. Each original image generates 10-15 variants by randomly combining these augmentation methods, while simultaneously updating the corresponding corner coordinates to ensure the annotations match the augmented images.

[0169] Dataset partitioning and management are the final preparatory steps before training. The augmented, complete dataset contains approximately 15,000-20,000 labeled images, divided into training, validation, and test sets in an 8:1:1 ratio. The partitioning follows a stratified sampling principle, ensuring that the distribution of samples of different sizes, materials, and states is roughly the same across each subset. Detailed records are kept of the dataset, including metadata for each sample (such as collection time, cardboard box specifications, material, and state category), facilitating subsequent analysis of performance for specific categories. The dataset undergoes a final quality check, removing or repairing problematic samples (such as incorrect labeling or blurred images) to ensure the reliability of the final training dataset.

[0170] Through the above data collection, annotation, and enhancement steps, a large-scale training dataset containing various specifications and materials of cardboard boxes in different states was constructed, providing a solid data foundation for subsequent deep learning model training.

[0171] Second, model architecture design is the primary task in system algorithm development. YOLOv8-Pose was chosen as the basic framework, a highly efficient single-stage object detection and keypoint localization model. Originally designed for human pose estimation (detecting 17 keypoints), YOLOv8-Pose was modified to meet the specific needs of cardboard box corner detection. Core modifications included: output header redesign, reducing the number of keypoints from 17 to 4 (corresponding to the cardboard box corners) while retaining bounding box detection functionality; loss function adjustment, adding corner point order consistency constraints to ensure the model outputs four corner points arranged in the order of top left, top right, bottom right, and bottom left; and inference process optimization, removing post-processing steps specific to human pose estimation to simplify the computation process. These modifications allowed the model to focus more on the cardboard box corner detection task, reducing computational complexity and improving inference speed.

[0172] To meet the real-time processing requirements of industrial environments, the lightweight MobileNetV3 was adopted as the backbone network, replacing the heavier CSPDarknet in the original YOLOv8-Pose. MobileNetV3 is a high-efficiency network designed specifically for mobile devices, using depthwise separable convolutions to significantly reduce the number of parameters and computational cost while maintaining good feature extraction capabilities. The MobileNetV3-Large variant was chosen to achieve a good balance between accuracy and speed. To further optimize computational efficiency, several network compression techniques were applied: channel pruning to reduce unimportant convolutional channels; knowledge distillation to transfer the "knowledge" of a larger model to a smaller model; and low-bit quantization to convert floating-point operations into integer operations, accelerating inference. These optimizations enabled the model to maintain accuracy while reducing the number of parameters by approximately 70% and computational cost by approximately 65%, meeting the requirements of real-time processing.

[0173] Transfer learning is a key technique for accelerating model training and improving performance. Instead of random initialization, pre-trained weights are used to initialize model parameters. Specifically, for the MobileNetV3 backbone, weights pre-trained on the ImageNet dataset are used, having learned rich visual feature representations. For layers related to detection and keypoint localization, YOLOv8-Pose weights trained on the COCO dataset are used, incorporating prior knowledge of keypoint localization. Due to task differences, a layered transfer learning strategy is employed: shallow network parameters (responsible for basic feature extraction) are fully transferred and frozen in the early training phase; deep network parameters (responsible for task-specific feature extraction) are partially transferred and fine-tuning begins immediately; output layer (redesigned) parameters are randomly initialized and training begins immediately. This strategy leverages the general feature representation capabilities of the pre-trained model while allowing the model to adapt to the specific cardboard corner detection task.

[0174] The training process employs a multi-stage strategy to progressively optimize model performance. The first stage, "warm-up training," lasts 10 epochs with a small learning rate (0.0001), training only newly added and modified layers while freezing the pre-trained backbone network, allowing the model to gradually adapt to the new task. The second stage, "joint training," lasts 40 epochs, unfreezing all network layers and using a moderate learning rate (0.001) to fine-tune the entire network. The third stage, "fine-tuning," lasts 50 epochs with a decreasing learning rate (starting from 0.0005 and decaying to 0.1 every 10 epochs), focusing on optimizing performance on difficult examples. The entire training process uses the Adam optimizer, an adaptive learning rate optimization algorithm that automatically adjusts the learning rate for each parameter, accelerating convergence and reducing the need for hyperparameter tuning. β1 is set to 0.9, β2 to 0.999, and the weight decay is 0.0005; these parameters were determined through multiple experiments to achieve optimal training results.

[0175] Loss function design is crucial for guiding model learning. A composite loss function is employed, comprising three parts: object detection loss, keypoint localization loss, and order consistency loss. The object detection loss uses the standard YOLO loss function, including bounding box position loss (CIoU loss) and confidence loss (binary cross-entropy). The keypoint localization loss combines smooth L1 loss and OKS (Object Keypoint Similarity) loss, where smooth L1 loss focuses on pixel-level precision localization, and OKS loss considers the relative importance and visibility of keypoints. The order consistency loss is a custom loss that penalizes incorrect corner point order to ensure the four corner points in the output are arranged in a predefined order. These three loss components are weighted and empirically set as follows: object detection loss 0.3, keypoint localization loss 0.6, and order consistency loss 0.1. These weights are slightly adjusted during training to balance the learning progress of different tasks.

[0176] The training environment and implementation details significantly impact the final performance. Training was performed on a GPU workstation using CUDA version 11.1, cuDNN version 8.0.5, Python version 3.8, and the deep learning framework PyTorch 1.8.1. A mixed-precision strategy was employed to accelerate training and reduce GPU memory usage without sacrificing accuracy. The batch size was set to 32; if GPU memory was insufficient, cumulative gradients (updating parameters every 4 mini-batches) were used to simulate the effect of large-batch training. Multiple regularization techniques were employed during training to prevent overfitting: Dropout (with a rate of 0.1) randomly discarded some neurons; weight decay penalized large weight values; and data augmentation randomly applied different augmentation methods in each training epoch. An early stopping strategy was also implemented, automatically stopping training if performance on the validation set did not improve for 10 consecutive epochs to avoid overfitting. The entire training process took approximately 8-10 hours under the above configuration, depending on the dataset size and training strategy.

[0177] Model evaluation and performance validation are crucial steps after training. Model performance was evaluated on independent test sets using multiple metrics: corner detection accuracy (corner position error <5 pixels considered correct) reached 98.5%; average corner position error was approximately 2.1 pixels; pose parameter calculation accuracy (vertical deviation angle error <0.5° considered correct) reached 99.0%; average inference time was 38ms on a standard industrial PC with an Intel i7 processor and 8GB RAM, meeting real-time processing requirements. Targeted tests were also conducted to evaluate the model's performance under specific conditions: detection accuracy differences for different cardboard box sizes were less than 1%; detection accuracy differences for different surface materials (especially reflective surfaces) were less than 2%; detection accuracy for different tilt angles decreased slightly under severe tilt (>15°), but still remained above 95%. These evaluation results demonstrate that the trained model possesses high accuracy and good robustness, meeting the needs of real-world production environments.

[0178] Model deployment and optimization are the final steps in applying the trained model to the real-world system. The PyTorch-trained model is converted to ONNX format and then further optimized using TensorRT, resulting in a significant improvement in inference speed (approximately 30-50%). To adapt to the characteristics of industrial control systems, an asynchronous model invocation mechanism is implemented, with image acquisition and processing performed in independent threads to avoid blocking the main control flow. Result caching and smoothing mechanisms are also implemented, using a weighted average of detection results from multiple consecutive frames to reduce the impact of occasional errors and improve the stability of pose parameters. These deployment optimizations ensure the model's efficient and stable performance in the real-world operating environment.

[0179] Through the above design, training and optimization steps, a lightweight deep learning model specifically for cardboard box corner detection was successfully developed. It meets the requirements of industrial applications in terms of both accuracy and speed, and provides reliable visual perception capabilities for cardboard box posture correction systems.

[0180] Example 5:

[0181] In this embodiment, based on the paper box posture parameters, the system divides the state into multiple levels according to the magnitude of the vertical deviation angle and the horizontal offset, and executes corresponding response strategies, including gripping point compensation, activation of the correction execution device, and anomaly marking processing, to obtain hierarchical linkage control commands, including:

[0182] Based on the carton posture parameters, when the vertical deviation angle is detected to be less than the first threshold, the compensation amount for the packing robot's gripping position is calculated, wherein the compensation amount is related to the vertical deviation angle and the carton height, and is transmitted to the robot controller in real time through the communication layer using a communication protocol to obtain the first-level control command;

[0183] Based on the carton posture parameters, when the vertical deviation angle is detected to be between the first threshold and the second threshold, the thrust application point height and push rod extension time are calculated based on the first-level control command. The thrust application point height is related to the vertical deviation angle and the carton height, and the push rod extension time is related to the vertical deviation angle and the conveyor belt speed. The thrust magnitude is adjusted through the equipment layer control signal to obtain the second-level control command.

[0184] According to the paper box posture parameters, when the vertical deviation angle is detected to be greater than the second threshold or when multiple corrections fail, based on the first-level control command and the second-level control command, the central control system controls the conveyor belt of the twisting conveyor to decelerate, the controller of the central control system controls the alarm light to light up and emits an audible prompt, the abnormal location and image are displayed in real time on the human-machine interface, the abnormality type and frequency are automatically recorded, and a third-level control command is obtained.

[0185] The hierarchical linkage control command is obtained based on the first-level control command, the second-level control command, and the third-level control command.

[0186] Specifically, firstly, in this step, the tilt degree of the cardboard box is determined based on the cardboard box posture parameters calculated from the deep learning recognition results. The vertical deviation angle refers to the angle between the cardboard box's central axis and the ideal vertical line; a positive value indicates a rightward tilt, and a negative value indicates a leftward tilt. The first threshold is the boundary value for the system to determine slight tilt, which was determined to be 5° through extensive experiments. This threshold is not fixed but dynamically adjusted according to the cardboard box's height-to-width ratio. For slender cardboard boxes with a height-to-width ratio greater than 2, the system lowers the threshold to 3°-4° because slender cardboard boxes are more prone to instability at the same angle; for flat cardboard boxes with a height-to-width ratio less than 1, the system raises the threshold to 6°-7° because these types of cardboard boxes have better stability.

[0187] When the vertical deviation angle of the cardboard box is detected to be less than the first threshold, it is determined to be in a slightly tilted state. At this time, there is no need to activate the physical correction device; instead, the gripping position of the robotic arm is adjusted to adapt to the actual posture of the cardboard box. The calculation of the gripping position compensation is based on geometric relationships, mainly considering two factors: the vertical deviation angle and the height of the cardboard box. The basic compensation calculation formula is ΔX = H·sin(θ)·k1, where H is the height of the cardboard box (millimeters), θ is the vertical deviation angle (radians), and k1 is the compensation coefficient, usually taken as 0.6-0.8. The physical meaning of this formula is: the center of mass of the tilted cardboard box has shifted relative to the geometric center, and the shift is approximately proportional to the sine of the tilt angle and the height of the cardboard box. The compensation coefficient k1 is less than 1 to account for the uneven weight distribution of the cardboard box.

[0188] In addition to the basic compensation amount, the horizontal offset of the cardboard box is also considered. The comprehensive compensation amount is calculated as ΔXc = ΔX + Δx·k2, where Δx is the horizontal offset (millimeters) and k2 is the horizontal compensation coefficient, typically ranging from 0.8 to 1.0. Horizontal compensation ensures that the robot arm can align with cardboard boxes that are off-center from the conveyor belt. The compensation amount is also fine-tuned based on the material and weight of the cardboard box; for example, the compensation coefficient is slightly increased for heavier cardboard boxes to provide a more accurate gripping position.

[0189] The calculated compensation amount needs to be transmitted to the robot controller in real time. The OPC UA protocol is used as the primary communication method. This is a standard protocol in industrial automation, offering platform independence and high reliability. The communication process includes the following steps: The central control system packages the compensation amount into a standard data structure, including the cardboard box ID, timestamp, compensation vector (compensation values ​​in the X, Y, and Z directions), and execution priority; the OPC UA server publishes the data to the communication network and triggers a data change notification; upon receiving the notification, the robot controller's OPC UA client immediately reads the compensation data; the client verifies the data validity (e.g., checking the timestamp to ensure data freshness), and then converts the compensation value into an offset in the robot's coordinate system; finally, the robot control program applies these offsets to the original target position to generate a corrected grasping path. The latency of the entire communication process is controlled within 20ms to ensure the system's real-time responsiveness.

[0190] The first-level control instructions are a set of structured control parameters, containing the following core elements: target position information, including original coordinates and compensated coordinates; execution parameters, such as gripping speed, acceleration, and gripping force; timing parameters, specifying the execution time and duration of the action; and status feedback requirements, specifying the type and frequency of status information that the robot needs to provide feedback. These instructions are passed to the execution layer through the interface functions of the robot controller, and are ultimately translated into motor rotation and pneumatic component movement to achieve precise gripping position compensation.

[0191] Second, when the vertical deviation angle of the cardboard box exceeds the first threshold but is less than the second threshold, it is determined to be in a moderate tilt state, requiring the activation of the correction actuator for physical correction. The second threshold is usually set at 15°, and will also be dynamically adjusted according to the aspect ratio of the cardboard box. A moderately tilted cardboard box cannot be stably gripped by gripping position compensation alone, but it has not reached the level of seriously affecting production, so it is suitable to be corrected by physical pushing. At this time, the system will simultaneously generate correction control instructions and gripping compensation instructions (as a backup plan), forming a second-level control instruction.

[0192] The height of the thrust application point is a key factor in the corrective effect, determining the vertical position of the push rod acting on the side of the cardboard box. The calculation of the thrust application point height considers two main factors: the vertical deviation angle and the height of the cardboard box. The basic calculation formula is h = H·(0.5+0.1·θ / θ1), where H is the height of the cardboard box, θ is the vertical deviation angle, and θ1 is the first threshold. The physical meaning of this formula is: when the cardboard box is slightly tilted, the application point should be close to the center of the cardboard box (approximately 50% of its height); as the tilt angle increases, the application point should be appropriately moved upwards (up to 60% of the height) to generate a greater corrective torque. Fine-tuning will be made according to the actual situation; for example, for cardboard boxes with a low center of gravity (such as those with heavy objects at the bottom), the application point will be adjusted downwards accordingly to avoid excessive displacement or deformation of the top of the cardboard box due to excessive thrust.

[0193] The system employs a segmented pusher design, typically with 3-5 vertically arranged pushers covering the possible height range of the cardboard box. Based on the calculated height of the push point of application, the nearest pusher is activated. If the push point of application is between two pushers, both pushers are activated simultaneously, and the push force is distributed inversely proportional to the distance between them, ensuring the force is near the ideal height. For example, if the ideal point of application is 25mm from the upper pusher and 15mm from the lower pusher, the lower pusher receives approximately 62.5% of the total push force, and the upper pusher receives approximately 37.5%.

[0194] The pusher extension time is another key parameter, determining the duration of the pushing force and directly affecting the correction effect. Calculating the pusher extension time requires considering three factors: the vertical deviation angle, the conveyor belt speed, and the cardboard material. The basic calculation formula is t = t0·(1+0.3·(θ-θ1) / (θ2-θ1))·(v0 / v)·km, where t0 is the base time (usually 100ms), θ is the vertical deviation angle, θ1 is the first threshold, θ2 is the second threshold, v is the current conveyor belt speed, v0 is the standard conveyor belt speed (e.g., 0.3m / s), and km is the material coefficient (e.g., 1.0 for rigid cardboard and 1.2 for flexible cardboard). This formula reflects the following: the larger the tilt angle, the longer the pushing force needs to be applied; the faster the conveyor belt speed, the shorter the time it takes for the cardboard to pass the pusher, requiring a faster response; different cardboard materials respond differently to external forces, with flexible materials requiring a longer application time.

[0195] Adjusting the thrust is another key parameter to ensure the correction effect. The thrust is adjusted by controlling the air pressure of the push rod through a proportional pneumatic valve. The thrust is directly proportional to the vertical deviation angle, and the basic calculation formula is F = F0·(1+0.5·(θ-θ1) / (θ2-θ1)), where F0 is the base thrust (usually 50-80N), θ is the vertical deviation angle, θ1 is the first threshold, and θ2 is the second threshold. This formula ensures that a smaller force is applied to slightly tilted cartons to avoid overcorrection, while a larger force is applied to severely tilted cartons approaching the second threshold to improve the correction success rate. The thrust is also adjusted according to the weight and material of the cartons; for example, for heavy cartons or harder materials, the thrust is increased by 20-30%.

[0196] The generation and transmission of control signals are crucial steps in executing the correction action. Pneumatic components are controlled by drivers at the device level. The control flow is as follows: The central control system generates correction commands, including push rod selection, thrust magnitude, and extension time; the commands are transmitted to the pneumatic controller via a fieldbus (such as Profinet or EtherCAT), with a transmission delay controlled within 10ms; after receiving the commands, the pneumatic controller outputs a control voltage through a digital-to-analog converter, driving the proportional valve to open to the specified position; the proportional valve adjusts the airflow, controlling the cylinder to extend with the specified force and speed; simultaneously, pressure and displacement sensors monitor the actual execution status, forming a closed-loop control; after execution, the execution results are fed back to the central control system for evaluating the correction effect and recording historical data.

[0197] The second-level control command is a comprehensive instruction set, including a correction control section and a gripping compensation section. The correction control section includes: pusher selection parameters, specifying which pushers(s) to activate; push force parameters, specifying the push force of each pusher; timing parameters, specifying the extension and retraction times; and feedback requirements, specifying the status information to be monitored. The gripping compensation section is similar to the first-level control command, but the compensation amount is adjusted appropriately to account for changes in posture after correction. Based on historical data and the current status, it estimates the posture of the cardboard box after correction and calculates the corresponding gripping compensation in advance, ensuring successful packing even if the correction effect is imperfect.

[0198] Third, when the vertical deviation angle of the cardboard box exceeds the second threshold (usually 15°) or the system repeatedly attempts to correct the same cardboard box but fails each time, it is considered a serious abnormal state, requiring the initiation of a third-level response. The criteria for continuous correction failure are: more than three correction attempts on the same cardboard box (with intervals less than 1 second), but the vertical deviation angle after each correction still exceeds the first threshold. This situation usually indicates that the cardboard box is in a non-standard state (such as deformation, damage, or foreign object interference) or that the correction device is malfunctioning, requiring manual intervention.

[0199] Slowing down the conveyor belt is the primary measure in a Level 3 response, aiming to slow the spread of the problem and buy time for manual intervention. The central control system sends speed adjustment commands to the frequency converter, reducing the conveyor belt speed to 30%-50% of its normal speed. The deceleration process is gradual to avoid sudden deceleration causing impact and vibration on other cartons. The duration of deceleration is automatically adjusted based on the severity of the abnormality and the speed of manual intervention, and can last up to 5-10 minutes. If the abnormality is resolved within this time, the system will automatically return to normal speed.

[0200] Alarm prompts are a crucial means of alerting operators. A multi-layered alarm mechanism is activated: visual alarms use a three-color alarm light (red indicates a serious anomaly requiring immediate action, yellow indicates a warning requiring prompt action, and green indicates normal system operation), flashing at a 1Hz frequency to emphasize urgency; audible alarms employ a distinguishable dual-tone alarm with a volume of approximately 80dB and a frequency of 2-3kHz (the most sensitive area for the human ear), using an intermittent buzzing mode (1 second continuous, 2-second interval) to avoid the desensitization effect caused by prolonged continuous alarms; tactile alarms are achieved through vibration devices (such as specially designed wristbands) worn by operators, ensuring timely alarm detection even in noisy environments. The alarm system supports zoned and hierarchical control, triggering alarms only in relevant areas to avoid interfering with other production areas.

[0201] Real-time display of the human-machine interface is crucial for assisting operators in quickly locating and resolving problems. When a Level 3 response is triggered, an anomaly alarm window pops up on the monitoring interface, containing the following: a real-time image of the abnormal cardboard box, highlighting the problem area (such as a severely tilted section); cardboard box location information, displayed precisely on the conveyor line diagram; anomaly type description, such as "severely tilted more than 15°" or "three consecutive correction failures"; suggested handling methods, such as "check if the cardboard box is deformed" or "check if the push rod pressure is normal"; and emergency operation buttons, such as "pause conveying" or "skip this cardboard box". The interface design follows ergonomic principles, using high-contrast colors, clear icons, and concise text to ensure quick understanding and operation even in emergency situations.

[0202] Automatic recording of anomaly data is fundamental to continuous system improvement. Each trigger of a Level 3 response records detailed anomaly information: time information, including the time of occurrence, duration, and processing time; cardboard box information, including dimensions, material type, production batch, and identification code; anomaly parameters, including vertical deviation angle, horizontal offset, and attitude change curve; system status, including equipment parameters, environmental conditions, and operation logs; and processing results, including operator actions and final solutions. This data is stored in a structured format in the database, supporting subsequent statistical analysis, trend mining, and root cause analysis. The system automatically calculates the frequency, distribution patterns, and correlation factors of various anomalies, generating anomaly analysis reports to help managers identify areas for system improvement.

[0203] The third-level control command is a comprehensive emergency response command set, comprising four parts: equipment control, alarm triggering, display control, and data logging. The equipment control part includes: conveyor belt speed adjustment commands, specifying the deceleration ratio and recovery conditions; equipment status switching commands, such as activating standby devices or switching operating modes; and system protection commands, ensuring equipment safety in abnormal situations. The alarm triggering part includes: alarm device selection, specifying which alarm devices to activate; alarm mode settings, specifying the mode and intensity of audible and visual alarms; and alarm duration control, specifying the conditions for automatic alarm cancellation. The display control part includes: abnormal information push, pushing abnormal data to designated display devices; interface element control, specifying the pop-up window position, size, and content; and interactive control, defining available interactive operations and permissions. The data logging part includes: recording content definition, specifying the data items and formats to be recorded; storage location specification, determining the database table and file path for data storage; and analysis triggering, specifying whether to initiate real-time analysis and report generation.

[0204] Fourth, the hierarchical linkage control command is a composite command structure that integrates three levels of control commands and establishes the linkage relationship between them. This design allows the system to flexibly switch response strategies according to real-time conditions, forming a continuous and smooth control flow and avoiding system instability that may be caused by isolated control actions. The core feature of the hierarchical linkage control command is that the "level can be upgraded or downgraded," and the system dynamically adjusts the response level based on real-time feedback.

[0205] Instruction integration is the first step in forming hierarchical linkage control instructions. The three levels of instructions are organized hierarchically into a tree structure: the top level is a response level selector, which selects the appropriate response level based on the paper tray attitude parameters and system state; the middle level contains the specific instruction content for each level of response, including the aforementioned parameters and control signals; the bottom level is a post-execution feedback processor, which adjusts subsequent responses based on the execution results. The inheritance relationship between instructions ensures that higher-level responses include key functions of lower-level responses. For example, although the third-level response focuses on exception handling, it still retains the function of grasping compensation, ensuring the continuity of system functionality.

[0206] The linkage mechanism is the core of the hierarchical linkage control command, defining the transition conditions and processes between different levels. Upward linkage conditions include: the vertical deviation angle exceeds the level threshold; execution feedback indicates poor response at the lower level; multiple consecutive identical anomalies indicate a persistent problem. Downward linkage conditions include: the corrected vertical deviation angle drops to a lower level range; the abnormal situation is effectively resolved; and the system returns to normal after manual intervention by the operator. The linkage process employs a smooth transition strategy to avoid frequent oscillations in response levels; for example, level switching is only triggered when the parameter stabilizes within the new range for a certain period (e.g., after three consecutive detections).

[0207] Timing control is crucial for ensuring the orderly execution of hierarchical linkage commands. An event-driven timing control mechanism is employed, defining key events and their processing logic: trigger events, such as detecting a cardboard box entering the field of view; status update events, such as receiving new attitude parameters; execution completion events, such as the end of a correction action; and abnormal events, such as detecting severe deviations or equipment malfunctions. Each event is associated with a specific processing flow, including condition evaluation, command generation, and feedback processing. The timing controller maintains an event queue to ensure events are processed according to priority and time order, avoiding command conflicts and resource contention.

[0208] Execution monitoring serves as the feedback loop for hierarchical, interconnected control commands, responsible for evaluating execution effectiveness and triggering necessary adjustments. The monitoring process includes: real-time status monitoring, continuously collecting system status data through various sensors; execution result evaluation, comparing expected and actual results; anomaly detection, identifying abnormal phenomena during execution; and adjustment decision-making, determining whether to adjust the response level or parameters based on the evaluation results. The monitoring system employs a predictive-comparison-adjustment closed-loop control model, continuously optimizing control effectiveness and improving the system's adaptability and robustness.

[0209] Through the above design and implementation, a complete hierarchical linkage control command system has been formed, which can take corresponding response measures according to different carton postures, from simple gripping position compensation to complex anomaly handling, to ensure the efficient and reliable execution of carton posture correction and packing process.

[0210] Example 6:

[0211] In this embodiment, the execution of the corresponding response strategy further includes security response control based on random polynomial time delay, including:

[0212] Based on the verticality deviation angle in the paper box posture parameters, a time delay function containing a deterministic polynomial part and a controlled random disturbance term is designed, wherein the controlled random disturbance term follows a normal distribution, thus obtaining a delay control model;

[0213] Based on the aforementioned delay control model, a dynamic delay is added to the calculation of the compensation amount for the gripping position of the packing robot in the first-level control command, so that the compensation action is delayed. The delay time changes nonlinearly with the verticality deviation angle, resulting in a first-level response command with random delay.

[0214] Based on the delay control model and the first-level response command, the push rod extension time in the second-level control command is adjusted, and the delay amount related to the verticality deviation angle is increased to obtain a second-level response command with random delay;

[0215] Based on the vertical deviation angle and horizontal offset in the paper box posture parameters, a polynomial boundary function is introduced to dynamically adjust the first threshold and the second threshold, and a safety status assessment model is developed. When the safety status assessment value is lower than the safety threshold, the model parameters of the delay control model are automatically adjusted. Based on the first-level response command, the second-level response command, and the third-level control command, a safety-enhanced hierarchical linkage control command is obtained.

[0216] Specifically, firstly, stochastic polynomial time delay is a specialized technique for enhancing system security. Its core idea is to introduce controlled random time delays into the response process of the control system, giving the system behavior a degree of uncertainty, thereby enhancing the system's resistance to external interference and malicious attacks. In the cardboard box posture correction system, this technique, by introducing randomness in the time dimension, prevents potential attackers from inferring and interfering with the system's behavior by observing its regular responses.

[0217] The deterministic polynomial part of the delay function is a fundamental component of the system response delay, reflecting the basic relationship between the delay time and the vertical deviation angle. This part adopts a quadratic polynomial form: T poly (θ) = a0 + a1·θ + a2·θ 2Where θ is the vertical deviation angle, and a0, a1, and a2 are coefficients. The coefficients are selected based on two principles: first, when the deviation angle is small, the delay time is relatively long, allowing the system sufficient time to assess the situation; second, as the deviation angle increases (indicating a more urgent situation), the delay time gradually decreases, accelerating the system's response speed. Typical coefficient values ​​determined through experiments are: a0 = 200ms (basic delay time), a1 = -10ms / degree (first-order coefficient, reflecting a linear decreasing trend), and a2 = -0.5ms / degree² (second-order coefficient, reflecting an accelerated decreasing trend). This set of coefficients results in a delay of approximately 180ms when the deviation angle is 2°, and a delay decreasing to approximately 50ms when the deviation angle approaches 15°, reflecting the principle of "rapid response in emergencies."

[0218] The controlled random disturbance term is a randomized component of the delay function, introducing uncertainty into the system. This term follows a restricted normal distribution: T rand = ε·σ·T poly (θ), where ε is a random variable following a standard normal distribution N(0,1), σ is the standard deviation coefficient controlling the intensity of the disturbance, and T poly (θ) is the value of the deterministic polynomial part. The standard deviation coefficient σ is typically set in the range of 0.1-0.2, meaning that the magnitude of the random disturbance is approximately 10%-20% of the deterministic delay time. For example, when the deterministic delay is 100ms, the actual delay may vary randomly within the range of 80-120ms. The system also limits the random disturbance to ensure that the total delay time is within a reasonable range: T min ≤ T poly (θ) + T rand ≤ T max T min It is the minimum allowable latency (typically 30ms, considering hardware response limitations), T max This is the maximum allowable latency (usually 300ms, to avoid excessively affecting system real-time performance).

[0219] The normal distribution is a probability distribution model for random perturbations. Its characteristic is that the probability density is highest near the mean, decreasing towards both sides, with most samples concentrated within a certain range around the mean. There are three main reasons for choosing the normal distribution as the perturbation model: First, the normal distribution is ubiquitous in nature and artificial systems, making system behavior appear more natural; second, the normal distribution has favorable mathematical properties, facilitating theoretical analysis and implementation; and finally, by controlling the standard deviation parameter, the degree of randomness can be precisely adjusted, balancing security and system performance requirements. The system generates normally distributed random numbers using the Box-Muller transformation method, which can efficiently convert a uniform distribution into a normal distribution, meeting the performance requirements of real-time systems.

[0220] The complete delay control model integrates deterministic and stochastic components: T delay (θ) = T poly (θ) +T rand The model is subject to the aforementioned upper and lower bound constraints. It also incorporates an adaptive adjustment mechanism that dynamically adjusts parameters based on system operating status and environmental conditions. For example, it increases randomness when potential interference is detected; reduces the base delay time when the system load is high; and slightly adjusts the polynomial coefficients when batch processing similar cardboard boxes to prevent predictable patterns. This adaptive characteristic enables the delay control model to minimize the impact on normal system function while ensuring safety.

[0221] The delay control model employs a layered design: the bottom layer consists of a high-precision timer providing millisecond-level time control; the middle layer comprises a random number generator and a delay calculator, responsible for generating specific delay times based on the model; and the top layer is a delay manager, responsible for coordinating the delay strategies of different control commands to ensure the consistency and security of the overall system behavior. Model parameters are stored in an encrypted configuration file, accessible and modifiable only by authorized system components, preventing security risks caused by unauthorized parameter modification.

[0222] Secondly, a delay control model is applied to the first-level control commands, primarily targeting the calculation and execution process of the gripping position compensation for the packing robot. After calculating the compensation amount, the corresponding adjustment is not executed immediately; instead, a dynamically calculated delay time is added before sending the execution command. This delayed execution mechanism makes the system response time non-deterministic, increasing the unpredictability of the system behavior.

[0223] The dynamic delay calculation is based on the aforementioned delay control model, but with specific optimizations for the first-level response. The first-level response deals with a slightly tilted cardboard box, a relatively non-urgent situation, therefore the base delay time is relatively long, typically set to a0 = 200-250 ms. As the vertical deviation angle approaches the first threshold (5°), the delay time gradually decreases, reflecting the principle of faster system response to cardboard boxes approaching critical states. Simultaneously, the standard deviation coefficient of the random disturbance is relatively large, typically set to σ = 0.15-0.2, introducing more pronounced temporal randomness.

[0224] The implementation of delayed execution of compensation actions involves the collaborative work of multiple system components. After the vision system detects the cardboard box and calculates its posture parameters, the central control system calculates the required gripping position compensation amount. However, instead of immediately sending this command to the robot controller, it places the instruction in a delay queue and calculates a specific delay time based on a delay control model. A timer is started, and only after the specified delay time is reached is the instruction retrieved from the queue and sent to the robot controller. To ensure that the delay does not affect functionality, the compensation amount is calculated in advance, allowing sufficient time buffer to ensure that even with the added delay, the instruction arrives in time before the robot needs to perform the gripping action.

[0225] The nonlinear variation of delay time is key to enhancing the unpredictability of a system. Due to the nonlinear characteristics of the polynomial part and the presence of random perturbations, similar vertical deviation angles can produce significantly different delay times. For example, two cardboard boxes with deviation angles of 2.0° and 2.1° should theoretically have very similar compensation amounts and response times, but in reality, the system may exhibit delay differences of 20-30 ms or even greater. This nonlinearity and randomness make it difficult for external observers to accurately predict when the system will respond to a specific state, increasing the complexity and security of the system's behavior.

[0226] The Level 1 response command with random delay adds timing control attributes to the original Level 1 control command. These attributes include: a calculation timestamp, recording the exact time the command was generated; a delay parameter, containing the calculated delay time and a random factor; an execution time window, specifying the effective execution time range of the command; exceeding this range invalidates the command; and a synchronization flag, used to ensure the correct execution order of multiple related commands. These additional attributes enable the control system to precisely manage the timing dimension of command execution, improving safety while ensuring functional integrity.

[0227] Then, based on the delay control model and the first-level response command, the push rod extension time in the second-level control command is adjusted, and the delay amount related to the verticality deviation angle is increased to obtain a second-level response command with random delay.

[0228] The second-level control command primarily targets moderately tilted cardboard boxes. Its core action is activating a pneumatic actuator for physical correction. The actuator extension time is adjusted, and a dynamically calculated delay is added, introducing some uncertainty to the correction action's execution time. Unlike the first-level response, the second-level response handles more urgent situations; therefore, the delay strategy is adjusted to ensure safety without unduly impacting the correction effect.

[0229] The adjustment of the push rod extension time is based on two parts: basic adjustment and random delay. Basic adjustment is similar to the calculation in the aforementioned second-level control command, determining the basic extension time of the push rod based on the verticality deviation angle, conveyor belt speed, and cardboard material. Random delay is based on a delay control model, but with different parameter configurations: the basic delay time a0 is shorter, typically set to 100-150ms, reflecting the higher urgency of the second-level response; the first-order coefficient a1 is larger, causing the delay time to decrease more rapidly as the deviation angle increases; the standard deviation coefficient σ is smaller, typically set to 0.1-0.15, introducing moderate randomness without excessively affecting the correction effect.

[0230] The delay calculation takes into account the specific impact of the vertical deviation angle. When the deviation angle is close to the first threshold (just entering the second-level response range), the delay is relatively large because the situation is not relatively urgent; when the deviation angle is close to the second threshold (approaching the third-level response range), the delay is significantly reduced to ensure a rapid response to severely tilted cartons. The system also dynamically adjusts the delay parameters based on historical data. For example, if it finds that the success rate of carton correction is low within a certain angle range, the delay time within that range will be reduced accordingly to improve response speed and correction effect.

[0231] The implementation mechanism for random delays is similar to that of the first-level response, but with some optimizations in details. A high-priority timer is used to manage the delayed execution of the second-level response, ensuring more precise timing control. The delay instruction queue is prioritized, with instructions for the paper tray having higher priority due to larger vertical deviation angles, and these instructions are executed first in resource contention situations. The system also implements a delay cancellation mechanism; if a significant change in the paper tray's state is detected during the delay period (such as self-correction due to external factors or further tilting), the original delay is immediately cancelled, and the response strategy is recalculated.

[0232] The Level 2 response command with random delay extends the functional attributes of the original Level 2 control command. New attributes include: pre-delay state recording, capturing the carton's attitude parameters at the time of command generation; post-delay state prediction, estimating the possible state of the carton during delayed execution; parameter adjustment, specifying how to adjust execution parameters based on the difference between the actual and predicted states during execution; and failure handling strategy, defining how to handle situations where the command times out or environmental conditions change, rendering the command inapplicable. These extensions enable the Level 2 response command to adapt to uncertainties in delayed execution environments, maintaining the effectiveness of the response.

[0233] Finally, the polynomial boundary function is a mathematical model used to dynamically adjust the decision threshold of a system. Its core idea is to transform the originally fixed threshold into a dynamic boundary influenced by multiple factors. In this system, the polynomial boundary function is used to adjust the first and second thresholds, making the boundary of the hierarchical response more flexible and precise, and adaptable to different cardboard box characteristics and environmental conditions.

[0234] The dynamic adjustment of the first threshold uses a bivariate polynomial function: θ1(Δx, H / W) = θ 10 + c1·Δx + c2·(H / W-1) + c3·Δx·(H / W), where θ 10 Here, Δx is the base threshold (typically 5°), H / W is the aspect ratio of the cardboard box, and c1, c2, and c3 are coefficients. This function reflects the following patterns: the larger the horizontal offset, the smaller the allowable vertical deviation angle (c1 is negative, typically -0.05° / mm); the larger the aspect ratio (the longer and thinner the cardboard box), the smaller the allowable vertical deviation angle (c2 is negative, typically -1.5°); and the interaction effect between horizontal offset and aspect ratio (c3 is typically -0.01° / mm). For example, for a standard cardboard box (H / W=1.5) with no horizontal offset, the first threshold remains at 5°; however, for a long and thin cardboard box (H / W=2.5) with a 10mm horizontal offset, the first threshold may drop to about 2.5°, reflecting the characteristic that the cardboard box is more prone to instability in this case.

[0235] The dynamic adjustment of the second threshold uses a similar polynomial function, but with different coefficients and base values: θ²(Δx, H / W) = θ 20 + d1·Δx + d2·(H / W-1) + d3·Δx·(H / W), where θ 20 The first threshold is the base threshold (typically 15°). The second threshold is usually adjusted more significantly than the first, reflecting a higher sensitivity of the judgment of severe abnormal conditions to the characteristics of the carton. It is also important to ensure a reasonable interval (typically at least 5°) between the two thresholds to avoid frequent oscillations in the grading response due to the thresholds being too close together.

[0236] The safety status assessment model is a comprehensive evaluation system that considers multiple factors to quantify the current safety status of the system and guide the parameter adjustment of the delay control model. The assessment model considers the following core factors: response triggering patterns, analyzing the triggering frequency and distribution patterns of responses at all levels to identify possible regularities; parameter change trends, monitoring the statistical distribution and change trends of vertical deviation angle and horizontal offset; abnormal event frequency, recording and analyzing the occurrence frequency and time distribution of various abnormal events; and environmental condition changes, monitoring the potential impact of changes in the production environment (such as temperature, humidity, vibration, etc.) on the system.

[0237] The safety status assessment value is calculated using a weighted scoring method: S = w1·S1 + w2·S2 + w3·S3 + w4·S4, where S1 to S4 are the scores of each factor (ranging from 0 to 1, with 1 representing the safest), and w1 to w4 are the corresponding weights (summing up to 1). The calculation of each factor's score is based on a specific algorithm. For example, the response triggering mode score S1 uses entropy measurement to quantify the randomness of the response sequence; the higher the entropy value (the more random the pattern), the higher the score. The parameter change trend score S2 uses statistical variance analysis to detect the uniformity and stability of parameter distribution. The abnormal event frequency score S3 is based on the frequency and clustering of abnormal events; the lower the frequency and the more dispersed the distribution, the higher the score. The environmental condition score S4 is based on the stability and expected range of environmental parameters. The safety threshold is typically set at 0.7. When the assessment value is below this threshold, it indicates that the system may be in an unsafe state, requiring adjustment of the delay control model parameters to enhance safety.

[0238] The automatic adjustment of delay control model parameters is an adaptive adjustment based on the safety status assessment results. When the assessed value is lower than the safety threshold, the following parameters are adjusted according to the specific situation: random disturbance intensity, by increasing the standard deviation coefficient σ to introduce stronger randomness; basic delay time, by adjusting the coefficient a0 to change the basic level of delay; and nonlinearity, by adjusting the higher-order coefficients of the polynomial to enhance the nonlinear variation of the delay time. The adjustment strategy is based on the principle of "targeted reinforcement of problem factors," making targeted adjustments to the main problem factors found in the assessment. For example, if a clear pattern is found in the response triggering mode (low S1 score), the random disturbance intensity is increased; if the parameter distribution is found to be too concentrated (low S2 score), the nonlinearity is enhanced to expand the distribution range of the delay time.

[0239] The enhanced security hierarchical linkage control commands integrate the aforementioned Level 1, Level 2, and Level 3 response commands, and add a security enhancement layer to form a complete security enhancement control system. The security enhancement layer includes the following components: a dynamic threshold manager, responsible for real-time calculation and updating of response thresholds; a delay policy coordinator, ensuring consistency in delay policies across all response levels; a security status monitor, continuously assessing the system's security status and triggering necessary adjustments; and an adaptive parameter adjuster, dynamically adjusting delay model parameters based on monitoring results. These components work together to improve system security and anti-interference capabilities while ensuring normal system functionality.

[0240] The execution flow of hierarchical linkage control commands includes multiple safety mechanisms: command verification to ensure the source of the command is trustworthy and its content is complete; execution time check to verify whether the command is within the valid time window; state consistency check to confirm that the current state is consistent with the command's expectation; resource availability check to verify whether the resources required for execution are available; and a rollback mechanism to safely roll back to a stable state in the event of execution anomalies. These mechanisms ensure stable and secure operation even in complex delayed execution environments.

[0241] By comprehensively applying the above technologies, a safe response control based on random polynomial time delay is realized. While maintaining the original functions, the system's safety and anti-interference capabilities are significantly enhanced, making the carton posture correction and packing process more reliable and safe.

[0242] Example 7:

[0243] In this embodiment, the method of using a deep learning model to identify the corner coordinates of the cardboard box further includes robust pose recognition with residual noise enhancement, comprising:

[0244] Based on the algorithm layer in the closed-loop control system architecture, a residual noise linear layer is inserted into the feature extraction network of the deep learning model. The residual noise linear layer enhances the feature expression capability by superimposing a noise function on the linear transformation result, thus obtaining a feature extraction network with noise enhancement capability.

[0245] Based on the feature extraction network with noise enhancement capability, a strategy for dynamically adjusting noise intensity with training progress is designed. When the number of training iterations is less than 30% of the total number of iterations, high-intensity noise with a noise standard deviation of 0.1 is used; when the number of training iterations is greater than 70% of the total number of iterations, low-intensity noise with a noise standard deviation of 0.01 is used; and when the number of training iterations is between 30% and 70% of the total number of iterations, a linearly decaying noise standard deviation is used, thus obtaining an adaptive noise scheduling mechanism.

[0246] The model is trained based on the aforementioned adaptive noise scheduling mechanism to obtain a lightweight attitude detection model with enhanced robustness.

[0247] Specifically, firstly, the residual noise linear layer is a specially designed neural network layer. Its core idea is to introduce controlled noise interference on the basis of standard linear transformation to improve the network's robustness to input variations. In the task of cardboard box corner detection, this technique can effectively cope with interference factors such as changes in image quality, changes in lighting conditions, and partial occlusion, improving the model's stability and accuracy in complex real-world environments.

[0248] The basic operation of a standard linear layer (fully connected layer) is a linear transformation: y = Wx + b, where x is the input vector, W is the weight matrix, b is the bias vector, and y is the output vector. This linear transformation learns the mapping relationship between input features and output targets during training, but it is prone to oversensitivity to noise and small changes in the training data, resulting in insufficient generalization ability.

[0249] The innovation of the residual noise linear layer lies in the introduction of a noise function, mathematically expressed as: y = Wx + b + N(x), where N(x) is the noise function, generating noise dependent on the input x. The noise function is designed using an input-dependent approach: N(x) = η⊙x, where η is a randomly generated noise matrix, and ⊙ represents element-wise multiplication (Hadamard product). The elements of the noise matrix η follow a normal distribution with a mean of 0 and a standard deviation of σ, where σ is a hyperparameter controlling the noise intensity. This design ensures that the noise magnitude is proportional to the amplitude of the input features, moderately disturbing important features (large amplitudes) and relatively less disturbing weak features (small amplitudes), thus maintaining the relative importance of the features.

[0250] Residual connections are another key feature of this layer, ensuring smooth learning of the network. Similar to residual connections in ResNet, the complete expression for the residual noisy linear layer is: y = x + (Wx + b + N(x)), where the first term x is the identity mapping (shortcut connection), and the second term is the noisy linear transformation. This design allows the network to maintain its basic information transfer capability even under strong noise conditions, avoiding gradient problems and promoting a more stable training process.

[0251] In the model implementation, residual noise linear layers are strategically inserted at multiple locations within the feature extraction network. A typical configuration is as follows: low-noise-intensity layers are inserted in shallow networks (closer to the input) to help the model resist noise and variations in the input image; medium-noise-intensity layers are inserted in mid-level networks to enhance the diversity of feature representations; and high-noise-intensity layers are inserted in deep networks (closer to the output) to encourage the model to learn more robust high-level feature representations. This layered configuration enables the model to be robust to interference at each level of abstraction.

[0252] The enhanced feature representation ability is reflected in three aspects: enhanced feature diversity, as noise interference forces the network to explore more possible feature combinations, reducing reliance on a single feature; improved feature robustness, as the features learned by the network are more stable to input changes and noise, reducing the risk of overfitting; and enhanced feature discriminability, as the network can better distinguish key features from background noise, improving detection accuracy.

[0253] In cardboard box corner detection tasks, residual noise linear layers are particularly helpful in handling the following challenges: differences in surface reflectivity of cardboard boxes made of different materials; variations in lighting conditions in the production environment; interference from printed patterns on the cardboard box surface; slight image blurring caused by conveyor belt vibration; and the effects of partial occlusion and shadows. By introducing controlled noise during training, the model learns to adapt to these interference factors, exhibiting higher stability in real-world deployment environments.

[0254] The feature extraction network with noise enhancement capabilities is enhanced based on the original MobileNetV3 backbone network using the aforementioned techniques. The improved network significantly improves the robustness and adaptability of feature representation while maintaining computational efficiency, providing a more reliable feature foundation for subsequent corner detection tasks. Experiments show that adding a residual noise linear layer slightly improves the model's accuracy on the standard test set (approximately 1-2%), while the performance improvement on the noisy test set is more significant (approximately 5-8%), demonstrating the effectiveness of this technique in enhancing model robustness.

[0255] Secondly, the adaptive noise scheduling mechanism is a strategy that dynamically adjusts the noise intensity during training. Its design is similar to the simulated annealing algorithm, introducing greater randomness in the early stages of training to promote global exploration, and gradually reducing randomness as training progresses to focus on local optimization. This strategy can balance the model's exploration ability and convergence, improving the model's robustness while ensuring that training can successfully converge to a good solution.

[0256] The high-intensity noise phase in the early training stage (iterations < 30%) is the exploration phase. At this stage, a noise standard deviation σ = 0.1 is set, meaning the noise amplitude in the residual noise linear layer is relatively large, potentially reaching ±20% of the input feature amplitude (approximately within the 2σ range). High-intensity noise serves several purposes: it promotes extensive exploration, forcing the network to try various feature combinations and avoiding premature immersion in local optima; it enhances regularization, significantly reducing the risk of overfitting, especially when training data is limited; and it improves robustness, enabling the network to learn strategies against various disturbances from the outset. During this phase, the network's loss function may fluctuate significantly, and the learning process may appear unstable, but this is precisely the crucial process for cultivating the network's robustness.

[0257] The linear decay phase during the middle of training (between 30% and 70% of the iterations) is a transition phase. The noise standard deviation decreases linearly from 0.1 to 0.01, calculated using the formula σ. t = σ high - (σ high - σ low )·(t - T start ) / (T end -T start), where t is the current iteration number, T start It is the decay start time (30% of the total number of iterations), T end It is the decay end time (70% of the total number of iterations), σ high =0.1, σ low =0.01. The advantages of linear decay rather than step change are: avoiding training instability caused by sudden changes in noise intensity; giving the network enough time to adapt to gradually changing conditions; and simulating scenarios where interference gradually decreases in the real world, which is more in line with actual application environments. In this stage, the network's loss function gradually stabilizes, performance metrics begin to improve continuously, and the model gradually finds a parameter configuration that balances robustness and accuracy.

[0258] The low-intensity noise phase in the later stages of training (iterations > 70%) is the fine-tuning phase. At this point, the noise standard deviation σ is set to 0.01, and the noise amplitude is relatively small, typically not exceeding ±2% of the input feature amplitude. The purpose of low-intensity noise is to: maintain moderate interference to prevent the model from overfitting the training data; provide a continuous but gentle regularization effect; and allow the model to fine-tune parameters, improving accuracy under standard conditions. At this stage, the network has established basic resistance to noise; the focus now is on improving performance under normal conditions, achieving the optimal balance between robustness and accuracy.

[0259] Dynamic adjustments are implemented using a callback mechanism. Before each training batch begins, the current training progress percentage p = current iteration / total iterations Then, based on the progress, the current noise standard deviation to be used is determined. The calculated standard deviation value is set into each residual noise linear layer through the model parameter update interface. Visual monitoring of noise scheduling is also implemented, displaying the noise intensity change curve and current training status in real time during training, helping researchers understand and adjust the training process.

[0260] The adaptive noise scheduling mechanism also includes feedback adjustment capabilities, enabling dynamic fine-tuning of scheduling parameters based on validation set performance. The system periodically (e.g., every 5 epochs) evaluates model performance on the validation set, paying particular attention to performance differences under varying noise conditions. If the model's adaptability to noise is found to be insufficient (e.g., excessive performance differences between noisy and noise-free conditions), the system extends the high-intensity noise phase or slows down the decay rate; if the model performs poorly under standard conditions, the system accelerates the decay to a low-noise phase, increasing the time for fine-tuning. This feedback mechanism allows noise scheduling to adapt to the needs of specific datasets and models, further improving training effectiveness.

[0261] Overall, the adaptive noise scheduling mechanism, through careful design of noise intensity throughout the training cycle, helps the network gradually build up its resistance to various interferences, while ensuring that the model can converge to a parameter configuration that combines accuracy and robustness, providing a solid foundation for the accurate and reliable detection of cardboard box corners.

[0262] Finally, the model training process is a crucial step in applying the aforementioned residual noise linear layer and adaptive noise scheduling mechanism to actual network learning. The training employs a carefully designed process and strategy to ensure that the model can fully utilize noise enhancement techniques to improve robustness while maintaining lightweight characteristics and high accuracy.

[0263] The training data preparation was enhanced based on Example 4, with a particular emphasis on diversity and coverage. In addition to the aforementioned data augmentation techniques, the system added targeted noise samples: synthetic noise samples, generated by adding various types and intensities of noise (Gaussian noise, salt-and-pepper noise, speckle noise, etc.) to standard images; environmental interference samples, simulating various real-world environmental disturbances such as lighting variations, reflections, shadows, and partial occlusion; and extreme examples, including low contrast, high exposure, and severe tilting boundary conditions. These enhanced samples ensure that the model is exposed to various possible disturbances during training, building comprehensive resilience.

[0264] The training configuration uses the same hardware environment as the base model, but the training strategy has been adjusted. The batch size is set to 24, slightly smaller than the 32 of the base model, to accommodate the additional memory requirements introduced by the residual noise linear layers. A cosine annealing strategy is used for the learning rate, with an initial learning rate of 0.001, which is adjusted in conjunction with the noise scheduling cycle: a larger learning rate is used during periods of high noise intensity to promote exploration; the learning rate decreases accordingly as the noise intensity decreases to facilitate fine-tuning. The total number of training epochs has been increased to 120 epochs (compared to 100 epochs for the base model), providing the model with more learning time, especially since more iterations are needed to build robustness under noisy conditions.

[0265] The loss function design optimizes noise sensitivity. The system combines an improved smoothed L1 loss with noisy-weighted keypoint similarity loss (OKS). The smoothed L1 loss is insensitive to outliers, mitigating the impact of noise on loss calculation; the noisy-weighted OKS loss, through an adaptive weighting mechanism, reduces loss sensitivity in high-noise phases and restores normal sensitivity in low-noise phases, helping the model maintain appropriate learning signal strength across different training stages. Dynamic adjustment of loss weights is synchronized with noise scheduling, ensuring consistency and stability during training.

[0266] The training process is monitored using a multi-metric tracking method, recording and visualizing the following key metrics in real time: corner detection accuracy under standard conditions to evaluate the model's basic performance; corner detection accuracy under noisy conditions to evaluate the model's robustness; the difference in accuracy between the two conditions to evaluate the model's adaptability to noise; detection accuracy across different tilt angle ranges to evaluate the model's performance in various poses; and the statistical distribution of activation values ​​in each layer to monitor the impact of noise on the network's internal state. These metrics are visually presented through visualization tools, helping researchers gain a comprehensive understanding of training progress and model behavior.

[0267] The early stopping strategy and model selection employ a comprehensive evaluation method. Unlike traditional early stopping based solely on validation set accuracy, the system introduces a "robustness-aware" early stopping criterion: the early stopping mechanism is triggered when the weighted sum of the accuracy on the standard validation set and the accuracy on the noisy validation set does not improve for 10 consecutive epochs. Model selection also considers not only standard accuracy but also the checkpoint that best balances robustness and accuracy, typically the model with the highest weighted sum of standard and noisy accuracy. This strategy ensures that the final selected model possesses both high accuracy and good anti-interference capabilities.

[0268] Post-training evaluation employs a multi-dimensional testing approach to comprehensively assess the model's performance across various aspects: accuracy testing evaluates corner position accuracy and pose parameter calculation accuracy on a standard test set; robustness testing assesses performance degradation on test sets with added noise of different types and intensities; scene adaptability testing evaluates performance on specific test sets under different lighting conditions, backgrounds, and cardboard types; speed testing measures inference time and resource consumption on target hardware; and boundary testing evaluates model performance under extreme conditions (such as severe tilt or partial occlusion). These test results comprehensively reflect the model's performance characteristics and application potential.

[0269] The robust and lightweight attitude detection model is the final product of the training process, possessing the following core characteristics: high accuracy, achieving a corner detection accuracy of 98.5% on the standard test set, and calculating attitude parameters (vertical deviation angle and horizontal offset) with an accuracy better than 99%; strong robustness, with performance degradation of less than 3% on a test set with added moderate noise (σ=0.05), far superior to the 8-10% degradation of the base model; high efficiency, with single-frame processing time remaining within 40ms on the target industrial PC, meeting real-time processing requirements; and strong adaptability, performing well on different materials, sizes of cardboard boxes, and environmental conditions. These characteristics enable the model to stably and reliably perform cardboard box corner detection tasks in complex industrial environments, providing accurate positional information for subsequent attitude correction and packing operations.

[0270] Model deployment and practical application are the final steps, transforming the trained model into a practical component of the production system. Model quantization techniques are employed to convert floating-point models into integer or half-precision floating-point models, further reducing computational resource requirements and improving inference speed. A hot-swap mechanism for models is implemented, allowing model updates without interrupting system operation. Adaptive inference strategies are developed to dynamically adjust inference parameters based on current workload and image quality, balancing speed and accuracy requirements. A performance monitoring mechanism is established to continuously evaluate the model's performance in real-world environments and promptly identify potential problems. These deployment optimizations ensure that the model can run efficiently and stably in real-world production environments, fully leveraging its accuracy and robustness advantages.

[0271] Example 8:

[0272] In this embodiment, the model training based on the adaptive noise scheduling mechanism further includes:

[0273] Based on the adaptive noise scheduling mechanism, the corner detection loss function of the deep learning model is improved into a weighted combined loss function that includes the original loss function and a smooth loss function that introduces Gaussian blur, thus obtaining the smooth loss function.

[0274] Based on the smoothing loss function, the model parameters of the deep learning model are updated using an exponential moving average method, so that the model parameters at the current time are determined by a weighted combination of the parameters at the previous time and the optimized parameters of the current batch, thus obtaining the soft target network update mechanism.

[0275] Model training is completed based on the soft target network update mechanism. A noise perception evaluation index is developed to measure the performance difference of the model under different noise levels. The noise injection strategy in the adaptive noise scheduling mechanism is dynamically adjusted to obtain the lightweight attitude detection model with enhanced robustness.

[0276] Specifically, firstly, the smoothing loss function is a specially designed deep learning optimization objective function. It improves the model's adaptability to noise and interference by combining the traditional precise matching objective with a smoothing objective that tolerates a certain degree of deviation. In the cardboard box corner detection task, the smoothing loss function can mitigate the negative impacts of inaccurate labeling and slight target wobbling during training, making the model more robust and with stronger generalization ability.

[0277] The original loss function adopts the loss design from the basic cardboard box corner detection model, mainly consisting of two parts: bounding box detection loss (based on CIoU) and corner coordinate regression loss (based on smoothed L1 loss). The bounding box detection loss guides the model to accurately locate the position and size of the entire cardboard box, and its expression is L... bbox = CIoU(box pred boxgt CIoU is a bounding box matching metric that comprehensively considers intersection-union ratio, center point distance, and aspect ratio. Corner coordinate regression loss guides the model to accurately predict the positions of the four corner points; its expression is L. corner = SmoothL1(corners pred corners gt The SmoothL1 loss function is a composite loss function that uses squared error in the small error region and absolute error in the large error region, with the smooth transition point typically set around 1.0. The overall expression for the original loss function is L... orig = λ1·L bbox + λ2·L corner λ1 and λ2 are weighting coefficients, usually set to λ1=1.0 and λ2=2.0, reflecting the higher importance of corner location in the task.

[0278] The smoothing loss component of Gaussian blur is an innovative extension of the original loss function. Its core idea is to transform deterministic labels into probability distributions, tolerating minor deviations in corner positions. This is achieved by converting the precise coordinates of corner points into a two-dimensional Gaussian distribution centered at that point, representing the probability of the corner point's possible location. The standard deviation σ of the Gaussian kernel is a key parameter controlling the degree of blur; a smaller σ produces a sharper distribution, emphasizing precise positioning; a larger σ produces a smoother distribution, tolerating more positional deviations. An adaptive σ setting is used, dynamically adjusted according to the cardboard box size and image resolution, typically within the range of 2-5 pixels. The smoothing loss is calculated based on the difference between the predicted heatmap and the target heatmap, using KL divergence or mean squared error as the metric. The expression for the smoothing loss component is L... smooth = MSE(heatmap pred heatmap gt ), where heatmap gt Heatmap is a target heatmap generated by applying Gaussian blur to the corner coordinates. pred It is a heatmap of the corner locations predicted by the model.

[0279] Weighted combination is the key mechanism that combines the original loss function and the smoothing loss. The combination strategy employs a dynamic weighting method, with weights changing according to the training process and noise level. The complete expression for the smoothing loss function is L. total = (1-α)·L orig + α·L smoothα is the weighting coefficient of the smoothing loss. The setting of α changes in tandem with the adaptive noise scheduling mechanism: in the high-noise stage (early training), α is larger (usually 0.5-0.7), emphasizing the fault tolerance of the smoothing loss; in the low-noise stage (late training), α is smaller (usually 0.1-0.3), emphasizing the accuracy of the original loss function. The change of α adopts a segmented strategy similar to the noise intensity: a high value is maintained for the first 30% of training, a low value is maintained for the last 70% of training, and a linear transition occurs in the intermediate stages. This dynamic weighting strategy ensures that the model establishes resistance to noise in the early training stage and maintains high accuracy in the later training stage.

[0280] The implementation of the smoothing loss function involves several technical details. First, the Gaussian blur generation employs a fast implementation method, avoiding direct calculation of the complete Gaussian kernel. Instead, it utilizes the separability of the Gaussian function, applying a one-dimensional Gaussian filter first in the x-direction and then in the y-direction, significantly reducing computational complexity. Second, the resolution of the heatmap is typically lower than that of the original image; for example, if the original image is 640×480, the heatmap might be 160×120. This downsampling design reduces computation and increases positional tolerance. Third, to handle multi-corner scenes, the system generates an independent heatmap channel for each corner, and the final smoothing loss is the average of the losses from all channels. Finally, the system implements a batch processing mechanism to efficiently process multiple samples in the training batch.

[0281] The smoothing loss function offers several advantages. First, it increases the model's tolerance to inaccurate corner annotations, mitigating the impact of fluctuations in data annotation quality. Second, it enhances the model's adaptability to image noise and slight target movement, improving stability in complex environments. Third, it improves gradient flow, mitigating the vanishing gradient problem, especially as the model approaches convergence in the later stages of training. Finally, it facilitates the model's learning of blurred boundary cases, such as unclear cardboard box edges or partially occluded corners. Experiments show that the model trained with the smoothing loss function achieves comparable accuracy on the standard test set to the original loss function, but significantly outperforms the original model on low-quality images, improving corner detection accuracy by 5-8 percentage points.

[0282] Second, the soft-target network update mechanism is a special method for updating model parameters. Unlike the direct parameter replacement in traditional deep learning, it employs a gradual parameter fusion strategy to smoothly change model parameters, improving training stability and model generalization ability. The core of this mechanism is the Exponential Moving Average (EMA), a smoothing technique commonly used in time series analysis, which is used in deep learning to stabilize the model parameter update process.

[0283] The basic principle of exponential moving average (EMA) is to perform a weighted average of historical data, with the weights decaying exponentially over time, giving more weight to recent data. In model parameter updates, the EMA is calculated using the formula θ. t = β·θ (t-1) + (1-β)·θ' t , where θ t θ is the EMA parameter at time t. (t-1) These are the EMA parameters at time t-1, θ' t t represents the parameter update value calculated by the optimizer at time t, and β is the decay rate (usually set to 0.9995-0.9999). This update method makes the changes in model parameters smoother, reduces parameter oscillations caused by stochastic gradient descent, and improves the model's consistent performance across different data.

[0284] The model parameters at the current moment are determined using the aforementioned EMA formula, which is a comprehensive assessment of historical parameter trajectories. The decay rate β is a key parameter controlling the degree to which historical information is retained. A larger β indicates a more persistent influence of historical parameters and a smoother model change; a smaller β indicates a more sensitive model response to new data and a more rapid change. The choice of decay rate needs to balance stability and adaptability: in the early stages of training, a slightly smaller β value (e.g., 0.9995) can be used to enable the model to learn quickly; in the later stages of training, a larger β value (e.g., 0.9999) is used to enhance stability. This system employs a dynamic adjustment strategy, where the β value changes in tandem with training progress and noise levels, forming an adaptive EMA mechanism.

[0285] A soft-target network is a model network maintained using the EMA (Effective Mapping) update mechanism, existing in parallel with the standard training network. The system maintains two sets of network parameters: the training network parameters θ', directly updated by the optimizer; and the soft-target network parameters θ, derived from the training network parameters via EMA. In practice, the soft-target network requires no additional forward computation, remaining only in the parameter space, consuming minimal additional memory without increasing computational burden. During training, the standard network is used for gradient calculation and parameter updates; during inference and evaluation, the soft-target network generates the final prediction, leveraging its smoother and more stable characteristics to provide more reliable results.

[0286] Weighted combination determination is the core operation for updating parameters in soft-target networks. After each batch of parameter updates is completed by the optimizer, an EMA update is performed: first, the decay rate β applied to the current batch is calculated, taking into account training progress and a preset decay curve; then, the latest parameters θ' of the trained network are obtained. t Next, the new parameters θ of the soft target network are calculated according to the EMA formula. t Finally, the calculation results are stored as the current parameters of the soft-target network. The entire process is performed in the background, without interfering with the normal training process, and the soft-target network parameters are only used when evaluating and saving checkpoints.

[0287] The implementation of the soft-target network update mechanism involves several key technical aspects. First, the initialization strategy sets the soft-target network parameters to the exact same values ​​as the training network parameters, ensuring that the two networks start differentiating from the same point. Second, the decay rate scheduling dynamically adjusts the β value according to the training progress, typically using a step-like or linear change strategy. Third, parameter truncation is an optional optimization; when the EMA parameter differs too much from the training parameters (such as in the early training phase), it can limit the maximum change in a single update to prevent outliers from affecting the network. Finally, the checkpoint saving mechanism needs to store the parameters of both the training network and the soft-target network simultaneously, facilitating recovery after training interruptions.

[0288] The soft-target network update mechanism works synergistically with the aforementioned adaptive noise scheduling and smoothing loss function. Noise scheduling introduces controllable randomness, prompting the model to explore more possible solutions; the smoothing loss function provides a fault-tolerant optimization objective, mitigating the impact of labeling errors and noise; and the soft-target update mechanism smooths the parameter change path, filters out random fluctuations, and preserves stable trends. The combination of these three techniques forms a complete robust training framework that allows the model to fully explore the solution space while ensuring the stability and generalization ability of the final solution.

[0289] The soft-objective network update mechanism makes the model parameter learning process smoother and more stable, reduces the risk of overfitting, and improves the model's adaptability to different data distributions, especially exhibiting stronger robustness in the presence of noise and interference. Experimental results show that, compared with the standard update method, soft-objective update can improve the model's consistent performance across different test sets, with an average accuracy improvement of 1-3 percentage points, particularly significant on difficult examples.

[0290] Third, the model training process is based on the aforementioned technologies, integrating a smooth loss function, a soft-target network update mechanism, and adaptive noise scheduling to form a complete robustness enhancement training framework. The training process includes standard steps such as data loading, forward propagation, loss calculation, backpropagation, parameter update, and model evaluation, but specific robustness enhancement techniques are incorporated into each step.

[0291] Noise perception evaluation metrics are a set of specifically designed measures to comprehensively evaluate model performance under different noise conditions and guide dynamic adjustments during training. This set of metrics includes the following core components: Noise Sensitivity (NS), which measures how sensitive the model's performance is to changes in noise levels; it is calculated by dividing the standard deviation of accuracy at different noise levels by the average accuracy. A smaller NS value indicates a less sensitive model to noise and stronger robustness; Noise Tolerance (NT), which measures the model's ability to maintain performance under specific noise levels; it is calculated as the ratio of accuracy under noise to accuracy without noise; a closer NT value to 1 indicates stronger noise tolerance; Noise Performance Curve (NPC), a complete curve showing the model's accuracy changing with noise intensity, reflecting the model's behavior across the entire noise range; the smoothness of the curve is an important indicator of robustness; and Keypoint Stability (KS), which measures the stability of corner prediction positions under noise interference; it is calculated as the spatial dispersion of multiple prediction results; a smaller KS value indicates more stable predictions. These metrics, used together, provide a multi-dimensional assessment of model robustness.

[0292] The evaluation process employs a standardized testing protocol to ensure comparability and repeatability of results. A noise test set is constructed, containing multiple noise levels (typically five, ranging from no noise to strong noise); each level uses the same original image but adds noise of varying intensities; noise types include Gaussian noise, salt-and-pepper noise, contrast variations, and partial occlusion. For each noise level, fundamental metrics such as corner detection accuracy, pose parameter accuracy, and inference time are calculated, and the aforementioned noise perception evaluation metrics are then calculated based on these fundamental metrics. Evaluation results are presented visually, including performance-noise curves and radar charts, to intuitively reflect the model's robustness characteristics.

[0293] Dynamically adjusting the noise injection strategy is a key step in optimizing the training process based on evaluation results. The system periodically (e.g., every 10 epochs) evaluates the noise perception of the current model and adjusts the parameters of the adaptive noise scheduling mechanism based on the evaluation results. Specific adjustment strategies include: adjusting the noise intensity range based on the noise sensitivity index (high sensitivity expands the range to promote more exploration, low sensitivity narrows the range to accelerate convergence); adjusting the duration of the high-noise phase based on the noise tolerance index (low tolerance extends the high-noise phase to enhance robust training); adjusting the noise decay rate based on the noise performance curve (slowing the decay rate near the noise level in the steeply declining region of the performance curve to increase the training density in that region); and adjusting the noise distribution characteristics of the residual noise layer based on the keypoint stability index (increasing noise interference in unstable feature regions to encourage the model to learn more stable representations). This dynamic adjustment forms an adaptive optimization loop, enabling the training process to automatically adjust the strategy according to the current state of the model to achieve optimal robustness.

[0294] The final training process integrates all the above techniques, forming a complete training cycle: Initialization phase: creating the training network and soft-target network, and initializing adaptive noise scheduling parameters; Training loop phase: performing forward propagation (with residual noise injection) on each batch of data, calculating the smoothing loss function, updating the training network parameters through backpropagation, and applying the soft-target network update; Evaluation phase: periodically performing noise-aware evaluation, calculating various indicators, and generating evaluation reports; Adjustment phase: dynamically optimizing the noise scheduling parameters and training hyperparameters based on the evaluation results; repeating the above loop until the stopping condition is met. The entire process forms a closed-loop optimization system, continuously improving the balance between the model's robustness and accuracy.

[0295] The robust and lightweight attitude detection model is the culmination of this series of technologies and processes. This model possesses the following significant characteristics: high accuracy, achieving over 99% corner detection accuracy (error < 5 pixels) on the standard test set; strong robustness, with accuracy decreasing by no more than 2 percentage points under moderate noise (σ=0.05), far superior to the 8-10 percentage points of the baseline model; high efficiency, with only about 30% of the original YOLOv8-Pose model's parameter count, and single-frame inference time controlled within 35ms on the target industrial PC; and strong stability, with high consistency in inference results across multiple inferences for the same input, and a corner position standard deviation of < 1 pixel. These characteristics enable the model to reliably and stably perform cardboard box corner detection tasks in complex and ever-changing industrial environments, providing accurate positional information for subsequent attitude correction and packing operations.

[0296] Model evaluation and validation are crucial steps in confirming the performance of the final model. Model performance is evaluated on multiple test sets: a standard test set to assess basic accuracy; a noisy test set to assess robustness; a real-world image set to assess adaptability to real-world environments; and an extreme case set to assess boundary condition handling capabilities. Multiple metrics are used in the evaluation: corner location error, measuring the average distance between predicted and actual corners; pose parameter error, measuring the accuracy of vertical deviation angle and horizontal offset calculations; detection success rate, measuring the proportion of times the model can successfully detect all four corners; and processing speed, measuring the average processing time per frame. Evaluation results are documented in a report containing detailed metric data, performance curves, and comparative analysis, serving as a reference for model optimization and application.

[0297] Example 9:

[0298] In this embodiment, the method of recording system operating parameters and abnormal data based on the closed-loop control system architecture, extracting the attitude anomaly rate from the system operating parameters and the abnormal data, establishing a correlation model between the system operating parameters and the attitude anomaly rate, searching for the optimal parameter combination through an optimization algorithm and automatically adjusting the equipment parameters, also includes parameter adaptive control based on a polynomial-Fourier-spline hybrid approach, including:

[0299] Based on the closed-loop control system architecture, a pseudo-linear input-output model of the carton torsion conveying system is established. The pseudo-linear input-output model represents the output attitude anomaly rate as the product of the feature matrix constructed based on historical data and the parameter vector to be estimated, plus the error term, to obtain the pseudo-linear control model.

[0300] Based on the pseudolinear control model, a hybrid basis function space containing polynomial basis functions, Fourier basis functions, and cubic spline basis functions is constructed. The torsional conveying speed, limit rod spacing, guide plate angle, carton height, carton width, carton thickness, and carton material type in the system operating parameters are mapped to the hybrid basis function space to obtain the feature matrix.

[0301] Based on the feature matrix and the parameter vector to be estimated in the pseudo-linear control model, the recursive least squares algorithm with a forgetting factor is used to update the parameter vector in real time to obtain adaptive parameter estimates.

[0302] Based on the adaptive parameter estimation and the pseudo-linear control model, the observables of the system are defined to represent the anomaly rate and the trigger frequency of each level of response per unit time. The observables of the system under different parameter combinations are predicted. An adaptive control law is designed so that the adjustment amount of the torsional conveying speed, the limit rod spacing and the guide plate angle are gradient-dependent with the difference between the current value and the target value of the observables of the system. The torsional conveying parameters are adjusted in real time through the equipment layer to obtain the parameter optimization results.

[0303] Specifically, firstly, in the field of industrial automation, the pseudolinear input-output model is a mathematical modeling method that approximates a nonlinear system as a linear system. Unlike traditional linear models, the pseudolinear model allows input variables to be transformed nonlinearly before being multiplied by the parameter vector, thus capturing the nonlinear characteristics of the system while retaining the simplicity and intuitiveness of the linear model. In this system, the mathematical expression of the pseudolinear input-output model represents the attitude anomaly rate y as the product of the feature matrix X and the parameter vector β plus the error term ε: y = Xβ + ε.

[0304] First, determine the input and output variables of the model. Input variables (i.e., system operating parameters) include: torsional conveyor speed v (unit: m / s, typical range: 0.1-0.5 m / s), representing the linear velocity of the cardboard box moving on the conveyor belt; limit bar spacing d (unit: mm, typical range: 100-300 mm), representing the distance between the limit bars at the end of the conveyor belt, used to control the width through which the cardboard box passes; guide plate angle α (unit: degrees, typical range: 30-60°), representing the angle between the guide plates on both sides of the conveyor belt and the horizontal plane, affecting the cardboard box turning process; cardboard box height H (unit: mm, typical range: 200-500 mm); cardboard box width W (unit: mm, typical range: 150-350 mm); cardboard box thickness D (unit: mm, typical range: 5-30 mm); and cardboard box material type m (a categorical variable, such as 1=corrugated paper, 2=white cardboard, 3=coated paper, etc.). The output variable is the posture abnormality rate r, which is defined as the proportion of cardboard boxes that exhibit posture abnormalities (vertical deviation angle exceeding the first threshold) per unit time (e.g., per hour), expressed as a percentage.

[0305] Data collection and preprocessing are prerequisites for model building. During normal production, operating parameters and corresponding anomaly rates are continuously recorded, with one data point automatically generated every hour. To ensure data quality, the system implements a data validation mechanism to remove obvious outliers (such as abnormal readings caused by sensor malfunctions) and incomplete records. Continuous variables (such as speed and distance) are standardized, transforming the data to a range with a mean of 0 and a standard deviation of 1, reducing the impact of dimensional differences on the model. For categorical variables (such as material type), the system uses one-hot encoding to convert them into numerical representations.

[0306] The core of the model is establishing a mapping relationship between historical data and the anomaly rate. A sliding window method is used, taking data from the most recent 72 hours (approximately 3 days) to build the model each time. This ensures sufficient data to support stable modeling while reflecting the latest operational state of the system. Each model update fully utilizes all historical data within the window, but to highlight the impact of new data, a time-weighted mechanism is introduced, giving newer data points higher weights. Specifically, the weight of each data point is a function of its time t: w i = exp(-λ(t now - t i ), where λ is the attenuation factor (usually set to 0.01-0.05), t now It is the current time, t i It is the recording time of the data point.

[0307] The error term ε in the model represents uncaptured random factors and measurement errors. It is assumed that the error term follows a normal distribution with a mean of zero, and that the errors are independent of each other. In practical applications, this assumption is periodically tested, and the model's applicability is evaluated through residual analysis (such as tests for the normality of residuals and autocorrelation analysis). If significant violations of the assumptions are found, the model structure is adjusted, or an autocorrelation correction term is considered.

[0308] The advantage of pseudolinear control models lies in their ability to leverage the computational efficiency of linear models while capturing complex system behavior through nonlinear transformations of input variables. This model is particularly suitable for industrial processes like cardboard box torsion conveyor systems, which exhibit significant nonlinear characteristics but are not extremely complex. Another important characteristic of the model is its interpretability; engineers can analyze the values ​​of the parameter vector β to understand the degree and direction of the influence of each input factor on the anomaly rate, thereby guiding adjustments in actual production.

[0309] Second, the hybrid basis function space is a composite set of functions used to transform the original input variables into features that better capture the nonlinear characteristics of the system. This system uses a combination of three types of basis functions to construct the hybrid space, each with its own advantages: polynomial basis functions are suitable for describing smooth nonlinear relationships; Fourier basis functions excel at expressing periodic changes; and cubic spline basis functions can accurately fit the nonlinear characteristics of local intervals. This hybrid strategy significantly enhances the model's expressive power, enabling it to adapt to various complex system behaviors.

[0310] Polynomial basis functions are the most fundamental nonlinear transformation method, generating new features by raising the input variables to different powers. Applying a polynomial transformation to each continuous variable (such as velocity v, spacing d, angle α, etc.) generates first-order terms (original variables), second-order terms (squared), and third-order terms (cubic). For example, for a torsional conveyor velocity v, the feature {v, v...} is generated. 2 v3 The main advantages of polynomial basis functions are their simple form, high computational efficiency, and good ability to describe smoothly changing nonlinear relationships. Limiting the polynomial order to no more than 3 is a result of engineering experience—while higher-order polynomials theoretically have stronger fitting capabilities, they are prone to overfitting and are numerically unstable.

[0311] Fourier basis functions transform input variables into frequency domain features using trigonometric functions, making them particularly suitable for capturing periodic variations in systems. Fourier features are constructed for each key continuous variable (primarily the torsional conveyor speed and guide plate angle), in the form {sin(ωx), cos(ωx), sin(2ωx), cos(2ωx)}, where ω is the fundamental frequency, determined based on the physical meaning of the variable. For example, during torsional conveying, due to the mechanical characteristics of the conveyor belt, the anomaly rate may fluctuate periodically with speed; Fourier basis functions can effectively capture this pattern. By analyzing the autocorrelation and spectral characteristics of historical data, the ω value is automatically adjusted to ensure that the Fourier features match the actual periodicity of the system.

[0312] Cubic spline basis functions are piecewise polynomials that can flexibly fit the nonlinear characteristics of local intervals while maintaining continuous smoothness. This paper implements a truncated cubic spline basis function. For each continuous variable x, the basis function φ(x) = max(0, (x-ξ)) is defined at a preset node ξ. 3 By analyzing the distribution of historical data, the node positions for each variable are automatically determined, typically selecting 3-5 nodes evenly within the variable range. For example, if the velocity v ranges from 0.1 to 0.5 m / s, nodes might be set at {0.15, 0.25, 0.35, 0.45}, generating four spline basis functions. The advantage of spline basis functions is their ability to accurately describe the nonlinear characteristics of a variable within a specific interval; for instance, the anomaly rate might rise sharply within a certain velocity range while remaining relatively stable in other ranges.

[0313] Capturing variable interaction effects is a crucial step in feature construction. This involves considering not only the nonlinear transformations of individual variables but also the interactions between them. Specifically, for physically related variable pairs, such as velocity v and spacing d, or angle α and box height H, second-order interaction terms like v·d and α·H are constructed. These interaction terms describe how the effect of one variable depends on the value of another; for example, the effect of increased velocity on the anomalous rate might be more significant when the spacing is small. By analyzing the correlations and partial correlation plots of variables in historical data, significant interaction effects are automatically identified and incorporated into the model.

[0314] Feature selection is a crucial step in avoiding the "curse of dimensionality." A complete mixture basis function space can contain hundreds of features; directly using all features leads to excessive model complexity, computational burden, and a high risk of overfitting. A two-stage feature selection strategy is employed: first, L1-regularized LASSO regression is performed to filter features that significantly contribute to the output; then, forward stepwise regression is applied to further refine the feature set selected by LASSO, ultimately choosing the optimal feature set that balances model complexity and prediction accuracy. This process typically keeps the number of features between 20 and 30, significantly reducing model complexity while maintaining good predictive ability.

[0315] The feature matrix X is the core data structure of the model. Each row represents a historical observation point, and each column represents a feature generated based on the mixed basis function space. The completed feature matrix typically has the following characteristics: the number of columns is much greater than the number of original input variables, reflecting the enhanced expressive power brought about by basis function expansion; there may be correlations between features, but highly redundant features have been removed through the feature selection process; the feature values ​​have been standardized to ensure that features of different magnitudes have an equal impact on the model; the feature set contains various nonlinear transformations and interaction terms, which can comprehensively capture the complex behavior of the system.

[0316] Third, the Recursive Least Squares (RLS) algorithm with a forgetting factor is an online parameter estimation method, particularly suitable for dynamically changing systems. Unlike batch processing algorithms (such as standard least squares), RLS does not require processing all historical data at once. Instead, it incrementally updates model parameters as new data arrives, significantly reducing computational load and storage requirements. The introduction of the forgetting factor allows the algorithm to gradually "forget" the influence of old data, adapting more quickly to changes in system characteristics. This is especially important for industrial processes such as cardboard box torsion conveyor systems, which may change slowly over time (e.g., due to equipment wear and tear, environmental changes).

[0317] Algorithm initialization is the first step in RLS. Initial estimates β0 for the parameter vector β are set, typically based on batch least squares results from historical data or domain knowledge. Simultaneously, the covariance matrix P0 is initialized, representing the uncertainty of the parameter estimates. It is usually set to a large diagonal matrix (e.g., the identity matrix multiplied by a large constant) to indicate low confidence in the initial estimates. A forgetting factor λ is also set, typically in the range of 0.95-0.99. Smaller values ​​allow the model to "forget" old data more quickly, resulting in greater adaptability but potentially leading to fluctuations in parameter estimates; larger values ​​make the estimates more stable but slightly less adaptable.

[0318] The arrival of data triggers the update process. When new data points (input parameters and corresponding anomaly rates) are collected, the input parameters are first mapped to feature vectors x through the mixture basis function space, corresponding to the columns of the feature matrix X. Then, the core step of RLS update is executed: calculating the prediction error e = y - x. T ·β, which is the difference between the actual observed anomaly rate and the model prediction; calculate the gain vector k = P·x / (λ + x). T ·P·x), determine how each parameter should be adjusted to reduce error; update the parameter vector β = β + k·e, adjusting the parameters in the direction of reducing prediction error; update the covariance matrix P = (P - k·x) T ·P) / λ reflects the uncertainty in the new parameter estimation. The computational complexity of this process is quadratic with the number of features, but since feature selection has kept the number of features within a reasonable range, the computational burden is perfectly acceptable on industrial computers.

[0319] The selection of the forgetting factor λ is crucial to the algorithm's performance. An adaptive forgetting factor strategy is employed, dynamically adjusting the value of λ based on the statistical characteristics of the prediction error. Specifically, the mean squared error (MSE) of the prediction error is maintained within a sliding window. When the MSE continuously increases (indicating potential changes in system characteristics), the value of λ is slightly decreased to enhance the algorithm's adaptability; when the MSE stabilizes or decreases, the value of λ is slightly increased to improve estimation stability. The range of λ variation is typically limited to ±0.02 to avoid instability caused by over-adjustment. This adaptive strategy enables the algorithm to maintain robust estimation when system characteristics are relatively stable and to quickly adapt to new states when system characteristics change.

[0320] Anomaly detection and handling are crucial for ensuring algorithm robustness. A multi-layered anomaly detection mechanism is implemented: prediction error detection, marking an anomaly when a single prediction error exceeds three times the historical MSE; parameter jump detection, marking an anomaly when a single change in any parameter exceeds three times its standard deviation; and update stability detection, marking numerical instability when the condition number of the covariance matrix exceeds a preset threshold. For detected anomalies, corresponding measures are taken: reducing the weight of anomalous data points to minimize their impact on parameter updates; introducing regularization terms to improve the matrix condition number when numerical instability occurs; and triggering alarms when anomalies persist, indicating potential need for manual intervention or model reconstruction.

[0321] Parameter interpretation and validation is the final step in the algorithm's output. It not only outputs the parameter vector β, but also calculates the standard error and statistical significance of each parameter, helping engineers understand which factors significantly affect the outlier rate. The system also evaluates the model's predictive ability through cross-validation, periodically (e.g., daily) testing the prediction accuracy on retained data using the latest parameters, generating metrics such as R² and MAE (mean absolute error). These evaluation results are displayed visually on the system dashboard, enabling operators to monitor the model's health and reliability.

[0322] The result of adaptive parameter estimation is a continuously updated set of model parameters that reflect the impact of various factors on the anomalous rate under the current system state. These parameters are not only used to predict the anomalous rate under different operating conditions, but also provide a basis for optimized control, guiding the system on how to adjust operating parameters to reduce the anomalous rate.

[0323] Fourth, observable metrics are a comprehensive set of indicators for evaluating system performance. They include not only the main output—the attitude anomaly rate *r*—but also the trigger frequencies of different response levels: Level 1 response frequency *f1* (representing the rate of minor anomalies), Level 2 response frequency *f2* (representing the rate of moderate anomalies), and Level 3 response frequency *f3* (representing the rate of severe anomalies). This multi-dimensional assessment provides a more comprehensive description of the system's state, as a simple total anomaly rate might mask the distribution variations of anomalies of different severity. These metrics are automatically calculated based on real-time records, typically expressed in "events per thousand cartons," providing a more intuitive reflection of actual production conditions.

[0324] Based on a pseudolinear control model, it is possible to predict observables under different parameter combinations. Given a set of candidate parameters (v, d, α, etc.), they are first mapped to an eigenvector x through a mixture of basis functions. Then, the prediction anomaly rate r' = x is calculated using the current parameter estimate β. T • β. Using a similar method, the response frequencies f1', f2', and f3' at each level can also be predicted. This predictive capability allows the system to assess the potential impact of adjustments before actually tweaking parameters, avoiding potential production disruptions or quality fluctuations caused by blind experimentation.

[0325] The parameter optimization objective is to minimize the weighted performance index J = w1r + w2f2 + w3f3 + w4C, where w1 to w4 are weighting coefficients, and C is the cost of parameter changes (to prevent frequent and significant parameter variations). The weighting reflects the relative importance of different indicators: typically w3 > w2 > w1, indicating a greater emphasis on reducing severe anomalies; w4 is adjusted based on production stability requirements, with a higher value making the system more inclined to maintain parameter stability. The weights are not fixed and are dynamically adjusted based on production plans, cardboard box types, and historical performance. For example, increasing w3 when handling high-value products strengthens the suppression of severe anomalies; increasing w4 when equipment status fluctuates reduces the frequency of parameter changes.

[0326] The adaptive control law is the core algorithm for parameter tuning. Based on the gradient descent principle, it adjusts the parameters along the gradient direction of the objective function J. A model-based adaptive control law is implemented: Δv = -η v · J / v;Δd = -η d · J / d;Δα = -η α · J / α, where η v η d η α It's the learning rate, which controls the step size. Gradient. J / v is calculated using the parameters and feature maps of the pseudolinear model. The learning rate is not fixed; the system employs an adaptive learning rate strategy: an initial small learning rate is set, which is gradually increased as the system's confidence in the model increases (reflected by the covariance matrix of the parameter estimates); when parameter oscillations or poor control performance are detected, the learning rate is automatically reduced. This strategy maintains adjustment agility while avoiding excessive parameter fluctuations.

[0327] The constraints on parameter adjustment ensure that the system operates within a physically feasible range. An effective range (e.g., the range of velocity v is 0.1-0.5 m / s) and a maximum adjustment amplitude (e.g., a single velocity adjustment does not exceed ±0.05 m / s) are set for each parameter. When the calculated adjustment exceeds these constraints, the system automatically truncates to the boundary value. Furthermore, the system implements a state-based adjustment suppression mechanism: when the system detects abnormal operating conditions (e.g., unstable sensor readings, abnormal equipment vibration), the learning rate is temporarily reduced or automatic adjustment is completely paused, resuming automatic optimization only after the state returns to normal. These constraints and protection mechanisms ensure the safety and reliability of the parameter optimization process.

[0328] Real-time adjustment execution is the step that translates optimization decisions into actual control actions. Adjustment commands are sent to each actuator through the control interface at the equipment level: speed adjustment uses a frequency converter to control the motor speed; spacing adjustment uses a stepper motor to control the limit lever position; and angle adjustment uses a servo motor to control the guide plate angle. To ensure a smooth transition, the system employs a ramp control strategy, causing parameters to gradually change to the target value rather than abruptly changing. Throughout the adjustment process, key status indicators, such as motor current and position feedback, are monitored. If any abnormality is detected, the adjustment is immediately stopped and the system restores to safe values ​​to prevent production risks caused by execution failures.

[0329] The evaluation of optimization results is the feedback loop of closed-loop control. Within a specific time window (usually 1-2 hours) after each parameter adjustment, changes in observables of the system are closely monitored. The improvement rate R is calculated by comparing performance indicators before and after the adjustment. imp = (J before - J after ) / J before Evaluate the optimization effect. Based on the evaluation results, the system adjusts subsequent optimization strategies: if the improvement is significant (R... imp If the improvement is >10%, the learning rate may be appropriately increased to accelerate the optimization process; if the improvement is not significant (R... imp If the learning rate is <2%, consider reducing the learning rate or introducing new features; if performance degrades (R0), the learning rate may be reduced or new features may be considered; if performance degrades (R0), the learning rate may be reduced. imp If the parameter value is less than 0, the system may roll back to previous parameters and re-evaluate the model. This feedback-based adaptive strategy enables the system to continuously learn and improve, gradually approaching its optimal operating state.

[0330] The parameter optimization results include not only the found optimal parameter combinations but also a complete record of the optimization process. Each adjusted parameter value, corresponding performance indicators, and environmental conditions (such as cardboard box type and production rate) are stored in a database to build an optimization knowledge base. This data supports various advanced analyses: parameter sensitivity analysis to identify parameters with the greatest impact on performance; optimal range analysis to find the best operating range for each parameter; and conditional correlation analysis to study the differences in optimal parameters under different conditions. The analysis results are presented in intuitive charts to help engineers understand system behavior and guide future equipment adjustments and improvements.

[0331] In summary, a complete data-driven optimization system was constructed based on a hybrid multinomial-Fourier-spline parameter adaptive control method. Through the organic combination of pseudolinear modeling, feature space construction, recursive parameter estimation, and adaptive control law, continuous learning and optimization are achieved, making the carton twisting conveying process more stable and efficient, significantly reducing the posture anomaly rate, and improving the reliability and production efficiency of subsequent packing processes.

[0332] Example 10:

[0333] In this embodiment, calculating the gripping position of the packing robot based on the carton posture parameters and the hierarchical linkage control command includes:

[0334] Based on the visual inspection device and the packing robot in the closed-loop control system architecture, the transformation relationship between the camera coordinate system and the robot coordinate system is established through hand-eye calibration technology. The suction force of the suction cup and the reading of the vacuum pressure sensor are calibrated, a suction force estimation model is established, and the calibrated robot gripping system is obtained.

[0335] Based on the calibrated robotic gripping system and the cardboard box posture parameters, a center of gravity gripping strategy is used to calculate the center of gravity position of the cardboard box. The center point of the suction cup is aligned with the center of gravity position of the cardboard box, and the distance between the two suction cups is automatically adjusted according to the height and width of the cardboard box to obtain the optimal gripping position planning.

[0336] Specifically, firstly, hand-eye calibration is a fundamental technology for the collaborative work of machine vision and robotic arms. It is used to establish a precise transformation relationship between the camera coordinate system and the robotic arm coordinate system, enabling the target position detected by the vision system to be correctly converted into the movement commands of the robotic arm. In a cardboard box packing system, calibration accuracy directly affects the accuracy of the gripping position, therefore, high-precision calibration methods and rigorous verification procedures are required.

[0337] The calibration process begins with selecting a suitable calibration tool. A specially designed calibration plate is used, printed with a precise checkerboard pattern (usually 8×8 or 9×9 squares), with each square measuring 20mm × 20mm and a printing accuracy better than 0.05mm. The calibration plate is mounted on the end effector of the robotic arm and can move with the robotic arm. This configuration is a "hand-eye separation" configuration (i.e., the camera is fixed above the work area and operates independently of the robotic arm), suitable for applications requiring a wide field of view, such as cardboard box packing.

[0338] Calibration data acquisition employs a multi-pose method. The robot arm moves the calibration plate to multiple positions (typically 15-20) within the workspace. These positions are evenly distributed across the robot arm's working area, covering the possible range of positions for future cardboard box gripping. At each position, two sets of data are recorded: the end effector pose (position and orientation, denoted as T) provided by the robot arm controller. robot The pose of the calibration board recognized by the vision system (calculated by recognizing the checkerboard pattern, denoted as T) and the pose of the calibration board recognized by the vision system (calculated by recognizing the checkerboard pattern, denoted as T) camera To reduce the impact of random errors, data was collected three times at each location, and the average value was used as the final record. The data acquisition process is highly automated, and a complete calibration typically takes only 15-20 minutes.

[0339] Coordinate transformation calculation is the core step in calibration. The Tsai method (a classic hand-eye calibration algorithm) is used to calculate the transformation matrix T between the camera coordinate system and the robot arm base coordinate system. eye_hand The transformation satisfies the equation: T camera ×T eye_hand = T eye_hand × T robot The optimal T is found through least squares optimization. eye_handThe goal is to minimize the transformation error at all acquisition locations. To improve computational stability, various numerical optimization techniques are implemented, such as Singular Value Decomposition (SVD) and the Levenberg-Marquardt algorithm, which can effectively handle measurement noise and numerical errors.

[0340] Calibration accuracy verification is a crucial step in ensuring reliable results. The following verification methods are employed: reprojection error analysis, calculating the difference between the actual position of the calibration board corner points in the image and the predicted position calculated using the transformation matrix, with the average error controlled within 0.5 pixels; position accuracy testing, controlling the robot arm to move to a random position (non-calibrated position) within the workspace, comparing the visually estimated position with the robot arm's actual position, with the error controlled within ±1 mm; and attitude accuracy testing, comparing the visually estimated attitude with the robot arm's actual attitude, with the angle error controlled within ±0.5°. Only after passing all verification tests is the calibration result accepted by the system and put into use.

[0341] Saving and applying the calibration results is the final step in the operation process. The calculated transformation matrix T is then... eye_hand The data is stored in a configuration file and records metadata such as calibration time, environmental conditions, and verification results. In actual operation, when the vision system detects the cardboard box and calculates its position P in the camera coordinate system... camera At that time, the position in the robot's coordinate system is obtained through transformation: P robot = T eye_hand × P camera This conversion process is executed in a real-time control loop with a delay of less than 5ms, ensuring that visual information can be promptly converted into robot control commands.

[0342] Suction cup force calibration is the second critical calibration step in a robotic gripper system. Suction cup force directly affects the stability and reliability of gripping, especially for cardboard boxes with different materials and surface conditions. Precise adjustment of the suction force is necessary to ensure safe gripping without damage. The calibration process establishes the relationship between suction cup control parameters (such as vacuum pump power and proportional valve opening) and actual suction force, while also considering the influence of different cardboard box materials.

[0343] Vacuum pressure sensor calibration employs a comparative method. A high-precision reference sensor (accuracy better than ±0.1 kPa) is connected in parallel with the working sensor to be calibrated. A series of preset vacuum levels (typically from 0 to -80 kPa, at 5 kPa intervals) are measured under identical conditions. After recording two sets of readings, a calibration curve is established through linear regression or polynomial fitting, mapping the working sensor readings to accurate vacuum pressure values. The calibration process also includes temperature compensation. Measurements are repeated under different temperature conditions (typically within the range of 15-35℃, at 5℃ intervals) to establish a model relating temperature to reading deviation, ensuring measurement accuracy remains consistent even with temperature variations in actual production environments.

[0344] The suction force estimation model is a model relating the actual suction force of the suction cup to the vacuum pressure, suction cup area, and cardboard material. The basic principle is that the suction force F is directly proportional to the vacuum pressure P and the effective suction area A, but is affected by the material's influence coefficient k. m Adjustments were made. Actual suction force was measured experimentally under different conditions. A force gauge was used to directly measure the maximum tensile force the suction cup could withstand. Data collected included: different vacuum levels (from -20 kPa to -80 kPa); different cardboard materials (corrugated paper, white cardboard, coated paper, etc.); different surface conditions (smooth, rough, coated, etc.); and different suction cup diameters (40 mm, 60 mm, 80 mm). Based on the experimental data, a suction force estimation model incorporating these factors was established. This model can predict the required vacuum pressure under given conditions to generate sufficient suction force for safe gripping of the cardboard box.

[0345] The suction control system consists of a proportional vacuum generator, pressure sensors, and a control algorithm. The proportional vacuum generator precisely controls the vacuum level within the suction cups, with a response time of less than 50ms and an adjustment accuracy of ±2kPa. It implements PID-based closed-loop control, reading pressure sensor data in real time and adjusting vacuum generator parameters to ensure the actual pressure tracks the target value. The control algorithm specifically considers the dynamic characteristics of the suction cups at the moment of contact with the cardboard box, implementing a soft-start strategy: initially using a lower pressure for suction, gradually increasing to the target pressure after confirming good contact, avoiding impact and airflow noise. For dual-suction cup systems, the controller supports independent control of the pressure of each suction cup, achieving differential suction control, which is particularly important for gripping unevenly distributed or tilted cardboard boxes.

[0346] The integration and application of calibration information integrates hand-eye calibration and suction calibration information into the robotic gripper system. A comprehensive configuration file was created, containing transformation matrices, suction model parameters, and control parameters, all of which collectively support gripping decisions. An automatic recalibration trigger mechanism was implemented, automatically prompting the operator to perform recalibration when a decline in positioning or gripping performance is detected (such as multiple consecutive gripping failures or increased positional deviation). Typically, hand-eye calibration is recommended monthly, or immediately after adjustments to the camera or robotic arm position; suction calibration is recommended quarterly, or after replacing the suction cup or repairing the vacuum system.

[0347] Secondly, the center-of-grab strategy is a grasping method that takes into account the mass distribution of an object. Unlike simple geometric center grasping, it can better handle uneven or asymmetrical objects. In cardboard box grasping, especially for tilted boxes, the center of gravity is usually off-center from the geometric center, and accurately finding the center of gravity position is the key to stable grasping.

[0348] The calculation of the cardboard box's center of gravity position is based on attitude parameters and a physical model. For a standard vertical cardboard box, the center of gravity is assumed to be located at the geometric center, i.e., halfway up the height. However, for a tilted cardboard box, the center of gravity position will shift. The lateral shift of the center of gravity ΔX is calculated based on the vertical deviation angle θ and the box height H: ΔX = H·sin(θ) / 2·k adj , where k adj This is an adjustment factor, typically 0.85-0.95, taking into account the fact that the weight of the cardboard box may be slightly greater at the bottom. The horizontal offset Δx is also considered and directly added to the gripping position calculation. The complete center of gravity coordinate calculation includes three-dimensional spatial positioning: the X-coordinate considers the geometric center, the center of gravity offset due to tilt, and the horizontal offset; the Y-coordinate (depth direction) is usually fixed at the center of the cardboard box; the Z-coordinate is set at 45%-50% of the cardboard box height, slightly below the geometric center, considering the typically heavier bottom.

[0349] The gripping position optimization considers several constraints. First, there's the stability constraint: the suction cup position must be within a stable area on the cardboard box surface, typically avoiding the area within 10% of the edge to prevent air leakage and unstable gripping. Second, there's interference avoidance: ensuring the robot arm doesn't collide with surrounding equipment or other cardboard boxes during approach, gripping, and lifting. Third, there's the dynamic range constraint: the selected position must be within the robot arm's working range, with sufficient operational margin. Finally, there's the timing constraint: the gripping action must be completed after the cardboard box reaches the designated position and before it leaves, considering the robot arm's movement time and the conveyor belt speed. A heuristic algorithm is used to find the feasible gripping position closest to the ideal center of gravity while satisfying these constraints.

[0350] Dual suction cup spacing adjustment is a key technology for improving gripping stability. The optimal spacing d is automatically calculated based on the cardboard box width W. opt = W·k width , where k width This is a proportionality coefficient, typically set at 0.6-0.7, representing the ratio of the suction cup spacing to the width of the cardboard box. This ratio was determined through extensive experimentation, ensuring gripping stability while adapting to cardboard boxes of different sizes. The influence of the cardboard box height (H) is also considered: for slender cardboard boxes with a height / width ratio (H / W) greater than 2, the spacing coefficient is appropriately increased (by 5%-10%) to provide better stability support; for flat cardboard boxes with a ratio less than 1, the spacing is slightly reduced. Spacing adjustment is achieved via an electric slide rail on the robotic arm's end effector, with an adjustment range of 100-300mm, an accuracy of ±1mm, and an adjustment time of less than 0.5 seconds, allowing adjustment to be completed before the cardboard box reaches the gripping position.

[0351] The gripping posture planning includes trajectory design in three stages: approach, gripping, and lifting. In the approach stage, the robot moves from its standby position to the pre-grasping position (approximately 50mm from the cardboard box surface), employing joint space planning to ensure smooth, rapid, and wide-range movement. In the gripping stage, the robot moves linearly from the pre-position to the gripping position, gradually decreasing in speed, with force control used for the final 10mm to ensure gentle contact. In the lifting stage, after confirming stable adhesion, the robot first lifts vertically approximately 50mm, then moves along an optimized path to the placement position. The entire planning process considers the robot's dynamic characteristics, ensuring smooth changes in acceleration and velocity, reducing vibration and impact, and achieving a trajectory generation time of less than 50ms to meet real-time control requirements.

[0352] Obstacle avoidance is a crucial mechanism for safe grasping. A 3D model of the workspace is maintained, including fixed obstacles (such as equipment structures and guardrails) and dynamic obstacles (such as other cardboard boxes and mobile devices). Fixed obstacle information is pre-entered into the system; dynamic obstacles are detected and tracked in real-time by a vision system. A modified Rapid Expanding Random Tree (RRT) algorithm is used for real-time path planning, automatically adjusting the trajectory to avoid obstacles when a potential collision risk is detected. If a safe path cannot be found, the grasping operation is paused and an alarm is issued, awaiting manual intervention. This proactive obstacle avoidance capability enables the system to operate safely and reliably in complex environments, reducing downtime and damage risks.

[0353] Verifying the grasping result is a crucial step in closed-loop control. Multiple sensors verify the success of the grasp: a vacuum sensor detects the adsorption state; stable pressure within the expected range indicates good adsorption. Force / torque sensors monitor the force and torque during lifting; abnormal values ​​may indicate partial detachment or deformation of the cardboard box. The vision system tracks the cardboard box's position and orientation, confirming synchronization with the robotic arm's movement. If any verification step fails, the current operation is immediately aborted, attempting to safely return the cardboard box or triggering an exception handling procedure. Verification results are logged for subsequent analysis and system optimization.

[0354] The output of the optimal gripping position planning is a complete set of gripping parameters, including: target position coordinates (X, Y, Z), specifying the spatial position of the suction cup center point; suction cup spacing d, specifying the center distance between the two suction cups; attitude parameters (pitch, roll, yaw angles), specifying the spatial orientation of the robot's end effector; approach and lift trajectories, specifying the motion paths before and after gripping; and timing parameters, specifying the execution time and duration of each action. These parameters are sent to the robot controller in real time via the control bus to guide the execution of precise gripping actions. A historical database of gripping parameters is also maintained, recording the optimal gripping parameters for different cardboard box types and attitudes. Statistical learning is used to continuously optimize the gripping strategy and improve the first-time gripping success rate.

[0355] Example 11:

[0356] In this embodiment, the adjustment of suction cup suction force distribution to achieve torque compensation includes:

[0357] Based on the cardboard box posture parameters and the dynamic adjustment command for the gripping point in the graded linkage control command, for cardboard boxes whose vertical deviation angle is less than the first threshold, a dual suction cup differential suction force control is designed. The suction force of the tilting side suction cup is set as the product of the standard suction force and the first correlation coefficient of the vertical deviation angle, and the suction force of the stable side suction cup is set as the product of the standard suction force and the second correlation coefficient of the vertical deviation angle. The standard suction force is pre-calibrated and determined according to the weight and material type of the cardboard box. The vacuum pressure of each suction cup is adjusted in real time through a proportional adjustment valve and a PID control algorithm to obtain a torque compensation control command.

[0358] According to the torque compensation control command, the adsorption state of the dual suction cups is monitored in real time by a vacuum pressure sensor. When the vacuum pressure difference between the tilted suction cup and the stable suction cup exceeds a preset pressure threshold or the adsorption stability evaluation index is lower than the stability threshold, compensation control is activated. The pitch angle compensation amount of the packing robot is adjusted by the servo motor of the packing robot, or the vacuum pressure increment of the tilted suction cup is increased by the proportional adjustment valve to obtain a dynamic compensation adjustment command.

[0359] Based on the dynamic compensation adjustment command, the grasping process data is recorded through the closed-loop control system architecture. The grasping process data includes the vertical deviation angle, the vacuum pressure of the tilt side suction cup, the vacuum pressure of the stable side suction cup, the pitch angle compensation amount, the vacuum pressure increment, and the grasping success rate. The grasping process data is analyzed through the data analysis platform of the application layer to optimize the first correlation coefficient and the second correlation coefficient, thereby obtaining the adaptive grasping control command of the robot arm.

[0360] Specifically, firstly, dual-suction cup differential suction control is a gripping strategy designed for tilted cartons. The core idea is to adjust the suction distribution of the two suction cups to generate a corrective torque, counteracting the unbalanced torque caused by the carton's tilt. This maintains the carton's stability during lifting, preventing further tilting or tipping. Compared to traditional uniform suction control, differential suction control is better suited for cartons with slight tilt (vertical deviation angle less than a first threshold, typically 5°), reducing reliance on physical correction and improving overall system efficiency.

[0361] Determining the standard suction force is the benchmark for differential control. Standard suction force P stdDefined as the minimum suction force required to safely grasp a cardboard box under ideal conditions (vertical box, flat surface). An experimental model was established to model the relationship between standard suction force and cardboard box parameters, primarily considering two factors: the box weight W and the material type m. An incremental method was used in the experiment: starting with a lower suction force, it was gradually increased to the minimum value that allowed for safe grasping (including a 50% safety margin), and this value was recorded as the standard suction force. Standard suction force lookup tables were established for different weight ranges (e.g., light-duty <0.5kg, medium-duty 0.5-2kg, heavy-duty >2kg) and material types (e.g., corrugated paper, white cardboard, coated paper). In actual operation, the standard suction force P was obtained by looking up the table based on the estimated weight of the current cardboard box (based on an empirical model of volume and material) and the identified material type. std The typical range is between -30 kPa and -70 kPa.

[0362] Differential suction force distribution is based on the principle of torque balance. For tilted cartons, the two suction cups are divided into a "tilted side" (the lower side in the tilt direction of the carton) and a "stable side" (the higher side). The suction force P of the tilted side suction cup is... low Set as P low = P std ·(1+k1·θ), the suction force P of the stable side suction cup high Set as P high = P std ·(1-k2·θ), where θ is the vertical deviation angle (in radians), and k1 and k2 are correlation coefficients determined experimentally, typically k1=0.05-0.08 ( / degree) and k2=0.02-0.04 ( / degree). This setup results in a stronger suction force on the tilted side and a slightly weaker suction force on the stable side, with the difference increasing as the tilt angle increases. When the cardboard box is lifted, this force distribution generates a corrective torque, opposite in direction to the tilt direction and proportional in magnitude to the degree of tilt, effectively counteracting the overturning torque caused by the shift in the center of gravity of the cardboard box.

[0363] Determining the correlation coefficients k1 and k2 is crucial to the system design. Excessively large coefficients can lead to overcompensation of the torque, causing the cardboard box to tilt in the opposite direction during lifting; conversely, insufficient compensation will fail to effectively counteract the overturning torque. The system uses a controlled variable experiment to determine the optimal coefficients: one coefficient is fixed, while the other is adjusted, and the posture changes of the cardboard box during lifting are recorded under different combinations. The combination of coefficients that makes the cardboard box closest to a vertical state is selected. Experiments show that k1 usually needs to be greater than k2 because increasing suction is more effective than decreasing it, and the tilting side plays a dominant role in torque compensation. The influence of the cardboard box height is also considered. For slender cardboard boxes with a height / width ratio greater than 2, the coefficients automatically increase by 10%-20%, as these types of boxes are more prone to instability due to their higher center of gravity.

[0364] The proportional control valve is the actuator for achieving precise suction control. Each suction cup is equipped with an independent proportional vacuum control valve, capable of continuously adjusting the vacuum pressure within the range of 0 to -80 kPa, with a response time of less than 50 ms and an adjustment accuracy of ±2 kPa. The control valve employs electro-proportional control, receiving 0-10V analog signals or digital communication commands, and supports update frequencies up to 100 Hz to meet real-time control requirements. Valve characteristic compensation is also implemented; through table lookup or polynomial fitting, the linear control signal is mapped to a nonlinear valve response, ensuring a linear relationship between vacuum pressure and control commands, simplifying control algorithm design.

[0365] The PID control algorithm is the core of the closed-loop suction control. An independent PID control loop is implemented for each suction cup, reading pressure sensor feedback in real time, calculating the deviation from the target pressure, and generating an appropriate control signal to adjust the proportional valve opening. The control algorithm adopts the standard PID form but incorporates several improvements: gain scheduling, dynamically adjusting PID parameters according to different operating points to adapt to the nonlinear characteristics of the vacuum system; anti-integral saturation, limiting the growth of the integral term and preventing control overshoot; derivative term filtering, reducing the impact of noise on derivative control; and feedforward compensation, using the system model to predict control requirements and improve response speed. Control parameters are initially determined using the Ziegler-Nichols method and then fine-tuned through actual testing. Typical parameters are P=0.5-0.8, I=0.1-0.3, D=0.05-0.1, with a sampling period of 10ms. A self-diagnostic function for the control loop is also implemented, monitoring control performance indicators such as rise time, steady-state error, and volatility, automatically adjusting parameters or triggering alarms when anomalies are detected.

[0366] The torque compensation control command is the output of the differential suction control, containing detailed execution parameters: the target vacuum level of the two suction cups, in kPa, accurate to one decimal place; pressure settling time, specifying the time required to reach the target pressure, typically 100-200ms; stabilization period requirement, specifying the minimum time to maintain stable suction, typically 500ms; pressure fluctuation tolerance, defining the acceptable pressure fluctuation range, typically ±5% of the target pressure; and anomaly response strategy, specifying the handling method when an abnormal pressure is detected. These parameters are transmitted to the execution unit via the device-level control bus to guide the precise implementation of differential suction. The control command is updated every 100ms to ensure that torque compensation can adapt to minute changes in the carton's posture, providing continuous and stable control.

[0367] Secondly, the vacuum pressure sensor is a key component for monitoring the adsorption state. Each suction cup is equipped with an independent high-precision pressure sensor with a measurement range of 0 to -100 kPa, a resolution better than 0.1 kPa, a response time of less than 5 ms, and supports sampling frequencies up to 200 Hz. The sensor is installed in the gas path between the suction cup and the vacuum generator, as close to the suction cup as possible to minimize the impact of gas path delay. The sensor adopts a temperature compensation design, maintaining stable performance within the range of 15-45℃, adapting to changes in factory ambient temperature. An automatic sensor calibration function is also implemented, periodically (usually weekly) checking the accuracy of readings at known pressure points, automatically adjusting calibration parameters or prompting sensor replacement if deviations are detected.

[0368] Adsorption state monitoring employs a multi-parameter analysis method. It not only records absolute pressure values ​​but also calculates several derived indicators: pressure build-up rate, representing the speed at which vacuum pressure rises, reflecting the sealing condition between the suction cup and the cardboard box surface; pressure stability, representing the magnitude of pressure fluctuations under stable conditions, reflecting the airtightness of the gas path and the stability of the gripping; pressure difference, representing the pressure difference between the two suction cups, reflecting the actual effect of torque compensation; and suction uniformity, representing the degree of matching between the actual pressure distribution and the ideal distribution. These indicators are updated at a frequency of 100Hz, providing a comprehensive real-time assessment of the adsorption state. Based on these indicators, adsorption quality is judged into four levels: "excellent," "good," "average," and "unstable," with corresponding control strategies implemented for each level.

[0369] Abnormal state detection is based on threshold judgment and pattern recognition. Two key thresholds are set: pressure difference threshold ΔP. threshold Typically set to 10-15 kPa, this represents the maximum allowable pressure difference between the two suction cups; the stability threshold S threshold The value is typically set to 0.7-0.8 (out of 1.0), representing the minimum requirement for adsorption stability assessment. The stability assessment index S is a comprehensive score, calculated using the following formula: , where σ p It is the standard deviation of pressure. This is the average pressure; p1 and p2 are the pressures of the two suction cups. max This refers to the maximum permissible pressure difference. Furthermore, machine learning-based anomaly pattern recognition has been implemented. By analyzing the characteristics of pressure time series, specific types of anomalies, such as air leaks, partial suction cup detachment, and cardboard box deformation, are identified. This advanced anomaly detection can identify and address potential risks before problems escalate.

[0370] The compensation control trigger mechanism employs a hysteresis design. When a pressure difference exceeding ΔP is detected... threshold Or the stability index S is lower than S threshold If the condition persists for more than 100ms, compensation control is triggered; when the indicator returns to the normal range (pressure difference less than 0.8·ΔP), compensation control is triggered.threshold And S is greater than 1.1·S threshold When the duration exceeds 200ms, the system deactivates the compensation state. This hysteresis-based triggering mechanism avoids frequent switching when the indicator fluctuates near the critical value, improving control stability. It also implements a gradual triggering strategy, applying mild compensation for minor anomalies and gradually increasing the compensation intensity as the anomaly severity increases, ensuring a smooth control transition and avoiding shocks and vibrations caused by sudden changes.

[0371] Attitude compensation is an important supplement to torque compensation. When adsorption instability is detected, physical compensation is first attempted by adjusting the attitude of the robot's end effector. The pitch joint of the robot is controlled to fine-tune the attitude angle, with the adjustment amount being θ. adj Proportional to the detected pressure difference: θ adj = k angle ·(p1-p2), where k angle This is a proportionality coefficient, typically 0.1-0.2° / kPa. The adjustment range is limited to ±3°, sufficient to provide effective compensation without causing excessive deviation in the gripping posture. The posture adjustment employs a smooth transition to avoid carton wobbling caused by sudden changes. This mechanical compensation is particularly suitable for handling uneven adsorption caused by uneven or deformed carton surfaces. By changing the contact angle between the suction cup and the surface, it improves the sealing effect and reduces the risk of air leakage.

[0372] Suction adjustment is another effective compensation method. When pressure imbalance or unstable adsorption is detected, dynamic compensation can also be achieved by adjusting the suction distribution. The main strategy is to increase the vacuum pressure of the tilted suction cup; the increment ΔP is inversely proportional to the deviation of the stability index S: ΔP = k pressure ·(S threshold -S), where k pressure This is a proportionality coefficient, typically 50-100 kPa. The incremental value is limited to within 30% of the original target pressure to avoid excessive increases that may cause cardboard box deformation. Suction adjustment reacts faster than attitude compensation (usually within 50 ms), making it particularly suitable for handling sudden changes in adsorption conditions. However, long-term, significant increases in suction may damage the cardboard box. Therefore, attitude compensation should be prioritized, and suction should only be significantly increased when attitude compensation is insufficient.

[0373] The comprehensive compensation strategy automatically selects the most suitable compensation method or a combination of multiple methods based on the nature and severity of the anomaly. A decision tree algorithm is implemented to select the compensation strategy according to the anomaly type (e.g., large pressure difference, instability on both sides, unilateral instability), cardboard box characteristics (e.g., weight, material, size), and current state (e.g., lifting stage, moving speed). For example, for slight pressure imbalance in lightweight cardboard boxes, posture adjustment is prioritized; for severe instability in heavy cardboard boxes, suction force and posture adjustment are simultaneously increased; for detected air leakage, the suction force of the affected suction cup is significantly increased and the posture is slightly adjusted to maximize the compensation effect. This intelligent decision-making ensures the system can cope with various anomalies and maintain grasping stability.

[0374] Dynamic compensation adjustment commands are real-time control commands generated by the system in response to abnormal conditions. They include the following core parameters: attitude adjustment amount (specifying the pitch angle that the robot's end effector needs to change); pressure adjustment amount (specifying the increase or decrease in vacuum pressure for each suction cup); execution priority (specifying the order of command execution in resource contention situations); duration (specifying the time range for compensation to be maintained); and feedback requirements (specifying the status feedback indicators that need to be monitored). These parameters are sent to the execution unit through a priority control channel to ensure timely response to abnormal conditions and reduce the risk of cardboard box detachment or damage.

[0375] Third, recording the grasping process data is the foundation for continuous system improvement. A high-precision time-series database is used to record complete process data for each grasping operation, with a sampling frequency of 100Hz to ensure the capture of rapidly changing dynamic characteristics. The recorded content is comprehensive, including not only basic posture and pressure data, but also: cardboard box characteristic data (size, material, weight estimation, etc.); environmental conditions (temperature, humidity, production speed, etc.); robot arm status (position, speed, acceleration, etc.); and grasping results (success, failure, or anomaly). Each record is accompanied by a unique identifier and timestamp, supporting subsequent precise queries and correlation analysis. The system employs a hierarchical storage strategy: recent data (usually within 7 days) is stored in its entirety with a high sampling rate; historical data is downsampled and stored long-term, typically for 3-6 months.

[0376] Success rate is a key metric for evaluating system performance. A multi-level success standard is defined: complete success (no abnormalities throughout the grasping, transporting, and placing process); basic success (minor abnormalities during the process, but the cardboard box is ultimately placed correctly); partial success (the cardboard box is grasped but placement is not ideal, requiring fine-tuning); failure (the cardboard box fails to be grasped or falls off during the process). Success rates are calculated across different dimensions: by cardboard box type (different sizes, materials); by posture characteristics (different deviation angle ranges); by time period (hours, shifts, dates); and by control parameters (different correlation coefficient settings). These multi-dimensional success rate statistics provide direct performance feedback for parameter optimization, guiding the optimization direction.

[0377] The data analytics platform is a comprehensive analysis and decision support system that integrates data storage, processing, analysis, and visualization capabilities. The platform employs a layered architecture: the bottom layer is a time-series database storing raw process data; the middle layer is an analytics engine implementing various statistical and machine learning algorithms; and the top layer is a visualization and interactive interface providing intuitive data display and operational functions. The platform supports multiple analytical methods: descriptive statistics, calculating the distribution characteristics and correlations of various parameters; comparative analysis, comparing performance differences under different parameter settings; trend analysis, tracking the changing trends of key indicators over time; and predictive modeling, predicting the success rate of data capture under specific conditions. These analytical results are presented in the form of dashboards, trend charts, and reports, supporting the decision-making processes of engineers and managers.

[0378] Correlation coefficient optimization is one of the core tasks of data analysis. A data-driven approach was used to optimize k1 (tilt side coefficient) and k2 (stability side coefficient). The optimization process included several steps: data grouping, grouping the captured records by cardboard box type and deviation angle range to ensure reasonable comparison; parameter grid generation, generating uniformly distributed parameter combinations within feasible k1 and k2 ranges, typically using a 5×5 or 7×7 grid; performance evaluation, calculating the weighted performance score for each parameter combination, considering factors such as success rate, stability indicators, and energy consumption; parameter screening, identifying the optimal parameter region; and fine-grained optimization, performing a more granular search within the optimal region to find the locally optimal parameter combination. Optimization results show that the optimal correlation coefficient varies with cardboard box type and working conditions; there is no universally applicable "optimal value," which is why the system needs continuous adaptive optimization.

[0379] Machine learning models represent an advanced parameter optimization approach. Supervised learning methods are employed to establish a mapping model from working conditions to optimal coefficients. Specifically, the Gradient Boosting Decision Tree (GBDT) algorithm is used, with cardboard box characteristics (size, material, etc.), environmental conditions, and orientation parameters as input features, and optimal k1 and k2 values ​​as output targets. Model training utilizes the best-performing records from historical data, and cross-validation ensures generalization ability. The learning process is incremental, with the model updated periodically (e.g., weekly) using newly collected data to adapt to long-term changes in production conditions. In practical applications, the model can predict the optimal coefficients under current conditions in real time with an accuracy of ±0.005, significantly outperforming fixed-coefficient strategies, especially when dealing with non-standard cardboard boxes or boundary conditions.

[0380] The robotic arm's adaptive gripping control commands are a comprehensive control scheme generated by the system based on optimization results. It includes the following core components: basic parameters, such as suction cup position, spacing, and attitude angle; torque compensation parameters, including optimized k1 and k2 correlation coefficients; dynamic adjustment rules, defining how to fine-tune parameters under different conditions; anomaly response strategies, specifying the handling methods when anomalies are detected; and monitoring requirements, defining the process data items that need to be recorded. These commands are distributed to the execution units through the system's multi-layered architecture, guiding the robotic arm to complete precise and reliable cardboard box gripping operations. Each command generation considers the current cardboard box characteristics and attitude state, achieving true adaptive control and significantly improving the system's ability and stability in handling various cardboard boxes.

[0381] Example 12:

[0382] like Figure 2 As shown, the present invention also provides a visual feedback-based cardboard box posture correction-packing collaborative control system, comprising:

[0383] System architecture building unit 10 is used to build a hardware architecture including a torsion conveyor, a vision inspection device, a correction execution device, a packing robot and a central control system. It designs a four-layer software system architecture including a device layer, a communication layer, an algorithm layer and an application layer, and establishes an information flow diagram from image acquisition to control command issuance to obtain a closed-loop control system architecture.

[0384] The paper box posture recognition unit 20 is used to acquire paper box images and complete calibration based on the closed-loop control system architecture, and then use a deep learning model to identify the corner coordinates of the paper box, calculate the vertical deviation angle and horizontal offset of the paper box, and obtain the paper box posture parameters.

[0385] The hierarchical linkage control unit 30 is used to divide the paper box posture parameters into multiple states according to the magnitude of the vertical deviation angle and the horizontal offset, and execute the corresponding response strategy, including gripping point compensation, start-up of the correction execution device and abnormal marking processing, to obtain hierarchical linkage control instructions.

[0386] The robotic arm adaptive gripping unit 40 is used to calculate the gripping position of the packing robot and adjust the suction force distribution of the suction cup to achieve torque compensation based on the carton posture parameters and the hierarchical linkage control command. It also monitors and compensates for the torque compensation in real time through sensors to obtain the robotic arm adaptive gripping control command.

[0387] The parameter optimization unit 50 is used to record system operating parameters and abnormal data based on the closed-loop control system architecture, extract the attitude anomaly rate from the system operating parameters and the abnormal data, establish a correlation model between the system operating parameters and the attitude anomaly rate, search for the optimal parameter combination through an optimization algorithm, and automatically adjust the equipment parameters to obtain the parameter optimization result.

[0388] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cardboard box posture correction-packing collaborative control method based on visual feedback, characterized in that, include: A hardware architecture including a torsion conveyor, a vision inspection device, a correction execution device, a packing robot, and a central control system is constructed. A four-layer software system architecture including an equipment layer, a communication layer, an algorithm layer, and an application layer is designed. An information flow diagram from image acquisition to control command issuance is established to obtain a closed-loop control system architecture. Based on the closed-loop control system architecture, after acquiring the cardboard box image and completing the calibration, a deep learning model is used to identify the corner coordinates of the cardboard box, calculate the vertical deviation angle and horizontal offset of the cardboard box, and obtain the cardboard box attitude parameters. Based on the paper box posture parameters, the system divides the data into multiple states according to the magnitude of the vertical deviation angle and the horizontal offset, and executes corresponding response strategies, including gripping point compensation, activation of the correction execution device, and anomaly marking processing, to obtain hierarchical linkage control commands, including: Based on the carton posture parameters, when the vertical deviation angle is detected to be less than a first threshold, a compensation amount for the gripping position of the packing robot is calculated. This compensation amount is related to the vertical deviation angle and the carton height, and is transmitted in real-time to the robot controller via the communication layer using a communication protocol to obtain a first-level control command. Based on the carton posture parameters, when the vertical deviation angle is detected to be between the first and second thresholds, the thrust application point height and push rod extension time are calculated based on the first-level control command. The thrust application point height is related to the vertical deviation angle and the carton height, and the push rod extension time is related to the vertical deviation angle and the conveyor belt speed. The equipment layer control signal adjusts the thrust magnitude to obtain the second-level control command; based on the carton posture parameters, when the vertical deviation angle is detected to be greater than the second threshold or multiple consecutive correction failures occur, based on the first-level control command and the second-level control command, the central control system controls the conveyor belt of the torsional conveyor to decelerate, the controller of the central control system controls the alarm light to light up and emits an audible prompt, the abnormal location and image are displayed in real time on the human-machine interface, and the abnormality type and frequency are automatically recorded to obtain the third-level control command; based on the first-level control command, the second-level control command, and the third-level control command, the hierarchical linkage control command is obtained; Based on the carton posture parameters and the hierarchical linkage control command, the gripping position of the packing robot is calculated and the suction cup force distribution is adjusted to achieve torque compensation. The robot is monitored in real time by the sensor and compensation adjustment is performed to obtain the robot's adaptive gripping control command. Based on the closed-loop control system architecture, the system operating parameters and abnormal data are recorded. The attitude anomaly rate in the system operating parameters and the abnormal data is extracted. A correlation model between the system operating parameters and the attitude anomaly rate is established. The optimal parameter combination is searched through an optimization algorithm and the equipment parameters are automatically adjusted to obtain the parameter optimization result.

2. The method according to claim 1, characterized in that, The hardware architecture includes a torsion conveyor, a vision inspection device, a correction execution device, a packing robot, and a central control system. A four-layer software system architecture, comprising an equipment layer, a communication layer, an algorithm layer, and an application layer, is designed. An information flow diagram from image acquisition to control command issuance is established, resulting in a closed-loop control system architecture, including: Based on the existing cardboard box twisting conveyor line and packing station, an industrial camera, a pneumatic push rod correction device, a controller, and a human-machine interface are added to construct a hardware architecture that includes the twisting conveyor, the vision inspection device, the correction execution device, the packing robot, and the central control system, thus obtaining the physical implementation basis of the system. Based on the physical implementation of the system, a four-layer software system architecture is designed, including a device layer driver, a communication layer OPC UA protocol data interaction network, an algorithm layer attitude recognition algorithm and response strategy decision algorithm, and an application layer operation interface and data analysis platform, to obtain software function support. Based on the physical implementation of the system and the software functionality, the industrial camera captures images of the cardboard box and transmits them to the attitude recognition algorithm in the algorithm layer to obtain attitude data. The attitude data is then transmitted to the response strategy decision algorithm in the algorithm layer via the OPC UA protocol data interaction network in the communication layer to obtain control commands. The control commands are then sent to the correction execution device and the packing robot in the device layer to complete the execution actions and obtain execution data. Finally, the data analysis platform in the application layer collects the execution data to optimize the parameters, resulting in the closed-loop control system architecture.

3. The method according to claim 2, characterized in that, The process of acquiring and calibrating the cardboard box image based on the closed-loop control system architecture includes: Based on the vision inspection device in the closed-loop control system architecture, an industrial camera with a resolution of 2048×1536 is installed above the conveyor belt outlet of the torsional conveyor, equipped with a polarizing filter to eliminate reflective interference, and a ring LED is used to provide uniform illumination, thus obtaining the vision inspection hardware configuration. Based on the aforementioned visual inspection hardware configuration, the camera's intrinsic and extrinsic parameters are calibrated using the checkerboard calibration method, establishing a mapping relationship between pixel coordinates and actual physical coordinates to obtain the calibrated camera system. Based on the calibrated camera system, when the cardboard box is conveyed to the detection area by the conveyor belt of the twisting conveyor, the industrial camera captures an image of the cardboard box and transmits it to the algorithm layer in the closed-loop control system architecture to obtain the image of the cardboard box to be identified.

4. The method according to claim 3, characterized in that, The method of using a deep learning model to identify the corner coordinates of a cardboard box includes: Based on the algorithm layer in the closed-loop control system architecture, images of cardboard boxes of various specifications and materials in different states are collected, and the coordinates of the four corner points are marked as key feature points. The dataset is expanded to a preset number of samples by methods such as rotation, scaling, brightness adjustment, and adding noise to obtain the training dataset. Based on the training dataset, the output layer of the YOLOv8-Pose model was modified to adapt to the requirements of cardboard box four-corner point detection. MobileNetV3 was used as the backbone network, and the model parameters of the YOLOv8-Pose model were initialized using the transfer learning method. Then, the model was trained iteratively using the Adam optimizer with an adaptive learning rate to obtain a lightweight pose detection model with a recognition accuracy greater than 98% and a single frame processing time of less than 50ms.

5. The method according to claim 1, characterized in that, The execution of the corresponding response strategy also includes security response control based on random polynomial time delay, including: Based on the verticality deviation angle in the paper box posture parameters, a time delay function containing a deterministic polynomial part and a controlled random disturbance term is designed, wherein the controlled random disturbance term follows a normal distribution, thus obtaining a delay control model; Based on the aforementioned delay control model, a dynamic delay is added to the calculation of the compensation amount for the gripping position of the packing robot in the first-level control command, so that the compensation action is delayed. The delay time changes nonlinearly with the verticality deviation angle, resulting in a first-level response command with random delay. Based on the delay control model and the first-level response command, the push rod extension time in the second-level control command is adjusted, and the delay amount related to the verticality deviation angle is increased to obtain a second-level response command with random delay; Based on the vertical deviation angle and horizontal offset in the paper box posture parameters, a polynomial boundary function is introduced to dynamically adjust the first threshold and the second threshold, and a safety status assessment model is developed. When the safety status assessment value is lower than the safety threshold, the model parameters of the delay control model are automatically adjusted. Based on the first-level response command, the second-level response command, and the third-level control command, a safety-enhanced hierarchical linkage control command is obtained.

6. The method according to claim 5, characterized in that, The method of using a deep learning model to identify the corner coordinates of the cardboard box also includes robust attitude recognition with residual noise enhancement, including: Based on the algorithm layer in the closed-loop control system architecture, a residual noise linear layer is inserted into the feature extraction network of the deep learning model. The residual noise linear layer enhances the feature expression capability by superimposing a noise function on the linear transformation result, thus obtaining a feature extraction network with noise enhancement capability. Based on the feature extraction network with noise enhancement capability, a strategy for dynamically adjusting noise intensity with training progress is designed. When the number of training iterations is less than 30% of the total number of iterations, high-intensity noise with a noise standard deviation of 0.1 is used; when the number of training iterations is greater than 70% of the total number of iterations, low-intensity noise with a noise standard deviation of 0.01 is used; and when the number of training iterations is between 30% and 70% of the total number of iterations, a linearly decaying noise standard deviation is used, thus obtaining an adaptive noise scheduling mechanism. The model is trained based on the aforementioned adaptive noise scheduling mechanism to obtain a lightweight attitude detection model with enhanced robustness.

7. The method according to claim 6, characterized in that, The model training based on the adaptive noise scheduling mechanism further includes: Based on the adaptive noise scheduling mechanism, the corner detection loss function of the deep learning model is improved into a weighted combined loss function that includes the original loss function and a smooth loss function that introduces Gaussian blur, thus obtaining the smooth loss function. Based on the smoothing loss function, the model parameters of the deep learning model are updated using an exponential moving average method, so that the model parameters at the current time are determined by a weighted combination of the parameters at the previous time and the optimized parameters of the current batch, thus obtaining the soft target network update mechanism. Model training is completed based on the soft target network update mechanism. A noise perception evaluation index is developed to measure the performance difference of the model under different noise levels. The noise injection strategy in the adaptive noise scheduling mechanism is dynamically adjusted to obtain the lightweight attitude detection model with enhanced robustness.

8. The method according to claim 1, characterized in that, The system, based on the closed-loop control system architecture, records system operating parameters and abnormal data, extracts the attitude anomaly rate from the system operating parameters and the abnormal data, establishes a correlation model between the system operating parameters and the attitude anomaly rate, searches for the optimal parameter combination through an optimization algorithm, and automatically adjusts the equipment parameters. It also includes parameter adaptive control based on a polynomial-Fourier-spline hybrid approach, comprising: Based on the closed-loop control system architecture, a pseudo-linear input-output model of the carton torsion conveying system is established. The pseudo-linear input-output model represents the output attitude anomaly rate as the product of the feature matrix constructed based on historical data and the parameter vector to be estimated, plus the error term, to obtain the pseudo-linear control model. Based on the pseudolinear control model, a hybrid basis function space containing polynomial basis functions, Fourier basis functions, and cubic spline basis functions is constructed. The torsional conveying speed, limit rod spacing, guide plate angle, carton height, carton width, carton thickness, and carton material type in the system operating parameters are mapped to the hybrid basis function space to obtain the feature matrix. Based on the feature matrix and the parameter vector to be estimated in the pseudo-linear control model, the recursive least squares algorithm with a forgetting factor is used to update the parameter vector in real time to obtain adaptive parameter estimates. Based on the adaptive parameter estimation and the pseudo-linear control model, the observables of the system are defined to represent the anomaly rate and the trigger frequency of each level of response per unit time. The observables of the system under different parameter combinations are predicted. An adaptive control law is designed so that the adjustment amount of the torsional conveying speed, the limit rod spacing and the guide plate angle are gradient-dependent with the difference between the current value and the target value of the observables of the system. The torsional conveying parameters are adjusted in real time through the equipment layer to obtain the parameter optimization results.

9. The method according to claim 1, characterized in that, The step of calculating the gripping position of the packing robot based on the carton posture parameters and the hierarchical linkage control command includes: Based on the visual inspection device and the packing robot in the closed-loop control system architecture, the transformation relationship between the camera coordinate system and the robot coordinate system is established through hand-eye calibration technology. The suction force of the suction cup and the reading of the vacuum pressure sensor are calibrated, a suction force estimation model is established, and the calibrated robot gripping system is obtained. Based on the calibrated robotic gripping system and the cardboard box posture parameters, a center of gravity gripping strategy is used to calculate the center of gravity position of the cardboard box. The center point of the suction cup is aligned with the center of gravity position of the cardboard box, and the distance between the two suction cups is automatically adjusted according to the height and width of the cardboard box to obtain the optimal gripping position planning.

10. The method according to claim 7, characterized in that, The adjustment of suction cup force distribution to achieve torque compensation includes: Based on the cardboard box posture parameters and the dynamic adjustment command for the gripping point in the graded linkage control command, for cardboard boxes whose vertical deviation angle is less than the first threshold, a dual suction cup differential suction force control is designed. The suction force of the tilting side suction cup is set as the product of the standard suction force and the first correlation coefficient of the vertical deviation angle, and the suction force of the stable side suction cup is set as the product of the standard suction force and the second correlation coefficient of the vertical deviation angle. The standard suction force is pre-calibrated and determined according to the weight and material type of the cardboard box. The vacuum pressure of each suction cup is adjusted in real time through a proportional adjustment valve and a PID control algorithm to obtain a torque compensation control command. According to the torque compensation control command, the adsorption state of the dual suction cups is monitored in real time by a vacuum pressure sensor. When the vacuum pressure difference between the tilted suction cup and the stable suction cup exceeds a preset pressure threshold or the adsorption stability evaluation index is lower than the stability threshold, compensation control is activated. The pitch angle compensation amount of the packing robot is adjusted by the servo motor of the packing robot, or the vacuum pressure increment of the tilted suction cup is increased by the proportional adjustment valve to obtain a dynamic compensation adjustment command. Based on the dynamic compensation adjustment command, the grasping process data is recorded through the closed-loop control system architecture. The grasping process data includes the vertical deviation angle, the vacuum pressure of the tilt side suction cup, the vacuum pressure of the stable side suction cup, the pitch angle compensation amount, the vacuum pressure increment, and the grasping success rate. The grasping process data is analyzed through the data analysis platform of the application layer to optimize the first correlation coefficient and the second correlation coefficient, thereby obtaining the adaptive grasping control command of the robot arm.

11. A cardboard box posture correction-packing collaborative control system based on visual feedback, characterized in that, include: The system architecture building unit is used to construct the hardware architecture including a torsion conveyor, a vision inspection device, a correction execution device, a packing robot and a central control system. It designs a four-layer software system architecture including an equipment layer, a communication layer, an algorithm layer and an application layer, and establishes an information flow diagram from image acquisition to control command issuance to obtain a closed-loop control system architecture. The paper box posture recognition unit is used to acquire paper box images and complete calibration based on the closed-loop control system architecture, and then use a deep learning model to identify the corner coordinates of the paper box, calculate the vertical deviation angle and horizontal offset of the paper box, and obtain the paper box posture parameters. The hierarchical linkage control unit is used to divide the carton posture parameters into multiple states based on the magnitude of the vertical deviation angle and the horizontal offset, and execute corresponding response strategies, including gripping point compensation, activation of the correction execution device, and abnormal marking processing, to obtain hierarchical linkage control commands. These commands include: calculating the compensation amount for the gripping position of the packing robot when the vertical deviation angle is detected to be less than a first threshold, wherein the compensation amount is related to the vertical deviation angle and the carton height, and is transmitted in real time to the robot controller via the communication layer using a communication protocol to obtain a first-level control command; and calculating the thrust application point height and push rod extension time based on the first-level control command when the vertical deviation angle is detected to be between the first and second thresholds, wherein the thrust application point height and push rod extension time are calculated based on the first-level control command. The point height is related to the vertical deviation angle and the height of the cardboard box; the push rod extension time is related to the vertical deviation angle and the conveyor belt speed. The thrust magnitude is adjusted through the equipment layer control signal to obtain the second-level control command. According to the cardboard box posture parameters, when the vertical deviation angle is detected to be greater than the second threshold or multiple consecutive correction failures occur, based on the first-level control command and the second-level control command, the central control system controls the conveyor belt of the torsional conveyor to decelerate, the controller of the central control system controls the alarm light to light up and emits an audible prompt, the abnormal location and image are displayed in real time on the human-machine interface, and the abnormality type and frequency are automatically recorded to obtain the third-level control command. Based on the first-level control command, the second-level control command, and the third-level control command, the hierarchical linkage control command is obtained. The adaptive gripping unit of the robotic arm is used to calculate the gripping position of the packing robot and adjust the suction force distribution of the suction cup to achieve torque compensation based on the posture parameters of the carton and the hierarchical linkage control command. The unit monitors and compensates for the torque compensation in real time through sensors to obtain the adaptive gripping control command of the robotic arm. The parameter optimization unit is used to record system operating parameters and abnormal data based on the closed-loop control system architecture, extract the attitude anomaly rate from the system operating parameters and the abnormal data, establish a correlation model between the system operating parameters and the attitude anomaly rate, search for the optimal parameter combination through an optimization algorithm, and automatically adjust the equipment parameters to obtain the parameter optimization result.

Images