A method and system for frame forming control based on industrial vision

Abstract

This application discloses a frame forming control method and system based on industrial vision, relating to the field of industrial vision. The method includes: statistically analyzing the gray board conveying time of the target gray board; performing anomaly verification on the target gray board; outputting an alarm message if an anomaly is found on the target gray board; acquiring a first face paper image if no anomaly is found on the target gray board; determining the face paper adsorption pose of the target robotic arm; calculating the initial adsorption force and moving speed of the target robotic arm; setting the robotic arm adsorption parameters; controlling the target robotic arm to adsorb and move the target face paper directly above the target gray board; acquiring a second face paper image and a gray board image; performing pose estimation of the target face paper and target gray board, and setting the robotic arm bonding parameters; completing the bonding process to obtain a laminated gray board; and bending the laminated gray board to obtain a frame semi-finished product. This application can effectively improve the quality of finished products manufactured through frame forming.

Description

Technical Field

[0001] This application relates to the field of industrial vision, and more particularly to a frame forming control method and system based on industrial vision. Background Technology

[0002] In the packaging manufacturing industry, frame forming, as a key process, is widely used in the production of outer packaging for products such as food, beverages, wine, tea, cigarettes, and pharmaceuticals. It directly affects the aesthetics and practicality of the product. With the continuous improvement of people's living standards, the requirements for product packaging are also increasing. Therefore, improving the quality of frame forming has become a core demand in the industry. One of the steps that directly affects the quality of frame forming is the positioning and bonding of the face paper and the grey board.

[0003] Existing methods for improving the positioning and bonding accuracy between the face paper and the gray board involve using a pre-set control system to control a robotic arm to perform the positioning and bonding. This method lacks a real-time feedback mechanism and cannot adjust bonding parameters based on dynamic factors such as the deformation parameters of the face paper and gray board, temperature fluctuations, and humidity changes in the production environment. Especially for face paper and gray board made from different raw materials, the resulting deformation and the degree to which temperature and humidity parameters affect them vary. Relying on a pre-set control system for positioning and bonding may lead to deviations in the bonding position or air bubbles after bonding, affecting the quality of the frame forming and thus the aesthetics and practicality of the final product, causing significant losses to the packaging manufacturing plant. Summary of the Invention

[0004] This application provides a method and system for controlling frame forming based on industrial vision, which is used to improve the quality of finished products manufactured through frame forming.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, an industrial vision-based frame forming control method is provided, which includes:

[0007] When the target gray board is detected to be conveyed to the first preset position in the target production line, the timing device preset on the target production line is activated, and the gray board conveying time consumed by the target gray board to pass through the first preset position is counted by the timing device.

[0008] Collect production parameters of the target production line, and combine the production line parameters and gray board conveying time to complete the anomaly verification of the target gray board;

[0009] If the target gray board has an anomaly, an alarm message will be output.

[0010] If there is no gray board abnormality in the target gray board, the first image of the target paper is acquired using the image acquisition device preset in the target production line;

[0011] Based on the first paper image, the paper adsorption posture of the target robotic arm, which is pre-set above the target production line, is determined using image recognition technology.

[0012] The initial adsorption force and moving speed of the target robotic arm are calculated based on the production parameters.

[0013] Set the tissue paper adsorption pose, initial adsorption force, and moving speed as the robot arm adsorption parameters for the target robot arm;

[0014] When it is predicted that the target gray board will be delivered to the second preset position in the target production line, the target robotic arm is controlled to adsorb and move the target paper to directly above the target gray board according to the robotic arm adsorption parameters.

[0015] Acquire images of the second side of the target paper and the gray board, respectively;

[0016] The pose estimation of the target paper and the target gray board is completed by combining the second paper image and the gray board image respectively, and the robot arm fitting parameters are set for the target robot arm based on the pose estimation results;

[0017] The target robotic arm is controlled to complete the bonding process between the target gray board and the target face paper according to the robotic arm bonding parameters, so as to obtain the laminated gray board;

[0018] The gray board is bent and laminated using a pre-set frame forming equipment on the target production line to obtain a semi-finished frame.

[0019] Optionally, production parameters include production line parameters, raw material parameters, and environmental parameters. Production line parameters include production line operation parameters and production line structure parameters. Raw material parameters include gray board size, gray board material, face paper size, face paper material, face paper quality, and face paper thickness. Environmental parameters include production line temperature and production line humidity.

[0020] Optionally, the step of combining production line parameters and gray board conveying time to complete the anomaly verification of the target gray board includes the following steps:

[0021] The theoretical conveying time of the target gray board is calculated by combining the production line operating parameters and gray board size;

[0022] Calculate the absolute value of the time difference between the theoretical conveying time and the gray board conveying time;

[0023] If the absolute value of the time difference is greater than the preset time difference threshold, it is determined that the target gray board has a gray board anomaly.

[0024] If the absolute value of the time difference is less than or equal to the preset time difference threshold, it is determined that there is no grayboard anomaly in the target grayboard.

[0025] Optionally, determining the tissue adhesion pose of the target robotic arm, which is pre-set above the target production line, based on the first tissue image and using image recognition technology includes the following steps:

[0026] Preprocess the first sheet image;

[0027] An edge detection algorithm is used to perform edge detection on the preprocessed first sheet image, and the edge points of the preprocessed first sheet image are obtained based on the edge detection results.

[0028] The edge points are mapped to the parameter space using the Hough transform. Based on the edge point mapping results, all rectangular edges of the preprocessed first sheet are detected, and the pixel coordinates of all intersection points between all rectangular edges are calculated.

[0029] Complete the coordinate transformation of all intersection pixel coordinates, and determine the paper adsorption pose of the target robotic arm above the target production line based on the coordinate transformation results.

[0030] Optionally, the coordinate transformation of all intersection pixel coordinates is completed, and the paper adsorption pose of the target robotic arm above the target production line is determined based on the coordinate transformation results, including the following steps:

[0031] Acquire the device parameters of the image acquisition device, including device intrinsic parameters, depth information, and device pose;

[0032] For any intersection point pixel coordinates, the intersection point pixel coordinates are converted into paper imaging coordinates based on the intrinsic parameter information;

[0033] Based on the pinhole imaging model and combined with intrinsic parameter information and depth information, the three-dimensional transformation of the paper imaging coordinates is completed to obtain the coordinates of the acquisition device.

[0034] The rotation matrix and translation vector between the image acquisition device and the target robotic arm preset above the target pipeline are calculated by combining the device pose and the robotic arm pose in the pipeline structure parameters.

[0035] By combining the rotation matrix and translation vector to correct the coordinates of the acquisition device, the relative coordinates of the paper are obtained;

[0036] The edge center coordinates and face center coordinates of the target paper are calculated based on the relative coordinates of all the paper sheets. The paper adsorption position of the target robotic arm is then planned by combining the edge center coordinates and face center coordinates.

[0037] Calculate the direction vector of the line connecting the relative coordinates of adjacent sheets, and fit the normal vector of the target sheet based on the direction vector of the line.

[0038] The paper adsorption posture of the target robotic arm is planned based on the paper normal vector;

[0039] The tissue adsorption position and tissue adsorption posture are integrated into the tissue adsorption posture of the target robotic arm.

[0040] Optionally, calculating the initial adsorption force and moving speed of the target robotic arm based on production parameters includes the following steps:

[0041] The initial adsorption force of the target robotic arm is determined by combining environmental and raw material parameters and based on a pre-constructed adsorption force parameter table.

[0042] The vertical movement speed of the target robotic arm is determined based on the robotic arm pose in the pipeline operation parameters and pipeline structure parameters.

[0043] The horizontal movement speed of the target robotic arm is determined based on the raw material station spacing and vertical movement speed in the production line operating parameters and structural parameters of the production line.

[0044] The vertical and horizontal movement speeds are integrated into the target robotic arm's movement speed.

[0045] Optionally, combining the second paper image and the gray board image to perform attitude estimation for the target paper and the target gray board respectively, and setting the robotic arm fitting parameters for the target robotic arm based on the attitude estimation results, includes the following steps:

[0046] Preprocess the second face paper image, which includes a front face paper image and a back face paper image;

[0047] The edge detection algorithm was used to detect all the edges of the second paper image after preprocessing.

[0048] The preprocessed second face paper image is segmented based on all face paper edges to obtain the face paper region, which includes the face paper front region and the face paper back region.

[0049] Using image recognition technology to detect wrinkles on the front side of tissue paper;

[0050] If the wrinkle detection fails, the wrinkle type of the target paper is determined based on the wrinkle recognition result, and the initial adsorption force is adjusted according to the wrinkle type until the wrinkle detection passes.

[0051] If the wrinkle detection passes, the center point coordinates of all suction cup areas in the area on the back of the face paper are extracted, and the target robotic arm is judged to have adsorption deviation based on the center point coordinates.

[0052] If the target robotic arm has an adsorption deviation, the edge bonding parameters of the target paper are determined based on the adsorption deviation.

[0053] If the target robotic arm has no adsorption deviation, the edge sag curvature of the target paper is calculated using the least squares method, and the edge bonding parameters of the target paper are determined based on the edge sag curvature.

[0054] Obtain the device parameters of the image acquisition device, and combine the device parameters with the gray board image to locate the paper-attaching posture of the target robotic arm;

[0055] Set the edge bonding parameters and the face paper bonding pose to the robot arm bonding parameters of the target robot arm.

[0056] Optionally, extracting the center point coordinates of all suction cup areas within the area on the back of the paper, and determining whether the target robotic arm has adsorption deviation based on the center point coordinates includes the following steps:

[0057] Fit the minimum bounding rectangle of the back area of ​​the paper and extract the coordinates of all vertices of the minimum bounding rectangle;

[0058] Statistically analyze the pixel values ​​of the back area of ​​the face paper, and then use the Hough circle detection algorithm to detect all suction cup areas in the back area of ​​the face paper based on the pixel values.

[0059] Extract the center coordinates of all suction cup areas, and determine the center point coordinates of all suction cup areas based on the center coordinates of all suction cup areas;

[0060] By combining the coordinates of the center point and all vertices, the vertical distance between the center point of all suction cup areas and the edge of the back of the face paper is calculated, and the vertical distance is used to determine whether the target robotic arm has adsorption deviation.

[0061] Optionally, calculating the edge sag curvature of the target paper using the least squares method includes the following steps:

[0062] For any edge of the face paper, multiple edge sampling points are uniformly extracted from the edge of the face paper according to a preset interval parameter;

[0063] All edge sampling points are fitted to edge lines using the least squares method;

[0064] Calculate the perpendicular distance between all edge sampling points and the edge line;

[0065] The edge sag curvature of the target face paper is calculated based on the vertical distance of all edges of the face paper.

[0066] Secondly, this application provides a frame forming control system based on industrial vision, comprising:

[0067] The memory is configured to store instructions; and

[0068] The processor is configured to retrieve instructions from memory and, when executing the instructions, to implement the industrial vision-based frame forming control method as described in the first aspect.

[0069] By comparing the target gray board's conveying time at the first preset position, as measured by a timing device, with the theoretical conveying time calculated based on the production line operating parameters and gray board dimensions, it is determined whether the target gray board has any abnormalities. If an abnormality is found, an alarm message is output, reminding the production line workers to remove the target gray board with the abnormality. Only when the target gray board is free of abnormalities will the subsequent bonding process continue, thereby reducing production time and material waste. To improve the bonding accuracy between the target face paper and the target gray paper, various parameters of the target robotic arm need to be set reasonably before bonding. For example, this application uses a series of image processing steps to accurately locate the intersection pixel coordinates of the target face paper and converts these coordinates into the face paper adsorption pose of the target robotic arm. This reduces the deformation of the target face paper caused by gravity after being adsorbed by the robotic arm. Simultaneously, by combining environmental and material parameters to set the initial adsorption force of the target robotic arm, wrinkles in the target face paper due to excessive adsorption force can be effectively avoided. In summary, this application can effectively avoid wrinkles or bubbles when the target face paper is bonded to the target gray board, thereby improving the qualification rate of the final product, reducing raw material waste in the factory, and lowering production costs.

[0070] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0071] Figure 1 A flowchart illustrating a frame forming control method based on industrial vision provided in an embodiment of this application;

[0072] Figure 2 A schematic diagram illustrating the adsorption position of a suction cup and a target paper sheet, provided as an embodiment of this application;

[0073] Figure 3 This is an example diagram showing an adsorption deviation of a suction cup provided in an embodiment of this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0075] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0076] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0077] Figure 1 The illustration schematically shows a flowchart of a frame forming control method based on industrial vision according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a frame forming control method based on industrial vision, which may include the following steps:

[0078] S101. When the target gray board is detected to be conveyed to the first preset position in the target production line, the timing device preset on the target production line is activated, and the gray board conveying time consumed by the target gray board to pass through the first preset position is counted by the timing device.

[0079] In this embodiment, a photoelectric sensor, consisting of a transmitter and a receiver, is pre-installed at a first preset position on the target production line. These are mounted on opposite sides of the production line at the same horizontal level, ensuring that the light emitted by the transmitter is accurately received by the receiver. Together, they form a detection beam used to detect whether the target gray board has arrived at or departed from it. Simultaneously, a timing device, accurate to the millisecond level, is installed on the production line. This timing device includes a signal receiving module, a timing module, and a data storage module, and has been connected to the photoelectric sensor via circuitry and signal matching to ensure that the electrical signal emitted by the photoelectric sensor is received and accurately identified by the timing device. As the target gray board is continuously conveyed on the production line at a preset speed, its front end gradually enters the detection area between the transmitter and receiver of the photoelectric sensor as it approaches the first preset position. The instant the front end of the target gray board completely blocks the detection beam, the receiver, unable to receive the light emitted by the transmitter, immediately generates an activation signal and transmits this signal to the signal receiving module of the timing device via a connecting line. Upon receiving the start signal, the signal receiving module of the timing device immediately sends a start command to the timing module. The timing module then starts and enters timing mode the instant it receives the command, accurately recording the start time. When the rear end of the target gray board completely leaves the detection area and no longer blocks the detection beam, the receiver receives the light emitted by the transmitter again, generating a stop signal which is transmitted to the signal receiving module of the timing device. Upon receiving the stop signal, the signal receiving module sends a stop command to the timing module, which immediately stops timing and records the stop time. Finally, by calculating the time difference between the start time and the stop time, the gray board transport time consumed by the target gray board passing through the first preset position can be obtained.

[0080] S102. Collect the production parameters of the target production line, and combine the production line parameters and gray board conveying time to complete the anomaly verification of the target gray board.

[0081] In this embodiment, the production line operating parameters include the production line speed and the production line pause time. The theoretical conveying time of the target gray board can be calculated by dividing the gray board size by the production line speed. Then, the time difference between gray board conveying times is subtracted from the theoretical conveying time, and the absolute value of the time difference is obtained. If the absolute value of the time difference is greater than a preset time difference threshold, it indicates that the target gray board is too large or too small compared to a normal gray board, i.e., there is a gray board abnormality. The causes of gray board abnormalities may include edge damage or positional misalignment. This method allows for the pre-elimination of target gray boards with serious abnormalities, saving production resources on the target production line. Furthermore, by avoiding the use of damaged or abnormally sized target gray boards in packaging box production, the factory's product qualification rate is indirectly improved.

[0082] S103. If the target gray board has an abnormality, an alarm message will be output.

[0083] In this embodiment, when an anomaly is detected in the target gray board, a visual or audible alarm is triggered on the target production line. The visual alarm is typically a warning light; when an anomaly is detected, the warning light illuminates, alerting relevant operators to handle the abnormal target gray board. An audible alarm can be a buzzer. Simultaneously, the frequency of gray board anomalies is recorded. If the anomaly frequency exceeds a preset threshold, the preceding cutting processes need to be investigated for anomalies. This aims to improve the factory's raw material utilization rate and reduce production costs.

[0084] S104. If there is no gray board abnormality in the target gray board, the first face paper image of the target face paper is acquired using the image acquisition device preset in the target production line.

[0085] In this embodiment, when an abnormality is detected in the target gray board, subsequent steps can be performed. To ensure that the target robotic arm can accurately adsorb the target paper and move it directly above the target panel, an image acquisition device positioned above the target assembly line is needed to acquire a first image of the target paper. The image acquisition device is an industrial camera, such as a 2D area scan camera or a 3D depth camera.

[0086] S105. Based on the first paper image and using image recognition technology, determine the paper adsorption posture of the target robotic arm that is preset above the target production line.

[0087] In this embodiment, the first sheet image is preprocessed, including grayscale conversion and filtering for noise reduction. Next, edge points of the first sheet image are extracted using an edge detection algorithm; the commonly used edge detection algorithm is the Canny algorithm. Then, the Hough line detection algorithm is used to detect the four rectangular edges of the target sheet in the first sheet image. Based on the polar coordinates of the four rectangular edges and using Cramer's rule, the pixel coordinates of the intersection points between the rectangular edges are calculated. The intersection pixel coordinates are then converted into sheet adhesion poses that can be directly used to set the target robotic arm. According to the device parameters of the image acquisition device, the intersection pixel coordinates are first converted into sheet imaging coordinates, which are the physical pose descriptions of the edge intersection points (the points corresponding to the intersection pixel coordinates) of the target sheet on the imaging plane of the image acquisition device. Next, these two-dimensional sheet imaging coordinates are converted into three-dimensional coordinates, namely, the acquisition device coordinates. The acquisition device coordinates refer to the coordinates of the edge intersection points in the image acquisition device coordinate system constructed with the optical center of the image acquisition device as the origin. Based on the pinhole imaging model, the three-dimensional transformation of the sheet imaging coordinates is completed to obtain the acquisition device coordinates. Next, based on the relative positions of the image acquisition device and the target robotic arm, the coordinates of the acquisition device are translated and rotated to obtain the three-dimensional coordinates of the intersection point of the target paper's edges in the target robotic arm's coordinate system, i.e., the relative coordinates of the paper. Then, to minimize excessive sagging of the target paper's edges when the target robotic arm adsorbs it, the paper's adsorption position and posture need to be planned based on the relative coordinates, i.e., the paper's adsorption pose. Through this method, the paper's adsorption pose can be accurately calculated, minimizing significant edge sagging or wrinkles after the target paper is adsorbed by the target robotic arm. This prevents wrinkles or air bubbles from forming when the target paper is bonded to the target gray board, thereby improving the quality of the factory's products and ensuring factory efficiency.

[0088] S106. Calculate the initial adsorption force and moving speed of the target robotic arm based on the production parameters.

[0089] In this embodiment, an experimental robotic arm identical to the target robotic arm is first used, with identical suction cup material, suction cup aperture, and suction cup layout. Next, an adsorption force experiment is conducted using the face paper used in all packaging boxes produced by the factory, and the experimental results are compiled into an adsorption force parameter table. This table allows for the lookup of the corresponding minimum and maximum adsorption forces based on environmental and material parameters, and the average of these two values ​​is taken as the initial adsorption force. Then, the vertical distance between the target robotic arm and the target production line is calculated based on the robotic arm pose in the production line operation parameters; the vertical axis coordinate of the robotic arm pose represents the vertical distance. The production line operation parameters include production line speed and production line pause time. The pause time refers to the period of time during which the target gray board is paused when it reaches the second preset position, allowing the target face paper to accurately adhere to the target base plate. To save time in the bonding process, the target robotic arm completes the adsorption step of the target face paper and moves it directly above the target gray board at the same time the target gray board is transported to the second preset position. When the target gray board stops moving, the subsequent bonding steps are immediately executed. First, the minimum vertical movement speed of the target robotic arm is calculated based on the vertical distance between the target robotic arm and the target production line, as well as the production line pause time. This is to ensure the smooth execution of the bonding steps. Since the target robotic arm needs to perform structural adjustments during the bonding steps, the minimum vertical movement speed is appropriately increased based on the production experience of this frame semi-finished product. The transport time of the target gray board from the preset first position to the preset second position is calculated based on the production line speed. Since the target robotic arm also needs to move vertically during the adsorption step, the vertical movement time is calculated based on the vertical movement speed and vertical distance. After subtracting the vertical movement time from the transport time, the minimum horizontal movement speed is calculated based on the distance between the raw material stations. Similarly, based on the production experience of this frame semi-finished product, the minimum horizontal movement speed is appropriately increased to become the horizontal movement speed of the target robotic arm. By integrating the vertical and horizontal movement speeds, the target robotic arm's movement speed is obtained. This step ensures seamless integration of the target robotic arm's movements with the target production line's cycle time, significantly improving production efficiency.

[0090] S107. Set the paper adsorption pose, initial adsorption force, and moving speed as the adsorption parameters of the target robotic arm.

[0091] In this embodiment, the tissue paper adsorption posture, initial adsorption force, and moving speed are fed back to the controller of the target robotic arm. The controller sets the robotic arm adsorption parameters of the target robotic arm. In subsequent steps, the target robotic arm moves the adsorption box of the target tissue paper according to the set robotic arm control parameters.

[0092] S108. When it is predicted that the target gray board will be transported to the second preset position in the target production line, control the target robotic arm to adsorb and move the target paper to directly above the target gray board according to the robotic arm adsorption parameters.

[0093] In this embodiment, the time point at which the target gray board is transported to the second preset position in the target production line can be calculated based on the distance between the first preset position and the second preset position, as well as the production line speed. Before reaching this time point, the target robotic arm begins to descend, adsorb the target face paper, rise, and move horizontally. When the target gray board reaches the second preset position, the target robotic arm, adsorbing the target face paper, is just above the target gray board, facilitating the subsequent face paper lamination step.

[0094] S109. Acquire images of the second side of the target paper and the gray board, respectively.

[0095] In this embodiment, an image acquisition device of the same model is used to acquire images of the second face paper and the gray board. The second face paper image includes an image of the front of the face paper and an image of the back of the face paper. The gray paper image and the back of the face paper image are acquired by an image acquisition device positioned directly above the target production line, while the front of the face paper image is acquired using an image acquisition device positioned to the side of the target production line.

[0096] S110. Combine the second paper image and the gray board image to perform attitude estimation of the target paper and the target gray board respectively, and set the robot arm fitting parameters for the target robot arm based on the attitude estimation results.

[0097] In this embodiment, the second face paper image is first preprocessed, including grayscale conversion, filtering and noise reduction, and contrast enhancement. Next, an edge detection algorithm is used to detect all face paper edges in the preprocessed second face paper image. Face paper edges refer to edge points; the commonly used edge detection algorithm is Canny edge detection. Based on the face paper edge filtering results, the face paper region is segmented from the background of the second face paper image. This step is to reduce the impact of complex background regions on subsequent wrinkle recognition. Since face paper wrinkles mainly manifest as local grayscale anomalies and the presence of fine edges, during the edge detection stage, if an edge point exists within the face paper region, and the edge length obtained after fitting the edge point is greater than a preset length threshold, then the corresponding position in the face paper region is marked as an edge anomaly. During the wrinkle detection stage, if the target face paper is not marked as an edge anomaly, the target face paper wrinkle detection is considered successful. If the target paper is marked as having edge anomalies, a small sliding window (e.g., 5×5 pixels) is set and slids pixel by pixel across the back area of ​​the paper. The variance of the grayscale values ​​of each pixel within the window is calculated. The larger the variance, the more drastic the grayscale fluctuation within the window. The back area of ​​the paper with a variance greater than a preset variance threshold is marked as a grayscale anomaly region. The region marked as having edge anomalies is considered an edge anomaly region. If the overlap between the edge anomaly region and the grayscale anomaly region is greater than a preset overlap threshold, the wrinkle detection of the target paper is deemed to have failed.

[0098] The overlapping area between edge anomaly regions and grayscale anomaly regions is defined as the wrinkled region. The number of pixels in the wrinkled region is counted, and a conversion coefficient between pixels and actual size is obtained through camera calibration, such as a checkerboard calibration method. The product of the number of pixels in the wrinkled region and the conversion coefficient is calculated to obtain the wrinkled area. Simultaneously, the area of ​​the face paper is calculated based on the face paper size. The wrinkled area is divided by the face paper area to obtain the wrinkle percentage. Next, connected component analysis is performed on the wrinkled regions. The 8-neighborhood method can be used, where two adjacent pixels that are both in wrinkled regions are considered connected, and adjacent wrinkled regions are merged into a single connected component. Based on the wrinkle percentage, the wrinkles in the target face paper are divided into local wrinkles and global wrinkles. Then, based on the number of wrinkles, they are further divided into few wrinkles, medium wrinkles, and many wrinkles. For local wrinkles, the suction force of the suction cups near the wrinkle is appropriately reduced according to the number of wrinkles. For global wrinkles, the suction force of all suction cups is reduced according to the number of wrinkles.

[0099] If the wrinkle detection passes, further inspection is needed to ensure the target robotic arm's suction cup is centered on the target face paper. Adsorption deviation will cause varying degrees of sagging at the edges of the face paper. Directly laminating the face paper in this case may result in wrinkles on the final laminated grey board, affecting the quality of the final product. Next, the edge lamination parameters of the target face paper are determined based on the direction of the adsorption deviation. These parameters include the edge lamination sequence and local lifting amplitude. If the target robotic arm has no adsorption deviation, the edge sag curvature of all long and wide sides of the target face paper needs to be calculated using the least squares method. The local lifting amplitude of the target robotic arm and the edge lamination sequence of the face paper are adjusted based on the edge sag curvature to prevent wrinkles on the laminated grey board caused by edge sag during lamination. The device parameters of the image acquisition device are obtained. Combined with these parameters and the grayscale image, the paper-attaching pose of the target robotic arm is located. The calculation process for the paper-attaching pose is the same as that for the paper-adsorption pose. It primarily involves preprocessing the grayscale image, using an edge detection algorithm to detect edges, and then locating all intersection pixel coordinates of the grayscale image using the Hough interpolation algorithm. After a series of transformations, the paper-attaching position and pose for the target robotic arm are obtained. Integrating these two parameters yields the paper-attaching pose of the target robotic arm. Finally, the edge-attaching parameters and the paper-attaching pose are set as the robotic arm's attachment parameters. These steps ensure precise attachment of the target paper to the target grayscale image without wrinkles, thus guaranteeing the aesthetic appeal of the packaging boxes produced in this factory.

[0100] S111. Control the target robotic arm to complete the bonding process between the target gray board and the target face paper according to the robotic arm bonding parameters, and obtain the laminated gray board.

[0101] In this embodiment, the target robotic arm is controlled to adjust the target robotic arm according to the bonding parameters and move downwards to bond. It stays in the bonding area for a certain period of time, such as 0.5s, to ensure that the target paper and the target gray board are pressed together. After bonding is completed, the target robotic arm is controlled to move upwards to the original position, that is, the position where it stayed before moving downwards. Then it moves horizontally to the top of the next target paper and repeats the above process to continue to complete the bonding step of the next target paper and the target gray board.

[0102] S112. Using a pre-set frame forming device on the target production line, bend and laminate the gray board to obtain a semi-finished frame.

[0103] In this embodiment, the laminated gray board is transported to the frame forming station and fixed in a preset position by positioning clamps (such as pneumatic baffles) on both sides of the production line. The laminated gray board is then bent in a preset order by a frame forming device, such as a pneumatic folding machine, which is pre-set on the target production line. This forms the four sides of the lidless box, resulting in a semi-finished frame.

[0104] In one embodiment, the production parameters include production line parameters, raw material parameters, and environmental parameters. The production line parameters include production line operation parameters and production line structure parameters. The raw material parameters include gray board size, gray board material, face paper size, face paper material, face paper quality, and face paper thickness. The environmental parameters include production line temperature and production line humidity.

[0105] In this embodiment, the production line operating parameters include the production line speed and the production line pause time. The pause time refers to the temporary cessation of transport when the target gray board is transported to the second preset position to ensure that the target face paper can be accurately adhered to the target gray board. The production line structural parameters include the raw material station spacing and the robotic arm pose. The raw material station spacing refers to the distance between the position of the target face paper during the adsorption step and the position of the target gray board (the second preset position) during the lamination step. The robotic arm pose refers to the coordinates of the origin of the target robotic arm coordinate system in the world coordinate system. Based on these coordinates, the vertical distance between the target robotic arm and the target production line can be calculated. The raw material parameters can be directly obtained from the factory's database according to the type of the final product. Environmental parameters can be obtained in real time from temperature and humidity sensors preset around the target production line.

[0106] In one embodiment, the anomaly verification of the target gray board, which combines production line parameters and gray board conveying time, includes the following steps:

[0107] The theoretical conveying time of the target gray board is calculated by combining the production line operating parameters and gray board size;

[0108] Calculate the absolute value of the time difference between the theoretical conveying time and the gray board conveying time;

[0109] If the absolute value of the time difference is greater than the preset time difference threshold, it is determined that the target gray board has a gray board anomaly.

[0110] If the absolute value of the time difference is less than or equal to the preset time difference threshold, it is determined that there is no grayboard anomaly in the target grayboard.

[0111] In this embodiment, the production line operating parameters include the production line speed and the production line pause time. The theoretical transport time of the target gray board can be calculated by dividing the gray board size by the production line speed. Then, the theoretical transport time is subtracted from the time difference between gray board transport times, and the absolute value of the time difference is obtained. If the absolute value of the time difference is greater than a preset time difference threshold, it indicates that the target gray board is either too large or too small compared to a normal gray board, i.e., there is a gray board anomaly. The causes of gray board anomalies may include edge damage or positional deviation. For example, when the target gray board has edge damage, its effective length through the first preset position is reduced, resulting in a shorter gray board transport time than the theoretical transport time. Conversely, when the target gray board's position is deviated, its path through the first preset position may become longer, causing the gray board transport time to be longer than the theoretical transport time. Since the time difference of the target gray board exceeds the time difference threshold, it indicates a serious anomaly that is difficult to compensate for by adjusting the parameters of the target robotic arm. This could lead to serious defects in the final packaged boxes, such as internal wrinkles, rendering them unsaleable and wasting production time and paper. Therefore, an alarm is directly output to remind factory staff to remove the target gray board with the anomaly. There are many reasons why the target gray board might exhibit this anomaly. For example, edge damage could be due to protrusions, burrs, or foreign objects on the conveyor track of the production line, causing friction and collision between the gray board edge and these parts. Alternatively, it could be due to severe wear or dull blades in the previous cutting process, resulting in edge tearing when cutting the target gray board. Gray board misalignment could also be caused by the impact of surrounding airflow.

[0112] In one embodiment, determining the tissue-adsorption pose of the target robotic arm, which is pre-set above the target assembly line, based on the first tissue image and using image recognition technology includes the following steps:

[0113] Preprocess the first sheet image;

[0114] An edge detection algorithm is used to perform edge detection on the preprocessed first sheet image, and the edge points of the preprocessed first sheet image are obtained based on the edge detection results.

[0115] The edge points are mapped to the parameter space using the Hough transform. Based on the edge point mapping results, all rectangular edges of the preprocessed first sheet are detected, and the pixel coordinates of all intersection points between all rectangular edges are calculated.

[0116] Complete the coordinate transformation of all intersection pixel coordinates, and determine the paper adsorption pose of the target robotic arm above the target production line based on the coordinate transformation results.

[0117] In this embodiment, the preprocessing steps include grayscale conversion and filtering / denoising. A common grayscale conversion method is the weighted average method, which converts the color first-side paper image into a grayscale image. Filtering / denoising methods include Gaussian filtering and median filtering, which reduce noise interference in subsequent steps. Edge points in the first-side paper image are extracted using an edge detection algorithm. A commonly used edge detection algorithm is the Canny algorithm. The Canny algorithm calculates the horizontal and vertical gradients of pixels in the first-side paper image using the Sobel operator, obtaining the gradient magnitude. It iterates through the first-side paper image, retaining only pixels with local maximum values ​​in the gradient direction, refining the edges to a single pixel width to eliminate blurring effects, and setting high and low thresholds: pixels above the high threshold are directly identified as edges; pixels between the high and low thresholds are retained if connected to the high-threshold edge, otherwise suppressed. Finally, all retained edge points are output. Next, the Hough transform is used to map the edge points to the parameter space, that is, to convert the two-dimensional coordinate representation of the pixel into polar coordinates, such as ρ = xcosθ + ysinθ, where ρ is the polar radius obtained after the pixel is mapped, θ is the polar angle, and x and y are the two-dimensional x-coordinate and two-dimensional y-coordinate of the pixel. All possible (ρ, θ) combinations for each edge point are counted (θ ranges from 0 to 180°, with a resolution of 1°; ρ resolution is set to 1 pixel to adapt to the target paper size). The number of votes for each (ρ, θ) is counted by an accumulator, and (ρ, θ) pairs with a number of votes higher than a preset voting threshold (e.g., accumulator value > 15% of the total number of edge points) are selected as candidate lines in the corresponding image space. Then, based on the prior knowledge of the rectangle of the packaging boxes produced by the factory (parallel opposite sides, perpendicular adjacent sides), the candidate lines are classified, such as the parallel group: lines with a difference of θ ≤ 2° are grouped together (e.g., horizontal side θ ≈ 0°, vertical side θ ≈ 90°), and each group must have exactly 2 lines (corresponding to a pair of opposite sides of the rectangle). Perpendicular Verification: The difference in angle (θ) between two sets of straight lines must be approximately 90° (e.g., the 0° and 90° sets) to verify the perpendicular relationship between adjacent sides. Several straight lines are ultimately selected. Since the rectangular edges of the target paper should be located around its perimeter, the four rectangular edges of the target paper are obtained based on the positions of all the straight lines. Then, the pixel coordinates of the intersection points between the rectangular edges are calculated using the polar coordinates of the four rectangular edges and Cramer's rule. Specifically, assuming the four rectangular edges are L1(ρ1,θ1), L2(ρ2,θ2), L3(ρ3,θ3), and L4(ρ4,θ4), where L1∥L3, L2∥L4, and L1⊥L2, a system of equations is then established for the rectangular edges with perpendicular relationships. Taking rectangular edge L1 as an example, the system of equations is as follows:

[0118] ​​

[0119] Next, the pixel coordinates of the intersection point between rectangle side L1 and rectangle side L2 are solved using Cramer's rule. The same method can be used to solve for the pixel coordinates of other intersection points. Cramer's rule is an important theorem in linear algebra for solving systems of linear equations with the same number of variables and equations. Its core idea is to directly obtain the solution to the system of equations by calculating the determinant, which is especially applicable when the coefficient matrix of the system of equations is invertible (i.e., the determinant is not zero).

[0120] Next, the intersection pixel coordinates are converted into the paper-adsorption pose that can be directly used to set the target robotic arm. Based on the device parameters of the image acquisition device, the intersection pixel coordinates are first converted into paper imaging coordinates. The paper imaging coordinates describe the physical pose of the edge intersection points (the points corresponding to the intersection pixel coordinates) of the target paper on the imaging plane of the image acquisition device. Then, these two-dimensional paper imaging coordinates are converted into three-dimensional coordinates, namely the acquisition device coordinates. The acquisition device coordinates refer to the coordinates of the edge intersection points in the image acquisition device coordinate system constructed with the optical center of the image acquisition device as the origin. The three-dimensional transformation of the paper imaging coordinates is completed based on the pinhole imaging model to obtain the acquisition device coordinates. The pinhole imaging model originates from the pinhole imaging experiment, where a baffle with a small hole separates the "object" and the "screen." Light emitted from the object passes through the small hole and forms an inverted image on the screen. The pinhole imaging is abstracted as an image acquisition device model (camera model). The optical center corresponds to the position of the small hole and is the "origin" where the light converges. It is also the simplified optical center of the camera lens. The object space contains the object being photographed, i.e., the paper, and the imaging plane corresponds to the photosensitive element of the image acquisition device. Based on the pinhole imaging model, and combining the intrinsic parameters and depth information of the image acquisition device, the two-dimensional paper imaging coordinates can be transformed into three-dimensional acquisition device coordinates. Then, according to the relative position of the image acquisition device and the target robotic arm, the acquisition device coordinates are translated and rotated to obtain the three-dimensional coordinates of the intersection point of the target paper's edges in the target robotic arm coordinate system, i.e., the relative coordinates of the paper. Next, to minimize the sagging of the target paper's edges when the target robotic arm adsorbs the target paper, the paper adsorption position and posture need to be planned based on the relative coordinates, i.e., the paper adsorption pose. The paper adsorption position refers to the contact point between the target robotic arm's suction cup and the target paper. If there is only one suction cup, its center point should coincide with the center point of the target paper as much as possible. Alternatively, if there are multiple suction cups, the minimum bounding rectangle of all suction cups can be constructed, ensuring that the center point of the minimum bounding rectangle coincides with the center point of the target paper, thus achieving precise adsorption. Figure 2If the target paper is offset on the horizontal plane, the suction angle of the target robotic arm must also be adjusted. This requires ensuring that all four sides of the smallest bounding rectangle of the target robotic arm's suction cup are parallel to the four sides of the target paper. Furthermore, the paper's suction posture refers to the posture of the target robotic arm's suction plane. To avoid wrinkles in the target paper, the suction plane of the target robotic arm must be parallel to the target paper. Using the above method, the paper's suction posture can be accurately calculated, minimizing significant edge sagging or wrinkles after the target paper is suctioned by the target robotic arm. This prevents wrinkles or air bubbles when the target paper is bonded to the target gray board, thereby improving the quality of the factory's products and ensuring factory efficiency.

[0121] In one embodiment, completing the coordinate transformation of all intersection pixel coordinates and determining the paper-adsorption pose of the target robotic arm above the target assembly line based on the coordinate transformation result includes the following steps:

[0122] Acquire the device parameters of the image acquisition device, including device intrinsic parameters, depth information, and device pose;

[0123] For any intersection point pixel coordinates, the intersection point pixel coordinates are converted into paper imaging coordinates based on the intrinsic parameter information;

[0124] Based on the pinhole imaging model and combined with intrinsic parameter information and depth information, the three-dimensional transformation of the paper imaging coordinates is completed to obtain the coordinates of the acquisition device.

[0125] The rotation matrix and translation vector between the image acquisition device and the target robotic arm preset above the target pipeline are calculated by combining the device pose and the robotic arm pose in the pipeline structure parameters.

[0126] By combining the rotation matrix and translation vector to correct the coordinates of the acquisition device, the relative coordinates of the paper are obtained;

[0127] The edge center coordinates and face center coordinates of the target paper are calculated based on the relative coordinates of all the paper sheets. The paper adsorption position of the target robotic arm is then planned by combining the edge center coordinates and face center coordinates.

[0128] Calculate the direction vector of the line connecting the relative coordinates of adjacent sheets, and fit the normal vector of the target sheet based on the direction vector of the line.

[0129] The paper adsorption posture of the target robotic arm is planned based on the paper normal vector;

[0130] The tissue adsorption position and tissue adsorption posture are integrated into the tissue adsorption posture of the target robotic arm.

[0131] In this embodiment, the image acquisition device refers to an industrial camera, such as a 2D area array camera or a 3D depth camera. The device intrinsic parameters refer to the camera intrinsic parameters, which are calculated by a camera calibration algorithm, such as Zhang's calibration method. These parameters mainly include focal length, principal point, and pixel physical size. Depth information refers to the distance from the target point in the scene (such as the center point of the target paper) to the optical center of the camera. This distance can be directly acquired by a depth camera. If it is a stereo camera, the same target can be photographed by the left and right cameras, the pixel parallax can be calculated, and the depth information can be calculated by combining the baseline (distance between the two cameras) and the camera focal length. Alternatively, the depth information can be calculated based on the spatial coordinate difference between the target paper and the image acquisition device, according to the spatial coordinates of the target paper in the world coordinate system.

[0132] The pixel coordinate system takes the top left corner of the image as its origin, while the imaging coordinate system takes the intersection of the optical axis and the photosensitive chip (principal point) as its origin. The transformation needs to eliminate the effect of pixel discretization. The transformation formula is as follows:

[0133]

[0134] in, , Each pixel on the image acquisition device is located at... (Horizontal axis) Physical dimensions in the (vertical axis) direction (unit: mm / pixel, determined by intrinsic parameters). , This represents the coordinates of the principal point in the pixel coordinate system, determined based on intrinsic parameter information. , ) represents the pixel coordinates of the i-th intersection point. , The image coordinates of the paper, i=1,2,3,4.

[0135] Next, based on the pinhole imaging model, there is a perspective relationship between points on the imaging plane and three-dimensional points in the coordinate system of the image acquisition device. Therefore, based on the focal length information in the intrinsic parameters of the image acquisition device, and the depth of the intersection pixel in the coordinate system of the image acquisition device corresponding to the intersection pixel coordinates (i.e., depth information), the paper imaging coordinates can be converted to the acquisition device coordinates using a three-dimensional transformation formula, as follows:

[0136]

[0137] in, For depth information, For focal length information, ( , , () represents the coordinates of the i-th acquisition device.

[0138] Next, based on the device pose and the robotic arm pose, the rotation matrix and translation vector between the image acquisition device and the target robotic arm are calculated. The transposed pose refers to the three-dimensional coordinates of the origin of the image acquisition device's coordinate system in a preset world coordinate system. The robotic arm pose refers to the three-dimensional coordinates of the origin of the target robotic arm's coordinate system in the same world coordinate system. For example, the world coordinate system could be centered at the starting point of the target conveyor belt, with the direction of movement along the conveyor belt as the positive horizontal axis, the horizontal direction perpendicular to the surface of the conveyor belt as the vertical axis, and the direction perpendicular to the ground upwards as the positive vertical axis. Based on the three-dimensional coordinates of the image acquisition device and the target robotic arm in the same world coordinate system, their rotation matrix and translation vector can be directly calculated. Finally, based on the transposed pose and the robotic arm pose, the rotation angle of the image acquisition device's coordinate system about the vertical axis of the target robotic arm's coordinate system is calculated. Substituting this rotation angle into the rotation matrix calculation formula, we obtain the rotation matrix between the two, which is as follows:

[0139]

[0140] The steps for calculating the rotation angle include: first, selecting two orthogonal unit vectors (such as the X-axis and Y-axis) in the coordinate system of the image acquisition device; then, through coordinate transformation (i.e., subtracting the position offset of the origins of the two coordinate systems), obtaining the coordinates of the two orthogonal unit vectors in the target robotic arm coordinate system; and finally, calculating the inverse tangent value of these coordinates to obtain the rotation angle. The translation vector is the coordinate difference between the origin of the target robotic arm coordinate system and the origin of the image acquisition device coordinate system.

[0141] The rotation matrix describes the mounting angle of the image acquisition device relative to the target robotic arm base, while the translation vector represents the positional offset between the origin of the image acquisition device's coordinate system and the origin of the robotic arm's coordinate system. Given the rotation matrix and translation vector, the coordinates of the acquisition device can be corrected to the relative coordinates of the paper in the target robotic arm's coordinate system using the following formula:

[0142]

[0143] in, For rotation matrix, , and These represent the coordinate differences between the origin of the target robotic arm's coordinate system and the origin of the image acquisition device's coordinate system on the horizontal, vertical, and y-axis, respectively. It refers to matrix transpose.

[0144] After converting all intersection pixel coordinates to relative coordinates of the target paper, the average of the relative coordinates of two surfaces on the same edge is calculated to obtain the edge center coordinates. The edge center coordinates refer to the coordinates of the center point of the edge of the target paper. Then, the average of the four relative coordinates of the target paper is calculated to obtain the face center coordinates. The face center coordinates refer to the coordinates of the exact center point of the target paper. The edge center coordinates are used to ensure that the suction cup adsorption angle of the target robotic arm is parallel or perpendicular to the angle of the center line of the target paper. The face center coordinates are used to ensure that the adsorption center of the suction cup is the exact center of the target paper. For example, if there is a row of suction cups at the adsorption end of the target robotic arm, the adsorption position of the suction cup located at the exact center is the exact center of the target paper. This ensures that the adsorption force is evenly distributed on the target paper and minimizes the risk of the target paper shifting or deforming during the adsorption process. For the relative coordinates of two adjacent sheets, the direction vector of the connecting line is calculated. This connecting line direction vector is also the direction vector of the four sides of the target sheet. Based on the calculated connecting line direction vector, the normal vector of the target sheet is fitted. This is used to plan the suction posture of the target robotic arm's suction cup, ensuring that the suction cup plane is aligned with the normal vector direction, i.e., parallel to the target sheet plane, thus ensuring uniform force on the target sheet. Integrating the sheet's suction position and suction posture, the sheet suction pose of the target robotic arm is obtained.

[0145] In one embodiment, calculating the initial adsorption force and moving speed of the target robotic arm based on production parameters includes the following steps:

[0146] The initial adsorption force of the target robotic arm is determined by combining environmental and raw material parameters and based on a pre-constructed adsorption force parameter table.

[0147] The vertical movement speed of the target robotic arm is determined based on the robotic arm pose in the pipeline operation parameters and pipeline structure parameters.

[0148] The horizontal movement speed of the target robotic arm is determined based on the raw material station spacing and vertical movement speed in the production line operating parameters and structural parameters of the production line.

[0149] The vertical and horizontal movement speeds are integrated into the target robotic arm's movement speed.

[0150] In this embodiment, an experimental robotic arm identical to the target robotic arm is first used, with identical suction cup material, suction cup aperture, and suction cup layout. Next, an adsorption force experiment is conducted using the face paper used in all the packaging boxes produced by the factory. Various parameters of the face paper used in different packaging boxes are recorded, including thickness, weight, and material. Simultaneously, the smoothness of the face paper is categorized into high, medium, and low grades based on its material. Humidifiers and air conditioners are used to set different levels of temperature and humidity, aiming to cover the actual temperature and humidity range of the production process as comprehensively as possible. Temperature and humidity can be divided into different levels, from high to low, such as Level 1, Level 2, Level 3, and Level 4. First, set an initial adsorption force, such as a negative pressure of 1.0 kPa. Control the robotic arm to adsorb tissue paper according to the initial adsorption force, while using an infrared sensor to monitor the amount of tissue paper sagging at the edge. When the sagging exceeds a preset first threshold (e.g., 1 mm) or the tissue paper falls off, gradually increase the fixed negative pressure to improve the adsorption force, for example, by 0.2 kPa each time, until the sagging is less than a second threshold (e.g., 0.5 mm). Record the negative pressure at this point as the minimum adsorption force. Then, continue to gradually increase the fixed negative pressure based on the minimum adsorption force, while using a high-definition camera to capture images of the tissue paper. When wrinkles appear in the tissue paper image, record the negative pressure at this point, subtract the fixed negative pressure from it, and use this as the maximum adsorption force. Repeat the same method for multiple experiments, and compile the experimental results into an adsorption force parameter table. Using this adsorption force parameter table, the corresponding minimum and maximum adsorption forces can be found based on environmental and raw material parameters. The average of the two is taken as the initial adsorption force.

[0151] Next, the vertical distance between the target robotic arm and the target production line is calculated based on the robotic arm pose in the production line operation parameters. The vertical distance is represented by the vertical axis coordinate of the robotic arm pose. The production line operation parameters include the production line speed and the production line pause time. The production line pause time refers to the period of time during which the target gray board is paused when it is transported to the second preset position, allowing the target face paper to be accurately bonded to the target base plate. To save time in the bonding process, the target robotic arm completes the target face paper adsorption step and moves the target face paper directly above the target gray board at the same time the target gray board is transported to the second preset position. When the target gray board stops moving, the subsequent series of bonding steps are immediately executed. The minimum vertical movement speed of the target robotic arm is first calculated based on the vertical distance between the target robotic arm and the target production line and the production line pause time. This is to ensure the smooth execution of the bonding steps. Since the target robotic arm needs to perform structural adjustments and other operations during the bonding steps, the minimum vertical movement speed is appropriately increased based on the production experience of this frame semi-finished product. The transport time of the target gray board from the preset first position to the preset second position is calculated based on the production line speed. Since the target robotic arm also needs to move vertically during the adsorption step, the vertical movement time is calculated based on the vertical movement speed and vertical distance. After subtracting the vertical movement time from the transport time, the minimum horizontal movement speed is calculated by combining it with the distance between the raw material stations. Similarly, based on the production experience of this semi-finished frame, the minimum horizontal movement speed is appropriately increased to obtain the horizontal movement speed of the target robotic arm. The vertical and horizontal movement speeds are integrated to obtain the movement speed of the target robotic arm. This step can seamlessly connect the operation of the target robotic arm with the operating rhythm of the target production line as much as possible, greatly improving production efficiency.

[0152] In one embodiment, the pose estimation of the target paper and the target gray board is completed by combining the second paper image and the gray board image, and the robotic arm fitting parameters are set for the target robotic arm based on the pose estimation results, including the following steps:

[0153] Preprocess the second face paper image, which includes a front face paper image and a back face paper image;

[0154] The edge detection algorithm was used to detect all the edges of the second paper image after preprocessing.

[0155] The preprocessed second face paper image is segmented based on all face paper edges to obtain the face paper region, which includes the face paper front region and the face paper back region.

[0156] Using image recognition technology to detect wrinkles on the front side of tissue paper;

[0157] If the wrinkle detection fails, the wrinkle type of the target paper is determined based on the wrinkle recognition result, and the initial adsorption force is adjusted according to the wrinkle type until the wrinkle detection passes.

[0158] If the wrinkle detection passes, the center point coordinates of all suction cup areas in the area on the back of the face paper are extracted, and the target robotic arm is judged to have adsorption deviation based on the center point coordinates.

[0159] If the target robotic arm has an adsorption deviation, the edge bonding parameters of the target paper are determined based on the adsorption deviation.

[0160] If the target robotic arm has no adsorption deviation, the edge sag curvature of the target paper is calculated using the least squares method, and the edge bonding parameters of the target paper are determined based on the edge sag curvature.

[0161] Obtain the device parameters of the image acquisition device, and combine the device parameters with the gray board image to locate the paper-attaching posture of the target robotic arm;

[0162] Set the edge bonding parameters and the face paper bonding pose to the robot arm bonding parameters of the target robot arm.

[0163] In this embodiment, the front image of the face paper refers to the image of the side of the target face paper facing the target production line, and the back image refers to the image of the side of the face paper being attracted by the suction cup of the target robotic arm. The preprocessing steps include grayscale conversion, filtering and noise reduction, and contrast enhancement. Grayscale conversion involves converting the color image to a grayscale image to reduce color interference and unify the calculation dimensions; commonly used methods include weighted average. Common filtering and noise reduction methods include Gaussian filtering and median filtering. Contrast enhancement strengthens the grayscale difference between the edges of the target face paper and the background. If wrinkles exist in the target face paper, it can also enhance the accuracy of wrinkle recognition. Common methods for enhancing contrast include histogram equalization or setting an adaptive threshold. Next, an edge detection algorithm is used to detect all face paper edges in the preprocessed second face paper image. Face paper edges refer to edge points. A commonly used edge detection algorithm is Canny edge detection, whose main steps are calculating gradient magnitude and direction and using dual thresholds to filter edges. Based on the face paper edge filtering results, the face paper region is segmented from the background of the second face paper image. This step is to reduce the impact of complex background regions on subsequent wrinkle recognition. Because the tissue paper area may contain wrinkles, to ensure that the segmentation of the tissue paper area is not affected by these wrinkles, a contour extraction method can be used. The contour with the largest area and shape closest to a rectangle is selected as the tissue paper area contour, and the tissue paper area is segmented based on this contour. Since tissue paper wrinkles are mainly manifested as local grayscale anomalies and the presence of fine edges, during the edge detection stage, if edge points exist within the tissue paper area, and the edge length obtained after fitting these edge points is greater than a preset length threshold, then the corresponding position in the tissue paper area is marked as an edge anomaly. The front tissue paper area refers to the tissue paper area segmented from the front tissue paper image, and the back tissue paper area refers to the tissue paper area segmented from the back tissue paper image.

[0164] During the wrinkle detection phase, if the target paper is not marked as an edge anomaly, the wrinkle detection is considered successful. If the target paper is marked as an edge anomaly, a small sliding window (e.g., 5×5 pixels) is set and slides pixel by pixel across the back area of ​​the paper. The variance of the grayscale values ​​of the pixels within each window is calculated. The larger the variance, the more severe the grayscale fluctuation within the window. The back area of ​​the paper with a variance greater than a preset variance threshold is marked as a grayscale anomaly area. The area marked as an edge anomaly is then used as the edge anomaly area. If the overlap between the edge anomaly area and the grayscale anomaly area is greater than a preset overlap threshold, the wrinkle detection of the target paper is considered unsuccessful. The overlap can be calculated based on the ratio of the overlapping pixel area to the total pixel area of ​​the two anomaly areas.

[0165] The overlapping area between edge anomaly regions and grayscale anomaly regions is defined as the wrinkle region. The number of pixels in the wrinkle region is counted. Using camera calibration, such as a checkerboard calibration method, a conversion coefficient between pixels and actual size is obtained (e.g., 1 pixel = 0.01 mm²). The product of the number of pixels in the wrinkle region and the conversion coefficient is calculated to obtain the wrinkle area. Simultaneously, the area of ​​the face paper is calculated based on the face paper size. The wrinkle area is divided by the face paper area to obtain the wrinkle percentage. Next, connected component analysis is performed on the wrinkle regions. An 8-neighborhood method can be used, where two adjacent pixels that are both in wrinkle regions are considered connected, and adjacent wrinkle regions are merged into one connected component. If the shortest distance between two connected components is less than a preset distance threshold, such as 5 mm, these two connected components are merged into one connected component. After completing the above steps, the number of connected components is counted to obtain the number of wrinkles. Based on the wrinkle percentage, the wrinkles of the target face paper are divided into local wrinkles and global wrinkles. For example, wrinkles with a wrinkle percentage greater than 20% are classified as global wrinkles, and wrinkles with a wrinkle percentage less than or equal to 20% are classified as local wrinkles. Next, wrinkles are categorized into few, medium, and many wrinkles based on their number. For example, 1-2 wrinkles are classified as few, 3-4 as medium, and 5 or more as many. For localized wrinkles, the suction force of the suction cups near the wrinkle is reduced accordingly, for example, 0.1 kPa for a few wrinkles, 0.2 kPa for a medium wrinkle, and 0.3 kPa for a large wrinkle. For global wrinkles, the suction force of all suction cups is reduced based on the number of wrinkles. After one reduction in suction force, images of the target paper are acquired for wrinkle detection. If two consecutive detections show no change in wrinkle area or number, or if the number of detections reaches the preset maximum threshold (e.g., three), detection stops, and the subsequent step of detecting adsorption deviation is performed. This is because two consecutive test results remain unchanged, indicating that the current adsorption force adjustment scheme has no practical effect on improving wrinkles. This could be due to physical defects in the paper itself, and the processing time for a single target paper is too long, potentially affecting the overall capacity of the target production line. Therefore, a maximum number of tests is set to avoid unnecessary time wastage. If the adjusted test results show that the wrinkles of the target paper have disappeared, the adsorption force of the target robotic arm at this time, as well as the raw material parameters of the target paper and the environmental parameters at this time, are recorded. The adsorption force parameter table is then updated based on these parameters and the adsorption force.

[0166] If the wrinkle detection passes, further inspection is needed to determine if the suction cup of the target robotic arm is centered on the target paper. If there is a suction deviation, such as if the suction position is too far to the left, right, top, or bottom, refer to... Figure 3All of these factors can cause varying degrees of sagging at the edges of the target paper. If the paper is directly laminated at this point, wrinkles may appear on the resulting laminated grey board, affecting the quality of the final product. Next, the edge lamination parameters of the target paper are determined based on the direction of the adsorption deviation. For example, if the adsorption position is too high, it indicates a greater sagging at the lower edge of the target paper. Therefore, the angle of the target robotic arm on the horizontal plane needs to be adjusted during lamination. This involves slightly raising the side of the target robotic arm adsorbed at the lower edge of the target paper. The degree of raising is determined by the adsorption deviation; a larger deviation requires a larger rise, and vice versa. This localized raising counteracts the sagging. When the target robotic arm is about to perform the lamination step, the end with the smaller sagging edge contacts the target grey board first. After contact, the target robotic arm slowly rotates and flattens around the lamination edge (rotation speed ≤ 1° / s), allowing the edge with the larger sagging edge to gradually contact the target grey board, completing the lamination and avoiding wrinkles caused by "gravity difference." Another scenario exists where the adsorption position is biased towards the upper left, lower left, upper right, or lower right. In this case, the corner opposite to the biased position needs to be adhered to first. For example, if the adsorption position is biased towards the upper left, it means the lower right corner of the target paper sags more. Therefore, by locally raising the lower right position of the target robotic arm, making its lower right end slightly higher than its upper left end, the adsorption process involves first adhering to the upper left corner of the target paper, then sequentially adhering to the lower right, lower left, and finally the lower right corner. The edge adsorption parameters of the target paper, including the edge adsorption order and the local lifting amplitude, are determined based on the adsorption bias. If the target robotic arm has no adsorption bias, the edge sag curvature of all long and wide sides of the target paper needs to be calculated using the least squares method. A higher edge sag curvature indicates a greater degree of edge sag of the target paper. If the droop curvature of any long side of the target paper is greater than the preset first curvature threshold, or the droop curvature of any long side is greater than the preset second curvature threshold, and the second curvature threshold is less than the first curvature threshold, it indicates that the suction force of the target robotic arm suction cup is insufficient. In this case, the suction force of the target robotic arm suction cup is appropriately increased, and the front image of the target paper is re-acquired. The droop curvature is recalculated. If the droop curvature of all long sides is less than or equal to the first curvature threshold, and the droop curvature of all short sides is less than or equal to the first curvature threshold, the current suction force is used as the edge bonding parameter for determining the target paper based on the droop curvature, and the suction force parameter table is updated based on the current suction force. If the suction force of the target robotic arm's suction cup is increased twice consecutively, and the drooping curvature of its edge remains unchanged, or if there is still a long or short side that is greater than the first curvature threshold or the second curvature threshold, then for the long or short side with abnormal curvature, the local lifting amplitude of the target robotic arm and the paper edge bonding sequence are adjusted according to the position of the edge sampling point corresponding to the largest edge vertical distance in the edge of the paper.For example, if the edge with abnormal curvature is the long left side of the target face paper, then the position of the edge sampling point corresponding to the maximum vertical distance of the edge, determined when calculating the edge sag curvature of this long side, is obtained within the edge of the face paper. If this edge sampling point is located lower on the long side, then the lower end of the target robotic arm needs to be locally lifted, because the lower end of the target face paper will sag and bend at this time. Furthermore, during lamination, the upper end of the target face paper needs to be laminated first, and then lamination should proceed downwards from the upper end, and vice versa. This method prevents wrinkles from appearing on the laminated gray board after lamination due to edge sag and bending during the lamination process. The device parameters of the image acquisition device are obtained. Combined with these parameters and the grayscale image, the paper-attaching pose of the target robotic arm is located. The calculation process for the paper-attaching pose is the same as that for the paper-adsorption pose. It primarily involves preprocessing the grayscale image, using an edge detection algorithm to detect edges, obtaining the edge points of the grayscale image, and then using the Hough inter-pixel detection algorithm to locate the pixel coordinates of all intersection points in the grayscale image. After a series of transformations on these pixel coordinates, the paper-attaching position and pose suitable for the target robotic arm are obtained. Integrating these two parameters yields the paper-attaching pose of the target robotic arm. Finally, the edge-attaching parameters and the paper-attaching pose are set as the robotic arm's attachment parameters. Through these steps, the target paper can be accurately attached to the target grayscale image while ensuring no wrinkles occur during the attachment process, thus guaranteeing the aesthetics of the packaging boxes produced in this factory.

[0167] In one embodiment, extracting the center point coordinates of all suction cup areas within the area on the back of the tissue paper, and determining whether the target robotic arm has adsorption deviation based on the center point coordinates includes the following steps:

[0168] Fit the minimum bounding rectangle of the back area of ​​the paper and extract the coordinates of all vertices of the minimum bounding rectangle;

[0169] Statistically analyze the pixel values ​​of the back area of ​​the face paper, and then use the Hough circle detection algorithm to detect all suction cup areas in the back area of ​​the face paper based on the pixel values.

[0170] Extract the center coordinates of all suction cup areas, and determine the center point coordinates of all suction cup areas based on the center coordinates of all suction cup areas;

[0171] By combining the coordinates of the center point and all vertices, the vertical distance between the center point of all suction cup areas and the edge of the back of the face paper is calculated, and the vertical distance is used to determine whether the target robotic arm has adsorption deviation.

[0172] In this embodiment, because the edges of the back area of ​​the tissue paper droop, the back area may not be a standard rectangle. Therefore, it is necessary to fit the minimum bounding rectangle of the back area of ​​the tissue paper. After edge detection, the minAreaRect function can be used to fit the minimum bounding rectangle of the back area of ​​the tissue paper. The minAreaRect function can output the center point coordinates, width, height, and rotation angle of the minimum bounding rectangle. Then, the boxPoints function can be used to convert the rectangle parameters into the pixel coordinates of the four vertices of the minimum bounding rectangle, i.e., vertex coordinates. Next, the pixel values ​​of the back area of ​​the tissue paper are calculated. Pixel values ​​refer to the grayscale values ​​of the back area of ​​the tissue paper. Suspected suction cup areas are then selected based on these grayscale values. First, the average pixel value of the back area of ​​the tissue paper is calculated. Then, a dynamic threshold is set based on the suction cup color. For example, for black or other dark-colored suction cups, the dynamic threshold can be a-0.5b, where 'a' is the average pixel value and 'b' is a preset fixed parameter, such as 20. Areas with grayscale values ​​less than a-0.5b are marked as suspected suction cup areas. For transparent suction cups, the dynamic threshold can be set to a+0.5b, marking areas with grayscale values ​​greater than a+0.5b as suspected suction cup areas. Then, the Canny edge detection algorithm is used to detect edges in the suspected suction cup areas. Next, a Hough transform is performed on the detected edge points. Circle detection parameters are set based on the actual size of the suction cup, including the suction cup radius and the center-to-center distance between suction cups, thus selecting suction cup areas that match the actual size of the suction cup. Both the Hough circle detection algorithm and the Hough line detection algorithm belong to the Hough algorithm family. Their Hough transform steps are similar, but the difference lies in that the parameter space of the Hough circle detection algorithm is three-dimensional, and the voting step proceeds from center to radius. Therefore, the center coordinates can be directly obtained after the Hough transform process. Since suction cups are basically circular, the Hough circle detection algorithm can be used to detect the suction cup region. Then, based on the coordinates of all the circle centers, the coordinates of the center points of all suction cup regions are calculated, i.e., the center point coordinates, referring to... Figure 3The center point of the suction cup area refers to the center point of the region formed by all suction cup areas. For example, if there is only one suction cup area, the coordinates of the center circle are the coordinates of the center point. If there are three suction cup areas, and the three suction cup areas are evenly arranged in a row, the coordinates of the center circle of the middle suction cup area are used as the coordinates of the center point. Alternatively, if there are many suction cup areas arranged in multiple rows, the smallest bounding rectangle of all suction cup areas can be constructed based on their center coordinates and the radius of each suction cup area. The coordinates of the center of this smallest bounding rectangle are used as the coordinates of the center point of all suction cup areas. Since the center coordinates and the coordinates of the four vertices are based on the pixel coordinate system of the back of the face paper, no coordinate transformation is required. After calculating the coordinates of the center point of all suction cup areas, the vertical distance between the center point and the edge of the face paper on the back of the face paper is calculated using the distance formula from a point to a line. The edge of the face paper refers to the four sides of the back of the face paper, including two short sides and two long sides. Calculate the difference in perpendicular distances from the center point to the two short sides of the back area of ​​the tissue paper to obtain the short side distance difference. Similarly, calculate the difference in perpendicular distances from the center point to the two long sides of the back area of ​​the tissue paper to obtain the long side distance difference. If the short side distance difference is greater than a preset first distance threshold, it indicates that the suction cup of the target robotic arm is positioned too high or too low during adsorption. If the long side distance difference is greater than a preset second distance threshold (the first distance threshold is greater than the second distance threshold), it indicates that the suction cup of the target robotic arm is positioned too left or too right during adsorption. If either of these situations occurs, it indicates that the target robotic arm has an adsorption deviation. The possible cause of adsorption deviation is that there are certain errors in the intrinsic parameters, depth information, and device pose of the image acquisition device, which leads to deviations in the calculated adsorption position of the face paper. This results in different drooping amplitudes on the four sides of the target face paper. The above parameter errors are difficult to avoid, and the influence of gravity on the target face paper is also unavoidable. Therefore, it is necessary to perform adsorption deviation verification through the above steps, and fine-tune the parameters of the target robotic arm before the bonding step based on the adsorption deviation verification results. That is, by adjusting the edge bonding parameters of the target face paper, the problem of uneven edge drooping amplitude caused by gravity and parameter errors can be overcome, avoiding wrinkles or curling edges of the target face paper during bonding, thereby improving the quality pass rate of the factory products.

[0173] In one embodiment, calculating the edge sag curvature of the target paper using the least squares method includes the following steps:

[0174] For any edge of the face paper, multiple edge sampling points are uniformly extracted from the edge of the face paper according to a preset interval parameter;

[0175] All edge sampling points are fitted to edge lines using the least squares method;

[0176] Calculate the perpendicular distance between all edge sampling points and the edge line;

[0177] The edge sag curvature of the target face paper is calculated based on the vertical distance of all edges of the face paper.

[0178] In this embodiment, for any edge of the paper, multiple edge sampling points are uniformly extracted at preset intervals (e.g., 10 pixels) to ensure that the edge sampling points represent the overall edge shape of the target paper. The least squares method is used to fit all edge sampling points to an edge line. The core idea is to find an edge line that minimizes the sum of the squared distances from all edge sampling points to this edge line. Specifically, the average of the x-coordinates and y-coordinates of all edge sampling points is calculated first, and the center position of all edge sampling points is found. The optimal fitted edge line must pass through this center position. Next, the difference between the x-coordinate and the average x-coordinate of each edge sampling point, and the difference between the y-coordinate and the average y-coordinate are calculated. A matrix is ​​then constructed using all the x-coordinate differences and y-coordinate differences. The eigenvalues ​​and eigenvectors of this matrix are calculated, and the eigenvector with the smallest eigenvalue is the direction corresponding to the edge line. Knowing the direction of the edge line and the guaranteed center position, the optimal edge line can be determined. Finally, the perpendicular distances from all edge sampling points to the edge line are calculated. The maximum vertical distance of the edge / the length of the long side or the length of the wide side (the length or width of the target paper) can be directly used as the edge sag curvature of all long sides and all wide sides of the target paper, respectively.

[0179] This application also provides a frame forming control system based on industrial vision, including:

[0180] The memory is configured to store instructions; and

[0181] The processor is configured to retrieve instructions from memory and, when executing instructions, to implement the aforementioned industrial vision-based frame forming control method.

[0182] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0183] The memory can be an internal storage unit of a computer device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) provided on the computer device. Furthermore, the memory can be a combination of internal storage units and external storage devices of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0184] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the above-described industrial vision-based frame forming control method.

[0185] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0186] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0187] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0188] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0189] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0190] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0191] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0192] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0193] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A frame forming control method based on industrial vision, characterized in that, The method includes the following steps: S101. When the target gray board is detected to be conveyed to the first preset position in the target production line, the timing device preset on the target production line is activated, and the gray board conveying time consumed by the target gray board to pass through the first preset position is counted by the timing device. S102. Collect the production parameters of the target production line, and combine the production line parameters and gray board conveying time to complete the anomaly verification of the target gray board; S103. If the target gray board has an abnormality, an alarm message will be output. S104. If there is no gray board abnormality in the target gray board, the first image of the target paper is acquired using the image acquisition device preset in the target production line. S105. Determine the paper adsorption posture of the target robotic arm, which is preset above the target assembly line, based on the first paper image and using image recognition technology. S106. Calculate the initial adsorption force and moving speed of the target robotic arm based on the production parameters; S107. Set the paper adsorption pose, initial adsorption force and moving speed as the robotic arm adsorption parameters of the target robotic arm. S108. When it is predicted that the target gray board is transported to the second preset position in the target production line, control the target robotic arm to adsorb and move the target paper to directly above the target gray board according to the robotic arm adsorption parameters. S109. Acquire images of the second side of the target paper and the gray board, respectively; S110. Combine the second paper image and the gray board image to perform attitude estimation of the target paper and the target gray board respectively, and set the robot arm fitting parameters for the target robot arm based on the attitude estimation results. S111. Control the target robotic arm to complete the bonding process between the target gray board and the target face paper according to the robotic arm bonding parameters, and obtain the laminated gray board; S112. Using a pre-set frame forming device on the target production line, bend and laminate the gray board to obtain a semi-finished frame. S110 includes the following steps: Preprocess the second face paper image, which includes a front face paper image and a back face paper image; The edge detection algorithm was used to detect all the edges of the second paper image after preprocessing. The preprocessed second face paper image is segmented based on all face paper edges to obtain the face paper region, which includes the face paper front region and the face paper back region. Using image recognition technology to detect wrinkles on the front side of tissue paper; If the wrinkle detection fails, the wrinkle type of the target paper is determined based on the wrinkle recognition result, and the initial adsorption force is adjusted according to the wrinkle type until the wrinkle detection passes. If the wrinkle detection passes, the center point coordinates of all suction cup areas in the area on the back of the face paper are extracted, and the target robotic arm is judged to have adsorption deviation based on the center point coordinates. If the target robotic arm has an adsorption deviation, the edge bonding parameters of the target paper are determined based on the adsorption deviation. If the target robotic arm has no adsorption deviation, the edge sag curvature of the target paper is calculated using the least squares method, and the edge bonding parameters of the target paper are determined based on the edge sag curvature. Obtain the device parameters of the image acquisition device, and combine the device parameters with the gray board image to locate the paper-attaching posture of the target robotic arm; Set the edge bonding parameters and the face paper bonding pose to the robot arm bonding parameters of the target robot arm.

2. The method according to claim 1, characterized in that, The production parameters include production line parameters, raw material parameters, and environmental parameters. Production line parameters include production line operation parameters and production line structure parameters. Raw material parameters include gray board size, gray board material, face paper size, face paper material, face paper quality, and face paper thickness. Environmental parameters include production line temperature and production line humidity.

3. The method according to claim 2, characterized in that, The process of combining production line parameters and gray board conveying time to complete the anomaly verification of the target gray board includes the following steps: The theoretical conveying time of the target gray board is calculated by combining the production line operating parameters and gray board size; Calculate the absolute value of the time difference between the theoretical conveying time and the gray board conveying time; If the absolute value of the time difference is greater than the preset time difference threshold, it is determined that the target gray board has a gray board anomaly. If the absolute value of the time difference is less than or equal to the preset time difference threshold, it is determined that there is no grayboard anomaly in the target grayboard.

4. The method according to claim 1, characterized in that, The step of determining the tissue adsorption pose of the target robotic arm, which is pre-set above the target production line, based on the first tissue image and using image recognition technology includes the following steps: Preprocess the first sheet image; An edge detection algorithm is used to perform edge detection on the preprocessed first sheet image, and the edge points of the preprocessed first sheet image are obtained based on the edge detection results. The edge points are mapped to the parameter space using the Hough transform. Based on the edge point mapping results, all rectangular edges of the preprocessed first sheet are detected, and the pixel coordinates of all intersection points between all rectangular edges are calculated. Complete the coordinate transformation of all intersection pixel coordinates, and determine the paper adsorption pose of the target robotic arm above the target production line based on the coordinate transformation results.

5. The method according to claim 4, characterized in that, The process of completing the coordinate transformation of all intersection pixel coordinates and determining the paper adsorption pose of the target robotic arm above the target assembly line based on the coordinate transformation results includes the following steps: Acquire the device parameters of the image acquisition device, including device intrinsic parameters, depth information, and device pose; For any intersection point pixel coordinates, the intersection point pixel coordinates are converted into paper imaging coordinates based on the intrinsic parameter information; Based on the pinhole imaging model and combined with intrinsic parameter information and depth information, the three-dimensional transformation of the paper imaging coordinates is completed to obtain the coordinates of the acquisition device. The rotation matrix and translation vector between the image acquisition device and the target robotic arm preset above the target pipeline are calculated by combining the device pose and the robotic arm pose in the pipeline structure parameters. By combining the rotation matrix and translation vector to correct the coordinates of the acquisition device, the relative coordinates of the paper are obtained; The edge center coordinates and face center coordinates of the target paper are calculated based on the relative coordinates of all the paper sheets. The paper adsorption position of the target robotic arm is then planned by combining the edge center coordinates and face center coordinates. Calculate the direction vector of the line connecting the relative coordinates of adjacent sheets, and fit the normal vector of the target sheet based on the direction vector of the line. The paper adsorption posture of the target robotic arm is planned based on the paper normal vector; The tissue adsorption position and tissue adsorption posture are integrated into the tissue adsorption posture of the target robotic arm.

6. The method according to claim 2, characterized in that, The calculation of the initial adsorption force and moving speed of the target robotic arm based on production parameters includes the following steps: The initial adsorption force of the target robotic arm is determined by combining environmental and raw material parameters and based on a pre-constructed adsorption force parameter table. The vertical movement speed of the target robotic arm is determined based on the robotic arm pose in the pipeline operation parameters and pipeline structure parameters. The horizontal movement speed of the target robotic arm is determined based on the raw material station spacing and vertical movement speed in the production line operating parameters and structural parameters of the production line. The vertical and horizontal movement speeds are integrated into the target robotic arm's movement speed.

7. The method according to claim 1, characterized in that, The steps of extracting the center point coordinates of all suction cup areas within the area on the back of the paper and determining whether the target robotic arm has adsorption deviation based on the center point coordinates include the following: Fit the minimum bounding rectangle of the back area of ​​the paper and extract the coordinates of all vertices of the minimum bounding rectangle; Statistically analyze the pixel values ​​of the back area of ​​the face paper, and then use the Hough circle detection algorithm to detect all suction cup areas in the back area of ​​the face paper based on the pixel values. Extract the center coordinates of all suction cup areas, and determine the center point coordinates of all suction cup areas based on the center coordinates of all suction cup areas; By combining the coordinates of the center point and all vertices, the vertical distance between the center point of all suction cup areas and the edge of the back of the face paper is calculated, and the vertical distance is used to determine whether the target robotic arm has adsorption deviation.

8. The method according to claim 1, characterized in that, The calculation of the edge sag curvature of the target paper using the least squares method includes the following steps: For any edge of the face paper, multiple edge sampling points are uniformly extracted from the edge of the face paper according to a preset interval parameter; All edge sampling points are fitted to edge lines using the least squares method; Calculate the perpendicular distance between all edge sampling points and the edge line; The edge sag curvature of the target face paper is calculated based on the vertical distance of all edges of the face paper.

9. A frame forming control system based on industrial vision, characterized in that, include: The memory is configured to store instructions; as well as The processor is configured to retrieve instructions from memory and, when executing the instructions, to implement the industrial vision-based frame forming control method according to any one of claims 1 to 8.

Images