Enclosure frame forming control method and system based on industrial vision

Abstract

The invention discloses an enclosure frame forming control method and system based on industrial vision, and relates to the field of industrial vision, and the method comprises the steps: carrying out the statistics of gray board conveying time of a target gray board; the abnormity verification of the target grey board is completed; if the target grey board is abnormal, outputting alarm information; if the target grey board does not have grey board abnormity, collecting a first surface paper image; determining a facial tissue adsorption pose of the target mechanical arm; the initial adsorption force and the moving speed of the target mechanical arm are calculated; mechanical arm adsorption parameters of the target mechanical arm are set; the target mechanical arm is controlled to adsorb and move the target facial tissue to the position over the target grey board; collecting a second surface paper image and a grey board image; attitude estimation of the target surface paper and the target grey board is completed, and fitting parameters of a mechanical arm are set; completing a laminating process to obtain a laminated grey board; bending and laminating the grey board to obtain an enclosure frame semi-finished product. The quality of a finished product manufactured through enclosure frame forming can be effectively improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of industrial vision, and in particular to a frame forming control method and system based on industrial vision. BACKGROUND

[0002] In the field of packaging manufacturing, frame forming, as a key process, is widely used in the production process of outer packaging of food, beverage, wine, tea, cigarette, medicine and other products, which can directly affect the appearance and practicability of the product. With the continuous improvement of people's living standards, the requirements for product packaging are also getting higher and higher, so how to improve the quality of frame forming has become the core demand of the industry. One of the steps that directly affect the quality of frame forming is the positioning and bonding of the face paper and the gray board.

[0003] The existing method for improving the positioning and bonding precision between the face paper and the gray board is to control the positioning and bonding of the face paper and the gray board by a pre-set control system. This method lacks real-time feedback mechanism and cannot adjust the bonding parameters according to the deformation parameters of the face paper and the gray board, temperature fluctuations, humidity changes and other dynamic factors in the production environment. Especially for face paper and gray board produced from different raw materials, the deformation generated and the influence degree of temperature, humidity and other parameters on them all have certain differences. Relying on the pre-set control system to complete the positioning and bonding of the face paper and the gray board may cause deviation in the positioning and bonding position or bubbles after bonding, affecting the quality of frame forming, thereby affecting the appearance and practicability of the final product, causing serious loss to the packaging factory. SUMMARY

[0004] Embodiments of the present application provide a frame forming control method and system based on industrial vision for improving the quality of products manufactured by frame forming.

[0005] To achieve the above-mentioned purpose, embodiments of the present application adopt the following technical solutions: In a first aspect, a frame forming control method based on industrial vision is provided, which comprises: When it is detected that the target gray board is conveyed to the first preset position in the target assembly line, a timing device preset on the target assembly line is started, and the gray board conveying time consumed by the target gray board passing through the first preset position is counted by the timing device; The production parameters of the target assembly line are collected, and the abnormality of the target gray board is checked in combination with the assembly line parameters and the gray board conveying time; If the target gray board has gray board abnormality, an alarm information is outputted; If the target gray board has no gray board abnormality, a first face paper image of the target face paper is collected by an image collection device preset on the target assembly line; determine a face paper suction pose of a target mechanical arm preset above a target assembly line according to the first face paper image and by using image recognition technology; calculate an initial suction force and a moving speed of the target mechanical arm according to the production parameters; set the face paper suction pose, the initial suction force and the moving speed as mechanical arm suction parameters of the target mechanical arm; when it is predicted that the target face paper is conveyed to a second preset position in the target assembly line, control the target mechanical arm to suck and move the target face paper to directly above the target face paper according to the mechanical arm suction parameters; collect a second face paper image and a face paper image of the target face paper and the target face paper respectively; complete pose estimation of the target face paper and the target face paper respectively by combining the second face paper image and the face paper image, and set mechanical arm lamination parameters for the target mechanical arm according to the pose estimation results; control the target mechanical arm to complete a lamination process of the target face paper and the target face paper according to the mechanical arm lamination parameters, and obtain a laminated face paper; fold the laminated face paper by using a frame forming device preset in the target assembly line to obtain a frame semi-finished product.

[0006] Optionally, the production parameters include assembly line parameters, raw material parameters and environmental parameters, the assembly line parameters include assembly line running parameters and assembly line structure parameters, the raw material parameters include face paper size, face paper material, face paper size, face paper material, face paper quality and face paper thickness, and the environmental parameters include assembly line temperature and assembly line humidity.

[0007] Optionally, the abnormality checking of the target face paper by combining the assembly line parameters and the face paper conveying time includes the following steps: calculate a theoretical conveying time of the target face paper by combining the assembly line running parameters and the face paper size; calculate an absolute value of a time difference between the theoretical conveying time and the face paper conveying time; if the absolute value of the time difference is greater than a preset time difference threshold, it is determined that the target face paper has a face paper abnormality; if the absolute value of the time difference is less than or equal to the preset time difference threshold, it is determined that the target face paper does not have a face paper abnormality.

[0008] Optionally, the face paper suction pose of the target mechanical arm preset above the target assembly line is determined according to the first face paper image and by using image recognition technology, including the following steps: preprocess the first face paper image; perform edge detection on the preprocessed first face paper image by using an edge detection algorithm, and obtain edge points of the preprocessed first face paper image according to the edge detection result; The edge points are mapped to a parameter space by using a Hough transform, all rectangular edges of the preprocessed first paper sheet are detected according to the mapping result of the edge points, and all intersection pixel coordinates between the all rectangular edges are calculated; The coordinate conversion of the all intersection pixel coordinates is completed, and a paper sheet suction pose of a target mechanical arm pre-set above a target flow line is determined according to a coordinate conversion result.

[0009] Optionally, the coordinate conversion of the all intersection pixel coordinates is completed, and a paper sheet suction pose of a target mechanical arm pre-set above a target flow line is determined according to a coordinate conversion result, including the following steps: Obtaining device parameters of the image acquisition device, the device parameters including device intrinsic parameters, depth information and device pose; For any intersection pixel coordinate, the intersection pixel coordinate is converted into a paper sheet imaging coordinate according to the intrinsic information; Based on a pinhole imaging model and combined with the intrinsic information and the depth information, three-dimensional conversion of the paper sheet imaging coordinate is completed to obtain a collection device coordinate; Combined with the device pose and the mechanical arm pose in the flow line structure parameters, a rotation matrix and a translation vector between the image acquisition device and the target mechanical arm pre-set above the target flow line are calculated; Combined with the rotation matrix and the translation vector, the collection device coordinate is corrected to obtain a paper sheet relative coordinate; According to all the paper sheet relative coordinates, edge center coordinates and face center coordinates of the target paper sheet are calculated, and combined with the edge center coordinates and the face center coordinates, a paper sheet suction position of the target mechanical arm is planned; A line direction vector between adjacent paper sheet relative coordinates is calculated, and a paper sheet normal vector of the target paper sheet is fitted according to the line direction vector; According to the paper sheet normal vector, a paper sheet suction pose of the target mechanical arm is planned; The paper sheet suction position and the paper sheet suction pose are integrated into a paper sheet suction pose of the target mechanical arm.

[0010] Optionally, the initial suction force and the moving speed of the target mechanical arm are calculated according to the production parameters, including the following steps: Combined with the environmental parameters and the raw material parameters and based on a pre-constructed suction force parameter table, the initial suction force of the target mechanical arm is determined; According to the flow line running parameters and the mechanical arm pose in the flow line structure parameters, a vertical moving speed of the target mechanical arm is determined; According to the flow line running parameters, the raw material station distance in the flow line structure parameters and the vertical moving speed, a horizontal moving speed of the target mechanical arm is determined; The vertical moving speed and the horizontal moving speed are integrated into a moving speed of the target mechanical arm.

[0011] Optionally, the posture estimation of the target face paper and the target gray board is respectively completed by combining the second face paper image and the gray board image, and the mechanical arm fitting parameters of the target mechanical arm are set according to the posture estimation results, and the method comprises the following steps: preprocessing the second face paper image, the second face paper image comprising a face paper front image and a face paper back image; detecting all face paper edges of the second face paper image after preprocessing by using an edge detection algorithm; segmenting the face paper region of the second face paper image after preprocessing according to all face paper edges, the face paper region comprising a face paper front region and a face paper back region; detecting wrinkles in the face paper front region by using image recognition technology; if the wrinkle detection fails, determining the wrinkle type of the target face paper according to the wrinkle recognition result, and adjusting the initial suction force according to the wrinkle type until the wrinkle detection passes; if the wrinkle detection passes, extracting the center point coordinates of all suction cup regions in the face paper back region, and judging whether the target mechanical arm has suction deviation based on the center point coordinates; if the target mechanical arm has suction deviation, determining the edge fitting parameters of the target face paper according to the suction deviation; if the target mechanical arm has no suction deviation, calculating the edge sag curvature of the target face paper by using the least square method, and determining the edge fitting parameters of the target face paper according to the edge sag curvature; obtaining device parameters of the image acquisition device, positioning the face paper fitting pose of the target mechanical arm in combination with the device parameters and the gray board image; setting the edge fitting parameters and the face paper fitting pose as the mechanical arm fitting parameters of the target mechanical arm.

[0012] Optionally, the center point coordinates of all suction cup regions in the face paper back region are extracted, and whether the target mechanical arm has suction deviation is judged based on the center point coordinates, and the method comprises the following steps: fitting the minimum circumscribed rectangle of the face paper back region, and extracting all vertex coordinates of the minimum circumscribed rectangle; counting the pixel value of the face paper back region, and detecting all suction cup regions in the face paper back region according to the pixel value by using the Hough circle detection algorithm; extracting the center coordinates of all suction cup regions, and determining the center point coordinates of all suction cup regions according to all center coordinates; calculating the vertical distance between the center points of all suction cup regions and the face paper edge of the face paper back region in combination with the center point coordinates and all vertex coordinates, and judging whether the target mechanical arm has suction deviation according to the vertical distance.

[0013] Optionally, the edge sag curvature of the target face paper is calculated by using the least square method, and the method comprises the following steps: For any face paper edge, a plurality of edge sampling points are uniformly extracted from the face paper edge according to a preset interval parameter; All edge sampling points are fitted into an edge straight line by using a least square method; Edge vertical distances between all edge sampling points and the edge straight line are calculated; A target face paper edge sag curvature is calculated based on all edge vertical distances of all face paper edges.

[0014] In a second aspect, the application provides an industrial vision-based frame forming control system, comprising: a memory configured to store instructions; and a processor configured to call the instructions from the memory and capable of realizing the industrial vision-based frame forming control method according to the first aspect when executing the instructions.

[0015] Through the above technical solution, by comparing the gray board conveying time of the target gray board passing through the first preset position counted by the timing device and the theoretical conveying time calculated according to the pipeline operation parameters and the gray board size, it is judged whether the target gray board has a gray board abnormality. If there is a gray board abnormality, an alarm information will be output to remind the staff of the target pipeline to remove the target gray board with a gray board abnormality. Only when the target gray board does not have a gray board abnormality, the subsequent process of bonding will continue, thereby reducing the production time and the waste of raw materials. In order to improve the bonding accuracy between the target face paper and the target gray paper, before bonding, it is also necessary to set various parameters of the target mechanical arm reasonably. For example, the present application accurately locates the intersection pixel coordinates of the target face paper through a series of image processing steps, and converts the intersection pixel coordinates into the face paper suction pose of the target mechanical arm, thereby reducing the deformation of the target face paper after being sucked by the mechanical arm due to gravity. At the same time, the initial suction force of the target mechanical arm is set in combination with the environmental parameters and the raw material parameters, which can effectively avoid wrinkles of the target face paper due to excessive suction force. In summary, the present application can effectively avoid wrinkles or bubbles when the target face paper and the target gray board are bonded, thereby improving the qualified rate of the final product, reducing the raw material loss of the factory, and reducing the production cost.

[0016] Other features and advantages of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of an industrial vision-based frame forming control method provided by the present application embodiment; Figure 2 A suction position diagram of a suction cup and a target face paper provided by the present application embodiment; Figure 3An example diagram of a suction cup appearing adsorption deviation is provided for the embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are merely used to explain and illustrate the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0019] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative positional relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional indications also change accordingly.

[0020] In addition, if the embodiments of the present application involve descriptions such as “first”, “second”, etc., the descriptions of “first”, “second”, etc. are only for description purposes, and cannot be understood as indicating or implying the relative importance of the technical features indicated or the number of the technical features indicated. Therefore, the features limited by “first”, “second” can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it, and when the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist, and is also not within the scope of protection claimed by the present application.

[0021] Figure 1 A flowchart of a surrounding frame forming control method based on industrial vision according to the embodiments of the present application is schematically shown. As shown in Figure 1 The embodiments of the present application provide a surrounding frame forming control method based on industrial vision, which can include the following steps: S101, when it is detected that the target gray board is conveyed to the first preset position in the target flow line, a timing device preset on the target flow line is started, and the gray board conveying time consumed by the target gray board passing through the first preset position is counted by the timing device.

[0022] In the embodiment, a photoelectric sensor is pre-installed at a first preset position in the target flow line, which is composed of a transmitter and a receiver, both of which are installed on both sides of the target flow line and are on the same horizontal line, and the light emitted by the transmitter can be accurately received by the receiver, forming a detection light beam for detecting whether the target gray board reaches or leaves the target flow line. Meanwhile, a timing device is installed on the target flow line, which can be accurate to the millisecond level, and the timing device has a signal receiving module, a timing module and a data storage module, and has completed circuit connection and signal matching debugging with the photoelectric sensor, ensuring that the electrical signal emitted by the photoelectric sensor can be received by the timing device and accurately identified. When the target gray board is continuously conveyed on the target flow line at a preset flow line speed, as the target gray board continuously approaches the first preset position, the front end of the target gray board will gradually enter the detection area between the transmitter and the receiver of the photoelectric sensor. When the front end of the target gray board completely blocks the detection light beam, the receiver will immediately generate an open electrical signal because it cannot receive the light emitted by the transmitter, and the signal will be transmitted to the signal receiving module of the timing device through the connection line. After the signal receiving module of the timing device receives the open electrical signal, it will immediately send a start instruction to the timing module, and the timing module will start and enter the timing state at the moment it receives the instruction, accurately recording the start time at that moment. When the rear end of the target gray board completely leaves the detection area and no longer blocks the detection light beam, the receiver re-receives the light emitted by the transmitter, and immediately generates a stop electrical signal and transmits it to the signal receiving module of the timing device. After the signal receiving module receives the stop electrical signal, it sends a stop instruction to the timing module, and the timing module immediately stops timing and records the stop time at that moment. Finally, the time difference between the start time and the stop time is calculated, and the gray board conveying time of the target gray board passing through the first preset position is obtained.

[0023] S102, collect the production parameters of the target flow line, and complete the abnormality check of the target gray board in combination with the flow line parameters and the gray board conveying time.

[0024] In the embodiment, the flow line running parameters include the flow line running speed and the flow line pause time, and the theoretical conveying time of the target gray board can be calculated by dividing the size of the gray board by the flow line speed. Then, the time difference between the theoretical conveying time and the gray board conveying time is subtracted, and the absolute value of the time difference is obtained. If the absolute value of the time difference is greater than the preset time difference threshold, it means that the size of the target gray board is too large or too small compared with the normal gray board, that is, the target gray board is abnormal. The reasons for the abnormality of the gray board may be that the edge of the gray board is damaged or the position of the gray board is deviated. Through the above method, the target gray board with serious abnormalities can be pre-removed, saving the production resources of the target flow line, and at the same time, the target gray board with damage or size abnormalities is not used for producing packaging boxes, indirectly improving the product qualification rate of the factory.

[0025] S103, if the target gray board exists gray board abnormality, output alarm information.

[0026] In this embodiment, when it is detected that the target gray board exists gray board abnormality, the visual alarm or the audible alarm of the target flow line will be triggered. The visual alarm is generally a warning light, which will be turned on when the gray board abnormality is detected, reminding the relevant operating personnel to handle the target gray board with gray board abnormality. The audible alarm can use a buzzer. At the same time, the frequency of the target gray board appearing gray board abnormality will be recorded. Once the abnormal frequency exceeds the preset frequency threshold, the previous cutting process needs to be checked for abnormality. In this way, the raw material utilization rate of the factory can be improved, and the production cost can be reduced.

[0027] S104, if the target gray board does not exist gray board abnormality, the first face paper image of the target face paper is collected by using the image collection device preset on the target flow line.

[0028] In this embodiment, when it is detected that the target gray board does not exist gray board abnormality, the subsequent steps can be performed. In order to ensure that the target mechanical arm can accurately adsorb the target face paper and move above the target face board, the first face paper image of the target face paper needs to be collected by using the image collection device preset above the target flow line. The image collection device is an industrial camera, such as a 2D area array camera or a 3D depth camera.

[0029] S105, according to the first face paper image and by using image recognition technology, the face paper adsorption pose of the target mechanical arm preset above the target flow line is determined.

[0030] In the embodiment, the first side paper image is preprocessed first, and the preprocessing steps include graying and filtering denoising. Then the edge detection algorithm is used to extract the edge points of the first side paper image, and the commonly used edge detection algorithm is Canny algorithm. Then the Hough straight line detection algorithm is used to detect the four rectangular edges of the target side paper in the first side paper image. According to the polar coordinates of the four rectangular edges and using the Cramer rule, the full intersection pixel coordinates between the rectangular edges are solved. The coordinates of the intersection pixel coordinates are converted into a paper suction pose that can be directly used to set the target mechanical arm. According to the device parameters of the image acquisition device, the intersection pixel coordinates are first converted into the paper imaging coordinates, which are the physical pose description of the edge intersection point (the point corresponding to the intersection pixel coordinates) of the target side paper on the imaging plane of the image acquisition device. Then the two-dimensional coordinates of the paper imaging coordinates are converted into three-dimensional coordinates, i.e. the acquisition device coordinates, which refer to the coordinates of the edge intersection point in the image acquisition device coordinate system constructed with the image acquisition device optical center as the origin. Based on the pinhole imaging model, the three-dimensional conversion of the paper imaging coordinates is completed to obtain the acquisition device coordinates. Then according to the relative position of the image acquisition device and the target mechanical arm, the acquisition device coordinates are translated and rotated to obtain the three-dimensional coordinates of the edge intersection point of the target side paper in the target mechanical arm coordinate system, i.e. the paper relative coordinates. Then in order to avoid the edge of the target side paper from sagging too much when the target mechanical arm sucks the target side paper, the paper suction position and the paper suction pose, i.e. the paper suction pose, need to be planned according to the paper relative coordinates. Through the above method, the paper suction pose can be accurately calculated, and the target side paper can be maximally avoided from being sucked by the target mechanical arm with a large edge sag or paper wrinkles, so as to avoid the target side paper from being wrinkled or bubbled when it is attached to the target gray board, thereby improving the quality of the products of the factory and ensuring the efficiency of the factory.

[0031] S106, calculate the initial suction force and moving speed of the target mechanical arm according to the production parameters.

[0032] In the embodiment, first, an experimental manipulator completely same as the target manipulator is used, for example, the material, the aperture, the layout of the suction cup are all completely same. Then, the suction force experiment is carried out by using the face paper used by all the packaging box goods produced by the factory, and the experimental results are arranged into a suction force parameter table. According to the environmental parameters and the raw material parameters, the corresponding minimum suction force and maximum suction force can be queried, and the average of the two is taken as the initial suction force. Then, the vertical distance between the target manipulator and the target assembly line is calculated according to the manipulator pose in the assembly line running parameters. The vertical distance is the vertical axis coordinate of the manipulator pose. The assembly line running parameters include the assembly line speed and the assembly line pause time. The assembly line pause time refers to when the target gray board is transported to the second preset position, it will be paused for a period of time, so that the target face paper can be accurately attached to the target bottom plate. In order to save the time of the attachment process, at the same time when the target gray board is transported to the second preset position, the target manipulator completes the suction step of the target face paper and moves the target face paper to the top of the target gray board. When the target gray board stops moving, a series of subsequent attachment steps are immediately executed. First, the minimum vertical moving speed of the target manipulator is calculated according to the vertical distance between the target manipulator and the target assembly line and the assembly line pause time, which is to ensure the smooth execution of the attachment step. Since the target manipulator needs to complete the structure adjustment and other operations in the attachment step, the minimum vertical moving speed is appropriately increased according to the production experience of the surrounding frame semi-finished product and taken as the vertical moving speed. The transportation time of the target gray board from the first preset position to the second preset position is calculated according to the assembly line speed. Since the target manipulator also needs to move vertically in the suction step, the vertical moving time is calculated according to the vertical moving speed and the vertical distance. After subtracting the transportation time from the vertical moving time, the minimum horizontal moving speed is calculated in combination with the raw material station spacing. Similarly, the minimum horizontal moving speed is appropriately increased according to the production experience of the surrounding frame semi-finished product and taken as the horizontal moving speed of the target manipulator. The vertical moving speed and the horizontal moving speed are integrated to obtain the moving speed of the target manipulator. This step can as much as possible seamlessly connect the running action of the target manipulator with the running beat of the target assembly line, greatly improving the production efficiency.

[0033] S107, setting the face paper suction pose, the initial suction force and the moving speed as the manipulator suction parameters of the target manipulator.

[0034] In the embodiment, the face paper suction pose, the initial suction force and the moving speed are fed back to the controller of the target manipulator, and the manipulator suction parameters of the target manipulator are set by the controller. In the subsequent step, the target manipulator completes the suction box movement of the target face paper according to the set manipulator control parameters.

[0035] S108, when it is predicted that the target gray board is transported to the second preset position in the target flow line, the target mechanical arm is controlled to adsorb and move the target face paper to the position directly above the target gray board according to the mechanical arm adsorption parameters.

[0036] In the embodiment, according to the distance between the first preset position and the second preset position and the flow line speed, the time point when the target gray board is transported to the second preset position in the target flow line can be calculated. Before the time point is reached, the target mechanical arm starts to perform the mechanical arm descending, adsorbing the target face paper, mechanical arm ascending, and mechanical arm horizontal moving. When the time point is reached, that is, the moment when the target gray board reaches the second preset position, the target mechanical arm adsorbs the target face paper just to reach the position directly above the target face paper to the target gray board, which is convenient for performing the subsequent face paper lamination step.

[0037] S109, the second face paper image and the gray board image of the target face paper and the target gray board are collected respectively.

[0038] In the embodiment, the same type of image collection device is used to collect the second face paper image and the gray board image. The second face paper image includes the face paper front image and the face paper back image. The image collection device arranged directly above the target flow line is used to collect the gray paper image and the face paper back image. The face paper front image is collected by the image collection device preset on the side of the target flow line.

[0039] S110, the pose estimation of the target face paper and the target gray board is completed respectively by combining the second face paper image and the gray board image, and the mechanical arm lamination parameters are set for the target mechanical arm according to the pose estimation result.

[0040] In the embodiment, the second face paper image is pre-processed, and the pre-processing steps include grayscale processing, filter denoising and contrast enhancement. Then, an edge detection algorithm is used to detect all the face paper edges of the second face paper image after pre-processing, and the face paper edge refers to an edge point. A commonly used edge detection algorithm is Canny edge detection. According to the face paper edge screening result, the face paper region is separated from the background of the second face paper image, which is to reduce the influence of the complex background region on the subsequent wrinkle recognition. Since the face paper wrinkle mainly shows local gray abnormality and small edge, if there is an edge point in the face paper region and the length of the edge obtained by fitting the edge point is greater than a preset length threshold, the corresponding position in the face paper region is marked as an edge abnormality. In the wrinkle detection stage, if the target face paper is not marked as an edge abnormality, it is determined that the target face paper wrinkle detection is passed. If the target face paper is marked as an edge abnormality, a small size sliding window (such as 5x5 pixels) is set, which is slid in the face paper back region pixel by pixel, and the variance of the pixel gray value in each window is calculated. The greater the variance, the more intense the gray fluctuation in the window. The face paper back region with a variance greater than a preset variance threshold is marked as a gray abnormality region. The region marked as an edge abnormality is regarded as an edge abnormality region, and if the overlap degree between the edge abnormality region and the gray abnormality region is greater than a preset overlap degree threshold, it is determined that the wrinkle detection of the target face paper fails.

[0041] The overlapping region between the edge abnormality region and the gray abnormality region is regarded as a wrinkle region. The number of pixels in the wrinkle region is counted, the conversion coefficient between the pixel and the actual size is obtained through camera calibration such as chessboard calibration, the product of the number of pixels in the wrinkle region and the conversion coefficient is calculated, and the wrinkle area of the wrinkle region is obtained. At the same time, the face paper area is calculated according to the face paper size, the wrinkle area is divided by the face paper area, and the wrinkle ratio is obtained. Then, the wrinkle region is subjected to connected component analysis, which can adopt 8-neighborhood method, that is, two pixels are adjacent and both are wrinkle region pixels, which are regarded as connected, and adjacent wrinkle regions are merged into one connected domain. According to the wrinkle ratio, the wrinkle of the target face paper is divided into local wrinkle and global wrinkle. Then, according to the number of wrinkles, the wrinkles are divided into a small amount of wrinkles, a medium amount of wrinkles and a large amount of wrinkles. For local wrinkles, the suction force of the suction cup near the wrinkle is appropriately reduced according to the number of wrinkles. If it is a global wrinkle, the suction force of all suction cups is reduced according to the number of wrinkles.

[0042] If the wrinkle detection passes, further detection is needed to determine whether the target mechanical arm is adsorbed at the center position of the target face paper. If there is adsorption deviation, it will cause the edges of the target face paper to sag to varying degrees. At this time, if the face paper is directly laminated, it may cause the finished laminated gray board to have wrinkles, affecting the quality of the final product. Then, according to the deviation direction of the adsorption deviation, the edge lamination parameters of the target face paper are determined, and the edge lamination parameters of the target face paper include the face paper edge lamination sequence and the local lifting amplitude. If the target mechanical arm does not have adsorption deviation, the least square method is used to calculate the edge sag curvature of all long edges and all wide edges of the target face paper. According to the edge sag curvature, the local lifting amplitude and the face paper edge lamination sequence of the target mechanical arm are adjusted to prevent the target face paper from being laminated to cause the finished laminated gray board to have wrinkles due to edge sagging and bending. The device parameters of the image acquisition device are obtained, and the face paper lamination pose of the target mechanical arm is located by combining the device parameters and the gray board image. The calculation process of the face paper lamination pose is the same as that of the face paper adsorption pose, mainly by preprocessing the gray board image, using an edge detection algorithm to detect the edges of the preprocessed gray board image to obtain the edge points of the gray board image, and then locating all intersection pixel coordinates of the gray board image according to the Hough transform algorithm. After a series of transformations on all intersection pixel coordinates of the gray board image, the face paper lamination position and the face paper lamination pose of the target mechanical arm are obtained, and the face paper lamination position and the face paper lamination pose are integrated to obtain the face paper lamination pose of the target mechanical arm. Finally, the edge lamination parameters and the face paper lamination pose are set as the mechanical arm lamination parameters of the target mechanical arm. Through the above steps, the target face paper can be accurately laminated with the target gray board, and wrinkles can be prevented during the lamination process to ensure the beauty of the packaging box produced by the factory.

[0043] S111, controlling the target mechanical arm to complete the lamination process of the target gray board and the target face paper according to the mechanical arm lamination parameters to obtain a laminated gray board.

[0044] In this embodiment, the target mechanical arm is controlled to move downward according to the mechanical arm lamination parameters to adjust the target mechanical arm to move downward to laminate, and to stay in the lamination area for a period of time, for example, 0.5s, to ensure that the target face paper is pressed with the target gray board. After completing the lamination, the target mechanical arm is controlled to move upward to the original position, i.e., the position before moving downward, and then move horizontally to the top of the next target face paper, and repeat the above process to continue the lamination steps of the next target face paper and the target gray board.

[0045] S112, bending the laminated gray board by using the frame forming device pre-set in the target assembly line to obtain a frame semi-finished product.

[0046] In the embodiment, the assembled gray board is transported to the frame forming station, fixed at a preset position by positioning clamps (such as pneumatic baffles) on both sides of the assembly line, and bent by a multi-axis mechanical arm of a frame forming device (such as a pneumatic folder) preset on the target assembly line in a preset order to form four sides of a coverless box, thereby obtaining a frame semi-finished product.

[0047] In one embodiment, the production parameters include assembly line parameters, raw material parameters, and environmental parameters. The assembly line parameters include assembly line operation parameters and assembly line structure parameters. The raw material parameters include gray board size, gray board material, face paper size, face paper material, face paper quality, and face paper thickness. The environmental parameters include assembly line temperature and assembly line humidity.

[0048] In the embodiment, the assembly line operation parameters include assembly line operation speed and assembly line pause time. The assembly line pause time refers to the time when the target gray board is temporarily stopped during transportation to the second preset position to ensure that the target face paper can be accurately attached to the target gray board. The assembly line structure parameters include raw material station spacing and mechanical arm pose. The raw material station spacing refers to the distance between the position of the target face paper in the adsorption step and the position of the target gray board (second preset position) when the attachment step is performed. The mechanical arm pose refers to the coordinates of the origin of the target mechanical arm coordinate system in the world coordinate system, from which the vertical distance between the target mechanical arm and the target assembly line can be calculated. The raw material parameters can be directly obtained from the database of the factory according to the type of the final product. The environmental parameters can be obtained in real time from the temperature sensor and humidity sensor preset around the target assembly line.

[0049] In one embodiment, the abnormality verification of the target gray board in combination with the assembly line parameters and the gray board transportation time includes the following steps: Calculate the theoretical transportation time of the target gray board in combination with the assembly line operation parameters and the gray board size; Calculate the absolute value of the time difference between the theoretical transportation time and the gray board transportation time; If the absolute value of the time difference is greater than the preset time difference threshold, it is determined that the target gray board has a gray board abnormality; If the absolute value of the time difference is less than or equal to the preset time difference threshold, it is determined that the target gray board does not have a gray board abnormality.

[0050] In the embodiment, the pipeline running parameters include pipeline running speed and pipeline pause time, and the theoretical conveying time of the target grey board can be calculated by dividing the size of the grey board by the pipeline speed. Then, the theoretical conveying time is subtracted by the time difference between the grey board conveying time, and the absolute value of the time difference is obtained. If the absolute value of the time difference is greater than the preset time difference threshold, it indicates that the size of the target grey board is too large or too small compared with the normal grey board, that is, the grey board is abnormal. The reasons for the abnormality of the grey board may be that the edge of the grey board is damaged, the position of the grey board is deviated, etc. For example, when the target grey board has edge damage, the effective length passing through the first preset position is reduced, resulting in that the conveying time of the grey board is shorter than the theoretical conveying time. When the target grey board has position deviation, the path passing through the first preset position may be lengthened, resulting in that the conveying time of the grey board is longer than the theoretical conveying time. Since the time difference of the target grey board exceeds the time difference threshold, it indicates that the abnormality of the target grey board is more serious, and it is difficult to compensate by adjusting the parameters of the target mechanical arm, which may result in that the finally produced packaging box has serious defects, such as wrinkles in the packaging box, and cannot be put into sales, which not only wastes production time, but also causes waste of paper. Therefore, an alarm information is directly output to remind the factory staff to remove the target grey board with abnormality. There are many reasons for the target grey board to have the above-mentioned abnormality, for example, the edge damage of the grey board may be caused by the existence of protrusions, burrs or foreign matters on the conveying track of the target pipeline, and the friction and collision between the edge of the grey board and these parts, thereby causing damage. It may also be caused by the serious wear or dull blade of the cutting tool in the previous cutting process, resulting in edge tearing when cutting the target grey board. The position deviation of the grey board may be caused by the impact of the surrounding airflow.

[0051] In one embodiment, determining the paper suction pose of the target mechanical arm preset above the target pipeline according to the first paper image and using image recognition technology includes the following steps: preprocessing the first paper image; performing edge detection on the preprocessed first paper image using an edge detection algorithm, and obtaining edge points of the preprocessed first paper image according to the edge detection result; mapping the edge points to the parameter space using Hough transformation, detecting all rectangular edges of the preprocessed first paper according to the mapping result of the edge points, and calculating all intersection pixel coordinates between the rectangular edges; performing coordinate conversion on all intersection pixel coordinates, and determining the paper suction pose of the target mechanical arm preset above the target pipeline according to the coordinate conversion result.

[0052] In the present embodiment, the preprocessing step includes graying and filter denoising, and a commonly used graying method is a weighted value method, which converts the colored first side paper image into a gray image. The filter denoising method includes Gaussian filtering, median filtering and the like, and the filter denoising can reduce the interference of noise on the subsequent steps. An edge detection algorithm is used to extract the edge points of the first side paper image, and a commonly used edge detection algorithm is a Canny algorithm. The Canny algorithm calculates the horizontal direction and vertical direction gradient of the pixel points in the first side paper image by using a Sobel operator to obtain the gradient amplitude. The first side paper image is traversed, only the local maximum value pixels in the gradient direction are retained, the edge is thinned to a single pixel width, the blur effect is eliminated, and high and low threshold values are set: the pixels higher than the high threshold value are directly determined as the edge; the pixels between the high and low threshold values are retained if they are connected with the high threshold value edge, otherwise they are suppressed, and finally all retained edge points are output. Then, the Hough transform is used to map the edge points to the parameter space, that is, the two-dimensional coordinate representation of the pixel points is converted into polar coordinates, such as p = xcosq + ysinq, p is the polar radius obtained after the pixel point is mapped, q is the polar angle, and x and y are the two-dimensional horizontal coordinate and two-dimensional vertical coordinate of the pixel point. All possible (p, q) combinations of each edge point are counted (q range 0~180°, resolution set to 1°; p resolution set to 1 pixel, adapt to the target side paper size), the number of votes for each (p, q) is counted by an accumulator, and the (p, q) pair with a vote number higher than a preset vote threshold (such as accumulator value > 15% of the total number of edge points) is screened out as a candidate straight line in the corresponding image space. Then, based on the rectangular prior knowledge of the packaging box produced by the factory (the opposite sides are parallel, and the adjacent sides are perpendicular), the candidate straight lines are classified, such as parallel group: the straight lines with a q difference of less than or equal to 2° are grouped together (such as horizontal side q ≈ 0° and vertical side q ≈ 90°), and each group needs to have and only have 2 straight lines (corresponding to a pair of opposite sides of the rectangle). Vertical verification: the q difference of the two groups of straight lines needs to be approximately 90° (such as 0° and 90° groups), and the adjacent side perpendicular relationship is verified. Finally, a plurality of straight lines are screened out, and then, since the rectangular sides of the target side paper should be located around the target side paper, the positions of all the straight lines are screened to obtain the four rectangular sides of the target side paper. Then, the polar coordinates of the four rectangular sides are used to solve the pixel coordinates of the intersection points between the rectangular sides by using the Cramer rule. Specifically, assuming that the four rectangular sides are L1(p1, q1), L2(p2, q2), L3(p3, q3) and L4(p4, q4), wherein L1||L3, L2||L4, and L1 L2, then the rectangular sides with a perpendicular relationship are combined into a system of equations, and the combined equations of the rectangular side L1 and the rectangular side L1 are as follows: ‌‌ Then, the intersection pixel coordinates between the rectangular side L1 and the rectangular side L1 are solved by the Cramer rule The same method is used to solve other intersection pixel coordinates. Cramer's rule is an important theorem in linear algebra for solving linear equations with the same number of variables and equations. Its core idea is to directly obtain the solution of the equation set through determinant calculation, especially suitable for the case where the coefficient matrix of the equation set is invertible (i.e., the determinant is not zero).

[0053] Then the coordinates of the intersection pixel coordinates are converted into a face paper adsorption pose that can be directly used to set the target mechanical arm. According to the device parameters of the image acquisition device, the intersection pixel coordinates are first converted into face paper imaging coordinates, which are a physical pose description of the edge intersection points (points corresponding to the intersection pixel coordinates) of the target face paper on the imaging plane of the image acquisition device. Then, the two-dimensional face paper imaging coordinates are converted into three-dimensional coordinates, i.e., acquisition device coordinates, which refer to the coordinates of the edge intersection points in the image acquisition device coordinate system constructed with the image acquisition device optical center as the origin. Based on the pinhole imaging model, the three-dimensional conversion of the face paper imaging coordinates is completed, and the acquisition device coordinates are obtained. The pinhole imaging model is derived from the pinhole imaging experiment. A baffle with a small hole is used to separate the "object" and the "screen". The light emitted by the object passes through the small hole and forms an inverted image on the screen. The pinhole imaging is abstracted as an image acquisition device model (camera model). The optical center corresponds to the position of the small hole, which is the "origin" of the light convergence and also the optical center of the simplified camera lens. The object space includes the object to be photographed, i.e., the face paper, and the imaging plane corresponds to the image acquisition device's photosensitive element. Based on the pinhole imaging model, combined with the device parameters and depth information of the image acquisition device, the two-dimensional face paper imaging coordinates can be converted into three-dimensional acquisition device coordinates. Then, according to the relative position of the image acquisition device and the target mechanical arm, the acquisition device coordinates are translated and rotated to obtain the three-dimensional coordinates of the edge intersection points of the target face paper in the target mechanical arm coordinate system, i.e., the face paper relative coordinates. Then, in order to avoid the edge of the target face paper from drooping too much when the target mechanical arm adsorbs the target face paper, the face paper adsorption position and the face paper adsorption pose, i.e., the face paper adsorption pose, need to be planned according to the face paper relative coordinates. The face paper adsorption position refers to the position of the target mechanical arm's suction disc and the target face paper. If it is a suction disc, the center point of the suction disc needs to coincide with the center point of the target face paper as much as possible. In addition, if there are multiple suction discs, the minimum circumscribed rectangle of all the suction discs can be constructed to make the center point of the minimum circumscribed rectangle coincide with the center point of the target face paper, and the center coincidence point is obtained to complete the precise adsorption, as shown in Figure 2If the target paper is offset in the horizontal plane, the target robot arm's suction angle also needs to be adjusted, and the four edges of the minimum circumscribed rectangle of the target robot arm's suction disc need to be parallel to the four edges of the target paper. In addition, the paper suction posture refers to the posture of the target robot arm's suction plane. In order to avoid wrinkles in the target paper, it is necessary to ensure that the target robot arm's suction plane is parallel to the target paper. Through the above method, the paper suction pose can be accurately calculated, and the target paper can be maximally avoided from being greatly sagged or wrinkled after being sucked by the target robot arm, so as to avoid wrinkles or bubbles when the target paper is attached to the target gray board, thereby improving the quality of the products of the factory and ensuring the efficiency of the factory.

[0054] In one embodiment, the coordinate conversion of all intersection pixel coordinates is completed, and the paper suction pose of the target robot arm preset above the target pipeline is determined according to the coordinate conversion result, including the following steps: Obtain the device parameters of the image acquisition device, including the device intrinsic parameters, depth information and device pose; For any intersection pixel coordinate, the intersection pixel coordinate is converted into a paper imaging coordinate according to the intrinsic information; Based on the pinhole imaging model and combined with the intrinsic information and the depth information, the three-dimensional conversion of the paper imaging coordinate is completed to obtain the acquisition device coordinate; Combined with the device pose and the robot arm pose in the pipeline structure parameters, the rotation matrix and the translation vector between the image acquisition device and the target robot arm preset above the target pipeline are calculated; Combined with the rotation matrix and the translation vector, the acquisition device coordinate is corrected to obtain the paper relative coordinate; According to all the paper relative coordinates, the edge center coordinates and the face center coordinates of the target paper are calculated, and the paper suction position of the target robot arm is planned combined with the edge center coordinates and the face center coordinates; Calculate the direction vector of the connecting line between adjacent paper relative coordinates, and fit the paper normal vector of the target paper according to the direction vector; According to the paper normal vector, the paper suction posture of the target robot arm is planned; The paper suction position and the paper suction posture are integrated into the paper suction pose of the target robot arm.

[0055] In the present embodiment, the image acquisition device refers to an industrial camera such as a 2D area array camera or a 3D depth camera, the device intrinsic parameter refers to a camera intrinsic parameter, which is calculated by a camera calibration algorithm such as Zhang's calibration method, mainly including focal length, principal point, pixel physical size, the depth information refers to the distance from the target point (such as the center point of the target face paper) in the scene to the camera optical center, which can be directly acquired by the depth camera, if it is a binocular camera, the same target can be shot by the left and right cameras, the pixel disparity is calculated, combined with the baseline (the distance between the two cameras) and the camera focal length to calculate, or according to the spatial coordinates of the target face paper in the world coordinate system, the depth information can be calculated according to the spatial coordinate difference between the target face paper and the image acquisition device.

[0056] The pixel coordinate system takes the upper left corner of the image as the origin, while the imaging coordinate system takes the intersection of the optical axis and the photosensitive chip (principal point) as the origin, and the conversion needs to eliminate the influence of pixel discretization, and the conversion formula is as follows:

[0057] wherein, , are the physical sizes (unit: mm / pixel, determined by intrinsic information) of a single pixel on the image acquisition device in the x-axis direction (x-axis), (y-axis) direction. , represent the coordinates of the principal point in the pixel coordinate system, which are determined according to the intrinsic information, (x0, y0) is the face paper imaging coordinate of the i-th intersection pixel coordinate (xi, yi), i=1, 2, 3, 4. , ,

[0058] Then according to the pinhole imaging model, the points on the imaging plane and the three-dimensional points in the image acquisition device coordinate system have a perspective relationship, therefore, the face paper imaging coordinates can be converted into the acquisition device coordinates according to the focal length information in the image acquisition device intrinsic information, and the depth of the intersection pixel corresponding to the intersection pixel coordinate in the image acquisition device coordinate system, i.e. the depth information, and using the three-dimensional conversion formula, the three-dimensional conversion formula is as follows:

[0059] wherein, is the depth information, is the focal length information, (xi, yi) is the i-th acquisition device coordinate. , ,

[0060] ​​​Then, the rotation matrix and translation vector between the image acquisition device and the target robotic arm are calculated based on the device posture and the robotic arm posture. The transposed posture refers to the three-dimensional coordinates of the origin of the image acquisition device coordinate system in the preset world coordinate system, and the robotic arm posture refers to the three-dimensional coordinates of the origin of the target robotic arm coordinate system in the same world coordinate system. For example, the world coordinate system can be centered on the starting point of the target conveyor belt, with the direction of movement along the target conveyor belt as the positive direction of the horizontal axis, the horizontal direction perpendicular to the surface of the target conveyor belt as the vertical axis, and the direction perpendicular to the ground upward as the positive direction of the vertical axis. The rotation matrix and translation vector of the image acquisition device and the target robotic arm can be directly calculated based on their three-dimensional coordinates in the same world coordinate system. The rotation angle of the image acquisition device coordinate system around the vertical axis of the target robotic arm coordinate system is calculated based on the transposed posture and the robotic arm posture. , substitute the rotation angle into the rotation matrix calculation formula to obtain the rotation matrix between the two. The rotation matrix calculation formula is as follows:

[0061] The rotation angle calculation steps include: first selecting two orthogonal unit vectors (such as the X-axis and Y-axis) in the image acquisition device coordinate system. Then, through coordinate transformation (subtracting the position offset of the two coordinate system origins), the coordinates of the two orthogonal unit vectors in the target manipulator coordinate system are obtained. The inverse tangent of these coordinates is then calculated to determine the rotation angle. The translation vector is the difference between the origin of the target manipulator coordinate system and the origin of the image acquisition device coordinate system.

[0062] The rotation matrix describes the angle of the image acquisition device relative to the target robotic arm base, and the translation vector represents the offset between the origin of the image acquisition device's coordinate system and the origin of the robotic arm's coordinate system. Given the rotation matrix and translation vector, the following formula can be used to correct the acquisition device's coordinates to the paper-relative coordinates in the target robotic arm's coordinate system:

[0063] in, is the rotation matrix, 、 and are the coordinate differences between the origin of the target manipulator coordinate system and the origin of the image acquisition device coordinate system on the horizontal, vertical and vertical axes, respectively. It refers to the matrix transpose.

[0064] After converting all the intersection pixel coordinates into the face paper relative coordinates, the coordinate average of the two face paper relative coordinates on the same edge in the target face paper is calculated to obtain the edge center coordinate, which refers to the center point coordinate of the edge of the target face paper. Then the coordinate average of the four face paper relative coordinates is calculated to obtain the face center coordinate, which refers to the center point coordinate of the positive center of the target face paper. The edge center coordinate is to ensure that the suction angle of the target mechanical arm is parallel or perpendicular to the angle of the center line of the target face paper, and the face center coordinate is to ensure that the suction center of the suction cup is the positive center of the target face paper. For example, the target mechanical arm has a row of suction cups at the suction end, and the suction position of the suction cup located at the positive center is the positive center of the target face paper, so as to ensure that the suction force is evenly distributed on the target face paper and to avoid the target face paper from being offset or deformed during the suction process. For the two adjacent face paper relative coordinates, the direction vector of the connecting line formed after the connection of the two is calculated, that is, the connecting line direction vector, which is also the direction vector of the four edges of the target face paper. The normal vector of the target face paper is fitted according to the calculated connecting line direction vector, so as to plan the suction posture of the suction cup of the target mechanical arm and ensure that the suction cup plane is consistent with the direction of the normal vector, that is, the suction cup plane is parallel to the target face paper plane, so as to ensure that the target face paper is evenly stressed. The face paper suction position and the face paper suction posture are integrated to obtain the face paper suction pose of the target mechanical arm.

[0065] In one embodiment, calculating the initial suction force and the moving speed of the target mechanical arm according to the production parameters comprises the following steps: combining the environmental parameters and the raw material parameters and determining the initial suction force of the target mechanical arm based on the pre-constructed suction force parameter table; determining the vertical moving speed of the target mechanical arm according to the mechanical arm pose in the pipeline operation parameters and the pipeline structure parameters; determining the horizontal moving speed of the target mechanical arm according to the raw material station distance in the pipeline operation parameters and the pipeline structure parameters and the vertical moving speed; integrating the vertical moving speed and the horizontal moving speed into the moving speed of the target mechanical arm.

[0066] In the present embodiment, first, an experimental robot arm completely identical to the target robot arm is used, such as the material, aperture, and layout of the suction cup, etc. are completely identical. Then, the suction force experiment is performed using the face paper used by all the packaging box products produced by the factory, and the parameters of the face paper used by different packaging box products are recorded, including thickness, grammage, material, etc. At the same time, the smoothness of the face paper is divided into high, medium, and low grades according to the material. At the same time, different grades of temperature and humidity are set by using humidifier and air conditioner, etc. to cover the actual temperature and humidity range of the actual production process as much as possible. The temperature and humidity can be divided into different ranges such as first, second, third, and fourth grades from high to low. First, set an initial suction force, such as a suction force of 1.0 kPa negative pressure, control the experimental robot arm to adsorb the face paper according to the initial suction force, and at the same time, use the infrared sensor to monitor the amount of sagging of the edge of the face paper. When the amount of sagging of the edge of the face paper is greater than a first preset threshold value (such as 1 mm) or the face paper falls, the fixed negative pressure is gradually increased to increase the suction force, such as increasing by 0.2 kPa each time, until the amount of sagging of the edge of the face paper is less than a second threshold value (such as 0.5 mm), and the negative pressure at this time is recorded as the minimum suction force. Then, on the basis of the minimum suction force, the fixed negative pressure is continuously increased step by step, and at the same time, the high-definition camera is used to shoot the image of the face paper. When wrinkles appear in the image of the face paper, the negative pressure at this time is recorded, and the maximum suction force is obtained by subtracting the fixed negative pressure. The same method is used to perform multiple experiments, and the experimental results are arranged into a suction force parameter table. Through the suction force parameter table, the corresponding minimum suction force and maximum suction force can be queried according to the environmental parameters and raw material parameters, and the average of the two is taken as the initial suction force.

[0067] Then, a vertical distance between the target robot and the target assembly line is calculated according to a robot pose in the pipeline running parameters, and a vertical axis coordinate of the robot pose is the vertical distance. The pipeline running parameters include a pipeline speed and a pipeline pause time. The pipeline pause time refers to a time period during which the target paperboard is paused when the target paperboard is transported to a second preset position, so as to facilitate accurate attachment of the target paper to the target baseboard. In order to save the attachment time, the target robot completes the target paper adsorption step at the same time when the target paperboard is transported to the second preset position, and moves the target paper to the top of the target paperboard. When the target paperboard stops moving, a series of subsequent attachment steps are immediately executed. The minimum vertical moving speed of the target robot is calculated according to the vertical distance between the target robot and the target assembly line and the pipeline pause time, so as to ensure smooth execution of the attachment step. Since the target robot needs to complete structural adjustment and other operations in the attachment step, the minimum vertical moving speed is appropriately increased according to the production experience of the frame semi-finished product and is used as the vertical moving speed. The transportation time of the target paperboard from the first preset position to the second preset position is calculated according to the pipeline speed. Since the target robot also needs to vertically move in the adsorption step, the vertical moving time is calculated according to the vertical moving speed and the vertical distance. The minimum horizontal moving speed is calculated by subtracting the vertical moving time from the transportation time and combining the raw material station spacing. Similarly, the minimum horizontal moving speed is appropriately increased according to the production experience of the frame semi-finished product and is used as the horizontal moving speed of the target robot. The vertical moving speed and the horizontal moving speed are integrated to obtain the moving speed of the target robot. This step can seamlessly connect the target robot running action with the running beat of the target assembly line as much as possible, and greatly improves the production efficiency.

[0068] In one embodiment, the pose estimation of the target paper and the target paperboard is completed by combining the second paper image and the paperboard image, and the robot attachment parameters of the target robot are set according to the pose estimation result, including the following steps: The second paper image is preprocessed, and the second paper image includes a front paper image and a back paper image; All paper edges of the preprocessed second paper image are detected by using an edge detection algorithm; The paper region of the preprocessed second paper image is segmented according to all the paper edges, and the paper region includes a front paper region and a back paper region; The wrinkle detection of the front paper region is completed by using image recognition technology; If the wrinkle detection fails, the wrinkle type of the target paper is determined according to the wrinkle recognition result, and the initial adsorption force is adjusted according to the wrinkle type until the wrinkle detection passes; If the wrinkle detection passes, the center point coordinates of all suction cup regions in the back surface area of the face paper are extracted, and whether the target mechanical arm has suction deviation is determined based on the center point coordinates; If the target mechanical arm has suction deviation, the edge fitting parameters of the target face paper are determined according to the suction deviation; If the target mechanical arm does not have suction deviation, the edge sag curvature of the target face paper is calculated by using the least square method, and the edge fitting parameters of the target face paper are determined according to the edge sag curvature; The device parameters of the image acquisition device are obtained, and the face paper fitting pose of the target mechanical arm is located combining the device parameters and the gray board image; The edge fitting parameters and the face paper fitting pose are set as the mechanical arm fitting parameters of the target mechanical arm.

[0069] In the embodiment, the face paper front surface image refers to the image of the side of the target face paper facing the target flow line, and the face paper back surface image refers to the image of the side adsorbed by the suction cup of the target mechanical arm. The preprocessing step includes grayscale processing, filter denoising and contrast enhancement. The grayscale processing refers to converting a color image into a grayscale image to reduce color interference and unify the calculation dimension. The commonly used method is weighted average filtering. The commonly used filter for filter denoising is Gaussian filtering, median filtering, etc. Contrast enhancement can enhance the gray difference between the edge of the target face paper and the background. If there are wrinkles in the target face paper, the recognition accuracy of the wrinkles can also be enhanced. The commonly used method for contrast enhancement is histogram equalization or setting an adaptive threshold. Then, an edge detection algorithm is used to detect all the face paper edges of the second face paper image after preprocessing. The face paper edge refers to an edge point. The commonly used edge detection algorithm is Canny edge detection, which mainly includes calculating the gradient amplitude and direction and using double thresholds to screen the edge. According to the face paper edge screening result, the face paper region and the background of the second face paper image are segmented. This step is to reduce the influence of the complex background region on the subsequent wrinkle recognition. Since there may be wrinkles inside the face paper region, in order to ensure that the segmentation of the face paper region will not be affected by the wrinkles inside, a contour extraction method can be used to screen out the contour with the largest area and the closest rectangular shape as the face paper region contour, and the face paper region is segmented according to the face paper region contour. Since the face paper wrinkles mainly manifest as local gray abnormalities and small edges, if there are edge points in the face paper region and the length of the edge obtained by fitting the edge points is greater than a preset length threshold, the corresponding position in the face paper region is marked as an edge abnormality. The face paper front surface region refers to the face paper region segmented from the face paper front surface image, and the face paper back surface region refers to the face paper region segmented from the face paper back surface image.

[0070] In the crease detection stage, if the target face paper is not marked as edge abnormal, it is determined that the target face paper crease detection passes. If the target face paper is marked as edge abnormal, a small size sliding window (such as 5x5 pixels) is set, which is slid pixel by pixel in the back area of the face paper, and the variance of the gray value of each window is calculated. The greater the variance, the more intense the gray fluctuation in the window. The back area of the face paper with a variance greater than the preset variance threshold is marked as a gray abnormal area. The area marked as edge abnormal is regarded as an edge abnormal area. If the overlap between the edge abnormal area and the gray abnormal area is greater than the preset overlap threshold, it is determined that the crease detection of the target face paper fails. The overlap can be calculated according to the proportion of the overlapping pixel area to the total pixel area of the two abnormal areas.

[0071] The overlapping area between the edge abnormal area and the gray abnormal area is taken as the wrinkle area. The pixel number of the wrinkle area is counted, and the product of the pixel number and the conversion coefficient of the pixel to the actual size (such as 1 pixel = 0.01 mm2) is calculated to obtain the wrinkle area of the wrinkle area. At the same time, the face paper area is calculated according to the size of the face paper, and the wrinkle area is divided by the face paper area to obtain the wrinkle ratio. Then, the wrinkle area is subjected to connected component analysis, which can adopt the 8-neighborhood method, that is, two pixels are adjacent and both are wrinkle area pixels, which are considered to be connected, and adjacent wrinkle areas are merged into one connected domain. If the shortest distance between two connected domains is less than a preset distance threshold (such as 5 mm), the two connected domains are merged into one connected domain. After the above steps are completed, the number of connected domains is counted to obtain the number of wrinkles. According to the wrinkle ratio, the wrinkles of the target face paper are divided into local wrinkles and global wrinkles, for example, the wrinkle ratio is greater than 20%, which is divided into global wrinkles, and the wrinkle ratio is less than or equal to 20%, which is divided into local wrinkles. Then, according to the number of wrinkles, the wrinkles are divided into a small amount of wrinkles, a medium amount of wrinkles, and a large amount of wrinkles, such as 1-2 wrinkles, which are divided into a small amount of wrinkles, 3-4 wrinkles, which are divided into a medium amount of wrinkles, and more than 5 wrinkles, which are divided into a large amount of wrinkles. For local wrinkles, the suction force of the suction cup near the wrinkles is appropriately reduced according to the number of wrinkles, for example, a small amount of wrinkles is reduced by 0.1 kPa suction force, a medium amount of wrinkles is reduced by 0.2 kPa suction force, and a large amount of wrinkles is reduced by 0.3 kPa suction force. If it is a global wrinkle, the suction force of all suction cups is reduced according to the number of wrinkles. After completing the suction force reduction once, the image of the target face paper is continuously collected for wrinkle detection, and if the detection results of two consecutive times show that the wrinkle area and the number of wrinkles do not change or the detection times reach a preset maximum number threshold (such as three times), the detection is stopped, and the subsequent detection step of suction deviation is directly performed. This is because the continuous two detection results do not change, which indicates that the current suction force adjustment scheme has no actual effect on the improvement of the wrinkles, for example, the physical defects of the face paper itself, and the processing time of a single target face paper is too long, which may affect the overall productivity of the target pipeline, therefore, the maximum number threshold is set to avoid invalid time consumption. If the adjusted detection result shows that the wrinkles of the target face paper disappear, the suction force of the target mechanical arm at this time, the raw material parameters of the target face paper, and the environmental parameters at this time are recorded, and the suction force parameter table is updated according to these parameters and the suction force.

[0072] If the wrinkle detection passes, further detection is needed to detect whether the suction cup of the target mechanical arm is suctioned at the center position of the target face paper, if there is suction deviation, if the suction position is left, right, up or down, refer to Figure 3, will cause the edges of the subsequent target face paper to sag to varying degrees. At this time, if direct face paper lamination is performed, it may cause the finished laminated gray board to have wrinkles, affecting the quality of the final product. Then, according to the deviation direction of the adsorption deviation, the edge lamination parameters of the target face paper are determined. For example, if the adsorption position is slightly above, it means that the lower end position of the target face paper has a larger sag amplitude, so the angle of the target mechanical arm in the horizontal plane needs to be adjusted during lamination, that is, the target mechanical arm slightly lifts up on one side of the lower end position of the target face paper. The lifting amplitude is determined according to the adsorption deviation. The larger the adsorption deviation, the larger the lifting amplitude. Conversely, the smaller the adsorption deviation, the smaller the lifting amplitude. By lifting locally, the edge sag amplitude is offset. When the target mechanical arm is about to perform the lamination step, the end with a smaller edge sag amplitude is allowed to contact the target gray board first. After contact, the target mechanical arm rotates slowly to flatten (rotation speed ≤ 1° / s) with the lamination side as the axis, so that the edge with a larger sag amplitude gradually contacts the target gray board, completing the lamination and avoiding wrinkles caused by "gravity difference". There is also a situation where if the adsorption position is slightly left up, left down, right up, or right down, the corner opposite to the position deviation needs to be laminated first. For example, if the adsorption position is slightly left up, it means that the right lower corner of the target face paper has a larger sag amplitude, so the right lower position of the target mechanical arm is lifted locally to make the right lower end of the target mechanical arm slightly higher than the left upper end. The left upper corner of the target face paper is laminated first, followed by the right lower corner, the left lower corner, and finally the right lower corner. The edge lamination parameters of the target face paper, including the face paper edge lamination sequence and the local lifting amplitude, are determined according to the adsorption deviation. If the target mechanical arm has no adsorption deviation, the least squares method is used to calculate the edge sag curvature of all long edges and all short edges of the target face paper. The higher the edge sag curvature, the higher the edge sag amplitude of the target face paper. If the edge sag curvature of any long edge is greater than a preset first curvature threshold, or the edge sag curvature of any short edge is greater than a preset second curvature threshold, which is less than the first curvature threshold, it means that the adsorption force of the target mechanical arm suction cup is insufficient, so the adsorption force of the target mechanical arm suction cup is appropriately increased, and the face paper front image of the target face paper is reacquired to recalculate the edge sag curvature. If the edge sag curvatures of the long edges are all less than or equal to the first curvature threshold, and the edge sag curvatures of the short edges are all less than or equal to the first curvature threshold, the adsorption force at this time is used as the edge lamination parameter of the target face paper according to the edge sag curvature, and the adsorption force parameter table is updated according to the adsorption force at this time. If the adsorption force of the target mechanical arm suction cup is increased twice in succession, and the edge sag curvature does not change or still exists for the long edge or short edge greater than the first curvature threshold or the second curvature threshold, then for the long edge or short edge with abnormal curvature, the local lifting amplitude of the target mechanical arm and the face paper edge lamination sequence are adjusted according to the position of the edge sampling point corresponding to the maximum edge vertical distance in the face paper edge.For example, if the edge with abnormal curvature is the long edge on the left side of the target paper, the position of the edge sampling point corresponding to the maximum edge vertical distance determined when calculating the sagging curvature of the long edge is obtained. If the edge sampling point is at the lower position of the long edge, the lower end of the target robot arm needs to be locally lifted, because the lower end of the target paper sags at this time, and when the target paper is attached, the upper end of the target paper needs to be attached first, and then the lower end of the target paper is attached, and vice versa. By this method, it can be prevented that the target paper is attached to the target paper after the target paper is attached to the target paper, and the target paper is attached to the target paper. The image acquisition device is obtained, and the device parameters are obtained. The device parameters and the gray board image are combined to locate the paper attachment pose of the target robot arm. The calculation process of the paper attachment pose is the same as that of the paper suction pose, mainly by preprocessing the gray board image, using an edge detection algorithm to detect the edge of the preprocessed gray board image to obtain the edge point of the gray board image, and then locating all the intersection pixel coordinates of the gray board image according to the Hough line detection algorithm. After a series of transformations are performed on all the intersection pixel coordinates of the gray board image, the paper attachment position and the paper attachment attitude of the target robot arm are obtained, and the paper attachment position and the paper attachment attitude are integrated to obtain the paper attachment pose of the target robot arm. Finally, the edge attachment parameters and the paper attachment pose are set as the robot attachment parameters of the target robot arm. Through the above steps, the target paper can be accurately attached to the target gray board, and at the same time, wrinkles can be prevented during the attachment process to ensure the appearance of the packaging box produced by the factory.

[0073] In one embodiment, the center point coordinates of all suction disc regions in the back surface region of the paper are extracted, and whether the target robot arm has suction deviation is determined based on the center point coordinates, including the following steps: The minimum circumscribed rectangle of the back surface region of the paper is fitted, and all vertex coordinates of the minimum circumscribed rectangle are extracted; The pixel values of the back surface region of the paper are counted, and all suction disc regions in the back surface region of the paper are detected according to the pixel values and using the Hough circle detection algorithm; The center coordinates of all suction disc regions are extracted, and the center point coordinates of all suction disc regions are determined according to all center coordinates; The vertical distance between the center point of all suction disc regions and the paper edge of the back surface region of the paper is calculated based on the center point coordinates and all vertex coordinates, and whether the target robot arm has suction deviation is determined according to the vertical distance.

[0074] In the present embodiment, since the face paper back area may not be a standard rectangle due to the sag of the edge of the face paper back area, it is necessary to fit the minimum circumscribed rectangle of the face paper back area. After the edge detection is completed, the minimum circumscribed rectangle of the face paper back area can be fitted by using the minAreaRect function, which can output the center point coordinates, width and height, and rotation angle of the minimum circumscribed rectangle and other rectangular parameters. Then the boxPoints function can be used to convert the rectangular parameters into the pixel coordinates of the four vertices of the minimum circumscribed rectangle, i.e. the vertex coordinates. Then the pixel value of the face paper back area is obtained, which refers to the gray value of the face paper back area. Then the suspected sucker area is screened according to the gray value. First, the average pixel value of the face paper back area is calculated, and then the dynamic threshold is set according to the color of the sucker. The suspected sucker area is screened by the dynamic threshold. For example, if it is a black or other dark sucker, the dynamic threshold can be a-0.5b, a is the average pixel value, and b is a preset fixed parameter, which can be 20. The area with a gray value less than a-0.5b is marked as a suspected sucker area. If it is a transparent sucker, the dynamic threshold can be set to a+0.5b, and the area with a gray value greater than a+0.5b is marked as a suspected sucker area. Then the Canny edge detection algorithm is used to detect the edge of the suspected sucker area, and then the detected edge points are subjected to Hough transformation. The circular detection parameters are set according to the actual size of the sucker, including the sucker radius, the center distance between the suckers, etc., so as to screen the sucker area that meets the actual size of the sucker. The Hough circle detection algorithm and the Hough line detection algorithm both belong to the Hough algorithm, and their Hough transformation steps are similar, except that the parameter space of the Hough circle detection algorithm is three-dimensional, and the voting step is first the center and then the radius. Therefore, the center coordinates can be directly obtained after the Hough transformation process is completed. Since the sucker is basically a circular sucker, the Hough circle detection algorithm can be used to detect the sucker area. Then the center point coordinates of all sucker areas are calculated according to all center coordinates, i.e. the center point coordinates. For reference Figure 3The center point of the suction disc area refers to the center point of the area composed of all suction disc areas. For example, if there is only one suction disc area, the center point coordinate is the center coordinate of the circle. If there are three suction disc areas, and the three suction disc areas are uniformly arranged in a row, the center coordinate of the circle of the suction disc area in the middle position is taken as the center point coordinate. In addition, there can be more suction disc areas, which are arranged in multiple rows. The center point coordinate of the entire suction disc area can be constructed based on the center coordinates of the circles and the radii of the suction disc areas. The coordinates of the center of the minimum circumscribed rectangle are taken as the center point coordinates of the entire suction disc area. Since the center coordinates of the circles and the four vertex coordinates are based on the pixel coordinate system of the back surface area of the face paper, coordinate conversion is not required. After calculating the center point coordinates of the entire suction disc area, the vertical distance between the center point and the edge of the back surface area of the face paper is calculated using the distance formula from a point to a straight line. The edge of the face paper refers to the four edges of the back surface area of the face paper, including two short edges and two long edges. The difference between the vertical distances from the center point to the two short edges of the back surface area of the face paper is calculated to obtain the short edge distance difference. Similarly, the difference between the vertical distances from the center point to the two long edges of the back surface area of the face paper is calculated to obtain the long edge distance difference. If the short edge distance difference is greater than a first preset distance threshold, it indicates that the suction disc attachment position of the target robot arm is biased upward or downward. If the long edge distance difference is greater than a second preset distance threshold (the first distance threshold is greater than the second distance threshold), it indicates that the suction disc attachment position of the target robot arm is biased to the left or right. Once any of the above conditions occurs, it indicates that the target robot arm has an attachment deviation. The possible reason for the attachment deviation is that there is a certain error in the device internal parameters, depth information, and device pose of the image acquisition device, which causes the calculated face paper attachment position to deviate, thereby causing the four edges of the target face paper to have different sag amplitudes. The above parameter errors are difficult to avoid, and the influence of gravity on the target face paper is also unavoidable. Therefore, the above steps are required for attachment deviation verification, and the parameters of the target robot arm are adjusted before the fitting step based on the attachment deviation verification result, i.e., by adjusting the edge fitting parameters of the target face paper to overcome the problem of uneven edge sag amplitude caused by gravity and parameter errors, to avoid wrinkles or edge lifting during fitting, thereby improving the quality pass rate of factory products.

[0075] In one embodiment, calculating the edge sag curvature of the target face paper using the least squares method includes the following steps: For any face paper edge, a plurality of edge sampling points are uniformly extracted from the face paper edge according to a preset interval parameter; All edge sampling points are fitted into an edge straight line using the least squares method; The edge vertical distance between all edge sampling points and the edge straight line is calculated; The edge sag curvature of the target face paper is calculated based on all edge vertical distances of all face paper edges.

[0076] In this embodiment, for any paper edge, multiple edge sampling points are uniformly extracted at a preset interval (e.g., 10 pixels) to ensure that the edge sampling points represent the overall shape of the target paper edge. Least squares fitting is then used to fit all edge sampling points to an edge line. The key is to find an edge line that minimizes the sum of the squared distances from all edge sampling points to the line. Specifically, the calculation begins by calculating the average horizontal and vertical coordinates of all edge sampling points. The center position of all edge sampling points is then determined; the optimal fitting edge line must pass through this center position. Next, the difference between the horizontal coordinate and the average horizontal coordinate, as well as the difference between the vertical coordinate and the average vertical coordinate, is calculated for each edge sampling point. A matrix is ​​constructed using all the horizontal and vertical coordinate differences, and the eigenvalues ​​and eigenvectors of this matrix are calculated. The eigenvector with the smallest eigenvalue corresponds to the direction of the edge line. Knowing the direction of the edge line and the center position it must pass through, the optimal edge line can be determined. Next, the perpendicular distances from all edge sampling points to the edge line are calculated. The maximum edge vertical distance / long side length or wide side length (the length or width of the target paper) can be directly used as the edge droop curvature of all long sides and all wide sides of the target paper respectively.

[0077] The present application also provides an industrial vision-based enclosure forming control system, including: a memory configured to store instructions; and The processor is configured to call instructions from the memory and implement the above-mentioned industrial vision-based frame forming control method when executing the instructions.

[0078] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0079] Among them, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD) or flash memory card (FC) equipped on the computer device. In addition, the memory can also be a combination of an internal storage unit and an external storage device of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or is to be output. This application does not impose any restrictions on this.

[0080] The embodiment of the present application further provides a machine readable storage medium, which stores instructions for causing a machine to execute the industrial vision-based bounding box forming control method.

[0081] Those skilled in the art should understand that the embodiment of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0082] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems), computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be realized by computer program instructions, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks.

[0083] These computer program instructions can also be stored in a computer readable storage medium capable of guiding the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction means, which realizes the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks.

[0084] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide a process for realizing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified by one or more blocks.

[0085] In a typical configuration, the computing device includes one or more processors (CPU), input / output interface, network interface and memory.

[0086] Memory can include non-persistent memory, Random Access Memory (RAM), and / or non-volatile memory, such as Read Only Memory (ROM) or flash memory, in a computer readable medium. Memory is an example of computer readable media.

[0087] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "comprising", "including", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0089] The above only is an embodiment of the present application, and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A frame forming control method based on industrial vision, characterized in that: The method comprises the following steps: When it is detected that the target gray board is transported to the first preset position in the target assembly line, a timing device preset on the target assembly line is turned on, and the gray board transport time consumed by the target gray board passing the first preset position is counted by the timing device; Collect the production parameters of the target production line, and complete the abnormality check of the target gray board by combining the production line parameters and gray board delivery time; If the target gray plate has gray plate abnormality, an alarm message will be output; If the target gray plate has no gray plate abnormality, the first surface paper image of the target surface paper is captured by using an image capture device preset in the target production line; Determine the paper adsorption posture of a target robotic arm preset above the target assembly line based on the first paper image and using image recognition technology; Calculate the initial adsorption force and moving speed of the target robot arm based on production parameters; Set the tissue paper adsorption posture, initial adsorption force and movement speed as the manipulator adsorption parameters of the target manipulator; When it is predicted that the target gray plate is delivered to the second preset position in the target assembly line, the target robotic arm is controlled to adsorb and move the target surface paper to just above the target gray plate according to the robotic arm adsorption parameters; respectively collecting a second surface paper image and a gray card image of the target surface paper and the target gray card; Combine the second paper image and the gray plate image to complete the pose estimation of the target paper and the target gray plate respectively, and set the robot arm fitting parameters for the target robot arm according to the pose estimation results; Control the target robotic arm to complete the laminating process of the target gray board and the target surface paper according to the robotic arm laminating parameters to obtain a laminated gray board; The frame forming equipment pre-installed on the target production line is used to bend and laminate the gray boards to obtain a semi-finished frame.

2. The method according to claim 1, characterized in that The production parameters include assembly line parameters, raw material parameters and environmental parameters. The assembly line parameters include assembly line operation parameters and assembly line structure parameters. The raw material parameters include gray board size, gray board material, facial paper size, facial paper material, facial paper quality and facial paper thickness. The environmental parameters include assembly line temperature and assembly line humidity.

3. The method according to claim 2, characterized in that The method of completing the abnormality check of the target gray board by combining the pipeline parameters and the gray board delivery time includes the following steps: Calculate the theoretical delivery time of the target grey board by combining the operation parameters of the production line and the size of the grey board; Calculate the absolute value of the time difference between the theoretical conveying time and the gray board conveying time; If the absolute value of the time difference is greater than the preset time difference threshold, it is determined that the target gray plate has a gray plate abnormality; If the absolute value of the time difference is less than or equal to the preset time difference threshold, it is determined that there is no gray plate abnormality in the target gray plate.

4. The method according to claim 1, wherein The method of determining the paper adsorption posture of the target robotic arm preset above the target assembly line based on the first paper image and using image recognition technology includes the following steps: Preprocessing the first paper image; Performing edge detection on the preprocessed first paper image using an edge detection algorithm, and obtaining edge points of the preprocessed first paper image according to the edge detection result; The edge points are mapped to the parameter space using Hough transform, all rectangular edges of the pre-processed first paper are detected based on the edge point mapping results, and the pixel coordinates of all intersection points between all rectangular edges are calculated; Complete the coordinate transformation of all intersection pixel coordinates, and determine the tissue paper adsorption posture of the target robotic arm preset above the target assembly line based on the coordinate transformation results.

5. The method according to claim 4, characterized in that The steps of completing the coordinate conversion of all intersection pixel coordinates and determining the tissue paper adsorption posture of the target robotic arm preset above the target assembly line according to the coordinate conversion results include the following steps: Obtaining device parameters of the image acquisition device, the device parameters including device internal parameters, depth information, and device posture; For any intersection pixel coordinate, the intersection pixel coordinate is converted into the paper imaging coordinate according to the internal reference information; Based on the pinhole imaging model and combined with the internal reference information and depth information, the three-dimensional transformation of the tissue imaging coordinates is completed to obtain the acquisition device coordinates; The rotation matrix and translation vector between the image acquisition device and the target robotic arm preset above the target pipeline are calculated by combining the device posture and the robotic arm posture in the pipeline structure parameters; Combine the rotation matrix and translation vector to correct the coordinates of the acquisition device and obtain the relative coordinates of the paper; Calculate the edge center coordinates and surface center coordinates of the target paper based on the relative coordinates of all paper sheets, and plan the paper adsorption position of the target robot arm by combining the edge center coordinates and surface center coordinates; Calculate the direction vector of the line connecting the relative coordinates of adjacent tissue papers, and fit the tissue paper normal vector of the target tissue paper according to the direction vector of the line connecting the line; Plan the tissue paper adsorption posture of the target robot arm according to the tissue paper normal vector; The tissue paper adsorption position and tissue paper adsorption posture are integrated into the tissue paper adsorption posture of the target robot arm.

6. The method according to claim 2, characterized in that Calculating the initial adsorption force and moving speed of the target robot arm according to the production parameters includes the following steps: Determine the initial adsorption force of the target robot arm based on a pre-built adsorption force parameter table in combination with environmental parameters and raw material parameters; Determine the vertical moving speed of the target manipulator according to the manipulator posture in the assembly line operation parameters and the assembly line structure parameters; Determine the horizontal moving speed of the target robot arm according to the assembly line operation parameters, the raw material station spacing and the vertical moving speed in the assembly line structure parameters; The vertical movement speed and the horizontal movement speed are integrated into the movement speed of the target robot arm.

7. The method according to claim 1, characterized in that The steps of respectively completing the posture estimation of the target paper and the target gray board by combining the second paper image and the gray board image, and setting the robot arm fitting parameters for the target robot arm according to the posture estimation results include the following steps: Preprocessing the second paper image, where the second paper image includes a front side image and a back side image; Detecting all paper edges of the second paper image after preprocessing using an edge detection algorithm; Segmenting a pre-processed second paper image's paper area based on all paper edges, the paper area including a front side area and a back side area; Use image recognition technology to detect wrinkles on the front area of ​​facial tissue; If the wrinkle detection fails, the wrinkle type of the target paper is determined based on the wrinkle recognition result, and the initial adsorption force is adjusted according to the wrinkle type until the wrinkle detection passes; If the wrinkle detection passes, the center coordinates of all suction cup areas in the back area of ​​the tissue paper are extracted, and based on the center coordinates, it is determined whether the target robot arm has adsorption deviation; If the target robotic arm has an adsorption deviation, the edge fitting parameters of the target paper are determined based on the adsorption deviation; If the target robotic arm does not have adsorption deviation, the edge droop curvature of the target paper is calculated using the least squares method, and the edge lamination parameters of the target paper are determined based on the edge droop curvature; Obtain device parameters of the image acquisition device, and locate the paper laminating posture of the target robotic arm by combining the device parameters and the gray plate image; Set the edge fitting parameters and paper fitting pose as the target robot arm's robot fitting parameters.

8. The method according to claim 7, characterized in that The method of extracting the center point coordinates of all suction cup areas in the back area of ​​the tissue paper and determining whether the target robot arm has adsorption deviation based on the center point coordinates includes the following steps: Fit the minimum bounding rectangle of the back area of ​​the tissue paper and extract all vertex coordinates of the minimum bounding rectangle; Counting the pixel values ​​of the back area of ​​the tissue paper, and detecting all the suction cup areas in the back area of ​​the tissue paper based on the pixel values ​​and using the Hough circle detection algorithm; Extracting the center coordinates of all suction cup areas, and determining the center point coordinates of all suction cup areas based on all the center coordinates; The vertical distance between the center point of the entire suction cup area and the edge of the tissue paper in the back area is calculated by combining the center point coordinates and all vertex coordinates, and whether the target robot arm has adsorption deviation is determined based on the vertical distance.

9. The method according to claim 7, characterized in that The method of calculating the edge droop curvature of the target paper using the least squares method comprises the following steps: For any edge of the paper, multiple edge sampling points are evenly extracted from the edge of the paper according to the preset interval parameters; All edge sampling points are fitted into edge straight lines using the least squares method; Calculate the edge vertical distance between all edge sampling points and the edge straight line; The edge sag curvature of the target paper is calculated based on the vertical distances of all edges of the paper.

10. A frame forming control system based on industrial vision, characterized in that: include: a memory configured to store instructions; as well as A processor is configured to call instructions from a memory and implement the industrial vision-based frame forming control method according to any one of claims 1 to 9 when executing the instructions.

Images