Accurate attachment process optimization system based on computer vision

Abstract

The invention discloses an accurate attachment process optimization system based on computer vision, and the system comprises an image collection module which is used for obtaining a product and label image; the contour construction module is used for extracting edges and generating a contour point set; the pose recognition module is used for determining the spatial positions and angles of the product and the label based on the shape context and the Hungary algorithm; the coordinate mapping and compensation module is used for mapping the spatial pose to a tool coordinate system and generating a compensation amount; the track generation and motion control module is used for generating a smooth attaching track and driving an attaching mechanism to execute operation; the quality detection and distribution module is used for collecting the post-pasting image, judging the qualification and controlling the distribution of defective products; and the beat optimization module is used for dynamically adjusting the conveying speed and the camera triggering opportunity. According to the method, stable product and label pose recognition, rapid point pair matching and dynamic beat optimization based on the queuing theory can be realized under complex working conditions, so that the attachment precision and the production efficiency are both considered.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent manufacturing, and in particular to a precision bonding process optimization system based on computer vision. Background Technology

[0002] Existing automated labeling equipment largely relies on mechanical positioning and fixing fixtures to ensure product placement, followed by label transfer and application by the labeling head. This method lacks flexibility, making it difficult to adapt to products of different sizes and shapes, and the application accuracy is limited by product clamping errors, making deviations prone to occur. In recent years, computer vision has been introduced into the labeling process, using image acquisition and edge detection to achieve product and label positioning. However, most existing methods are based on general algorithms, lacking matching efficiency and robustness. When the product shape is complex or the label is rotated or deformed, positioning instability can easily occur.

[0003] In terms of process control, existing systems mostly adopt fixed conveying speeds and uniform triggering times, making it difficult to dynamically adjust according to the identification complexity and bonding time of different products. When the identification confidence decreases or the proportion of defective products increases, the lack of a scheduling optimization mechanism based on queuing models can easily lead to increased waiting time, disordered production rhythm, and difficulty in balancing bonding accuracy and production efficiency.

[0004] Therefore, how to provide a computer vision-based precision bonding process optimization system is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a computer vision-based precision bonding process optimization system. This invention introduces orientation weighting and an improved Hungarian algorithm hierarchical pruning mechanism in the construction of contour point sets and shape context matching, enabling stable spatial pose recognition of products and labels under complex shapes and rotation conditions, while reducing the computational scale of point-pair matching to meet real-time requirements. Furthermore, it combines a queuing theory-based cycle optimization method, introducing recognition confidence correction and defective product queue scheduling mechanisms during the modeling process, thereby dynamically adjusting the conveying speed and camera triggering timing to achieve a balance between bonding accuracy and production efficiency.

[0006] A computer vision-based precision bonding process optimization system according to an embodiment of the present invention includes the following modules:

[0007] The image acquisition module is used to acquire product images and label images;

[0008] The contour construction module is used to perform edge extraction on product images and label images and construct a set of contour points;

[0009] The pose recognition module is used to generate a descriptor subset based on the shape context algorithm and match it using the Hungarian algorithm to obtain the spatial position and angle of the product and label.

[0010] The coordinate mapping and compensation module is used to map the spatial position and angle of the product and label to the coordinate system of the attachment mechanism tool, compare and calculate the difference between the position and angle, and perform filtering and thresholding on the difference to generate the compensation amount.

[0011] The trajectory generation and motion control module is used to generate an attachment trajectory command based on the compensation amount, and send the attachment trajectory command to the motion control unit to drive the attachment mechanism to perform the attachment operation;

[0012] The quality inspection and diversion module is used to acquire post-application images after the application is completed, perform edge integrity analysis and position deviation calculation, and generate diversion control commands when the application is deemed unqualified.

[0013] The takt time optimization module is used to abstract the waiting time and service time of products in the conveying, positioning, attaching and diverting stages into queuing system parameters, and to use the takt time optimization method based on queuing theory model to calculate queue length, average waiting time and optimal service rate, and adjust conveying speed and camera triggering timing.

[0014] Furthermore, the modules are interconnected using the following method:

[0015] Perform edge extraction on the product image and label image to construct a set of contour points;

[0016] A descriptive subset is generated based on a shape context algorithm, and the spatial position and angle of the product and label are obtained after matching it with template data.

[0017] The spatial position and angle of the product and label are mapped to the coordinate system of the attachment mechanism tool. The difference between the position and angle is calculated by comparison, and the difference is filtered and thresholded to obtain the compensation amount.

[0018] The application trajectory command is generated based on the compensation amount and sent to the motion control unit to drive the application mechanism to perform the application operation, collect the post-application image, determine the application quality, and generate a diversion command to control the discharge of defective products when they are not qualified.

[0019] The waiting time and service time of products in the process of conveying, positioning, attaching and sorting are abstracted into queuing system parameters;

[0020] The queue length, average waiting time, and optimal service rate are calculated using a queuing theory-based takt optimization method, and the delivery speed and camera triggering timing are adjusted accordingly.

[0021] Furthermore, the step of constructing the contour point set includes:

[0022] The acquired images are processed to grayscale and contrast enhancement.

[0023] The Canny edge detection operator is used to extract boundary contour lines, and the main contour with the largest area or the boundary length that meets the threshold is retained through connected component analysis.

[0024] The contour curve is resampled at set intervals to form a contour point set.

[0025] Furthermore, the steps for determining the spatial position and angle include:

[0026] Reference points are selected sequentially from the set of contour points, and a logarithmic polar coordinate system is established in the neighborhood, dividing the neighborhood into several angular intervals and radial intervals;

[0027] The distribution of neighborhood points in each interval is counted to form a local histogram, and a direction weighting factor is introduced in the statistical process;

[0028] Neighboring points that are aligned with or orthogonal to the local tangent direction of the contour point have the highest set weight in the descriptor statistics, while neighboring points that deviate from the preset angle threshold have progressively lower weights.

[0029] Normalize the histogram descriptor subsets for all reference points;

[0030] The normalized histogram descriptor subset is matched with the histogram descriptor subset of the template image, and the point pair relationship is determined by the Hungarian algorithm.

[0031] Based on point-to-point relationships, the spatial position and angle of the product and label are calculated through rigid body registration.

[0032] Furthermore, the step of determining the point-pair relationship using the Hungarian algorithm includes:

[0033] An initial cost matrix is ​​constructed based on a descriptive subset of the product outline point set and the label outline point set, wherein the elements of the cost matrix represent the matching cost between point pairs.

[0034] Perform a two-stage pruning process before generating the cost matrix:

[0035] In the first stage, candidate point pairs are coarsely screened according to preset distance and angle thresholds, and only point pairs that meet the spatial relationship constraints are retained.

[0036] The second stage involves fine screening based on local geometric consistency rules in the coarse screening results. These geometric consistency rules include maintaining the relative polar angle order of adjacent sampling points and ensuring that the rate of change of the boundary normal does not exceed a set limit.

[0037] A sparse cost matrix is ​​constructed from the candidate point pair set obtained after two-stage pruning.

[0038] The Hungarian algorithm is run on the sparse cost matrix, sequentially performing the steps of finding feasible top labels, constructing feasible matches, iteratively updating labels and expanding matches, until all point pairs are assigned.

[0039] Output the optimal pair relationship between the product point set and the label point set.

[0040] Furthermore, the step of mapping the spatial position and angle of the product and label to the coordinate system of the attachment mechanism tool and generating compensation amounts includes:

[0041] By using hand-eye calibration, the mapping relationship between the camera coordinate system and the tool coordinate system of the attachment mechanism is obtained, and the spatial position and angle of the product and label are transformed into the tool coordinate system.

[0042] The spatial position and angle of the converted product and label are calculated to obtain the position difference and angle difference;

[0043] The position difference and angle difference are subjected to low-pass filtering, and a preset threshold is set;

[0044] No compensation is performed when the difference is less than or equal to the threshold.

[0045] When the difference exceeds the threshold, the corresponding compensation amount is output.

[0046] Furthermore, the step of generating the attachment trajectory command based on the compensation amount includes:

[0047] Based on the compensation amount, a trajectory point series is generated in the tool coordinate system of the attachment mechanism, and the trajectory is smoothed by the S-curve velocity planning method to obtain a smooth trajectory point series.

[0048] The smooth trajectory point series generates instructions according to the motion control interface format supported by the attachment mechanism and sends them to the motion control unit to drive the attachment mechanism to perform the attachment operation.

[0049] After the affixing is completed, the inspection camera captures the image after affixing, and performs edge integrity analysis and positional deviation calculation on the image to determine whether it is qualified or not.

[0050] When a condition is deemed unqualified, a diversion control command is generated.

[0051] Furthermore, the queuing theory-based time-lapse optimization method includes:

[0052] The time interval between products entering the conveying process is used as the arrival process parameter, and the average value within a certain time window is calculated to determine the arrival rate.

[0053] Visual recognition, attachment execution, and defective product sorting are treated as service processes, and the service time is recorded separately. The service time of the visual recognition process is adjusted according to the recognition confidence level. When the confidence level is lower than a preset threshold, the service time is increased.

[0054] Based on the arrival rate and each service time, calculate the queue utilization rate, average queue length and average waiting time, and simultaneously establish a normal product queue and a defective product queue.

[0055] During the queuing modeling process, when the proportion of defective products increases or the identification confidence level is continuously lower than the threshold, the scheduling priority of the defective product queue is increased.

[0056] Based on the calculated queue utilization, average waiting time, and queue priority, the conveying speed and camera triggering timing are dynamically adjusted.

[0057] The beneficial effects of this invention are:

[0058] This invention introduces edge detection and connected component filtering during the construction of the contour point set, and adds direction weighting processing to the shape context algorithm, so that the product and label can still obtain stable spatial position and angle information under complex shape or slight rotation conditions, which significantly improves the accuracy and robustness of attachment and positioning.

[0059] This invention improves the Hungarian algorithm in the point-pair matching process by using a hierarchical pruning method to remove point pairs that do not meet geometric constraints when constructing the cost matrix. This reduces the computational scale, improves matching efficiency, and can meet the real-time requirements of the bonding process.

[0060] This invention abstracts the conveying, positioning, attaching, and diversion processes into a queuing theory model, and introduces a recognition confidence correction and defective product queue scheduling mechanism in the calculation process. This enables the system to dynamically adjust the conveying speed and camera triggering timing according to real-time working conditions, thereby effectively optimizing the production cycle and balancing attaching quality and production efficiency. Attached Figure Description

[0061] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0062] Figure 1 This is a structural diagram of a computer vision-based precision bonding process optimization system proposed in this invention.

[0063] Figure 2 This is a flowchart of a computer vision-based precision bonding process optimization system proposed in this invention. Detailed Implementation

[0064] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0065] refer to Figure 1-2 A computer vision-based precision bonding process optimization system includes the following modules:

[0066] The image acquisition module is used to acquire product and label images; the product positioning camera is positioned above the conveyor belt, the label positioning camera is positioned downstream of the label attaching mechanism, and the detection camera is positioned at the front of the diversion mechanism. Each camera is connected to the data processing unit via an Ethernet interface and achieves synchronous acquisition triggered by a photoelectric sensor.

[0067] The contour construction module is used to perform edge extraction on product images and label images to construct a contour point set. The edge extraction uses the Canny operator, and the contour filtering is achieved through connected component analysis. Only the contours with the largest area or the boundary length that meets the threshold are retained, and the contours are resampled at a set interval to obtain a uniformly distributed contour point set.

[0068] The pose recognition module generates descriptive subsets based on a shape context algorithm and matches them using the Hungarian algorithm to obtain the spatial position and angle of the product and label. The generation of the descriptive subsets involves counting the distribution of neighborhood points in log-polar coordinates and introducing a direction weighting factor to enhance the sensitivity to local geometric orientation. Before running the Hungarian algorithm, a two-level pruning strategy is adopted: first, coarse screening is performed based on distance and angle thresholds, and then fine screening is performed based on local geometric consistency. After constructing a sparse cost matrix, the top label update and feasible matching steps of the Hungarian algorithm are executed, and finally the point pair relationship is output.

[0069] The coordinate mapping and compensation module is used to map the spatial position and angle of the product and label to the tool coordinate system of the attachment mechanism, compare and calculate the position and angle difference, and perform filtering and thresholding on the difference to generate the compensation amount; the relationship between the camera coordinate system and the tool coordinate system is obtained through hand-eye calibration, and the position difference and angle difference are smoothed by low-pass filtering and compared with the set threshold. When the difference exceeds the threshold, the corresponding compensation amount is output.

[0070] The trajectory generation and motion control module is used to generate an attachment trajectory command based on the compensation amount and send the attachment trajectory command to the motion control unit to drive the attachment mechanism to perform the attachment operation. The trajectory generation adopts an S-curve velocity planning method for smoothing, specifically a seven-segment polynomial interpolation or cubic spline function interpolation, to ensure the continuity of velocity, acceleration and jerk. The generated trajectory point series is then encapsulated into an instruction format that conforms to the motion control interface.

[0071] The quality inspection and diversion module is used to acquire post-application images after the application is completed, perform edge integrity analysis and position deviation calculation, and generate diversion control commands when the image is deemed unqualified. The edge integrity analysis is achieved through contour closure detection, the position deviation calculation is completed by comparison with the preset qualified area, and the diversion control commands are output to the actuator by the programmable logic controller.

[0072] The cycle time optimization module abstracts the waiting time and service time of products in the conveying, positioning, attaching, and diversion stages into queuing system parameters. It uses a cycle time optimization method based on queuing theory models to calculate queue length, average waiting time, and optimal service rate, and adjusts the conveying speed and camera triggering timing. The product arrival interval is regarded as a Poisson distribution input, and the service time is regarded as an exponential distribution parameter. An M / M / c queue model is established to calculate the utilization rate and average waiting time. The calculation incorporates identification confidence correction and defective product queue priority scheduling. Based on the results, the conveying speed and camera triggering timing are dynamically adjusted.

[0073] In this embodiment, the modules are interconnected using the following method:

[0074] The system acquires product and label images; the product positioning camera is positioned above the conveyor belt, the label positioning camera is positioned downstream of the affixing mechanism, and the detection camera is positioned at the front of the sorting mechanism. Each camera is connected to the data processing unit via an Ethernet interface and acquires images synchronously triggered by photoelectric sensor signals to ensure the integrity of image data related to the product, label, and post-application state.

[0075] Edge extraction is performed on product images and label images to construct a set of contour points. The edge extraction uses the Canny edge detection operator, and the contour filtering is achieved through connected component analysis. Only the main contour with the largest area or a boundary length exceeding a threshold is retained, and the contour is resampled at a set interval to obtain a uniformly distributed set of contour points.

[0076] A descriptive subset is generated based on a shape context algorithm, and its spatial position and angle of the product and label are obtained after matching with template data. The generation of the descriptive subset involves counting the distribution of neighborhood points in logarithmic polar coordinates, and introducing a direction weighting factor during the counting process. Neighboring points that are consistent with or orthogonal to the tangent direction receive the highest weight, while the weight of other neighborhood points is gradually reduced according to the degree of angular deviation. The Hungarian algorithm is used in the matching process, and a two-level pruning strategy is used to coarsely screen and finely screen candidate point pairs before running. Finally, the top label update and feasible matching steps of the Hungarian algorithm are executed on the sparse cost matrix to output the optimal point pair relationship. Based on the point pair relationship, the spatial position and angle of the product and label are calculated through rigid body registration.

[0077] The spatial position and angle of the product and label are mapped to the coordinate system of the attachment mechanism tool. The difference between the position and angle is calculated and then filtered and thresholded to obtain the compensation amount. The transformation relationship between the camera coordinate system and the tool coordinate system is obtained through hand-eye calibration. The position difference and angle difference are smoothed by low-pass filtering and compared with the set threshold. When the difference exceeds the threshold, the corresponding compensation amount is output.

[0078] The attachment trajectory command is generated based on the compensation amount and sent to the motion control unit to drive the attachment mechanism to perform the attachment operation. The trajectory generation generates a series of trajectory points in the tool coordinate system of the attachment mechanism based on the compensation amount, and smooths them using an S-curve velocity planning method, specifically a seven-segment polynomial interpolation, to ensure the continuity of velocity, acceleration, and jerk. The smoothed trajectory point series is then encapsulated into an instruction format that conforms to the motion control interface and sent to the servo driver and actuator.

[0079] The system acquires images after application and performs quality assessment. When an application is deemed unqualified, a diversion command is generated to control the discharge of defective products. The images are acquired by a detection camera, edge integrity analysis is performed through contour closure detection, and positional deviation is calculated by comparing with a preset qualified area. When an application is deemed unqualified, the diversion mechanism is controlled to remove the defective product.

[0080] The waiting time and service time of products in the conveying, positioning, attaching and diversion stages are abstracted as queuing system parameters. The queuing theory-based takt optimization method is used to calculate the queue length, average waiting time and optimal service rate, and adjust the conveying speed and camera triggering timing. The product arrival interval is regarded as a Poisson distribution input and the service time is regarded as an exponential distribution parameter. An M / M / c queue model is established to calculate the utilization rate and average waiting time. The identification confidence correction and defective product queue priority scheduling are introduced into the calculation. The conveying speed and camera triggering timing are dynamically adjusted according to the results.

[0081] In this embodiment, the step of constructing the contour point set includes:

[0082] The acquired product and label images are converted to grayscale, transforming the color images into single-channel grayscale images to remove interference from color information. Subsequently, histogram equalization is performed to enhance contrast, making the grayscale gradient changes in the edge areas more obvious.

[0083] The enhanced image is processed using the Canny edge detection operator. The Canny operator sequentially performs Gaussian filtering for noise reduction, gradient direction and magnitude calculation, non-maximum suppression, and double threshold connection steps to obtain continuous and refined edge contour lines.

[0084] Edge results are filtered using connected component analysis to remove pseudo-edge regions with an area smaller than a threshold, retaining only the main contour with the largest area and a boundary length greater than the threshold; the threshold is preset to a fixed value based on the product size and camera resolution.

[0085] The main contour curve is divided into a fixed number of sampling points according to the contour perimeter, forming a uniformly distributed contour point set, which is then used as input data for subsequent shape context descriptor generation and pose recognition calculation.

[0086] In this embodiment, the steps for determining the spatial position and angle include:

[0087] Reference points are selected sequentially from the set of contour points, and a logarithmic polar coordinate system is established in the neighborhood of each reference point. The neighborhood is divided into 12 angular intervals and 5 radial intervals to form a fixed two-dimensional grid.

[0088] The distribution of neighborhood points in each interval is used to form a local histogram. A direction weighting factor is introduced during the statistical process, so that neighborhood points that are consistent with or orthogonal to the local tangent direction of the reference point are given the highest weight in the histogram statistics, and the weight of neighborhood points that deviate from the preset angle threshold is reduced step by step according to the degree of angle deviation.

[0089] In the process of forming a local histogram of neighborhood points, a direction weighting factor is introduced, and the following fixed steps and thresholds are followed: 1) Calculate the angle α between the local tangent direction of the reference point and the direction from the reference point to the neighborhood point (taking the minimum angle between 0° and 180°); 2) Calculate the deviation δ from this angle to the nearest 0°, 90°, and 180°, where δ = min(|α−0°|,|α−90°|,|α−180°|); 3) Assign hierarchical weights based on δ and use them for histogram counting accumulation:

[0090] δ≤10°, weight 1.00;

[0091] 10° < δ ≤ 25°, weight 0.75;

[0092] 25° < δ ≤ 40°, weight 0.50;

[0093] δ>40°, weight 0.25.

[0094] In statistical analysis, the direction weighting factor directly replaces the "+1" count and is accumulated in the corresponding log-polar coordinate sector histogram bin; the threshold groups (10°, 25°, 40°) and weight groups (1.00, 0.75, 0.50, 0.25) are fixed values ​​in this implementation. This simultaneously strengthens the contribution of directions consistent with the tangent (0° / 180°) and orthogonal (90°), while gradually reducing the weight of other directions according to a clear hierarchical rule.

[0095] The histogram descriptor subsets generated from all reference points are normalized to eliminate the effects of differences in the number of points and scale, resulting in a descriptor subset with uniform dimensions.

[0096] The normalized histogram descriptor subset is matched point-by-point with the histogram descriptor subset of the template image to construct a cost matrix. The point-pair relationship is determined by the Hungarian algorithm improved by two-level pruning to ensure that the matching result meets the geometric consistency constraint.

[0097] Based on the obtained point-to-point relationships, rigid body registration is performed using the ICP algorithm. The rotation matrix and translation vector are calculated by iterating the nearest point, and the spatial position and angle of the product and label in the coordinate system of the attachment mechanism tool are determined accordingly, providing accurate pose data for compensation and trajectory generation.

[0098] In this embodiment, the step of determining the point pair relationship using the Hungarian algorithm includes:

[0099] An initial cost matrix is ​​constructed based on the descriptive subsets of the product outline point set and the label outline point set. The elements of the cost matrix represent the matching cost between point pairs, which is used to characterize the matching cost of the point pair.

[0100] Perform a two-stage pruning process before generating the cost matrix:

[0101] In the first stage, candidate point pairs are coarsely screened according to preset distance and angle thresholds, and point pairs with position differences greater than the set pixel threshold or angle differences greater than the set angle threshold are removed, leaving only point pairs that meet the spatial relationship constraints.

[0102] The second stage involves fine screening based on local geometric consistency rules in the coarse screening results. These rules include: maintaining the relative polar angle order of adjacent sampling points and ensuring that the rate of change of the boundary normal vector does not exceed a set limit, thereby ensuring that the retained point pairs satisfy local geometric continuity.

[0103] In the candidate point pair set obtained after two-stage pruning, a sparse cost matrix is ​​constructed, which contains only the cost value corresponding to the candidate point pair that satisfies the pruning conditions.

[0104] The Hungarian algorithm is run on the sparse cost matrix, sequentially performing the steps of finding feasible top labels, constructing feasible matches, iteratively updating labels and expanding matches, until all point pairs are assigned and the globally optimal matching relationship is obtained;

[0105] Finally, the optimal point pairing relationship between the product point set and the label point set is output as input for subsequent rigid body registration calculation.

[0106] In this embodiment, the step of mapping the spatial position and angle of the product and label to the coordinate system of the attachment mechanism tool and generating compensation amounts includes:

[0107] The mapping relationship between the camera coordinate system and the tool coordinate system of the attachment mechanism is obtained through hand-eye calibration. During the calibration process, a calibration plate with a checkerboard pattern is placed on the attachment station. After the camera acquires the calibration image, the orientation of the calibration plate in the camera coordinate system is calculated. At the same time, the orientation of the end of the attachment mechanism in the tool coordinate system is recorded. The fixed transformation matrix between the camera coordinate system and the tool coordinate system is obtained based on the AX=XB model.

[0108] The spatial positions and angles of the identified products and labels are transformed from the camera coordinate system to the tool coordinate system of the attachment mechanism using this transformation matrix;

[0109] The difference between the converted product pose and the label pose is calculated to obtain the three-dimensional position difference and the angle difference around the three axes.

[0110] The positional and angular differences were processed using first-order low-pass filtering to eliminate instantaneous fluctuations caused by camera shake or changes in lighting. Based on the filtered results, fixed thresholds were set: a positional difference threshold of 0.2 mm and an angular difference threshold of 0.3 degrees.

[0111] When the filtered difference is less than or equal to the above threshold, the system determines that the attachment accuracy is within the allowable range and does not generate a compensation amount; when the filtered difference exceeds the threshold, the system outputs the corresponding compensation amount and uses the compensation amount as the input parameter of the trajectory generation and motion control module.

[0112] In this embodiment, the compensation amount is calculated as follows:

[0113] The actual spatial position and angle of the product and label are obtained in the coordinate system of the attachment mechanism tool, and compared with the preset target attachment offset to obtain the translation difference along the X, Y and Z directions and the rotation angle difference around the X, Y and Z axes.

[0114] The difference is processed by a first-order low-pass filter to suppress instantaneous jitter and compared with a fixed threshold. The translation threshold is set to 0.20 mm and the rotation threshold is set to 0.30 degrees. The difference within the threshold range is set to zero.

[0115] The difference outside the threshold is corrected by a fixed gain, where the gain for X, Y and around Z is 1.00, the gain for Z direction is 0.80, and the gain around X and around Y is 0.70. A saturation limit is applied to ensure that a single compensation does not exceed the set upper limit.

[0116] The processed compensation values ​​are applied according to nodes during the trajectory planning process. The compensation for X, Y and Z directions is mainly applied during the alignment and fine-tuning stages before pressing. The Z direction compensation is applied during the pressing stage, and the compensation for X and Y directions is applied during the alignment stage.

[0117] The compensation values ​​in the six directions are quantized according to the controller resolution and aligned with the timestamps of the trajectory points. After safety verification, they are combined into a six-dimensional compensation vector, which serves as the input to the trajectory generation and motion control module.

[0118] In this embodiment, the step of generating the attachment trajectory command based on the compensation amount includes:

[0119] In the coordinate system of the attachment mechanism tool, the compensation amount is superimposed on the original attachment trajectory reference point to generate a complete trajectory point series.

[0120] The trajectory point sequence is smoothed using an S-curve velocity planning method. The planning method employs a seven-segment polynomial interpolation method to ensure that the velocity, acceleration, and jerk of the trajectory remain continuous in the acceleration, constant speed, and deceleration segments, thereby obtaining a smooth trajectory point sequence.

[0121] The trajectory point series is encapsulated into an instruction format that conforms to the motion control interface of the attachment mechanism and sent to the motion control unit via industrial Ethernet to drive the attachment mechanism to perform the attachment operation according to the trajectory.

[0122] After the application is completed, the camera captures the image after application, and the edge integrity is analyzed by contour closure detection. Then, the actual application area is compared with the preset qualified area to calculate the deviation and determine whether the application result is qualified.

[0123] When the result is unqualified, the system generates a diversion control command and outputs it to the drive unit of the diversion mechanism via PLC, thereby realizing the automatic rejection of defective products.

[0124] In this embodiment, the queuing theory-based rhythm optimization method includes:

[0125] The time interval between products entering the conveying stage on the production line is statistically analyzed. The average value is calculated using a sliding time window of fixed length, and this average value is used as a parameter for the arrival process.

[0126] Visual recognition, attachment execution, and defective product sorting are considered as three independent service processes, and their service times are recorded separately. The service time of the visual recognition process is adjusted according to the recognition confidence level. When the recognition confidence level is lower than a set threshold, the service time of this process is increased by a fixed compensation value.

[0127] Based on the arrival rate and service time, an M / M / c queue model is established to calculate the queue utilization rate, average queue length, and average waiting time. At the same time, normal product queues and defective product queues are established.

[0128] During the modeling process, when the proportion of defective products exceeds the set threshold, or when the identification confidence level is lower than the set threshold for several consecutive periods, the system increases the scheduling priority of the defective product queue, so that the defective product queue receives priority service.

[0129] Based on the calculation results of queue utilization, average waiting time, and queue priority, the system dynamically adjusts the conveyor belt speed and camera triggering timing, thereby optimizing the production cycle while ensuring the bonding quality.

[0130] In this embodiment, the system dynamically adjusts the conveyor belt speed and camera triggering timing according to the queue utilization rate, average waiting time, and queue priority output by the queuing theory model, following these rules:

[0131] Conveyor belt speed adjustment:

[0132] When the queue utilization rate is greater than 0.85 or the average waiting time exceeds 2 seconds, the system determines that the production line is approaching overload, and the control unit reduces the conveyor belt speed by 10% to reduce the arrival rate.

[0133] When the queue utilization rate is less than 0.50 and the average waiting time is less than 0.5 seconds, the system determines that the production line is idle, and the control unit increases the conveyor belt speed by 10% to increase production capacity.

[0134] The adjustment range of the conveyor belt speed shall not exceed 15% of the rated speed each time, and shall remain stable within 3 statistical periods.

[0135] Camera trigger timing adjustment:

[0136] When the priority of the defective product queue is increased, the camera trigger timing is moved forward by 20 milliseconds to ensure that the recognition images can be processed in advance, allowing time for diversion.

[0137] When the normal queue is under low load, the camera trigger timing is shifted back by 20 milliseconds to reduce invalid image acquisition.

[0138] The camera triggers adjustments within 50 milliseconds to ensure that image sharpness and exposure stability are not affected.

[0139] Integrated Dispatch:

[0140] When allocating cycle time between the normal product queue and the defective product queue, if the defective product ratio exceeds 10%, the system prioritizes the processing speed of the defective product queue and appropriately sacrifices the cycle time of the normal queue; if the defective product ratio is below this threshold, the normal queue is given priority again.

[0141] Example 1:

[0142] To verify the feasibility of this invention in practice, it was applied to an automatic labeling production line for food bottles, and a control group was set up for comparison.

[0143] The production line is set to operate at a speed of 120 bottles per minute, the attaching mechanism is a dual-station attaching head, and the conveyor line is 8 meters long.

[0144] The experimental group of this invention: implemented according to the method and system of this invention, including shape context orientation weighting, improved Hungarian pruning algorithm, six-dimensional compensation calculation, S-curve trajectory embedding and queuing theory beat optimization.

[0145] Control group: The existing conventional labeling system was used. Image recognition was based on standard template matching, trajectory planning adopted trapezoidal velocity planning, and the delivery and triggering strategies were fixed.

[0146] Image acquisition: Both groups used industrial cameras with a resolution of 2048×2048 to acquire images of the products and labels. The experimental group used photoelectric sensors to trigger synchronous acquisition, while the control group used fixed-period acquisition.

[0147] Contour and Matching: The experimental group used Canny edge detection and connected component analysis to construct a contour point set, and then performed direction-weighted shape context descriptor matching with the improved Hungarian algorithm; the control group used standard grayscale template matching.

[0148] Pose calculation and compensation: The experimental group calculated the spatial pose and generated a six-dimensional compensation value through ICP registration, while the control group had no compensation function.

[0149] Trajectory generation: The experimental group used S-curve trajectory smoothing with embedded compensation, while the control group used traditional trapezoidal velocity planning.

[0150] Quality inspection: Both groups underwent post-attachment image acquisition and edge integrity analysis using inspection cameras.

[0151] Cyclic timing optimization: The experimental group used the M / M / c queue model for cyclic timing scheduling, while the control group kept the conveying speed and camera triggering timing fixed.

[0152] Data collection cycle: 8 hours of continuous operation, 500 samples per hour, for a total of 4,000 data points collected.

[0153] Image acquisition: Both groups used industrial cameras with a resolution of 2048×2048 to acquire images of the products and labels. The experimental group used photoelectric sensors to trigger synchronous acquisition, while the control group used fixed-period acquisition.

[0154] Contour and Matching: The experimental group used Canny edge detection and connected component analysis to construct a contour point set, and then performed direction-weighted shape context descriptor matching with the improved Hungarian algorithm; the control group used standard grayscale template matching.

[0155] Pose calculation and compensation: The experimental group calculated the spatial pose and generated a six-dimensional compensation value through ICP registration, while the control group had no compensation function.

[0156] Trajectory generation: The experimental group used S-curve trajectory smoothing with embedded compensation, while the control group used traditional trapezoidal velocity planning.

[0157] Quality inspection: Both groups underwent post-attachment image acquisition and edge integrity analysis using inspection cameras.

[0158] Cyclic timing optimization: The experimental group used the M / M / c queue model for cyclic timing scheduling, while the control group kept the conveying speed and camera triggering timing fixed.

[0159] Data collection cycle: 8 hours of continuous operation, 500 samples per hour, for a total of 4,000 data points collected.

[0160] Table 1 Experimental Results

[0161] Test metrics Control group results Results of this invention Average positioning error (mm) 0.65 0.28 Average angular error (°) 1.2 0.4 Maximum positioning error (mm) 1.5 0.6 Maximum angular error (°) 2.5 0.9 Adhesion failure rate (%) 3.8 0.9 Average waiting time (s) 1.8 0.7 Peak wait time (s) 4.5 1.6 Queue utilization 0.88 0.76 Delivery speed fluctuation rate (%) 0 ±10 Production cycle time (bottles / minute) 115 120

[0162] As shown in Table 1, the system of this invention significantly outperforms the control group in terms of positioning accuracy, angle stability, labeling defect rate, average waiting time, and cycle time maintenance. The control group, due to its fixed cycle time and template matching, experienced positioning deviations when the bottle rotated slightly or the label deformed, resulting in a defect rate of 3.8%. In contrast, this invention reduces the probability of mismatched point pairs through direction-weighted shape context and a pruned Hungarian algorithm; it avoids trajectory jitter caused by instantaneous corrections by embedding compensation amounts into the S-curve trajectory; and it effectively balances recognition time and cycle time load by dynamically adjusting the conveying speed and camera triggering timing using a queuing theory model, maintaining the production capacity at the theoretical maximum of 120 bottles / minute and reducing the average defect rate to 0.9%.

[0163] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A computer vision-based precision bonding process optimization system, characterized in that, Includes the following modules: The image acquisition module is used to acquire product images and label images; The contour construction module is used to perform edge extraction on product images and label images and construct a set of contour points; The pose recognition module is used to generate a descriptor subset based on the shape context algorithm, and introduces an orientation weighting factor in the statistical process. It outputs the spatial position and angle of the product and label through two-level pruning and the Hungarian algorithm. The coordinate mapping and compensation module is used to map the spatial position and angle of the product and label to the coordinate system of the attachment mechanism tool, compare and calculate the difference between the position and angle, and perform filtering and thresholding on the difference to generate the compensation amount. The trajectory generation and motion control module is used to generate an attachment trajectory command based on the compensation amount, and send the attachment trajectory command to the motion control unit to drive the attachment mechanism to perform the attachment operation; The quality inspection and diversion module is used to acquire post-application images after the application is completed, perform edge integrity analysis and position deviation calculation, and generate diversion control commands when the application is deemed unqualified. The takt time optimization module is used to abstract the waiting time and service time of products in the conveying, positioning, attaching and diverting stages into queuing system parameters, and to use the takt time optimization method based on queuing theory model to calculate queue length, average waiting time and optimal service rate, and adjust conveying speed and camera triggering timing.

2. The precision attachment process optimization system based on computer vision according to claim 1, characterized in that, The modules are connected in the following way: Perform edge extraction on the product image and label image to construct a set of contour points; A descriptive subset is generated based on a shape context algorithm, and the spatial position and angle of the product and label are obtained after matching it with template data. The spatial position and angle of the product and label are mapped to the coordinate system of the attachment mechanism tool. The difference between the position and angle is calculated by comparison, and the difference is filtered and thresholded to obtain the compensation amount. The application trajectory command is generated based on the compensation amount and sent to the motion control unit to drive the application mechanism to perform the application operation, collect the post-application image, determine the application quality, and generate a diversion command to control the discharge of defective products when they are not qualified. The waiting time and service time of products in the process of conveying, positioning, attaching and sorting are abstracted into queuing system parameters; The queue length, average waiting time, and optimal service rate are calculated using a queuing theory-based takt optimization method, and the delivery speed and camera triggering timing are adjusted accordingly.

3. The precision attachment process optimization system based on computer vision according to claim 2, characterized in that, The steps for constructing the contour point set include: The acquired images are processed to grayscale and contrast enhancement. The Canny edge detection operator is used to extract boundary contour lines, and the main contour with the largest area or the boundary length that meets the threshold is retained through connected component analysis. The contour curve is resampled at set intervals to form a contour point set.

4. The computer vision-based precision bonding process optimization system according to claim 2, characterized in that, The steps for determining the spatial position and angle include: Reference points are selected sequentially from the set of contour points, and a logarithmic polar coordinate system is established in the neighborhood, dividing the neighborhood into several angular intervals and radial intervals; The distribution of neighborhood points in each interval is counted to form a local histogram, and a direction weighting factor is introduced in the statistical process; Neighboring points that are aligned with or orthogonal to the local tangent direction of the contour point have the highest set weight in the descriptor statistics, while neighboring points that deviate from the preset angle threshold have progressively lower weights. Normalize the histogram descriptor subsets for all reference points; The normalized histogram descriptor subset is matched with the histogram descriptor subset of the template image, and the point pair relationship is determined by the Hungarian algorithm. Based on point-to-point relationships, the spatial position and angle of the product and label are calculated through rigid body registration.

5. The precision attachment process optimization system based on computer vision according to claim 4, characterized in that, The steps for determining the point-pair relationship using the Hungarian algorithm include: An initial cost matrix is ​​constructed based on a descriptive subset of the product outline point set and the label outline point set, wherein the elements of the cost matrix represent the matching cost between point pairs. Perform a two-stage pruning process before generating the cost matrix: In the first stage, candidate point pairs are coarsely screened according to preset distance and angle thresholds, and only point pairs that meet the spatial relationship constraints are retained. The second stage involves fine screening based on local geometric consistency rules in the coarse screening results. These geometric consistency rules include maintaining the relative polar angle order of adjacent sampling points and ensuring that the rate of change of the boundary normal does not exceed a set limit. A sparse cost matrix is ​​constructed from the candidate point pair set obtained after two-stage pruning. The Hungarian algorithm is run on the sparse cost matrix, sequentially performing the steps of finding feasible top labels, constructing feasible matches, iteratively updating labels and expanding matches, until all point pairs are assigned. Output the optimal pair relationship between the product point set and the label point set.

6. The computer vision-based precision bonding process optimization system according to claim 2, characterized in that, The step of mapping the spatial position and angle of the product and label to the coordinate system of the attachment mechanism tool and generating compensation amounts includes: By using hand-eye calibration, the mapping relationship between the camera coordinate system and the tool coordinate system of the attachment mechanism is obtained, and the spatial position and angle of the product and label are transformed into the tool coordinate system. The spatial position and angle of the converted product and label are calculated to obtain the position difference and angle difference; The position difference and angle difference are subjected to low-pass filtering, and a preset threshold is set; No compensation is performed when the difference is less than or equal to the threshold. When the difference exceeds the threshold, the corresponding compensation amount is output.

7. The precision attachment process optimization system based on computer vision according to claim 2, characterized in that, The step of generating the attachment trajectory command based on the compensation amount includes: Based on the compensation amount, a trajectory point series is generated in the tool coordinate system of the attachment mechanism, and the trajectory is smoothed by the S-curve velocity planning method to obtain a smooth trajectory point series. The smooth trajectory point series generates instructions according to the motion control interface format supported by the attachment mechanism and sends them to the motion control unit to drive the attachment mechanism to perform the attachment operation. After the affixing is completed, the inspection camera captures the image after affixing, and performs edge integrity analysis and positional deviation calculation on the image to determine whether it is qualified or not. When a condition is deemed unqualified, a diversion control command is generated.

8. The precision attachment process optimization system based on computer vision according to claim 2, characterized in that, The queuing theory-based time optimization method includes: The time interval between products entering the conveying process is used as the arrival process parameter, and the average value within a certain time window is calculated to determine the arrival rate. Visual recognition, attachment execution, and defective product sorting are treated as service processes, and the service time is recorded separately. The service time of the visual recognition process is adjusted according to the recognition confidence level. When the confidence level is lower than a preset threshold, the service time is increased. Based on the arrival rate and each service time, calculate the queue utilization rate, average queue length and average waiting time, and simultaneously establish a normal product queue and a defective product queue. During the queuing modeling process, when the proportion of defective products increases or the identification confidence level is continuously lower than the threshold, the scheduling priority of the defective product queue is increased. Based on the calculated queue utilization, average waiting time, and queue priority, adjust the conveying speed and camera triggering timing.

Images