A high-precision pillow bagging method and system based on visual recognition

Abstract

The present invention relates to the field of intelligent manufacturing technology, and discloses a high-precision pillow core bagging method and system based on visual recognition. The high-precision pillow core bagging method based on visual recognition includes: extracting a small number of highly representative feature points through a KSVD algorithm and an optimal sparse representation solution; calculating weights through a graph attention network and realizing consistency constraints by using an energy minimization method; identifying causal dependencies and retaining high-information edge connections through a PC algorithm; generating a bagging path based on a dynamic RRT algorithm and realizing force feedback execution by using impedance control, establishing a multi-objective quality evaluation system, and automatically adjusting system parameters through Bayesian optimization to perform feedback optimization; the present invention extracts a small number of highly representative feature points through a KSVD algorithm and a LARS algorithm, combines causal critical sampling with path planning and execution control, and reduces computational complexity.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and more specifically, to a high-precision pillow bagging method and system based on visual recognition. Background Art

[0002] In modern household goods production lines, pillow core bagging is a critical process, and its automation level directly impacts product quality and production efficiency. However, due to the flexibility, diverse shapes, and diverse materials of pillow cores, traditional bagging automation equipment faces the following technical challenges:

[0003] First, the irregular shape of pillow cores and their easy deformation make it difficult for traditional rigid object recognition algorithms to accurately locate them. Second, the high-speed operation of the production line requires millisecond-level response speeds for the algorithms, while existing feature extraction algorithms such as SIFT and ORB have high computational complexity and large redundancy. Third, traditional methods process each feature point independently, and overall positioning is prone to failure when some features are blocked. Finally, the computational efficiency of the fully connected graph structure is low, making it difficult to adapt to high-speed production environments.

[0004] Currently, the market lacks an intelligent system that can accurately locate and bag flexible objects in a high-speed production environment. There is an urgent need for an efficient, accurate and robust pillow core visual recognition and bagging technology solution. Summary of the Invention

[0005] The present invention provides a high-precision pillow core bagging method and system based on visual recognition, which solves the technical problems of accurate positioning and bagging of flexible objects in a high-speed production environment in related technologies.

[0006] The present invention provides a high-precision pillow bagging method based on visual recognition, comprising:

[0007] Extract a small number of highly representative feature points through the KSVD algorithm and optimal sparse representation solution;

[0008] Build a graph structure representation based on feature points, calculate weights through the graph attention network, and use energy minimization to implement consistency constraints;

[0009] According to the graph structure representation, PC algorithm is used to identify causal dependencies and retain high-information edge connections, optimizing feature points and topological relationships;

[0010] According to the optimized feature points and topological relationships, a bagging path is generated based on a dynamic RRT algorithm and impedance control is used to implement force feedback execution. The steps of generating a bagging path based on a dynamic RRT algorithm and implementing force feedback execution by impedance control include:

[0011] Based on the optimal path planning of dynamic RRT, a 7-dimensional configuration space is constructed and the sampling strategy is dynamically adjusted during the tree expansion process;

[0012] Impedance-controlled force feedback is implemented, and the compliance of the robot arm at different stages of bagging is controlled by adjusting the stiffness and damping parameters. An abnormality detection mechanism is also set to automatically suspend execution in abnormal situations. The mathematical model of impedance control is as follows:

[0013] ;

[0014] in and are the actual position and the expected position, and are the actual speed and the expected speed, and are the actual acceleration and the expected acceleration, is an external force, 、 and are the mass, damping and stiffness matrices respectively;

[0015] A multi-objective quality evaluation system is established for feature points, graph structure representation, optimized feature points, topological relationships, and bagging paths, and system parameters are automatically adjusted through Bayesian optimization to perform feedback optimization. The steps of establishing the multi-objective quality evaluation system and automatically adjusting system parameters through Bayesian optimization include:

[0016] Bagging quality assessment based on visual feedback, using a convolutional neural network structure to evaluate position accuracy, angle consistency, wrinkle degree, time efficiency, and smoothness;

[0017] Multi-objective Bayesian optimization tuning uses Gaussian process surrogate models to automatically adjust system parameters;

[0018] Adaptive memory and continuous learning extract and store successful bagging experience, reuse historical experience through similarity calculation and continuously optimize system performance.

[0019] Furthermore, the step of extracting a small number of highly representative feature points by using the KSVD algorithm and the optimal sparse representation solution includes:

[0020] Feature point detection and description, constructing feature atom dictionary and extracting potential feature points through KSVD algorithm;

[0021] Optimal sparse representation solution: Use the LARS algorithm to solve the optimal sparse representation of the input pillow core image;

[0022] Geometric consistency feature point screening, using the improved RANSAC variant algorithm to screen key feature points with geometric consistency;

[0023] Sub-pixel precise positioning, using gradient information and quadratic interpolation method to achieve sub-pixel feature point positioning.

[0024] Furthermore, the step of calculating weights through the graph attention network and implementing consistency constraints using the energy minimization method includes:

[0025] Graph structure representation construction, representing the feature point set as a graph structure;

[0026] Graph attention network weight calculation, using graph attention network to calculate the importance weights between nodes;

[0027] Message passing and feature integration, integrating the information of adjacent feature points through an iterative message passing mechanism;

[0028] Consistency constraint optimization uses energy minimization method to achieve structural consistency constraints of feature point sets.

[0029] Furthermore, the steps of identifying causal dependencies and retaining high-information edge connections by the PC algorithm and optimizing feature points and topological relationships include:

[0030] Causal dependency identification, using PC algorithm to identify the true causal dependency between feature points;

[0031] Importance scoring function construction, designing an edge importance scoring function that comprehensively considers mutual information, spatial distance and geometric consistency;

[0032] Critical edge sampling, retaining TopK high-information edge connections based on importance scores;

[0033] Adaptive sparsity adjustment dynamically adjusts graph sparsity based on actual application scenario requirements and computing resource constraints.

[0034] Furthermore, the adaptive sparsity adjustment dynamically adjusts the sampling threshold by constructing a resource precision balance function, and the balance function is:

[0035] ;

[0036] in Indicated in the sparse factor The equilibrium function value under Indicated in the sparsity factor The recognition accuracy metric under represents the corresponding computational cost metric, and are the accuracy weight and cost weight coefficients respectively.

[0037] Furthermore, the bagging quality assessment establishes a 5-dimensional quality evaluation vector, and the calculation formula is:

[0038] ;

[0039] in represents the quality evaluation vector, represents the position accuracy score, represents the angle consistency score, Indicates the wrinkle score, represents the time efficiency score, Represents the smoothness score.

[0040] Furthermore, the adaptive memory and continuous learning achieve continuous optimization of system performance through experience pattern extraction and storage, experience similarity calculation, experience migration and reuse, and incremental learning and optimization. When the number of experiences exceeds a threshold, experiences with low usage frequency and low quality scores are removed.

[0041] A high-precision pillow bagging system based on visual recognition, used to execute the above-mentioned high-precision pillow bagging method based on visual recognition, comprising:

[0042] A visual acquisition unit, used to obtain pillow core image information and output it to a feature processing module;

[0043] The feature extraction processing module is used to extract a small number of highly representative feature points from the pillow core image and achieve sub-pixel positioning of the feature points through sparse representation and geometric consistency screening;

[0044] The structural relationship analysis module is used to construct the topological structure between feature points and identify key connection relationships through causal modeling, thereby ensuring recognition accuracy while reducing computational complexity;

[0045] The trajectory planning and execution module is used to generate the optimal bagging path based on the feature point analysis results and achieve smooth execution through adaptive force control strategy;

[0046] The quality monitoring and optimization module is used to evaluate bagging quality and dynamically adjust system parameters based on a multi-objective optimization algorithm to achieve continuous performance optimization.

[0047] The beneficial effects of the present invention are: extracting a small number of highly representative feature points through the KSVD algorithm and the LARS algorithm, combining causal critical sampling with path planning and execution control, reducing computational complexity, and rapidly completing feature extraction and processing, meeting the real-time processing requirements of ultra-high-speed production lines;

[0048] Sub-pixel positioning technology using gradient information and quadratic interpolation, combined with structural consistency constraints for energy minimization, provides a stable and accurate spatial reference for high-precision bagging.

[0049] Utilizing graph structure representation and structural constraint recovery technology, the system can accurately restore the complete structure through topological relationships even when some feature points are obscured or deformed, significantly improving recognition success rates compared to traditional methods and effectively reducing production interruptions.

[0050] By dynamically adjusting sparsity parameters and fusing multi-resolution features, the system can adaptively adjust processing parameters based on different pillow core types, production environments, and computing resource constraints, achieving the optimal accuracy-efficiency balance under various conditions.

[0051] By using causal critical sampling to sparsely connect graphs and combining it with a gated message passing mechanism, the system's memory usage and computational load are reduced, enabling the entire algorithm to run smoothly on common industrial computing platforms without the need for dedicated GPU acceleration, significantly reducing deployment and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a high-precision pillow bagging method based on visual recognition in the present invention;

[0053] Figure 2 is a flow chart of step 1 in the present invention;

[0054] Figure 3 It is a flow chart of step 2 in the present invention;

[0055] Figure 4 It is a flow chart of step 3 in the present invention;

[0056] Figure 5 It is a flow chart of step 4 in the present invention;

[0057] Figure 6 It is a flow chart of step 5 in the present invention. DETAILED DESCRIPTION

[0058] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0059] At least one embodiment of the present invention discloses a high-precision pillow bagging method based on visual recognition, such as Figures 1 to 6 As shown, the following steps are included:

[0060] Step 1: Extract a small number of highly representative feature points using the KSVD algorithm and optimal sparse representation solution;

[0061] This step aims to extract a small number of highly representative key feature points from the pillow core image to reduce computational complexity and increase processing speed. It specifically includes the following sub-steps:

[0062] Step 1.1, feature point detection and description;

[0063] This application uses the KSVD algorithm to process a large number of pillow core sample images and construct a feature atom dictionary D. The specific process is as follows:

[0064] Input image area information (Include image regions, each with pixels), initialize an overcomplete dictionary ,in is the number of dictionary atoms (usually set to ).

[0065] Fixed overcomplete dictionary , the sparse representation coefficients are obtained by solving the following optimization problem :

[0066] ;

[0067] in Minimize , Representation matrix The square of the Frobenius norm, Represents a vector The number of nonzero elements in , Represents a sparse coefficient matrix No. Column (corresponding to sparse representation of image regions), is the preset sparse constraint parameter (controls the maximum number of dictionary atoms), is the number of pillow core image areas, is the number of pixels in the pillow core image area, Indicates constraints.

[0068] This step is implemented by the Orthogonal Matching Pursuit (OMP) algorithm, which iteratively greedily selects the dictionary atoms most relevant to the current residual, ensuring that the reconstruction error is minimized while satisfying the sparsity constraint.

[0069] Update the overcomplete dictionary column by column :For the List , calculate the representation error matrix:

[0070] ;

[0071] in represents the error matrix, Indicates pillow core image area information, Overcomplete dictionary No. List, Represents a sparse coefficient matrix No. OK, Indicates that All indexes except Perform the summation, Represents a transpose operation.

[0072] Error matrix Apply Singular Value Decomposition (SVD):

[0073] ;

[0074] in Represents the error matrix SVD decomposition of and are the left singular vector matrix and the right singular vector matrix obtained by SVD decomposition, is a diagonal matrix of singular values, express The first diagonal element of the matrix is ​​then updated for and update the first column of for Multiply The first column of Represents a transpose operation.

[0075] Repeated fixed overcomplete dictionary Until convergence or the maximum number of iterations is reached .

[0076] In the pillow bagging application scenario, the KSVD algorithm of this application is suitable for extracting key feature points including pillow edge contour, material texture and suture line. By configuring appropriate parameters, such as the number of dictionary atoms obtained by training different batches of pillow samples Set to 256, sparse constraint parameter Set to 8, the maximum number of iterations is 50. The algorithm can effectively cope with changes in visual features caused by lighting changes, surface wrinkles, etc. in the production environment.

[0077] Step 1.2, optimal sparse representation solution;

[0078] Based on the constructed feature atom dictionary, this application uses the Least Angle Regression (LARS) algorithm to solve the optimal sparse representation of the input pillow core image. The specific process is as follows:

[0079] For the input pillow core image I, first extract the potential feature point set:

[0080] ;

[0081] in 、 、 Respectively represent the first, second, and potential feature points, is the total number of potential feature points extracted;

[0082] And its corresponding local descriptor:

[0083] ;

[0084] in 、 、 Respectively represent the first, second, and The local descriptor of feature points, is the total number of potential feature points extracted;

[0085] Then, for each descriptor , solve the following optimization problem:

[0086] ;

[0087] in for The L1 norm of (the sum of the absolute values ​​of all elements), represents the square of the Euclidean distance, It is an overcomplete dictionary. is the sparse coefficient vector to be solved, is the reconstruction error threshold, Represents a minimize operation.

[0088] The LARS algorithm solves the above problem by gradually increasing the set of active variables to ensure the sparsity of the solution.

[0089] Through this sub-step, each feature point is represented as a sparse linear combination of the dictionary D, and a sparse coefficient matrix is ​​obtained:

[0090] ;

[0091] in represents a sparse coefficient matrix, 、 、 Respectively represent the first, second, and The sparse coefficient vector of feature points, is the total number of potential feature points extracted.

[0092] Step 1.3, screening of geometrically consistent feature points;

[0093] This application uses an improved variant of the Random Sample Consensus (RANSAC) algorithm to filter out key feature points with geometric consistency from the sparse representation results. The specific process is as follows:

[0094] First, the significance score of each feature point is calculated based on the sparse coefficient matrix A:

[0095] ;

[0096] in Indicates the The significance score of feature points, yes The L1 norm of yes the number of nonzero elements in ;

[0097] Then, a weighted RANSAC variant algorithm is used to iteratively perform the following steps:

[0098] Randomly select 4 feature points with high significance scores as the initial set;

[0099] Calculate the geometric transformation model defined by these four points (such as affine transformation);

[0100] Count the number of other feature points that are consistent with the geometric transformation model;

[0101] The model with the largest number of consistent points is retained.

[0102] Therefore, 8 to 15 feature points that are consistent with the best model are finally selected as the key feature point set:

[0103] ;

[0104] in represents a set of key feature points, 、 、 Respectively represent the first, second, and Key feature points, The number of key feature points selected is usually 8 to 15.

[0105] Step 1.4: Sub-pixel precise positioning;

[0106] For the selected key feature point set F, this application uses gradient information and quadratic interpolation method to achieve sub-pixel accurate positioning. The specific process is as follows:

[0107] For each feature point, a two-dimensional quadratic function is constructed in its local neighborhood:

[0108] ;

[0109] in Indicated in coordinates The quadratic function value at 、 、 、 、 、 The quadratic terms are , quadratic term , cross terms , one-time item , one-time item The coefficients of the constant term are obtained by fitting the gradient values ​​of the surrounding pixels using the least squares method; and are the horizontal and vertical coordinates of the pixel, Indicates a constant offset for a function.

[0110] In addition, the calculation function The extreme point position of :

[0111] ;

[0112] ;

[0113] in and Represents functions respectively right and The partial derivative of and are the horizontal and vertical coordinates of the pixel respectively;

[0114] Solve the above equations to obtain the feature point coordinates with sub-pixel accuracy.

[0115] Through this step, the positioning accuracy of feature points is improved to the sub-pixel level (approximately ±0.3 pixels), providing a more accurate position reference for subsequent high-precision bagging.

[0116] It can be seen that the output of this step is a set of key feature points with precise positioning:

[0117] ;

[0118] in Represents a set of key feature points for precise positioning, 、 、 Respectively represent the first, second, and The exact coordinates of the feature points, The number of key feature points selected is usually 8-15.

[0119] Step 2: Build a graph structure representation based on feature points, calculate weights through the graph attention network, and implement consistency constraints using energy minimization methods;

[0120] This step aims to construct the topological structure between feature points, capture the spatial relationship between points, and improve the robustness of feature representation. It specifically includes the following sub-steps:

[0121] Step 2.1, graph structure representation construction;

[0122] This application represents the feature point set as a graph structure:

[0123] ;

[0124] in Represents the constructed graph structure, Represents the vertex set, corresponding to the feature points, Represents an edge set and encodes the geometric relationship between feature points.

[0125] The specific implementation process is as follows:

[0126] First, the feature point set obtained in step 1 As a vertex set of a graph:

[0127] ;

[0128] in represents the vertex set of the graph, 、 、 Respectively represent the first, second, and vertices, vertices Corresponding feature points , is the number of key feature points selected;

[0129] Then, the connection relationship is constructed based on the geometric distance and descriptor similarity, that is, the edge set .

[0130] For any two points and , calculate the edge weights between them:

[0131] ;

[0132] in Representing feature points and feature points The Euclidean distance between Representation descriptor and descriptors The difference of and are the scaling parameters for distance and features, represents the natural exponential function, Representing feature points and feature points The edge weights between .

[0133] It should be understood that the final graph structure It contains the spatial distribution information of feature points and the similarity of local features, providing a topological representation basis for subsequent processing.

[0134] Step 2.2, graph attention network weight calculation;

[0135] Use graph attention network to calculate the importance weights between nodes , to capture the relative importance of feature points. The specific implementation process is as follows:

[0136] For the graph Each edge in , calculate the attention coefficient:

[0137] ;

[0138] ;

[0139] in and They are feature points and are feature points The eigenvector of is the shared linear transformation matrix, is the attention vector, represents the feature connection operation, Representing feature points The set of adjacent feature points, is the rectified linear unit activation function with a negative slope, Represents a vector The transpose of is a feature point and The unnormalized attention score of is the normalized attention weight, represents the natural exponential function, represents the transpose operation, Representing feature points Its neighboring feature points The index value of the unnormalized attention score between Indicates the feature points, Represents the summation symbol.

[0140] The specific structure of the graph attention network includes: input layer, feature conversion layer, attention calculation layer and output layer. The input layer receives the descriptor and position information of the feature points, and the feature dimension is ; The feature conversion layer passes the weight matrix Mapping input features to dimensional space; the attention calculation layer uses the attention vector Calculate the attention coefficient between node pairs; the output layer fuses neighborhood information according to the attention weight to generate updated node feature representation.

[0141] In the pillow bagging application scenario, the graph attention network proposed in this application can be configured as a three-layer structure. The first layer maps the feature dimension from 128 to 64, the second layer maintains the dimension, and the third layer maps to 32-dimensional output features. The negative slope parameter of the LeakyReLU is set to 0.2, and batch normalization is added after each layer to accelerate convergence and improve stability. To address the differences in feature importance of different parts of the pillow, regional adaptive weights can be introduced into the attention mechanism to make the network pay more attention to the relationship between feature points in key bagging areas (such as the opening end and corners).

[0142] Through the attention mechanism, the system can automatically learn the importance of connections between different feature points, allowing the network to focus on more informative connections.

[0143] Step 2.3, message delivery and feature integration;

[0144] This application uses an iterative message passing mechanism to integrate the information of adjacent feature points and enhance the representation capability of nodes. The specific implementation process is as follows:

[0145] For an L-layer graph attention network, in layer l, each node updates its features in the following way:

[0146] ;

[0147] in Represents the lth layer node Features, represents the attention coefficient calculated at layer l, is the weight matrix of layer l, represents a nonlinear activation function, Representation node The set of neighbor nodes of represents the updated node features, Represents the summation symbol.

[0148] It should be noted that through multi-layer message passing, each node not only contains its own information, but also integrates the contextual information of its topological neighborhood, thereby obtaining a richer representation.

[0149] Step 2.4, consistency constraint optimization;

[0150] This application uses the energy minimization method to implement structural consistency constraints on feature point sets and improve recognition stability. The specific implementation process is as follows:

[0151] Construct the energy function:

[0152] ;

[0153] in represents the total energy function, represents the location set of feature points, Represents data items, measurement points consistency with observed data; Represents structural items, measurement points and Structural constraints between It is a trade-off factor that controls the relative importance of data items and structural items; Indicates the sum of all feature points; represents the sum over all edges.

[0154] ;

[0155] in represents the location set of feature points, 、 、 Respectively represent the first, second, and The location of the feature points, is the total number of feature points.

[0156] In addition, the above energy function is minimized through iterative optimization algorithms (such as gradient descent), so that the feature point set can meet the structural constraints while maintaining consistency with the observed data, thereby improving the robustness of the overall recognition.

[0157] Therefore, the output of this step is a representation of feature points with topological relationships:

[0158] ;

[0159] in is the representation of feature points with topological relationships, is a node set, is an edge set, It is the final layer node feature set, which contains the spatial layout and relationship information of feature points;

[0160] ;

[0161] in represents the feature set of the final layer nodes, 、 、 Respectively represent the first, second, and The node in The feature representation of the layer, is the total number of feature points, is the total number of layers in the graph attention network.

[0162] Step 3: Based on the graph structure representation, the PC algorithm is used to identify causal dependencies and retain high-information edge connections, optimizing feature points and topological relationships.

[0163] This step aims to identify and retain the most informative connections in the graph structure, remove redundant edges, and improve computational efficiency. It specifically includes the following sub-steps:

[0164] Step 3.1, identification of causal dependencies;

[0165] This application uses the PC (Peter-Clark) algorithm to identify the true causal dependencies between feature points and eliminate non-causal connections. The specific implementation process is as follows:

[0166] First, the initial graph structure is assumed to be a fully connected graph; then, the significance level (usually set to 0.05) iteratively execute the PC algorithm:

[0167] For each pair of variables , test their conditional independence ,in is a subset of variables, Indicates that under a given set of conditions In this case, the variable and Conditional independence;

[0168] If the independence condition is met, remove and The edge between

[0169] Gradually increase the condition set Repeat the above process;

[0170] The mathematical representation of the PC algorithm is:

[0171] ;

[0172] in For the dataset, is the significance level.

[0173] It should be noted that the PC algorithm can remove statistically relevant but non-causal connections between feature points and retain connections with causal relationships, making the graph structure more concise and meaningful.

[0174] Step 3.2, importance scoring function construction;

[0175] Design an edge importance scoring function that comprehensively considers mutual information, spatial distance, and geometric consistency to provide guidance for subsequent edge sampling. The specific implementation process is as follows:

[0176] For the graph Each edge in , calculate its importance score:

[0177] ;

[0178] in Representing feature points and feature points The mutual information between them measures their information correlation; Represents the spatial distance between feature points; represents a geometric consistency measure that evaluates the structural stability between two points; 、 and Represent the weight coefficients of mutual information, spatial distance and geometric consistency, respectively, to balance the importance of different factors; Represents an edge Importance rating; represents the scoring function; represents the natural exponential function.

[0179] Mutual Information The calculation is as follows:

[0180] ;

[0181] in Representing feature points and feature points The mutual information between them measures their information correlation, and They are feature points and feature points The characteristic variable space of is the joint probability distribution, and is the marginal probability distribution, represents the summation symbol, Indicates the logarithmic sign.

[0182] Step 3.3, critical edge sampling;

[0183] Based on importance scores, we retain the topK high-information edge connections to achieve graph sparseness. The specific implementation process is as follows:

[0184] Computational Graph The importance scores of all edges in ;

[0185] Sort all edges by rating from high to low;

[0186] Before selection The edges with the highest scores are used as key connections to form a sparse edge set ;

[0187] Dynamically determined based on the threshold adjustment mechanism value:

[0188] ;

[0189] in represents the number of edges retained, and are the upper and lower bounds of the number of retained edges, is the original edge set size, is the sparsity factor, usually set to 0.1-0.3, represents the minimum function, represents the maximum value function, Represents an edge Importance rating.

[0190] In addition, the graph structure after sparsification is expressed as:

[0191] ;

[0192] in represents the graph structure after sparsification, represents the original vertex set, represents the edge set after sparsification;

[0193] The number of edges is significantly reduced, while the most informative connections are retained.

[0194] Step 3.4, adaptive sparsity adjustment;

[0195] Based on the actual application scenario requirements and computing resource constraints, the graph sparsity is dynamically adjusted to balance accuracy and efficiency. The specific implementation process is as follows:

[0196] Construct a resource-precision balance function:

[0197] ;

[0198] in Indicated in the sparse factor The equilibrium function value under Indicated in the sparse factor The recognition accuracy metric under represents the corresponding computational cost metric, and are the accuracy weight and cost weight coefficients respectively.

[0199] It can be seen that by evaluating different The balance function value under the value, choose Maximize the optimal sparsity factor :

[0200] ;

[0201] in represents the optimal sparsity factor, Indicates that the function The corresponding maximum value value.

[0202] based on Dynamically adjust the sampling threshold so that the system can achieve the best efficiency-accuracy balance under different working conditions.

[0203] Therefore, the output of this step is a sparse graph structure after causal critical sampling:

[0204] ;

[0205] in represents the sparse graph structure after causal critical sampling, represents the original vertex set, represents the sparse edge set, Represents a vertex feature set.

[0206] Step 4: Based on the optimized feature points and topological relationships, a bagging path is generated based on the dynamic RRT algorithm and impedance control is used to implement force feedback execution;

[0207] This step generates the optimal bagging path based on the obtained pillow core posture and three-dimensional morphology information, and controls the robotic arm to perform the bagging action.

[0208] Step 4.1, optimal path planning based on dynamic RRT*;

[0209] This application uses an improved dynamic RRT* (Rapidly-exploring Random Trees Star) algorithm for path planning, which can effectively handle possible slight deformation and position changes of the pillow core. The specific implementation steps are as follows:

[0210] Configuration space construction, treating the end effector of the robotic arm as a mass point, and establishing a 7-dimensional configuration space:

[0211] ;

[0212] in Represents the configuration space, Indicates location, Express gesture, additional The dimension indicates how open or closed the bag opening is.

[0213] Initialize the random tree with the current configuration of the robot arm As the root node, a random tree structure is constructed in the configuration space:

[0214] ;

[0215] in represents a random tree, is a collection of nodes, is the edge set.

[0216] Set the target area Cover the area around the end of the pillow to be bagged.

[0217] To expand the tree, iterate the following steps until the maximum number of iterations (10,000 in this example) is reached or a path that meets the conditions is found:

[0218] By probability (0.3 in this example) directly samples the target area, otherwise randomly samples in the configuration space ;

[0219] In the tree Find the distance Nearest Node ;

[0220] from towards Expand in a fixed step (0.1 in this example) to get a new node ;

[0221] If connected and There is no collision on the path, then Surrounding search radius The set of all nodes in ;

[0222] from Choose the value from the starting point to The node with the minimum path cost As The parent node of

[0223] for Each node in If you pass arrive If the path cost is less than the original path cost and there is no collision, reconnect The parent node is .

[0224] In this embodiment, the path cost function is defined as:

[0225] ;

[0226] in 、 and are distance metrics representing the position, posture, and gripper opening and closing states, respectively. It is the degree of similarity to the pillow surface. 、 、 、 are the weight coefficients of position distance, posture distance, gripper state distance and approximation degree (in this embodiment, they are 0.4, 0.3, 0.1, and 0.2 respectively). Indicates that from the configuration To configuration Path cost.

[0227] Dynamic adjustment: During path execution, whenever the pillow core posture change is detected to exceed a threshold (5mm or 3 degrees in this example), the following operations are performed:

[0228] Preserve the parts of the existing tree structure that are still valid;

[0229] The update starting point is the current configuration;

[0230] The tree is expanded again until a new feasible path is found.

[0231] Path smoothing and time parameterization.

[0232] The found path is smoothed by applying cubic spline interpolation method, and then the optimal time allocation is calculated by a two-stage time optimal parameterization method while satisfying the constraints of the robot arm joint velocity and acceleration.

[0233] In response to the special requirements of pillow cover bag applications, this embodiment makes the following improvements to the standard RRT* algorithm:

[0234] A pillow core deformation prediction model is added to enable path planning to adapt to slight pillow core deformation;

[0235] An adaptive sampling strategy was designed to increase sampling density during key stages of bagging (e.g., approaching the pillow opening, aligning with the opening, and passing through the opening);

[0236] An online path adjustment mechanism based on visual feedback is introduced to respond to state changes during the bagging process in real time.

[0237] Step 4.2, force feedback execution based on impedance control;

[0238] To ensure smooth and safe bagging, this application uses an impedance control method to control the robotic arm's execution. The core concept is to model the robotic arm as a spring-damper system. By adjusting the stiffness and damping parameters, the robotic arm can appropriately yield when encountering unexpected resistance, avoiding damage to the pillow core or bagging tool.

[0239] The mathematical model of impedance control is as follows:

[0240] ;

[0241] in and are the actual position and the expected position, and are the actual speed and the expected speed, and are the actual acceleration and the expected acceleration, is an external force, 、 and are the mass, damping and stiffness matrices respectively.

[0242] At different stages of the bagging process, this embodiment dynamically adjusts the impedance parameters:

[0243] Approaching stage:

[0244] High stiffness ( ), to ensure precise positioning;

[0245] Bagging mouth alignment stage:

[0246] Medium stiffness ( ), small adjustments are allowed;

[0247] Insertion phase:

[0248] Low stiffness ( ), provide sufficient flexibility;

[0249] Bagging propulsion phase: Gradually increase stiffness to ensure stable propulsion.

[0250] in Represents a diagonal matrix, where the elements on the diagonal correspond to the stiffness coefficients in the three translation directions of X-axis, Y-axis, and Z-axis and the three rotation directions around X-axis, Y-axis, and Z-axis.

[0251] This embodiment also implements an anomaly detection mechanism based on force sensors. When the detected force or torque exceeds a preset threshold (20N and 2Nm, respectively, in this embodiment), the system automatically pauses execution, retraces to a safe position, and then replans the path.

[0252] Step 5: Based on the feature points, graph structure representation, optimized feature points, topological relationships, and bagging paths, a multi-objective quality evaluation system is established and the system parameters are automatically adjusted through Bayesian optimization for feedback optimization.

[0253] Conduct confidence assessment on the recovery results and adjust system parameters based on the assessment results. The specific implementation process is as follows:

[0254] Step 5.1, bagging quality assessment based on visual feedback;

[0255] This application designs a multi-objective quality evaluation system to evaluate the quality of the bagging process in real time. The specific implementation steps are as follows:

[0256] Define the bagging quality evaluation index system, including the following key indicators:

[0257] Position accuracy ( ): Deviation between bagging position and target position;

[0258] Angular consistency ( ): The degree of consistency between the bagging angle and the main axis of the pillow core;

[0259] Wrinkle degree ( ): the number and severity of wrinkles produced during bagging;

[0260] Time efficiency ( ): Time consumption to complete bagging;

[0261] Smoothness ( ): The smoothness of the robot arm's movement during bagging.

[0262] A visual quality inspection model is established, and a convolutional neural network structure is used to evaluate the bagging results. The network structure is as follows:

[0263] Input layer: receives the visual image after bagging (640×480×3);

[0264] Feature extraction layer: contains 5 convolution blocks, each of which consists of 2 3×3 convolution layers and a maximum pooling layer;

[0265] Feature fusion layer: fuses the extracted features with the force sensing data during the bagging process;

[0266] Output layer: Generate 5-dimensional quality evaluation vector:

[0267] ;

[0268] in represents the quality evaluation vector, represents the position accuracy score, represents the angle consistency score, Indicates the wrinkle score, represents the time efficiency score, Represents the smoothness score.

[0269] Calculate the comprehensive quality score and sum the weighted indicators to get the comprehensive quality score:

[0270] ;

[0271] in is the weight coefficient of each indicator (in this embodiment, they are 0.3, 0.25, 0.2, 0.15, and 0.1 respectively), Indicates the quality indicators, represents the comprehensive quality score, Indicates the sum of 5 indicators.

[0272] A quality assessment feedback mechanism is established. Based on the quality assessment results, the system provides real-time feedback on the bagging quality and triggers the tuning process when the quality score is lower than the threshold (0.75 in this embodiment).

[0273] Step 5.2, multi-objective Bayesian optimization tuning;

[0274] To improve system performance, this application uses a multi-objective Bayesian optimization algorithm to automatically adjust system parameters. The specific implementation steps are as follows:

[0275] Define the parameter optimization space. In this embodiment, the following key parameters are selected for optimization:

[0276] Feature extraction threshold ( ): The threshold parameter that controls the number of feature points;

[0277] Topology relationship construction parameters ( ): Control the complexity of the topology structure;

[0278] Path planning weight coefficient ( to ): Control the weights of path planning;

[0279] Impedance control parameters ( to ): Controls the impedance characteristics of the robotic arm during execution.

[0280] Construct a Gaussian process proxy model and build a mapping model from parameters to quality indicators to reduce the number of actual experiments:

[0281] ;

[0282] in Represents the parameter vector The corresponding quality index function is, is the mean function, is the kernel function (this example uses the Matérn5 / 2 kernel function), and Represents the current parameters and historical parameters respectively, represents a Gaussian process, Indicates that it obeys the distribution.

[0283] Iterate the optimization process, performing the following iterative steps:

[0284] Update the Gaussian process model based on the current observation data;

[0285] Calculate the acquisition function value (this embodiment uses the expected improvement EI as the acquisition function):

[0286] ;

[0287] in is the currently known optimal parameter. Representation parameters The expected improvement value of Indicates the expected value, represents the maximum value function, It is the best quality indicator currently known. is a parameter Corresponding quality indicators.

[0288] Select the next set of parameters to evaluate:

[0289] ;

[0290] in represents the next set of parameters to be evaluated, Indicates The corresponding parameter when the maximum value is obtained .

[0291] Perform bagging with the new parameters and evaluate quality metrics.

[0292] New observations are added to the dataset and the iterations are repeated until the termination condition is reached.

[0293] Parameter update and application: select one of the Pareto optimal solutions as the system parameters, and give priority to the parameter combination with the highest comprehensive quality score:

[0294] ;

[0295] in is the set of Pareto optimal solutions, represents the optimal parameter combination, Represents the Pareto optimal solution set Chinese envoy Get the maximum value of the parameter , Representation parameters The corresponding comprehensive quality score.

[0296] Step 5.3, adaptive memory and continuous learning;

[0297] This application designs an adaptive memory mechanism to store successful bagging experiences and continuously optimize system performance. The specific implementation steps are as follows:

[0298] Empirical pattern extraction and storage. For each successful bagging operation, its key features are extracted and stored:

[0299] ;

[0300] in is feature information, is the control parameter, is the execution path, is the quality score, Indicates the An experience sample.

[0301] Experience similarity calculation: When encountering a new bagging task, calculate the similarity with historical experience:

[0302] ;

[0303] in and They are feature similarity function and environmental condition similarity function, and Represent the weight coefficients of feature similarity and environmental condition similarity, Indicates new task experience and historical experience The similarity between and Represent the feature information of the new task and the feature information of historical experience respectively, and They represent the environmental condition parameters of the new task and the environmental condition parameters in historical experience respectively.

[0304] Experience migration and reuse: select the historical experience with the highest similarity for parameter migration:

[0305] ;

[0306] ;

[0307] in Indicates the initialization path parameters, Indicates the The path parameters of the empirical samples, Indicates the index of the experience sample with the highest similarity, Indicates the index corresponding to the maximum value of the function , Indicates new task experience and historical experience The similarity between them.

[0308] Incremental learning and optimization, continuously updating the experience library based on execution results:

[0309] Successful cases: directly added to the experience library;

[0310] Failure cases: Analyze the reasons for failure and update the restrictions of relevant experience items;

[0311] Experience library pruning: When the number of experiences exceeds a threshold, experiences with low usage frequency and low quality scores are removed.

[0312] A high-precision pillow bagging system based on visual recognition, used to execute the above-mentioned high-precision pillow bagging method based on visual recognition, comprising:

[0313] A visual acquisition unit, used to obtain pillow core image information and output it to a feature processing module;

[0314] The feature extraction processing module is used to extract a small number of highly representative feature points from the pillow core image and achieve sub-pixel positioning of the feature points through sparse representation and geometric consistency screening;

[0315] The structural relationship analysis module is used to construct the topological structure between feature points and identify key connection relationships through causal modeling, thereby ensuring recognition accuracy while reducing computational complexity;

[0316] The trajectory planning and execution module is used to generate the optimal bagging path based on the feature point analysis results and achieve smooth execution through adaptive force control strategy;

[0317] Quality monitoring and optimization module, used to evaluate bagging quality and dynamically adjust system parameters based on multi-objective optimization algorithms to achieve continuous performance optimization

[0318] Here, the present invention provides an implementation example: taking the pillow core production line of a company in a household goods manufacturing enterprise as an example.

[0319] A company specializing in the production of household goods produces over 2 million sets of high-end pillow cores annually. The company's existing pillow core bagging process relied primarily on manual labor, which presented the following issues:

[0320] Low efficiency: On average, each worker can only complete 50 to 60 pillow bagging operations per hour;

[0321] High labor costs: The bagging workshop requires a large number of workers to work in three shifts;

[0322] Unstable quality: Manual operation is prone to wrinkles, deviations and other quality problems;

[0323] Workers' occupational health issues: Long-term repetitive operations lead to a high incidence of occupational diseases such as wrist and shoulder problems among workers.

[0324] In order to solve the above problems, a company decided to introduce this patented technology to realize the automation transformation of the pillow core bagging process.

[0325] This patented technology is applied to three pillow core production lines of a certain company. Each production line consists of the following components:

[0326] Industrial robot: 6-axis collaborative robot with a load of 6kg and a working radius of 850mm;

[0327] Vision System:

[0328] Two industrial cameras (resolution 1920×1080, frame rate 60fps) are fixedly installed on both sides of the production line;

[0329] A depth camera (resolution 640×480, depth accuracy ±2mm) is installed at the end of the robot;

[0330] Computing platform: Industrial computer equipped with NVIDIA RTX 3080 GPU for visual processing and path planning;

[0331] End effector: Custom pneumatic gripper with force / tactile sensors;

[0332] Conveyor system: Conveyor belt with positioning mechanism to ensure pillow cores and bags reach the working area.

[0333] The system is integrated into the existing production line, covers an area of ​​about 4m², and does not affect the normal operation of other processes.

[0334] According to the specific production needs of a company, the system is configured with the following key parameters:

[0335] Visual recognition parameters:

[0336] Number of feature points: 128 feature points are extracted from each pillow surface;

[0337] Color space: Use HSV color space for image processing;

[0338] Recognition threshold: Confidence greater than 0.85 is considered a valid recognition;

[0339] Path planning parameters:

[0340] Number of path sampling points: 64 key points for each path planning;

[0341] Safety distance threshold: The minimum distance from obstacles should be kept above 50mm;

[0342] Path smoothness: 5th-order spline interpolation is used to ensure smooth robot motion;

[0343] Execution control parameters:

[0344] Force control threshold: The maximum allowable contact force during bagging is 15N;

[0345] Speed ​​configuration: 300mm / s in approach phase, 100mm / s in bagging phase;

[0346] Impedance parameters: The main diagonal elements of the stiffness matrix are [600, 600, 400, 200, 200, 200] N / m.

[0347] A company's application implementation is divided into three stages:

[0348] Phase 1: Single-machine testing (2 weeks);

[0349] Establish a test platform to conduct offline testing of visual recognition algorithms and path planning algorithms;

[0350] Single-machine bagging test was conducted using pillow core samples of 10 different specifications;

[0351] Collect test data and optimize algorithm parameters;

[0352] Phase II, trial operation (1 month):

[0353] Install the system on production line 1 and integrate it with the existing production system;

[0354] The system and manual operations are run in parallel for comparative testing;

[0355] Continuously adjust system parameters through Bayesian optimization algorithm;

[0356] Collect operation data and analyze system performance and stability;

[0357] Phase 3, full deployment (3 months):

[0358] Deploy the optimized system to all three production lines;

[0359] Establish a production data management system to monitor bagging quality in real time;

[0360] Training of operation and maintenance personnel;

[0361] Develop standard operating procedures and emergency response plans.

[0362] After 6 months of operation, the application of this patented technology in a certain company has achieved the following results:

[0363] Bagging speed: The system can complete an average of 220 pillow bagging operations per hour, which is 3.7 times the original manual efficiency.

[0364] Continuous working capability: The system can operate continuously for 24 hours, requiring only a small amount of maintenance time;

[0365] Changeover time: The switching time between pillow cores of different specifications is reduced from 30 minutes to 5 minutes.

[0366] Bagging quality pass rate: increased from 92% in manual operation to 98.7% in automated system;

[0367] Quality consistency: The quality fluctuation between product batches is reduced, and the standard deviation is reduced by 78%;

[0368] Special process compatibility: The system is capable of handling complex products such as special fabrics and special-shaped pillow cores.

[0369] Labor costs: The number of people on each production line has been reduced from 12 to 2, saving approximately RMB 1.2 million in labor costs annually;

[0370] Material saving: Due to the improvement of bagging accuracy, the bagging loss rate has been reduced from 2.8% to 0.5%, saving about RMB 800,000 in material costs annually;

[0371] Energy consumption: Compared with 24-hour manual operation, the energy consumption of the automated system is reduced by 35%;

[0372] Return on investment: System investment is recovered within 18 months;

[0373] Improved working environment: reduced repetitive work for workers and lowered the incidence of occupational diseases;

[0374] Technology accumulation: The company has accumulated technical capabilities in automation, visual recognition, etc. through this project;

[0375] Brand image: Smart manufacturing upgrades have enhanced the company's market competitiveness and brand image.

[0376] During the implementation process of a certain company, this patented technology also made the following innovations and optimizations based on actual production needs:

[0377] Developed a quick change mechanism to achieve fast switching between pillowcases of different specifications, with a changeover time as low as 5 minutes;

[0378] Added a fabric status detection function before bagging, which can automatically detect whether the bag is twisted or deformed;

[0379] An online bagging quality assessment system was developed to detect and classify bagging results in real time;

[0380] A production data management module has been added to achieve seamless integration with the enterprise MES system.

[0381] Successful application at a company has demonstrated that the high-precision pillow bagging method based on visual recognition is not only technically feasible but also has significant economic and social benefits in a real-world production environment. The system has excellent adaptability and scalability, allowing it to be applied to various pillow production lines.

[0382] Through continuous parameter optimization and accumulated experience, there is room for further improvement in system performance. Within the next two years, one company plans to upgrade the system to handle pillows with more complex shapes and further increase the bagging speed to 300 pillows per hour.

[0383] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. A high-precision pillow bagging method based on visual recognition, characterized in that: The following steps are involved: Extract a small number of highly representative feature points through the KSVD algorithm and optimal sparse representation solution; Build a graph structure representation based on feature points, calculate weights through the graph attention network, and use energy minimization to implement consistency constraints; According to the graph structure representation, PC algorithm is used to identify causal dependencies and retain high-information edge connections, optimizing feature points and topological relationships; According to the optimized feature points and topological relationships, a bagging path is generated based on a dynamic RRT algorithm and impedance control is used to implement force feedback execution. The steps of generating a bagging path based on a dynamic RRT algorithm and implementing force feedback execution by impedance control include: Based on the optimal path planning of dynamic RRT, a 7-dimensional configuration space is constructed and the sampling strategy is dynamically adjusted during the tree expansion process; Impedance-controlled force feedback is implemented, and the compliance of the robot arm at different stages of bagging is controlled by adjusting the stiffness and damping parameters. An abnormality detection mechanism is also set to automatically suspend execution in abnormal situations. The mathematical model of impedance control is as follows: ； in and are the actual position and the expected position, and are the actual speed and the expected speed, and are the actual acceleration and the expected acceleration, is an external force, 、 and are the mass, damping and stiffness matrices respectively; A multi-objective quality evaluation system is established for feature points, graph structure representation, optimized feature points, topological relationships, and bagging paths, and system parameters are automatically adjusted through Bayesian optimization to perform feedback optimization. The steps of establishing the multi-objective quality evaluation system and automatically adjusting system parameters through Bayesian optimization include: Bagging quality assessment based on visual feedback, using a convolutional neural network structure to evaluate position accuracy, angle consistency, wrinkle degree, time efficiency, and smoothness; Multi-objective Bayesian optimization tuning uses Gaussian process surrogate models to automatically adjust system parameters; Adaptive memory and continuous learning extract and store successful bagging experience, reuse historical experience through similarity calculation and continuously optimize system performance.

2. The high-precision pillow bagging method based on visual recognition according to claim 1, characterized in that: The step of extracting a small number of highly representative feature points by using the KSVD algorithm and the optimal sparse representation solution includes: Feature point detection and description, constructing feature atom dictionary and extracting potential feature points through KSVD algorithm; Optimal sparse representation solution: Use the LARS algorithm to solve the optimal sparse representation of the input pillow core image; Geometric consistency feature point screening, using the improved RANSAC variant algorithm to screen key feature points with geometric consistency; Sub-pixel precise positioning, using gradient information and quadratic interpolation method to achieve sub-pixel feature point positioning.

3. The high-precision pillow bagging method based on visual recognition according to claim 1, characterized in that: The steps of calculating weights through the graph attention network and implementing consistency constraints using the energy minimization method include: Graph structure representation construction, representing the feature point set as a graph structure; Graph attention network weight calculation, using graph attention network to calculate the importance weights between nodes; Message passing and feature integration, integrating the information of adjacent feature points through an iterative message passing mechanism; Consistency constraint optimization uses energy minimization method to achieve structural consistency constraints of feature point sets.

4. The high-precision pillow bagging method based on visual recognition according to claim 1, characterized in that: The steps of identifying causal dependencies and retaining high-information edge connections by the PC algorithm and optimizing feature points and topological relationships include: Causal dependency identification, using PC algorithm to identify the true causal dependency between feature points; Importance scoring function construction, designing an edge importance scoring function that comprehensively considers mutual information, spatial distance and geometric consistency; Critical edge sampling, retaining TopK high-information edge connections based on importance scores; Adaptive sparsity adjustment dynamically adjusts graph sparsity based on actual application scenario requirements and computing resource constraints.

5. The high-precision pillow bagging method based on visual recognition according to claim 4, characterized in that: The adaptive sparsity adjustment dynamically adjusts the sampling threshold by constructing a resource precision balance function, and the balance function is: ； in Indicated in the sparsity factor The equilibrium function value under Indicated in the sparsity factor The recognition accuracy metric under represents the corresponding computational cost metric, and are the accuracy weight and cost weight coefficients respectively.

6. The high-precision pillow bagging method based on visual recognition according to claim 1, characterized in that: The bagging quality assessment establishes a 5-dimensional quality evaluation vector, and the calculation formula is: ； in represents the quality evaluation vector, represents the position accuracy score, represents the angle consistency score, Indicates the wrinkle score, represents the time efficiency score, Represents the smoothness score.

7. The high-precision pillow bagging method based on visual recognition according to claim 1, characterized in that: The adaptive memory and continuous learning achieve continuous optimization of system performance through experience pattern extraction and storage, experience similarity calculation, experience migration and reuse, and incremental learning and optimization. When the number of experiences exceeds a threshold, experiences with low usage frequency and low quality score are removed.

8. A high-precision pillow bagging system based on visual recognition, characterized in that: A high-precision pillow bagging method based on visual recognition for executing any one of claims 1-7, comprising: A visual acquisition unit, used to obtain pillow core image information and output it to a feature processing module; The feature extraction processing module is used to extract a small number of highly representative feature points from the pillow core image and achieve sub-pixel positioning of the feature points through sparse representation and geometric consistency screening; The structural relationship analysis module is used to construct the topological structure between feature points and identify key connection relationships through causal modeling, thereby ensuring recognition accuracy while reducing computational complexity; The trajectory planning and execution module is used to generate the optimal bagging path based on the feature point analysis results and achieve smooth execution through adaptive force control strategy; The quality monitoring and optimization module is used to evaluate bagging quality and dynamically adjust system parameters based on a multi-objective optimization algorithm to achieve continuous performance optimization.

Images