A deep learning-based bin packing path planning method and system

Abstract

This invention relates to the field of packing path planning technology, and discloses a packing path planning method and system based on deep learning. The method includes: collecting data of objects to be packed to construct three-dimensional coordinate parameters; using a graph convolutional network to perform spatial topology modeling to generate a geometric topology graph and calculate path density coefficients; extracting spatial principal component vectors from the graph Laplacian matrix spectral decomposition of the path density coefficients, and generating path planning weights through a genetic algorithm; dynamically updating baseline path nodes based on the weights to generate optimized paths, and controlling the trajectory of a robotic arm. The system includes modules for three-dimensional data acquisition, topology modeling, density coefficient calculation, spectral decomposition, weight optimization, path generation, and motion control. This invention, through the combination of deep learning and intelligent algorithms, achieves complex geometric feature modeling and path optimization, improving packing efficiency, accuracy, and system adaptability, and is suitable for packing scenarios such as logistics and warehousing.

Description

Technical Field

[0001] This invention relates to the field of packing path planning technology, specifically to a packing path planning method and system based on deep learning. Background Technology

[0002] In logistics, warehousing, and intelligent manufacturing, packing is a critical and high-frequency operation, and the rationality of its path planning directly affects operational efficiency, equipment wear and tear, and cargo safety. Traditional packing path planning methods mainly rely on human experience or classic algorithms, such as genetic algorithms and simulated annealing algorithms. However, these methods have significant limitations when dealing with objects with complex geometries, dynamically changing packing spaces, and multi-objective optimization problems.

[0003] Traditional methods struggle to efficiently capture the geometric features and spatial topological relationships of objects. For objects with irregular shapes and complex surface structures to be packed, manually designed feature extraction rules or simple geometric parameter descriptions often fail to accurately reflect the spatial constraints between objects, leading to problems such as collisions and path redundancy during path planning. For example, when packing multiple irregularly shaped parts, traditional methods may generate unreasonable paths due to their inability to accurately model the adjacency relationships between parts, increasing the robot arm's energy consumption and operation time.

[0004] Traditional algorithms are inefficient when handling high-dimensional data and dynamic optimization. As the complexity of bin-packing scenarios increases, factors such as object size, weight, spatial density, and robotic arm motion parameters also rise, leading to a dramatic increase in data dimensionality. The computational complexity of traditional algorithms grows exponentially, making it difficult to meet real-time requirements. Furthermore, traditional methods typically employ fixed optimization strategies, failing to dynamically adjust path planning weights based on real-time bin-packing status, resulting in poor system adaptability. For example, when the object distribution density changes in a certain area of ​​the bin-packing space, traditional methods struggle to quickly reallocate path planning weights, potentially causing frequent path conflicts or detours for the robotic arm in high-density areas.

[0005] Traditional path planning methods lack sufficient coordination with robotic arm motion control. Path planning results often fail to adequately consider the kinematic and dynamic constraints of the robotic arm, such as joint angle limits and speed limitations. This can lead to generated paths that may not be feasible during actual execution or require extensive post-processing adjustments, impacting packing efficiency. For example, paths generated by traditional methods may require the robotic arm to move beyond its joint angle range, forcing the system to replan the path, increasing operational uncertainty and time costs.

[0006] With the development of deep learning technology, it has demonstrated powerful feature learning and data processing capabilities in fields such as image recognition, natural language processing, and complex system modeling. Applying deep learning to the field of bin packing path planning holds promise for solving the challenges of traditional methods in geometric feature modeling, high-dimensional data processing, and dynamic optimization. However, deep learning-based bin packing path planning technology is still in the exploratory stage. How to combine deep learning models with the specific needs of bin packing path planning to build an efficient, accurate, and robust path planning system remains a pressing technical problem to be solved. Summary of the Invention

[0007] The purpose of this invention is to provide a deep learning-based bin packing path planning method to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a bin packing path planning method based on deep learning, the method comprising:

[0009] Collect the size, weight, and geometric feature data of the objects to be packed, and construct the three-dimensional coordinate parameters of the packing space;

[0010] A graph convolutional network is used to perform spatial topology modeling on geometric feature data, generate a geometric topology graph, and calculate the path density coefficient based on the spatial density variation between nodes in the topology graph.

[0011] The graph Laplacian matrix of the path density coefficients of different packing areas is spectrally decomposed to extract spatial principal component vectors. The principal component vectors are then iteratively optimized using a genetic algorithm to generate path planning weights for each area.

[0012] The baseline path nodes are dynamically updated based on the path planning weights to generate an optimized path, and the movement trajectory of the robotic arm is controlled based on the optimized path.

[0013] Preferably, the spatial topology modeling includes:

[0014] Geometric feature data is converted into a three-dimensional spatial tensor, and dimensional differences are eliminated through normalization.

[0015] Multi-layer graph convolutional layers are used to aggregate neighborhood features of the three-dimensional spatial tensor to generate an adjacency matrix between nodes;

[0016] The adjacency matrix is ​​optimized using a dynamic routing mechanism to output a geometric topology graph.

[0017] Preferably, the path density coefficient calculation includes:

[0018] Select a baseline path node in any packing area and calculate the spatial density distribution of its n neighboring nodes;

[0019] The ratio of the standard deviation of spatial density to the Euclidean distance of the reference node is used as the local density quantity.

[0020] Calculate the path residual between the baseline node and the associated nodes, and take its absolute value as the global density quantity;

[0021] The harmonic mean of the local density and the global density is used as the path density coefficient.

[0022] Preferably, the spatial principal component vector extraction includes:

[0023] Construct a graph Laplacian matrix of path density coefficients for different packing regions, and perform spectral clustering decomposition on it to obtain feature vectors;

[0024] The feature vectors whose cumulative contribution rate exceeds a set threshold are selected to form the principal component subspace;

[0025] Projecting the graph Laplacian matrix onto the principal component subspace yields the dimensionality-reduced spatial principal component vectors.

[0026] Preferably, the path planning weight generation includes:

[0027] The spatial principal component vectors are standardized, and their cosine similarity with the historical best path vectors is calculated.

[0028] The similarity scores are input into the genetic algorithm population, and initial weights are generated through selection, crossover, and mutation operations.

[0029] The initial weights are locally optimized using an adaptive sliding window, and the path planning weights for each region are output.

[0030] Preferably, the generated optimization path includes:

[0031] The node update parameters are obtained by multiplying the path planning weights by tensors with the baseline path nodes.

[0032] The deviation between the node update parameters and the baseline path is calculated, and the deviation is corrected using a dynamic programming algorithm.

[0033] The corrected node sequence is connected into a continuous trajectory to generate an optimized path.

[0034] Preferably, the method further includes:

[0035] A 3D packing trajectory space is constructed based on the optimized path, and the intersection points and conflict areas in the trajectory are extracted.

[0036] When the number of intersections exceeds the set threshold or the area of ​​the conflict zone is larger than the limit, the path is judged to be in an abnormal state, and a trajectory adjustment command is generated.

[0037] Preferably, the construction of the three-dimensional packing trajectory space includes:

[0038] The optimized path is mapped to a three-dimensional coordinate system according to the motion sequence to generate a trajectory point cloud set.

[0039] A point cloud reconstruction algorithm is used to perform three-dimensional voxel modeling of the trajectory point cloud and calculate the surface curvature of the voxel model.

[0040] The curvature distribution is enhanced by using a nonmaximum suppression algorithm to eliminate noise interference.

[0041] Preferably, the method further includes:

[0042] A mixed integer programming model of path planning parameters and robotic arm motion parameters is established, and the branch and bound method is used to solve for the optimal motion strategy.

[0043] Spatially align the optimal motion strategy with the optimized path to generate a path scheme with the highest packing efficiency;

[0044] The solution to the mixed integer programming model includes:

[0045] The objective function is defined as a linear combination of path length and motion energy consumption, and the constraint is the limit of the joint angle of the robotic arm;

[0046] The objective function is relaxed and decomposed into path parameter subproblems and motion parameter subproblems.

[0047] The two subproblems are solved iteratively and alternately until convergence, and the optimal motion strategy that satisfies the constraints is output.

[0048] Preferably, the present invention further includes a deep learning-based bin packing path planning system, the system comprising:

[0049] The 3D data acquisition module is used to acquire the size, weight, and geometric feature data of the object to be packed, and to construct a 3D spatial coordinate matrix;

[0050] The topology modeling module uses graph convolutional networks to aggregate neighborhood features of the three-dimensional spatial coordinate matrix and generates a geometric topology graph through a dynamic routing mechanism.

[0051] The density coefficient calculation module generates local and global density quantities based on the spatial density distribution of nodes in the geometric topology graph, and calculates their harmonic mean as the path density coefficient.

[0052] The spectral decomposition module constructs a graph Laplacian matrix for the path density coefficients of different packing regions and extracts spatial principal component vectors through spectral clustering decomposition.

[0053] The weight optimization module uses a genetic algorithm to iteratively optimize the spatial principal component vectors and generate path planning weights for each region.

[0054] The path generation module dynamically updates the baseline path nodes based on path planning weights, corrects path deviations through dynamic programming algorithms, and generates optimized paths.

[0055] The motion control module converts the optimized path into control commands for the robotic arm, which then drives the actuator to complete the packing operation via a real-time communication protocol.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] In geometric feature modeling, spatial topology modeling of object geometric feature data is performed using graph convolutional networks, effectively capturing the adjacency relationships and topological structure between objects in three-dimensional space. Converting geometric feature data into a three-dimensional spatial tensor and normalizing it eliminates the influence of dimensional differences, ensuring the model can accurately learn the features of objects at different scales. Neighborhood feature aggregation operations in multi-layer graph convolutional layers generate adjacency matrices between nodes, and then a dynamic routing mechanism optimizes the topological connections, thereby constructing a geometric topology graph that accurately reflects the spatial structure. This process overcomes the limitations of traditional methods that rely on manually designed features, enabling the system to automatically learn feature representations of complex geometric shapes. This provides richer and more accurate spatial information for subsequent path planning, effectively reducing path conflicts and collisions caused by inaccurate geometric feature descriptions.

[0058] In path density coefficient calculation and spatial principal component analysis, the path density coefficient is calculated by comprehensively considering both local and global density values, which can fully reflect the spatial density distribution characteristics of the container loading area. Local density is based on the ratio of the standard deviation of spatial density of neighboring nodes to the Euclidean distance, reflecting the details of density changes in local areas. Global density is measured by the absolute value of the path residuals between the baseline node and associated nodes, reflecting the path coherence across the entire scope. The harmonic mean of both is used as the path density coefficient, taking into account both local details and global trends, providing a more reasonable density reference index for path planning. Furthermore, spectral decomposition is performed on the graph Laplacian matrix of the path density coefficient to extract spatial principal component vectors with cumulative contribution rates exceeding a threshold. This achieves dimensionality reduction of high-dimensional data, reducing computational complexity while preserving key feature information and improving the efficiency of path planning.

[0059] In the process of generating weights and optimizing paths during path planning, the application of genetic algorithms enables iterative optimization of the principal component vectors in the spatial model. Through standardization and cosine similarity calculation, the principal component vectors are correlated with historically optimal path vectors, ensuring that the initial weight generation is guided by prior knowledge. The adaptive sliding window local optimization operation allows the weights to be dynamically adjusted based on the real-time packing status, enhancing the system's adaptability to complex scenarios. Based on the path planning weights, the baseline path nodes are dynamically updated, and path deviations are corrected using dynamic programming algorithms. The generated optimized paths can more rationally allocate path planning weights to different regions, reducing path redundancy and improving the robotic arm's motion efficiency. Simultaneously, constructing a 3D packing trajectory space and detecting abnormal path states in real time can promptly identify intersections and conflict areas. When an anomaly is detected, trajectory adjustment commands are generated, effectively avoiding collision risks during robotic arm movement and improving the safety and reliability of the packing operation.

[0060] Regarding the coordination with robotic arm motion control, a mixed integer programming model is established, integrating path planning parameters and robotic arm motion parameters. The objective function is a linear combination of path length and motion energy consumption, with the limits of robotic arm joint angles as constraints. The optimal motion strategy is solved using a branch-and-bound method, and spatial alignment with the optimized path ensures that the generated path is not only geometrically sound but also fully considers the actual motion capabilities of the robotic arm. This approach, combining path planning and motion control, avoids the disconnect between path planning and execution found in traditional methods, enabling the robotic arm to complete packing operations more efficiently and safely, significantly improving packing efficiency and overall system performance. Attached Figure Description

[0061] Figure 1 This is a schematic diagram illustrating the working principle of the deep learning-based packing path planning method described in this invention.

[0062] Figure 2 Workflow diagram for spatial topology modeling;

[0063] Figure 3 To optimize the path generation flowchart;

[0064] Figure 4 This is a flowchart for path anomaly detection and adjustment. Detailed Implementation

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] Please see Figure 1-Figure 4 This invention relates to a bin packing path planning method based on deep learning, the specific implementation steps of which are as follows:

[0067] The system collects dimensional, weight, and geometric feature data of the objects to be packed, constructing three-dimensional coordinate parameters for the packing space. Specifically, it acquires the length, width, and height of the objects using devices such as 3D scanners, obtains weight data using weighing sensors, and simultaneously collects geometric feature data such as surface curvature and edge features. This data is then mapped onto a three-dimensional coordinate system to form three-dimensional coordinate parameters containing information such as the object's position and shape, providing basic spatial information for subsequent path planning.

[0068] A graph convolutional network (GCNN) is used to perform spatial topology modeling on geometric feature data, generating a geometric topology graph. Path density coefficients are then calculated based on the spatial density variations between nodes in the topology graph. Geometric feature data is input into the GCNN, and the network's operations uncover the spatial relationships between nodes, constructing a geometric topology graph that reflects the object's geometric structure. Within this topology graph, the spatial density distribution around each node is analyzed, and path density coefficients are obtained through a specific calculation method. These coefficients characterize the complexity of path planning and the distribution of feasible regions.

[0069] The path density coefficients of different binning regions are transformed into graph Laplacian matrices through spectral decomposition to extract spatial principal component vectors. These principal component vectors are then iteratively optimized using a genetic algorithm to generate path planning weights for each region. Specifically, for different binning regions, the path density coefficients are converted into graph Laplacian matrices, and spectral decomposition is used to extract principal component vectors reflecting the main spatial characteristics. Then, leveraging the global search capability of the genetic algorithm, the principal component vectors are optimized to generate path planning weights suitable for each region, guiding the direction of path optimization.

[0070] The baseline path nodes are dynamically updated based on path planning weights to generate optimized paths, and the movement trajectory of the robotic arm is controlled based on these optimized paths. Using pre-defined baseline path nodes as a foundation, and combining them with the generated path planning weights, the nodes are adjusted and updated to form a better sequence of path nodes. These node sequences are connected into a continuous optimized path, which is then converted into control commands for the robotic arm, driving it to complete the box-packing operation according to the planned trajectory.

[0071] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0072] Example 1:

[0073] Based on the overall scheme described above, the specific implementation of spatial topology modeling is as follows: First, geometric feature data is converted into a three-dimensional spatial tensor, and dimensional differences are eliminated through normalization. Geometric feature data encompasses multi-dimensional information such as surface curvature, edge angles, and vertex coordinates. This data may exist in different dimensions (e.g., length in meters, angle in radians), and direct processing can lead to model training bias. Therefore, geometric feature data needs to be mapped to a three-dimensional spatial tensor. For example, each geometric feature point can be represented as an element in the tensor, with its coordinate values ​​corresponding to the tensor's dimensions. Normalization is performed through linear transformations (e.g., scaling the data to the [0,1] interval) or standardization methods (e.g., subtracting the mean, dividing by the standard deviation) to bring data of different dimensions to the same numerical scale, ensuring that subsequent graph convolutional networks treat all types of features equally.

[0074] Multi-layer graph convolutional layers are used to aggregate neighborhood features of a 3D spatial tensor, generating an adjacency matrix between nodes. Graph Convolutional Networks (GCNs) are an effective tool for processing non-Euclidean data. Their core idea is to update the features of the current node through feature propagation from neighboring nodes. In a 3D spatial tensor, each node represents a feature point in the geometric structure of an object (such as a vertex or curvature extremum). The initial feature vector of a node contains its geometric attributes (such as coordinates and curvature values). Each layer of the multi-layer graph convolutional layers performs the following operations: for each node, it collects the feature vectors of its neighboring nodes (the neighborhood range can be determined by setting a radius or using the k-nearest neighbor algorithm), and aggregates these features through weighted summation (the weights are learned from the convolutional kernel parameters) to generate a new feature representation for the node. After multiple convolutions, the node features not only contain their own geometric attributes but also incorporate neighborhood structural information. The adjacency matrix is ​​generated based on the spatial association between nodes. If the distance between two nodes in 3D space is less than a preset threshold or they belong to the k-nearest neighbor group, the corresponding position in the adjacency matrix is ​​set to 1; otherwise, it is 0. This matrix reflects the topological connections between nodes.

[0075] A dynamic routing mechanism optimizes the topology of the adjacency matrix, outputting a geometric topology graph. This mechanism, inspired by capsule networks, optimizes the connection strength between nodes by iteratively updating routing weights. Specifically, during initialization, each node assigns equal routing weights to its neighboring nodes. Then, for each node, its predicted feature vectors for its neighboring nodes are calculated based on the current routing weights, and similarity matching (e.g., calculating the dot product) is performed between these vectors and the actual feature vectors of the neighboring nodes. The routing weights are updated based on the similarity results; higher similarity results in higher routing weights, indicating a more important connection between the two nodes. After several iterations, the connection weights in the adjacency matrix no longer change significantly, resulting in the optimized topology. The geometric topology graph is presented in graph form, with nodes representing geometric feature points and edges representing optimized adjacency relationships. The edge weights reflect the importance of feature propagation between nodes. This graph provides a structured geometric representation for subsequent path density coefficient calculations.

[0076] In practical implementation, the dimension of the 3D spatial tensor can be set according to the complexity of the object. For example, for simple geometric objects, a third-order tensor composed of 3D coordinates (x, y, z) can be used; for complex objects, dimensions such as curvature and normal vectors can be added to form a higher-order tensor. The choice of normalization method needs to be combined with the data distribution. If the data follows a normal distribution, the normalization method is more suitable; if the data range is known and boundary values ​​need to be preserved, linear normalization is better. The number of layers in the graph convolutional layer is usually 2-4 layers. Too few layers may not be able to fully capture high-order neighborhood features, while too many layers may lead to overfitting. The number of neighborhood nodes k can be set by empirical values ​​(such as k=8 or k=16), or it can be adaptively adjusted according to the curvature of the object surface. Smaller k values ​​are selected for areas with larger curvature (such as corners) to focus on local features, and larger k values ​​are selected for areas with smaller curvature (such as planes) to fuse more neighborhood information. The number of iterations for dynamic routing is generally 3-5 times. Each iteration requires calculating and updating the routing weights of all nodes until the weight change is less than a preset threshold (such as 0.01), at which point the iteration stops.

[0077] Through the above steps, the geometric feature data is transformed from its original scattered state into a structured geometric topology graph. This process not only preserves the spatial geometric information of the objects, but also enhances the semantic association between nodes through neighborhood feature aggregation and topology connection optimization, providing a more accurate spatial topology basis for the calculation of path density coefficients in subsequent path planning.

[0078] Example 2:

[0079] Based on the overall scheme described above, the calculation process for the path density coefficient is as follows: First, a reference path node is selected for any packing area, and the spatial density distribution of its n nearest neighbors is calculated. The reference path node can be selected based on the geometric center of the packing area or the key turning point of the initial path planning to ensure the representativeness of the calculation. The determination of the n nearest neighbors can be achieved through the k-nearest neighbor algorithm in three-dimensional space. That is, for the reference node, the nearest n nodes (n is a preset positive integer, such as n=10 or n=20) are searched in the space formed by the three-dimensional coordinate parameters. The spatial density distribution is calculated with the reference node as the center, and the distribution frequency of the n nearest neighbors in different spatial directions is statistically analyzed. For example, the three-dimensional space is divided into several cubic grids, and the number of nodes in each grid is calculated to form a density histogram centered on the reference node. This histogram reflects the degree of clustering and distribution pattern of the neighboring nodes in space.

[0080] The ratio of the standard deviation of spatial density to the Euclidean distance of the reference node is used as the local density measure. The standard deviation of spatial density is obtained by statistically calculating the density values ​​in the density histogram. A larger standard deviation indicates a more dispersed spatial distribution of neighboring nodes; conversely, a smaller standard deviation indicates a more concentrated distribution. The Euclidean distance of the reference node refers to the distance between the reference node and the centroid of the set of neighboring nodes. The centroid coordinates are calculated by the arithmetic mean of the coordinates of neighboring nodes. The formula for calculating the local density measure is: Local density measure = Standard deviation of spatial density / Euclidean distance. This ratio measures the relative magnitude of the dispersion of node distribution relative to spatial distance within a local region centered on the reference node. For example, if the standard deviation is small and the Euclidean distance is large, it indicates that the node distribution within the local region is concentrated and far from the reference node, resulting in a low local density; if the standard deviation is large and the Euclidean distance is small, it indicates that the node distribution is dispersed but close to the reference node, resulting in a high local density.

[0081] The path residual between the baseline node and associated nodes is calculated, and its absolute value is taken as the global density. Associated nodes refer to nodes directly connected to the baseline node in the initial path planning. The path residual is the coordinate difference between the baseline node and associated nodes in three-dimensional space, calculated by determining the coordinate differences along the x, y, and z axes, and then using vector norms to determine the magnitude of the path residual. Taking the absolute value, the global density reflects the degree of spatial difference between the baseline node and associated nodes on the path. For example, a larger path residual indicates a greater spatial distance between the baseline node and associated nodes, requiring the path planning to consider a larger span, resulting in a higher global density; conversely, a smaller path residual indicates a closer spatial distance, better path continuity, and a lower global density.

[0082] Finally, the harmonic mean of the local and global density values ​​is used as the path density coefficient. The formula for calculating the harmonic mean is: Path density coefficient = 2 × (Local density × Global density) / (Local density + Global density). The harmonic mean is sensitive to smaller values, balancing the influence of local and global density values ​​and avoiding the bias of a single indicator. For example, when the local density is high but the global density is low, the harmonic mean will favor the smaller global density, making the path density coefficient better reflect the coherence of the global path. When the two values ​​are similar, the path density coefficient is the geometric mean of the two, comprehensively reflecting the density characteristics of both local and global density.

[0083] In practical implementation, the selection of the reference path node can be combined with the binning area partitioning strategy. For example, the binning space can be divided into several cubic sub-regions, and the center node of each sub-region can be selected as the reference node. The value of k (i.e., n) in the k-nearest neighbor algorithm needs to be adjusted according to the size and density of the objects in the bin. For small, densely packed objects, n can be set to a larger value (e.g., n=30) to capture the distribution characteristics of more neighboring nodes; for large, sparsely packed objects, n can be set to a smaller value (e.g., n=5) to avoid including irrelevant nodes. The cubic grid partitioning granularity of the spatial density distribution can be set according to the geometric accuracy requirements of the object. For example, when the accuracy requirement is high, the grid side length can be set to 1 / 10 of the smallest size of the object; when the accuracy requirement is low, it can be set to 1 / 5 of the average size of the object. The calculation of the path residual needs to consider the initial direction of the path planning. For example, in the Cartesian coordinate system, the direction of the robotic arm movement is taken as the x-axis, and the vertical direction is taken as the y and z axes. The residual components of each axis are calculated separately to ensure that the global density quantity can accurately reflect the spatial differences in the path direction.

[0084] When calculating the harmonic mean, care must be taken to avoid cases where the denominator is zero. When both the local and global density values ​​are zero (i.e., the reference node has no neighboring or associated nodes, which usually occurs in isolated nodes or path start / end points), the path density coefficient can be set to a preset minimum value (e.g., 0.1) to ensure the robustness of the algorithm. Furthermore, the dimensions of the local and global density values ​​must be consistent, which can be achieved through normalization (e.g., scaling both to the [0,1] interval) to ensure the physical meaning of the harmonic mean is clear.

[0085] Through the above steps, the path density coefficient can comprehensively reflect the local spatial distribution characteristics and global path connection characteristics of the reference nodes in the packing area, providing a quantitative basis for the subsequent construction of the graph Laplacian matrix and the extraction of spatial principal component vectors.

[0086] Example 3:

[0087] Based on the overall scheme described above, the specific steps for extracting the spatial principal component vector are as follows: First, construct the graph Laplacian matrix of path density coefficients for different packing regions, and then perform spectral clustering decomposition to obtain eigenvectors. The path density coefficient matrix is ​​composed of the path density coefficients of each reference node calculated in Example 2. The rows of the matrix correspond to different packing regions, and the columns correspond to the reference nodes within the regions. The construction of the graph Laplacian matrix is ​​based on this coefficient matrix. The specific process is as follows: First, calculate the similarity matrix between nodes. The similarity is defined as the Gaussian kernel function value of the path density coefficients of two nodes, i.e., similarity = exp(-||coefficient i - coefficient j||² / (2σ²)), where σ is the bandwidth parameter, which can be automatically determined by the median of the coefficient differences between all pairs of nodes. Next, construct the degree matrix, whose diagonal elements are the sum of the similarities of the corresponding nodes, and the off-diagonal elements are zero. Finally, the graph Laplacian matrix is ​​defined as the degree matrix minus the similarity matrix. Perform spectral clustering decomposition on this matrix, i.e., solve its characteristic equation to obtain the eigenvalues ​​and the corresponding eigenvectors.

[0088] The principal component subspace is constructed by selecting eigenvectors whose cumulative contribution rate exceeds a set threshold. The eigenvalues ​​are arranged in descending order, and the cumulative contribution rate is the ratio of the sum of the first k eigenvalues ​​to the sum of all eigenvalues. After setting a threshold (e.g., 85% or 90%), the smallest k eigenvectors are selected such that the cumulative contribution rate exceeds the threshold, and these k eigenvectors constitute the principal component subspace. This space retains the main information of the original path density coefficient matrix while achieving data dimensionality reduction. For example, if the original matrix has 100 nodes, spectral decomposition may yield 100 eigenvectors, but if the cumulative contribution rate of only the first 10 eigenvectors exceeds 90%, then these 10 eigenvectors are selected to form the principal component subspace, reducing the data dimensionality from 100 to 10.

[0089] Finally, the graph Laplacian matrix is ​​projected onto the principal component subspace to obtain the dimensionality-reduced spatial principal component vectors. The projection process is achieved through matrix multiplication, specifically multiplying the graph Laplacian matrix by the eigenvector matrix of the principal component subspace. The result is the dimensionality-reduced spatial principal component vectors. These vectors retain the main features of the original data while removing noise and redundant information, allowing subsequent genetic algorithm optimization to be performed in a lower-dimensional space, improving computational efficiency and optimization accuracy.

[0090] In practical implementation, the similarity matrix can be calculated using a parallel computing strategy, distributing node pairs to multiple computing units for simultaneous processing to accelerate computation. The choice of the bandwidth parameter σ is crucial for constructing the similarity matrix; too high a value will cause the similarity between all nodes to approach 1, while too low a value will make the similarity matrix resemble a sparse matrix, both of which are detrimental to spectral decomposition. An initial σ value can be calculated by taking the median of the coefficient differences between all node pairs, and then fine-tuned based on the data distribution.

[0091] Spectral clustering decomposition can employ iterative methods such as the power method and the Lanczos algorithm to solve for eigenvalues ​​and eigenvectors. These methods are highly efficient when dealing with large-scale matrices. When selecting eigenvectors whose cumulative contribution rate exceeds a threshold, care must be taken to avoid selecting too many eigenvectors, which would result in insignificant dimensionality reduction, while also preventing the selection of too few eigenvectors, which could lead to the loss of important information.

[0092] Projection operations on principal component subspaces must ensure matrix dimension matching, which can be verified using matrix transpose and multiplication rules. The dimensionality-reduced principal component vectors can be visualized, for example, by plotting 2D or 3D scatter plots to observe the distribution of different bin regions in the principal component space, providing intuitive guidance for subsequent genetic algorithm optimization.

[0093] The entire process of extracting principal component vectors in the spatial domain is based on rigorous mathematical operations and data processing steps, ensuring the accuracy and repeatability of the transformation from path density coefficients to principal component vectors. Dimensionality reduction not only reduces data redundancy but also highlights key features, providing more effective input data for subsequent path planning weight optimization.

[0094] Example 4:

[0095] Based on the overall scheme described above, the generation process of path planning weights is as follows: First, the spatial principal component vectors are standardized, and their cosine similarity with the historical best path vectors is calculated. The spatial principal component vectors are extracted in Example 3, and their dimension has been reduced to the principal component subspace (e.g., 10 dimensions) through spectral decomposition. Standardization is achieved by subtracting the vector mean and dividing by the standard deviation, so that the numerical distribution of each dimension has zero mean and unit variance, avoiding the influence of differences in dimensions on the similarity calculation. The historical best path vectors are stored in the system database, which is the set of principal component vectors that have performed best in the optimized path in the historical bin packing task, reflecting the feature patterns of past successful cases. The cosine similarity is obtained by calculating the dot product of two vectors and dividing by the product of their magnitudes, with a value range of [-1, 1]. The larger the value, the more similar the vector directions are, that is, the higher the feature matching degree between the current principal component vector and the historical best path.

[0096] Similarity scores are input into the genetic algorithm population, and initial weights are generated through selection, crossover, and mutation operations. The genetic algorithm population consists of multiple individuals, each representing a set of candidate weight parameters (e.g., a vector with the same dimension as the principal component vector, where each element represents the weight coefficient of the corresponding principal component). The initial population can be generated randomly or with minor perturbations based on historical best weights. Selection uses a roulette wheel or tournament selection method, filtering high-quality individuals based on their fitness (e.g., similarity value); individuals with higher fitness have a greater probability of being selected. Crossover is implemented through single-point or uniform crossover, combining the weight parameters of selected individuals to generate offspring. For example, single-point crossover randomly selects a crossover point in the weight vector, swapping the parameter values ​​of two parent individuals after that point. Mutation randomly perturbs the individual's weight parameters with a low probability (e.g., 0.01), such as by adding a small random number following a normal distribution, to maintain population diversity and avoid getting trapped in local optima. After several generations (e.g., 50 generations), initial weights with high fitness are generated.

[0097] Finally, an adaptive sliding window is used to locally optimize the initial weights, outputting the path planning weights for each region. The size of the adaptive sliding window is dynamically adjusted according to the distribution density of the principal component vectors. A larger window (containing 10 individuals) is used in regions where weight changes are gradual to accelerate optimization, while a smaller window (containing 3 individuals) is used in regions where gradient changes are rapid to fine-tune the process. The optimization algorithm within the window can employ a local search method, such as generating multiple candidate solutions within the neighborhood of the current weight vector (e.g., within a range of 0.1 for each parameter), calculating their corresponding path planning performance evaluation values ​​(e.g., a comprehensive index of path length and smoothness), and selecting the optimal solution as the local optimum within the window. The window slides gradually along the weight parameter space until all regions are traversed or a stopping condition is met (e.g., the optimal solution does not change significantly within 5 consecutive generations of windows). The final path planning weights for each region are a set of coefficients corresponding to the dimensions of the principal component vectors, used to adjust the update direction and magnitude of the baseline path nodes.

[0098] In practical implementation, standardization requires saving the mean and standard deviation parameters from the training phase to ensure consistent standardization of new data during the testing phase. The storage of historical best path vectors needs to include corresponding binning scene labels (such as object size range, bin shape, etc.) to match the most similar historical vectors in the current scene. The population size of the genetic algorithm is typically set to 50-200 individuals, and the number of iterations is set according to computational resources and accuracy requirements, generally 50-100 generations. The crossover probability is typically set to 0.6-0.9, and the mutation probability to 0.001-0.01, to balance search speed and global exploration capability.

[0099] The dynamic adjustment mechanism of the adaptive sliding window can be achieved by calculating the variance of the weight parameters within the current window. A larger variance indicates a more dispersed distribution, requiring the window to be narrowed to focus on local details; a smaller variance indicates a more concentrated distribution, allowing the window to be expanded to explore a wider area. The calculation of the path planning performance evaluation value needs to be consistent with the goal of subsequent path optimization, such as aiming for the shortest path length and the fewest turns. A comprehensive evaluation function is constructed, but actual experimental data is not required; only the relative value is calculated based on the mathematical relationship between the parameters.

[0100] The entire path planning weight generation process combines historical experience data with intelligent optimization algorithms. It standardizes data scale, explores the weight space using the global search capability of genetic algorithms, and then performs local fine-tuning through adaptive sliding windows to ensure that the generated weights can both draw on successful historical patterns and adapt to the unique needs of the current packing scenario.

[0101] Example 5:

[0102] Based on the overall scheme described above, the optimization path generation and related extension steps are as follows: The path planning weights are multiplied with the baseline path nodes using tensors to obtain the node update parameters. The path planning weights are generated by Example 4 and are a set of coefficients corresponding to the dimensions of the principal component vector (e.g., a weight vector with dimension 10). The baseline path nodes are discrete points in the initial planned path, each containing three-dimensional coordinates (x, y, z) and attributes such as a timestamp. The tensor multiplication operation multiplies the coordinate components of each baseline path node element-wise with the corresponding weight coefficients. For example, for the node coordinate vector... and weight vector Node update parameters The calculation formula is:

[0103]

[0104] Where: when the weight value When the weight value is increased, the coordinate value of the corresponding dimension is magnified; when the weight value is increased... At that time, reduce the coordinate values ​​of the corresponding dimension; This represents the element-wise multiplication of vectors (Hadamard product). This parameter... This reflects the magnitude and direction of the weight's adjustment to the node coordinates.

[0105] The deviation between the updated node parameters and the baseline path is calculated, and a dynamic programming algorithm is used to correct the path based on this deviation. The deviation is the difference between the updated node coordinates and the original baseline path node coordinates, i.e. Dynamic programming is used to optimize an adjusted node sequence while considering path continuity and robotic arm motion constraints. The specific steps are as follows: Arrange the baseline path nodes in chronological order to form a state space, where each state represents the updated coordinates and cumulative deviation of the node; define the transition cost as the weighted sum of the Euclidean distance between adjacent nodes and the change in deviation (the weights can be set according to path smoothness requirements); recursively calculate the minimum cost path from the starting point to the ending point to obtain the corrected node coordinate sequence. For example, for a path containing n nodes, the dynamic programming table... Let $\frac{i}{i}$ represent the minimum cumulative cost when the $i$-th node is in state $j$. The optimal solution is obtained by traversing all possible state transitions.

[0106] The corrected node sequence is connected to form a continuous trajectory, generating an optimized path. The corrected node sequence is arranged sequentially on the time axis, and adjacent nodes are connected by cubic spline interpolation or Bézier curve fitting to form smooth curves, ensuring that the robotic arm's motion trajectory has no abrupt changes. During interpolation, the kinematic constraints of the robotic arm, such as maximum joint velocity and acceleration limits, must be satisfied. Interpolation parameters (such as the tension coefficient of the spline curve) are adjusted to ensure the trajectory conforms to physical feasibility. The generated optimized path is represented by a continuous sequence of coordinate points and corresponding time parameters, which can be directly used as input commands for the robotic arm's motion control.

[0107] The process also includes the following steps: constructing a 3D packing trajectory space based on the optimized path, and extracting intersection points and conflict areas in the trajectory. Specifically, this involves mapping the optimized path to a 3D coordinate system according to the motion time sequence, generating a trajectory point cloud set containing timestamps. ,in Let be the trajectory points at time t. A point cloud reconstruction algorithm (such as voxelization meshing) is used to perform 3D voxelization modeling of the trajectory point cloud. The 3D space is divided into uniform cubic voxels, and voxel units are filled according to the distribution of trajectory points to form a voxel model containing the trajectory shape. The surface curvature of the voxel model is calculated. The normal vector of each voxel surface point is estimated using the finite difference method, and then the Gaussian curvature and mean curvature are calculated. The curvature value reflects the degree of curvature of the trajectory surface. A non-maximum suppression algorithm is used to enhance the curvature distribution. That is, for each voxel point, if its curvature value is not a local maximum, it is suppressed (set to zero), retaining points with significant curvature, eliminating noise interference, and thus highlighting sharp corners, intersections, and other characteristic areas in the trajectory.

[0108] When the number of intersections exceeds a set threshold or the area of ​​the conflict zone exceeds a limit, the path is deemed abnormal, and a trajectory adjustment command is generated. Intersection detection involves traversing the trajectory point cloud set and calculating the spatial distance and time difference between any two points. If the distance is less than a preset threshold (e.g., 1.5 times the size of the robotic arm's end effector) and the time difference is less than the minimum safe time interval, it is identified as a potential intersection. A conflict zone is defined as a continuous voxel region in the voxel model whose curvature value exceeds a threshold; its area is calculated by counting the number of conflicting voxels. If the number of intersections exceeds the threshold (e.g., 5) or the area of ​​the conflict zone exceeds a limit (e.g., 100 voxel units), an anomaly handling mechanism is triggered, generating a trajectory adjustment command containing adjustment direction and magnitude, and returning to the path correction stage for re-optimization.

[0109] A mixed-integer programming model is established, integrating path planning parameters and robotic arm motion parameters. The branch-and-bound method is used to solve for the optimal motion strategy. The variables in the mixed-integer programming model include the discrete coordinates of path nodes (integer variables) and the joint angles of the robotic arm (continuous variables). The objective function is defined as a linear combination of path length and motion energy consumption, i.e. ,in This is the total path length. The integral energy consumption of the joint torque. and The weighting coefficients (can be set according to the optimization objectives of packing efficiency and energy consumption) are used. Constraints include the limits of the robotic arm joint angles, speed, and acceleration. When using the branch and bound method, the objective function is first relaxed, ignoring integer constraints to obtain a continuous relaxation problem, from which the lower bound is obtained. Then, the integer variables are branched into different subproblems, and the lower bound and feasible solutions of each subproblem are recursively solved. Branches with lower bounds higher than the current optimal solution are pruned until the optimal solution satisfying the integer constraints is found.

[0110] The optimal motion strategy and optimized path are spatially aligned to generate a path scheme with the highest packing efficiency. Spatial alignment is achieved through coordinate transformation, mapping the optimal solution of the robot arm's joint space to Cartesian space and matching it with the coordinate points of the optimized path. The time parameters of the trajectory are adjusted to ensure motion synchronization. The final generated path scheme includes optimized trajectory coordinates, joint angle sequences, and time planning, ensuring that the robot arm completes the packing operation with the shortest path and lowest energy consumption while satisfying physical constraints.

[0111] This embodiment utilizes techniques such as tensor operations, dynamic programming, voxel modeling, and mixed-integer programming to achieve a complete process from path weight generation to optimized trajectory generation, anomaly detection, and strategy optimization. Each step is based on data processing and algorithmic logic, avoiding the introduction of hypothetical experimental effect descriptions, meeting the technical detail requirements of the specific implementation methods in the patent application, and ensuring the feasibility and efficiency optimization of the path through multi-layered verification mechanisms (such as intersection detection and constraint solving).

[0112] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0113] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A bin packing path planning method based on deep learning, characterized in that, include: Collect the size, weight, and geometric feature data of the objects to be packed, and construct the three-dimensional coordinate parameters of the packing space; A graph convolutional network is used to perform spatial topology modeling on geometric feature data, generate a geometric topology graph, and calculate the path density coefficient based on the spatial density variation between nodes in the topology graph. The graph Laplacian matrix of the path density coefficients of different packing areas is spectrally decomposed to extract spatial principal component vectors. The principal component vectors are then iteratively optimized using a genetic algorithm to generate path planning weights for each area. The baseline path nodes are dynamically updated based on the path planning weights to generate an optimized path, and the movement trajectory of the robotic arm is controlled based on the optimized path. The path density coefficient calculation includes: Select a baseline path node in any packing area and calculate the spatial density distribution of its n neighboring nodes; The ratio of the standard deviation of spatial density to the Euclidean distance of the reference node is used as the local density quantity. Calculate the path residual between the baseline node and the associated nodes, and take its absolute value as the global density quantity; The harmonic mean of the local density and the global density is used as the path density coefficient. The extraction of spatial principal component vectors includes: Construct a graph Laplacian matrix of path density coefficients for different packing regions, and perform spectral clustering decomposition on it to obtain feature vectors; The feature vectors whose cumulative contribution rate exceeds a set threshold are selected to form the principal component subspace; Projecting the graph Laplacian matrix onto the principal component subspace yields the dimensionality-reduced spatial principal component vectors. The path planning weight generation includes: The spatial principal component vectors are standardized, and their cosine similarity with the historical best path vectors is calculated. The similarity scores are input into the genetic algorithm population, and initial weights are generated through selection, crossover, and mutation operations. The initial weights are locally optimized using an adaptive sliding window, and the path planning weights for each region are output.

2. The bin packing path planning method based on deep learning according to claim 1, characterized in that, The spatial topology modeling includes: Geometric feature data is converted into a three-dimensional spatial tensor, and dimensional differences are eliminated through normalization. Multi-layer graph convolutional layers are used to aggregate neighborhood features of the three-dimensional spatial tensor to generate an adjacency matrix between nodes; The adjacency matrix is ​​optimized using a dynamic routing mechanism to output a geometric topology graph.

3. The bin packing path planning method based on deep learning according to claim 1, characterized in that, The generated optimization path includes: The node update parameters are obtained by multiplying the path planning weights by tensors with the baseline path nodes. The deviation between the node update parameters and the baseline path is calculated, and the deviation is corrected using a dynamic programming algorithm. The corrected node sequence is connected into a continuous trajectory to generate an optimized path.

4. The bin packing path planning method based on deep learning according to claim 1, characterized in that, Also includes: A 3D packing trajectory space is constructed based on the optimized path, and the intersection points and conflict areas in the trajectory are extracted. When the number of intersections exceeds the set threshold or the area of ​​the conflict zone is larger than the limit, the path is judged to be in an abnormal state, and a trajectory adjustment command is generated.

5. The bin packing path planning method based on deep learning according to claim 4, characterized in that, The construction of the three-dimensional packing trajectory space includes: The optimized path is mapped to a three-dimensional coordinate system according to the motion sequence to generate a trajectory point cloud set. A point cloud reconstruction algorithm is used to perform three-dimensional voxel modeling of the trajectory point cloud and calculate the surface curvature of the voxel model. The curvature distribution is enhanced by using a nonmaximum suppression algorithm to eliminate noise interference.

6. The bin packing path planning method based on deep learning according to claim 1, characterized in that, Also includes: A mixed integer programming model of path planning parameters and robotic arm motion parameters is established, and the branch and bound method is used to solve for the optimal motion strategy. Spatially align the optimal motion strategy with the optimized path to generate a path scheme with the highest packing efficiency; The solution to the mixed integer programming model includes: The objective function is defined as a linear combination of path length and motion energy consumption, and the constraint is the limit of the joint angle of the robotic arm; The objective function is relaxed and decomposed into path parameter subproblems and motion parameter subproblems. The two subproblems are solved iteratively and alternately until convergence, and the optimal motion strategy that satisfies the constraints is output.

7. A deep learning-based bin packing path planning system, characterized in that, include: The 3D data acquisition module is used to acquire the size, weight, and geometric feature data of the object to be packed, and to construct a 3D spatial coordinate matrix; The topology modeling module uses graph convolutional networks to aggregate neighborhood features of the three-dimensional spatial coordinate matrix and generates a geometric topology graph through a dynamic routing mechanism. The density coefficient calculation module generates local and global density quantities based on the spatial density distribution of nodes in the geometric topology graph, and calculates their harmonic mean as the path density coefficient. The spectral decomposition module constructs a graph Laplacian matrix for the path density coefficients of different packing regions and extracts spatial principal component vectors through spectral clustering decomposition. The weight optimization module uses a genetic algorithm to iteratively optimize the spatial principal component vectors and generate path planning weights for each region. The path generation module dynamically updates the baseline path nodes based on path planning weights, corrects path deviations through dynamic programming algorithms, and generates optimized paths. The motion control module converts the optimized path into control commands for the robotic arm, and drives the actuator to complete the packing operation through a real-time communication protocol; The calculation of the path density coefficient includes: Select a baseline path node in any packing area and calculate the spatial density distribution of its n neighboring nodes; The ratio of the standard deviation of spatial density to the Euclidean distance of the reference node is used as the local density quantity. Calculate the path residual between the baseline node and the associated nodes, and take its absolute value as the global density quantity; The harmonic mean of the local density and the global density is used as the path density coefficient. The extraction of the spatial principal component vectors includes: Construct a graph Laplacian matrix of path density coefficients for different packing regions, and perform spectral clustering decomposition on it to obtain feature vectors; The feature vectors whose cumulative contribution rate exceeds a set threshold are selected to form the principal component subspace; Projecting the graph Laplacian matrix onto the principal component subspace yields the dimensionality-reduced spatial principal component vectors. The generation of the path planning weights includes: The spatial principal component vectors are standardized, and their cosine similarity with the historical best path vectors is calculated. The similarity scores are input into the genetic algorithm population, and initial weights are generated through selection, crossover, and mutation operations. The initial weights are locally optimized using an adaptive sliding window, and the path planning weights for each region are output.

Images