Intelligent carton stacking method and system

Abstract

The present application relates to the field of material handling technology, and in particular to an intelligent stacking method and system. The method comprises the following steps: obtaining the top features of the stack; obtaining the outline features of the stack; creating a first sliding time window, the first sliding time window containing the top features of the stack and the outline features of the stack after a plurality of consecutive stacking operations to form a first feature matrix; inputting the first feature matrix into a dynamic weight calculation module to output a fusion weight; creating a second sliding time window, the second sliding time window containing the top features of the stack and the outline features of the stack after a plurality of consecutive stacking operations, performing weighted fusion according to the aforementioned fusion weight to form a second feature matrix; inputting the second feature matrix into a position compensation calculation module to calculate the position compensation amount of the current carton group unit when it is placed. The present application can better stack cartons.

Description

Technical Field

[0001] The present application relates to the field of material handling technology, and in particular to an intelligent stacking method and system. Background Art

[0002] Currently, with the rapid development of logistics automation and intelligent manufacturing, automated stacking technology has been widely used in warehousing, logistics, manufacturing and other fields. Automated stacking systems can significantly improve operational efficiency and reduce labor costs.

[0003] Mainstream automated palletizing systems primarily utilize a rigid positioning method based on preset coordinates. This method targets standard goods of regular dimensions. Ideal stacking position coordinates are typically pre-set, and the robot performs the palletizing operation according to a fixed placement trajectory and fixed cargo position.

[0004] However, in the stacking operation of a carton group consisting of a plurality of unexpanded cartons, the prior art still has some problems, resulting in poor stacking effect of this type of cartons. Summary of the Invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides an intelligent carton stacking method, which can better stack cartons.

[0006] In a first aspect, the present application proposes an intelligent carton stacking method, which comprises the following steps:

[0007] According to a preset stacking path, stacking carton group units consisting of multiple unexpanded cartons onto a pallet one by one to form a stack body; the preset stacking path includes a target placement position for each carton group unit;

[0008] Before placing each carton unit, the following steps are also included:

[0009] The top vision system is used to collect data on the top of the stack and obtain the top features of the stack;

[0010] The side vision system is used to collect data on the side of the stack to obtain the stack contour features;

[0011] Creating a first sliding time window, wherein the first sliding time window includes stack top features and stack outline features after a recent plurality of consecutive stacking operations to form a first feature matrix;

[0012] Inputting the first feature matrix into the dynamic weight calculation module to output the fusion weight of the stack top feature and the fusion weight of the stack contour feature;

[0013] Creating a second sliding time window, the second sliding time window contains the stack top features and stack outline features after the most recent multiple consecutive stacking operations, and performing weighted fusion on the stack top features and stack outline features within the second sliding time window according to the fusion weights obtained above to form a second feature matrix;

[0014] The second characteristic matrix is ​​input into the position compensation calculation module to calculate the position compensation of the current carton group unit when it is placed;

[0015] The target placement position of the current carton group unit is adjusted according to the position compensation amount.

[0016] Optionally, the following steps are also included before placing each carton group unit:

[0017] After the carton group unit is grasped, the contour features of the carton group unit are acquired by the lateral vision system;

[0018] Inputting the contour features of the carton group unit into the carton group stability judgment module to determine whether the carton group unit is in a stable or unstable state;

[0019] If the current carton group unit is determined to be in a stable state, the carton group unit is placed by directly releasing it;

[0020] If the current carton group unit is determined to be in an unstable state, the carton group unit is placed in a compaction and release manner to cause the current stack surface to expose its true state;

[0021] The unstable state includes a bulging state of the carton group unit and an internal loose state of the carton group unit.

[0022] Optionally, the carton group stability judgment module includes:

[0023] A carton group unit image segmentation unit, used to segment the side image area of ​​the current carton group unit from the image acquired by the side vision system;

[0024] a first convolutional neural network, configured to receive the side image region as input and output a state classification result of the carton group unit;

[0025] The classification results include: normal state, bulging state and internal loose state.

[0026] Optionally, the dynamic weight calculation module includes:

[0027] a first recurrent neural network, configured to receive the first feature matrix as input;

[0028] The first recurrent neural network encodes the time series features in the first feature matrix to output a historical state encoding vector;

[0029] A weight generation unit composed of a fully connected neural network is connected to the output end of the first recurrent neural network, and is used to calculate and output the fusion weight of the stack top feature and the fusion weight of the stack contour feature based on the historical state encoding vector.

[0030] Optionally, the position compensation amount calculation module includes:

[0031] a second recurrent neural network, configured to receive the second feature matrix as input;

[0032] The second recurrent neural network encodes the weighted fusion time series features in the second feature matrix to output a compensation decision encoding vector;

[0033] A compensation amount calculation unit composed of a fully connected neural network is connected to the output end of the second recurrent neural network, and is used to regressively calculate and output the position compensation amount of the current carton group unit in the three-dimensional direction based on the compensation decision coding vector.

[0034] Optionally, the intelligent carton stacking method further includes:

[0035] The dynamic weight calculation module also determines whether the placement position of the current carton group unit needs to be adjusted based on the first characteristic matrix;

[0036] The dynamic weight calculation module outputs the fusion weight only when it is determined that adjustment is needed.

[0037] Optionally, the second sliding time window is larger than the first sliding time window.

[0038] In the second aspect, the present application proposes an intelligent carton stacking system, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the intelligent carton stacking method as described in any one of the first aspects.

[0039] The technical solution provided by this application has the following advantages compared with the existing technology:

[0040] One of its beneficial effects and its working principle is:

[0041] In the application scenario of the present application, the carton stacking process is a process of bundling several unexpanded cartons into groups with strapping to form carton group units, and then placing them one by one on a pallet to form a multi-layer stack.

[0042] However, since the carton group unit is composed of several unexpanded cartons tied together with straps, the surface of the cartons is relatively smooth, and the unexpanded cartons may slip due to vibration, and a spring-like state may be formed between the cartons. These factors inevitably lead to the fact that as the carton group units are stacked layer by layer, the actual position and shape of each stacked carton group unit after placement may have slight but cumulative deviations from the theoretical preset position, and this cumulative deviation will continue to amplify as the stacking height increases, ultimately leading to a decrease in the stability and accuracy of the entire stack.

[0043] Furthermore, if too much attention is paid to the top features of the stack, the overall contour of the stack may be neglected, causing the stack to tilt or even overturn over time. Conversely, if too much emphasis is placed on the overall stack, local placement accuracy may be sacrificed, leading to increasingly obvious local unevenness on the top layer, ultimately reducing the effective load-bearing capacity of the carton stack.

[0044] Therefore, this application proposes a cardboard stacking method, which collects the top features of the stack through an overhead vision system and obtains the overall contour features of the stack through a side vision system, and then inputs this information into two hierarchical sliding time windows for time series analysis and processing.

[0045] First, the first sliding time window is used to collect historical feature data after multiple consecutive stacking operations and form the first feature matrix. Then, the dynamic weight calculation module calculates the optimal fusion weight of the stack top feature and the stack contour feature based on the current stack state and the historical evolution trend of the stack.

[0046] This dynamic weighting mechanism can adaptively adjust the distribution of attention between local accuracy and overall stability according to the different stages of the stacking process and the real-time changes in the stability of the stack.

[0047] Then, in the second sliding time window, the pile top features and pile contour features are weightedly fused according to the calculated fusion weights to form a second feature matrix that comprehensively considers historical changes, current status and future evolution trends introduced through the dynamic attention mechanism.

[0048] The final position compensation calculation module calculates the position compensation required for the current carton group unit when it is placed based on this second characteristic matrix, so as to better adjust the target placement position.

[0049] Therefore, the present application can effectively eliminate cumulative position deviations, and achieve a balance between placement accuracy and overall stability through a dynamic weight adjustment mechanism, thereby improving the stability of carton group stacking.

[0050] Therefore, the present application can better stack carton groups.

[0051] The second beneficial effect and its working principle are:

[0052] There is a certain degree of randomness when bundling cartons, and the stability of each carton unit itself is limited by the tightness of the strapping used when bundling the cartons.

[0053] Inevitably, when bundling cartons, the strapping may be too tight or too loose. If the strapping is too tight, the cartons in the cartons will be pulled up on both sides by the strapping, causing the center of the cartons to bulge slightly under the force, making the spring tendency of the cartons even stronger. If the strapping is too loose, the cartons in the cartons will slip or shift against each other to a greater extent.

[0054] When a group of cartons is severely bulging or loose inside, if they are still placed in a normal direct release manner, this unstable state will often be covered up until multiple layers are stacked and the cartons are under pressure, which will suddenly show up in a certain layer, causing the stack outline and the top stack surface to suddenly change.

[0055] Therefore, the present application proposes an active intervention method for compaction release, that is, when the above-mentioned unstable state of the carton group unit is detected, after the carton group unit is moved to the stack surface, pressure is applied to the current carton group unit to expose the true stack surface shape in advance, so that this change can be perceived before the next placement, avoiding the masking of this change after stacking multiple layers of carton group units, and avoiding internal stress and slip mutations of the stack, thereby improving the stability of the overall stack. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic flow chart of the intelligent carton stacking method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solution in this application will be described below with reference to the accompanying drawings.

[0058] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application may also be implemented in other ways than those described herein. It is apparent that the embodiments described in the specification are only some of the embodiments of the present application, not all of them. It should be noted that the embodiments of the present application and the features therein may be combined with each other unless there is a conflict.

[0059] First, refer to Figure 1 As shown, the present application proposes an intelligent carton stacking method, which includes the following steps:

[0060] S1: According to a preset stacking path, stacking carton group units consisting of multiple unexpanded cartons onto a pallet one by one to form a stack body; the preset stacking path includes a target placement position for each carton group unit.

[0061] Specifically, the preset stacking path includes the three-dimensional position of each carton group unit when it is placed in sequence to complete a stacking operation. In the embodiment of the present application, the three-dimensional position is the target placement position to which the center point of the carton group unit needs to be moved.

[0062] An industrial robot arm equipped with a pneumatic suction cup gripper is used to grab carton groups one by one from the assembly line and perform stacking operations according to the three-dimensional position sequence.

[0063] The three-dimensional position sequence for placing the carton group units can be manually configured in advance.

[0064] S2: Before placing the carton group unit each time, the following steps are also included:

[0065] After the carton group unit is grasped, the contour features of the carton group unit are acquired through the lateral vision system.

[0066] Among them, the industrial robot arm will move the carton group unit to a preset position before placing the carton group unit, so that the side vision system can capture the contour features of the carton group unit.

[0067] Specifically, in the embodiment of the present application, the Canny edge detection algorithm is used to extract the contour features of the carton group unit.

[0068] Inputting the contour features of the carton group unit into the carton group stability judgment module to determine whether the carton group unit is in a stable or unstable state;

[0069] If it is determined that the current carton group unit is in a stable state, the carton group unit is placed in a direct release manner.

[0070] Among them, direct release means:

[0071] After the industrial robotic arm moves the carton unit to the target placement position, it immediately turns off the adsorption function of the pneumatic suction cup, allowing the carton unit to fall naturally and fit the stack surface. No additional pressure or dwell time is applied during the entire release process, ensuring maximum placement efficiency.

[0072] If it is determined that the current carton group unit is in an unstable state, the carton group unit is placed in a compaction and release manner to cause the current stack surface to expose its true state.

[0073] Among them, compaction release means:

[0074] After the industrial robotic arm moves the carton unit to the target placement position, it closes the pneumatic suction cup, allowing the carton unit to fall naturally and fit the stack surface. It then presses the pneumatic suction cup down to contact the carton unit and applies a contact pressure of 15-20N to drive the stack surface to deform towards a stable state under the action of external force.

[0075] The unstable state of the current carton group unit includes a bulging state of the carton group unit and an internal loose state of the carton group unit.

[0076] The bulging state of the carton group unit is that the strapping is too tight. The two sides of the carton of the carton group unit will be tied up by the strapping, causing the middle of the carton group unit to bulge slightly under the force.

[0077] The internal loose state of the carton group unit means that the strapping is too loose, and the cartons in the carton group unit will slip or shift with each other on a larger scale, which is manifested as the entire carton group being offset when the carton group unit is lifted.

[0078] Specifically, in the embodiment of the present application, the carton group stability judgment module includes:

[0079] A carton group unit image segmentation unit, used to segment the side image area of ​​the current carton group unit from the image acquired by the side vision system;

[0080] a first convolutional neural network, configured to receive the side image region as input and output a state classification result of the carton group unit;

[0081] The classification results include: normal state, bulging state and internal loose state.

[0082] Specifically, in the embodiment of the present application, the training process of the first convolutional neural network is:

[0083] Multiple side images of carton group units in different states are collected as training data sets, including normal state images, bulging state images and internal loose state images. Manual labeling is used to ensure label accuracy.

[0084] The first convolutional neural network is trained using side-view images of carton units from the training dataset as input and corresponding state labels (normal, bulging, and loose inside) as labels. The first convolutional neural network is trained to classify the states of the carton units. The cross-entropy loss function is used in the training process.

[0085] S3: Collect data on the top of the stack through the overhead vision system to obtain the top features of the stack;

[0086] The overhead vision system uses an Intel depth camera, installed at a height of 9 meters directly above the stack (based on the initial placement of the stack support), with a frame rate of 30 fps. This camera captures RGB images and depth information from the top of the stack, extracting its 3D morphological features using point cloud processing technology.

[0087] Specifically, in the embodiment of the present application, the overhead vision system first performs image denoising and depth map completion after each capture. A bilateral filter is used to denoise the RGB image. A fast bilateral stereo matching algorithm is then used to complete the missing depth information, converting the resulting image into a 3D point cloud.

[0088] The top plane of the stack is identified using the RANSAC plane detection algorithm.

[0089] Based on the detected plane, the top of the pile is divided into 20×20 grids, and the average height value of each grid is calculated to form a 400-dimensional feature vector of the top of the pile.

[0090] The side vision system is used to collect data on the side of the stack to obtain the stack contour features;

[0091] The lateral vision system uses two Baoshina industrial cameras, which are installed at a height of 1.8 meters on both sides of the stack to obtain the contour morphological features of the front and side of the entire stack.

[0092] Specifically, in the embodiment of the present application, the Canny edge detection algorithm is used to extract the stack body contour. The extracted contour is processed by the Douglas-Peucker algorithm to ultimately form a 256-dimensional stack body contour feature vector containing the coordinates of the contour key points.

[0093] S4: Create a first sliding time window, which contains the top features and stack contour features of the stack after multiple consecutive stacking operations to form a first feature matrix; input the first feature matrix into a dynamic weight calculation module to output the fusion weight of the top features of the stack and the fusion weight of the stack contour features.

[0094] Specifically, in the embodiment of the present application, the length of the first sliding time window is set to 8, that is, it includes the feature data after the latest 8 stacking operations.

[0095] After completing each stacking operation, the system combines the latest stack top features (400 dimensions) and stack contour features (256 dimensions) into a 656-dimensional comprehensive feature vector, and stores them in the first sliding window in chronological order to form an 8×656 first feature matrix.

[0096] Specifically, the dynamic weight calculation module is a first composite neural network consisting of a first recurrent neural network and a fully connected neural network:

[0097] a first recurrent neural network, configured to receive the first feature matrix as input;

[0098] The first recurrent neural network encodes the time series features in the first feature matrix to output a historical state encoding vector.

[0099] The historical state encoding vector can characterize the correlation between the local accuracy represented by the stack top feature and the global stability represented by the stack contour feature in the stacking history contained in the first sliding time window.

[0100] A weight generation unit composed of a fully connected neural network is connected to the output end of the first recurrent neural network, and is used to calculate and output the fusion weight of the stack top feature and the fusion weight of the stack contour feature based on the historical state encoding vector.

[0101] Specifically, in the embodiment of the present application, the first recurrent neural network adopts a bidirectional LSTM architecture, and the specific parameters are as follows:

[0102] Input dimension: 656 (400-dimensional stack top features + 256-dimensional stack outline features).

[0103] Hidden layer dimension: 256 dimensions;

[0104] Number of layers: 2-layer bidirectional LSTM;

[0105] Activation function: tanh activation function;

[0106] Output dimension: 512 (concatenated from the forward and backward outputs of the bidirectional LSTM);

[0107] The weight matrices of the forget gate, input gate, and output gate of the LSTM are initialized using the Xavier initialization method, and the bias vector is initialized to zero.

[0108] The weight generation unit consists of a three-layer fully connected neural network:

[0109] First layer: input 512 dimensions, output 256 dimensions, activation function is ReLU;

[0110] Second layer: input 256 dimensions, output 128 dimensions, activation function is ReLU;

[0111] The third layer: input is 128-dimensional, output is 3-dimensional, and the activation function is Sigmoid;

[0112] The output 3D vectors represent:

[0113] The fusion weight output represents a vector with a value of 0 or 1.

[0114] Fusion weight of the top feature of the pile

[0115] and the fusion weight of the stack contour features ,

[0116] And through normalization to ensure + = 1.

[0117] Among them, the fusion weight is output only when the fusion weight output representation vector takes the value of 1, so as to determine whether the placement position of the current carton group unit needs to be adjusted.

[0118] By outputting a fusion weight output representation vector, the weight generation unit determines whether the placement of the current carton group unit needs to be adjusted based on the first feature matrix. Specifically, the weight generation unit generates a fusion weight output representation vector based on the historical state encoding vector obtained through the first feature matrix.

[0119] When the representation vector takes the value of 1, it indicates that the placement position of the current carton group unit needs to be adjusted, and then the corresponding fusion weight is output.

[0120] When the representation vector takes the value of 0, it indicates that the placement position of the current carton group unit does not need to be adjusted, and the corresponding fusion weight is not output.

[0121] Through this conditional judgment mechanism, the present application implements an on-demand calculation strategy. Only when the fusion weight output representation vector indicates that the placement position of the current carton group unit needs to be adjusted (when the value is 1), will the subsequent position compensation calculation process with a large amount of calculation be triggered, thereby effectively reducing the computational burden of the system.

[0122] Specifically, the dynamic weight calculation module is trained through the following steps:

[0123] The first feature matrix sequences and their corresponding weight annotation data under multiple different stacking scenarios are collected as training sets for the dynamic weight calculation module. Each set of data contains the first feature matrix sequence, as well as the fusion weight output representation vector (0 or 1) annotated by stacking experts according to the stability status of the stack, the stack top feature fusion weight (a floating point number between 0-1), and the stack contour feature fusion weight (a floating point number between 0-1), to ensure that the weight annotation reflects the balance between the local accuracy of the top and the global stability in actual stacking.

[0124] The first feature matrix sequence in the training set is used as the input of the first composite network.

[0125] The corresponding fusion weight output representation vector, the stack top feature fusion weight, and the stack outline feature fusion weight are used as labels for the first composite network to train a dynamic weight calculation module. The training process uses the mean square error loss function combined with the binary cross entropy loss function.

[0126] S5: Create a second sliding time window, which is larger than the first sliding time window. The second sliding time window contains the top features and outline features of the stack after multiple consecutive stacking operations. The top features and outline features of the stack in the second sliding time window are weightedly fused according to the fusion weights obtained above to form a second feature matrix; the second feature matrix is ​​input into the position compensation calculation module to calculate the position compensation of the current carton group unit when it is placed.

[0127] Specifically, in the embodiment of the present application, the length of the second sliding time window is set to 12, which contains the feature data of the last 12 stacking operations. This window also stores a 656-dimensional comprehensive feature vector to form a 12×656 feature matrix

[0128] According to the fusion weight output in S4, the 12×656 feature matrix in the second sliding time window is weightedly fused to form a second feature matrix of 12×656.

[0129] Specifically, the weighted fusion process is performed according to the following formula: For the i-th stacking operation (i=1, 2, ..., 12) in the second sliding time window, the fused feature vector [i] = × Top features of the pile[i] + × Stack outline features [i], where and They are respectively the stack top feature fusion weight and the stack outline feature fusion weight output by the dynamic weight calculation module in S4.

[0130] Specifically, the position compensation calculation module is a second composite neural network composed of a second recurrent neural network and a fully connected neural network:

[0131] a second recurrent neural network, configured to receive the second feature matrix as input;

[0132] The second recurrent neural network encodes the weighted fusion time series features in the second feature matrix to output a compensation decision encoding vector;

[0133] A compensation amount calculation unit composed of a fully connected neural network is connected to the output end of the second recurrent neural network, and is used to regressively calculate and output the position compensation amount of the current carton group unit in the three-dimensional direction based on the compensation decision coding vector.

[0134] Specifically, the second recurrent neural network also uses a bidirectional LSTM architecture, with the following parameters:

[0135] Input dimension: 656 dimensions (comprehensive feature dimension after weighted fusion);

[0136] Hidden layer dimension: 512 dimensions;

[0137] Number of layers: 3-layer bidirectional LSTM;

[0138] Activation function: tanh activation function;

[0139] Output dimension: 1024 (concatenated from the forward and backward outputs of the bidirectional LSTM);

[0140] The compensation calculation unit uses a four-layer fully connected neural network:

[0141] First layer: input 1024 dimensions, output 512 dimensions, activation function is ReLU;

[0142] Second layer: input 512 dimensions, output 256 dimensions, activation function is ReLU;

[0143] The third layer: input 256 dimensions, output 128 dimensions, activation function is ReLU;

[0144] The fourth layer has a 128-dimensional input and a 3-dimensional output, and the activation function is linear.

[0145] The output 3D vectors represent the position compensation amounts in the X, Y, and Z directions respectively.

[0146] Specifically, the position compensation calculation module is trained through the following steps:

[0147] Data containing the second characteristic matrices and their corresponding position compensation values ​​under multiple different stacking scenarios are collected as the training set for the second composite neural network. Each set of data contains the second characteristic matrix and the three-dimensional position compensation values ​​(compensation values ​​in the X, Y, and Z directions) annotated by stacking experts based on the actual stacking deviation.

[0148] Using the second feature matrix in the training set as the input of the second composite neural network;

[0149] The corresponding 3D position compensation amount annotated data is used as the label of the second composite neural network, and the position compensation amount calculation module is trained based on this. The mean square error loss function is used in the training process.

[0150] S6: Adjusting the target placement position of the current carton group unit according to the position compensation amount.

[0151] Specifically, the system adjusts the preset target placement position of the current carton group unit according to the three-dimensional compensation amount (ΔX, ΔY, ΔZ) output by the position compensation amount calculation module.

[0152] Adjusted target placement position X' = original target placement position X + ΔX;

[0153] Adjusted target placement position Y' = original target placement position Y + ΔY;

[0154] Adjusted target placement position Z' = original target placement position Z + ΔZ.

[0155] Among them, ΔX and ΔY correspond to position compensation in the horizontal plane, and ΔZ corresponds to height compensation in the vertical direction.

[0156] After receiving the adjusted target placement position, the industrial robot arm moves the center position of the currently grasped carton group unit to the adjusted target placement position, and then performs the placement operation.

[0157] In summary, the technical solution provided by this application has the following advantages compared with the existing technology:

[0158] One of its beneficial effects and its working principle is:

[0159] In the application scenario of the present application, the carton stacking process is a process of bundling several unexpanded cartons into groups with strapping to form carton group units, and then placing them one by one on a pallet to form a multi-layer stack.

[0160] However, since the carton group unit is composed of several unexpanded cartons tied together with straps, the surface of the cartons is relatively smooth, and the unexpanded cartons may slip due to vibration, and a spring-like state may be formed between the cartons. These factors inevitably lead to the fact that as the carton group units are stacked layer by layer, the actual position and shape of each stacked carton group unit after placement may have slight but cumulative deviations from the theoretical preset position, and this cumulative deviation will continue to amplify as the stacking height increases, ultimately leading to a decrease in the stability and accuracy of the entire stack.

[0161] Furthermore, if too much attention is paid to the top features of the stack, the overall contour of the stack may be neglected, causing the stack to tilt or even overturn over time. Conversely, if too much emphasis is placed on the overall stack, local placement accuracy may be sacrificed, leading to increasingly obvious local unevenness on the top layer, ultimately reducing the effective load-bearing capacity of the carton stack.

[0162] Therefore, this application proposes a cardboard stacking method, which collects the top features of the stack through an overhead vision system and obtains the overall contour features of the stack through a side vision system, and then inputs this information into two hierarchical sliding time windows for time series analysis and processing.

[0163] First, the first sliding time window is used to collect historical feature data after multiple consecutive stacking operations and form the first feature matrix. Then, the dynamic weight calculation module calculates the optimal fusion weight of the stack top feature and the stack contour feature based on the current stack state and the historical evolution trend of the stack.

[0164] This dynamic weighting mechanism can adaptively adjust the distribution of attention between local accuracy and overall stability according to the different stages of the stacking process and the real-time changes in the stability of the stack.

[0165] Then, in the second sliding time window, the pile top features and pile contour features are weightedly fused according to the calculated fusion weights to form a second feature matrix that comprehensively considers historical changes, current status and future evolution trends introduced through the dynamic attention mechanism.

[0166] The final position compensation calculation module calculates the position compensation required for the current carton group unit when it is placed based on this second characteristic matrix, so as to better adjust the target placement position.

[0167] Therefore, the present application can effectively eliminate cumulative position deviations, and achieve a balance between placement accuracy and overall stability through a dynamic weight adjustment mechanism, thereby improving the stability of carton group stacking.

[0168] Therefore, the present application can better stack carton groups.

[0169] The second beneficial effect and its working principle are:

[0170] There is a certain degree of randomness when bundling cartons, and the stability of each carton unit itself is limited by the tightness of the strapping used when bundling the cartons.

[0171] Inevitably, when bundling cartons, the strapping may be too tight or too loose. If the strapping is too tight, the sides of the cartons will be pulled up by the strapping, causing the center of the cartons to bulge slightly under the force, making the spring tendency of the cartons even stronger. If the strapping is too loose, the cartons will slip or shift more significantly.

[0172] When a group of cartons is severely bulging or loose inside, if they are still placed in a normal direct release manner, this unstable state will often be covered up until multiple layers are stacked and the cartons are under pressure, which will suddenly show up in a certain layer, causing the stack outline and the top stack surface to suddenly change.

[0173] Therefore, the present application proposes an active intervention method for compaction release, that is, when the above-mentioned unstable state of the carton group unit is detected, after the carton group unit is moved to the stack surface, pressure is applied to the current carton group unit to expose the true stack surface shape in advance, so that this change can be perceived before the next placement, avoiding the masking of this change after stacking multiple layers of carton group units, and avoiding internal stress and slip mutations of the stack, thereby improving the stability of the overall stack.

[0174] In the second aspect, an embodiment of the present application proposes an intelligent carton stacking system, including a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the intelligent carton stacking method as described in the above embodiment.

[0175] It should be noted that, in this document, relational terms such as "first" and "second" are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or device comprising the element. Furthermore, in the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent either A or B. "And / or" herein is merely a description of an associative relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, in the description of the embodiments of the present application, “plurality” refers to two or more than two.

[0176] The foregoing description is intended only to provide specific embodiments of the present application, which will enable those skilled in the art to understand and implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments described herein, but is intended to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent carton stacking method, comprising the following steps: According to a preset stacking path, stacking carton group units consisting of multiple unexpanded cartons onto a pallet one by one to form a stack body; the preset stacking path includes a target placement position for each carton group unit; It is characterized in that before placing the carton group unit each time, the following steps are also included: The top vision system is used to collect data on the top of the stack and obtain the top features of the stack; The side vision system is used to collect data on the side of the stack to obtain the stack contour features; Creating a first sliding time window, wherein the first sliding time window includes stack top features and stack outline features after a recent plurality of consecutive stacking operations to form a first feature matrix; Inputting the first feature matrix into the dynamic weight calculation module to output the fusion weight of the stack top feature and the fusion weight of the stack contour feature; Creating a second sliding time window, the second sliding time window includes the stack top features and stack outline features after the most recent multiple consecutive stacking operations, and performing weighted fusion on the stack top features and stack outline features within the second sliding time window according to the fusion weights of the stack top features and the fusion weights of the stack outline features obtained above to form a second feature matrix; The second characteristic matrix is ​​input into the position compensation calculation module to calculate the position compensation of the current carton group unit when it is placed; Adjust the target placement position of the current carton group unit according to the position compensation amount; The dynamic weight calculation module includes: a first recurrent neural network, configured to receive the first feature matrix as input; The first recurrent neural network encodes the time series features in the first feature matrix to output a historical state encoding vector; a weight generation unit composed of a fully connected neural network, the weight generation unit being connected to an output end of the first recurrent neural network and being configured to calculate and output a fusion weight of the stack top feature and a fusion weight of the stack outline feature based on the historical state encoding vector; The position compensation calculation module includes: a second recurrent neural network, configured to receive the second feature matrix as input; The second recurrent neural network encodes the weighted fusion time series features in the second feature matrix to output a compensation decision encoding vector; A compensation amount calculation unit composed of a fully connected neural network is connected to the output end of the second recurrent neural network, and is used to regressively calculate and output the position compensation amount of the current carton group unit in the three-dimensional direction based on the compensation decision coding vector.

2. The intelligent carton stacking method according to claim 1, characterized in that: Each time before placing the carton group unit, the following steps are also included: After the carton group unit is grasped, the contour features of the carton group unit are acquired by the lateral vision system; Inputting the contour features of the carton group unit into the carton group stability judgment module to determine whether the carton group unit is in a stable or unstable state; If the current carton group unit is determined to be in a stable state, the carton group unit is placed by directly releasing it; If the current carton group unit is determined to be in an unstable state, the carton group unit is placed in a compaction and release manner to cause the current stack surface to expose its true state; The unstable state includes a bulging state of the carton group unit and an internal loose state of the carton group unit.

3. The intelligent carton stacking method according to claim 2, characterized in that: The carton group stability judgment module includes: A carton group unit image segmentation unit, used to segment the side image area of ​​the current carton group unit from the image acquired by the side vision system; a first convolutional neural network, configured to receive the side image region as input and output a state classification result of the carton group unit; The state classification results include: normal state, bulging state and internal loose state.

4. The intelligent carton stacking method according to claim 1, characterized in that: The intelligent carton stacking method further comprises: The dynamic weight calculation module also determines whether the placement position of the current carton group unit needs to be adjusted based on the first characteristic matrix; The dynamic weight calculation module outputs the fusion weight only when it is determined that adjustment is needed.

5. The intelligent carton stacking method according to claim 1, characterized in that: The second sliding time window is larger than the first sliding time window.

6. Intelligent carton stacking system, characterized by: It includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the intelligent carton stacking method as described in any one of claims 1 to 5.

Images