Intelligent carton stacking method and system

Abstract

The invention relates to the technical field of material carrying, in particular to an intelligent stacking method and system. The method comprises the following steps: acquiring top features of a stack body; stack body contour features are obtained; a first sliding time window is created, the first sliding time window comprises stack body top features and stack body contour features after a plurality of continuous stacking operations, and a first feature matrix is formed; inputting the first feature matrix into a dynamic weight calculation module to output a fusion weight; a second sliding time window is created, the second sliding time window comprises stack body top features and stack body contour features after a plurality of continuous stacking operations, weighted fusion is carried out according to the obtained fusion weight, and a second feature matrix is formed; and inputting the second characteristic matrix into a position compensation amount calculation module, and calculating to obtain the position compensation amount of the current carton group unit when the carton group unit is placed. By means of the carton stacking device, cartons can be stacked better.

Description

Technical Field

[0001] This application relates to the technical field of material handling, and particularly to an intelligent stacking method and system for cartons. Background Art

[0002] Currently, with the rapid development of logistics automation and intelligent manufacturing, automated stacking technology has been widely applied in fields such as warehousing, logistics, and manufacturing. The automated stacking system can significantly improve operation efficiency and reduce labor costs.

[0003] The mainstream automated stacking systems mainly adopt a rigid positioning method based on preset coordinates. Such methods are for standard goods with regular sizes. Usually, the ideal stacking position coordinates are preset, and the robot performs the stacking operation according to a fixed placement trajectory and the position of the goods.

[0004] However, in the stacking operation of a carton group composed of multiple unopened cartons, there are still some problems in the existing technology, resulting in poor stacking effects for this type of carton. Summary of the Invention

[0005] To solve the above technical problems or at least partially solve the above technical problems, this application provides an intelligent carton stacking method, which can stack cartons better.

[0006] In a first aspect, this application proposes an intelligent carton stacking method, and the intelligent carton stacking method includes the following steps: Stack each carton group unit composed of multiple unopened cartons onto a pallet one by one according to a preset stacking path to form a stack; the preset stacking path includes the target placement position of each carton group unit; Before each placement of a carton group unit, the following steps are further included: Collect data on the top of the stack through an overhead vision system to obtain the top characteristics of the stack; Collect data on the side of the stack through a lateral vision system to obtain the contour characteristics of the stack; Create a first sliding time window, which includes the top characteristics and contour characteristics of the stack after the most recent consecutive multiple stacking operations to form a first feature matrix; Input the first feature matrix into a dynamic weight calculation module to output the fusion weight of the top characteristics of the stack and the fusion weight of the contour characteristics of the stack; Create a second sliding time window, which includes the top characteristics and contour characteristics of the stack after the most recent consecutive multiple stacking operations, and perform weighted fusion on the top characteristics and contour characteristics of the stack within the second sliding time window according to the obtained fusion weights to form a second feature matrix; Input the second feature matrix into the position compensation amount calculation module to calculate the position compensation amount of the current carton group unit during placement; Adjust the target placement position of the current carton group unit according to the position compensation amount.

[0007] Optionally, before placing the carton group unit each time, the following steps are further included: After the carton group unit is grasped, obtain the contour features of the carton group unit through the lateral vision system; Input the contour features of the carton group unit into the carton group stability judgment module to determine whether the carton group unit is in a stable or unstable state; If it is determined that the current carton group unit is in a stable state, place the carton group unit in a direct release manner; If it is determined that the current carton group unit is in an unstable state, place the carton group unit in a compaction release manner to prompt the current stack surface to expose its true state; Wherein, the unstable state includes the bulging state of the carton group unit and the internal loose state of the carton group unit.

[0008] Optionally, the carton group stability judgment module includes: The carton group unit image segmentation unit is used to segment the side image area of the current carton group unit from the image obtained by the lateral vision system; The first convolutional neural network is used to receive the side image area as input and output the state classification result of the carton group unit; The classification results include: normal state, bulging state, and internal loose state.

[0009] Optionally, the dynamic weight calculation module includes: The first recurrent neural network is used to receive the first feature matrix as input; The first recurrent neural network encodes the time series features in the first feature matrix to output a historical state encoding vector; The weight generation unit composed of a fully connected neural network, the weight generation unit is connected to the output end of the first recurrent neural network, and is used to calculate and output the fusion weight of the stack top features and the fusion weight of the stack contour features according to the historical state encoding vector.

[0010] Optionally, the position compensation amount calculation module includes: The second recurrent neural network is used to receive the second feature matrix as input; The second recurrent neural network encodes the weighted fusion time series features in the second feature matrix to output a compensation decision encoding vector; A compensation amount calculation unit composed of a fully connected neural network, the compensation amount calculation unit is connected to the output end of the second recurrent neural network, and is used to calculate and output the position compensation amount of the current carton group unit in three-dimensional directions according to the compensation decision coding vector.

[0011] Optionally, the intelligent carton stacking method further includes: The dynamic weight calculation module also judges whether it is necessary to adjust the placement position of the current carton group unit according to the first feature matrix; When it is judged that adjustment is needed, the dynamic weight calculation module outputs the fusion weight.

[0012] Optionally, the second sliding time window is greater than the first sliding time window.

[0013] In a second aspect, the present application proposes an intelligent carton stacking system, including a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the intelligent carton stacking method as described in any one of the first aspects.

[0014] The technical solution provided by the present application has the following advantages compared with the prior art: One of its beneficial effects and its working principle lie in: In the application scenario of the present application, the carton stacking process is a process of bundling several unfolded cartons into groups with binding tapes, forming carton group units, and then placing them on the pallet one by one to form a multi-layer stack.

[0015] However, since the carton group unit is formed by bundling several unfolded cartons into groups with binding tapes, the surface of the carton is relatively smooth, and there may be a certain amount of slippage between the unfolded cartons due to vibration, and a spring-like state will also be formed between the cartons. These factors will inevitably lead to that with the stacking of carton group units layer by layer, the actual position and shape of each stacked carton group unit after placement may have a slight but cumulative deviation from the theoretical preset position, and this cumulative deviation will increase continuously with the increase of the stacking height, ultimately leading to a decrease in the stability and accuracy of the entire stack.

[0016] And if too much attention is paid to the characteristics of the top of the stack, the overall stack contour characteristics of the entire stack may be ignored, making the stack prone to overall tilt or even overturn during long-term stacking. On the contrary, if too much attention is paid to the overall situation, the local placement accuracy may be sacrificed, resulting in an increasingly obvious problem of unevenness at the top layer, ultimately reducing the effective bearing capacity of the carton group.

[0017] Therefore, the present application proposes a carton stacking method. This method collects the top features of the stack through an overhead vision system, and at the same time obtains the overall contour features of the stack through a lateral vision system. Then, this information is respectively input into two hierarchically progressive sliding time windows for time-series analysis and processing.

[0018] First, the first sliding time window is responsible for collecting historical feature data after multiple consecutive recent stacking operations and forming a first feature matrix. Subsequently, the dynamic weight calculation module calculates the optimal fusion weights of the top features of the stack and the contour features of the stack based on the current state of the stack and the historical evolution trend of the stack.

[0019] This dynamic weight mechanism can adaptively adjust the attention allocation to local accuracy and overall stability according to different stages of the stacking process and the real-time changes in the stability of the stack.

[0020] Furthermore, in the second sliding time window, the top features of the stack and the contour features of the stack are weighted and fused according to the calculated fusion weights, forming a second feature matrix that comprehensively considers historical changes, the current state, and the future evolution trend introduced through the dynamic attention mechanism.

[0021] The final position compensation amount calculation module calculates the position compensation amount required for the current carton group unit during placement based on this second feature matrix, so as to better adjust the target placement position.

[0022] Therefore, the present application can effectively eliminate cumulative position deviations, and through the dynamic weight adjustment mechanism, it achieves a balance between placement accuracy and overall stability, improving the stability of carton group stacking.

[0023] Therefore, the present application can better stack carton groups.

[0024] The second beneficial effect and its working principle are as follows: When cartons are bundled, there is a certain degree of randomness, and the stability of each carton group unit itself is limited by the tightness of the bundling straps used during carton bundling.

[0025] Inevitably, when carton group units are bundled, there will be situations of over-tight bundling or over-loose bundling. If the bundling is over-tight, the two sides of the cartons in the carton group unit will be tied up by the bundling straps, resulting in the middle part of the carton group unit being slightly bulged due to the force, making the spring tendency of the carton group unit more pronounced; if the bundling is over-loose, there will be a greater degree of mutual sliding or displacement between the cartons in the carton group unit.

[0026] When the carton group shows a serious bulging or loose internal state, if it is still placed in the general direct release manner, this unstable state is often masked until after multi-layer stacking, and it suddenly appears on a certain layer after being pressed by multiple layers of cartons, resulting in problems such as sudden changes in the outline of the stack and the top surface of the stack.

[0027] Therefore, this application proposes an active intervention method of compaction and release, that is, when it is detected that the carton group unit has the above-mentioned unstable state, after moving the carton group unit to the stack surface, the current carton group unit is also pressed to expose its true stack surface form in advance, so that this change can be sensed before the next placement, avoiding the masking of this change due to the stacking of multiple layers of carton group units, and avoiding the internal stress and slip mutation of the stack, thereby improving the overall stacking stability. Brief Description of the Drawings

[0028] Figure 1 It is a schematic flow chart of the intelligent carton stacking method provided by the embodiment of this application. Detailed Embodiments

[0029] Next, the technical solutions in this application will be described in conjunction with the drawings.

[0030] Many specific details are set forth in the following description in order to fully understand this application, but this application can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of this application, rather than all the embodiments. It should be noted that, without conflict, the embodiments of this application and the features in the embodiments can be combined with each other.

[0031] In the first aspect, referring to Figure 1 as shown, this application proposes an intelligent carton stacking method, and the intelligent carton stacking method includes the following steps: S1: According to the preset stacking path, stack the carton group units composed of multiple unfolded cartons one by one onto the pallet to form a stack; the preset stacking path includes the target placement position of each carton group unit.

[0032] Specifically, the preset stacking path includes the three-dimensional positions of each carton group unit during placement in sequence to complete a stacking operation. In the embodiment of this application, the three-dimensional position is the target placement position where the center point of the carton group unit needs to move.

[0033] An industrial robotic arm equipped with a pneumatic suction cup gripper is used to grab the carton group units one by one from the assembly line and perform the stacking operation of the carton group units according to this sequence of three-dimensional positions.

[0034] The sequence of three-dimensional positions for placing the carton group units can be manually configured in advance by humans.

[0035] S2: Before each placement of the carton group unit, the following steps are further included: After the carton group unit is grasped, the contour features of the carton group unit are obtained through a lateral vision system.

[0036] Among them, before placing the carton group unit, the industrial robotic arm will first move the carton group unit to a preset position so that the lateral vision system can capture the contour features of the carton group unit.

[0037] Specifically, in the embodiment of the present application, the Canny edge detection algorithm is used to extract the contour features of the carton group unit.

[0038] The contour features of the carton group unit are input into the carton group stability judgment module to determine whether the carton group unit is in a stable or unstable state; If it is determined that the current carton group unit is in a stable state, the carton group unit is placed in a direct release manner.

[0039] Among them, the meaning of direct release is: After the industrial robotic arm moves the carton group unit to the target placement position, the adsorption function of the pneumatic suction cup is immediately turned off, so that the carton group unit naturally falls and fits on the stack surface. No additional pressure or residence time is applied during the entire release process to ensure the maximization of the placement efficiency.

[0040] If it is determined that the current carton group unit is in an unstable state, the carton group unit is placed in a compaction release manner to prompt the current stack surface to expose its true state.

[0041] Among them, the meaning of compaction release is: After the industrial robotic arm moves the carton group unit to the target placement position, the pneumatic suction cup is turned off, so that the carton group unit naturally falls and fits on the stack surface. Then the pneumatic suction cup is pressed down to contact the carton group unit and a contact pressure is applied, and the pressure magnitude is 15 - 20 N, so as to drive the stack surface to deform towards a stable state under the action of an external force.

[0042] Among them, the unstable state of the current carton group unit includes the bulging state of the carton group unit and the internal loose state of the carton group unit.

[0043] The bulging state of the carton group unit is that the bundling tape is tied too tightly, and both sides of the cartons in the carton group unit will be tied up by the bundling tape, resulting in a slight bulge in the middle of the carton group unit.

[0044] The internal loose state of the carton group unit is that the bundling tape is tied too loosely, then there will be a larger-scale mutual slip or displacement between the cartons in the carton group unit, manifested as an offset state of the whole carton group when the carton group unit is lifted.

[0045] Specifically, in the embodiments of the present application, the carton group stability judgment module includes: A carton group unit image segmentation unit, configured to segment the side image area of the current carton group unit from the image obtained by the lateral vision system; A first convolutional neural network, configured to receive the side image area as input and output the state classification result of the carton group unit; The classification results include: normal state, bulging state, and internal loose state.

[0046] Specifically, in the embodiments of the present application, the training process of the first convolutional neural network is as follows: Collect the side images of multiple carton group units in different states as the training data set, including normal state images, bulging state images, and internal loose state images, and ensure the label accuracy by manual annotation.

[0047] Use the side images of the carton group units in the training data set as the input of the first convolutional neural network, and use the corresponding state category labels (normal state, bulging state, internal loose state) as labels to train the first convolutional neural network for classifying the state of the carton group units. The cross-entropy loss function is used in the training process.

[0048] S3: Collect data on the top of the stack through the overhead vision system to obtain the top features of the stack; The overhead vision system uses an Intel depth camera, which is installed 9 meters above the stack (based on the initial setting position of the stack pallet), and the frame rate is 30fps. This camera is used to obtain the RGB image and depth information of the top of the stack, and extract the three-dimensional morphological features of the top of the stack through point cloud processing technology.

[0049] Specifically, in the embodiments of the present application, after each shot by the overhead vision system, image denoising and depth map completion processing are first performed. The bilateral filter is used to denoise the RGB image. Then, the fast bilateral stereo matching algorithm is used to complete the missing depth information and convert it into three-dimensional point cloud data.

[0050] Identify the top plane of the stack through the RANSAC plane detection algorithm.

[0051] Based on the detected plane, the top of the stack is divided into a 20×20 grid, and the average height value of each grid is calculated to form a 400-dimensional top feature vector of the stack.

[0052] Collect data on the side of the stack through the lateral vision system to obtain the stack contour features; The lateral vision system uses two Baumer industrial cameras, which are respectively installed at a height of 1.8 meters on both sides of the stack to obtain the contour morphological features of the entire front and side of the stack.

[0053] Specifically, in the embodiment of the present application, the Canny edge detection algorithm is used to extract the contour of the stack body. The extracted contour is processed by the Douglas-Peucker algorithm, and finally a 256-dimensional stack body contour feature vector containing the coordinates of the contour key points is formed.

[0054] S4: Create a first sliding time window, where the first sliding time window includes the top feature of the stack body and the contour feature of the stack body after multiple consecutive recent stacking operations, so as to form a first feature matrix; input the first feature matrix into the dynamic weight calculation module to output the fusion weight of the top feature of the stack body and the fusion weight of the contour feature of the stack body.

[0055] Specifically, in the embodiment of the present application, the length of the first sliding time window is set to 8, that is, it includes the feature data after the recent 8 stacking operations.

[0056] After each layer of stacking operation is completed, the system combines the latest top feature of the stack body (400 dimensions) and the contour feature of the stack body (256 dimensions) into a 656-dimensional comprehensive feature vector, and stores it in the first sliding window in chronological order to form an 8×656 first feature matrix.

[0057] Specifically, the dynamic weight calculation module is a first composite neural network composed of a first recurrent neural network and a fully connected neural network: The first recurrent neural network is used to receive the first feature matrix as input; The first recurrent neural network encodes the time series features in the first feature matrix to output a historical state encoding vector.

[0058] The historical state encoding vector can represent the correlation between the local accuracy represented by the top feature of the stack body and the global stability represented by the contour feature of the stack body in the stacking history included in the first sliding time window.

[0059] A weight generation unit composed of a fully connected neural network, the weight generation unit is connected to the output end of the first recurrent neural network, and is used to calculate and output the fusion weight of the top feature of the stack body and the fusion weight of the contour feature of the stack body according to the historical state encoding vector.

[0060] Specifically, in the embodiment of the present application, the first recurrent neural network adopts a bidirectional LSTM architecture, and the specific parameters are as follows: Input dimension: 656 dimensions (400 dimensions of the top feature of the stack body + 256 dimensions of the contour feature of the stack body); Hidden layer dimension: 256 dimensions; Number of layers: 2 layers of bidirectional LSTM; Activation function: tanh activation function; Output dimension: 512 dimensions (concatenated by the forward and backward outputs of the bidirectional LSTM); The weight matrices of the forget gate, input gate, and output gate of the LSTM are initialized using the Xavier initialization method, and the bias vectors are initialized to zero.

[0061] The weight generation unit consists of three fully connected neural networks: The first layer: takes 512 dimensions as input and outputs 256 dimensions, with the activation function being ReLU; The second layer: takes 256 dimensions as input and outputs 128 dimensions, with the activation function being ReLU; The third layer: takes 128 dimensions as input and outputs 3 dimensions, with the activation function being Sigmoid; The 3D vector output represents: The fusion weight output representation vector, with values of 0 or 1.

[0062] The fusion weight of the top features of the stack and the fusion weight of the stack contour features , and through normalization to ensure + = 1.

[0063] Among them, when the value of the fusion weight output representation vector is 1, the fusion weight is output, so as to determine whether it is necessary to adjust the placement position of the current carton group unit.

[0064] By outputting the fusion weight output representation vector, the weight generation unit realizes the judgment of whether it is necessary to adjust the placement position of the current carton group unit according to the first feature matrix. That is, the weight generation unit obtains the fusion weight output representation vector according to the historical state encoding vector obtained through the first feature matrix.

[0065] When the value of this representation vector is 1, it indicates that the placement position of the current carton group unit needs to be adjusted, and then the corresponding fusion weight is output.

[0066] When the value of the representation vector is 0, it indicates that the placement position of the current carton group unit does not need to be adjusted, and then the corresponding fusion weight is not output.

[0067] Through this conditional judgment mechanism, the present application realizes the on-demand calculation strategy. Only when the fusion weight output representation vector indicates that the placement position of the current carton group unit needs to be adjusted (when the value is 1), will the subsequent calculation process of the relatively large position compensation amount be triggered, thereby effectively reducing the calculation burden of the system.

[0068] Specifically, the dynamic weight calculation module is trained through the following steps: Collect the first feature matrix sequences under multiple different stacking scenarios and their corresponding weight annotation data as the training set of the dynamic weight calculation module. Each set of data includes the first feature matrix sequence, as well as the fused weight output representation vector (0 or 1), the stack top feature fusion weight (a floating point number between 0 and 1), and the stack contour feature fusion weight (a floating point number between 0 and 1) annotated by the stacking expert according to the stack stability condition, ensuring that the weight annotation reflects the balance requirements of the top local accuracy and global stability in actual stacking.

[0069] Use the first feature matrix sequences in the training set as the input of the first composite network. Use the corresponding fused weight output representation vector, stack top feature fusion weight, and stack contour feature fusion weight as the labels of the first composite network, and thus train the dynamic weight calculation module. The mean square error loss function combined with the binary cross-entropy loss function is used in the training process.

[0070] S5: Create a second sliding time window, where the second sliding time window is larger than the first sliding time window. The second sliding time window includes the stack top features and stack contour features after multiple consecutive recent stacking operations. Weightedly fuse the stack top features and stack contour features within the second sliding time window according to the fusion weights obtained above to form a second feature matrix; input the second feature matrix into the position compensation amount calculation module to calculate the position compensation amount when the current carton group unit is placed.

[0071] Specifically, in the embodiment of the present application, the length of the second sliding time window is set to 12, including the feature data of the recent 12 stacking operations. This window also stores a 656-dimensional comprehensive feature vector to form a 12×656 feature matrix. According to the fusion weights output in S4, weightedly fuse the 12×656 feature matrix in the second sliding time window to form a 12×656 second feature matrix.

[0072] Specifically, the weighted fusion process is carried out according to the following formula: For the i-th stacking operation (i = 1, 2,..., 12) in the second sliding time window, the fused feature vector [i] = × stack top feature [i] + × stack contour feature [i], where and are respectively the stack top feature fusion weight and stack contour feature fusion weight output by the dynamic weight calculation module in S4.

[0073] Specifically, the position compensation amount calculation module is a second composite neural network composed of a second recurrent neural network and a fully connected neural network: A second recurrent neural network for receiving the second feature matrix as an input; The second recurrent neural network encodes the weighted fusion time series features in the second feature matrix to output a compensation decision encoding vector; A compensation amount calculation unit composed of a fully connected neural network, the compensation amount calculation unit is connected to the output end of the second recurrent neural network, and is used to regressively calculate and output the position compensation amount of the current carton group unit in three-dimensional directions according to the compensation decision encoding vector.

[0074] Specifically, the second recurrent neural network also adopts a bidirectional LSTM architecture, and the specific parameters are as follows: Input dimension: 656 dimensions (the comprehensive feature dimension after weighted fusion); Hidden layer dimension: 512 dimensions; Number of layers: 3 layers of bidirectional LSTM; Activation function: tanh activation function; Output dimension: 1024 dimensions (concatenated by the forward and backward outputs of the bidirectional LSTM); The compensation amount calculation unit adopts a four-layer fully connected neural network: The first layer: Input 1024 dimensions, output 512 dimensions, and the activation function is ReLU; The second layer: Input 512 dimensions, output 256 dimensions, and the activation function is ReLU; The third layer: Input 256 dimensions, output 128 dimensions, and the activation function is ReLU; The fourth layer: Input 128 dimensions, output 3 dimensions, and the activation function is linear; The output 3D vector represents the position compensation amounts in the X, Y, and Z directions respectively.

[0075] Specifically, the position compensation amount calculation module is obtained through the following steps: Collect the second feature matrices under multiple different stacking scenarios and their corresponding position compensation amount annotation data as the training set of the second composite neural network, where each group of data includes the second feature matrix and the three-dimensional position compensation amounts (compensation values in the X, Y, and Z directions) annotated by the stacking expert according to the actual stacking deviation situation.

[0076] Use the second feature matrix in the training set as the input of the second composite neural network; Use the corresponding three-dimensional position compensation amount annotation data as the label of the second composite neural network, and thus train the position compensation amount calculation module. The mean square error loss function is used in the training process.

[0077] S6: Adjust the target placement position of the current carton group unit according to the position compensation amount.

[0078] Specifically, the system adjusts the preset target placement position of the current carton group unit according to the three-dimensional compensation amounts (ΔX, ΔY, ΔZ) output by the position compensation amount calculation module.

[0079] The adjusted target placement position X' = the original target placement position X + ΔX; The adjusted target placement position Y' = the original target placement position Y + ΔY; The adjusted target placement position Z' = the original target placement position Z + ΔZ.

[0080] Among them, ΔX and ΔY respectively correspond to the position compensation in the horizontal plane, and ΔZ corresponds to the height compensation in the vertical direction.

[0081] After receiving the adjusted target placement position, the industrial robotic arm moves the central position of the currently grasped carton group unit to the adjusted target placement position, and then performs the placement operation.

[0082] In summary, the technical solution provided by this application has the following advantages compared with the prior art: One of its beneficial effects and its working principle lie in: In the application scenario of this application, the carton stacking process is a process of bundling several unopened cartons into groups with bundling straps, forming carton group units, and then placing them on the pallet one by one unit by unit to form a multi-layer stack.

[0083] However, since the carton group unit is formed by bundling several unopened cartons with bundling straps, the surface of the carton is relatively smooth, and there may be a certain amount of slippage between the unopened cartons due to vibration, and a state similar to a spring will also be formed between the cartons. These factors will inevitably lead to that with the gradual stacking of carton group units, the actual position and shape of each stacked carton group unit after placement may have a slight but cumulative deviation from the theoretical preset position, and this cumulative deviation will continuously increase with the increase of the stacking height, ultimately resulting in a decrease in the stability and accuracy of the entire stack.

[0084] And if too much attention is paid to the characteristics of the top of the stack, the stack contour characteristics of the overall stack may be ignored, making the stack prone to overall tilt or even overturn during long-term stacking. On the contrary, if too much attention is paid to the overall, the local placement accuracy may be sacrificed, resulting in an increasingly obvious problem of unevenness at the top layer, ultimately reducing the effective bearing capacity of the carton group.

[0085] Therefore, the present application proposes a method for stacking cartons. This method collects the top features of the stack body through an overhead vision system, and at the same time obtains the overall contour features of the stack body through a lateral vision system. Then, this information is respectively input into two hierarchically progressive sliding time windows for time-series analysis and processing.

[0086] First, the first sliding time window is responsible for collecting historical feature data after multiple consecutive stacking operations and forming a first feature matrix. Subsequently, the dynamic weight calculation module calculates the optimal fusion weights of the top features of the stack body and the contour features of the stack body based on the current state of the stack body and the historical evolution trend of the stack body.

[0087] This dynamic weight mechanism can adaptively adjust the attention allocation to local accuracy and overall stability according to different stages of the stacking process and the real-time changes in the stability of the stack body.

[0088] Furthermore, in the second sliding time window, the top features of the stack body and the contour features of the stack body are weighted and fused according to the calculated fusion weights, forming a second feature matrix that comprehensively considers historical changes, current state, and future evolution trends introduced through the dynamic attention mechanism.

[0089] Finally, the final position compensation amount calculation module calculates the position compensation amount required when placing the current carton group unit based on this second feature matrix, so as to better adjust the target placement position.

[0090] Therefore, the present application can effectively eliminate cumulative position deviations, and through the dynamic weight adjustment mechanism, it realizes the balance between placement accuracy and overall stability, improving the stability of carton group stacking.

[0091] Therefore, the present application can better stack carton groups.

[0092] The second beneficial effect and its working principle are as follows: When bundling cartons, there is a certain degree of randomness, and the stability of each carton group unit itself is limited by the tightness of the bundling straps used during carton bundling.

[0093] Inevitably, when bundling carton group units, there will be situations where the bundling is too tight or too loose. If the bundling is too tight, the two sides of the cartons in the carton group unit will be tied up by the bundling straps, resulting in a slight bulge in the middle of the carton group unit, making the spring tendency of the carton group unit more pronounced; if the bundling is too loose, there will be a greater degree of mutual sliding or displacement inside the cartons.

[0094] When the carton group appears in a severely bulged or internally loose state, if it is still placed in a general direct release manner, this unstable state is often masked until, after multi-layer stacking, it suddenly appears on a certain layer after being pressed by multiple layers of cartons, resulting in problems such as sudden changes in the stack contour and the top stack surface.

[0095] Therefore, this application proposes an active intervention method of compaction and release, that is, when it is detected that the carton group unit has the above-mentioned unstable state, after moving the carton group unit to the stack surface, the current carton group unit is also pressured to expose the true stack surface form in advance, so that this change can be sensed before the next placement, avoiding the masking of this change due to the stacking of multiple layers of carton group units, and avoiding the internal stress and slip mutation of the stack, thereby improving the overall stacking stability.

[0096] In a second aspect, an embodiment of this application proposes an intelligent carton stacking system, including a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the intelligent carton stacking method as described in the above embodiment.

[0097] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Additionally, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element. Moreover, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. And, in the description of the embodiments of this application, "a plurality" means two or more than two.

[0098] The above are only specific embodiments of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments described herein, but rather will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. Intelligent carton stacking method, the intelligent carton stacking method comprising the following steps: Stacking carton group units composed of multiple unexpanded cartons onto a pallet one by one according to a preset stacking path to form a stack body; the preset stacking path includes the target placement positions of each carton group unit; Characterized in that, before each placement of a carton group unit, the following steps are further included: Collecting data of the top of the stack body through an overhead vision system to obtain the top characteristics of the stack body; Collecting data of the side of the stack body through a lateral vision system to obtain the contour characteristics of the stack body; Creating a first sliding time window, the first sliding time window including the top characteristics and the contour characteristics of the stack body after the most recent consecutive multiple stacking operations to form a first feature matrix; Inputting the first feature matrix into a dynamic weight calculation module to output the fusion weights of the top characteristics of the stack body and the fusion weights of the contour characteristics of the stack body; Creating a second sliding time window, the second sliding time window including the top characteristics and the contour characteristics of the stack body after the most recent consecutive multiple stacking operations, and performing weighted fusion on the top characteristics and the contour characteristics of the stack body within the second sliding time window according to the obtained fusion weights to form a second feature matrix; Inputting the second feature matrix into a position compensation amount calculation module to calculate the position compensation amount when the current carton group unit is placed; Adjusting the target placement position of the current carton group unit according to the position compensation amount.

2. The intelligent carton stacking method according to claim 1, wherein Before each placement of a carton group unit, the following steps are further included: After the carton group unit is grasped, obtaining the contour characteristics of the carton group unit through a lateral vision system; Inputting the contour characteristics of the carton group unit into a carton group stability judgment module to determine whether the carton group unit is in a stable or unstable state; If it is determined that the current carton group unit is in a stable state, placing the carton group unit in a direct release manner; If it is determined that the current carton group unit is in an unstable state, placing the carton group unit in a compaction release manner to prompt the current stack surface to expose its true state; Wherein, the unstable state includes the bulging state of the carton group unit and the internal loose state of the carton group unit.

3. The intelligent carton stacking method according to claim 2, wherein, The carton group stability judgment module includes: A carton group unit image segmentation unit for segmenting the side image area of the current carton group unit from the image obtained by the lateral vision system; A first convolutional neural network for receiving the side image area as input and outputting the state classification result of the carton group unit; The classification results include: normal state, bulging state, and internal loose state.

4. The intelligent carton stacking method according to claim 1, characterized in that, The dynamic weight calculation module includes: A first recurrent neural network for receiving the first feature matrix as input; The first recurrent neural network encodes the time series features in the first feature matrix to output a historical state encoding vector; A weight generation unit composed of a fully connected neural network, the weight generation unit being connected to the output end of the first recurrent neural network for calculating and outputting the fusion weights of the top characteristics of the stack body and the fusion weights of the contour characteristics of the stack body according to the historical state encoding vector.

5. The intelligent carton stacking method according to claim 1, characterized in that, The position compensation amount calculation module includes: A second recurrent neural network for receiving the second feature matrix as an input; The second recurrent neural network encodes the weighted fusion time series features in the second feature matrix to output a compensation decision encoding vector; A compensation amount calculation unit composed of a fully-connected neural network, the compensation amount calculation unit being connected to the output end of the second recurrent neural network, and being configured to regressively calculate and output the position compensation amount of the current carton group unit in three-dimensional directions according to the compensation decision encoding vector.

6. The intelligent carton stacking method according to claim 1, wherein The intelligent carton stacking method further includes: The dynamic weight calculation module further determines whether to adjust the placement position of the current carton group unit according to the first feature matrix; When it is determined that adjustment is required, the dynamic weight calculation module outputs the fusion weight.

7. The intelligent carton stacking method according to claim 1, wherein The second sliding time window is greater than the first sliding time window.

8. Intelligent carton stacking system, characterized in that, Comprising a processor and a memory, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the intelligent carton stacking method according to any one of claims 1-7.

Images