Stack type graph generation method and device, palletizing robot, and robot palletizing method

Abstract

The application relates to the technical field of stacking control, and provides a stacking type graph generation method and device, a stacking robot and a robot stacking method. The method comprises the following steps: obtaining an original stacking type graph based on current frame depth information after each stacking operation is completed; performing difference value processing on the original stacking type graph of a previous frame and the original stacking type graph of a current frame to obtain a difference value graph; performing contour detection on the difference value graph to obtain position information of a newly added stacking object in the difference value graph; updating height values in an output stacking type graph of the previous frame according to the position information, and outputting the updated stacking type graph as an output stacking type graph of the current frame. The method can obtain a more detailed stacking type graph by using the information of the previous frame and the current frame, is beneficial to stacking of small objects, and can further improve the volume rate of stacking.

Description

Technical Field

[0001] This application relates to the field of palletizing control technology, and in particular to a method, apparatus, palletizing robot, and robot palletizing method for generating pallet pattern diagrams. Background Technology

[0002] Robots are being used more and more widely, for example, replacing human workers in industrial logistics and intelligent sorting scenarios for palletizing operations, thus saving significant labor costs. Palletizing refers to the process of stacking materials on pallets according to certain rules, and palletizing planning algorithms are one of the keys to achieving efficient work for palletizing robots. As the input to the palletizing planning algorithm, the pallet shape analysis map is an important factor affecting the performance of the algorithm. Existing pallet shape analysis methods mainly use the depth map of the current frame to calculate the point cloud, then analyze the space occupancy of the point cloud within the stacking space, and thus determine which space is occupied, thereby obtaining the pallet shape analysis map.

[0003] However, current methods for generating stacking analysis maps have the following drawbacks: because the space covered by the depth camera's field of view is a cone-shaped space, after calculating the point cloud information from the depth map, taking cardboard boxes as an example, a certain range of blind spots will occur at the bottom of the topmost cardboard box. Figure 1 As shown, the gray areas at both ends represent blind spots. These blind spots will result in many regions of unknown depth appearing in the stacking diagram, which is detrimental to the spatial planning of the stacking planning algorithm. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, palletizing robot, and robot palletizing method for generating pallet diagrams.

[0005] In a first aspect, embodiments of this application provide a method for generating a stack pattern diagram, including:

[0006] After each palletizing operation is completed, the original pallet shape diagram based on the current frame depth information is obtained;

[0007] The original stacking pattern diagram of the previous frame and the original stacking pattern diagram of the current frame are subjected to difference processing to obtain a difference diagram;

[0008] Contour detection is performed on the difference map to obtain the position information of the newly added stacking object in the difference map;

[0009] The height value in the output stack pattern diagram of the previous frame is updated based on the location information, and the updated stack pattern diagram is output as the output stack pattern diagram of the current frame.

[0010] In some embodiments, prior to the initial palletizing operation, the pallet diagram generation method further includes:

[0011] Obtain the three-dimensional spatial position of the empty pallet to obtain information on each corner point of the pallet's bottom surface;

[0012] Divide the planar area enclosed by all corner points into a grid, select one corner point as the origin, and set the initial height value of each grid to zero to obtain the initial stacking pattern.

[0013] In some embodiments, obtaining the original stacking pattern diagram based on the current frame depth information after each stacking operation includes:

[0014] After each palletizing operation is completed, the depth map of the current frame is obtained through the depth camera;

[0015] The depth map is converted into a point cloud, and the converted point cloud is projected onto the plane containing the bottom surface of the pallet.

[0016] The target point cloud located in the planar area on the bottom surface of the pallet after projection is selected, and the grid coordinates of each target point cloud projected into the divided grid are calculated to obtain the original pallet shape diagram of the current frame; wherein, the height value of each grid in the original pallet shape diagram is determined according to the depth value corresponding to the relevant target point cloud.

[0017] In some embodiments, the height value of each grid in the original stacking diagram is determined based on the depth value corresponding to the relevant target point cloud, including:

[0018] Calculate the height value of each target point cloud projected onto the planar region on the bottom surface of the pallet, and based on the grid coordinates, take the maximum height value of the points located in the same grid as the height value of the current grid.

[0019] In some embodiments, the contour detection of the difference map is performed prior to:

[0020] For grids in the difference map whose height difference is less than a preset distance threshold, the height difference is set to zero to obtain the first optimized difference map.

[0021] In some embodiments, the stack pattern generation method further includes:

[0022] The difference map after the first optimization is filtered using a sliding window to obtain the difference map after the second optimization.

[0023] In some embodiments, filtering the first optimized difference map using a sliding window includes:

[0024] The difference map is slid sequentially using a K*K window to count the number of grids with non-zero height differences within the sliding window at the current position.

[0025] If the ratio of the number of non-zero grids to the total number of grids is greater than a preset ratio threshold, the height difference at the current position is retained; otherwise, the height difference at the current position is set to zero.

[0026] In some embodiments, the location information is the minimum bounding rectangle of the newly added stacking object in the difference map; updating the height value of the output stacking map of the previous frame according to the location information includes:

[0027] Obtain the height value of the minimum bounding rectangle region corresponding to the original stacking diagram of the current frame;

[0028] If the height value is greater than the height value at the same position in the output stack pattern of the previous frame, then the height value is updated; otherwise, the height value in the output stack pattern of the previous frame remains unchanged.

[0029] Secondly, embodiments of this application provide a robotic palletizing method, including:

[0030] The stack diagram is generated using the aforementioned stack diagram generation method;

[0031] The stacking diagram is input into the stacking planning algorithm to perform stacking planning and obtain stacking parameters;

[0032] Perform the corresponding palletizing operation according to the palletizing parameters.

[0033] Thirdly, embodiments of this application provide a stack pattern diagram generation apparatus, comprising:

[0034] The acquisition module is used to obtain the original stacking pattern diagram based on the current frame depth information after each stacking operation is completed.

[0035] The difference module is used to perform difference processing on the original stacking pattern diagram of the previous frame and the original stacking pattern diagram of the current frame to obtain a difference diagram.

[0036] The detection module is used to perform contour detection on the difference map to obtain the position information of the newly added stacking object in the difference map;

[0037] The update output module is used to update the height value of the output stack pattern diagram of the previous frame according to the position information, and output the updated stack pattern diagram as the output stack pattern diagram of the generated current frame.

[0038] Fourthly, embodiments of this application provide a robotic palletizing device, comprising:

[0039] The stack diagram generation module is used to generate a stack diagram using the aforementioned stack diagram generation method.

[0040] The stacking pattern planning module is used to take the stacking pattern diagram as input to the stacking pattern planning algorithm and perform stacking pattern planning to obtain stacking parameters;

[0041] The palletizing execution module is used to perform corresponding palletizing operations according to the palletizing parameters.

[0042] Fifthly, embodiments of this application provide a palletizing robot, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the pallet pattern generation method or the robot palletizing method.

[0043] Sixthly, embodiments of this application provide a readable storage medium storing a computer program that, when executed on a processor, implements the pallet pattern generation method or the robot palletizing method.

[0044] The embodiments of this application have the following beneficial effects:

[0045] The stacking pattern generation method proposed in this application obtains the position information of newly added stacking objects in the difference map by combining the original stacking pattern of the previous frame and the original stacking pattern of the current frame and performing difference processing. Then, the height value of the output stacking pattern of the previous frame is updated according to the position information to generate the output stacking pattern of the current frame. This method can obtain a more refined stacking pattern by utilizing information from previous and next frames, reducing the impact of blind spots and improving the stacking volume ratio. It is especially beneficial for stacking small objects, such as small cardboard boxes, which can be placed in narrow spaces such as gaps, further improving stacking efficiency. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A schematic diagram of the blind spot in the field of view of a depth camera is shown;

[0048] Figure 2 A first flowchart of the stack pattern diagram generation method according to an embodiment of this application is shown;

[0049] Figure 3 A second flowchart of the stack pattern diagram generation method according to an embodiment of this application is shown;

[0050] Figure 4 A schematic diagram of a rectangular pallet is shown;

[0051] Figure 5 A third flowchart of the stack type diagram generation method according to an embodiment of this application is shown;

[0052] Figure 6 A fourth flowchart of the stack type diagram generation method according to an embodiment of this application is shown;

[0053] Figure 7 A flowchart of a robot palletizing method according to an embodiment of this application is shown;

[0054] Figure 8 This paper shows a schematic diagram of a stack pattern generation device according to an embodiment of the present application;

[0055] Figure 9 A schematic diagram of a robotic palletizing device according to an embodiment of this application is shown.

[0056] Explanation of key component symbols:

[0057] 100-Package pattern diagram generation device; 110-Acquisition module; 120-Differential module; 130-Detection module; 140-Update output module; 200-Robot palletizing device; 210-Package pattern diagram generation module; 220-Package pattern planning module; 230-Packaging execution module. Detailed Implementation

[0058] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0059] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0060] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0061] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0062] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0063] This application proposes a method for generating stacking pattern diagrams. It utilizes the difference between depth maps of consecutive frames to obtain a difference map, and then uses the height information in the difference map to update the stacking pattern diagram of the previous frame, thereby obtaining the stacking pattern diagram of the current frame. This method effectively solves the problem of blind spots in existing solutions, resulting in more accurate stacking pattern diagrams, which in turn helps to improve the upper limit of stacking planning algorithms.

[0064] The following examples illustrate the method for generating stack diagrams.

[0065] Figure 2 A flowchart illustrating a method for generating a stack pattern diagram according to an embodiment of this application is shown. Exemplarily, the method includes the following steps:

[0066] S110: After each palletizing operation is completed, obtain the original pallet pattern diagram based on the current frame depth information.

[0067] It is understood that the palletizing operation described above refers to the robot sequentially stacking objects onto a pallet according to the stacking instructions. These objects can include, but are not limited to, relatively regular objects such as cardboard boxes and bagged packages; this is not a limitation here. In this embodiment, the shape of the pallet is not limited. For example, based on the planar shape of the bottom of the pallet, it can include, but is not limited to, rectangles, squares, circles, triangles, rings, etc., and this is not a limitation here.

[0068] As an example, depth information can be acquired using a depth camera after each palletizing operation to obtain updated depth information. Then, point cloud transformation is performed based on the latest depth information to obtain corresponding point cloud information, and finally, point cloud projection is used to obtain the original pallet shape diagram at the corresponding time point.

[0069] It is understood that the original stacking pattern diagram refers to the stacking pattern diagram (Height Map) directly calculated using the depth information collected in real time. This differs from the output stacking pattern diagram mentioned later, which is the final output and is obtained primarily through optimization of the original stacking pattern diagram. The Height Map is an M x N matrix, where each point represents the height of the pallet at that location, and M and N are both positive integers.

[0070] As an alternative, the method further includes positioning the empty pallet before the first palletizing operation to obtain the dimensions of an initial pallet pattern corresponding to the dimensions of the pallet bottom surface.

[0071] like Figure 3 As shown, prior to step S110 above, the method for generating the stack pattern diagram further includes:

[0072] S100: Obtain the three-dimensional spatial position of the empty pallet to obtain the corner information of the bottom surface of the pallet, divide the area enclosed by all corner points into grids, select one corner point as the origin and set the initial height value of each grid to zero to obtain the initial pallet shape diagram.

[0073] As an example, before stacking begins, the empty pallet is positioned. This can be achieved using a depth camera or other positioning devices to obtain the pallet's positional information. The three-dimensional spatial coordinates of each corner point on the pallet's bottom surface are then extracted to obtain the planar area enclosed by all corner points. It can be understood that the shape of this planar area is related to the shape of the pallet's bottom surface. For example, if the pallet is rectangular... Figure 4 As shown, the planar region is formed by the four corner points of the bottom surface.

[0074] Then, the planar region is divided into an MxN grid with a certain resolution, where the initial height of each grid is 0. One corner point is designated as the origin to construct the initial stacking pattern. It can be understood that the size of this stacking pattern is always MxN.

[0075] Next, the initial pallet pattern diagram is fed into the palletizing planning algorithm to obtain the pose of the first object to be palletized, so that the robot can perform the first palletizing operation. Then, after the first palletizing operation is completed, step S110 is executed, which is to obtain the original pallet pattern diagram of the current frame.

[0076] For example, in one implementation, such as Figure 5 As shown, step S110 includes the following sub-steps:

[0077] S111: After each palletizing operation is completed, the depth map of the current frame is obtained through the depth camera.

[0078] For example, a depth camera can be set up above or directly above the pallet. After each palletizing operation, the height of the pallet changes, and the depth camera collects depth information to obtain the current depth map. This current depth map is then used as the depth map of the current frame.

[0079] S112, perform point cloud conversion on the depth map, and project the converted point cloud onto the plane where the bottom surface of the pallet is located.

[0080] S113: Select the target point cloud within the area enclosed by all corner points on the bottom surface of the pallet after projection, and calculate the grid coordinates of each target point cloud projection within the divided grid to obtain the original pallet shape diagram of the current frame. The height value of each grid in the original pallet shape diagram is determined based on the depth value corresponding to the relevant target point cloud.

[0081] Next, the depth information is converted into a point cloud to obtain the point cloud image corresponding to the depth map; then, the point cloud is projected onto the plane containing the bottom surface of the pallet. After the point cloud is projected, the point cloud within the area formed by the projection onto the bottom surface of the pallet can be used as the target to generate the pallet shape map.

[0082] As an example, after filtering out all the target point clouds, the planar coordinates of each target point cloud in the M×N grid can be calculated, as well as the height of these target point clouds from the bottom of the pallet, thus obtaining the original pallet shape map of the current frame (denoted as Map_cur).

[0083] Optionally, when calculating the height value, considering that a single grid in an M×N grid may contain multiple points, the maximum height value of points within the same grid can be used as the height value of the current grid for each grid in the original stacking diagram. It is understood that selecting the maximum height value can avoid issues such as compression or incorrect stacking positions.

[0084] S120, perform difference processing on the original stacking pattern diagram of the previous frame and the original stacking pattern diagram of the current frame to obtain a difference diagram.

[0085] In this embodiment, the original stacking pattern map of the current frame will be post-processed to obtain an optimized stacking pattern map (denoted as Map_out) based on the depth maps of the previous and next frames, which is convenient for the stacking planning algorithm to calculate.

[0086] As an example, by subtracting the original stacking pattern map (Map_cur) calculated in the current frame from the original stacking pattern map (Map_cur) calculated in the previous frame, a difference map of size M×N can be obtained.

[0087] Furthermore, by performing contour detection on the difference map, the position information of the newly added stacked object in the difference map can be fitted, such as the minimum bounding rectangle.

[0088] As a preferred option, such as Figure 6 As shown, before performing contour detection on the difference map, the method further includes denoising and optimizing the difference map. Exemplarily, the method also includes:

[0089] S150, for grids in the difference map whose height difference is less than the preset distance threshold, perform the height difference operation to zero to obtain the difference map after the first optimization.

[0090] For example, if a height value in the difference map is less than a preset distance threshold (e.g., 5mm), the height value at that location is set to 0. This operation aims to remove noise in the difference map caused by minute errors in the depth camera's accuracy.

[0091] S160, use a sliding window to filter the difference map after the first optimization to obtain the difference map after the second optimization.

[0092] Specifically, a K*K window can be used to slide the first optimized difference map sequentially to count the number of grids with non-zero height differences within the sliding window at the current position. Here, K is an integer greater than 0. If the ratio of the number of non-zero grids to the total number of grids is greater than a preset threshold, the height difference at the current position is retained; otherwise, the height difference at the current position is set to zero. This yields the second optimized difference map, on which contour detection is performed.

[0093] For example, a sliding window of size 3*3 or 5*5 can be selected for filtering. If the height value of a point in the difference graph is 0, then that position is skipped; if it is not 0, then the number of non-zero grids in the sliding window at that position is counted. If the proportion of non-zero grids to the total number of grids is greater than a preset proportion threshold (e.g., set to 50% to 70%), then the height value of that position in the difference graph is retained; otherwise, the height value of that position is set to 0.

[0094] S130, Perform contour detection on the difference map to obtain the position information of the newly added stacking object in the difference map.

[0095] As an example, the position information of the newly added stacking object can be obtained through contour detection. For example, in one implementation, it can be the minimum bounding rectangle of the newly added stacking object in the difference map, etc., which is not limited here. The minimum bounding rectangle will be used as the normalized representation of the newly added stacking object on the difference map.

[0096] S140, based on the position information, update the height value in the output stack pattern diagram of the previous frame, and output the updated stack pattern diagram as the output stack pattern diagram of the generated current frame.

[0097] For example, taking the positional information of the minimum bounding rectangle as an example, during the update, the height value of the corresponding minimum bounding rectangle region in the original stacking map of the current frame can be retrieved and overwritten onto the corresponding region in the output stacking map (Map_out) of the previous frame. Specifically, the overwriting rule can be: if the height value in the current frame is greater than the height value at the same position in the output stacking map of the previous frame, then the height value is updated, that is, the height value of the previous frame is overwritten; otherwise, the height value in the output stacking map of the previous frame remains unchanged, that is, the height value of the previous frame is retained.

[0098] Finally, the updated output stack map (Map_out) of the previous frame is used as the output stack map (Map_out) of the current frame, and then input into the stack planning algorithm for further processing.

[0099] Compared to existing schemes that generate pallet patterns based on single-frame depth maps, this embodiment proposes generating pallet patterns based on depth maps from consecutive frames. This effectively solves the problem of blind spots in existing schemes, improving the accuracy of the palletizing planning algorithm and thus increasing actual palletizing efficiency. Furthermore, it is particularly beneficial for small stacking objects such as cartons and bagged packages, thereby increasing the palletizing volume ratio.

[0100] Figure 7 A flowchart of a robotic palletizing method according to an embodiment of this application is shown. Exemplarily, the robotic palletizing method includes the following steps:

[0101] S210, the stacking diagram is generated using the stacking diagram generation method of the above embodiment.

[0102] S220, the stacking diagram is input into the stacking planning algorithm to perform stacking planning in order to obtain stacking parameters.

[0103] Regarding the stacking planning mentioned above, optimization solutions can be found using algorithms such as greedy algorithms, genetic algorithms, and simulated annealing algorithms. The specific solution can be determined based on actual needs and is not limited here.

[0104] For example, the stacking parameters used for stacking pattern planning may include, but are not limited to, the number of stacking layers, the layer arrangement, the number of horizontal layers, the number of vertical layers, unit parameters, and stacking gaps. Furthermore, the relationship between stacking layers may include overlapping, alternating, rotating, and alternating types.

[0105] S230, perform the corresponding palletizing operation according to the palletizing parameters.

[0106] As an example, during the robot's stacking calculation process, the stacking point data will be calculated in the pallet coordinate system based on the stacking parameters; then, the stacking point data in the pallet coordinate system will be converted to the robot coordinate system data, and finally the robot will be controlled to perform the stacking operation.

[0107] It is understood that the methods and options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0108] By employing the method described in the above embodiments, a more refined palletizing diagram can be generated, which is beneficial for the stacking of small objects (e.g., small cardboard boxes can be placed in narrow spaces such as gaps), thereby further improving the upper limit of the palletizing planning algorithm.

[0109] Figure 8 A schematic diagram of a stack pattern diagram generation apparatus according to an embodiment of this application is shown. Exemplarily, the stack pattern diagram generation apparatus 100 includes:

[0110] The acquisition module 110 is used to acquire the original stacking pattern diagram based on the current frame depth information after each stacking operation is completed.

[0111] The difference module 120 is used to perform difference processing on the original stacking pattern diagram of the previous frame and the original stacking pattern diagram of the current frame to obtain a difference diagram.

[0112] The detection module 130 is used to perform contour detection on the difference map to obtain the position information of the newly added stacking object in the difference map;

[0113] The update output module 140 is used to update the height value of the output stack pattern diagram of the previous frame according to the position information, and output the updated stack pattern diagram as the output stack pattern diagram of the generated current frame.

[0114] It is understood that the apparatus in this embodiment corresponds to the stack pattern diagram generation method in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0115] Figure 9 A schematic diagram of a robotic palletizing device according to an embodiment of this application is shown. Exemplarily, the robotic palletizing device 200 includes:

[0116] The stack diagram generation module 210 is used to generate a stack diagram using the stack diagram generation method described in the above embodiment.

[0117] The stacking pattern planning module 220 is used to input the stacking pattern diagram into a preset stacking pattern planning algorithm to perform stacking pattern planning in order to obtain stacking parameters;

[0118] The palletizing execution module 230 is used to perform corresponding palletizing operations according to the palletizing parameters.

[0119] It is understood that the apparatus in this embodiment corresponds to the robot palletizing method in the above embodiments, and the options in the above embodiments are also applicable to this embodiment, so they will not be described again here.

[0120] This application also proposes a palletizing robot, which can be applied to different scenarios, such as warehouses, logistics, and intelligent sorting, for palletizing different objects. It should be understood that the shape of the palletizing robot is not limited; for example, it can be a robotic arm, a humanoid robot, etc. Exemplarily, the palletizing robot includes a processor and a memory. In addition, it may be equipped with sensing peripherals such as a depth camera. The memory stores a computer program, and the processor executes the computer program to cause the palletizing robot to perform the pallet pattern generation method or the robot palletizing method of the above embodiments.

[0121] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0122] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0123] In addition, this application also provides a computer-readable storage medium for storing the computer program used in the above-described palletizing robot, wherein when the computer program is executed on a processor, it implements the pallet pattern generation method or the robot palletizing method of the above embodiments.

[0124] For example, the computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0126] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0127] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for generating stack type diagrams, characterized in that, include: After each palletizing operation is completed, an original pallet shape image based on the depth information of the current frame is obtained. The acquisition of the original pallet shape image includes: acquiring the depth image of the current frame through a depth camera; performing point cloud transformation on the depth image and projecting the transformed point cloud onto the plane where the bottom surface of the pallet is located; filtering out the target point clouds located in the planar area of ​​the bottom surface of the pallet after projection, and calculating the grid coordinates of each target point cloud projected into the grid within the grid that divides the planar area, to obtain the original pallet shape image of the current frame. The original stacking pattern diagram of the previous frame and the original stacking pattern diagram of the current frame are subjected to difference processing to obtain a difference diagram; Contour detection is performed on the difference map to obtain the position information of the newly added stacking object in the difference map; The height value in the output stack pattern diagram of the previous frame is updated based on the location information, and the updated stack pattern diagram is output as the output stack pattern diagram of the current frame.

2. The method for generating stack pattern diagrams according to claim 1, characterized in that, Before the first palletizing operation, the following also applies: Obtain the three-dimensional spatial position of the empty pallet to obtain information on each corner point of the pallet's bottom surface; Divide the planar area enclosed by all corner points into a grid, select one corner point as the origin, and set the initial height of each grid to zero to obtain the initial stacking pattern.

3. The method for generating stack pattern diagrams according to claim 2, characterized in that, The height value of each grid in the original stacking diagram is determined based on the depth value corresponding to the relevant target point cloud.

4. The method for generating stack pattern diagrams according to claim 3, characterized in that, The height values ​​of each grid in the original stacking diagram are determined based on the depth values ​​corresponding to the relevant target point clouds, including: Calculate the height value of each target point cloud projected onto the planar region on the bottom surface of the pallet, and based on the grid coordinates, take the maximum height value of the points located in the same grid as the height value of the current grid.

5. The method for generating stack pattern diagrams according to claim 1, characterized in that, Before performing contour detection on the difference map, the process further includes: For grids in the difference map whose height difference is less than a preset distance threshold, the height difference is set to zero to obtain the first optimized difference map.

6. The method for generating stack pattern diagrams according to claim 5, characterized in that, Also includes: The difference map after the first optimization is filtered using a sliding window to obtain the difference map after the second optimization.

7. The method for generating stack pattern diagrams according to claim 6, characterized in that, The step of filtering the first optimized difference map using a sliding window includes: The difference map is slid sequentially using a K*K window to count the number of grids with non-zero height differences within the sliding window at the current position, where K is an integer greater than 0; If the ratio of the number of non-zero grids to the total number of grids is greater than a preset ratio threshold, the height difference at the current position is retained; otherwise, the height difference at the current position is set to zero.

8. The method for generating a stack pattern diagram according to any one of claims 1 to 7, characterized in that, The location information is the minimum bounding rectangle of the newly added stacking object in the difference map; updating the height value of the output stacking map of the previous frame according to the location information includes: Obtain the height value of the minimum bounding rectangle region corresponding to the original stacking diagram of the current frame; If the height value is greater than the height value at the same position in the output stack pattern of the previous frame, then the height value is updated; otherwise, the height value in the output stack pattern of the previous frame remains unchanged.

9. A robotic palletizing method, characterized in that, include: A stacking diagram is generated using the method described in any one of claims 1 to 8; The stacking diagram is input into the stacking planning algorithm to perform stacking planning and obtain stacking parameters; Perform the corresponding palletizing operation according to the palletizing parameters.

10. A stacking diagram generation device, characterized in that, include: The acquisition module is used to acquire the original pallet shape map based on the depth information of the current frame after each palletizing operation. The acquisition of the original pallet shape map includes: acquiring the depth map of the current frame through a depth camera; performing point cloud transformation on the depth map and projecting the transformed point cloud onto the plane where the bottom surface of the pallet is located; filtering out the target point clouds located in the planar area of ​​the bottom surface of the pallet after projection, and calculating the grid coordinates of each target point cloud projected into the grid within the grid that divides the planar area, so as to obtain the original pallet shape map of the current frame. The difference module is used to perform difference processing on the original stacking pattern diagram of the previous frame and the original stacking pattern diagram of the current frame to obtain a difference diagram. The detection module is used to perform contour detection on the difference map to obtain the position information of the newly added stacking object in the difference map; The update output module is used to update the height value of the output stack pattern diagram of the previous frame according to the position information, and output the updated stack pattern diagram as the output stack pattern diagram of the generated current frame.

11. A robotic palletizing device, characterized in that, include: A stacking diagram generation module is used to generate a stacking diagram using the method described in any one of claims 1 to 8; The stacking pattern planning module is used to take the stacking pattern diagram as input to the stacking pattern planning algorithm and perform stacking pattern planning to obtain stacking parameters; The palletizing execution module is used to perform corresponding palletizing operations according to the palletizing parameters.

12. A palletizing robot, characterized in that, The palletizing robot includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the pallet pattern generation method of any one of claims 1-8 or the robot palletizing method of claim 9.

13. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed on a processor, implements the pallet pattern generation method according to any one of claims 1-8 or the robot palletizing method according to claim 9.

Images