Product Packaging and Adaptive Packaging Selection Using Computer Vision

Abstract

A system and method for packaging a shipping container includes: detecting, by a computing device, the presence of a shipping container; generating, by the computing device, a 3D point cloud including a 3D perspective representation of the interior of the shipping container and a surface of at least one item disposed within the shipping container; analyzing, by the computing device, the 3D point cloud to identify a void area within the interior of the shipping container, the void area being identified as being absent from at least one item; selecting, by the computing device, a percentage from a plurality of possible percentages based on characteristics of the void area; and dispensing, by the computing device, protective packaging material into the void area such that the selected percentage of the void area is filled with the protective packaging material.

Description

[Background technology]

[0001] When a product is purchased online, it is packaged and sent to the purchaser via a parcel delivery service. The product is often sent in either a flexible mailer or a box lined and / or filled with protective packaging material. The protective packaging material may include, for example, bubble wrap, packing paper dunnage, biodegradable packing peanuts, foam, corrugated dunnage, and / or inflatable cushions (or pillows). An individual places the product in the box, obtains the protective packaging material, fills the box with the protective packaging material, and seals the box for shipping. Known processes use human assistance throughout the box filling / sealing process. Summary of the Invention

[0002] The present disclosure relates to a method for packaging a shipping container, the method including: detecting, by a computing device, the presence of a shipping container; generating, by the computing device, a 3D point cloud including a 3D perspective representation of the interior of the shipping container and a surface of at least one item disposed within the shipping container; analyzing, by the computing device, the 3D point cloud to identify a void area within the interior of the shipping container, the void area being identified as being absent from the at least one item; selecting, by the computing device, a percentage from a plurality of possible percentages based on characteristics of the void area; and dispensing, by the computing device, protective packaging material into the void area such that the selected percentage of the void area is filled with the protective packaging material.

[0003] The present disclosure also relates to a system comprising a processor and a non-transitory computer-readable storage medium with programming instructions configured to cause the processor to implement a method for packaging a shipping container, the programming instructions including instructions for detecting the presence of the shipping container, generating a 3D point cloud including a 3D perspective representation of the interior of the shipping container and a surface of at least one item disposed within the shipping container, analyzing the 3D point cloud to identify a void area within the shipping container, the void area being identified as being absent from at least one item, selecting a percentage from a plurality of possible percentages based on characteristics of the void area, and dispensing protective packaging material into the void area such that the selected percentage of the void area is filled with the protective packaging material.

[0004] The present disclosure further relates to a non-transitory computer-readable medium storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform operations including: detecting the presence of a shipping container, generating a 3D point cloud including a 3D perspective representation of the interior of the shipping container and a surface of at least one item disposed within the shipping container, analyzing the 3D point cloud to identify a void area within the shipping container, the void area being identified as being absent from at least one item, selecting a percentage from a plurality of possible percentages based on characteristics of the void area, and dispensing protective packaging material into the void area such that the selected percentage of the void area is filled with the protective packaging material. [Brief explanation of the drawings]

[0005] The present solution will be described with reference to the following drawings, in which like numerals represent like items throughout: [Figure 1] FIG. 1 is a diagram of an exemplary system. [Figure 2] FIG. 1 is a diagram of a shipping container with a void area divided into void area sections. [Figure 3]FIG. 1 is a diagram of a shipping container with a void area divided into void area sections. [Figure 4] 1 is a graph illustrating the non-linear relationship between void area cubic volume and the number of inflatable cushions used to fill it. [Figure 5A] 1 is a flow chart of an exemplary method for filling, closing, and sealing a shipping container. [Figure 5B] 1 is a flow chart of an exemplary method for filling, closing, and sealing a shipping container. [Figure 6] FIG. 1 is an illustration of a computing device. DETAILED DESCRIPTION OF THE INVENTION

[0006] It will be readily understood that the components of the embodiments as generally described herein and illustrated in the accompanying drawings could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of various embodiments, as represented in the figures, is not intended to limit the scope of the disclosure, but is merely representative of various embodiments. While various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0007] The present solution may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the present solution is therefore indicated not by this detailed description but by the appended claims. All changes that come within the meaning and range of equivalency of the claims are intended to be embraced within their scope.

[0008] Throughout this specification, references to features, advantages, or similar terms do not imply that all of the features and advantages that may be realized by the present solution should be, or should be, in any single embodiment of the present solution. Rather, language referring to features and advantages is understood to mean that the particular feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present solution. Thus, throughout this specification, discussions of features and advantages and similar terms may, but do not necessarily, refer to the same embodiment.

[0009] Furthermore, the described features, advantages, and characteristics of the solution may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the solution may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the solution.

[0010] Throughout this specification, references will be made to "one embodiment," "an embodiment," or similar language that means that a particular feature, configuration, or characteristic described in connection with the illustrated embodiment is included in at least one embodiment of the solution. Thus, throughout this specification, "in one embodiment," "in an embodiment," and similar language may, but do not necessarily, all refer to the same embodiment.

[0011] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. As used herein, the word "including" means "including but not limited to."

[0012] As described above, products purchased online are packaged and sent to the purchaser via a parcel delivery service. Products are often sent in either flexible mailers or boxes lined and / or filled with protective packaging. The protective packaging can include cushioning or void-filling materials, such as bubble wrap, packing paper dunnage, foam, and / or inflatable cushions (or pillows). Individuals or mechanical robots can pick and place the product into a selected box or mailer. This is then followed by obtaining the appropriate protective packaging to pack or fill the void within the box, occupying the space and protecting the product for transport. Known processes use human assistance throughout the picking, box or mailer selection, sorting of products for the order, and decision-making regarding the type of interior packaging, along with the box or style of mailer, prior to the fill / seal process.

[0013] Distribution centers need a way to package consumer-purchased products at high speeds with limited or no human assistance. Therefore, traditional solutions are labor-intensive, independent, and uncoordinated, resulting in increased human input due to the use of more packaging material or the use of the wrong type of packaging to protect the product. The solution described herein provides an automated or semi-automated method for items to be identified, packaged with protective packaging, and sent to relatively large destinations. The automation or semi-automation reduces costs, the amount of labor required, and the amount of human error.

[0014] The product or products are configured to be placed within a packaging container or between packaging containers or articles being transported or stored to protect the articles, fill voids within a container such as a packaging container, and / or prevent or inhibit articles from shifting within the packaging container. Although there is overlap between the following categories, exemplary categories of protective packaging materials include protective fillers and block-and-brace materials.

[0015] Protective fillers are typically provided individually or as multiple units configured to be placed within the void space to provide a desired level of packing. Such units are typically of a predetermined size or may have a predetermined dimension and be selectively configurable in another dimension, such as length. In some instances, the size of the protective filler may be configurable in multiple or all of those dimensions. Protective fillers typically do not assume a solid shape corresponding to the space around the items, but rather are resiliently flexible to compressibly fit around the corners, edges, and sides of the packaged items to fill or cushion the item(s) within the space around the items. Protective fillers include, for example, void fillers and cushioning materials.

[0016] Void fillers typically provide minimal cushioning properties and are relatively soft. They are typically used to fill empty spaces within a packaging container to reduce movement within the container of lightweight items that are fragile or susceptible to damage, and to prevent internal movement that could damage items such as glass bottles when multiple products are placed in a box. Examples of void fillers include crushed paper dunnage with a fairly light compression loft pattern and other easily compressible void fillers.

[0017] Cushioning materials are designed to cushion packaged items and provide varying degrees of protection against shocks and vibrations, which may occur up to 20-30 times during an average shipping cycle. Examples of cushioning materials include inflatable air pillows and cushions, bubble wrap, paper dunnage with lofted construction capable of withstanding moderate shocks and vibrations, foam sheets, and packing peanuts. Void fillers and cushioning materials are typically provided as one or more similarly sized units of a common, pre-defined size, although for some applications, void fillers or cushioning materials from other protective fillers can be custom-sized.

[0018] The void fillers or cushioning materials used are typically selected to sufficiently fill the void space by dispensing lengths of material for placement within the container to perform the desired protective function. Some void fillers or cushioning materials can be used to enclose or surround an item, such as expandable paper or bubble wrap that can be used to wrap items such as bottles, jars, or fragile items.

[0019] Systems are used to accelerate the packaging process and reduce waste by optimizing the amount of protective packaging (e.g., inflatable cushions) used to fill each shipping container (e.g., box). Traditional systems rely mostly on human judgment to guess the appropriate boxes, amount of packaging, and use the appropriate packaging materials based on what is being packed. In some cases, cameras have improved in quality over the years as prices have decreased. In this solution, machine learning can enable cameras and other sensors to detect when a shipping container is in proximity to the dispensing system, automatically calculate the size of the detected shipping container, use cameras or other sensors to detect when the center of the shipping container is aligned with a dispenser, generate a 2D image from the top of the carton to identify a portion of the contents of the shipping container, identify areas inside the shipping container that are void or absent of any objects or items, calculate the volume of the identified void areas, optionally segment the void areas, communicate to a timer that dispenses a quantity of protective packaging material to fill the void areas or each void area segment (e.g., based on the size of the void area, the size of the void area segments, and / or the type of product placed in the shipping container), and / or control a dispenser to dispense protective packaging material into the shipping container.

[0020] The solution will now be described in relation to packaging for purchased goods, but the solution of the invention can also be used in other applications when objects need to be delivered from a start location to a destination location.

[0021] 1 provides a diagram of a system 100 generally configured to facilitate the packaging and transportation of items 110. The items may include a variety of products, including perishable items (e.g., food) and / or non-perishable items (e.g., apparel, electronics, auto parts, beauty products, personal care items, books, household appliances, entertainment tickets, fashion accessories, footwear, office supplies, sporting goods, toys, video games, watches, eyeglasses, and / or jewelry).

[0022] System 100 includes packing stations 102, a sensor system 106, an optional inflation module 160, dispensers 118, 122, and a container closer / sealer apparatus 126. These components are communicatively coupled to each other and to one or more computing devices 132 via a network 130 (e.g., the Internet or an intranet). Each packing station 102 includes a conveyor belt 104 controlled by a controller 190. Any known or later-known conveyor belt and controller may be used herein. Optional storage bin(s) may be provided at the end of conveyor belt 104 for temporary storage and / or transport of sealed shipping containers to a sorting / transport station.

[0023] As shown in FIG. 1 , the sensor system 106 is positioned relatively close to the packing station 102, suitable for generating sensor data regarding shipping container contents 108, 170, 180 being moved by the conveyor belt 104 toward the dispensers 118, 122. The shipping containers 108, 170, 180 each have one or more items 110 disposed therein that may or may not need to be protected from damage during shipping by protective packaging dispensed by the dispensers 118, 122. The shipping containers may include, but are not limited to, shipping boxes, shipping bags, and / or shipping envelopes. The sensor system 106 may include, but is not limited to, monocular cameras, digital cameras, radar sensors, lidar sensors, time-of-flight (ToF) cameras, and / or other sensor devices. Two or more sensor types may be used in conjunction to generate additional data or to verify point cloud accuracy, particularly with opaque or reflective surfaces.

[0024] The dispensers 118, 122 can have any suitable configuration for producing the desired type of protective filler material. For example, the dispensers 118, 122 may be configured to convert a supply of material into protective packaging material. In this case, the dispensers 118, 122 may include a paper dunnage machine, an expansion sealing device, and / or a foam-in-bag device. Alternatively, the dispensers 118, 122 may simply dispense protective packaging elements therefrom. In this case, the dispensers 118, 122 may store and dispense packing peanuts, individual lengths of bubble wrap, individual lengths of paper dunnage, and / or other types of protective packaging elements. The dispensers 118, 122 may also include a cutting mechanism for cutting pieces of bubble wrap and / or paper dunnage from a roll stored therein. In other scenarios, the dispensers 118, 122 are coupled to an external device that supplies the protective packaging material dispensed by the dispensers 118, 122. The external device may include, but is not limited to, an expansion seal module.

[0025] The inflation module 160 is configured to inflate inflatable protective packaging material, such as cushions, pillows, and bladders. The inflated protective packaging material is then provided to the dispenser 118 for storage and dispensed into the shipping container as it passes underneath. A single dispenser 118 may be provided in the system 100. Alternatively, two or more dispensers 118, 122 may be provided in the system 100, configured to dispense the same or different types of protective packaging material depending on the protection required. For example, the first dispenser 118 may dispense an inflated cushion, while the second dispenser 122 may dispense paper dunnage. The inflation module 160 and / or the dispensers 118, 122 may be manually controlled by the individual 136 or autonomously controlled by the computing device 132. This solution is described below in the context of autonomous control.

[0026] As the shipping container 108 moves on the conveyor belt 104 in a direction 114, a proximity sensor 116 detects when the shipping container 108 approaches the sensor system 106. The proximity sensor 116 may include, but is not limited to, a beam brake sensor, a scanner (e.g., a barcode scanner), and / or a camera. The sensor system 106 may generate sensor data continuously or periodically in response to a triggering event. The triggering event may include, but is not limited to, alignment of the center of the shipping container 108 with an axis 150 of the sensor system 106. The shipping container 108 is shown in FIG. 1 as comprising a shipping box. As mentioned above, the shipping container may alternatively comprise, for example, a shipping bag or a shipping envelope.

[0027] In some scenarios, the system 100 performs an operation to detect when the center of the shipping container is aligned with the axis 150 of the sensor system 106. This can be accomplished, for example, using the sensor devices of the sensor system 106 and the computing device 132. The first sensor device generates depth measurements. The computing device 132 uses the depth measurements to detect the leading edge 140 and trailing edge 142 of the shipping container as it moves on the conveyor belt in the direction 114. Each edge 140, 142 of the shipping container 108 is detected when the depth measurements are below a threshold value thr. The threshold value thr can be selected based on the known distance between the sensor system 106 and the conveyor system 104 and the known height of the shipping container sidewall (i.e., the flange extends upward, i.e., the flange (or flap) has not yet been bent at the score (or seam) 152). An additional alternative sensor can be added to identify a score 152 within the box, where flaps are folded to close the box and determine the length of the flap (flange), or depth measurement can be achieved by recognizing the score, which ultimately determines the depth of the box. These two edge detections are then used to determine the length L of the shipping container 108 (extending in the conveyor belt movement direction 114). The shipping container length L, the edge detection timing, and the known conveyor speed S are used to identify the time t when the center of the shipping container is aligned with axis 150. The conveyor belt 104 may optionally be controlled by the controller 190 to either (i) continuously move the shipping container 108 in the direction 114 at a predetermined speed, or (ii) momentarily stop when the center of the shipping container 108 is aligned with axis 150. In scenario (i), an encoder can be used on the moving conveyor to correlate void sampling with its axis of movement.

[0028] At time t, a second sensor device of the sensor system 106 is enabled or activated to generate sensor data useful for generating a 3D point cloud of the interior cavity of the shipping container 108. The 3D point cloud includes a plurality of points plotted on a 3D graph, providing a 3D surface map of the interior sidewall surfaces of the shipping container 108 and / or the exposed surfaces of any objects disposed within the shipping container. Thus, each point has an x-coordinate, a y-coordinate, and a z-coordinate.

[0029] The computing device 132 analyzes the 3D point cloud and other sensor data to identify one or more areas within the shipping container 108 that (i) are located under the flap seam (or score) 152 and (ii) are empty or absent of any objects. The identified areas are referred to herein as void areas. In some scenarios, the filtered contours of the shipping container flaps and the surrounding shipping container define the geometry of the shipping container footprint, and using automated depth measurements, the computing device 132 can determine the empty volume of the shipping container 108. From the topography of the contents and this empty shipping container volume, the remaining space can be calculated as void. The still image with the 3D point cloud allows machine vision to verify the true nature of the contents of the shipping container 108, especially shiny objects, transparent or opaque films, etc.

[0030] In other scenarios, the void area may be determined in other ways. The height H of the shipping container 108 in its fully packed state extends vertically from the bottom edge 154 of the box to the flap seam (or notch) 152. This height H may be calculated in various ways and / or obtained from the data store 134. For example, the height H may be calculated based on depth measurements associated with the leading edge 140 or trailing edge 142 of the shipping container 108. This depth measurement specifies the height h of the top edge of the flap relative to the top surface of the conveyor belt, a sensor device, or other reference point. H may be determined by dividing h by N (i.e., H = h / N) or by multiplying h by N, where N is any number greater than zero (e.g., 2). The ratio of h to H is known. The total interior volume of the box 108 can then be calculated using the length L, height H, and width W. The width W is determined from the 3D point cloud data and extends transversely to the direction of conveyor belt travel 114.

[0031] Additionally or alternatively, the box dimensions may be known in a warehouse management system (WMS) or other facility management software, in which case the box dimensions can be passed to a computing device when an identifier code, typically a barcode on a carton, is scanned.

[0032] A cube or other 3D shape can then be generated that represents the 3D shipping container having these dimensions. The cube or other 3D shape can then be overlaid on the 3D graph so that (i) its center is aligned with the center of the shipping container 108 and (ii) its bottom surface is aligned with a known location on the top surface of the conveyor belt 104. Any points of the 3D point cloud that lie outside the cube / 3D shape can be discarded or otherwise filtered. The remaining points of the cube / 3D shape and the 3D point cloud are then analyzed to identify one or more void regions within the shipping container 108. The cubic volume V of each identified void region can be calculated using known mathematical algorithms.

[0033] In response to the identified void areas, the computing device 132 can store information 160 in the data store 134, such as identifying the shipping container 108, identifying each void area detected within the shipping container 108, specifying the location of each void area within the shipping container 108, specifying the cubic volume V of each void area, and / or specifying the type of each object placed within the shipping container 108. The object type can be determined according to known or hereafter known object type detection techniques using 3D point clouds, scanner data (e.g., barcode data), machine vision, and / or accessible product data drawn from a 3D scanner used to size the product, a database with pre-programmed sizing, or manually sized products.

[0034] The computing device 132 may then determine whether the cubic volume V of each void area is large enough to be filled with at least a minimum amount of protective packaging material (e.g., a minimum number of inflatable cushions or a minimum length of paper dunnage (which may be measured in a linear projection)). If not, the shipping container 108 may optionally be moved toward the downstream container closer / sealer apparatus 126 without having protective packaging material dispensed therein by the dispensers 118, 122.

[0035] In that case, the computing device 132 performs other operations to facilitate the filling of the shipping container 108 with protective packaging material 120, 124 by the dispensers 118, 122. For each void area, the computing device 132 may select a percentage between 0% and 100% based on (i) the type of packaging material from a plurality of packaging material types, and / or (ii) the cubic volume V associated with the void area and / or the type of object placed in the shipping container 108 below or adjacent to the void area. For example, the computing device 132 may select paper dunnage and 60% when the cubic volume V has a first value and the type of object is a blanket, or alternatively, select an inflatable cushion (or pillow) and 95% when the volume has a second value and the type of object is glassware. The percentage represents the minimum amount of void area to be filled with protective packaging material.

[0036] The relationship between the cubic volume V of the void area and the percentage of the void area to be filled with protective packaging material may be non-linear. A graph illustrating an exemplary non-linear relationship is provided in Figure 4. The non-linear relationship may be determined empirically based on collected information and / or a best curve-fitting algorithm.

[0037] Once an inflatable type of protective packaging is selected, the computing device 132 can further determine the amount by which the protective packaging should be inflated. For example, the computing device 132 previously determined that an inflatable cushion of a given size (e.g., 5×8 inches in an uninflated state) should be used to fill the box 108. The computing device 132 then accesses, for example, a look-up table (LUT) to obtain the amount of inflation of the inflatable cushion so that at least 75% of the void area within the box is filled thereby. The LUT can be indexed by an identifier of a cushion of a given size, an identifier of the type of object placed within the shipping container, the shipping container type (e.g., box), the dimensions of the shipping container (i.e., L, W, H), and / or the percentage of void area to be filled by the cushion. The LUT can identify different inflation amounts for different types of objects, different sized cushions, different types of shipping containers, different sized shipping containers, and / or different fill rates.

[0038] Once the packaging paper dunnage is selected, the computing device 132 may further determine whether the paper dunnage should be stacked, folded (e.g., accordion-folded), loosely rolled, tightly rolled, loosely crushed, or tightly crushed. For example, the computing device 132 previously determined that individual sizes of paper dunnage should be used to fill the box 108. The computing device 132 then accesses the LUT, for example, to obtain information indicating whether the paper should be loosely or tightly crushed. The LUT may be indexed by an identifier for the individual sizes of paper dunnage, an identifier for the type of object placed in the shipping container, the shipping container type (e.g., box), shipping container dimensions (i.e., L, W, H), and / or the percentage of void area to be filled with the paper dunnage. The LUT may identify different types or amounts of wrinkles for different types of objects, different individual sizes of paper dunnage, different types of shipping containers, different sizes of shipping containers, and / or different fill rates.

[0039] The computing device 132 then provides information to the inflation module 160 and / or dispensers 118, 122 useful for filling the shipping container 108 with the selected parameters. This information may include, but is not limited to, a void area identifier, a void area location within the shipping container (e.g., a quadrant identifier), the type of protective packaging material used in each void area (e.g., inflatable cushions or paper dunnage), the number of protective packaging units for each void area (e.g., 10 cushions or 20 individually sized pieces of paper dunnage), the amount of protective packaging material for each void area (e.g., 18 feet per cubic foot of paper dunnage), the amount of inflation, the type of paper or dunnage folding (e.g., accordion), the type of paper or dunnage rolling (e.g., loose or tight), and / or the type of creasing (e.g., loose or tight). In some scenarios, a large void area may be segmented into two or more void area sections of the same or different sizes. Thus, this information may also include void area division identifiers and locations within the shipping container. The same or different types of protective packaging may be used in the void areas and / or void area sections. For example, a large inflatable cushion 172 may be used in a first quadrant void area 174 inside the shipping container 170, and a small inflatable cushion 176 may be used in a second quadrant void area 178 inside the shipping container. Alternatively, an inflatable cushion 120 may be used in a first quadrant void area 182 inside the shipping container 180, and crushed paper dunnage 124 may be used in a second quadrant void area 184 inside the shipping container.

[0040] The protective packaging material can be inserted into the shipping container manually by an individual 136 or automatically by a dispenser 118, 122. In the latter case, the computing device 132 is configured to control the operation of the inflation module 160 and / or the dispenser 118, 122. Any known or later-known inflation module for inflating packing cushions, pillows, and / or bags can be used herein. Also, any known or later-known dispenser can be used herein. In some scenarios, the dispenser is specifically modified to move the dispensing head 190 along multiple axes (e.g., x-axis, y-axis, and / or z-axis), rotationally or at an angle, so that the shipping material is dispensed into multiple void areas and / or multiple void area sections (e.g., quadrant sections) according to instructions received from the computing device 132.

[0041] Once the shipping container is filled with protective packaging, it is closed and sealed at block 126. Any known or hereafter known container closer / sealer device can be used herein. In some scenarios, the container closer / sealer device comprises one or more articulated arms with grippers at their distal ends and controllable joints along the elongated arms. Thus, articulated arms and controllers are well known. If the shipping container is a box, the flaps 130 are bent at seams 152 and joined together (e.g., via tape, PSA, or low-temperature adhesive and / or staples). Other devices 190 may be provided downstream of the container closer / sealer device 126. The other devices may include, but are not limited to, a labeling device that applies one or more labels to the shipping container 200.

[0042] As mentioned above, in some scenarios, the void area can be segmented. FIG. 2 provides an illustration of a shipping container 200 including a void area 202 divided into multiple void area sections 204, 206, and 208. The same or different protective packaging can be distributed among the void area sections 204, 206, and 208. If the same inflatable protective packaging is used in the void area sections 204, 206, and 208, the same or different amounts of inflation can be used for each void area section 204, 206, and 208. Other segmentation techniques can be used. For example, as illustrated by the top view of a shipping container 300 provided in FIG. 3, the void area can be segmented into quadrants, resulting in four void area sections 304, 306, 308, and 310.

[0043] 5A and 5B provide a flow chart of an example method 500 for packaging an item (e.g., item 110 of FIG. 1 ) within a shipping container (e.g., shipping container 108 of FIG. 1 , shipping container 170 of FIG. 1 , shipping container 180 of FIG. 1 , shipping container 200 of FIG. 2 , or shipping container 300 of FIG. 3 ). Method 500 may include more or fewer operations than those shown in FIGS. 5A and 5B . Some or all of the operations of method 500 may be performed in the same or a different order than that shown. Some or all of the operations of method 500 may be performed by a computing device (e.g., computing device 132 of FIG. 1 and / or computing device 600 of FIG. 6 ) and / or a processor (e.g., central processing unit 606 of FIG. 6 ).

[0044] Method 500 begins at block 502 and continues at block 504, where a proximity sensor (e.g., proximity sensor 116 in FIG. 1 ) detects when a shipping container is in proximity to a sensor system (e.g., sensor system 106 in FIG. 1 ). As used herein, the term “proximity” can refer to a predetermined distance between a leading edge of a shipping container (e.g., leading edge 140 in FIG. 1 ) and an axis of a sensor system (e.g., axis 150 in FIG. 1 ), or the presence of a shipping container in an area or at a location on a packing station. The predetermined distance can be selected according to a given application. For example, the predetermined distance can be between 1 and 10 feet, ≦10 feet, ≦5 feet, ≦2 feet, ≦1 foot, or ≦10 inches.

[0045] Next, in block 506, a computing device (e.g., computing device 132 of FIG. 1) detects a trigger event. The trigger event may include, but is not limited to, alignment of the center of the shipping container with the axes of the sensor system. This detection may be performed in a variety of ways. One such technique using depth measurements of the leading and training edges (e.g., leading edge 140 and trailing edge 142) of the shipping container is described above in connection with FIG. 1. This technique may be used here to determine the time t at which the center of the shipping container is aligned with the axes of the sensor system.

[0046] In block 508, sensors of the sensor system are enabled or activated, thereby generating sensor data at time t. The sensors may include, but are not limited to, a Time of Flight (ToF) camera, a lidar sensor, a radar sensor, and / or other sensors capable of generating sensor data that can be used to generate a 3D point cloud. The ToF camera provides a point cloud topography and visible light imagery. In block 510, a 3D point cloud for the interior cavity of the shipping container is generated. The 3D point cloud provides a surface map of the exposed interior sidewall surfaces of the shipping container and / or the exposed surfaces of any items placed inside the shipping container.

[0047] The computing device analyzes the 3D point cloud at 512 to identify a void area within the shipping container (e.g., void area 202 of FIG. 2 or void area 302 of FIG. 3). At block 514, a cubic volume V is calculated for the void area. If the cubic volume V is not large enough to be filled with a minimum amount of protective packaging material [block 516: NO], the method 500 continues at block 518, where the shipping container is moved toward a downstream container closer / sealer device (e.g., device 126 of FIG. 1) without dispensing any protective packaging material therein. The method 500 then proceeds to block 542 of FIG. 5B, which will be described below.

[0048] If the cubic volume V is large enough to be filled with a minimal amount of protective packaging material [block 516: YES], method 500 continues with optional blocks 520-526. These optional operations 520-526 include segmenting the void area, detecting the type of item placed within the shipping container, selecting, for the void area or each segment (e.g., void area section 204 of FIG. 2, void area section 206 of FIG. 2, void area section 208 of FIG. 2, void area section 304 of FIG. 3, void area section 306 of FIG. 3, void area section 308 of FIG. 3, or void area section 310 of FIG. 3), a protective packaging type based on its cubic volume and / or associated item type, and / or selecting, for the void area or each segment, a percentage thereof to be filled with protective packaging material based on its cubic volume and / or associated item type. Any known or later-known object detection technology may be used in block 522. For example, a machine learning algorithm trained to detect a particular type of object based on sensor data including 3D point cloud data can be used. The machine learning algorithm can include, but is not limited to, a neural network. Database queries can also be performed using, for example, scanner data (e.g., barcode data).

[0049] Upon completion of block 516 or any operation, method 500 continues to block 528 of FIG. 5B. As shown in FIG. 5B, block 528 includes determining whether the selected protective packaging is inflatable. Inflatable protective packaging may include, but is not limited to, cushions, pillows, and / or bladders. If the selected protective packaging is not inflatable [block 528: NO], method 500 proceeds to block 534, described below. In contrast, if the protective packaging is inflatable [block 528: YES], method 500 continues with optional blocks 530 and 532. Optional operations include selecting an inflation amount for the void region or each segment thereof based on specific criteria and / or having an inflation module (e.g., inflation module 160 of FIG. 1) inflate the transport material and dispense it to a dispenser (e.g., dispenser 118 of FIG. 1). The criteria by which the selection in block 520 is made may include, but are not limited to, shipping container characteristics, protective packaging characteristics, item characteristics, selected percentage values, and / or other criteria. Shipping container characteristics may include, for example, type and / or size. Protective packaging characteristics may include, for example, type and / or size. Item characteristics may include, but are not limited to, type, size, and / or fragility.

[0050] At decision block 534, the computing device determines whether the selected protective packaging material is paper dunnage. If not [block 534: NO], method 500 proceeds to block 538, described below. If it is paper dunnage [block 534: YES], method 500 continues with the operations of optional block 536 or block 538. Block 536 optionally includes selecting, for the void area or each segment thereof, whether the paper should be folded or crumpled based on certain criteria. The criteria may be the same or different from the criteria used to make the selection in block 530. Thus, the criteria used in block 536 may include, but are not limited to, shipping container characteristics, protective packaging characteristics, article characteristics, a selected percentage value, and / or other criteria.

[0051] In block 538, the computing device controls autonomous operation of a dispenser (e.g., dispensers 118, 122 of FIG. 1 ) to dispense transport material into the void area and / or void area sections. The autonomous operation can include, but is not limited to, actuation motors, actuation belts, actuation wheels, enabling air vectoring, opening valves, actuation latches, actuation of telescoping shafts, articulated arms, and / or moveable joints.

[0052] Air knives or other suitable mechanisms can be used to guide inflatable protective packaging, such as cushions, pillows, and bags, into the shipping container. Articulating steering vanes of polymeric or metallic materials can also be used to steer the inflatable protective packaging into the shipping container. A chute may be provided from a dispenser with drive belts on each side of the chute to insert units of inflatable protective packaging into the appropriate portion of the shipping container. The insertion point for any of the above transport mechanisms can be manipulated into the correct location within the shipping container, for example, by moving an insertion funnel linearly (e.g., upward, downward, or sideways) or at an angle. Once the protective packaging unit is inserted, the transport mechanism can use brushes, a set of elastomeric strands spanning the top of the shipping container (which allows the inflatable protective packaging to be pushed through but not bounce back), or flaps on the shipping container itself to help contain the shipping material within the shipping container. In addition to or instead of a containment mechanism, the deposition area may be isolated from the environment to avoid air currents that may prevent the inflatable protective packaging from reaching their target zone or that may subsequently displace the inflatable protective packaging.

[0053] Paper dunnage and inflatable protective packaging materials, such as cushions, pillows, and bladders, can be inserted by various drive mechanisms or guided through a chute with an articulating head. For example, paper dunnage or inflatable protective packaging materials can be conveyed by parallel drive belts on the chute side, driven by elastomeric drive wheels. The drive wheels can be articulated to steer the paper dunnage or inflatable protective packaging materials laterally. In the case of paper dunnage, such articulation creates transverse folds in the collapsed paper web, allowing the paper dunnage to be tied back and forth within the shipping container. The insertion point of the paper dunnage or inflatable protective packaging materials can be manipulated into the correct location within the shipping container by moving the insertion funnel linearly (e.g., upward, downward, sideways) or at an angle. The amount of paper dunnage or inflatable protective packaging materials is determined by the void volume and is subdivided into smaller zones to precisely adjust the ratio of dunnage or inflatable protective packaging materials to the void.

[0054] The shipping container is closed and sealed at 540. Block 542 is then executed to end the method 500 or perform other actions. Typically, a shipping label is printed and affixed to the package at this stage.

[0055] Referring now to Figure 6, there is shown a computing device 600. The sensor system 106 of Figure 1, the computing device 132 of Figure 1, and / or the controller 190 of Figure 1 may be the same as or similar to the computing device 600. Thus, a description of the computing device 600 is sufficient to understand these components of the system 100.

[0056] In some scenarios, the solution is used in a client-server architecture, and therefore the computing device architecture shown in Figure 6 is sufficient to understand the details of the client computing device and the server.

[0057] Computing device 600 may include more or fewer components than those shown in Figure 6. However, the components shown are sufficient to disclose an exemplary solution implementing the present solution. The hardware architecture of Figure 6 represents one implementation of a representative computing device configured to provide an improved item return process as described herein. Thus, computing device 600 of Figure 6 implements at least a portion of the methods described herein.

[0058] Some or all components of computing device 600 may be implemented as hardware, software, and / or a combination of hardware and software. Hardware includes, but is not limited to, one or more electronic circuits. Electronic circuits may include, but are not limited to, passive components (e.g., resistors and capacitors) and / or active components (e.g., amplifiers and / or microprocessors). The passive and / or active components may be adapted, configured, and / or programmed to perform one or more of the methods, procedures, or functions described herein.

[0059] As shown in FIG. 6 , computing device 600 comprises a user interface 602, a central processing unit (“CPU”) 606, a system bus 610, memory 612 connected to and accessible by other portions of computing device 600 via system bus 610, a system interface 660, and hardware entities 614 connected to system bus 610. The user interface may include input and output devices that facilitate user-software interaction to control the operation of computing device 600. Input devices include, but are not limited to, a physical and / or touch keyboard 650. The input devices may be connected to computing device 600 via a wired (serial or wired LAN) or wireless connection (e.g., a Bluetooth® connection or a WiFi connection). Output devices include, but are not limited to, a speaker 652, a display 654, and / or a light-emitting diode 656. The system interface 660 is configured to facilitate wired or wireless communication with external devices (e.g., network nodes such as access points).

[0060] At least some of the hardware entities 614 perform actions that involve accessing and using memory 612, which may be Random Access Memory (“RAM”), a disk drive, a Compact Disc Read Only Memory (“CD-ROM”), or a remote “cloud”-based process. The hardware entities 614 may include a disk drive unit 616 with a computer-readable storage medium 618 having stored thereon one or more sets of instructions 620 (e.g., software code) configured to implement one or more of the methods, procedures, or functions described herein. The instructions 620 may also reside, completely or at least partially, within the memory 612 and / or within the CPU 606 during their execution by the computing device 600. The memory 612 and the CPU 606 may also constitute machine-readable media. As used herein, the term “machine-readable medium” refers to a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions 620. The term "machine-readable medium," as used herein, also refers to any medium capable of storing, encoding, or carrying a set of instructions 620 for execution by computing device 600, causing computing device 600 to perform any one or more of the methods of this disclosure.

[0061] While the present solution has been shown and described with respect to one or more implementations, those skilled in the art will be able to make equivalent changes and modifications upon reading and understanding this specification and the accompanying drawings. In addition, while particular features of the present solution may be disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desirable and advantageous for any given or particular application. Thus, the breadth and scope of the present solution should not be limited by any of the above-described embodiments. Rather, the scope of the present solution should be defined according to the following claims and their equivalents.

[0062] Without excluding further possible embodiments, certain exemplary embodiments are summarized in the following sections.

[0063] Clause 1: A method for packing a shipping container, comprising: detecting, by a computing device, the presence of a shipping container; generating, by the computing device, a 3D point cloud including a 3D perspective representation of the interior of the shipping container and a surface of at least one item disposed within the shipping container; analyzing, by the computing device, the 3D point cloud to identify a void area within the shipping container, wherein the void area is identified as being free of the at least one item; selecting, by the computing device, a percentage from a plurality of possible percentages based on characteristics of the void area; and dispensing, by the computing device, protective packaging material into the void area such that the selected percentage of the void area is filled with the protective packaging material.

[0064] Clause 2: The method of clause 1, wherein the shipping container comprises a box, a bag, or an envelope or padded mailer.

[0065] Clause 3: The method of any of the above method clauses, further comprising dividing the void region into void region sections and selecting different percentages from a plurality of possible percentages for at least two of the void region sections based on characteristics of the void region sections.

[0066] Clause 4: A method according to any of the above method clauses, wherein the characteristics of the void area sections include at least one of a cubic volume, a relative position of the void area sections with respect to each other, and a relative position of the void area sections with respect to at least one item disposed within the shipping container.

[0067] Clause 5: The method of any of the above method clauses, wherein the cubic volume of the void area has a non-linear relationship to the percentage of the void area to be filled with protective packaging material.

[0068] Clause 6: The method of any of the above method clauses, further comprising detecting, by the computing device, the occurrence of a trigger event and enabling a sensor responsive to the trigger event.

[0069] Clause 7: The method of any of the above method clauses, wherein the trigger event comprises alignment of the center of the shipping container with the axis of the sensor.

[0070] Clause 8: A method according to any of the above method clauses, wherein the detection of the trigger event is based on a depth measurement of at least one edge of the shipping container.

[0071] Clause 9: The method of any of the above method clauses, further comprising calculating a cubic volume of the void region.

[0072] Clause 10: The method of any of the above method clauses, wherein the characteristics of the void region include a cubic volume.

[0073] Clause 11: The method of any of the above method clauses, further comprising closing and sealing the shipping container without any protective packaging material dispensed therein when the cubic volume is not large enough to be filled with the threshold amount of protective packaging material.

[0074] Clause 12: The method of any of the above method clauses, wherein dispensing protective packaging material into the void region is performed when the cubic volume is large enough to be filled with a threshold amount of protective packaging material.

[0075] Clause 13: The method of any of the method clauses above, further comprising selecting a protective packaging type from a plurality of possible protective packaging types based on the cubic volume or the at least one item type when the cubic volume is large enough to be filled with a threshold amount of protective packaging.

[0076] Clause 14: A method according to any of the above method clauses, wherein said plurality of protective packaging types includes at least an inflatable cushion, packing paper or foam in place.

[0077] Clause 15: The method of any of the above method clauses, further comprising selecting an inflation amount from a plurality of possible inflation amounts when the selected protective packaging type includes inflatable protective packaging.

[0078] Clause 16: A method according to any of the above method clauses, wherein the inflation amount is selected based on at least one of a characteristic of the shipping container, a characteristic of the protective packaging material, a characteristic of at least one item placed within the shipping container, and a selected percentage.

[0079] Clause 17: The method of any of the above method clauses, further comprising causing an expansion module to expand the transport material by a selected expansion amount.

[0080] Clause 18: The method of any of the above method clauses, further comprising selecting whether to pack, stack, fold, roll, or crumple the transport material when the selected protective packaging type includes packing paper.

[0081] Clause 19: A method according to any of the method clauses above, wherein a selection is made as to whether the transport material should be packed, stacked, folded, rolled or crushed based on at least one of the characteristics of the transport container, the characteristics of the protective packaging material, the characteristics of at least one item placed in the transport container, and the selected percentage.

[0082] Clause 20: A system comprising: a processor; and a non-transitory computer-readable storage medium comprising programming instructions configured to cause the processor to implement a method for packaging a shipping container, the programming instructions comprising: instructions to detect the presence of the shipping container; instructions to generate a 3D point cloud comprising a 3D perspective representation of an interior of the shipping container and a surface of at least one item disposed within the shipping container; instructions to analyze the 3D point cloud to identify a void area within the shipping container; instructions to select a percentage from a plurality of possible percentages based on characteristics of the void area; and instructions to dispense protective packaging material into the void area such that the selected percentage of the void area is filled with the protective packaging material.

[0083] Clause 21: A non-transitory computer-readable medium storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform operations including: detecting the presence of a shipping container; generating a 3D point cloud including a 3D perspective representation of the interior of the shipping container and a surface of at least one item disposed within the shipping container; analyzing the 3D point cloud to identify a void area within the interior of the shipping container; selecting a percentage from a plurality of possible percentages based on characteristics of the void area; and dispensing protective packaging material into the void area such that the selected percentage of the void area is filled with protective packaging material.

[0084] The breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents. [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 488,023, filed March 2, 2023, the contents of which are incorporated herein by reference in their entirety.

Claims

1. 1. A method for packaging a shipping container, comprising: detecting, by a computing device, the presence of the shipping container; generating, by the computing device, a 3D point cloud including a 3D perspective representation of an interior of the shipping container and a surface of at least one item disposed within the shipping container; analyzing, by the computing device, the 3D point cloud to identify void areas within the shipping container, the void areas identifying the absence of the at least one item; selecting, by the computing device, a percentage from a plurality of possible percentages based on characteristics of the void region; and dispensing, by the computing device, protective packaging material into the void area such that the selected percentage of the void area is filled with protective packaging material.

2. The method of claim 1 , wherein the shipping container comprises a box, a bag, or an envelope or padded mailer.

3. dividing the void region into void region sections; The method of claim 1 , further comprising: selecting different percentages from a plurality of possible percentages for at least two of the void area sections based on characteristics of the void area sections.

4. 4. The method of claim 3, wherein the characteristics of the void area sections include at least one of a cubic volume, a relative position of the void area sections to one another, and a relative position of the void area sections to the at least one item disposed within the shipping container.

5. 10. The method of claim 1, wherein the cubic volume of the void area has a non-linear relationship to the percentage of the void area to be filled with the protective packaging material.

6. detecting, by the computing device, the occurrence of a trigger event; The method of claim 1 , further comprising: enabling a sensor responsive to the trigger event.

7. The method of claim 6 , wherein the trigger event comprises alignment of a center of the shipping container with an axis of the sensor.

8. The method of claim 7 , wherein detecting the trigger event is based on a depth measurement of at least one edge of the shipping container.

9. The method of claim 1 further comprising calculating a cubic volume of the void region.

10. The method of claim 9 , wherein the characteristics of the void region include the cubic volume.

11. 10. The method of claim 9, further comprising closing and sealing the shipping container without dispensing any protective packaging material therein when the cubic volume is not large enough to be filled with a threshold amount of protective packaging material.

12. 10. The method of claim 9, wherein dispensing the protective packaging material into the void region occurs when the cubic volume is large enough to be filled with a threshold amount of protective packaging material.

13. 10. The method of claim 9, further comprising: when the cubic volume is large enough to be filled with the threshold amount of protective packaging, selecting a protective packaging type from a plurality of possible protective packaging types based on the cubic volume or the type of the at least one item.

14. 14. The method of claim 13, wherein the plurality of protective packaging types includes at least an inflatable cushion, packing paper, or foam in place.

15. 14. The method of claim 13, further comprising selecting an inflation amount from a plurality of possible inflation amounts when the selected protective packaging type includes an inflatable protective packaging.

16. 16. The method of claim 15, wherein the expansion amount is selected based on at least one of a characteristic of the shipping container, a characteristic of the protective packaging, a characteristic of the at least one item disposed within the shipping container, and the selected percentage.

17. 16. The method of claim 15, further comprising causing an expansion module to expand the transport material by the selected expansion amount.

18. 14. The method of claim 13, further comprising selecting whether the shipping material should be baled, stacked, folded, rolled, or crushed when the selected protective packaging type includes wrapping paper.

19. 20. The method of claim 18, wherein a selection is made as to whether the shipping material should be packed, stacked, folded, rolled, or crushed based on at least one of the characteristics of the shipping container, the characteristics of the protective packaging, the characteristics of the at least one item placed within the shipping container, and the selected percentage.

20. a processor; 1. A system including a non-transitory computer-readable storage medium including programming instructions configured to cause the processor to implement a method for packaging a shipping container, the method comprising: The programming instructions: instructions to detect the presence of the shipping container; instructions to generate a 3D point cloud including a 3D perspective representation of an interior of the shipping container and a surface of at least one item disposed within the shipping container; instructions for analyzing the 3D point cloud to identify void regions within the shipping container, the void regions identifying the absence of the at least one item; instructions for selecting a percentage from a plurality of possible percentages based on characteristics of the void region; and instructions to dispense protective packaging material into the void area such that the selected percentage of the void area is filled with protective packaging material.

21. 1. A non-transitory computer-readable medium storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform an operation, comprising: The operation is Detecting the presence of a shipping container; generating a 3D point cloud including a 3D perspective representation of an interior of the shipping container and a surface of at least one item disposed within the shipping container; analyzing the 3D point cloud to identify void areas within the shipping container, the void areas identifying the absence of the at least one item; selecting a percentage from a plurality of possible percentages based on characteristics of the void region; and dispensing protective packaging material into the void area such that the selected percentage of the void area is filled with protective packaging material.

Images