Methods and systems for highlighting box surfaces and edges during the dimensioning of mobile boxes

Abstract

Method comprising: Obtaining a three-dimensional (3D) point cloud from a depth sensor (318) when the depth sensor (318) is positioned such that a target indicator (204) appears on a first surface (108) of an object; Processing the 3D point cloud to identify an extent of the first surface (108); Further processing the 3D point cloud to identify a second surface (110) adjacent and perpendicular to the first surface (108) and to identify an extent of the second surface (110); and Displaying at least a portion of the 3D point cloud via a user interface (312), wherein displaying at least a portion of the 3D point cloud comprises displaying the identified first surface (108) in a first color (702) and the identified second surface (110) in a second color (802) different from the first color (702); wherein identifying the second surface (110) comprises: identifying a first edge of the first surface (108); identifying a plurality of suitable second surfaces (110); Identifying at least one edge of at least one of the identified suitable second surfaces (110); and Identifying the second surface (110) as a suitable second surface (110) having an edge that extends coextensively with the identified first edge of the first surface (108).

Description

BACKGROUND OF THE INVENTION

[0001] Precise measurements of packages before shipment are a critical task in the transportation and logistics industry. It is beneficial for users to be able to measure the size of a package or packages from an image that also includes background data (e.g., other objects). This can be challenging when viewing an image in a graphical user interface (“GUI”). When determining package sizes, it is difficult to distinguish package images from the background image in a GUI. Good differentiation of package images and background image is important for accurately measuring package sizes. Accordingly, there is a need for methods and systems for highlighting crate surfaces and edges when dimensioning mobile crates.

[0002] US 2012 / 0256916 A1 describes a device for processing point cloud data. It extracts features of an object from its point cloud data and automatically generates a three-dimensional model. The device comprises a non-planar region removal unit for removing points of non-planar regions from the point cloud data, and a plane labeling unit for adding identical labels to points in the same planes, except for the points removed by the non-planar region removal unit, to segment the point cloud data into planes. The device also comprises a three-dimensional edge extraction unit and a two-dimensional edge extraction unit.

[0003] US 2013 / 0 181 983 A1 describes a device for processing point cloud data, comprising a unit for removing non-planar regions, a plane marking unit, and a contour calculation unit. The non-planar region removal unit removes point cloud data related to non-planar regions from point cloud data, since the non-planar regions represent a high computational burden. In the point cloud data, a two-dimensional image of an object is linked to three-dimensional coordinate data of multiple points that form the two-dimensional image. The plane marking unit adds plane markings to the point cloud data, in which the data of the non-planar regions are removed.The contour calculation unit calculates a contour of the object using local flat planes based on a local area connected to the designated plane. BRIEF DESCRIPTION OF THE DIFFERENT VIEWS OF THE DRAWINGS

[0004] The accompanying figures, in which like reference numerals designate identical or functionally similar elements throughout the several views, together with the following detailed description, are part of the disclosure and serve to further illustrate embodiments of concepts comprising the claimed invention and to explain various principles and advantages of those embodiments. Fig. shows an example scenario including a plurality of example packages and a user holding an example portable computing device, according to some embodiments. Fig. shows a first exemplary front view of the computer device of Fig. according to some embodiments. Fig. shows an exemplary architectural view of the computer device of Fig. according to some embodiments. Fig. shows an example method according to some embodiments. Fig. shows a first outline view of the exemplary packages of Fig. according to some embodiments. Fig. shows a second outline view of the exemplary packages of Fig. according to some embodiments. Fig. shows a second exemplary front view of the computer device of Fig. according to some embodiments. Fig. shows a third exemplary front view of the computer device of Fig. according to some embodiments. Fig. shows a third outline view of the exemplary packages of Fig. according to some embodiments. Fig. shows a fourth exemplary front view of the computer device of Fig. according to some embodiments.

[0005] Those skilled in the art will appreciate that elements in the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to enhance understanding of embodiments of the present invention.

[0006] The apparatus and method steps have been represented, where appropriate, by conventional symbols in the drawings which show only those specific details relevant to an understanding of the embodiments of the present invention, so as not to obscure the disclosure with details which would be readily apparent to those skilled in the art having recourse to the present description. DETAILED DESCRIPTION

[0007] Disclosed herein are methods and systems for highlighting box surfaces and edges when dimensioning mobile boxes.

[0008] According to the invention, a method is provided comprising (a) obtaining a three-dimensional (3D) point cloud from a depth sensor when the depth sensor is positioned such that a target indicator appears on a first surface of an object, (b) processing the 3D point cloud to identify an extent of the first surface, (c) further processing the 3D point cloud to identify a second surface adjacent and perpendicular to the first surface and to identify an extent of the second surface, and (d) displaying at least a portion of the 3D point cloud via a user interface, wherein displaying at least a portion of the 3D point cloud comprises displaying the identified first surface in a first color and the identified second surface in a second color different from the first color.Identifying the second surface comprises (i) identifying a first edge of the first surface, (ii) identifying a plurality of suitable second surfaces, (iii) identifying the at least one edge of at least one of the identified suitable second surfaces, and (iv) identifying the second surface as a suitable second surface having an edge that extends coextensively with the identified first edge of the first surface.

[0009] According to the invention, a system is provided which comprises a user interface, a depth sensor, a processor and a data memory containing instructions executable by the processor to perform at least the functions described in the preceding paragraph.

[0010] In at least one embodiment, the depth sensor and the user interface are both components of a single handset.

[0011] In at least one embodiment, the target indicator appearing on the first surface of the object comprises the target indicator appearing in the user interface to be positioned on the first surface of the object.

[0012] In at least one embodiment, the target indicator is shaped like a square.

[0013] In at least one embodiment, the target indicator is shaped like crosshairs.

[0014] In at least one embodiment, the target indicator appearing on the first surface of the object comprises the target indicator actually projected onto the first surface of the object.

[0015] In at least one embodiment, the target indicator appears at a target indicator location on the first surface, and identifying the extent of the first surface comprises (i) calculating a target indicator normal unit vector at the target indicator location; and (ii) identifying the extent of the first surface as a first surface group of points in the 3D point cloud, wherein the first surface group is contiguous and includes the target indicator location, and wherein each point in the first surface group has a corresponding normal unit vector that corresponds to the target indicator normal unit vector.

[0016] In at least one embodiment, identifying the second surface comprises (i) identifying a second surface point that is (a) adjacent to the first surface and (b) has a second surface point normal unit vector that is perpendicular to the target indicator normal unit vector; and (ii) identifying a surface that has the second surface point as the second surface.

[0017] In at least one embodiment, identifying the extent of the second surface comprises identifying the extent of the second surface as a group of points of the second surface in the 3D point cloud, wherein the second surface group is contiguous and includes the second surface point, and wherein each point in the second surface group has a corresponding normal unit vector equal to the second surface point normal unit vector.

[0018] According to the invention, identifying the second surface comprises (i) identifying a first edge of the first surface, (ii) identifying a plurality of suitable second surfaces, (iii) identifying the at least one edge of at least one of the identified suitable second surfaces, and (iv) identifying the second surface as a suitable second surface having an edge that extends coextensively with the identified first edge of the first surface.

[0019] In at least one embodiment, one of the first and second colors is orange and the other of the first and second colors is either blue or green.

[0020] In at least one embodiment, the method further comprises identifying at least two edges among the identified first and second surfaces, and displaying at least a portion of the 3D point cloud comprises displaying the at least two identified edges in a third color that is different from both the first and second colors.

[0021] In at least one embodiment, each of the first, second, and third colors has a mutual contrasting relationship with each of the other two of the first, second, and third colors.

[0022] In at least one embodiment, the at least two edges among the identified first and second surfaces comprise two edges of the first surface and one edge of the second surface, all connected at a common corner of the object.

[0023] In at least one embodiment, the third color is yellow.

[0024] In at least one embodiment, one of the first and second colors is orange and the other of the first and second colors is either blue or green.

[0025] In at least one embodiment, the first and second colors have a mutual contrasting relationship.

[0026] In at least one embodiment, displaying a given surface in a given color comprises displaying the extent of the given surface in the given color.

[0027] In at least one embodiment, the method further comprises, while the identified first surface in the first color and the identified second surface in the second color are displayed via the user interface, receiving a data acquisition command via the user interface, and in response thereto storing volumetric data characteristic of the object.

[0028] Furthermore, any of the variations and permutations described herein may be implemented with respect to any embodiments, including with respect to any method embodiments and with respect to any system embodiments. Furthermore, this flexibility and multiple applicability of embodiments exists despite the use of somewhat different language (e.g., process, method, steps, functions, set of functions, and the like) to describe and characterize such embodiments.

[0029] Before proceeding with this Detailed Description, it should be noted that the entities, connections, arrangements, and the like illustrated and described in the various figures are presented by way of example and not by way of limitation. Thus, any and all statements or other references to what a particular figure "depicts," what a particular element or entity in a particular figure "is" or "has," and any and all similar statements—which, taken in isolation and out of context, may be read as absolute and therefore limiting—can only be properly read as constructively preceded by a phrase such as "In at least one embodiment,..." And for reasons of brevity and clarity of illustration, this implied heading clause will not be repeated in this Detailed Description.

[0030] Fig. shows an example scenario including a plurality of example packages and a user holding an example portable computing device, according to some embodiments. In particular, Fig. a handset 102, packages 104, and a user 106. The packages 104 are substantially cubic in shape (i.e., each side of each package 104 is substantially rectangular in shape (e.g., square)) and could have any number of different sizes, as illustrated in examples 104A, 104B, and 104C. Also in Fig. depicted are a top surface 108 and a front surface 110 of the package 104A, as well as (i) a first edge 108A and a second edge 108B of the top surface 108A, and (ii) a first edge 110A and a second edge 110B of the front surface 110. In the depicted example scenario, the front surface 110 of the package 104A faces the user 106.

[0031] The user 106 may position the handset 102 such that the handset 102 is generally directed toward the packages 104 and is directed toward a surface of a package. In the illustrated and described example, the user 106 positions the handset 102 such that the handset 102 (and in particular, a target indicator of the handset 102) is directed toward a first surface 108 (i.e., the top surface in the example scenario described in Fig. shown) of the package 104A. This is exemplary and not limiting, as the user 106 could point the handset 102 at any surface of any package.

[0032] Fig. shows a first exemplary front view of the computer device of Fig. according to some embodiments. In particular, Fig. the handset 102 with a touchscreen 202 and a target indicator 204. In Fig. The target indicator 204 is displayed on the touchscreen 202 such that it appears to the user 106 that the target indicator 204 appears on the top surface 108 of the package 104A. In some embodiments, the target indicator 204 comprises a light beam (e.g., a laser beam) that is actually projected from the handset 102 onto a surface of a package. In any event, in operation, the target indicator 204 appears to the user viewing the touchscreen 202 of the handset 102 to appear on a surface of a package (when properly aligned). As previously mentioned in the illustrated and described example, the target indicator 204 appears to be on the top surface 108 of the package 104A.

[0033] Fig. shows an architectural view of the exemplary computing device of Fig. according to some embodiments. The handheld device 102 may be configured to perform the functions described herein and, as illustrated, includes a communications interface 302, a processor 304, a data memory 306 (containing program instructions 308 and operational data 310), a user interface 312 (including the touchscreen 202 and a data acquisition element 314), peripherals 316 (including a depth sensor 318), and a communications bus 320. This arrangement is presented by way of example and not by way of limitation, as other example arrangements could be described herein.

[0034] The communication interface 302 may be configured to be operable for communication according to one or more wireless communication protocols, some examples of which include LMR, LTE, APCO P25, ETSI DMR, TETRA, Wi-Fi, Bluetooth, and the like. The communication interface 302 may also, or instead, include one or more wired communication interfaces (for communication, for example, according to Ethernet, USB, and / or one or more other protocols). The communication interface 302 may include any necessary hardware (e.g., chipsets, antennas, Ethernet interfaces, etc.), any necessary firmware, and any necessary software for performing one or more forms of communication with one or more other entities as described herein.

[0035] Processor 304 may include one or more processors of any type deemed suitable by those skilled in the art, with some examples including a general microprocessor and a dedicated digital signal processor (DSP).

[0036] The data storage 306 may take the form of any non-transitory computer-readable medium or a combination of such media, with some examples including flash memory, read-only memory (ROM), and random-access memory (RAM), to name a few, as any one or more types of non-transitory data storage technology deemed suitable by those skilled in the relevant art could be used. As in Fig. As shown, data storage 306 includes program instructions 308 executable by processor 304 to perform various functions described herein, and is further shown to include operational data 310, which may include one or more data values ​​stored and / or accessed by handheld device 102 in performing one or more of the functions described herein. In at least one embodiment, handheld device 102 includes a Matrix Lab (MATLAB) program instruction 308. And, of course, other examples could be cited.

[0037] The user interface 312 may include one or more input devices (i.e., components and the like) and / or one or more output devices (i.e., components and the like). With respect to input devices, the user interface 312 may include one or more touchscreens, buttons, switches, microphones, and the like. With respect to output devices, the user interface 312 may include one or more displays, speakers, light-emitting diodes (LEDs), and the like. Furthermore, one or more components (e.g., an interactive touchscreen and a display) of the user interface 312 may provide both user input and user output functionality. As in Fig. The user interface 312 includes the touchscreen 202 and the data acquisition element 314. Other handheld user interface components may also be present, as known to those skilled in the art.

[0038] The peripherals 316 may include any handset 102 accessory, component, or the like that can be accessed and used by the handset 102 during operation. As shown in Fig. As shown, the peripherals 316 include the depth sensor 318. There are a number of types of depth sensors 318 that could be used, possibly one that includes an RGB sensor, possibly leap motion, possibly Intel perceptual computing, possibly Microsoft Kinect, among numerous other possibilities that could be listed here. There are also a number of depth sensing methods that could be implemented by the depth sensor 318, possibly using stereo triangulation, possibly using time of flight, possibly using a coded aperture, among numerous other possibilities that could be listed here. This set of information (i.e.points) is referred to herein as a 3D point cloud (or sometimes simply a point cloud); each point in such a cloud corresponds to the perceived packets 104 at a corresponding location in the field of view of the depth sensor 318.

[0039] Fig. shows an exemplary method according to some embodiments. In particular, Fig. a method 400 comprising steps 402, 404, 406, and 408, which is described below by way of example as being performed by the handset 102, although the method 400 could generally be performed by any suitably equipped, programmed, and configured computing device.

[0040] In step 402, the handheld device 102 obtains a 3D point cloud using the depth sensor 318 at a time when the handheld device 102 is positioned such that the target indicator 204 appears on the top surface 108 of the package 104A. The 3D point cloud includes a plurality of points, each having a corresponding normal vector. The target indicator 204 appearing on the top surface 108 of the package 104A appears in the user interface 202. The target indicator 204 may appear in the user interface 202 in various ways. In one embodiment, the target indicator 204 is shaped like a square or crosshairs. In another embodiment, the target indicator 204 is projected onto the first surface of the package 104A. Of course, other example implementations are possible.

[0041] In step 404, the handheld device 102 processes the 3D point cloud to identify an extent of the first surface of the parcel 104A. The handheld device 102 may do this by first calculating a target indicator normal unit vector, as shown in Fig. is shown. In this case and elsewhere in this disclosure, the term normal unit vector is used to refer to a normal vector that has a unit length (i.e., default, standard, etc.) to enable comparisons of different normal unit vectors with one another in a context where the orientation (i.e., direction) of such normal vectors is important and where the magnitude of such normal vectors is not of particular importance.

[0042] A normal vector is a local geometric property of a 3D surface and is specific to a particular point. It should be apparent to one of ordinary skill in the art that a reliable estimation of the normal vector at a given point within a point cloud dataset thus depends on identifying the neighboring points of the given point. There are a number of different ways in which the handheld device 102 can identify the neighboring points of the normal vector and calculate the normal vector at a point within the 3D point cloud. In at least one embodiment, the handheld device 102 uses a fixed number of Euclidean nearest neighbor points to estimate the normal vector at a given point within the 3D point cloud.In another embodiment, the handheld device 102 identifies the neighbor point of the normal vector by constructing a polygon mesh to identify the neighboring points according to the connected polygon facets. The handheld device 102 then calculates the normal vector by fitting the identified neighboring points to a plane and using the normal of the fitted plane as the normal vector. Of course, other example implementations are possible. In some embodiments, the handheld device 102 uses a library-provided function to calculate the normal vector of a surface at a given point. One example is MATLAB's "surfnorm" function.

[0043] Fig. shows a first outline view of the sample packages of Fig. according to some embodiments. In particular, Fig. an outline of the packages 104. The outline should generally correspond to the packages 104 shown in Fig. are shown to help the reader visualize an example real-world scenario from which the example 3D point cloud could be derived, collected, or the like. In addition, for illustration purposes, various points in the 3D point cloud are Fig. shown with corresponding normal unit vectors corresponding to the top surface 108 of the package 104A. In actual implementations, any number of normal unit vectors could be calculated, since the various normal unit vectors shown in Fig. are for illustrative purposes only and are not to be understood as exhaustive.

[0044] As in Fig. Illustrated are the target indicator 204, a target indicator normal unit vector 502, and a first surface group of normal unit vectors 504. The target indicator normal unit vector 502 is a normal vector corresponding to a point representative of the position of the target indicator 204 on the package 104A.

[0045] Back to Fig. Where, in step 404, reference is made to how the handheld device 102 may identify an extent of the top surface 108 of the parcel 104A. In at least one embodiment, the handheld device 102 identifies the extent of the top surface 108 as a first surface group of points having respective normal unit vectors 504 equal to the target indicator normal unit vector 502 in the 3D point cloud, this first surface group of points being contiguous and including the point at which the target indicator normal unit vector 502 originates from the top surface 108 of the parcel 104A. Given that the top surface 108 of the parcel 104A may not be completely flat, in at least one embodiment, two normal unit vectors may be considered equal to each other if their directions are within a marginal tolerance of each other. And, of course, other example implementations are possible.

[0046] In step 406, the handheld device 102 further processes the 3D point cloud to identify a second surface—in this case, the front surface 110 of the package 104A. In at least one embodiment, the handheld device 102 may do this by first calculating a second surface point normal vector, as shown in Fig. is shown.

[0047] Fig. shows a second outline view of the exemplary packages of Fig. according to some embodiments. In particular, Fig. the outline of the parcels 104, the target indicator normal unit vector 502, the top surface 108, a first surface edge 606, the front surface 110, a second surface point 602, a second surface point normal vector 604, and a group of second surface normal unit vectors 608. The second surface point 602 is a point on the second surface 110. The second surface point normal unit vector 604 is a normal vector corresponding to the second surface point 602. Furthermore, the front surface 110 is adjacent to and perpendicular to the top surface 108 of the parcel 104A. The second surface group of normal unit vectors 608 represents the extent of the second surface 110 of the parcel 104A.

[0048] In operation, the handheld device 102 can thus calculate the target indicator normal unit vector 502 at the point where the target indicator 204 appears on the surface on which the user has chosen to orient the target indicator 204. In the parlance of this disclosure, this surface is called the first surface. In the example illustrated and described, the first surface is the top surface 108 of the packet 104A. In this manner, the target indicator normal unit vector 502 forms the normal unit vector for all points on the top surface 108. The handheld device 102 then determines the extent of the top surface 108 by progressing in several different directions in the point cloud data, calculating at each point the normal unit vector of the surface at that point. If the normal unit vector at a particular point matches the target indicator normal unit vector 502 (e.g.,close enough), the handset 102 considers this given point as still part of the upper surface 108 and continues outward to the next point.

[0049] Once the handheld device 102 reaches a point where the calculated normal unit vector is not equal to the target indicator normal unit vector 502 (e.g., not close enough), the handheld device determines that it has reached an edge of the top surface 108. In various embodiments, the handheld device 102 may require a certain number of points in a row, or a certain fraction of the previously determined number of points, to have a non-equal normal unit vector in order to determine that an edge of the top surface 108 has been reached. This can prevent anomalous data from being misinterpreted as an edge.

[0050] Once the handheld device 102 has performed this analysis in enough directions to identify the four edges of the top surface 108, the handheld device can then move beyond one of those edges to attempt to identify a second surface that is both adjacent to and perpendicular to the already identified first surface. Ideally, such a second surface would be visually confirmed by the user as a surface of the package 104A in which the user 106 is interested, although it could just as easily be a surface of a filing cabinet or a wall or another package or the like against which the package 104A is resting. In the example illustrated and described, the second surface is actually another surface of the package 104A; in this case, it is the front surface 110 of the package 104A.

[0051] The handheld device 102 can search for the second surface by advancing to points in the point cloud that are on the other side (from the target indicator point) of a now-identified edge of the upper surface 108 of the package 104A. In Fig. It can be seen that a leading edge 606 of the upper surface 108 has been identified. The handheld device 102 can proceed beyond this edge 606, continuing to calculate normal unit vectors at each point it passes until it reaches a point (such as point 602) that has a normal unit vector 604 that is perpendicular to the target indicator normal unit vector 502. The handheld device 102 can then follow a similar process as described above to continue in several different directions to identify the extent of the second surface. The extent of the second surface (i.e., the front surface 110 of the package 104A) is in Fig. graphically represented as the group 608 of normal unit vectors.

[0052] In some embodiments, the handheld device 102 will not interpret a given surface as "the second surface," as that term is used herein, unless that given surface shares an edge with an identified edge of the first surface. Thus, if the handheld device 102 initially calculated the normal unit vectors on a surface that was adjacent and perpendicular to the first surface, but did not share an edge (i.e., itself had an edge coextensive with an identified edge of the top surface 108), the handheld device 102 would discard that surface and move to another surface that is both adjacent and perpendicular to the first surface and check whether that next surface shares an edge with the top surface 108.Thus, while the front surface 110 shares the edge 606 with the top surface 108, some examples of surfaces that are adjacent and perpendicular to the top surface 108 but do not share an edge with the top surface 108 include walls, floors, sides of filing cabinets, sides of other differently sized boxes, etc. In one embodiment, for two surfaces to share an edge, an edge of one surface must be coextensive with an edge of the other surface. And, of course, other example implementations are possible.

[0053] Back to Fig. , in step 408, the handheld device 102 displays at least a portion of the 3D point cloud via the touchscreen 202, wherein displaying at least a portion of the 3D point cloud comprises the handheld device 102 displaying the identified top surface 108 of the package 104A in a first color and displaying the identified front surface 110 of the package 104A in a second color that is different from the first color.

[0054] The handheld device 102 can display the identified top surface 108 of the package 104A in any number of colors. Similarly, the handheld device 102 can display the identified front surface 110 of the package 104A in any number of colors. However, the top surface 108 of the package 104A and the front surface 110 of the package 104A are displayed in different colors. In at least one embodiment, the handheld device 102 displays, via the touchscreen 202, the top surface 108 of the package 104A in an orange color and the front surface 110 of the package 104A in a blue color. In another embodiment, the handheld device 102 displays, via the touchscreen 102, the top surface 108 of the package 104A in a red color and the front surface 110 of the package 104A in a purple color. And, of course, other example implementations are possible.

[0055] Fig. shows a second exemplary front view of the computer device of Fig. according to some embodiments. In particular, Fig. the handheld device 102, which includes the touchscreen 202 currently displaying the data capture element 314; the package 104A, which includes the top surface 108 (displayed in a top surface color 702), and the front surface 110. The top surface color 702 is depicted as a square grid pattern on the top surface 108 of the package 104A. The square grid pattern is intended to generally correspond to the top surface color 702 of the package 104A to help the reader visualize an example of a real-world scenario.It is further noted that in some embodiments, the handheld device 102 displaying a given surface in a given color takes the form of the handheld device 102 displaying a portion of the given surface in the given color; in other embodiments, the handheld device 102 displaying a given surface in a given color takes the form of the handheld device 102 displaying the full extent of the given surface in the given color.

[0056] Fig. shows a third exemplary front view of the computer device of Fig. according to some embodiments. In particular, Fig. all aspects that Fig. displayed, and additionally shows the front surface 110 of the package 104A displayed in a front surface color 802. The front surface color 802 is depicted as a diagonal grid pattern on the front surface 110 of the package 104A. The diagonal grid pattern is intended to generally correspond to the front surface color 802 on the front surface 110 to help the reader visualize an example of a real-world scenario. It is noted that two different geometric patterns are used to depict the top surface color 702 and the front surface color 802 to emphasize that these two colors 702 and 802 are distinct from each other. In at least one embodiment, the colors 702 and 802 have a mutually contrasting relationship (e.g., orange and green) with each other.

[0057] In some embodiments, the handheld device 102 also identifies at least two edges among the edges of the top surface 108 and the front surface 110. In at least some of these embodiments, when displaying at least a portion of the 3D point cloud, the handheld device 102 displays the at least two identified edges of the package 104A in a third color that is different from the top surface color 702 and the front surface color 802. In at least one such embodiment, the third color has a mutually contrasting relationship with both the top surface color 702 and the front surface color 802. In one embodiment, the third color is yellow (e.g., a bright, highlighter-like yellow).

[0058] Fig. shows a third outline view of the exemplary packages of Fig. according to some embodiments. In particular, Fig. the outline of the packets 104, the packet 104A, the top surface 108, the first edge of the top surface 108A, the second edge of the top surface 108B, the front surface 110, the first edge of the front surface 110A, the second edge of the front surface 110B, the target indicator normal unit vector 502, the second surface point 602, the normal unit vector 604 of the second surface point, and a common vertex 902.

[0059] As described above, according to one embodiment, once the handheld device 102 reaches a point where the calculated normal unit vector is not equal to (e.g., is not close enough to) the target indicator normal unit vector 502, the handheld device 102 identifies at least two edges of the top surface 108 by determining that it has reached the first edge of the top surface 108A. After further analysis in sufficient directions, the handheld device 102 identifies the second edge of the top surface 108B.

[0060] Further, as described above, in at least one embodiment, the handheld device 102 identifies the front surface 110 by progressing to points in the point cloud that lie on the other side (from the target indicator point) of the first edge of the top surface 108A of the package 104A. The handheld device 102 progresses beyond the first edge of the top surface 108A of the package 104A, calculating normal unit vectors at each point it passes until it reaches a point (such as point 602) that has the normal unit vector 604 of the second surface point perpendicular to the target indicator normal unit vector 502. The handheld device 102 then follows a similar process as described above to progress in several different directions to identify the first edge of the front surface 110A and the second edge of the front surface 110B.In one embodiment, the handset 102 identifies that edges 108A (which are also 110A), 108B, and 110B are connected to each other at the common vertex 902 of the packet 104A.

[0061] The handset 102 may display at least two identified edges of the package 104A in any number of third colors. However, three different colors are used for each of (i) the at least two identified edges of the package 104A, (ii) the top surface 108 of the package 104A, and (iii) the front surface 110 of the package 104A. For example, in one embodiment, the handset 102 displays, via the touchscreen 202, (i) the edges 108A / 110A, 108B, and 110B in a green color; (ii) the top surface 108 of the package 104A in an orange color; and (iii) the front surface 110 of the package 104A in a blue color. And, of course, other example implementations are possible.

[0062] Fig. shows a fourth exemplary front view of the computer device of Fig. according to some embodiments. In particular, Fig. the handset 102, the touchscreen 202, the data capture element 314, the package 104A, the first surface 108, the first surface color 702, the second surface 110, the second surface color 802, and a third edge color 1002.

[0063] The third edge color 1002 is illustrated as a dotted pattern on each of the edges 108A / 110A, 108B, and 110B of the package 104A. The dotted pattern is intended to generally correspond to the third edge color 1002 to help the reader visualize an example of a real-world scenario.

[0064] In some embodiments, while displaying at least a portion of the 3D point cloud with the identified surfaces and edges highlighted in three different colors, the handheld device 102 may receive a data collection command via the user interface (e.g., via the touchscreen 202). In some embodiments, this takes the form of detecting an actuation of the data collection element (e.g., a soft button) 314, which in Fig. is shown.

[0065] In an exemplary situation, the user may actuate the data capture element 314 if the user determines that the two highlighted surfaces (108 and 110) are both on the package in which the user 106 is interested (i.e., the one the user 106 has aimed at with the target indicator 204). Note that in cases where the handset 102 highlights the top surface 108 and also a second surface that is not part of the package 104A, the user may again attempt to change the angle, move the package 104A so that it is not near something else, and / or perform one or more other actions.

[0066] Upon detecting an actuation of the data capture element 314, the handheld device 102 may, in response, capture and store volumetric data characteristic of the package 104. This volumetric data could include various dimensions (e.g., length, width, depth) of the package 104. This volumetric data could include a calculated volume of the package 104. Of course, other example implementations are possible.

[0067] In some embodiments, the handheld device 102 performs a cost calculation based on a calculated volume to bill a customer. In some embodiments, the handheld device 102 transmits the volumetric data and / or one or more values ​​(e.g., shipping costs) derived therefrom to one or more other networked computer systems (e.g., a package workflow tracking system). Of course, other example implementations are possible.

[0068] While specific embodiments have been described in the foregoing specification, one of ordinary skill in the art will recognize that various modifications and changes may be made without departing from the scope of the invention as defined in the following claims. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present teachings.

[0069] The benefits, advantages, solutions to problems, and any elements that may result in the occurrence or enhancement of a benefit, advantage, or solution are not to be construed as critical, required, or essential features or elements in the claims. The invention is defined solely by the appended claims, including any amendments made during the pendency of this application and all equivalents of the claims as issued.

[0070] Furthermore, in this document, relational terms such as first and second, upper and lower, and the like may be used merely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprises," "comprising," "has," "having," "comprising," "includes," "includes," or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises, has, has, or contains a list of elements not only includes those elements, but may also include other elements not expressly listed or inherent in such process, method, product, or apparatus. An element that is "comprised of," "has," "has," or "includes"a" does not exclude, without further limitation, the existence of additional identical elements in the process, method, product, or apparatus comprising, having, including, or containing the element. The terms "a" and "an" are defined as one or more unless expressly stated otherwise herein. The terms "substantially," "generally," "approximately," "about," or any other version thereof are defined as being approximately understood by one of ordinary skill in the art, and in one non-limiting embodiment, the term is defined as within 10%, in another embodiment, within 5%, in another embodiment, within 1%, and in yet another embodiment, within 0.5%. The term "coupled," as used herein, is defined as being connected, but not necessarily directly and not necessarily mechanically.A device or structure that is “constructed” in a particular way is at least constructed that way, but may also be constructed in ways that are not listed.

[0071] It should be understood that some embodiments may be comprised of one or more generic or specialized processors (or "processing devices") such as microprocessors, digital signal processors, custom processors, and field-programmable gate arrays (FPGAs), and unique stored program instructions (including both software and firmware) controlling the one or more processors to, in conjunction with certain non-processor circuitry, implement some, most, or all of the functions of the method and / or apparatus described herein. Alternatively, some or all of the functions may be implemented by a state machine that does not have stored program instructions, or in one or more application-specific integrated circuits (ASICs) in which each function, or some combination of certain functions, are implemented as user-defined logic.Of course, a combination of the two approaches can be used.

[0072] Furthermore, an embodiment may be implemented as a computer-readable storage medium having computer-readable code stored thereon for programming a computer (e.g., including a processor) to perform a method as described and claimed herein. Examples of such computer-readable storage media include, but are not limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (read-only memory), a PROM (programmable read-only memory), an EPROM (erasable programmable read-only memory), an EEPROM (electrically erasable programmable read-only memory), and a flash memory.It is further believed that, notwithstanding possible significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, one of ordinary skill in the art will readily be able to generate such software instructions and programs and ICs with minimal experimentation when guided by the concepts and principles disclosed herein.

[0073] The Summary of Disclosure is provided to enable the reader to quickly ascertain the nature of the technical disclosure. It is provided with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Further, it can be seen from the foregoing Detailed Description that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This manner of disclosure should not be construed to reflect an intent that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims demonstrate, inventive subject matter lies in fewer than all of the features of a single disclosed embodiment.Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter.

Claims

[1] Method comprising: Obtaining a three-dimensional (3D) point cloud from a depth sensor (318) when the depth sensor (318) is positioned such that a target indicator (204) appears on a first surface (108) of an object; Processing the 3D point cloud to identify an extent of the first surface (108); Further processing the 3D point cloud to identify a second surface (110) adjacent and perpendicular to the first surface (108) and to identify an extent of the second surface (110); and Displaying at least a portion of the 3D point cloud via a user interface (312), wherein displaying at least a portion of the 3D point cloud comprises displaying the identified first surface (108) in a first color (702) and the identified second surface (110) in a second color (802) different from the first color (702); wherein identifying the second surface (110) comprises: identifying a first edge of the first surface (108); identifying a plurality of suitable second surfaces (110); Identifying at least one edge of at least one of the identified suitable second surfaces (110); and Identifying the second surface (110) as a suitable second surface (110) having an edge that extends coextensively with the identified first edge of the first surface (108). [2] The method of claim 1, wherein the depth sensor (318) and the user interface (312) are both components of a single handset (102). [3] The method of claim 1, wherein the target indicator (204) appearing on the first surface (108) of the object comprises the target indicator (204) appearing in the user interface (312) to be positioned on the first surface (108) of the object. [4] The method of claim 3, wherein the target indicator (204) is shaped like a square. [5] The method of claim 3, wherein the target indicator (204) is shaped like a crosshair. [6] The method of claim 1, wherein the target indicator (204) appearing on the first surface (108) of the object comprises actually projecting the target indicator (204) onto the first surface (108) of the object. [7] The method of claim 1, wherein the target indicator (204) appears at a target indicator location on the first surface (108), and wherein identifying the extent of the first surface (108) comprises: Calculating a target indicator normal unit vector at the target indicator location; and Identifying the extent of the first surface (108) as a first surface group of points in the 3D point cloud, wherein the first surface group is contiguous and includes the target indicator location, and wherein each point in the first surface group has a corresponding normal unit vector equal to the target indicator normal unit vector. [8] The method of claim 7, wherein identifying the second surface (110) comprises: Identifying a second surface point that (i) is adjacent to the first surface (108) and (ii) has a second surface point normal unit vector that is perpendicular to the target indicator normal unit vector; and Identifying a surface having the second surface point as the second surface (110). [9] The method of claim 8, wherein identifying the extent of the second surface (110) comprises identifying the extent of the second surface (110) as a second surface group of points in the 3D point cloud, the second surface group being contiguous and including the second surface point, and each point in the second surface group having a corresponding normal unit vector equal to the normal unit vector of the second surface point. [10] The method of claim 1, wherein: one of the first and second colors (702, 802) is orange; and the other of the first and second colors (702, 802) is either blue or green. [11] The method of claim 1, further comprising: Identifying at least two edges among the identified first and second surfaces (108, 110), wherein displaying at least a portion of the 3D point cloud comprises displaying the at least two identified edges in a third color that is different from both the first and second colors (802). [12] The method of claim 11, wherein each of the first, second and third colors has a mutual contrasting relationship with each of the other two of the first, second and third colors. [13] The method of claim 11, wherein the at least two edges among the identified first and second surfaces (108, 110) comprise two edges of the first surface (108) and one edge of the second surface (110), all connected at a common corner of the object. [14] The method of claim 11, wherein the third color is yellow. [15] The method of claim 14, wherein: one of the first and second colors (702, 802) is orange; and the other of the first and second colors (702, 802) is either blue or green. [16] The method of claim 1, wherein the first and second colors (702, 802) have a mutual high-contrast relationship. [17] The method of claim 1, wherein displaying a given surface in a given color comprises displaying the extent of the given surface in the given color. [18] The method of claim 1, further comprising: while displaying the identified first surface (108) in the first color (702) and the identified second surface (110) in the second color (802) via the user interface (312), receiving a data acquisition command via the user interface (312) and, in response thereto, storing volumetric data characteristic of the object. [19] System comprising: a user interface (312); a depth sensor (318); a processor (304); and a data memory containing instructions executable by the processor (304) to cause the system to perform a set of functions, the set of functions comprising: Obtaining a three-dimensional (3D) point cloud from the depth sensor (318) when the depth sensor (318) is positioned such that a target indicator (204) appears on a first surface (108) of an object; Processing the 3D point cloud to identify an extent of the first surface (108); Further processing the 3D point cloud to identify a second surface (110) adjacent and perpendicular to the first surface (108) and to identify an extent of the second surface (110); and Displaying at least a portion of the 3D point cloud via the user interface (312), wherein displaying at least a portion of the 3D point cloud comprises displaying the identified first surface (108) in a first color (702) and the identified second surface (110) in a second color (802) different from the first color (702); wherein identifying the second surface (110) comprises: identifying a first edge of the first surface (108); identifying a plurality of suitable second surfaces (110); Identifying at least one edge of at least one of the identified suitable second surfaces (110); and Identifying the second surface (110) as a suitable second surface (110) having an edge that extends coextensively with the identified first edge of the first surface (108).

Images