Transport segmentation for data capture systems
By obtaining the point cloud of the transport tool and using calibration data to select parts other than the main body and cage of the transport tool, the precise measurement of the package size is achieved, and the problem of time-consuming and error-prone measurement in the prior art is solved, and the measurement efficiency is improved and precision.
Patent Information
- Application Number
- CN202110214455.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-02-25
- Filing Date
- 2021-02-25
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-02-25
AI Technical Summary
The prior art has time-consuming and error-prone problems in measuring package sizes, especially when the package moves or has a dark surface, and the accuracy of the automatic measurement system is difficult to guarantee.
A server is designed to obtain a point cloud that depicts the volume of the transport tool, and select parts other than the transport tool body and cage based on calibration data to dimension the object. The system includes memory and processors, which are configured to obtain locations and marks associated with the transport tool, and use calibration data to select specific parts in the point cloud to achieve precise dimensioning of objects.
Improves the efficiency and accuracy of package size measurement, reduces manual measurement errors, can adapt to the measurement needs of moving or dark surface wraps, and provide more accurate dimensioning results.
Smart Images

Figure CN113379827B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to data capture systems and, more particularly, to transport segmentation for data capture systems. Background Art
[0002] The transportation and storage of objects such as packages may require knowledge of the dimensions of the packages. Such information may be used to optimize the use of available space in a container (e.g., a trailer), to determine the shipping or storage costs of a package, etc. However, the package dimensions may not be known in advance, so workers may need to obtain the package dimensions by manually measuring the package. Taking manual measurements is time consuming and prone to error. Systems for automatically measuring package dimensions may also suffer from reduced accuracy, for example, when measuring packages in motion, packages with dark (e.g., black) surfaces, etc. Summary of the invention
[0003] One or more embodiments of the present disclosure are directed to a server. The server includes: a memory storing calibration data; and a processor connected to the memory, the processor being configured to: obtain a point cloud depicting a capture volume containing a transport having a body and a holder carrying an object to be dimensioned; obtain a set of positions associated with the transport, the set of positions further associated with a marker positioned on the transport; based on the set of positions and the calibration data, select a first portion of the point cloud other than the body of the transport and a mast of the holder; based on the calibration data, select a second portion of the point cloud other than a base of the holder from the first portion; and dimension the object based on the second portion of the point cloud, wherein the calibration data includes positions of structural features of the transport relative to the marker positioned on the transport.
[0004] Additional embodiments of the present disclosure are directed to a method. The method includes: storing calibration data; obtaining a point cloud depicting a capture volume containing a transport having a body and a cage carrying an object to be dimensioned; obtaining a set of positions associated with the transport, the set of positions further associated with a marker positioned on the transport; selecting a first portion of the point cloud other than the body of the transport and a mast of the cage based on the set of positions and the calibration data; selecting a second portion of the point cloud other than a base of the cage based on the calibration data; and dimensioning the object based on the second portion of the point cloud, wherein the calibration data includes positions of structural features of the transport relative to the marker positioned on the transport. BRIEF DESCRIPTION OF THE DRAWINGS
[0005] The accompanying drawings, in which like reference numerals represent identical or functionally similar elements throughout the individual views, together with the following detailed description, are incorporated herein and form a part of the specification and are used to further illustrate embodiments of the concepts including the claimed invention and to explain the various principles and advantages of those embodiments.
[0006] Figure 1 is a block diagram of an example data capture system.
[0007] Figure 2 It shows Figure 1 A block diagram of the additional components of the system.
[0008] Figure 3 It shows Figure 1 FIG. 1 is a diagram of an example implementation of a data capture system.
[0009] Figure 4 is a flow chart of a data capture method for object dimensioning.
[0010] Figure 5 It is shown by Figure 1 Figure 2. Example point cloud generated by our system.
[0011] Figure 6 It shows that Figure 4 At block 420 of the method a graph of the cutting plane is generated.
[0012] Figure 7 It shows Figure 4 FIG. 4 is a diagram of an example execution of block 420 of the method.
[0013] Figure 8 It shows Figure 4 FIG. 4 is a diagram of an example execution of block 425 of the method.
[0014] Fig. 9 It shows Figure 4 FIG. 4 is another example execution of block 425 of the method.
[0015] Figure 10-11 It shows Figure 4 FIG. 4 is a diagram of an example execution of block 440 of the method.
[0016] Fig.12 It shows that Figure 4 A graph of a point density gradient employed in an example implementation of block 450 of the method.
[0017] Those skilled in the art will appreciate that the elements in the drawings are shown for simplicity and clarity and are not necessarily drawn to scale. For example, the sizes of some of the elements in the drawings may be magnified relative to other elements to help improve understanding of the embodiments of the present invention.
[0018] Apparatus and method components have been represented in the drawings by conventional symbols where appropriate, showing only those specific details relevant to an understanding of embodiments of the invention so as not to obscure the disclosure with details that would be apparent to one skilled in the art having the benefit of the description herein. DETAILED DESCRIPTION
[0019] An example disclosed herein relates to a server comprising: a memory storing calibration data; and a processor connected to the memory, the processor being configured to: obtain a point cloud depicting a capture volume containing a transport vehicle having a body and a holder carrying an object to be dimensioned; obtain a set of positions associated with the transport vehicle; based on the set of positions and the calibration data, select a first portion of the point cloud other than the body of the transport vehicle and a mast of the holder; based on the calibration data, select a second portion from the point cloud of the first portion other than a base of the holder; and dimension the object based on the second portion of the point cloud.
[0020] Additional examples disclosed herein relate to a method comprising: storing calibration data; obtaining a point cloud depicting a capture volume containing a transport vehicle having a body and a cage carrying an object to be dimensioned; obtaining a set of positions associated with the transport vehicle; based on the set of positions and the calibration data, selecting a first portion of the point cloud other than the body of the transport vehicle and a mast of the cage; based on the calibration data, selecting a second portion from the point cloud of the first portion other than a base of the cage; and dimensioning the object based on the second portion of the point cloud.
[0021] Figure 1 A data capture system 100 for object dimensioning is depicted. The data capture system 100 is configured to capture image data depicting an object within a capture volume. The image data (e.g., a set of two-dimensional images captured substantially simultaneously) can be processed to generate a point cloud representing the object to be dimensioned. The size of the object can then be determined based on the point cloud.
[0022] In particular, the system 100 includes a plurality of capture nodes 104. In the example shown, four capture nodes 104-1, 104-2, 104-3, and 104-4 are shown. In other examples, a greater or lesser number of capture nodes 104 may be provided. As will be discussed in more detail below, each capture node 104 includes a set of cameras that can be controlled to capture corresponding images of the capture volume mentioned above. More specifically, the camera of each capture node 104 captures an image of a given area of the capture volume. Thus, the coverage area of the capture node 104 substantially covers the entire capture volume. Based on the images captured by the camera of a given node 104, a computing device associated with the node 104 generates a point cloud (e.g., via execution of a photogrammetry application or other suitable point cloud generation mechanism).
[0023] Thus, in the example shown, the capture node 104 generates four point clouds, each of which depicts a respective region of the capture volume containing the object. These regions, and therefore the point clouds generated at the capture node 104, typically overlap at their boundaries.
[0024] The system 100 also includes a data capture server 108, also referred to simply as server 108. The server 108 is connected to the computing device of each of the capture nodes 104 and receives the point clouds mentioned above from the capture nodes 104. The server 108 is configured to combine the point clouds from the capture nodes 104 to produce a single point cloud that depicts at least a portion of the capture volume. As will be discussed in more detail below, the server 108 is further configured to select a portion of the combined point cloud that includes the object and excludes other structures (such as a forklift or other transportation vehicle carrying the object, etc.). Once the portion of the point cloud representing the object is selected, the server 108 can determine the size of the object.
[0025] Generating a point cloud for an area of the capture volume at each capture node 104 and combining the node-specific point clouds at the server 108 enables parallelization of computationally demanding point cloud generation tasks. Such parallelization may enable the system 100 to generate a complete point cloud based on a complete set of images from all cameras and determine the size of objects in the capture volume more quickly than a system that performs point cloud generation on a single device.
[0026] Figure 1Certain internal components of the server 108 are also shown. The server 108 includes a central processing unit (CPU), also referred to as a processor 112, interconnected with a non-transitory computer-readable storage medium such as a memory 116. The memory 116 includes any suitable combination of volatile memory (e.g., random access memory (RAM)) and non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory). The processor 112 and the memory 116 each include one or more integrated circuits (ICs).
[0027] The server 108 also includes a communication interface 120 that enables the server 108 to exchange data with other computing devices, such as via a network 124. Thus, the communication interface 120 includes any suitable hardware (e.g., a transmitter, a receiver, a network interface controller, etc.) that allows the server 108 to communicate over the network 124.
[0028] The server 108 may further include an input / output component 128, which may include a suitable combination of input devices (e.g., keyboard, mouse, microphone, etc.) and output devices (e.g., display, speaker, etc.). The components of the server 108 are interconnected by a communication bus (not shown) and are powered by a battery or other power source through the communication bus mentioned above or through a different power bus (not shown).
[0029] The memory 116 of the server 108 stores a plurality of applications, each of which includes a plurality of computer-readable instructions that can be executed by the processor 112. As discussed herein, execution of the instructions mentioned above by the processor 112 causes the server 108 to perform certain functions. Therefore, in the following discussion, these applications are configured to perform the functions. In this example, the memory 116 of the server 108 stores a point cloud generator application 132 and a dimensioning application 136 (also referred to herein as application 132 and application 136, respectively).
[0030] The server 108 is configured to acquire point clouds from at least a subset of the capture nodes 104 (up to and including all of the capture nodes 104) and generate a combined point cloud from the point clouds via the processor 112 executing the application 132. The server 108 may be configured to detect objects within the combined point cloud and determine the size of the objects via the execution of the application 136.
[0031] In other examples, application 132 and application 136 may be implemented as a single application, or as more than two applications. In other examples, processor 112 configured by executing application 132 and application 136 is implemented as one or more specifically configured hardware elements, such as a field programmable gate array (FPGA) and / or an application specific integrated circuit (ASIC). In further embodiments, the functionality implemented by server 108 via execution of an application (such as application 136) may be implemented by a separate computing device (such as a dimensioning server accessible via network 124) rather than server 108.
[0032] Go to Figure 2 , example components of a capture node 104 are shown. Figure 1 Each of the capture nodes 104 shown in FIG. 1 may include Figure 2 Components shown in and discussed below.
[0033] The capture nodes 104 include a plurality of image sensors 200-1, 200-2, ..., 200-n. For example, each capture node 104 may include four image sensors 200. The image sensors 200 may also be referred to as cameras 200. Each camera 200 has a different field of view (FOV) because the cameras 200 are placed at different physical locations relative to each other. The FOVs of the cameras 200 of a given capture node 104 overlap (e.g., 40% overlap, although more or less overlap is also contemplated). Figure 2 The FOV 202 for the capture node 104 is generally shown. The FOV 202 encompasses an area in the capture volume for which the capture node 104 can generate a point cloud. The point cloud can depict at least a portion of an object 203 within the capture volume.
[0034] The capture node 104 also includes a projector 204. In other examples, multiple projectors 204 may be included in the capture node 104. In further examples, some capture nodes 104 may include one or more projectors 204, while other nodes may omit the projector 204. The projector 204 may be controlled to project a structured light pattern onto the capture volume, for example, to illuminate an object 203 within the capture volume. The structured light pattern may be selected to be easily detected in an image captured by the camera 200 to facilitate the generation of the point cloud mentioned above.
[0035] The capture volume 104 also includes a computing device 208 connected to the camera 200 and the projector 204. The computing device 208 can control the camera 200 to capture images of the capture volume within the FOV 202, while the projector 204 illuminates the capture volume. The computing device 208 also generates a point cloud from the images captured by the camera 200. As will be apparent, the point cloud represents a portion of the capture volume, and thus may represent some, all, or none of the object 203 (depending on the location of the object 203 within the capture volume).
[0036] Go to Figure 3 , certain components of system 100 are shown in an example deployment. Figure 3 The example deployment shown in FIG. 1 includes the four capture nodes 104 mentioned above, for example, the camera 200 and the projector 204 of each node 104 are contained in a substantially cylindrical housing. A variety of other structural arrangements can also be used for each node 104.
[0037] For example, nodes 104 are supported by a central support structure 300 connected to nodes 104 via a passage 304. Nodes 104 may also be supported by support members extending from a ceiling (not shown) of the facility. When used in system 100, passage 304 may carry communication lines and cooling fluids (e.g., conditioned air, etc.). Thus, central support structure 300 may contain cooling or other conditioning equipment, and may also contain servers 108.
[0038] As previously described, the cameras 200 of the nodes 104 are positioned such that the field of view of each camera 200 encompasses at least a portion of the capture volume 308, such as a 10×10×10 foot volume. In addition, the fields of view of adjacent cameras 200 within each node 104 overlap, for example, by about 40%. Thus, the cameras 200 together provide substantially complete coverage of the capture volume 308 (e.g., each location in the capture volume 308 is within the field of view of at least two cameras 200).
[0039] The object 203 can be placed within the capture volume 308 to remain stationary during capture and dimensioning, or the object 203 can be transported through the capture volume 308 via any suitable locomotive mechanism. In other words, the system 100 can dimension the object 203 while it is stationary or in motion. In the example shown, the locomotive mechanism is a transportation vehicle, such as a forklift 309. The forklift 309 includes a body 310, for example, including a cockpit, seats for an operator, an engine, and other components. The forklift 309 also includes a cage 311 to support the object 203. In this example, the cage 311 includes a mast 311a, which can include a mast 311a, such as a mast 311a. Figure 3 The protective cover shown in the figure. The holder 311 also includes a base 311b, such as Figure 3Typically, mast 311a establishes a rear support surface for object 203, while base 311b establishes a lower support surface for object 203.
[0040] System 100 is configured to detect when object 203 enters capture volume 308 and, in response to such detection, control its components to configure projector 204 and camera 200 , capture images with camera 200 , and generate point cloud data.
[0041] In particular, the computing device 208 of each node 104 is configured to generate a point cloud from images captured by the camera 200 of the node 104. Thus, the point cloud generated by a given node 104 depicts a portion of a capture volume 308 corresponding to the FOV 202 of the node 104. The point cloud generated by the nodes 104 may use a local reference frame specific to each node 104, or may use a common reference frame 312 established for the capture volume 308 when the system 100 is deployed. When the nodes 104 generate the point cloud using the common reference frame 312, the computing device 208 of each node 104 may store calibration data defining the physical location of the camera 200 of the node 104 relative to the origin of the common reference frame 312. When the nodes 104 employ a local reference frame, the server 108 may register the node-specific point cloud to the common reference frame 312 using the calibration data.
[0042] The nodes 104 are also configured to detect a set of fiducial markers attached to the forklift 309 in the images captured and used for point cloud generation. The fiducial markers may include any of a variety of marker types, including IR reflective markers, reflective tape, etc. The markers are placed at predefined locations on the forklift 309, and when the markers are detected in the captured images, each node 104 is configured to provide the detected position of each marker to the server 108 (e.g., in the reference frame 312). In some examples, the markers may be encoded with different identifiers to distinguish each marker from other markers, as well as to mitigate false marker detections and enable the system to distinguish between different means of transport. Encoding the markers with different identifiers may also enable the server 108 to determine the orientation of the forklift, as the forklift may enter the capture volume 308 from various directions.
[0043] Thus, the nodes 104 can each provide a point cloud and one or more marker locations and identifiers to the server 108. As will be discussed below, the server 108 uses the marker locations and forklift-specific calibration data to select a portion of the combined point cloud that includes the object 203 but does not include the forklift 309 (both the body 310 and the cage 311). Isolating the object 203 facilitates subsequent dimensioning of the object 203.
[0044] The forklift 309 may also include a reference surface 314 placed thereon and detectable by attaching a fiducial marker, a predefined color or surface pattern, etc. As will be discussed below, in some examples, the location of the reference surface 314 detected by the node 104 may be reported to the server 108, and the server 108 may use the reference surface 314 to assess the quality (e.g., noise level) of the combined point cloud.
[0045] Go to Figure 4 , a method 400 for data capture for object dimensioning is shown. The method 400 will be described in conjunction with its execution in the system 100, and in particular, by the server 108. In general, execution of the method 400 enables the server 108 to obtain individual point clouds (each of which represents a portion of the capture volume 308) from the nodes 104, and generate a combined point cloud according to the common reference system 312. The server 108 can then determine the dimensions of the object 203 from the combined point cloud.
[0046] At block 405, the server 108 is configured to determine whether an object (e.g., object 203) is detected within or near the capture volume 308. Object detection at block 405 may be based on any of a variety of suitable triggering mechanisms. For example, when the object 203 enters the capture volume 308, a lidar sensor, an IR sensor, or the like disposed near the perimeter of the capture volume 308 may generate a detection signal. In other examples, at least one of the nodes 104 may be configured to transmit an image stream captured via at least one camera 200 to the server 108. The server 108 may process the image to detect an object moving therein. When the determination result at block 405 is positive, the server proceeds to block 410. When the determination result at block 405 is negative, the server 108 may repeat the execution of block 405.
[0047] At block 410, the server 108 generates a trigger command to cause at least one of the nodes 104 to capture an image and generate a point cloud therefrom. For example, the server 108 may select a subset of the nodes 104 or all of the nodes 104 and transmit the trigger command to each selected node 104. At each node 104 that receives the trigger command, the corresponding computing device 208 controls at least a subset of the cameras 200 to capture an image substantially simultaneously and generates a point cloud from the image via execution of a suitable point cloud generation mechanism (e.g., a photogrammetry application, etc.).
[0048] After generating and sending a trigger command to the nodes 104 to initiate point cloud generation at the nodes 104, the server 108 proceeds to block 415. At block 415, the server 108 is configured to receive point clouds from each of the nodes 104, as well as locations and identifiers of any fiducial markers in the reference frame 312 detected by the nodes 104. The server 108 is configured to generate a single combined point cloud from the point clouds received from the nodes 104. In some examples, the generation of the combined point cloud includes registering each received node-specific point cloud to the common reference frame 312. In other examples, the combined point cloud may be generated by selecting portions of each node-specific point cloud and excluding other portions; however, such functionality is beyond the scope of this discussion.
[0049] Figure 5 An example point cloud 501 generated at block 415 is shown depicting the forklift 309 and the object 203 . Figure 5 Fiducial markers 500, 504, 508, and 512 are also indicated. As previously described, the node 104 reports the locations of the markers 500-512 to the server 108, which is configured to detect the markers in the images captured via the camera 200. As received at the server 108, the locations (e.g., Figure 5 ) and identifiers (e.g. Figure 5 Each marker 500-512 is defined by the reference numerals shown in the figures.
[0050] At box 420, the server 108 is configured to select a first portion of the combined point cloud other than the body 310 of the transport and the mast 311a of the cage 311. The server 108 stores forklift calibration data in, for example, the memory 116, which defines the positions of various structural features of the forklift 309 relative to the positions of the markers 500-512. For example, the calibration data can define the positions of the markers 500-512 and the positions of other structural features of the forklift 309 in the local reference system. Based on the positions of the markers 500-512 in the reference system 312, the positions of other structural features of the forklift 309 in the reference system 312 can be determined. When different types of transports (such as forklifts) are deployed in the facility, the server 108 can store different calibration data for each forklift. At box 420, the server 108 can also select one of the calibration data sets based on, for example, a marker (e.g., a barcode) on the forklift (e.g., the reference surface 314). In other examples, the tags 500-512 themselves may encode a unique identifier, and the set of tags detected in the captured data may be referenced to determine which forklift is present.
[0051] In particular, refer to Figure 6, the forklift calibration data defines the position of the front surface 600 of the mast 311a of the cage 311 relative to the markers 500-512. For example, the forklift calibration data may define an offset 604 between a plane containing the front surface 600 and the marker 504, and an offset 608 between the plane and the marker 500. The offsets for the markers 508 and 512 ( Figure 6 604 and 608 and the locations of markers 500 and 504 in the point cloud, a cutting plane 612 is defined in the reference frame that is parallel to and may include the front surface 600. As will be apparent to one skilled in the art, for example, if certain markers were not detected in the initial data capture, the plane 612 may be generated based on a subset of the markers 500-512. In this example, at least three of the markers 500-512 may be employed to generate the plane 612.
[0052] Other mechanisms for generating the cutting plane 612 are also contemplated. For example, the forklift calibration data may define the orientation and distance from the mast plane (i.e., the plane that runs through the length of the mast 311a) to the cutting plane 612. In such implementations, the server 108 may generate the mast plane based on the locations of the markers 500-512 (e.g., using the markers 500-512 as points on the mast plane) and add the offsets described above to generate the cutting plane 612.
[0053] After the cutting plane 612 has been generated, the server 108 may be configured to select a first portion of the point cloud excluding points “behind” the cutting plane 612. In other words, the portion of the point cloud containing the markers 500-512 may be discarded or retained but excluded from subsequent processing. As a result of the execution of block 420, the server 108 selects the first portion 700 (e.g., Figure 7 ), the first portion 700 still represents the object 203 and at least a portion of the base 311b of the holder 311, but the rest of the forklift 309 is omitted.
[0054] Back to Figure 4 At block 425, the server 108 is configured to select a second portion of the point cloud 501 from the first portion 700. The second portion is selected to include the object 203, but exclude the remainder of the holder 311 (i.e., the base 311b still represented in the first portion 700). To select the second portion, the server 108 identifies the lower surface of the base 311b. For example, again referring to Figure 7, the server 108 can generate a bounding box 704, such as an axis-aligned bounding box (AABB) having faces parallel to the axes of the reference system 312. The bounding box has a lower surface 708 having a predefined angle relative to the cutting plane 612, which is defined in the forklift calibration data as the angle between the mast 311a and the base 311b. The lower surface 708 is positioned at the height (i.e., the position along the Z axis) of the lowest point (i.e., the point with the lowest Z coordinate) in the portion 700.
[0055] Bounding box 704 also includes side surfaces 712 and 716, e.g., parallel to the XZ plane of reference frame 312, located along the Y axis at the maximum and minimum Y values present in portion 700, respectively. Similarly, the top and front surfaces of bounding box 704 are defined by the range of points in portion 700. The back surface of bounding box 704 is coplanar with cutting plane 612.
[0056] Having identified the lower surface of the base 311b (assumed to be the lower surface 708 of the bounding box 704), the server 108 may apply an offset corresponding to the height of the base 311b to the lower surface to generate a further cutting plane. Figure 8 , as shown in the forklift calibration data, the cutting plane 800 is shown at an offset 804 corresponding to the height of the base 311b. To select the second portion of the point cloud from the first portion 700, the server 108 can discard any points below the cutting plane 800. Alternatively, the server 108 can discard any points between the cutting plane 800 and the lower surface 708 of the bounding box 704.
[0057] In a further example, the server 108 may generate a cutting box by defining the cutting plane 800 and the side cutting surfaces as described above. Fig. 9 In addition to the cutting plane 800 forming the upper surface of the cutting box, the server 108 can also determine the center of the bounding box 704 (in this example along the Y axis) and apply a relative offset 900 corresponding to half the distance between the prongs of the base 311b to the center. The distance between the prongs can be specified in the forklift calibration data. Based on the offset 900, the server 108 can define Fig. 9 The side surfaces of the cutting box 904 are shown. The server 108 may then discard any points from the portion 700 that are within the cutting box. Fig. 9 The method shown in may be desired to avoid excluding portions of the object that hang over the sides of the fork.
[0058] Back to Figure 4, at block 430, the server 108 is then configured to generate an object bounding box. As is apparent from the above discussion, the second portion of the point cloud substantially excludes all of the capture volume except for the object 203 itself. The server 108 may thus generate a bounding box corresponding to the outer boundary object 203 via any suitable plane fitting operation or other bounding box generation mechanism.
[0059] At block 435 , the server 108 is configured to determine the dimensions of the object 203 , such as the width, height, and / or length of the object 203 , the volume of the object, etc., based on the bounding box mentioned above.
[0060] In some examples, server 108 may be configured to refine the bounding box generated at block 430 prior to dimensioning object 203. When such refinement is implemented, Figure 4 As shown by the dashed line in , the server 108 proceeds from box 430 to box 440 before performing dimensioning at box 435. In general, the refinement of the bounding box at box 440 includes evaluating the point density of the second portion of the point cloud at various depths relative to the surface of the bounding box, and determining whether to adjust any of these surfaces. The point cloud may include a certain degree of noise in the points defining the surface of the object 203, which may cause the surface of the bounding box to be larger than the object 203 itself. The refinement of the bounding box at box 440 may enable the server 108 to increase the accuracy of the bounding box.
[0061] refer to Fig.10 , when executing block 440 , the server 108 may repeat the following sequence of operations for at least one surface of the bounding box, up to and including all surfaces of the bounding box. Fig.10 A bounding box 1000 (solid line) is shown containing the points (dashed line) defining the object 203. In other words, the points within the bounding box 1000 are the second portion selected at block 425, substantially excluding the entire forklift 309.
[0062] As shown in detail view 1004, a boundary 1008 of bounding box 1000 is generated based on a plurality of points 1012. To refine boundary 1008, go to Fig.11 , the server 108 is configured to generate a plurality of slices or regions parallel to the boundary 1008 and starting from the boundary 1008 into the object 203 at a predefined depth. Fig.11 Three example slices 1100 - 1 , 1100 - 2 , and 1100 - 3 are shown, although a fewer or greater number of slices may be employed in other examples.
[0063] For each slice 1100, the server 108 is configured to determine a point density or point count that indicates how densely the points 1012 populate the associated slice 1100. Fig.11 As shown, slice 1100-3 contains only a single point, while slice 1100-2 and slice 1100-1 contain eight points and six points, respectively.
[0064] The server 108 is configured to select the slice 1100 having the greatest point density (i.e., the slice 1100-2 in the illustrated example) and update the boundary 1008 to match the location of the slice 1100. Fig.11 As shown in the lower portion of FIG. 4 , boundary 1008 may be replaced by updated boundary 1104-2, which is located at the middle of slice 1100-2. In other examples, the boundary of the selected slice 1100 may be used instead of the middle of the selected slice 1100. When the above process is repeated for each of the remaining surfaces of the bounding box 1000, the updated bounding box may be used for dimensioning at block 435.
[0065] In some examples, using slices to assess the point density of the surface of the bounding box can also be used to assess the quality of the original point cloud. Figure 4 In some examples, server 108 may proceed from block 415 to block 445 instead of directly to block 420. At block 445, server 108 is configured to estimate a reference surface (such as Figure 3 Because the reference surface 314 is on the forklift 309, the position of the reference surface can be determined from the positions of the aforementioned markers 500-512 according to the forklift calibration data.
[0066] To estimate the noise of the reference surface 314, the server 108 may select a portion of the point cloud that contains only the reference surface 314 and perform the refinement process discussed above to determine the point density of each slice in a set of slices that encompass the points defining the reference surface 314. Because the reference surface 314 is planar, low levels of noise are expected to result in peaks in the display (such as Fig.12 The point density in the slice sequence of FIG. 1200 is shown in FIG. Fig.12 As shown in the graph 1204 in FIG. 1 , slower changes in point density may indicate excessive noise in the point cloud. Therefore, the server 108 may be configured to determine a gradient between point density measurements for each slice, where a larger gradient indicates a lower degree of noise. For example, an average of the above gradients may be used as an indicator of the level of noise in the point cloud.
[0067] At block 450, the server 108 determines whether the noise level determined at block 445 exceeds a threshold. For example, when the average gradient determined at block 445 exceeds a threshold, the determination at block 450 is positive, and the server 108 may proceed to block 455 instead of block 420. At block 455, the server 108 may generate an alert (e.g., to another computing device and / or via a display, speaker, etc. of the server 108) indicating that the point cloud quality is not suitable for dimensioning the object 203. In other examples, the server 108 may then perform noise reduction operations. When the determination at block 450 is negative, the process discussed above continues to block 420.
[0068] Specific embodiments have been described in the above description. However, it will be appreciated by those skilled in the art that various modifications and changes may be made without departing from the scope of the invention as set forth in the following claims. Therefore, the description and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the present teachings.
[0069] These benefits, advantages, solutions to problems, and any (multiple) elements that may make any benefit, advantage, or solution occur or become more prominent are not to be construed as critical, required, or essential features or elements of any or all claims. The invention is limited solely by the appended claims, including any amendments made during the pendency of this application and all equivalents of those claims issued as of the date of grant.
[0070] In addition, in this document, relational terms such as first and second, top and bottom, etc. can be used alone to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "include", "comprises", "has", "has", "includes", "contains", "covers", "covers" or any other variation thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device that includes, has, contains, covers a list of elements includes not only those elements but also other elements that are not explicitly listed or inherent to the process, method, article or device. Elements that begin with "includes one", "has one", "includes one", "covers one", in the absence of more constraints, do not exclude the presence of other identical elements in the process, method, article or device that includes, has, contains, covers the element. The terms "one" and "an" are defined as one or more, unless otherwise expressly stated herein. The terms "substantially," "approximately," "approximately," "about" or any other version of these terms are defined as close as understood by one of ordinary skill in the art, and in one non-limiting embodiment, within 10%, within 5%, within 1% in another embodiment, and within 0.5% in another embodiment. The term "coupled," as used herein, is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is "configured" in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
[0071] It is to be understood that some embodiments may include one or more special purpose processors (or "processing devices"), such as microprocessors, digital signal processors, custom processors, and field programmable gate arrays (FPGAs), and unique stored program instructions (including both software and firmware) that control the one or more processors to implement some, most, or all of the functions of the methods and / or devices described herein together with certain non-processor circuits. Alternatively, some or all of the functions may be implemented by a state machine without stored program instructions, or in one or more application specific integrated circuits (ASICs), where each function or certain combinations of certain functions are implemented as custom logic. Of course, a combination of these two approaches may also be used.
[0072] In addition, embodiments may be implemented as a computer-readable storage medium having computer-readable code stored thereon for programming a computer (e.g., including a processor) to perform the methods as described and claimed herein. Examples of such computer-readable storage media include, but are not limited to, hard disks, CD-ROMs, optical storage devices, magnetic storage devices, ROMs (read-only memories), PROMs (programmable read-only memories), EPROMs (erasable programmable read-only memories), EEPROMs (electrically erasable programmable read-only memories), and flash memory. In addition, it is expected that one of ordinary skill in the art, though making potentially significant efforts and many design choices motivated by, for example, available time, current technology, and economic considerations, will readily be able to generate such software instructions and programs and ICs with minimal experimentation when guided by the concepts and principles disclosed herein.
[0073] The abstract of the present disclosure is provided to allow the reader to quickly ascertain the nature of the present technical disclosure. This abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above-mentioned detailed description, it can be seen that various features are grouped together in various embodiments for the purpose of integrating the present disclosure. This method of disclosure should not be interpreted as reflecting the intention that the claimed embodiments require more features than those explicitly stated in the claims. On the contrary, as reflected in the following claims, the subject matter of the invention lies in less than all the features of a single disclosed embodiment. Therefore, the following claims are thus incorporated into the detailed description, wherein each claim represents itself as a subject matter claimed separately.
Claims
1. A server, comprising: a memory storing calibration data; as well as A processor, the processor is connected to the memory, and the processor is configured to: acquiring a point cloud depicting a capture volume containing a vehicle having a body and a cage carrying an object to be dimensioned; obtaining a set of locations associated with the transport, the set of locations further associated with markers located on the transport; selecting a first portion of the point cloud excluding the body of the vehicle and a mast of the cage based on the set of positions and the calibration data; selecting, from the point cloud of the first portion, a second portion excluding a base of the holder based on the calibration data; as well as dimensioning the object based on the second portion of the point cloud, Wherein the calibration data comprises positions of structural features of the vehicle relative to markers positioned on the vehicle.
2. The server according to claim 1, further comprising: Communication interface; The processor is further configured to receive initial point clouds from a plurality of capture nodes via the communication interface in order to acquire the point cloud, and combine the initial point clouds to generate the point cloud.
3. The server according to claim 1, characterized in that To select the first portion, the processor is configured to: generating a cutting plane corresponding to a front surface of the mast; and Points on predetermined sides of the cutting plane are discarded.
4. The server according to claim 3, characterized in that To select the second part, the processor is configured to: detecting a lower surface of the base; generating an upper surface of the base according to the calibration data; and Points between the lower surface and the upper surface are discarded.
5. The server according to claim 4, characterized in that To detect the lower surface, the processor is further configured to generate a bounding box containing the object and the base based on the first portion.
6. The server according to claim 5, characterized in that The processor is further configured to: identifying a center of the bounding box; generating a side surface corresponding to a side of the base according to the center and the calibration data; and Points between the lower surface, the upper surface, and the side surfaces are discarded.
7. The server according to claim 5, characterized in that The processor is configured to generate the lower surface having a predefined angle relative to the cutting plane.
8. The server according to claim 1, wherein: In order to dimension the object, the processor is further configured to: generating a bounding box corresponding to the object from the second portion of the point cloud; determining a point density for each of a plurality of slices adjacent to a surface of the bounding box; as well as An updated bounding box surface corresponding to one of the plurality of slices having a maximum point density is selected.
9. The server according to claim 8, characterized in that The processor is further configured to repeat the determination of the point density and the selection of an updated bounding box surface for each remaining surface of the bounding box.
10. The server according to claim 1, wherein: The processor is further configured to: detecting a reference surface of the vehicle based on the set of position and calibration data; and A noise level in the point cloud is estimated from the reference surface.
11. A method comprising: Store calibration data; acquiring a point cloud depicting a capture volume containing a vehicle having a body and a cage carrying an object to be dimensioned; obtaining a set of locations associated with the transport, the set of locations further associated with markers located on the transport; selecting a first portion of the point cloud excluding the body of the vehicle and a mast of the cage based on the set of positions and the calibration data; selecting, from the point cloud of the first portion, a second portion excluding a base of the holder based on the calibration data; as well as dimensioning the object based on the second portion of the point cloud, Wherein the calibration data comprises positions of structural features of the vehicle relative to markers positioned on the vehicle.
12. The method according to claim 11, characterized in that Acquiring the point cloud includes receiving initial point clouds from a plurality of capture nodes via a communication interface and combining the initial point clouds to generate the point cloud.
13. The method according to claim 11, characterized in that The first part of the selection includes: generating a cutting plane corresponding to a front surface of the mast; and Points on predetermined sides of the cutting plane are discarded.
14. The method according to claim 13, characterized in that The second part of the selection includes: detecting a lower surface of the base; generating an upper surface of the base according to the calibration data; and Points between the lower surface and the upper surface are discarded.
15. The method according to claim 14, characterized in that Detecting the lower surface includes generating a bounding box containing the object and the base based on the first portion.
16. The method of claim 15, further comprising: identifying a center of the bounding box; generating a side surface corresponding to a side surface of the base according to the center and the calibration data; as well as Points between the lower surface, the upper surface and the side surfaces are discarded.
17. The method of claim 15, further comprising generating the lower surface having a predefined angle relative to the cutting plane.
18. The method according to claim 11, characterized in that Dimensioning the object includes: generating a bounding box corresponding to the object from the second portion of the point cloud; determining a point density for each of a plurality of slices adjacent to a surface of the bounding box; and An updated bounding box surface corresponding to one of the plurality of slices having a maximum point density is selected.
19. The method of claim 18, further comprising: The determination of the point density and the selection of the updated bounding box surface are repeated for each remaining surface of the bounding box.
20. The method of claim 11, further comprising: detecting a reference surface of the vehicle based on the set of positions and the calibration data; as well as A noise level in the point cloud is estimated from the reference surface.
