Volume measuring system

Abstract

To provide a volume measuring method and device for a volume measuring system that can efficiently measure the weights and volumes of various sizes of objects in order to increase the efficiency of loading and transport in physical distribution.SOLUTION: A volume measuring method executed by a volume measuring device includes the steps of: determining information related to the conversion relationship between a sensing coordinate system related to a sensing direction and a reference plane coordinate system; acquiring, by a 3D sensing unit, a target depth image related to a target object located on a reference plane; acquiring point group data related to the sensing coordinate system on the basis of the target depth image; acquiring point group data related to the reference plane coordinate system by using the information related to the conversion relationship between the sensing coordinate system related to the sensing direction and the reference plane coordinate system; and determining the volume of the target object on the basis of the point group data related to the reference plane coordinate system.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to a Volume management System (VMS). [Background technology]

[0002] A home delivery service is a transportation service that accepts requests from individuals or companies to transport small amounts of cargo and provides all services from accepting the package to packaging, transportation, and delivery. Recently, there has been a boom in the trading of goods through domestic and international online shops, TV shopping, secondhand goods sales sites, etc., and the use of home delivery services to deliver goods for sale has been increasing rapidly. In addition, the frequency with which individuals use home delivery services to return delivered goods or exchange goods between individuals has also been increasing rapidly.

[0003] In order to improve the efficiency of loading and transportation in logistics, there is a demand for a volume management system (VMS) that can efficiently measure not only the weight but also the volume of objects of various sizes. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Korean Patent Publication No. 2003-0062774 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure is intended to solve the above-mentioned problems, and has an object to provide a volume measurement method and device for a volume measurement system. [Means for solving the problem]

[0006] A computing device-implemented method for solving the problems discussed above is disclosed.

[0007] The method may include: acquiring a target depth image relating to a target object located on a reference plane by a 3D sensing unit, where the 3D sensing unit has a sensing direction at an acute angle with respect to the reference plane; acquiring point cloud data relating to a sensing coordinate system based on the target depth image; and acquiring point cloud data relating to the reference plane coordinate system using information relating to a transformation relationship between the sensing coordinate system relating to the sensing direction and the reference plane coordinate system.

[0008] Alternatively, the information on the transformation relationship between the sensing coordinate system and the reference surface coordinate system may include a rotation transformation matrix and a translation transformation matrix.

[0009] Alternatively, the rotation transformation matrix may be a unit vector matrix of a reference plane based on the sensing coordinate system, and the translation transformation matrix may be a centroid matrix of point cloud data corresponding to the reference plane based on the sensing coordinate system.

[0010] Alternatively, the information on the transformation relationship between the sensing coordinate system and the reference surface coordinate system may be based on the following formula:

[0011]

number

[0012] Alternatively, the method may further include: determining information relating to a transformation relationship between a sensing coordinate system associated with the sensing direction and a reference plane coordinate system.

[0013] Alternatively, the step of determining information regarding the transformation relationship between the sensing coordinate system associated with the sensing direction and the reference plane coordinate system may include: acquiring a corrected depth image associated with the reference plane based on the sensing direction by the 3D sensing unit; acquiring point cloud data corresponding to the reference plane based on the corrected depth image; determining a plane equation associated with the reference plane based on the point cloud data corresponding to the reference plane; and determining information regarding the transformation relationship between the sensing coordinate system associated with the sensing direction and the reference plane coordinate system based on the plane equation associated with the reference plane.

[0014] Alternatively, the step of determining an equation of a plane relating to the reference surface based on point cloud data corresponding to the reference surface may include a step of determining the equation of the plane by applying Singular Value Decomposition to the point cloud data corresponding to the reference surface.

[0015] Alternatively, the point cloud data relating to the reference plane coordinate system may include a bird-eye view image corresponding to the target depth image.

[0016] Alternatively, the method may further include: determining a volume of the target object based on point cloud data related to the reference plane coordinate system.

[0017] Alternatively, the step of determining the volume of the target object based on point cloud data relating to the reference plane coordinate system may include the step of: if the target object has a rectangular prism shape, identifying a top surface of the target object to determine a height of the target object.

[0018] Alternatively, the step of determining the volume of the target object based on point cloud data associated with the reference plane coordinate system may include: if the target object has an irregular shape, determining a cubic bounding box of the target object based on point cloud data associated with the reference plane coordinate system; and determining approximate values ​​of the width, length and height of the target object using the cubic bounding box of the target object.

[0019] A non-transitory computer readable storage medium storing a computer program for solving the problems as mentioned above is disclosed.

[0020] The computer program includes instructions for causing one or more processors to execute the following method, which may include: acquiring a target depth image of a target object located on a reference plane by a 3D sensing unit, where the 3D sensing unit has a sensing direction at an acute angle with respect to the reference plane; acquiring point cloud data related to a sensing coordinate system based on the target depth image; and acquiring point cloud data related to the reference plane coordinate system using information related to a transformation relationship between the sensing coordinate system related to the sensing direction and the reference plane coordinate system.

[0021] A volume measuring device for solving the above-mentioned problems is disclosed.

[0022] The device includes: a 3D sensing unit having a sensing direction at an acute angle with respect to a reference plane; a base that forms the exterior of the volume measuring device and has an upper plane on which a target object can be placed, and a body structure that is arranged so that the 3D sensing unit faces the direction of the upper plane; and a control unit; wherein the control unit is capable of: acquiring a target depth image of the target object located on the reference plane using the 3D sensing unit, acquiring point cloud data related to a sensing coordinate system based on the target depth image, acquiring point cloud data related to the reference plane coordinate system using information related to the transformation relationship between the sensing coordinate system related to the sensing direction and the reference plane coordinate system, and determining the volume of the target object based on the point cloud data related to the reference plane coordinate system. Effect of the Invention

[0023] The present disclosure is intended to solve the above-mentioned problems, and can provide a volume measurement method and device for a volume measurement system. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a block diagram of a volume measurement device according to embodiments of the present disclosure. [Diagram 2] 1 is a diagram illustrating a process of acquiring a corrected depth image related to a reference plane based on a sensing direction according to several embodiments of the present disclosure. [Diagram 3] 1 is a diagram illustrating a corrected depth image showing a region corresponding to a reference surface according to various embodiments of the present disclosure. [Figure 4] 1 is a diagram illustrating a reference plane and corresponding point cloud data obtained from a rectified depth image according to embodiments of the present disclosure. [Diagram 5] 1 is a diagram illustrating a plane defined by point cloud data relating to a reference plane and a corresponding plane equation in accordance with several embodiments of the present disclosure. [Figure 6]1 is a diagram illustrating a sensing coordinate system and a reference plane coordinate system according to several embodiments of the present disclosure. [Figure 7] 1 is a diagram illustrating a step of acquiring a target depth image related to a target object located on a reference plane. [Figure 8] 1 is a diagram illustrating an exemplary target depth image acquired by a 3D sensing unit having a sensing direction at an acute angle with respect to a reference plane. [Figure 9] 11 is a diagram illustrating point cloud data related to a reference plane coordinate system acquired using information related to a conversion relationship between a sensing coordinate system related to a sensing direction and a reference plane coordinate system. [Figure 10] 1 is a flow chart of a volume measurement method according to some embodiments of the present disclosure. [Figure 11] FIG. 1 is a simplified generalized diagram of an exemplary computing environment in which embodiments of the present disclosure can be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Various embodiments and / or aspects are disclosed below with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are disclosed to aid in the overall understanding of one or more aspects. However, one of ordinary skill in the art of the present disclosure will recognize that such aspect(s) can be practiced without such specific details. The following description and accompanying drawings set forth certain exemplary aspects of one or more aspects in detail. However, these aspects are illustrative, and any of a variety of methods based on the principles of the various aspects may be utilized, and the description set forth herein is intended to include all such aspects and their equivalents. In particular, the terms "embodiment," "example," "aspect," "exemplary," and the like, as used herein, should not be construed as implying that any aspect or design described herein is superior or advantageous over other aspects or designs.

[0026] Hereinafter, the same or similar components are denoted by the same reference numerals regardless of the reference numerals in the drawings, and duplicated descriptions thereof will be omitted. In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may make the gist of the embodiments disclosed in this specification unclear, the detailed description will be omitted. In addition, the attached drawings are merely intended to facilitate understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings.

[0027] Although expressions such as "first" and "second" are used to describe various elements or components, these elements or components are not limited by these terms. These terms are merely used to distinguish one element or component from another element or component. Therefore, a first element or component described below may also be a second element or component within the technical concept of the present invention.

[0028] Unless otherwise defined, all terms (including technical and scientific terms) used in this specification can be used in a commonly understood sense by a person having ordinary knowledge in the technical field to which the present invention belongs. Furthermore, terms defined in a general dictionary are not to be interpreted ideally or excessively unless otherwise defined.

[0029] Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X utilizes A or B" is intended to mean one of the natural inclusive permutations. That is, if X utilizes A; X utilizes B; or X utilizes both A and B, then "X utilizes A or B" can apply to any of these. Additionally, the term "and / or" as used herein refers to and includes all possible combinations of one or more of the associated listed items. Additionally, the term "comprise" as a predicate and / or the modifier "comprise" are understood to mean the presence of the stated feature and / or component, but not to exclude the presence or addition of one or more other features, components and / or groups thereof. Additionally, unless a specific number is specified or the context is clear that a singular number is indicated, the singular number should generally be construed in this specification and claims to mean "one or more."

[0030] Also, as used herein, the terms "information" and "data" may be used interchangeably.

[0031] When an element or layer is referred to as "on" or "on" another element or layer, it does not only mean directly on the other element or layer, but also includes cases where there are other layers or elements between them. On the other hand, when an element is referred to as "directly on" or "directly above" an element, it means that there are no other elements or layers between them.

[0032] Spatially relative terms such as "below," "beneath," "lower," "above," "upper," and the like, may be used to easily describe the relationship of one component to another as depicted in the drawings. Spatially relative terms should be understood to include different orientations of each element in use or operation in addition to the orientation depicted in the drawings.

[0033] For example, if components depicted in the figures were turned upside down, a component described as "below" or "beneath" another component would then be located "above" the other component. Thus, the exemplary term "below" can encompass both an orientation of below and above. Components may be oriented in other directions, and thus spatially relative terms may have different interpretations depending on the orientation.

[0034] When a component is described as being "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly connected, coupled, or connected to the other component, but that there may be other components between them. On the other hand, when a component is described as being "directly connected," "directly coupled," or "directly connected" to another component, it should be understood that there are no other components between them.

[0035] The suffixes "module" and "section" used in the following description are used or mixed together for the sole purpose of facilitating the writing of the specification, and these suffixes themselves do not have any specific different meanings or roles.

[0036] The objectives and effects of the present disclosure, as well as the technical configuration for achieving them, will become clear by referring to the following detailed examples together with the accompanying drawings. In the description of the present disclosure, if it is determined that a specific description of a known function or configuration may make the gist of the present disclosure unclear, the detailed description will be omitted. The terms described below are defined in consideration of the functions in the present disclosure, and may change depending on the intentions or practices of users or operators.

[0037] However, the present disclosure is not limited to the embodiments disclosed below, and can be embodied in various forms. However, these embodiments are provided to make the present disclosure complete and to allow those skilled in the art to which the present disclosure pertains to fully understand the scope of the disclosure, and the present disclosure is defined by the scope of the claims. Therefore, the definition should be determined based on the contents described in the entire specification.

[0038] The accompanying drawings are merely intended to facilitate understanding of the embodiments disclosed in this specification, and are not intended to limit the technical ideas disclosed in this specification. It should be understood that the accompanying drawings include all modifications, equivalents, and alternatives included in the scope of the ideas and techniques of the present invention. FIG. 1 is a block diagram of a volumetric measurement device (1000) according to some embodiments of the present disclosure.

[0039] The configuration of the volume measuring device (1000) shown in Figure 1 is merely a simplified example. In various embodiments of the present disclosure, the volume measuring device (1000) may include other components for performing volume measurement functions and other functions, and only some of the disclosed components may constitute the volume measuring device (1000).

[0040] In some examples, the volume measuring device (1000) may be a device for a volume management system (VMS). For example, the volume measuring device (1000) may measure the volume of an object using depth information related to the object. In some examples, the volume measuring device (1000) may be an unmanned home delivery reception device. For example, the volume measuring device (1000) may measure the width, length, and height of objects of various sizes in real time by sensing the objects through the 3D sensing unit (140) in order to improve loading efficiency and transportation efficiency in logistics. However, the volume measuring device (1000) may include various devices.

[0041] Referring to FIG. 1, the volume measuring device (1000) may include a control unit (110), a storage unit (120), a network unit (130), a 3D sensing unit (140), and a body structure (150). However, the volume measuring device (1000) may further include other components depending on its application, and only some of the disclosed components may constitute the volume measuring device (1000). For example, if the volume measuring device (1000) is an unmanned home delivery reception device, the volume measuring device (1000) may further include at least one of a weight measuring unit, a camera unit, a user input unit, a display unit, and an audio output unit. However, the volume measuring device (1000) may include various components, without being limited thereto.

[0042] The control unit 110 is generally capable of processing the overall operation of the volume measuring device 1000. The control unit 110 processes signals, data, information, etc. input or output through components of the volume measuring device 1000, and can provide or process appropriate information or functions to a user by running application programs stored in the storage unit 120. In some examples, when the volume measuring device 1000 is an unmanned delivery reception device, the control unit 110 can process various operations for providing a delivery service together with the volume measuring function.

[0043] In some embodiments of the present disclosure, the control unit 110 can execute a volume measurement method according to the present disclosure, which will be described in detail with reference to FIGS.

[0044] The storage unit 120 may include a memory and / or a permanent storage medium, and the memory may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only Memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0045] The storage unit 120 can generally store various information used in the process of processing the overall operation of the volume measurement device 1000. According to several embodiments of the present disclosure, the storage unit 120 can store various data in the process of performing the volume measurement method. For example, the storage unit 120 can store a target depth image or a corrected depth image generated by the 3D sensing unit 140. In another example, the storage unit 120 can store various information received from the outside. The data stored in the storage unit 120 can be displayed on a display unit (not shown) or transferred to another device.

[0046] The network unit 130 can be configured regardless of the communication type, such as wired or wireless, and can be configured with various communication networks, such as a short-range communication network (LAN: Local Area Network) and a long-range communication network (WAN: Wide Area Network). The network unit 130 can transmit and receive various data in the process of performing the volume measurement method according to the present disclosure. The data received via the network unit 130 can be displayed on a display unit (not shown) or transferred to another device.

[0047] The 3D sensing unit 140 can obtain depth information. In some examples, the 3D sensing unit 140 can be based on a variety of technologies, such as a laser displacement sensor (profiler), a stereo camera, a laser or LIDAR distance measuring camera, a TOF camera / sensor, an RGB-D camera, a structured light camera, or a variety of other passive or active distance sensing technologies. In some examples, the 3D sensing unit 140 can generate a depth image, such as a target depth image or a rectified depth image. In this case, the 3D sensing unit 140 can generate point cloud data of the imaged object. The point cloud data can include a collection of 3D points in a space where each point (i) can be represented by (Xi, Yi, Zi). The point cloud data can represent a 3D object including the back, side, top, and bottom surfaces of the object. The 3D point (Xi, Yi, Zi) may represent a position in space where the camera can capture an object (visible), but is not limited thereto, and the 3D sensing unit 140 may obtain depth information in various ways.

[0048] The body structure 150 can form the exterior of the volume measuring device 1000. The exterior of the volume measuring device 1000 can be formed in various shapes depending on its application. The body structure 150 can include the above-mentioned control unit 110, storage unit 120, network unit 130, 3D sensing unit 140, and other components. However, the volume measuring device 1000 is not limited thereto, and can include various components depending on its application.

[0049] According to several embodiments of the present disclosure, the volume measuring device (1000) can measure the volume of an object placed on a reference plane using depth information (point cloud data) acquired through a 3D sensing unit (140) installed to have a sensing direction at an acute angle with respect to the reference plane. When the depth information is acquired along a sensing direction perpendicular to the object, the depth information acquired along the perpendicular sensing direction directly indicates the height of the object, so that the volume of the object can be easily determined.

[0050] However, in order to position the 3D sensing device vertically relative to the object, the volume measuring device must be large in size so that the 3D sensing device can be installed on top of the object, or must have an element such as a mount arm that protrudes far forward and causes inconvenience to the user. In particular, the 3D sensing device must have a minimum distance for measurement. Therefore, the higher the height of the object to be measured, the higher the sensing position of the 3D sensing device must be. In this case, there are disadvantages such as the overall size of the volume measuring device becoming larger, the space available for installation being limited, and increased manufacturing and installation costs.

[0051] According to several embodiments of the present disclosure, the volume measuring device (1000) can obtain point cloud data related to the reference plane coordinate system from the target depth image using information related to the transformation relationship between the sensing coordinate system and the reference plane coordinate system. The point cloud data related to the reference plane coordinate system can be displayed in the form of a bird-eye view image captured from a direction perpendicular to the object. Therefore, the volume measuring device (1000) of the present disclosure can provide the same effect as obtaining depth information along a sensing direction perpendicular to the object.

[0052] In addition, in the volume measuring device (1000) according to the present disclosure, since the 3D sensing unit (140) has a sensing direction at an acute angle with respect to the reference plane, the installation position of the 3D sensing unit (140) for ensuring the minimum separation distance for measurement may be lower than when the 3D sensing unit (140) is installed with a vertical sensing direction. In this case, the volume measuring device (1000) may have an advantage that it can be manufactured in a small size with low manufacturing costs and easy installation. In addition, the 3D sensing unit (140) installed with an acute sensing direction may provide an effect of being able to detect objects with a larger volume at the same angle of view compared to a vertical sensing direction. As a result, the present disclosure may provide a volume measuring device (1000) that provides an improved effect related to a VMS solution.

[0053] An example of how the volume measuring device (1000) according to the present disclosure executes a volume measuring method will be described below with reference to FIGS.

[0054] Referring to FIG. 2, the exemplary volume measuring device (1000) may be an unmanned parcel reception device. As shown in FIG. 2, the volume measuring device (1000) may include a body structure (150) that forms the appearance of the unmanned parcel reception device. In some examples, the volume measuring device (1000) may measure the volume of a target object located on a reference plane. The reference plane may mean a plane that is a reference when the volume measuring device (1000) measures the volume of an object. In other words, the reference plane may mean a plane whose height is 0 with respect to the object. Referring to FIG. 2, the body structure (150) may include a base (151) for stably installing the volume measuring device (1000) on the ground. The base (151) may have an upper plane on which a target object whose volume is to be measured can be placed. In this case, the 3D sensing unit (140) may be disposed in the body structure (150) so as to face the direction of the upper plane. In this case, the volume measuring device (1000) can set an area included in the upper plane of the base (151) as a reference plane for measuring the volume of the object.

[0055] The 3D sensing unit 140 may have a sensing direction that is at an acute angle with respect to the reference plane. As described above, when the 3D sensing unit 140 has a sensing direction that is at an acute angle with respect to the reference plane, the installation position of the 3D sensing unit 140 that can ensure a minimum separation distance may be lowered. In addition, the 3D sensing unit 140 may detect an object with a larger volume within the same angle of view.

[0056] In some examples, as shown in FIG. 2, the 3D sensing unit (140) may be installed on the upper part of a support table (152) extending upward from one side of the table (151). The 3D sensing unit (140) may be disposed so as to face the direction of the upper plane of the table (151) included in the reference plane. In this case, the 3D sensing unit (140) may have a sensing direction at an acute angle with respect to the reference plane. Referring to FIG. 2, the 3D sensing unit (140) having a sensing direction at an acute angle with respect to the reference plane may obtain a depth image expressed in a sensing coordinate system with the sensing direction as the Z axis. In other words, the control unit (110) may obtain a depth image expressed in the sensing coordinate system by the 3D sensing unit (140). Since the sensing direction is not perpendicular to the reference plane, it is not possible to directly calculate the height of an object located on the reference plane using a depth image related to the sensing coordinate system. Therefore, the volume measurement device (1000) can use information regarding the transformation relationship between the sensing coordinate system and the reference surface coordinate system to measure the volume.

[0057] According to several embodiments of the present disclosure, the control unit 110 can determine information related to a transformation relationship between a sensing coordinate system related to a sensing direction and a reference plane coordinate system.

[0058] To convert the sensing coordinate system into the reference plane coordinate system, the control unit 110 can determine information related to the conversion relationship between the sensing coordinate system related to the current sensing direction and the reference plane coordinate system. The process of determining information related to the conversion relationship between the sensing coordinate system related to the current sensing direction and the reference plane coordinate system can be referred to as calibration relative to the reference plane. Calibration relative to the reference plane can be performed when the volume measurement device 1000 is installed or when the position of the 3D sensing unit 140 is adjusted. An exemplary calibration relative to the reference plane can be performed by the following process.

[0059] According to some embodiments of the present disclosure, the control unit (110) can obtain a rectified depth image related to a reference plane based on a sensing direction by the 3D sensing unit (140). For example, when no object is placed on the reference plane, the 3D sensing unit (140) can obtain a rectified depth image including depth values ​​related to pixels in an area that falls within the angle of view. In this case, the rectified depth image can be an image of a Z-axis value (depth value) in an xy plane based on a sensing coordinate system.

[0060] According to several embodiments of the present disclosure, the control unit (110) can obtain point cloud data (210) corresponding to the reference surface based on the rectified depth image.

[0061] After acquiring the rectified depth image, the control unit (110) can designate a region in the rectified depth image that corresponds to the reference plane as the region of interest (200). FIG. 3 illustrates a rectified depth image in which a portion of an area in the upper plane of the platform (151) is designated as the region of interest (200) that corresponds to the reference plane. In some examples, the control unit (110) can use a segmentation method to designate the reference plane that is the region of interest (200) in the rectified depth image. For example, when the segmentation method is used, the control unit (110) can acquire point cloud data related to all pixels in the rectified depth image. The control unit (110) can determine a point cloud region that approximates a plane from the point cloud data acquired from the rectified depth image using a RANSAC (RANdom SAmple consensus) method or the like. In this case, the control unit (110) can determine the point cloud region that approximates a plane as the region of interest (200). The control unit (110) can then acquire the point cloud region that approximates the plane as point cloud data (210) corresponding to the reference plane. In some examples, multiple planes may be detected depending on the installation location of the volume measuring device (1000) or the like. In this case, the control unit (110) can determine the plane that is closest to the 3D sensing unit (140) as the reference plane. In FIG. 4, an exemplary reference plane and corresponding point cloud data (210) acquired from a corrective depth image are shown.

[0062] According to several embodiments of the present disclosure, the control unit (110) can determine a plane equation relating to the reference surface based on point cloud data (210) corresponding to the reference surface.

[0063] In some examples, to determine the information related to the transformation relationship, the control unit 110 can use point cloud data 210 corresponding to the reference plane to determine an equation of a plane that defines the reference plane. The control unit 110 can use various algorithms to approximate a plane using a plurality of three-dimensional points. For example, the control unit 110 can determine an equation of a plane related to the reference plane by applying Singular Value Decomposition (SVD) to the point cloud data 210 corresponding to the reference plane. An exemplary SVD process is as follows:

[0064] A point "p" expressed in the sensing coordinate system can be expressed by a vector as follows.

[0065]

number

[0066]

number

[0067] On the other hand, a point cloud matrix containing N points can be defined as follows:

[0068]

number

[0069]

number

[0070]

number

[0071]

number

[0072] Due to the nature of SVD, u 1 , u 2 , u 3 can be orthonormal vectors that are mutually orthogonal and have a magnitude of one.

[0073] Also, u 1 , u 2 , u 3 indicates the direction in which each point is distributed relative to the center, and these can be oriented in the direction with the largest variance in order. In other words, u 1 , u 2 is a vector lying in the plane to be approximated, so that each point can point along the two axes along which it is most widely distributed. Then, u 3 is the direction with the least distribution of points, so u 3 can be the normal vector of the plane to be approximated. Therefore, A, B, and C, which correspond to the normal vectors in the equation of the plane, can be obtained as follows.

[0074]

number

[0075] Also, due to the nature of the equation of the plane, D can be calculated as follows:

[0076]

number

[0077] Therefore, the equation of a plane that is approximated by a number of points in the reference plane can be obtained as follows:

[0078]

number

[0079] Through this SVD process, the control unit (110) can determine "Ax+By+Cz+D=0", which is the equation of the plane related to the reference plane. In FIG. 5, a plane (220) defined by the equation of the plane related to the reference plane is illustrated. As described later, the point cloud data (or the bird's-eye view image) related to the reference plane coordinate system converted using the information related to the conversion relationship can indicate height with reference to the plane (220) defined by the equation of the plane related to the reference plane. In other words, the plane (220) defined by the equation of the plane related to the reference plane can indicate a position where the height is 0.

[0080] According to several embodiments of the present disclosure, the control unit 110 can determine information related to a transformation relationship between the sensing coordinate system and the reference surface coordinate system based on an equation of a plane related to the reference surface, in which case the information related to the transformation relationship between the sensing coordinate system related to the sensing direction and the reference surface coordinate system can include a translation transformation matrix and a rotation transformation matrix.

[0081] Specifically, the coordinate transformation between the sensing coordinate system and the reference plane coordinate system can be performed by translation and rotation. The rotation transformation matrix that defines the rotation transformation can be determined depending on the direction of the x-axis and y-axis of the reference plane coordinate system. In some examples, referring to FIG. 6, when the distribution direction of the multiple points that constitute the point cloud data related to the reference plane is used as it is, the unit vector matrix U of the reference plane calculated in the process of performing SVD is used as the rotation transformation matrix, and the center matrix

[0082]

number

[0083]

number

[0084]

number

[0085] Information regarding the transformation relationship between the sensing coordinate system and the reference surface coordinate system, including the above-mentioned translation transformation matrix and rotation transformation matrix, is as follows:

[0086]

number

[0087] When information regarding the transformation relationship between the sensing coordinate system and the reference surface coordinate system is determined in accordance with the above-mentioned process, the control unit (110) can measure the volume of the target object using the information regarding the transformation relationship between the sensing coordinate system and the reference surface coordinate system.

[0088] An example in which the volume measuring device (1000), which is an unmanned home delivery reception device, measures the volume of a target object will be described below with reference to Figures 7 to 9. For ease of explanation, it is assumed based on the above example that the control unit (110) sets an area included in the upper plane of the platform (151) of the volume measuring device (1000), which is an unmanned home delivery reception device, as a reference plane for measuring the volume of the object, and then determines information related to the conversion relationship between the sensing coordinate system related to the sensing direction and the reference plane coordinate system.

[0089] According to some embodiments of the present disclosure, the control unit 110 can obtain a target depth image of a target object located on a reference plane by the 3D sensing unit 140. In this case, the 3D sensing unit 140 can have a sensing direction at an acute angle with respect to the reference plane.

[0090] 7 shows a state in which a target object is placed on a platform (151) of a volume measuring device (1000) which is an unmanned home delivery reception device. In this case, the control unit (110) can obtain a target depth image through the 3D sensing unit (140) having an acute sensing direction for an area included in the upper plane of the platform (151) of the volume measuring device (1000) which is set as a reference plane.

[0091] According to several embodiments of the present disclosure, the control unit (110) can acquire point cloud data related to a sensing coordinate system based on the target depth image. FIG. 8 illustrates an exemplary target depth image acquired by the 3D sensing unit 140 having a sensing direction at an acute angle with respect to the reference plane. As illustrated in FIG. 8, the target depth image may be an image of the z-axis value in the sensing coordinate system in the xy plane. The control unit 110 may acquire point cloud data related to the sensing coordinate system for every pixel of the target depth image. Because the sensing direction is not perpendicular to the reference plane, it is not possible to immediately calculate the height of an object located on the reference plane using the point cloud data related to the sensing coordinate system. Therefore, the control unit 110 may use information related to the transformation relationship between the sensing coordinate system and the reference plane coordinate system determined along the sensing direction for volumetric measurement of the target object.

[0092] Based on several embodiments of the present disclosure, the control unit (110) can acquire point cloud data relating to the reference plane coordinate system using information relating to the transformation relationship between the sensing coordinate system relating to the sensing direction and the reference plane coordinate system.

[0093] Specifically, the control unit (110) can convert the point cloud data related to the sensing coordinate system into a state in which the height of an object located on the reference plane can be calculated using information related to the conversion relationship between the sensing coordinate system and the reference plane coordinate system. FIG. 9 shows point cloud data related to the reference plane coordinate system converted using information related to the conversion relationship between the sensing coordinate system and the reference plane coordinate system. As shown in FIG. 9, the point cloud data related to the reference plane coordinate system can include a bird-eye view image corresponding to the target depth image. In other words, the point cloud data related to the reference plane coordinate system can include an image obtained by converting the target depth image in a direction perpendicular to the reference plane. Therefore, the control unit (110) can easily determine the height of the target object using the point cloud data related to the reference plane coordinate system.

[0094] According to several embodiments of the present disclosure, the control unit (110) can determine the volume of the target object based on the point cloud data related to the reference plane coordinate system.

[0095] As described above, since the point cloud data related to the reference plane coordinate system can include a bird's-eye view image corresponding to the target depth image, the control unit (110) can determine the height of the target object using the z-axis value of the point cloud data related to the reference plane coordinate system. Also, the control unit (110) can determine the horizontal / vertical lengths of the target object using the x-axis value and the y-axis value of the point cloud data related to the reference plane coordinate system. In this case, the control unit (110) can determine the volume of the target object using the height and the horizontal / vertical lengths of the target object.

[0096] In some examples, when the volume measuring device (1000) is an unmanned delivery device, the target object may have a rectangular parallelepiped shape similar to the shape of a typical parcel sent by delivery. When the target object has a rectangular parallelepiped shape, the control unit (110) may identify the top surface of the target object to determine the height of the target object. The control unit (110) may determine the area of ​​the target object using the x-axis value and the y-axis value of the identified top surface. Then, the control unit (110) may determine the height of the target object using the z-axis value of the identified top surface. In this case, the control unit (110) may measure the volume of the target object using the area and height of the rectangular parallelepiped target object.

[0097] In another example, the target object may have an irregular shape. In this case, the control unit (110) may use a cube-shaped bounding box to measure an approximation of the volume of the target object. Specifically, the control unit (110) may determine the cube-shaped bounding box of the target object based on point cloud data related to the reference plane coordinate system. The control unit (110) may use a rectangular bounding area of ​​the target object identified by the bird's-eye view image to determine the horizontal / vertical lengths of the cube-shaped bounding box of the target object. The control unit (110) may then determine the height of the cube-shaped bounding box using a portion of the z-axis value (height value) of the rectangular bounding area. For example, the control unit (110) may use the maximum value of the z-axis value of the rectangular bounding area to determine the height of the cube-shaped bounding box. In another example, in order to avoid using outliers, the control unit (110) may use the z-axis value (e.g., average value, median value, etc.) of the rectangular bounding area to determine the height of the cube-shaped bounding box. In this case, the control unit 110 can approximate and measure the volume of a target object having an irregular shape using the height and horizontal / vertical lengths of a rectangular parallelepiped cube-shaped bounding box.

[0098] FIG. 10 is a flow chart of a method for measuring volume, according to some embodiments of the present disclosure.

[0099] According to several embodiments of the present disclosure, the volume measurement method may include a step (s100) of determining information related to a transformation relationship between a sensing coordinate system related to a sensing direction and a reference plane coordinate system.

[0100] Alternatively, the step (s100) of determining information regarding the transformation relationship between the sensing coordinate system related to the sensing direction and the reference plane coordinate system may include the steps of: acquiring a corrected depth image related to the reference plane based on the sensing direction by the 3D sensing unit; acquiring point cloud data corresponding to the reference plane based on the corrected depth image; determining a plane equation related to the reference plane based on the point cloud data corresponding to the reference plane; and determining information regarding the transformation relationship between the sensing coordinate system related to the sensing direction and the reference plane coordinate system based on the plane equation related to the reference plane.

[0101] Alternatively, the step of determining an equation of a plane relating to the reference surface based on point cloud data corresponding to the reference surface may include a step of determining the equation of the plane by applying Singular Value Decomposition to the point cloud data corresponding to the reference surface.

[0102] According to some embodiments of the present disclosure, the volume measurement method may include acquiring (s200) a target depth image of a target object located on a reference plane by a 3D sensing unit, where the 3D sensing unit may have a sensing direction at an acute angle with respect to the reference plane.

[0103] According to several embodiments of the present disclosure, the volume measurement method may include a step (s300) of acquiring point cloud data related to a sensing coordinate system based on a target depth image.

[0104] Based on several embodiments of the present disclosure, the volume measurement method may include a step (s400) of acquiring point cloud data relating to a reference plane coordinate system using information relating to a transformation relationship between a sensing coordinate system relating to a sensing direction and a reference plane coordinate system.

[0105] According to several embodiments of the present disclosure, a volume measurement method may include determining (s500) a volume of a target object based on point cloud data associated with a reference plane coordinate system.

[0106] Alternatively, the step (s500) of determining the volume of the target object based on point cloud data relating to the reference plane coordinate system may include the step of: if the target object has a rectangular prism shape, identifying the top surface of the target object to determine the height of the target object.

[0107] Alternatively, the step (s500) of determining the volume of the target object based on point cloud data associated with the reference plane coordinate system may include: if the target object has an irregular shape, determining a cubic bounding box of the target object based on point cloud data associated with the reference plane coordinate system; and determining approximate values ​​of the width, length and height of the target object using the cubic bounding box of the target object.

[0108] The steps described in accordance with the various embodiments of the present disclosure are presented for illustrative purposes only, and some steps may be omitted, other steps may be added, and the steps may be performed in any order.

[0109] FIG. 11 is a simplified generalized diagram of an exemplary computing environment in which embodiments of the present disclosure can be implemented.

[0110] While the present disclosure has been described above as generally capable of being embodied in a computing device, those skilled in the art will appreciate that the present disclosure may also be embodied in combination with computer-executable instructions and / or other program modules capable of being executed on one or more computers and / or as a combination of hardware and software.

[0111] Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Additionally, those skilled in the art will appreciate that the disclosed methods can be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based consumer electronics, programmable consumer electronics, and the like, any of which can operate in conjunction with one or more associated devices.

[0112] Further, the embodiments described herein may be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0113] A computer typically includes a variety of computer-readable media. Any medium accessible by a computer can be a computer-readable medium, including volatile and non-volatile media, transitory and non-transitory media, and mobile and non-mobile media. By way of example and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media include volatile and non-volatile media, transitory and non-transitory media, mobile and non-mobile media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, DVD (digital video disk) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be accessed by a computer and used to store information.

[0114] Computer-readable transmission media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes all information delivery media. The term modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared and other wireless media. Combinations of any of the above media are also intended to be included within the scope of computer-readable transmission media.

[0115] An exemplary environment (1100) for implementing various aspects of the present disclosure is shown including a computer (1102) including a processing unit (1104), a system memory (1106), and a system bus (1108). The system bus (1108) couples system components, including, but not limited to, the system memory (1106), to the processing unit (1104). The processing unit (1104) can be any of a variety of commercially available processors. Dual processors and other multi-processor architectures can also be utilized as the processing unit (1104).

[0116] The system bus (1108) can be any of several types of bus structures that can be further interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory (1106) includes read only memory (ROM) (1110) and random access memory (RAM) (1112). The basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, or EEPROM, and contains the basic routines that support the exchange of information between the various components within the computer (1102), such as during start-up. The RAM (1112) can also include high-speed RAM, such as static RAM, for caching data.

[0117] The computer 1102 also includes an internal hard disk drive (HDD) 1114 (e.g., EIDE, SATA) - although the internal hard disk drive 1114 may be used externally in a suitable chassis (not shown) - a magnetic floppy disk drive (FDD) 1116 (e.g., for reading from and writing to a removable diskette 1118), and an optical disk drive 1120 (e.g., for reading from CD-ROM disks 1122 and for reading from and writing to other high capacity optical media such as DVDs). The hard disk drive 1114, magnetic disk drive 1116, and optical disk drive 1120 may be connected to the system bus 1108 by a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical drive interface 1128, respectively. The interface (1124) for implementing an external drive includes at least one or both of the following interface technologies: Universal Serial Bus (USB) and / or IEEE 1394.

[0118] These drives and their associated computer-readable media provide non-volatile storage for data, data structures, computer-executable instructions, and so forth. In the case of computer 1102, the drives and media accommodate the storage of any data in a suitable digital format. While the above description of computer-readable storage media refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will appreciate that other types of computer-readable storage media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and so forth, may also be used in the exemplary operating environment, and that any such media may include computer-executable instructions for performing the methods of the present disclosure.

[0119] A number of program modules may be stored in the drives and RAM (1112), including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136). All or portions of the operating system, applications, modules, and / or data may also be cached in RAM (1112). It will be appreciated that the present disclosure may be implemented with various commercially available operating systems or a combination of multiple operating systems.

[0120] A user may enter commands and information into the computer 1102 through one or more wired or wireless input devices, such as a keyboard 1138 and a pointing device, such as a mouse 1140. Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and the like. These and other input devices are often connected to the processing unit 1104 through an input device interface 1142 that is connected to the system bus 1108, but may also be connected through other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and the like.

[0121] A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface, such as a video adapter 1146. In addition to the monitor 1144, computers typically include other peripheral output devices (not shown), such as speakers, printers, etc.

[0122] The computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148, via wired and / or wireless communications. The remote computer(s) 1148 may be a workstation, computing device computer, router, personal computer, handheld computer, microprocessor-based entertainment device, peer device, or other conventional network node, and may include many or all of the components generally described in association with the computer 1102, although for simplicity, only a memory storage device 1150 is illustrated. The logical connections depicted include wired or wireless connections to a local area network (LAN) 1152 and / or larger networks, such as a long-distance network (WAN) 1154. Such LAN and WAN networking environments are commonplace in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which may connect to computer networks around the world, such as the Internet.

[0123] When used in a LAN networking environment, the computer 1102 connects to the local network 1152 through a wired and / or wireless communication network interface or adapter 1156. The adapter 1156 can facilitate wired or wireless communication to the LAN 1152, which may also include a wireless access point installed thereon for communicating with the wireless adapter 1156. When used in a WAN networking environment, the computer 1102 may include a modem 1158 or other means for establishing communications over the WAN 1154, such as connecting to a communications computing device in the WAN 1154 or through the Internet. The modem 1158, which may be internal or external and may be a wired or wireless device, connects to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules depicted for computer 1102, or portions thereof, may be stored in a remote memory / storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

[0124] The computer (1102) is operable to communicate with any wireless device or unit that is deployed and operates in wireless communication, such as printers, scanners, desktop and / or handheld computers, PDAs (portable data assistants), communication satellites, any equipment or location related to wirelessly detectable tags, and phones. This includes at least Wi-Fi and Bluetooth wireless technologies. Thus, the communication can be of a predefined structure, such as a traditional network, or in the simple case of ad hoc communication between at least two devices.

[0125] Wi-Fi (Wireless Fidelity) allows devices to connect to the Internet without being wired. Wi-Fi is a wireless technology, like a cell phone, that allows such devices, e.g., computers, to send and receive data indoors or outdoors, anywhere within range of a base station. Wi-Fi networks use IEEE 802.11 (a, b, g, etc.) radio technology to provide secure, reliable, and fast wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, e.g., at data rates of 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual bands).

[0126] Those of ordinary skill in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols and chips referred to in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0127] Those of ordinary skill in the art of the present disclosure will appreciate that the various exemplary logic blocks, modules, processors, means, circuits, and algorithm steps described in the description of the embodiments disclosed herein may be implemented by electronic hardware, various forms of programs (referred to herein as "software" for ease of description) or design code, or a combination of all of these. To clearly illustrate such interoperability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above with a focus on their functionality. Whether such functionality is implemented as hardware or software is determined by design constraints imposed on the particular application and the overall system. Those of ordinary skill in the art of the present disclosure may implement the described functionality in various ways for each particular application, and such implementation decisions should not be interpreted as departing from the scope of the present disclosure.

[0128] The various embodiments described herein may be implemented by a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" includes a computer program, carrier, or media accessible by any computer readable device. For example, computer readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROM, cards, sticks, key drives, etc.). Additionally, the various storage media described herein include one or more devices and / or other machine-readable media for storing information.

[0129] It should be understood that the particular order or hierarchy of the process steps depicted herein is an example of an exemplary approach. Based on design priorities, it should be understood that the particular order or hierarchy of the process steps can be rearranged within the scope of the present disclosure. The accompanying method claims provide elements of the various steps in a sample order, but are not meant to be limited to the particular order or hierarchy depicted.

[0130] The description of the embodiments set forth herein is provided to enable any person of ordinary skill in the art to which the disclosure pertains to the present disclosure to utilize or practice the present disclosure. Various modifications to such embodiments will be apparent to those of ordinary skill in the art to which the disclosure pertains, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited by the embodiments set forth herein, but is to be accorded the broadest scope consistent with the principles and novel features set forth herein.

Claims

1. 1. A method implemented by a computing device, comprising: acquiring a target depth image relating to a target object located on a reference plane by a 3D sensing unit, wherein the 3D sensing unit has a sensing direction at an acute angle with respect to the reference plane; acquiring point cloud data relating to a sensing coordinate system based on the target depth image; and acquiring point cloud data related to the reference plane coordinate system using information related to a transformation relationship between a sensing coordinate system related to the sensing direction and a reference plane coordinate system; Including, method.

2. The information regarding the transformation relationship between the sensing coordinate system and the reference surface coordinate system includes a rotation transformation matrix and a translation transformation matrix. The method of claim 1.

3. The rotation transformation matrix is ​​a unit vector matrix of a reference plane based on the sensing coordinate system, and the translation transformation matrix is ​​a center matrix of point cloud data corresponding to the reference plane based on the sensing coordinate system. The method of claim 2.

4. The information regarding the transformation relationship between the sensing coordinate system and the reference surface coordinate system is based on the following formula: The method according to claim 3. [0010]

5. determining information regarding a transformation relationship between a sensing coordinate system related to the sensing direction and the reference plane coordinate system; Further comprising: The method of claim 1.

6. The step of determining information related to a transformation relationship between a sensing coordinate system related to the sensing direction and the reference plane coordinate system includes: acquiring a rectified depth image with respect to a reference plane based on the sensing direction by the 3D sensing unit; obtaining point cloud data corresponding to a reference plane based on the corrected depth image; determining an equation of a plane associated with the reference surface based on point cloud data corresponding to the reference surface; and determining information relating to a transformation relationship between a sensing coordinate system relating to the sensing direction and the reference surface coordinate system based on an equation of a plane relating to the reference surface; Including, The method according to claim 5.

7. The step of determining an equation of a plane associated with the reference surface based on point cloud data corresponding to the reference surface includes: determining an equation of the plane by applying singular value decomposition to point cloud data corresponding to the reference plane; Including, The method according to claim 6.

8. The point cloud data relating to the reference plane coordinate system includes a bird-eye view image corresponding to the target depth image, The method of claim 1.

9. determining a volume of the target object based on point cloud data related to the reference plane coordinate system; Further comprising: The method of claim 1.

10. The step of determining a volume of the target object based on point cloud data related to the reference plane coordinate system includes: if the target object has a rectangular parallelepiped shape, identifying a top surface of the target object to determine a height of the target object; Including, 10. The method of claim 9.

11. The step of determining a volume of the target object based on point cloud data related to the reference plane coordinate system includes: determining a cubic bounding box of the target object based on point cloud data associated with the reference plane coordinate system if the target object has an irregular shape; and determining an approximation of the width, length and height of the target object using a cubic bounding box of the target object; Including, 10. The method of claim 9.

12. A non-transitory computer readable storage medium storing a computer program, the computer program including instructions for causing one or more processors to perform a method, the method comprising: acquiring a target depth image relating to a target object located on a reference plane by a 3D sensing unit, wherein the 3D sensing unit has a sensing direction at an acute angle with respect to the reference plane; acquiring point cloud data relating to a sensing coordinate system based on the target depth image; and acquiring point cloud data related to the reference plane coordinate system using information related to a transformation relationship between a sensing coordinate system related to the sensing direction and a reference plane coordinate system; Including, A non-transitory computer-readable storage medium for storing computer programs.

13. A volume measuring device, comprising: a 3D sensing portion having a sensing direction at an acute angle with respect to a reference plane; a body structure that forms the exterior of the volume measuring device and has an upper plane on which a target object can be placed, and the 3D sensing unit is disposed so as to face the upper plane; and Control unit; Including, The control unit: acquiring a target depth image relating to the target object located on a reference plane by the 3D sensing unit; Obtaining point cloud data relating to a sensing coordinate system based on the target depth image; Acquiring point cloud data relating to the reference plane coordinate system using information relating to a transformation relationship between a sensing coordinate system relating to the sensing direction and a reference plane coordinate system; and determining a volume of the target object based on point cloud data relating to the reference plane coordinate system; Volume measuring device.

Images