Measurement system

Abstract

To simply measure a shape of an object on the basis of a picked-up image.SOLUTION: A measurement apparatus comprises: an imaging part for imaging a marker and an object to be measured having side surfaces perpendicular to a placement surface and a top surface parallel to the placement surface; a first dimension calculation part for calculating an imaging length, which is a length in a three-dimensional imaging space; and a second dimension calculation part for calculating a real length, which is a length in a real space. The second dimension calculation part calculates the real length of a frame in the real space from the imaging length of a frame of the object to be measured in the imaging space in reference to the real length of the marker.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a technique for measuring the size of an object. [Background technology]

[0002] In logistics, the most common method of measuring package size is to pass the package through a gate equipped with an optical sensor, but this requires the installation of expensive dedicated equipment and is not suitable for measuring package sizes at the time of delivery.

[0003] In Patent Document 1, a stereo camera captures an image of a package from diagonally above to obtain two captured images. The stereo camera sends the two captured images to a server, which creates a depth map based on the principle of triangulation. The server calculates the vertex coordinates of the package based on the depth map to calculate the size of the package (see paragraphs

[0014] ,

[0015] ,

[0022] ,

[0023] , etc. in Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-211425 A [Patent Document 2] JP 2018-112521 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, it is assumed that all eight vertices will be measured (see paragraph

[0023] ), so it is considered necessary to capture images of the baggage from multiple directions. Furthermore, the measurement targets in Patent Document 1 are limited to rectangular parallelepiped-shaped bags moving on a conveyor route.

[0006] A primary object of the present invention is to provide a technique for easily measuring the shape of an object based on a captured image. [Means for solving the problem]

[0007] A measurement system in one embodiment of the present invention includes an imaging unit that images a marker and a measurement object having a side perpendicular to a mounting surface and an upper surface parallel to the mounting surface, a first dimension calculation unit that calculates an imaging length, which is a length in a three-dimensional imaging space, and a second dimension calculation unit that calculates a real length, which is a length in real space. The second dimension calculation unit refers to the actual length of the marker and calculates the actual length of the frame in the real space from the imaging length of the frame of the measurement object in the imaging space.

[0008] A measurement system in another aspect of the present invention includes an imaging unit that images a measurement object having a side perpendicular to a mounting surface and a top surface parallel to the mounting surface, and a dimension calculation unit that calculates an actual length, which is a length in real space. The dimension calculation unit identifies the position of the mounting surface based on the height of the imaging viewpoint from the mounting surface and the imaging angle, detects a first point on the side of the object to be measured that is in contact with the mounting surface from the captured image, identifies the coordinates of the first point in real space, detects a second point on the top surface of the object to be measured that is located vertically above the first point from the captured image, identifies the coordinates of the second point in real space, and calculates the height of the object to be measured based on the coordinates of the first point and the coordinates of the second point in real space. Effect of the Invention

[0009] According to the present invention, the shape of an object can be easily measured. [Brief description of the drawings]

[0010] [Figure 1] FIG. 13 is a schematic diagram for explaining a method for measuring the shape of a package. [Diagram 2] FIG. 2 is a schematic diagram showing a situation in which luggage is measured in the first embodiment. [Diagram 3] FIG. 2 is a functional block diagram of the measurement device according to the first embodiment. [Figure 4]FIG. 4 is a schematic diagram for explaining a method for measuring three sides in the first embodiment. [Diagram 5] 5 is a flowchart showing a process for measuring three sides of a package in the first embodiment. [Figure 6] FIG. 11 is a functional block diagram of a measurement device according to a second embodiment. [Figure 7] FIG. 11 is a schematic diagram for explaining a method for measuring three sides in the second embodiment. [Figure 8] 10 is a flowchart showing a process for measuring three sides of a package in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In the following, a method for measuring the dimensions of a package using markers will be described as a first embodiment, followed by a method for measuring the dimensions of a package in a case where some parameters, such as the height of the camera, are known. When the first and second embodiments are referred to collectively, or when no particular distinction is made between them, they will be referred to as "the present embodiment."

[0012] FIG. 1 is a schematic diagram for explaining a method for measuring the shape of a piece of luggage 200. As shown in FIG. The measurement device 100 (measurement system) in this embodiment is a smartphone equipped with an imaging function. The measurement device 100 may be a device dedicated to measurement.

[0013] First, the user places luggage 200 (measurement target) on floor surface 202, which is a placement surface. The luggage 200 is a rectangular parallelepiped object such as a cardboard box. Here, rectangular parallelepiped includes a "cube." Floor surface 202 is a horizontal plane, and the measurement target may have at least a top surface 204 parallel to the horizontal floor surface 202 and side surfaces perpendicular to floor surface 202. The measurement target may be a polygonal prism or cylinder.

[0014] The user holds the measuring device 100 diagonally above the luggage 200 and touches an image capture button for measurement (not shown) (hereinafter referred to as the "measurement button") that is displayed on the screen of the measuring device 100. When the measurement button is touched, the measuring device 100 captures an image of the luggage 200 and the floor surface 202.

[0015] The measuring device 100 in this embodiment measures the length of each of the three sides (length, width, and height) of the luggage 200 based on an image captured by a camera (imaging unit) using a method described below (hereinafter also referred to as "three-side measurement").

[0016] In the following, the horizontal direction is set as the XZ plane, and the Y axis is set in the height direction, as shown in Fig. 1. The coordinate values ​​of each point on the horizontal plane are called "XZ coordinates", the coordinate values ​​indicating the height of each point from the floor surface 202 are called "Y coordinates", and the XYZ coordinates of each point are called "three-dimensional coordinates".

[0017] [First embodiment] FIG. 2 is a schematic diagram showing a situation in which a baggage 200 is measured in the first embodiment. In the first embodiment, a marker M is placed on the floor surface 202. The marker M may be anything, such as a sticker, a piece of paper, or a coin, as long as it can be identified from the floor surface 202 by image recognition and is a thin object with a known length. The marker M may be placed on the top surface 204 of the baggage 200, but in the first embodiment, the marker M will be described as being placed on the floor surface 202.

[0018] In this embodiment, the measuring device 100 captures an image of the luggage 200 at a position and imaging angle that includes at least four vertices P1 to P4 of the luggage 200. The height (Y width) of the luggage 200 is the distance between vertices P1 and P2. The length (X width) of the luggage 200 is the distance between vertices P1 and P3. The width (Z width) of the luggage 200 is the distance between vertices P1 and P4.

[0019] Hereinafter, the length in the three-dimensional imaged space (virtual space) assumed by the measuring device 100 is referred to as the “imaged length,” and the actual length in the real space is referred to as the “real length.” In this embodiment, the object is to measure the three real lengths of the height, length, and width of the baggage 200.

[0020] FIG. 3 is a functional block diagram of the measuring device 100 in the first embodiment. Each component of the measuring device 100 is realized by hardware including computing units such as a CPU (Central Processing Unit) and various co-processors, storage devices such as memory and storage, and wired or wireless communication lines connecting them, and software stored in the storage devices and supplying processing instructions to the computing units. The computer programs may be composed of device drivers, an operating system, various application programs located at higher layers than these, and libraries that provide common functions to these programs. Each block described below represents a functional block, not a hardware-based configuration.

[0021] The measuring device 100 includes a camera 102 (imaging unit), a user interface processing unit 104, a data processing unit 106, and a data storage unit 108. The user interface processing unit 104 accepts operations from the user and is responsible for processing related to the user interface, such as image display and audio output. The data storage unit 108 stores various data. The data processing unit 106 executes various processes based on the captured image acquired by the camera 102, data acquired by the user interface processing unit 104, and data stored in the data storage unit 108. The data processing unit 106 also functions as an interface between the camera 102, the user interface processing unit 104, and the data storage unit 108.

[0022] The user interface processing unit 104 includes an input unit 110 that accepts input from a user, and an output unit 112 that outputs various types of information such as images and sounds to the user.

[0023] The data processing unit 106 includes a first dimension calculation unit 114 and a second dimension calculation unit 116 . The first dimension calculation unit 114 detects the frame (sides) of the baggage 200 from the captured image and calculates the captured length of the frame. The second dimension calculation unit 116 executes three-side measurement by calculating the actual length of the frame.

[0024] FIG. 4 is a schematic diagram for explaining a method for measuring three sides in the first embodiment. It is assumed that the focal length and the orientation (imaging angle) of the camera 102 are known. The measuring device 100 captures an image of the marker M. As described above, the marker M is placed on the floor surface 202. The first dimension calculation unit 114 identifies the position coordinates (three-dimensional coordinates) in a three-dimensional imaging space of the end points MQ1 and MQ2 of the marker M reflected in the captured image (imaging surface 120). The first dimension calculation unit 114 calculates the distance d between the end points MP1 and MP2 in this imaging space as the imaging length of the marker M.

[0025] The actual length D of the marker M is known. Next, the first dimension calculation unit 114 specifies the ratio r (=D / d) of the actual length D to the imaging length d as the ratio of the sizes of the imaging space and the actual space.

[0026] Next, the first dimension calculation unit 114 converts the position coordinates of the vertices P1 to P4 of the package 200 shown in the captured image into position coordinates (three-dimensional coordinates) in the captured space. The first dimension calculation unit 114 calculates the captured lengths of the height (P1 to P2), length (P1 to P3), and width (P1 to P4) of the package 200 in the captured space.

[0027] The second dimension calculation unit 116 calculates the actual lengths of the three sides of the package 200 by multiplying the captured image length by the ratio r for the height, length, and width of the package 200.

[0028] FIG. 5 is a flowchart showing the process of measuring three sides of the baggage 200 in the first embodiment. When the user holds the measuring device 100 over the baggage 200 and touches the measurement button, the following process starts. Specifically, the output unit 112 first displays a measurement button on the screen of the measuring device 100. The input unit 110 detects a touch on the measurement button. By simply touching the measurement button, the measuring device 100 automatically performs measurement of three sides of the baggage 200.

[0029] When the user touches the measurement button, the measuring device 100 captures an image of the baggage 200 from diagonally above, and the captured image is acquired (S10). The first dimension calculation unit 114 first calculates the captured length of the marker M shown in the captured image (S12). At this time, the first dimension calculation unit 114 also calculates the ratio r of the captured length of the marker M to its actual length.

[0030] Next, the first dimension calculation unit 114 detects frames (P1-P2), (P1-P3), and (P1-P4) corresponding to the height, length, and width as frames of the baggage 200 from the captured image (S14). The first dimension calculation unit 114 calculates the captured image lengths of the three frames from the position coordinates of the vertices P1, P2, and P3 in the captured image space (S16). The second dimension calculation unit 116 converts the captured image lengths of the frames into actual lengths based on the ratio r (S18).

[0031] [Second embodiment] In the second embodiment, the measuring device 100 captures an image of the baggage 200 at an imaging angle that includes the vertices P1, P2, P3, and P4 of the baggage 200, in the same manner as the imaging method shown in FIG. 2. In the second embodiment, the marker M is not placed on the floor surface 202. Meanwhile, the focal length and imaging angle of the measuring device 100 (camera 102), as well as the height H (actual length) of the imaging point of the camera 102 from the floor surface 202 are assumed to be known. The measuring device 100 may be installed in advance to match a specified imaging angle such as 30 degrees, or the imaging angle when actually installed may be set as a parameter in the measuring device 100 by a known means such as an inclination sensor. The same applies to the height, and the height H of the imaging point of the measuring device 100 from the floor surface 202 may be measured by a known means such as an altimeter or LIDAR (Laser Imaging Detection and Ranging), and set as a parameter in the measuring device 100.

[0032] FIG. 6 is a functional block diagram of the measuring device 100 in the second embodiment. The data processing unit 106 of the measuring device 100 in the second embodiment includes a dimension calculation unit 118. The dimension calculation unit 118 calculates the actual length of the frame (side of the baggage 200) based on the position coordinates in the imaging space by a method described later. The other configurations are similar to those of the measuring device 100 in the first embodiment.

[0033] FIG. 7 is a schematic diagram for explaining a method for measuring three sides in the second embodiment. In order to explain the measurement principle, a method for calculating the height (P1 to P2) and length (P1 to P3) of the luggage 200 will be described assuming that the luggage 200 is viewed from the side.

[0034] First, the imaging point of the camera 102 is set as the origin O. As described above, in the second embodiment, the height H as the actual length of the imaging point O is known. The tilt a (imaging angle) and focal length of the imaging surface 120 are also known.

[0035] Vertex P2 (first point) of the baggage 200 contacts the floor surface 202. Vertex P2 is projected as point Q2 on the imaging plane 120. Similarly, vertex P1 (second point) corresponds to point Q1 on the imaging plane 120, and vertex P3 (third point) corresponds to point Q3 on the imaging plane 120.

[0036] Since the height H and the inclination a are known, the dimension calculation unit 118 can specify the position of the floor surface 202 in the real space (the distance and direction from the imaging point O). In other words, the dimension calculation unit 118 can specify a plane equation of the floor surface 202 in the real space (the XZ plane that is the distance H below the imaging point O) based on parameters such as the height H. Next, the dimension calculation unit 118 specifies the position coordinates of the intersection of the floor surface 202 and the projection line T2 connecting the imaging point O and point Q2 as the position coordinates of the vertex P2 in the real space. Through the processing up to this point, the position coordinates of the vertex P2 in the real space are specified.

[0037] Next, the dimension calculation unit 118 specifies a vertical line S1 that is perpendicular to the floor surface 202 and passes through the vertex P2. The dimension calculation unit 118 specifies the position coordinates of the intersection of the vertical line S1 and the projection line T1 connecting the imaging point O and the point Q1 as the position coordinates of the vertex P1 in the real space.

[0038] The dimension calculation unit 118 specifies a plane equation of a second plane 206 that is parallel to the floor surface 202 and passes through the vertex P1, i.e., a plane that includes the top surface 204 of the baggage 200. The dimension calculation unit 118 specifies the position coordinates of the intersection of the second plane 206 and a projection line T3 that connects the imaging point O and the point Q3 as the position coordinates of the vertex P3 in real space. The position coordinates of the other vertex P4 can also be calculated in a similar manner.

[0039] The first dimension calculation unit 114 calculates the actual lengths of the three sides of the package 200 based on the position coordinates of the vertices P1 to P4 in the real space.

[0040] FIG. 8 is a flowchart showing the process of measuring three sides of the baggage 200 in the second embodiment. In the second embodiment as well, when the user holds the measuring device 100 over the baggage 200 and touches the measurement button, the following process is started.

[0041] When the user touches the measurement button, the measuring device 100 captures an image of the baggage 200 from diagonally above, and the captured image is acquired (S10). The dimension calculation unit 118 identifies a plane equation of the floor surface 202 (placing surface) from the captured image (S12).

[0042] Next, the dimension calculation unit 118 specifies the position coordinates in real space of the vertex P2, which is the intersection of the projection line T2 and the floor surface 202 (S24). The dimension calculation unit 118 specifies a line S1 that passes through the vertex P2 and is perpendicular to the floor surface 202 (S26). The dimension calculation unit 118 specifies the vertex P1, which is the intersection of the projection line T1 and the perpendicular line S1 (S28). Next, the dimension calculation unit 118 specifies a plane equation of the upper surface 204 of the luggage 200 (S30), and specifies the vertex P3 by finding the intersection of the projection line T3 and the upper surface 204 (S32). In a similar manner, the dimension calculation unit 118 specifies the vertex P4 (S32). The dimension calculation unit 118 calculates the actual lengths of the three sides of the luggage 200 based on the position coordinates of the vertices P1 to P4 (S34).

[0043] The measurement principle is as described above, but at the end, we will also mention a more specific calculation method in three-dimensional real space. First, the imaging point O is set as the origin, and three-dimensional vectors [Xi, Yi, Zi] from the imaging point O to each vertex Pi are created based on the two-dimensional coordinates in the captured image of vertex Q1, etc. (hereinafter referred to as "vertex vectors").

[0044] Next, based on the imaging angle a, the above vertex vectors are transformed into a world coordinate system (a coordinate system of the imaging space) so that the coordinate XZ plane is parallel to the floor surface 202. Projection lines T1 to T3 are defined by the transformed vertex vectors.

[0045] In this world coordinate system, a plane equation of the floor surface 202 in the real space is calculated. Since the height of the imaging point O in the real space is H, the plane equation of the floor surface 202 is Y=H.

[0046] The coordinates [X2, Y2, Z2] of the vertex P2 in the captured space are obtained as the intersection of the projection line T2 and the floor surface 202. Since Y2=H here, the coordinates of the vertex P2 in the real space are [X2*(Y2 / H),H,Z2*(Y2 / H)].

[0047] Find the linear equation L2 of the straight line S1 perpendicular to the floor surface 202 from the vertex P2. Since L2 = [X2 * (Y2 / H), s, Z2 * (Y2 / H)], by calculating the intersection point of the projection line T1 and the perpendicular line S1, P1 = [(X2 * Y2) / H, Y1 * (X2 * Y2) / (H * X1), (Z2 * Y2) / H)].

[0048] Since the height of vertex P1 of upper surface 204 is known, the plane equation of upper surface 204 of luggage 200 is Y=Y1*(X2*Y2) / (H*X1). Thereafter, the intersection of vertices P3 and P4 is found in a similar manner based on the plane equation of upper surface 204 of luggage 200. From the above, the three sides of luggage 200 can be measured.

[0049] [Summary] The measuring device 100 (measurement system) has been described above based on the embodiment. According to this embodiment, a user can measure three sides of a piece of luggage 200 with high accuracy by simply holding the measuring device 100 over the piece of luggage 200 and touching a measurement button. The user does not need to image the piece of luggage 200 from directly above, but only needs to image the piece of luggage 200 from a direction in which the three sides of the piece of luggage 200 are in view. In recent years, with the spread of flea market applications, there have been an increasing number of opportunities for individuals to sell their personal belongings over the Internet. Since a user can easily measure three sides of a piece of luggage 200 by imaging it with the measuring device 100, the user can easily register size information required when listing the item on a website.

[0050] In the first embodiment, the actual size of the luggage 200 shown in the captured image, i.e., the actual lengths of the three sides, can be measured by comparing the actual length of the marker M with the captured length. In the second embodiment, the three sides of the luggage 200 can be measured without using the marker M by appropriately setting the height and imaging angle of the luggage 200 and capturing images of the four vertices included in the three sides of the luggage 200. Therefore, even if the measuring device 100 is not equipped with a so-called depth sensor such as LIDAR, it is possible to easily measure the three sides of the luggage 200. If LIDAR is equipped, the measuring device 100 may obtain the actual height H of the imaging point O by the LIDAR.

[0051] The present invention is not limited to the above-mentioned embodiment and modified examples, and the components can be modified without departing from the gist of the present invention. Various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment and modified examples. In addition, some components can be deleted from all the components shown in the above-mentioned embodiment and modified examples.

[0052] [Variations] In the present embodiment, the measurement apparatus 100 is described as performing three-side measurement, but a measurement system may be configured by the measurement apparatus 100 and a server. The server may share part of the functions of the measurement apparatus 100 (for example, all or part of the first dimension calculation unit 114, the second dimension calculation unit 116, and the dimension calculation unit 118), thereby reducing the processing load of the measurement apparatus 100. A third apparatus other than the measurement apparatus 100 and the server may also take on part of the functions of the measurement apparatus 100 or the server. How to allocate a plurality of functions required to realize the present invention to one or more pieces of hardware may be determined in consideration of the processing capacity of each piece of hardware, the specifications required for the measurement system, and the like.

[0053] In the present embodiment, a description has been given assuming that an image of the luggage 200 is captured from diagonally above. In the first embodiment, the user may capture an image of the luggage 200 from directly above. By placing markers M on both the upper surface of the luggage 200 and the luggage 200, the measuring device 100 can measure the height of the luggage 200.

[0054] In the second embodiment, it is assumed that the height of the measurement device 100 from the floor surface 202 is known. As described above, the measurement device 100 may obtain the height (actual length) of the measurement device 100 from the floor surface 202 by a known measurement means such as a LIDAR or an altimeter. [Explanation of symbols]

[0055] 100 Measuring device, 102 Camera, 104 User interface processing unit, 106 Data processing unit, 108 Data storage unit, 110 Input unit, 112 Output unit, 114 First dimension calculation unit, 116 Second dimension calculation unit, 118 Dimension calculation unit, 120 Imaging surface, 200 Baggage, 202 Floor surface, 204 Top surface, 206 Second plane, M Marker, MP1 End point, MP2 End point

Claims

1. an imaging unit that images the marker and a measurement object having a side surface perpendicular to a placement surface and an upper surface parallel to the placement surface; a first dimension calculation unit that calculates an imaging length that is a length in a three-dimensional imaging space; a second dimension calculation unit that calculates an actual length that is a length in real space, The second dimension calculation unit calculates the actual length of the frame in the real space from an imaging length of the frame of the measurement object in the imaging space by referring to the actual length of the marker.

2. The first dimension calculation unit calculates an imaging length of the marker shown in a captured image, The measurement system according to claim 1 , wherein the second dimension calculation unit calculates the actual length of the frame based on a ratio of an imaged length of the marker to an actual length of the marker.

3. an imaging unit configured to image a measurement object having a side surface perpendicular to a placement surface and a top surface parallel to the placement surface; a dimension calculation unit that calculates an actual length that is a length in real space, The dimension calculation unit is identifying a position of the placement surface based on a height of an imaging viewpoint from the placement surface and an imaging angle; detecting a first point of a side surface of the measurement target object that is in contact with the placement surface from a captured image, and identifying coordinates of the first point in the real space; detecting a second point located vertically above the first point on an upper surface of the measurement object from the captured image, and identifying coordinates of the second point in the real space; A measurement system that calculates a height of the measurement object based on the coordinates of the first point and the coordinates of the second point in the real space.

4. The dimension calculation unit further Identifying the position of the upper surface of the measurement object from the identified placement surface; Detecting a third point, different from the second point, on the upper surface of the measurement object from the captured image, and specifying coordinates of the third point in the real space; The measurement system according to claim 3 , further comprising: a measuring unit configured to calculate a length of the measurement object based on the coordinates of the second point and the coordinates of the third point in the real space.

5. A function of capturing an image of the marker and a measurement object having a side surface perpendicular to the placement surface and a top surface parallel to the placement surface; A function of detecting a frame of the measurement object from a captured image; A function of calculating an imaging length, which is a length in a three-dimensional imaging space; a function of calculating an actual length of the frame in real space from an imaging length of the frame in the imaging space by referring to an actual length of the marker.

6. A function of capturing an image of a measurement object having a side surface perpendicular to a placement surface and a top surface parallel to the placement surface; A function of identifying a position of the placement surface based on a height of an imaging viewpoint from the placement surface and an imaging angle; a function of detecting a first point of a side surface of the measurement object that contacts the placement surface from a captured image and specifying coordinates of the first point in real space; a function of detecting a second point located vertically above the first point on an upper surface of the measurement object from the captured image and specifying coordinates of the second point in the real space; and a function of calculating a height of the object to be measured based on the coordinates of the first point and the coordinates of the second point in the real space.

Images