Measurement system

Abstract

To easily measure the shape of an object based on a picked-up image.SOLUTION: A measurement system includes an imaging unit that images a marker and a measurement target object, a height function derivation unit that derives a height function including a focal length of the imaging unit as an unknown focal length variable, an angle function derivation unit that derives an angle function including the focal length as the focal length variable, and a dimension calculation unit. The function calculation unit specifies coordinates of each of four or more end points of the measurement target object as a coordinate function based on the altitude function and the angle function, calculates temporary coordinate values from each of a plurality of coordinate functions when a mantissa value is set to a focal length variable, specifies a mantissa value when the temporary coordinate values satisfy an optimization condition as a maximum likelihood focal length, and calculates an actual length of the measurement target object based on the maximum likelihood focal length.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a technique for measuring the size of an object. [Background technology]

[0002] The most common method for measuring package size in logistics is to have the package pass through a gate equipped with an optical sensor, but this requires the installation of expensive dedicated equipment and is not suitable for measuring package sizes at the time of delivery.

[0003] In Patent Document 1, a stereo camera captures an image of a package from diagonally above, obtaining two captured images. The stereo camera sends the two captured images to a server, which creates a depth map based on the principle of triangulation. The server calculates the coordinates of the package's vertices based on the depth map, thereby calculating the package's size (see paragraphs

[0014] ,

[0015] ,

[0022] ,

[0023] , etc., in Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-211425 [Patent Document 2] Japanese Patent Application Publication No. 2018-112521 [Patent Document 3] Japanese Patent Application Laid-Open No. 2010-8352 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, it is assumed that all eight vertices will be measured (see paragraph

[0023] ), so it is considered necessary to capture images of the package from multiple directions. Furthermore, the measurement targets in Patent Document 1 are limited to rectangular parallelepiped packages moving on a conveyance path.

[0006] A primary object of the present invention is to provide a technique for easily measuring the shape of an object based on a captured image. [Means for solving the problem]

[0007] In another aspect of the present invention, a measurement system includes an imaging unit that images a marker of known size and a measurement object having sides perpendicular to a mounting surface and an upper surface parallel to the mounting surface; an altitude function derivation unit that derives an altitude function that indicates the altitude difference from the imaging viewpoint of the imaging unit to the marker as a function that includes the focal length of the imaging unit as an unknown focal length variable; an angle function derivation unit that derives an angle function that indicates the imaging angle of the imaging unit relative to the marker as a function that includes the focal length variable; and a dimension calculation unit that calculates an actual length, which is the length of the measurement object in real space. The dimension calculation unit includes a provisional calculation unit that determines the coordinates of each of the four or more endpoints of the object to be measured as a coordinate function including a focal length variable based on an altitude function and an angle function including a focal length variable; an optimization unit that calculates provisional coordinate values ​​of the four or more endpoints from each of the multiple coordinate functions when a provisional value is set to the focal length variable, and determines the provisional coordinate value when the provisional coordinate values ​​of the four or more endpoints satisfy the optimization condition as the most likely focal length; and a determination unit that calculates the actual length of the object to be measured based on the most likely focal length. [Effects of the Invention]

[0008] According to the present invention, the shape of an object can be easily measured. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 10 is a schematic diagram for explaining a method for measuring the shape of a package. [Figure 2] FIG. 2 is a schematic diagram showing a situation in which luggage is measured in the first embodiment. [Figure 3] FIG. 2 is a functional block diagram of the measurement device according to the first embodiment. [Figure 4] FIG. 2 is a schematic diagram for explaining a method for measuring three sides in the first embodiment. [Figure 5]4 is a flowchart showing the process of measuring three sides of a package in the first embodiment. [Figure 6] FIG. 10 is a functional block diagram of a measurement device according to a second embodiment. [Figure 7] FIG. 10 is a schematic diagram for explaining a method for measuring three sides in the second embodiment. [Figure 8] 10 is a flowchart showing the process of measuring three sides of a package in the second embodiment. [Figure 9] FIG. 11 is a schematic diagram showing a situation in which luggage is measured in the third embodiment. [Figure 10] FIG. 10 is a functional block diagram of a measurement device according to a third embodiment. [Figure 11] FIG. 11 is a schematic diagram for explaining a method for measuring three sides in the third embodiment. [Figure 12] 10 is a flowchart showing the process of measuring three sides of a package in the third embodiment. [Figure 13] 13 is a flowchart showing details of the optimization process in S50 of FIG. 12. [Figure 14] FIG. 10 is a schematic diagram for explaining a method for measuring three sides of a cylindrical package. DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following, a first embodiment will be described, which describes a method for measuring the dimensions of a package using markers. Next, a second embodiment will be described, which describes a method for measuring the dimensions of a package when some parameters, such as the height of the camera, are known. In a third embodiment, a method for measuring the dimensions of a package using markers when the focal length, height, and imaging angle of the camera are all unknown will be described. When the first, second and third embodiments are collectively referred to, or when no particular distinction is made, they will be referred to as "the present embodiment."

[0011] FIG. 1 is a schematic diagram for explaining a method for measuring the shape of a package 200. As shown in FIG. The measurement device 100 (measurement system) in this embodiment is a smartphone equipped with an imaging function. The measurement device 100 may also be a device dedicated to measurement.

[0012] First, the user places luggage 200 (measurement object) on floor surface 202, which serves as a placement surface. The luggage 200 is a rectangular parallelepiped object such as a cardboard box. Here, rectangular parallelepiped includes a "cube." Floor surface 202 is a horizontal plane, and the measurement object only needs to have a shape that has at least an upper surface 204 parallel to the horizontal floor surface 202 and side surfaces perpendicular to floor surface 202. The measurement object may also be in the shape of a polygonal prism or a cylinder.

[0013] The user holds the measuring device 100 diagonally above the luggage 200 and touches an image capturing button for measurement (not shown) (hereinafter referred to as the "measurement button") that is displayed on the screen of the measuring device 100. When the measurement button is touched, the measuring device 100 captures an image of the luggage 200 and the floor surface 202.

[0014] The measuring device 100 in this embodiment measures the length of each of the three sides (length, width, and height) of the luggage 200 based on an image captured by a camera (imaging unit) using a method described below (hereinafter also referred to as "three-side measurement").

[0015] In the following, the horizontal direction is defined as the XZ plane, and the Y axis is set in the height direction, as shown in Figure 1. The coordinate values ​​of each point on the horizontal plane are called "XZ coordinates," the coordinate values ​​indicating the height of each point from the floor 202 are called "Y coordinates," and the XYZ coordinates of each point are called "three-dimensional coordinates."

[0016] [First embodiment] FIG. 2 is a schematic diagram showing a situation in which a baggage 200 is measured in the first embodiment. In the first embodiment, a marker M is placed on the floor surface 202. The marker M may be a sticker, a piece of paper, a coin, or anything else, as long as it can be identified from the floor surface 202 by image recognition and is a thin object with a known length. The marker M may be placed on the top surface 204 of the baggage 200, but the first embodiment will be described assuming that the marker M is placed on the floor surface 202.

[0017] In this embodiment, the measuring device 100 captures images of the luggage 200 at positions and imaging angles that include at least four vertices (end points) P1 to P4 of the luggage 200. The height (Y width) of the luggage 200 is the distance between vertices P1 and P2. The length (X width) of the luggage 200 is the distance between vertices P1 and P3. The width (Z width) of the luggage 200 is the distance between vertices P1 and P4.

[0018] Hereinafter, the length in the three-dimensional imaged space (virtual space) assumed by the measuring device 100 will be referred to as the “imaged length,” and the actual length in the real space will be referred to as the “real length.” In this embodiment, the object is to measure the actual lengths of the height, length, and width of the luggage 200.

[0019] FIG. 3 is a functional block diagram of the measurement device 100 according to the first embodiment. Each component of the measuring device 100 is realized by hardware including computing units such as a CPU (Central Processing Unit) and various co-processors, storage devices such as memory and storage, and wired or wireless communication lines connecting them, as well as software stored in the storage devices and supplying processing instructions to the computing units. The computer program may be composed of device drivers, an operating system, various application programs located at higher levels than these, and libraries that provide common functions to these programs. Each block described below represents a functional block, not a hardware configuration.

[0020] The measurement device 100 includes a camera 102 (image capturing unit), a user interface processing unit 104, a data processing unit 106, and a data storage unit . The user interface processing unit 104 accepts operations from the user and is responsible for processing related to the user interface, such as image display and audio output. The data storage unit 108 stores various types of data. The data processing unit 106 executes various types of processing based on the captured image acquired by the camera 102, data acquired by the user interface processing unit 104, and data stored in the data storage unit 108. The data processing unit 106 also functions as an interface between the camera 102, the user interface processing unit 104, and the data storage unit 108.

[0021] The user interface processing unit 104 includes an input unit 110 that receives input from the user, and an output unit 112 that outputs various types of information such as images and sounds to the user.

[0022] The data processing unit 106 includes a first dimension calculation unit 114 and a second dimension calculation unit 116 . The first dimension calculation unit 114 detects the frame (sides) of the baggage 200 from the captured image and calculates the captured length of the frame. The second dimension calculation unit 116 performs three-side measurement by calculating the actual length of the frame.

[0023] FIG. 4 is a schematic diagram for explaining a method for measuring three sides in the first embodiment. It is assumed that the focal length and orientation (imaging angle) of the camera 102 are known. The measurement device 100 captures an image of the marker M. As described above, the marker M is placed on the floor surface 202. The first dimension calculation unit 114 identifies the position coordinates (three-dimensional coordinates) in a three-dimensional imaging space of the end points MQ1 and MQ2 of the marker M that appear in the captured image (imaging surface 120). The first dimension calculation unit 114 calculates the distance d between the end points MP1 and MP2 in this imaging space as the imaging length of the marker M.

[0024] The actual length D of the marker M is known. Next, the first dimension calculation unit 114 specifies the ratio r (=D / d) of the actual length D to the imaging length d as the ratio of the sizes of the imaging space and the actual space.

[0025] Next, the first dimension calculation unit 114 converts the position coordinates of the vertices P1 to P4 of the package 200 shown in the captured image into position coordinates (three-dimensional coordinates) in the captured space. The first dimension calculation unit 114 calculates the captured lengths of the height (P1 to P2), length (P1 to P3), and width (P1 to P4) of the package 200 in the captured space.

[0026] The second dimension calculation unit 116 calculates the actual lengths of the three sides of the package 200 by multiplying the image capture length by the ratio r for the height, length, and width of the package 200.

[0027] FIG. 5 is a flowchart showing the process of measuring three sides of the baggage 200 in the first embodiment. When the user holds the measuring device 100 over the baggage 200 and touches the measurement button, the following process starts. Specifically, the output unit 112 first displays a measurement button on the screen of the measuring device 100. The input unit 110 detects a touch on the measurement button. By simply touching the measurement button, the measuring device 100 automatically measures the three sides of the baggage 200.

[0028] When the user touches the measurement button, the measuring device 100 captures an image of the luggage 200 from diagonally above, and the captured image is acquired (S10). The first dimension calculation unit 114 first calculates the captured length of the marker M shown in the captured image (S12). At this time, the first dimension calculation unit 114 also calculates the ratio r of the captured length of the marker M to its actual length.

[0029] Next, the first dimension calculation unit 114 detects frames (P1-P2), frames (P1-P3), and frames (P1-P4) corresponding to the height, length, and width as frames of the package 200 from the captured image (S14). The first dimension calculation unit 114 calculates the captured image lengths of the three frames from the position coordinates of the vertices P1, P2, and P3 in the captured image space (S16). The second dimension calculation unit 116 converts the captured image lengths of the frames into actual lengths based on the ratio r (S18).

[0030] [Second embodiment] In the second embodiment, similarly to the imaging method shown in FIG. 2 , the measuring device 100 captures an image of the baggage 200 at an imaging angle that includes the vertices (end points) P1, P2, P3, and P4 of the baggage 200. In the second embodiment, the marker M is not placed on the floor surface 202. Meanwhile, the focal length and imaging angle of the measuring device 100 (camera 102), as well as the height H (actual length) of the imaging point of the camera 102 from the floor surface 202, are assumed to be known. The measuring device 100 may be installed in advance to match a specified imaging angle, such as 30 degrees, or the imaging angle at which the measuring device 100 is actually installed may be set as a parameter in the measuring device 100 using a known means such as an inclination sensor. Similarly, the height H of the imaging point of the measuring device 100 from the floor surface 202 may be measured using a known means such as an altimeter or LIDAR (Laser Imaging Detection and Ranging) and set as a parameter in the measuring device 100.

[0031] FIG. 6 is a functional block diagram of the measurement device 100 according to the second embodiment. The data processing unit 106 of the measuring device 100 in the second embodiment includes a dimension calculation unit 118. The dimension calculation unit 118 calculates the actual length of the frame (the side of the baggage 200) based on the position coordinates in the imaging space using a method described below. The other configurations are the same as those of the measuring device 100 in the first embodiment.

[0032] FIG. 7 is a schematic diagram for explaining a method for measuring three sides in the second embodiment. To explain the measurement principle, a method for calculating the height (P1 to P2) and length (P1 to P3) of the luggage 200 will be described, assuming that the luggage 200 is viewed from the side.

[0033] First, the imaging point of the camera 102 is set as the origin O (O). As described above, in the second embodiment, the height H of the imaging point O as an actual length is known. The tilt a (imaging angle) and focal length of the imaging surface 120 are also known.

[0034] Vertex P2 (first point) of baggage 200 contacts floor surface 202. Vertex P2 is projected as point Q2 on imaging plane 120. Similarly, vertex P1 (second point) corresponds to point Q1 on imaging plane 120, and vertex P3 (third point) corresponds to point Q3 on imaging plane 120.

[0035] Since the height H and the inclination a are known, the dimension calculation unit 118 can identify the position of the floor surface 202 in real space (the distance and direction from the imaging point O). In other words, the dimension calculation unit 118 can identify a plane equation of the floor surface 202 in real space (the XZ plane that is the distance H below the imaging point O) based on parameters such as the height H. Next, the dimension calculation unit 118 identifies the position coordinates of the intersection of the floor surface 202 and a projection line T2 connecting the imaging point O and point Q2 as the position coordinates of the vertex P2 in real space. Through the processing up to this point, the position coordinates of the vertex P2 in real space are identified.

[0036] Next, the dimension calculation unit 118 identifies a vertical line S1 that is perpendicular to the floor surface 202 and passes through the vertex P2. The dimension calculation unit 118 identifies the position coordinates of the intersection of the vertical line S1 and the projection line T1 connecting the imaging point O and the point Q1 as the position coordinates of the vertex P1 in real space.

[0037] The dimension calculation unit 118 determines the plane equation of a second plane 206 that is parallel to the floor surface 202 and passes through the vertex P1, i.e., a plane that includes the top surface 204 of the baggage 200. The dimension calculation unit 118 determines the position coordinates of the intersection of the second plane 206 and a projection line T3 connecting the imaging point O and point Q3 as the position coordinates of the vertex P3 in real space. The position coordinates of the other vertex P4 can also be calculated in a similar manner.

[0038] The first dimension calculation unit 114 calculates the actual lengths of the three sides of the package 200 based on the position coordinates of the vertices P1 to P4 in real space.

[0039] FIG. 8 is a flowchart showing the process of measuring three sides of the baggage 200 in the second embodiment. In the second embodiment as well, when the user holds the measuring device 100 over the baggage 200 and touches the measurement button, the following process is started.

[0040] When the user touches the measurement button, the measuring device 100 captures an image of the luggage 200 from diagonally above, and the captured image is acquired (S10). The dimension calculation unit 118 identifies a plane equation of the floor surface 202 (placing surface) from the captured image (S12).

[0041] Next, the dimension calculation unit 118 identifies the position coordinates in real space of vertex P2, which is the intersection of projection line T2 and floor surface 202 (S24). The dimension calculation unit 118 identifies a line S1 that passes through vertex P2 and is perpendicular to floor surface 202 (S26). The dimension calculation unit 118 identifies vertex P1, which is the intersection of projection line T1 and perpendicular line S1 (S28). Next, the dimension calculation unit 118 identifies a plane equation for the upper surface 204 of the package 200 (S30), and identifies vertex P3 by finding the intersection of projection line T3 and upper surface 204 (S32). In a similar manner, the dimension calculation unit 118 identifies vertex P4 (S34). The dimension calculation unit 118 calculates the actual lengths of the three sides of the package 200 based on the position coordinates of vertices P1 to P4 (S36).

[0042] The measurement principle is as described above, but finally, we will also mention a more specific calculation method in three-dimensional real space. First, the imaging point O is set as the origin, and based on the two-dimensional coordinates in the captured image of vertices Q1 and the like, three-dimensional vectors [Xi, Yi, Zi] from the imaging point O to each vertex Pi are created (hereinafter referred to as "vertex vectors").

[0043] Next, based on the imaging angle a, the above vertex vectors are transformed into a world coordinate system (a coordinate system of the imaging space) so that the coordinate XZ plane is parallel to the floor surface 202. Projection lines T1 to T3 are defined by the transformed vertex vectors.

[0044] In this world coordinate system, a plane equation of the floor surface 202 in the real space is calculated. Since the height of the imaging point O in the real space is H, the plane equation of the floor surface 202 is Y=H.

[0045] The coordinates [X2, Y2, Z2] of vertex P2 in the captured space are found as the intersection of projection line T2 and floor surface 202. Here, since Y2 = H, the coordinates of vertex P2 in the real space are [X2 * (Y2 / H), H, Z2 * (Y2 / H)].

[0046] Find the linear equation L2 of the line S1 that is perpendicular to the floor surface 202 from the vertex P2. Since L2 = [X2 * (Y2 / H), s, Z2 * (Y2 / H)], by calculating the intersection of the projection line T1 and the perpendicular line S1, P1 = [(X2 * Y2) / H, Y1 * (X2 * Y2) / (H * X1), (Z2 * Y2) / H)].

[0047] Now that the height of vertex P1 of upper surface 204 is known, the plane equation for upper surface 204 of luggage 200 is Y=Y1*(X2*Y2) / (H*X1). Thereafter, the intersection of vertices P3 and P4 is found in a similar manner based on the plane equation for upper surface 204 of luggage 200. From the above, the three sides of luggage 200 can be measured.

[0048] [Third embodiment] In the third embodiment, the focal length f of the camera 102, the imaging angle A, and the height H of the imaging point O are all unknown. To clarify that the focal length f is an unknown variable, the unknown focal length f is referred to as a "focal length variable f." As will be described in detail later, the imaging angle A is defined as a function including the focal length variable f. A function indicating the imaging angle A determined by the focal length variable f is referred to as an "angle function A(f)." Similarly, a function indicating the height of the imaging point O determined by the focal length variable f is referred to as an "altitude function H(f)."

[0049] FIG. 9 is a schematic diagram showing a situation in which the baggage 200 is measured in the third embodiment. In the third embodiment, a marker M is placed on the floor surface 202. As in the first embodiment, the marker M can be identified from the floor surface 202 by image recognition and can be a thin object with a known length. The marker M may be placed on the top surface 204 of the baggage 200, but the third embodiment will be described assuming that the marker M is placed on the floor surface 202.

[0050] A user captures an image of the upper surface 204 of the baggage 200 from diagonally above using a measuring device 100 with an imaging function. In the third embodiment, a prerequisite for trilateral measurement is that the four vertices P1 to P4 of the baggage 200 as well as the entire marker M are included in the captured image. Hereinafter, the line segment (frame) connecting vertex P1 (second point) and vertex P2 (first point) shown in FIG. 9 will be referred to as L1, the line segment connecting vertex P1 and vertex P3 (third point) as L2, and the line segment connecting vertex P1 and vertex P4 (fourth point) as L3. Also, the angle between line segment L1 and line segment L2 will be referred to as K1, the angle between line segment L1 and line segment L3 as K2, and the angle between line segment L2 and line segment L3 as K3. Because the baggage 200 is a rectangular parallelepiped, all of angles K1 to K3 are right angles (90 degrees).

[0051] FIG. 10 is a functional block diagram of a measurement device 100 according to the third embodiment. The data processing unit 106 of the measuring device 100 in the third embodiment includes a dimension calculation unit 130, an altitude function derivation unit 132, and an angle function derivation unit 134. The dimension calculation unit 118 calculates the actual lengths of the sides L1 to L3 of the luggage 200 based on position coordinates in the imaging space using a method described below. The altitude function derivation unit 132 derives an altitude function H(f) including a focal length variable f. The altitude function H(f) is a function that indicates the altitude difference between the imaging point O and the floor surface 202 (marker M). The angle function derivation unit 134 derives an angle function A(f) including the focal length variable f. The angle function A(f) is a function that indicates the imaging angle of the measuring device 100.

[0052] The dimension calculation unit 130 further includes a provisional calculation unit 140, an optimization unit 142, and a determination unit 144. The provisional calculation unit 140 calculates the position coordinates of the vertices P1 to P4 in real space based on an altitude function H(f) and an angle function A(f) using a method described below. However, since the position coordinates of the vertices P1 to P4 calculated by the provisional calculation unit 140 include an unknown focal length variable f, the position coordinates are expressed as a function including the focal length variable f (hereinafter referred to as a "coordinate function (f)"). The optimization unit 142 determines the maximum likelihood value of the focal length variable f (hereinafter referred to as the "maximum likelihood focal length F") based on the coordinate function (f) of the four vertices and the optimization conditions described below. The determination unit 144 determines the position coordinates of the vertices P1 to P4 based on the maximum likelihood focal length F and performs trilateral measurement. The other configuration is the same as that of the measurement device 100 in the first embodiment.

[0053] FIG. 11 is a schematic diagram for explaining a method for measuring three sides in the third embodiment. 7, the state of the baggage 200 when viewed from the side is assumed. In the third embodiment, the height H as the actual length of the imaging point O, the imaging angle A, and the focal length f are all unknown.

[0054] A user captures an image of a marker M using the measurement device 100. The size of the marker M is known. The altitude function derivation unit 132 derives an altitude function H(f) from the size of the marker M (known information). The angle function derivation unit 134 derives an angle function A(f) from the size of the marker M (known information). When a marker M of a known size is captured by a camera with a known focal length f, an algorithm for determining the altitude H and imaging angle A of the imaging point O is generally known. In the third embodiment, since the focal length f is unknown, the altitude H is determined as an altitude function H(f) including a focal length variable f (unknown). Similarly, the imaging angle A is determined as an angle function A(f) including a focal length variable f (unknown).

[0055] The horizontal direction of the captured image is the x-axis, and the vertical direction is the y-axis. In the third embodiment, when calculating the altitude function H(f) and the angle function A(f), it is assumed that the focal length fx in the x-axis direction and the focal length fy in the y-axis direction are the same (f = fx = fy). In addition, when calculating the altitude difference from marker M and the imaging angle, it is necessary to determine the imaging center of the captured image. In the third embodiment, it is assumed that the center point of the captured image, that is, the position corresponding to the midpoint of the x-axis and the midpoint of the y-axis, is the imaging center.

[0056] In summary, based on the captured image of the marker M and the above assumptions, the altitude function H(f) and angle function A(f), which include the focal length variable f as an unknown, are identified. If the focal length variable f can be determined, the altitude H and imaging angle A are also determined. The focal length variable f is determined by the optimization process described below.

[0057] Thereafter, coordinate functions R1(f) to R4(f) of vertices P1 to P4 are found in the same manner as in the second embodiment. Vertex P2 of baggage 200 contacts floor surface 202. Vertex P2 is projected as point Q2 on imaging plane 120. Similarly, vertex P1 (second point) corresponds to point Q1 on imaging plane 120, and vertex P3 (third point) corresponds to point Q3 on imaging plane 120.

[0058] The dimension calculation unit 118 determines a plane equation of the floor surface 202 in real space (an XZ plane that is a distance H(f) below the imaging point O) based on the altitude function H(f) and the angle function A(f). At this stage, the focal length variable f is uncertain, so the plane equation also includes the focal length variable f. Next, the dimension calculation unit 118 determines the position coordinates of the intersection of the floor surface 202 and a projection line T2 connecting the imaging point O and point Q2 as a coordinate function R2(f) that indicates the position coordinates of the vertex P2 in real space. In the following, all mathematical formulas C expressed as functions including the focal length variable f will be written as "C(f)".

[0059] The dimension calculation unit 118 identifies a vertical line S1(f) that is perpendicular to the floor surface 202 and passes through the coordinate function P2(f), and identifies a coordinate function R1(f) that indicates the position coordinates of the vertex P1 in real space.

[0060] The dimension calculation unit 118 determines the plane equation (f) of the plane including the upper surface 204 of the package 200, and determines the coordinate function R3(f) of the vertex P3. The coordinate function R4(f) of the other vertex P4 can be calculated in a similar manner. As described above, the coordinate functions corresponding to the vertices P1 to P4 are expressed as coordinate functions R1(f) to R4(f), respectively.

[0061] After finding coordinate functions R1(f) to R4(f), optimization unit 142 calculates the "maximum likelihood focal length F" as the most appropriate focal length f by an optimization process described below. By setting focal length variable f = maximum likelihood focal length F in coordinate functions R1(f) to R4(f), determination unit 144 finds the coordinate values ​​of vertices P1 to P4 in real space and then performs trilateral measurement.

[0062] FIG. 12 is a flowchart showing the process of measuring three sides of the baggage 200 in the third embodiment. In the third embodiment as well, when the user holds the measuring device 100 over the baggage 200 and touches the measurement button, the following process is started.

[0063] When the user touches the measurement button, the measuring device 100 captures an image of the baggage 200 and the marker M from diagonally above, and the captured image is acquired (S10). The camera 102 first identifies the marker M from the captured image (S40). The altitude function derivation unit 132 derives an altitude function H(f) based on the captured image of the marker M, and the angle function derivation unit 134 derives an angle function A(f) based on the captured image of the marker M (S42). The processing steps from S22 to S34 are the same as those in the second embodiment. However, because the focal length variable f is unknown at this stage, the coordinate functions R1(f) to R4(f) corresponding to the vertices P1 to P4 all include the unknown focal length variable f.

[0064] After the coordinate function R4(f) is derived from the coordinate function R1(f), the optimization unit 142 executes an optimization process to find the most likely focal length F (S50). Details of the optimization process will be explained with reference to the following FIG. 13. As a result of the optimization process, the determination unit 144 determines the position coordinates of the vertices P1 to P4, and thereby calculates the actual lengths of the sides L1 to L3 (S36).

[0065] FIG. 13 is a flowchart showing the details of the optimization process in S50 of FIG. An arbitrary variable range is set for the focal length variable f. For example, the variable range is set to 100 to 10000. In the following description, the minimum value of the variable range is f1 and the maximum value is fM.

[0066] The optimization unit 142 first temporarily sets the focal length variable f to the minimum value f1 (S60). The numerical value temporarily set for the focal length variable f is called the "temporary value." If the mantissa value f is not greater than fM (N in S62), the optimization unit 142 calculates coordinate functions R1(f) to R4(f) based on the mantissa value (S64). For example, if the mantissa value f=f1, the optimization unit 142 calculates the coordinate values ​​of the vertices P1 to P4 by substituting f=f1 into the coordinate functions R1(f) to R4(f). Hereinafter, the coordinate values ​​calculated based on the mantissa value are called "temporary coordinate values." Furthermore, the temporary coordinate value when the mantissa value is fx will be expressed as "temporary coordinate value (fx)."

[0067] The optimization unit 142 calculates a linear equation for the line segments L1(f1) to L2 based on the temporary coordinate values ​​R1(f1) to R4(f1) and calculates angles K1(f1) to K(f1) (S66). In this way, angles K1(f1) to K3(f1) can be calculated assuming that the focal length f is the temporary value f1.

[0068] Next, the optimization unit 142 calculates the total angle error E(f1) (S68). In the third embodiment, the total angle error E(f) is calculated as follows: error S1 (first error) + error S2 (second error) + error S3 (third error). Error S1 = absolute value of (angle K1(f1) - 90 degrees), error S2 = absolute value of (angle K2 - 90 degrees), error S3 = absolute value of (angle K3 - 90 degrees). Since the assumed angles of angles K1 to K3 are all 90 degrees, if the actual focal length is the same as mantissa value f1, the total angle error E(f1) will be zero. On the other hand, if the focal length is significantly different from mantissa value f1, the total angle error E(f1) will be a large value.

[0069] The optimization unit 142 increments the mantissa value of the focal length variable f, changing the focal length variable f to f=f1+1 (S70). After incrementing the focal length variable f, the process returns to S62. Next, the optimization unit 142 again calculates the temporary coordinate values ​​R1(f1+1) to R4(f1+1) and angles K1(f1+1) to K3(f1+1) of the vertices P1 to P4 based on the mantissa value "f1+1" of the focal length variable f, and then calculates the total angle error E(f1+1).

[0070] Similarly, the total angular error E(f) is calculated for all mantissa values ​​in the range of f1≦f≦fM. When f>fM (Y in S62), the optimization unit 142 selects the mantissa value f when the total angular error E(f) is at its smallest value as the focal length that satisfies the "optimization conditions," and sets this as the most likely focal length F (S72). For example, if the most likely focal length F=f1+m, this means that the focal length f=f1+m is the value closest to the true focal length.

[0071] The determination unit 144 calculates the position coordinates of the vertices P1 to P4 based on the most likely focal length F=f1+m (S74). As described above, the determination unit 144 calculates the actual lengths of the three sides L1 to L3 of the package 200 based on the position coordinates P1(f1+m) to P4(f1+m) (S36 in FIG. 12).

[0072] [Summary] The measurement device 100 (measurement system) has been described above based on the embodiment. According to this embodiment, a user can measure the three sides of a piece of luggage 200 with high accuracy simply by holding the measuring device 100 over the piece of luggage 200 and touching the measurement button. The user does not need to capture an image of the piece of luggage 200 from directly above, but can simply capture an image of the piece of luggage 200 from a direction that brings all three sides of the piece of luggage 200 into view. In recent years, with the spread of flea market applications, there have been more opportunities for individuals to sell their personal belongings over the Internet. Since users can easily measure the three sides of a piece of personal belongings by capturing an image of the piece of personal belongings with the measuring device 100, they can easily register the size information required when listing the item on a website.

[0073] In the first embodiment, the actual size of the luggage 200 shown in the captured image, i.e., the actual lengths of the three sides, can be measured by comparing the actual length of the marker M with the captured length. In the second embodiment, the three sides of the luggage 200 can be measured without using the marker M, simply by appropriately setting the height and imaging angle of the luggage 200 and capturing images of the four vertices included in the three sides of the luggage 200. Therefore, even if the measurement device 100 is not equipped with a so-called depth sensor such as LIDAR, it is possible to easily measure the three sides of the luggage 200. If LIDAR is equipped, the measurement device 100 may determine the actual height H of the imaging point O using the LIDAR.

[0074] In the third embodiment, even if the focal length f of a web camera is unknown, trilateral measurement is possible simply by using a marker M of a known size.

[0075] The present invention is not limited to the above-described embodiments and modifications, and the components can be modified without departing from the spirit of the invention. Various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments and modifications. Furthermore, some components can be omitted from all the components shown in the above-described embodiments and modifications.

[0076] [Variations] In the present embodiment, the measurement device 100 is described as performing three-side measurement. However, a measurement system may be configured by the measurement device 100 and a server. The server may share some of the functions of the measurement device 100 (for example, all or part of the first dimension calculation unit 114, the second dimension calculation unit 116, and the dimension calculation unit 118), thereby reducing the processing load on the measurement device 100. A third device other than the measurement device 100 and the server may also take on some of the functions of the measurement device 100 or the server. How to allocate multiple functions required to realize the present invention to one or more pieces of hardware may be determined in consideration of the processing capabilities of each piece of hardware, the specifications required for the measurement system, etc.

[0077] In this embodiment, a description has been given assuming that the luggage 200 is imaged from diagonally above. In the first embodiment, the user may image the luggage 200 from directly above. By placing markers M on both the top surface of the luggage 200 and the luggage 200 itself, the measuring device 100 can measure the height of the luggage 200.

[0078] In the second embodiment, it is assumed that the height of the measurement device 100 from the floor surface 202 is known. As described above, the measurement device 100 may obtain the height (actual length) of the measurement device 100 from the floor surface 202 by known measurement means such as LIDAR or an altimeter.

[0079] In the third embodiment, the optimization unit 142 provisionally sets a focal length variable f to a value ranging from the minimum value f1 to the maximum value fM, calculates the angular total error E(f) corresponding to all the mantissa values, and determines that the mantissa value when the angular total error E(f) is the minimum value is the most likely focal length F that satisfies the optimization conditions.

[0080] As a modified example, the optimization unit 142 may determine the focal length variable f when the total angle error E(f) is equal to or smaller than a predetermined threshold as the most likely focal length F. Alternatively, in the process of changing the focal length variable f, the optimization unit 142 may determine the focal length variable f when the total angle error E(f) changes from a decrease to an increase as the most likely focal length F.

[0081] The formula for calculating the total angle error E(f) only needs to be a monotonically increasing function of three types of variables: error S1 = {absolute value of the difference between angle K1 and the assumed angle (90 degrees)}, error S2 = {absolute value of the difference between angle K2 and the assumed angle (90 degrees)}, and error S3 = {absolute value of the difference between angle K3 and the assumed angle (90 degrees)}. For example, the total angle error E may be S1 × S2 × S3, or the total angle error E(f) = S1 + S2 + S3.

[0082] In the third embodiment, an image is captured so as to include four vertices (end points) of the baggage 200, but an image may be captured so as to include five or six vertices.

[0083] The third embodiment has been described assuming that the luggage 200 is a rectangular parallelepiped. The measuring device 100 can also measure the size (height and diameter) of a cylinder.

[0084] FIG. 14 is a schematic diagram for explaining a method for measuring three sides of a cylindrical baggage 200. As in the third embodiment, the cylindrical baggage 200 is imaged by the measuring device 100 together with the marker M. The height H as the actual length of the imaged point O, the imaged angle A, and the focal length f are all unknown.

[0085] The altitude function derivation unit 132 derives an altitude function H(f) from the size (known information) of the marker M. The angle function derivation unit 134 derives an angle function A(f) from the size (known information) of the marker M. The focal length f is unknown. Also, similar to the third embodiment, the focal length fx in the x-axis direction and the focal length fy in the y-axis direction are assumed to be the same, and the center point of the captured image is set as the imaging center.

[0086] Using the same algorithm as in the second embodiment, the coordinate function R2(f) of the end point P2 where the luggage 200 touches the floor surface is calculated, and the coordinate function R1(f) of the end point P1 located vertically above the end point P2 is calculated. The measuring device 100 calculates the plane equation of the top surface of the luggage 200, and calculates the coordinate function R3(f) of the end point P3 by connecting the center point C of the circular top surface, which appears as an ellipse in the captured image, to the end point P1. The side (P1-P3) is the diameter of the top surface of the luggage 200. Next, the dimension calculation unit 130 identifies the end points P4 and P5 of the side that is perpendicular to the side (P1-P3) and passes through the center C of the circular top surface, and thereby determines the coordinate functions R4(f) and R5(f) of the end points P4 and P5, respectively.

[0087] After determining coordinate functions R1(f) to R5(f), optimization unit 142 calculates "maximum likelihood focal length F" as the most appropriate focal length f through optimization processing, which will be described later. By setting focal length variable f = maximum likelihood focal length F in coordinate functions R1(f) to R5(f), determination unit 144 determines the coordinate values ​​of endpoints P1 to P5 in real space, and then calculates the height (P1-P2) of package 200 and the diameter of the top surface of package 200 (length L1 of side (P1-P3) or length L2 of side (P4-P5)).

[0088] In the optimization process, the difference between the length L1(f) of the side (P1-P3) and the length L1(f) of the side (P4-P5) is defined as error S1. The absolute value of the difference between the angle K1(f) formed by the side (P1-P2) and the side (P1-P3) and 90 degrees is defined as error S2. The absolute value of the difference between the angle K2(f) formed by the side (P1-P2) and the side (P4-P5) and 90 degrees is defined as error S3. The optimization unit 142 calculates error S1 + error S2 + error S3 as the total error, and calculates the f value when the total error is the smallest as the most likely focal length F. The determination unit 144 calculates the actual height and diameter of the package 200 based on the most likely focal length F.

[0089] Similarly, the measuring device 100 can also measure the size of a polygonal pillar-shaped package 200. [Explanation of symbols]

[0090] 100 Measuring device, 102 Camera, 104 User interface processing unit, 106 Data processing unit, 108 Data storage unit, 110 Input unit, 112 Output unit, 114 First dimension calculation unit, 116 Second dimension calculation unit, 118 Dimension calculation unit, 120 Imaging surface, 130 Dimension calculation unit, 132 Altitude function derivation unit, 134 Angle function derivation unit, 140 Temporary calculation unit, 142 Optimization unit, 144 Confirmation unit, 200 Baggage, 202 Floor surface, 204 Top surface, 206 Second plane, M Marker, MP1 End point, MP2 End point

Claims

1. an imaging unit that images a marker having a known size and a measurement object having a side surface perpendicular to a placement surface and an upper surface parallel to the placement surface; an altitude function derivation unit that derives an altitude function that indicates an altitude difference from an imaging viewpoint of the imaging unit to a marker as a function that includes a focal length of the imaging unit as an unknown focal length variable; an angular function derivation unit that derives an angular function that indicates an imaging angle of the imaging unit with respect to the marker as a function including the focal length variable; a dimension calculation unit that calculates an actual length, which is the length of the measurement object in real space; The dimension calculation unit a provisional calculation unit that specifies coordinates of each of four or more end points of the measurement object as a coordinate function including the focal length variable based on the altitude function and the angle function including the focal length variable; an optimization unit that calculates temporary coordinate values ​​of the four or more end points from each of a plurality of coordinate functions when a mantissa value is set to the focal length variable, and identifies the mantissa value when the temporary coordinate values ​​of the four or more end points satisfy an optimization condition as a most likely focal length; a determination unit that calculates an actual length of the measurement object based on the most likely focal length.

2. The preliminary calculation unit detecting a first point on a side surface of the measurement object that is in contact with the placement surface from the captured image, and specifying coordinates of the first point in the real space as a first coordinate function including the focal length variable; detecting a second point located vertically above the first point on the upper surface of the measurement object from the captured image, and specifying coordinates of the second point in the real space as a second coordinate function including the focal length variable; detecting a third point and a fourth point, each different from the second point, on the upper surface of the measurement object from the captured image, and specifying coordinates of the third point and the fourth point in the real space as a third coordinate function and a fourth coordinate function, each including the focal length variable; The optimization unit calculating first to fourth temporary coordinate values ​​based on the first to fourth coordinate functions when a mantissa value is set to the focal length variable; The measurement system according to claim 1 , wherein a mantissa value when the first to fourth temporary coordinate values ​​satisfy the optimization condition is specified as the most likely focal length.

3. The optimization unit 3. The measurement system according to claim 2, wherein a mantissa value when three types of errors satisfy the optimization condition, namely, a first error being an error between a first angle formed by a first side connecting the first temporary coordinate value and the second temporary coordinate value and a second side connecting the second temporary coordinate value and the third temporary coordinate value, and an assumed angle; a second error being an error between a second angle formed by a third side connecting the second coordinate value and the fourth coordinate value and the first side, and an assumed angle; and a third error being an error between a third angle formed by the second side and the third side, and an assumed angle, is identified as the most likely focal length.

4. The measurement system according to claim 3 , wherein the optimization unit determines that the optimization condition is met when a monotonically increasing function value including the first error, the second error, and the third error is minimized.

5. a function of capturing an image of a measurement object having a marker of known size, a side surface perpendicular to a placement surface, and an upper surface parallel to the placement surface; a function of deriving an altitude function that indicates an altitude difference from an imaging viewpoint of the imaging unit to a marker as a function that includes a focal length of the imaging unit as an unknown focal length variable; a function of deriving an angle function indicating an imaging angle of the imaging unit with respect to the marker as a function including the focal length variable; a function of identifying coordinates of each of four or more end points of the measurement object as a coordinate function including the focal length variable based on the altitude function and the angle function including the focal length variable; a function of calculating temporary coordinate values ​​of the four or more end points from each of a plurality of coordinate functions when a mantissa value is set to the focal length variable, and specifying the mantissa value when the temporary coordinate values ​​of the four or more end points satisfy an optimization condition as the most likely focal length; a measurement program that causes a computer to perform a function of calculating the actual length of the object to be measured based on the maximum likelihood focal length.

Images