Position measurement system, position measurement method, and program

Abstract

To enable positioning an object even if there is not a plurality of imaging apparatuses.SOLUTION: A position measurement system acquires, from an image acquired by a camera 30 (imaging apparatus) having a known imaging direction and a known installation position in a space 50, a position of a direct image resulting from directly receiving light from a light-emitting device 20 that is movable in the space 50 and a position of a reflection image by reflected light resulting from light emitted by the light-emitting device 20 being reflected by a floor surface 51 (predetermined plane) within the space 50, and acquires a three-dimensional position of the light-emitting device 20 in the space 50 on the basis of the position of the reflection image and the position of the direct image in the image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a position measurement system, a position measurement method, and a program. [Background technology]

[0002] Conventionally, a technique is known in which markers are captured by a plurality of cameras and the positions of the markers in space are measured (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2005 / 124687 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 requires multiple cameras (imaging devices), and therefore there is a risk that the position of the object may not be measured depending on the positioning situation.

[0005] The present invention has been made in consideration of the above-mentioned problems in the conventional technology, and an object of the present invention is to enable the positioning of an object without using a plurality of imaging devices. [Means for solving the problem]

[0006] In order to solve the above problem, the position measurement system of the present invention includes a control unit that acquires, from an image acquired by an imaging device whose installation position in space and shooting direction are known, the position of a direct image that directly receives light from a light-emitting device that can move in the space, and the position of a reflected image that is the light emitted by the light-emitting device and reflected by a predetermined plane in the space, and acquires the three-dimensional position of the light-emitting device in the space based on the position of the direct image and the position of the reflected image in the image. [Effects of the Invention]

[0007] According to the present invention, it is possible to measure the position of an object without using a plurality of imaging devices. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing a configuration of a position measurement system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing a configuration of a light emitting device. [Figure 3] FIG. 2 is a block diagram showing the configuration of a server. [Figure 4] FIG. 1 is a diagram for explaining three-dimensional positioning using a camera. [Figure 5] 10 is a flowchart showing a position measurement process executed in the server. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of a positioning system, a positioning method, and a program according to the present invention will be described in detail with reference to the drawings. Note that the embodiments described below are subject to various limitations that are technically preferable for implementing the present invention, but the technical scope of the present invention is not limited to the following embodiments and illustrated examples.

[0010] As shown in FIG. 1, the position measurement system 1 of this embodiment includes forklifts 10a, 10b, and 10c, light-emitting devices 20a, 20b, and 20c, cameras 30a, 30b, and 30c as imaging devices, a hub 31, and a server 40. The three-dimensional position of a space 50 to which the position measurement system 1 is applied is determined by coordinates of mutually orthogonal X-, Y-, and Z-axes. The X- and Y-axes are taken in directions including the horizontal plane, and the XY plane is parallel to a floor 51 of the space 50 (see FIG. 4). The Z-axis is an axis along the vertical direction. Note that while FIG. 1 schematically illustrates a top view of the space 50, the forklifts 10a, 10b, and 10c are shown as a side view of the forklifts traveling on the floor 51 of the space 50 (the up-down direction corresponds to the Z-axis direction), which differs from their actual orientation.

[0011] Forklifts 10a, 10b, and 10c move within space 50. The forklifts 10a, 10b, and 10c are equipped with forks 11a, 11b, and 11c that move in the vertical direction (Z-axis direction), respectively, and move between shelves 60a, 60b, 60c, and 60d provided within space 50 to carry cargo. Light-emitting devices 20a, 20b, and 20c are attached to the forklifts 10a, 10b, and 10c, respectively. Therefore, the light-emitting devices 20a, 20b, and 20c are movable within space 50. Hereinafter, when there is no need to distinguish between the forklifts 10a, 10b, and 10c, they will be referred to as "forklifts 10." When there is no need to distinguish between the forks 11a, 11b, and 11c, they will be referred to as "forks 11." When there is no need to distinguish between the light-emitting devices 20a, 20b, and 20c, they will be referred to as "light-emitting devices 20."

[0012] Each of the cameras 30a, 30b, and 30c is installed near the ceiling of the space 50 and captures images of the space 50. Hereinafter, when there is no need to distinguish between the cameras 30a, 30b, and 30c, they will be referred to as "camera 30." The camera 30 includes a lens, a light-receiving element, and the like. The camera 30 captures an optical image of light incident through the lens and generates two-dimensional image data. The camera 30 captures images continuously over time and acquires each successively acquired image data (frame) as video image data. The installation position (three-dimensional position) and shooting direction (three-dimensional direction) of each camera 30 in the space 50, as well as the specifications of the lens and light-receiving element, are known. Wherever the forklift 10 moves within the space 50, it will be within the imaging range of one of the cameras 30. For example, when the forklift 10 is traveling between shelves 60a and 60b, the forklift 10 will be within the imaging range of camera 30a. When the forklift 10 is traveling between shelves 60b and 60c, the forklift 10 comes within the imaging range of camera 30b. When the forklift 10 is traveling between shelves 60c and 60d, the forklift 10 comes within the imaging range of camera 30c.

[0013] The light emitting device 20 transmits information to be transmitted by modulating the color or brightness of light in the wavelength region of visible light. For example, the light emitting device 20 transmits its identification information (light source ID that can uniquely identify itself) by using a light emission pattern of three colors, red, green, and blue (the order of colors, the time interval between blinking, etc.).

[0014] The position measurement system 1 measures the position of the light emitting device 20 using visible light communication. The light emitting device 20 is a transmitter in visible light communication. A camera 30 is used as a receiver of a signal transmitted from the light emitting device 20. Each camera 30 is connected to a server 40 via a hub 31. The server 40 analyzes images continuously acquired over time by each camera 30, identifies the light source ID of the light emitting device 20 captured in the image, and acquires the three-dimensional position of the light emitting device 20 in space 50. Camera visible light communication can accurately detect the direction in which the light emitting device 20 is visible (reception angle) relative to the front direction of the camera 30 (the shooting direction as seen from the installation position).

[0015] As shown in FIG. 2 , the light-emitting device 20 includes a control unit 21, a memory 22, a switch 23, a battery 24, a drive unit 25, a three-color LED (Light Emitting Diode) 26, and the like. The control unit 21 is configured by a CPU (Central Processing Unit). The control unit 21 controls each unit of the light-emitting device 20 in accordance with a program stored in the memory 22. The memory 22 stores programs executed by the control unit 21, various data, and the like. For example, the memory 22 stores identification information of the light-emitting device 20 (its own light source ID). The switch 23 includes a switch for turning on / off the power of the light-emitting device 20, and the like. The battery 24 supplies power to each unit of the light-emitting device 20.

[0016] The driver 25 drives the three-color LED 26. The driver 25 generates a drive signal for temporally changing the light emission state of each color in the three-color LED 26, and outputs the drive signal to the three-color LED 26. The three-color LED 26 includes three LEDs of red, green, and blue. The controller 21 determines a light emission pattern corresponding to the light source ID stored in the memory 22, and causes the driver 25 to generate a drive signal according to the light emission pattern. The three-color LED 26 emits light in the light emission pattern corresponding to the light source ID in accordance with the drive signal. The light emission pattern represents information by the order in which the colors change, the time interval between blinking, etc.

[0017] As shown in FIG. 3, server 40 includes a control unit 41, a storage unit 42, an image input unit 43, a display unit 44, an operation unit 45, a communication unit 46, and the like. Control unit 41 is configured with a CPU. Control unit 41 controls each unit of server 40 in accordance with a program stored in storage unit 42. Storage unit 42 stores programs executed by control unit 41, various data, and the like. For example, storage unit 42 stores, for each camera 30, the installation position (three-dimensional position) and shooting direction (three-dimensional direction) of each camera 30 in space 50. Image data output from each camera 30 is input to image input unit 43.

[0018] The display unit 44 is composed of an LCD (Liquid Crystal Display) or the like, and displays various information according to display information instructed by the control unit 41. The operation unit 45 has an operation input unit such as a keyboard, touch panel, mouse, etc. The operation unit 45 accepts operation input by the user and outputs the operation information to the control unit 41. The communication unit 46 is composed of a network interface or the like, and performs data communication with an external device connected via a communication network.

[0019] The control unit 41 processes the image captured by the camera 30 to extract and decode the light emission pattern (change in light emission) of the light emitting device 20. The control unit 41 identifies the light emitting device 20 appearing in the image based on the light source ID obtained by decoding. Since the installation position and shooting direction of the camera 30 are known, the control unit 41 can obtain the direction of the light emitting device 20 as seen from the camera 30 by obtaining the position of the light emitting device 20 (bright spot) in the captured image.

[0020] Referring to FIG. 4, three-dimensional positioning of the light-emitting device 20 using a camera 30 (monocular camera) will be described. The light-emitting device 20, which is a visible light communication transmitter, is attached to the fork 11 of the forklift 10. By installing the light-emitting device 20 on the fork 11, the light-emitting device 20 can emit light at a position linked to the height of the fork 11. The space 50 in which the forklift 10 travels is, for example, an indoor factory or warehouse. The floor surface 51 of the factory or warehouse is often tiled or has a mirror-like finish, such as a painted or waxed smooth surface. Therefore, the image acquired by the camera 30 includes not only a direct image directly captured from the light from the light-emitting device 20, but also a reflected image resulting from the light emitted by the light-emitting device 20 being reflected by the floor surface 51.

[0021] The control unit 41 of the server 40 acquires, from the captured image acquired by the camera 30, the position of a direct image that directly receives light from the light-emitting device 20 and the position of a reflected image that is the light emitted by the light-emitting device 20 and reflected by a predetermined plane in the space 50. In this embodiment, a case will be described in which the "predetermined plane" is the floor surface 51. The control unit 41 acquires the position of a direct image and the position of a reflected image that correspond to the same identification information (light source ID) from the captured image. In other words, the control unit 41 acquires the position of direct light reception and the position of light that is reflected by the floor surface 51 for images that use the same light-emitting device 20 as a light source.

[0022] To detect the direct image (real image) and reflected image (virtual image) of the light-emitting device 20 from the captured image acquired by the camera 30, the control unit 41 searches for areas in the captured image where light (color, etc.) changes and searches for different areas that are synchronized with the change in light. To determine whether an image that changes synchronously is a direct image or a reflected image, one or a combination of the brightness, shape, and capture position of both images is used. For example, a reflected image reflected by a non-mirror floor surface 51 has lower brightness and is distorted compared to a direct image. Furthermore, a reflected image reflected by the floor surface 51 is captured at a lower position in the vertical direction than the direct image. Based on these differences, the control unit 41 distinguishes between the direct image and the reflected image of the light-emitting device 20. If the direct image or reflected image is distorted in the captured image, the center of the image's outline (the center of the circumscribing circle, the center of the inscribing circle, the center of gravity, etc.) is considered to be the position of the image.

[0023] The control unit 41 of the server 40 acquires the three-dimensional position of the light-emitting device 20 in the space 50 based on the position of the direct image of the light-emitting device 20 in the captured image and the position of the reflected image of the light-emitting device 20 in the captured image.

[0024] As shown in FIG. 4, the height from the floor surface 51 to the installation position of the camera 30 is H c , the height of the light emitting device 20 from the floor surface 51 is Z1, and the distance between the camera 30 and the light emitting device 20 in the vertical direction (Z-axis direction) is Z2. cis the distance between camera 30 and floor surface 51 in the Z-axis direction, and Z1 is the distance between light emitting device 20 and floor surface 51 in the Z-axis direction. D1 is the distance between camera 30 and light emitting device 20 on the horizontal plane (XY plane). θ1 is the angle of depression when light emitting device 20 is viewed from the installation position of camera 30, and θ2 is the angle of depression when reflection position 52 of light emitting device 20 on floor surface 51 is viewed from the installation position of camera 30. With the above definitions, equations (1) to (3) hold for D1, Z1, and Z2.

number

[0025] By rearranging the formulas (1) to (3), D1 and Z2 can be obtained by the formulas (4) and (5).

number

[0026] That is, the control unit 41 determines the installation height H c From the angle θ1 at which a direct image of light-emitting device 20 is visible (known), and the angle θ2 at which a reflected image of light-emitting device 20 is visible, it is possible to calculate the distance D1 between camera 30 and light-emitting device 20 on the XY plane and the distance Z2 between camera 30 and light-emitting device 20 in the Z-axis direction. As a result, control unit 41 can measure the relative three-dimensional position of light-emitting device 20 with camera 30 as the reference. Because the installation position and shooting direction of camera 30 are known, control unit 41 can calculate the three-dimensional position of light-emitting device 20 in space 50.

[0027] From the position of the direct image of light-emitting device 20 in the captured image, control unit 41 calculates a first angle (angle θ1), which is the angle between a line that is perpendicular to floor surface 51 and included in an imaginary plane that includes camera 30 and light-emitting device 20, and is parallel to floor surface 51, and a line segment connecting camera 30 and light-emitting device 20. The "imaginary plane" corresponds to the paper surface in FIG. 4. The "straight line that is included in the imaginary plane and is parallel to floor surface 51" is a line that runs along the left-right direction in FIG. 4.

[0028] The control unit 41 calculates a second angle (angle θ2), which is the angle between a "straight line parallel to the floor surface 51 contained in the virtual plane" and a line segment connecting the camera 30 and the reflection position 52 of the light emitted by the light emitting device 20 on the floor surface 51, from the position of the reflected image of the light emitting device 20 in the captured image.

[0029] From the first angle (angle θ1) and the second angle (angle θ2), the control unit 41 calculates a first distance (distance D1) that is the distance between the camera 30 and the light-emitting device 20 in a direction parallel to a "straight line included in the imaginary plane and parallel to the floor surface 51." From the first angle (angle θ1) and the second angle (angle θ2), the control unit 41 calculates a second distance (distance Z2) that is the distance between the camera 30 and the light-emitting device 20 in a direction perpendicular to the floor surface 51 (Z-axis direction).

[0030] The control unit 41 acquires the three-dimensional position of the light-emitting device 20 in the space 50 based on the first distance (distance D1) and the second distance (distance Z2). The control unit 41 can determine the direction of the light-emitting device 20 as seen from the camera 30 from the position of the direct image of the light-emitting device 20 in the captured image. Therefore, if the control unit 41 can determine the distance D1 between the camera 30 and the light-emitting device 20 on the XY plane and the distance Z2 between the camera 30 and the light-emitting device 20 in the Z-axis direction, it can calculate the three-dimensional position (XYZ coordinates) of the light-emitting device 20.

[0031] The position measurement process will be described with reference to Fig. 5. In the position measurement process, the control unit 41 of the server 40 analyzes, for each camera 30, the captured image acquired by that camera 30. The "installation position and shooting direction of the camera 30 in the space 50" used in the position measurement process is the installation position and shooting direction of the camera 30 that acquired the captured image to be analyzed.

[0032] First, the control unit 41 extracts an area where light is changing from the captured image acquired by the camera 30 (step S1). Next, the control unit 41 decodes the light source ID from the change in light (light emission pattern) (step S2). Here, the control unit 41 determines whether the light source ID has been decoded (step S3).

[0033] If it is determined that the decoding has been successful (step S3; YES), the control unit 41 identifies the photographed light-emitting device 20 based on the obtained light source ID (step S4). Next, the control unit 41 determines whether there are two points of light that indicate the same light source ID in the photographed image (step S5). If it is determined that there are two points of light that indicate the same light source ID (step S5; YES), the control unit 41 identifies the direct image and the reflected image of the light-emitting device 20 in the photographed image (step S6). Here, as described above, the control unit 41 distinguishes and detects the direct image and the reflected image of the light-emitting device 20 for the bright points in the photographed image based on the brightness, shape, photographing position, etc. Next, the control unit 41 corrects the position of the reflected image (step S7). For example, if the reflected image is distorted, the control unit 41 determines the center of the outline of the image as the position of the reflected image.

[0034] Next, control unit 41 calculates angle θ1 of the direct image as seen from camera 30 from the position of the direct image in the captured image, the installation position of camera 30 in space 50, and the shooting direction (step S8). As shown in Fig. 4, angle θ1 is the depression angle when light-emitting device 20 is viewed from the installation position of camera 30.

[0035] Furthermore, control unit 41 calculates angle θ2 of the reflected image as seen from camera 30 from the position of the reflected image in the captured image, the installation position of camera 30 in space 50, and the shooting direction (step S9). As shown in Fig. 4, angle θ2 is the depression angle when reflection position 52 on floor surface 51 of light-emitting device 20 is viewed from the installation position of camera 30.

[0036] Next, the control unit 41 calculates the angle θ1, the angle θ2, and the installation height H of the camera 30 using the above formulas (4) and (5). c From this, the control unit 41 calculates the distance D1 between the camera 30 and the light emitting device 20 on the XY plane and the distance Z2 between the camera 30 and the light emitting device 20 in the Z-axis direction (step S10). Next, the control unit 41 calculates the three-dimensional position of the light emitting device 20 in the space 50 based on the distance D1, the distance Z2, the installation position of the camera 30 in the space 50, and the shooting direction (step S11).

[0037] In step S5, if it is determined that there are no two points of light that indicate the same light source ID (step S5; NO), the control unit 41 estimates the height of the light-emitting device 20 and calculates the two-dimensional position of the light-emitting device 20 (step S12). Specifically, the control unit 41 estimates the height of the light-emitting device 20 when the forks 11 of the forklift 10 are lowered to the lowest position as the current height Z1 of the light-emitting device 20. c and the estimated height Z1 of the light emitting device 20, the distance Z2 between the camera 30 and the light emitting device 20 in the Z-axis direction is determined. Therefore, if the angle θ1 (the depression angle when the light emitting device 20 is viewed from the installation position of the camera 30) is obtained from the captured image, the distance D1 between the camera 30 and the light emitting device 20 on the XY plane can be calculated using equation (6).

number

[0038] The control unit 41 calculates the two-dimensional position of the light emitting device 20 on the XY plane based on the distance D1, the installation position of the camera 30 in the space 50, and the shooting direction. Furthermore, the control unit 41 can estimate the three-dimensional position of the light emitting device 20 in the space 50 by combining this with the estimated height Z1 of the light emitting device 20.

[0039] In step S3, if it is determined that decoding has not been possible (step S3; NO), the position measurement process ends after step S11 or step S12.

[0040] As described above, according to this embodiment, the control unit 41 of the server 40 acquires, from the captured image acquired by the camera 30, the position of a direct image that directly receives light from the light-emitting device 20, and the position of a reflected image that is the light emitted by the light-emitting device 20 and reflected by the floor surface 51 (a predetermined plane) in the space 50. The control unit 41 acquires the three-dimensional position of the light-emitting device 20 in the space 50 based on the position of the direct image of the light-emitting device 20 and the position of the reflected image of the light-emitting device 20 in the captured image. Therefore, the control unit 41 can determine the position of the target (light-emitting device 20) without multiple imaging devices.

[0041] Conventionally, 3D positioning using a single marker required multiple cameras, but according to this embodiment, highly accurate 3D positioning is possible even with a monocular camera. Conventionally, camera placement was designed so that multiple cameras could capture the entire positioning area. In contrast, in this embodiment, the number of cameras 30 installed is half that of the conventional system, reducing installation costs and the load when processing captured images. Furthermore, according to this embodiment, 3D positioning can be achieved even in situations where the height of the measurement object changes when using a monocular camera for positioning.

[0042] 4 from the captured image acquired by camera 30, and calculates distances D1 and Z2 shown in FIG. 4 from angles θ1 and θ2. Then, control unit 41 acquires the three-dimensional position of light-emitting device 20 in space 50 based on distances D1 and Z2. This enables control unit 41 to easily perform three-dimensional positioning of light-emitting device 20.

[0043] Furthermore, the control unit 41 obtains the positions of the direct image and the reflected image corresponding to the same light source ID from the captured image based on the light source ID transmitted by the light emitting device 20, and therefore can easily obtain the positions of images (direct image and reflected image) that use the same light emitting device 20 as a light source.

[0044] Conventionally, the light reflected from the light emitting device 20 by the floor surface 51 or the like had to be removed as noise from the captured image, but in this embodiment, by utilizing the reflected image, it is possible to measure the three-dimensional position of the light emitting device 20 even with a monocular camera 30.

[0045] The above-described embodiments are merely examples of the positioning system, the positioning method, and the program according to the present invention, and the present invention is not limited to these. The detailed configurations and operations of the devices constituting the system may also be modified as appropriate without departing from the spirit of the present invention.

[0046] In the above embodiment, an example has been described in which the three-dimensional position of the light-emitting device 20 is measured using reflection from the floor surface 51. Alternatively, reflection from the ceiling surface may be used. Furthermore, reflection from the wall surface may be used, although this would complicate the calculation method. By having the server 40 identify flat surfaces in the space 50 that are prone to reflecting light, more accurate three-dimensional positioning becomes possible. Furthermore, by having the server 40 identify multiple reflective surfaces, more accurate positioning can be achieved.

[0047] Furthermore, the reflection of the light emitted by the light-emitting device 20 on the "predetermined plane" does not have to be specular reflection. For example, even if a clear image is not obtained as a reflected image from the captured image, it is sufficient that an emission pattern with the same light source ID can be detected for a combination of a direct image and a reflected image using the same light-emitting device 20 as a light source.

[0048] Furthermore, when only one light emitting device 20 is used in the position measurement system 1, it is sufficient to be able to detect light (bright spots) from the captured image acquired by the camera 30, so the light emitting device 20 does not need to transmit a light source ID.

[0049] Furthermore, the computer-readable medium storing the program for executing each process is not limited to the above examples. Furthermore, a carrier wave may be used as a medium for providing program data via a communication line. [Explanation of symbols]

[0050] 1... position measurement system, 20 (20a, 20b, 20c)... light emitting device, 30 (30a, 30b, 30c)... camera (imaging device), 40... server, 41... control unit

Claims

1. From an image acquired by an imaging device whose installation position in space and shooting direction are known, a position of a direct image that directly receives light from a light-emitting device that is movable in the space, and a position of a reflected image formed by light that is emitted by the light-emitting device and reflected on a predetermined plane in the space are acquired; acquiring a three-dimensional position of the light-emitting device in the space based on a position of the direct image and a position of the reflected image in the image; A position measurement system comprising a control unit.

2. The control unit calculating a first angle, which is an angle between a line that is perpendicular to the predetermined plane and is included in a virtual plane that includes the image capture device and the light emitting device, and that is parallel to the predetermined plane, and a line segment that connects the image capture device and the light emitting device, from a position of the direct image in the image; calculating a second angle, which is an angle between the straight line and a line segment connecting the imaging device and a reflection position of light emitted by the light-emitting device on the predetermined plane, from the position of the reflected image in the image; calculating a first distance, which is a distance between the imaging device and the light emitting device in a direction parallel to the straight line, and a second distance, which is a distance between the imaging device and the light emitting device in a direction perpendicular to the predetermined plane, from the first angle and the second angle; acquiring a three-dimensional position of the light-emitting device in the space based on the first distance and the second distance; The position measurement system of claim 1 .

3. the light-emitting device modulates light in the visible light region to transmit identification information of the light-emitting device; the control unit acquires the position of the direct image and the position of the reflected image corresponding to the same identification information from the image.

3. A position measurement system according to claim 1 or 2.

4. a step of acquiring, from an image acquired by an imaging device whose installation position in space and shooting direction are known, the position of a direct image that directly receives light from a light-emitting device that is movable in the space, and the position of a reflected image that is formed by light that is emitted by the light-emitting device and reflected on a predetermined plane in the space; acquiring a three-dimensional position of the light-emitting device in the space based on a position of the direct image and a position of the reflected image in the image; A position measurement method including:

5. Computer, a means for acquiring, from an image acquired by an imaging device whose installation position in space and shooting direction are known, the position of a direct image that directly receives light from a light-emitting device that is movable in the space, and the position of a reflected image that is formed by light that is emitted by the light-emitting device and reflected on a predetermined plane in the space; a means for acquiring a three-dimensional position of the light-emitting device in the space based on a position of the direct image and a position of the reflected image in the image; A program to function as a

Images