Mobile object positioning device, mobile object positioning method and program

Abstract

To improve flexibility in the calculation of a position and an azimuth.SOLUTION: A server 200 obtains an image obtained by imaging of a marker 102a and a marker 102b with a camera 101a. Then, the server 200 corrects the distortion of the obtained image, thereby correcting positions of the marker 102a and the marker 102b. Furthermore, the server 200 converts image positions of the marker 102a and the marker 102b in the images after correction into three-dimensional positions of a camera 101a (forklift 100) by using a camera internal parameter and the like.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a mobile body positioning device, a mobile body positioning method, and a program.

Background Art

[0002] Conventionally, there has been a technique in which an imaging device attached to a mobile body captures a marker that is a light-emitting device in a space, and based on an image obtained by the imaging, calculates the position and orientation of the mobile body in a three-dimensional space (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above-described technique, there is a constraint that the markers must be arranged on the same plane, and there is a problem that the degree of freedom in installing the markers, and further, the degree of freedom in calculating the position and orientation is low.

[0005] The present invention has been made in view of such problems, and an object thereof is to improve the degree of freedom in calculating the position and orientation.

Means for Solving the Problems

[0006] To achieve the above object, a mobile body positioning device according to the present invention acquires coordinate positions of at least two light-emitting devices in an image based on an image including at least two light-emitting devices imaged by an imaging device attached to a mobile body from among a plurality of light-emitting devices whose coordinate positions in a three-dimensional space are known, the plurality of light-emitting devices including two light-emitting devices installed at arbitrary positions having different heights in the three-dimensional space, Based on the coordinate position in the three-dimensional space and the coordinate position in the image, at least one of the coordinate position of the moving object in the three-dimensional space and the direction the moving object is facing is determined. Execute the process. [Effects of the Invention]

[0007] According to the present invention, the degree of freedom in calculating position and orientation can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of a visible light communication system according to an embodiment of the present invention. [Figure 2] This figure shows an example of a server configuration. [Figure 3] This figure shows an example of the placement of a camera and marker in space. [Figure 4] This figure shows an example of a marker 3D position table. [Figure 5] This figure shows an example of camera internal parameters. [Figure 6] (a) is an example of an image before distortion correction, and (b) is an example of an image after distortion correction. [Figure 7] This figure shows an example of the image position and 3D position of a marker. [Figure 8] This figure shows an example of an XY plane vector. [Figure 9] (a) is an example of XY plane vectors before adjustment, and (b) and (c) are examples of XY plane vectors after adjustment. [Figure 10] This figure shows an example of a candidate line for camera position. [Figure 11] This figure shows an example of obtaining the intersection point between the candidate camera position line and the extension vector. [Figure 12] This is the first flowchart illustrating an example of server operation. [Figure 13] This is a second flowchart illustrating an example of server operation. [Figure 14]It is a third flowchart showing an example of the operation of the server. [Figure 15] It is a diagram showing an example of the XY plane vector when there are two cameras.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, a visible light communication system as a mobile body positioning device according to an embodiment of the present invention will be described with reference to the drawings.

[0010] FIG.  1 is a diagram showing the configuration of a visible light communication system. As shown in FIG.  1, the visible light communication system 1 includes a forklift 100 as a mobile body moving in a space 500, a camera 101a attached to the forklift 100, markers 102a, 102b, 102c which are light sources attached to the ceiling or side surfaces of the space 500 (hereinafter, when not limiting each of the markers 102a, 102b, 102c, they are appropriately referred to as "marker 102"), and a server 200 corresponding to the mobile body positioning device. The marker 102 includes an LED (Light Emitting Diode) not shown. The space 500 can be defined by three-dimensional parameters consisting of the coordinates of the X-axis (X coordinate), the coordinates of the Y-axis (Y coordinate), and the coordinates of the Z-axis (Z coordinate), and thereby the position of an object existing in the space 500 can be specified. Here, both the X-axis and the Y-axis are in the horizontal direction, are orthogonal to each other, the Z-axis is in the vertical direction, and is orthogonal to the X-axis and the Y-axis.

[0011] In the present embodiment, the camera 101a attached to the forklift 100 images the marker 102. The marker 102 has a known three-dimensional position in the space 500, performs color modulation with various transmission target information, and transmits the information by changing its emission color along the time series. The modulation method is to convert the transmission target information into bit data, and assign this data to the order of the three primary colors of light (R (red), G (green), B (blue)) with a predetermined regularity.

[0012] On the one hand, the server 200 demodulates the change in the emission color in the optical image obtained by the sequential continuous imaging by the camera 101a to acquire the information emitted by the marker 102, and calculates the three-dimensional position of the camera 101a in the space 500 as the three-dimensional position of the forklift 100.

[0013] FIG. 2 is a diagram showing an example of the configuration of the server 200. As shown in FIG. 2, the server 200 includes a control unit 202, a wireless communication unit 203, an image processing unit 204, a memory 205, an operation unit 206, a display unit 207, and a communication unit 208.

[0014] FIG. 3 is a diagram showing an example of the arrangement of the camera 101a and the marker 102 in the space 500. In the present embodiment, the server 200 calculates the three-dimensional position of the unknown camera 101a based on the three-dimensional position of the known marker 102 in the space 500.

[0015] The camera 101a images (receives light) the optical image incident through a lens (not shown), converts the image signal within the imaging angle of view into digital data, and generates an image. Further, the camera 101a performs imaging and image generation continuously in time, and transmits the consecutive images to the server 200 by wireless communication. The wireless communication unit 203 in the server 200 receives the image and outputs it to the image processing unit 204. The position of each pixel and image region in the image is defined in a two-dimensional coordinate plane composed of an X coordinate and a Y coordinate.

[0016] Returning to FIG. 2 for explanation again. The image processing unit 204 outputs the image (digital data) output from the wireless communication unit 203 as it is to the control unit 202, and also adjusts the image quality and image size for display on the display unit 207 and outputs it to the control unit 202. Further, the image processing unit 204 has a function of encoding, converting into a still image file, or converting into a moving image file using a compression encoding method such as JPEG (Joint Photographic Experts Group) or MPEG (Moving Photographic Experts Group).

[0017] The control unit 202 is composed of a processor, which is a CPU (Central Processing Unit). The control unit 202 controls the various functions of the server 200 by executing software processing according to a program stored in the memory 205 (for example, a program for realizing the operation of the server 200 as shown in Figures 12, 13, and 14, which will be described later).

[0018] Memory 205 is, for example, RAM (Random Access Memory) or ROM (Read Only Memory). Memory 205 stores various information (programs, etc.) used for control and other purposes in the server 200.

[0019] Furthermore, memory 205 stores the marker 3D position table shown in Figure 4 and the camera internal parameters shown in Figure 5. The marker 3D position table shown in Figure 4 associates the unique identification ID (ID) of each marker 102a, 102b, and 102c with the 3D position (X coordinate, Y coordinate, Z coordinate) of the marker 102a, 102b, and 102c in space 500.

[0020] Furthermore, the camera internal parameters shown in Figure 5 include image distortion correction data used to correct image distortion, the height (Z coordinate) of the camera 101a, the camera orientation matrix Rz(θ), which is a 3x3 matrix indicating the rotation of the camera 101a around the Z axis as orientation data indicating the direction the camera 101a is facing, and the camera positioning matrix Rc, which is a 3x3 matrix indicating the rotation of the camera 101a around the X axis and rotation around the Y axis as orientation data indicating the direction the camera 101a is facing.

[0021] Let's return to Figure 2 for further explanation. The operation unit 206 is an interface used to input user operations, consisting of a numeric keypad, function keys, etc. The display unit 207 is composed of, for example, an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), or an EL (Electroluminescence) display. The display unit 207 displays images according to the image signals output from the control unit 202. The communication unit 208 is, for example, a LAN (Local Area Network) card. The communication unit 208 communicates with external communication devices.

[0022] The control unit 202 comprises the following functions: an image acquisition unit 230, a marker identification unit 232, an image position correction unit 234, and a 3D position acquisition unit 236.

[0023] The image acquisition unit 230 acquires an image from the image processing unit 204. The marker identification unit 232 detects the position of marker 102 in the image (image position: X coordinate and Y coordinate in a two-dimensional plane corresponding to the image). Here, markers 102a, 102b, and 102c in space 500 emit light that cyclically changes in a pattern of three colors, R (red), G (green), and B (blue), modulated with an ID that can uniquely identify itself.

[0024] The marker identification unit 232 detects a cyclical pattern of three colors of light contained in each image obtained by the camera 101a. Furthermore, the marker identification unit 232 attempts to detect an ID corresponding to this pattern of three-color emission and to demodulate it back into an ID. If an ID is detected, the marker identification unit 232 obtains the image position of the marker 102 corresponding to that ID within the image.

[0025] Furthermore, the marker identification unit 232 obtains the three-dimensional position of the marker 102 in space 500 corresponding to the acquired ID by referring to the marker three-dimensional position table stored in memory 205.

[0026] The image position correction unit 234 corrects the image position of the marker 102. Specifically, as shown in Figure 6(a), the normal image 601a is distorted, and this distortion increases with distance from the center of the image. Therefore, the image position correction unit 234 corrects the distortion using the image distortion correction data of the camera's internal parameters shown in Figure 5. The image distortion correction data is data in which the amount of correction increases with distance from the center of the image. As a result, the distortion is corrected, resulting in the image 601b shown in Figure 6(b), and the positions (image positions) of the markers 102a, 102b, and 102c in the image 601b are corrected.

[0027] Furthermore, the image position correction unit 234 sets the markers 102 located at the four corners of the distortion-corrected image so that they are not used when acquiring the 3D position of the camera 101a. For example, as shown in Figure 6(b), triangular invalid regions 603a, 603b, 603c, and 603d are set at the four corners of the image 601b. Since marker 102c is located within the invalid region 603a, marker 102c is not used when acquiring the 3D position of the camera 101a, and only markers 102a and 102b are used when acquiring the 3D position of the camera 101a.

[0028] The 3D position acquisition unit 236 acquires the 3D position of camera 101a in space 500. Specifically, the 3D position acquisition unit 236 calculates the 3D position C=(cx,cy,cz) of camera 101a in space 500 using a known method, based on the image positions 610a and 102b of marker 102a in the image region 700 shown in Figure 7, the 3D positions of marker 102a and marker 102b obtained by referring to the marker 3D table, and the camera orientation matrix Rz(θ) and camera placement matrix Rc, which are pose data stored in memory 205.

[0029] Specifically, the 3D position acquisition unit 236 calculates a 3D direction vector (vx1, vy1, vz1) from camera 101a to marker 102a based on the 3D position of marker 102a, camera internal parameters, and pose data, in a camera-referenced relative system which is a coordinate system in a 3D space centered on camera 101a. It also calculates a 3D direction vector (vx2, vy2, vz2) from camera 101b to marker 102 based on the image position and pose data of marker 102b.

[0030] Next, the 3D position acquisition unit 236 calculates an XY plane vector V1 by projecting the 3D direction vector (vx1, vy1, vz1) from camera 101a to marker 102a onto the XY plane using a known method, based on the height difference between marker 102a and camera 101a and the camera placement matrix Rc1, in order to apply the 3D direction vector (vx1, vy1, vz1) from camera 101a to marker 102a to the positional relationship between camera 101a and marker 102a in 3D space. Similarly, the 3D position acquisition unit 236 calculates an XY plane vector V2 by projecting the 3D direction vector (vx2, vy2, vz2) from camera 101a to marker 102b onto the XY plane, based on the height difference between marker 102b and camera 101a and the camera placement matrix Rc2, in order to apply the 3D direction vector (vx2, vy2, vz2) from camera 101a to marker 102b to the positional relationship between camera 101a and marker 102b in 3D space. Figure 8 shows an example of XY plane vectors V1 and V2.

[0031] Furthermore, the 3D position acquisition unit 236 adjusts the XY plane vector V1 and the XY plane vector V2. This adjustment is to reduce errors in the calculated 3D position C=(cx,cy,cz) of the camera 101a in space 500, which may be due to errors in the rotation angle θ around the Z axis indicated by the camera orientation matrix Rz(θ).

[0032] For example, as shown in Figure 9(a), if the positional relationship between camera 101a and markers 102a and 102b on the XY plane does not match the XY plane vectors V1 and V2, the 3D position acquisition unit 236 adjusts the XY plane vectors V1 and V2.

[0033] For example, the 3D position acquisition unit 236 attempts to match the positional relationship between camera 101a and markers 102a and 102b with the XY plane vectors V1 and V2 by adjusting the rotation angle θ around the Z axis indicated by the camera orientation matrix Rz(θ).

[0034] Furthermore, in Figure 9(a), the 3D position acquisition unit 236 considers that the XY plane vector V1 is longer than the XY plane vector V2, and therefore assumes that the XY plane vector V1 has a larger error. As shown in Figure 9(b), the unit matches the positional relationship between camera 101a and marker 102b with the XY plane vector V2 without adjusting the length and direction of the XY plane vector V2, while adjusting the length and direction of the XY plane vector V1 to match the positional relationship between camera 101a and marker 102b with the XY plane vector V2.

[0035] Furthermore, as shown in Figure 9(c), the 3D position acquisition unit 236 adjusts the length and direction of both XY plane vectors V1 and XY plane vector V2 by matching the ratio of the adjustment amount of the length and direction of XY plane vector V1 to the ratio of the lengths of XY plane vector V1 and XY plane vector V2, thereby matching the positional relationship between camera 101a and marker 102b with XY plane vector V2.

[0036] The 3D position acquisition unit 236 uses the adjusted XY plane vectors V1 and V2 and the camera orientation matrix Rz(θ) to calculate the 3D direction vector (vx1, vy1, vz1), and further calculates the 3D position C=(cx, cy, cz) of the camera 101a in space 500, as well as the orientation of the camera 101a based on the camera orientation matrix Rz(θ) and the camera placement matrix Rc.

[0037] Furthermore, the 3D position acquisition unit 236 selects the best 3D position C=(cx,cy,cz) from among the 3D positions C=(cx,cy,cz) of the camera 101a stored in the memory 205, and sets it as the final 3D position of the camera 101a (coordinate position of the moving object). The best 3D position is, for example, the position in which the corresponding reprojection error is minimized among the 3D positions C of the camera 101a, or the position calculated by weighted averaging according to the reciprocal of the reprojection error. The 3D position acquisition unit 236 associates the obtained 3D position and orientation of the camera 101a with the ID of the marker 102 included in the image and stores it in the memory 205.

[0038] Furthermore, if initially two or more markers 102 were captured by camera 101a, but then the camera 101a moved and only one marker 102 was captured, the final three-dimensional position and orientation of camera 101a will be calculated based on the image position of that single marker 102.

[0039] Specifically, the 3D position acquisition unit 236 determines a 3D direction vector from camera 101a to marker 102 in a camera-reference relative system, which is a coordinate system in 3D space centered on camera 101a. The 3D direction vector is determined based on the image position and orientation data of marker 102. As shown in Figure 10, once the 3D direction vector V100 is determined, the position of camera 101a in 3D space (3D position) will be one of the positions on the camera position candidate line 710, which is the outer edge of a circle centered on position C10, which is on the same plane (XY plane) as camera 101a, directly below marker 102.

[0040] Furthermore, the 3D position acquisition unit 236 determines the inclination of the 3D direction vector with respect to the XY plane, and based on this inclination and the height difference between the camera 101a and the marker 102, it determines the horizontal distance (distance on the XY plane) between the camera 101a and the marker 102.

[0041] Next, the 3D position acquisition unit 236, based on the history of the 3D position and orientation of camera 101a corresponding to the ID of marker 102 stored in memory 205, determines a movement vector V110, as shown in Figure 11, starting from the 3D position of camera 101a from the measurement before last (position before last) P1 and ending at the 3D position of camera 101a from the measurement last time (previous position), and then determines an extension vector V120 by extending that movement vector V110. Furthermore, the 3D position acquisition unit 236 sets the intersection point C20 of the extension vector V120 and the camera position candidate line 710 as the current 3D position C20 of camera 101a.

[0042] Next, the 3D position acquisition unit 236 determines the change in orientation of camera 101a from the lateral movement (movement on the XY plane) of marker 102 in the image, and adds this change in orientation to the orientation from the previous measurement to obtain the current orientation of camera 101a. Furthermore, the 3D position acquisition unit 236 associates the determined 3D position and orientation of camera 101a with the ID of marker 102 included in the image and stores them in memory 205.

[0043] Next, we will explain the operation of server 200 by referring to the flowchart. Figures 12, 13, and 14 are flowcharts showing an example of the operation of server 200.

[0044] The image acquisition unit 230 within the control unit 202 acquires the image captured by the camera 101a from the image processing unit 204 (step S101). Next, the marker identification unit 232 analyzes the image and attempts to acquire the ID of the marker 102, and determines whether or not the ID was acquired (step S102). If the ID was not acquired (step S102; NO), the series of operations ends.

[0045] On the other hand, if an ID is obtained (step S102; YES), the marker identification unit 232 refers to the marker 3D position table and obtains the 3D position of the marker 102 in space 500 corresponding to the obtained ID (step S103).

[0046] The image position correction unit 234 corrects image distortion based on image distortion correction data of the camera's internal parameters (step S104). Furthermore, the image position correction unit 234 sets the image after distortion correction so that markers 102 located in invalid regions such as 603a are not used when acquiring the 3D position of the camera 101a (step S105).

[0047] Next, the 3D position acquisition unit 236 determines whether there are two or more markers 102 in the image (step S106). If there are two or more markers 102 in the image (step S106; YES), the operation proceeds to the one shown in Figure 13, and the 3D position acquisition unit 236 calculates a 3D direction vector from camera 101a to marker 102 in the camera-referenced relative frame based on the 3D position of marker 102a, camera internal parameters, and orientation data (step S201).

[0048] Next, the 3D position acquisition unit 236 calculates an XY plane vector by projecting the 3D direction vector onto the XY plane based on the height difference between the marker 102 and the camera 101a and the camera placement matrix Rc (step S202).

[0049] Next, the 3D position acquisition unit 236 selects two XY plane vectors from among two or more markers 102, with two markers 102 as their endpoints (step S203).

[0050] Next, the 3D position acquisition unit 236 adjusts the two XY plane vectors (step S204). Then, using the adjusted XY plane vectors and the camera orientation matrix Rz(θ), the 3D position acquisition unit 236 calculates the 3D position of the camera 101a in space 500, and calculates the orientation of the camera 101a based on the camera orientation matrix Rz(θ) and the camera placement matrix Rc, and stores it in memory 205 along with the ID of the marker 102 in the image (step S205).

[0051] Furthermore, the 3D position acquisition unit 236 determines whether the processing in steps S203 to S205 has been completed for all combinations of the two XY plane vectors (step S206). If it has not been completed (step S206; NO), the operation from step S203 onwards is repeated. On the other hand, if it has been completed (step S206; YES), the 3D position acquisition unit 236 selects the best 3D position and orientation of the camera 101a (step S207).

[0052] On the other hand, in step S106 of Figure 11, if there are no more than two markers 102 in the image (step S106; NO), that is, if there is only one marker 102 in the image, the operation proceeds to the one shown in Figure 14, and the 3D position acquisition unit 236 determines whether or not the ID of that single marker 102 has already been acquired during the acquisition of the 3D position and orientation of the camera 101a in the past (step S301). If the ID has not been acquired (step S301; NO), the series of operations ends.

[0053] On the other hand, if an ID has already been obtained (step S301; YES), the 3D position acquisition unit 236 determines a 3D direction vector from camera 101a to marker 102 in the camera-referenced relative system, which is a coordinate system in 3D space centered on camera 101a. Furthermore, the 3D position acquisition unit 236 determines the inclination of the 3D direction vector with respect to the XY plane, and based on this inclination and the height difference between camera 101a and marker 102, determines the horizontal distance (distance on the XY plane) between camera 101a and marker 102 (step S302).

[0054] Next, the 3D position acquisition unit 236 calculates a movement vector from the 3D position of camera 101a obtained from the measurement before last (position before last) to the 3D position of camera 101a obtained from the measurement last time (position last time), based on the history of the 3D position and orientation of camera 101a corresponding to the ID of marker 102 stored in memory 205, and calculates an extended vector by extending that movement vector (step S303).

[0055] Furthermore, the 3D position acquisition unit 236 sets the intersection point of the extension vector and the candidate camera position line as the current 3D position of the camera 101a (step S304).

[0056] Next, the 3D position acquisition unit 236 determines the change in orientation of camera 101a from the lateral movement of marker 102 in the image, and adds the amount of change in orientation to the orientation from the previous measurement to obtain the current orientation of camera 101a (step S305). Furthermore, the 3D position acquisition unit 236 associates the obtained 3D position and orientation of camera 101a with the ID of marker 102 included in the image and stores it in memory 205 (step S306).

[0057] In this embodiment, when the server 200 acquires an image obtained by the camera 101a capturing the marker 102, it corrects the position of the marker 102 by correcting the distortion of the image, and converts the image position of the marker 102 in the corrected image into the three-dimensional position of the camera 101a (forklift 100). In this embodiment, there are no restrictions on the installation position of the marker 102, and the degree of freedom in calculating the three-dimensional position and orientation of the forklift 100 can be improved.

[0058] Furthermore, in this embodiment, considering that image distortion increases with increasing distance from the center of the image, the server 200 performs image correction using image distortion correction data where the correction amount increases with increasing distance from the center of the image, and does not use the markers 102 located in invalid areas 603a at the four corners of the image when acquiring the 3D position of the camera 101a. This improves the accuracy of calculating the 3D position and orientation of the camera 101a.

[0059] Furthermore, in this embodiment, the server 200 adjusts the XY plane vectors, and if an error occurs in the three-dimensional position of the camera 101a due to an error in the rotation angle θ around the Z axis indicated by the camera orientation matrix Rz(θ), that error can be reduced.

[0060] Furthermore, in this embodiment, the best combination of the three-dimensional position and orientation of the camera 101a is selected or acquired based on the three-dimensional position and orientation of the camera 101a acquired for each combination of the two markers 102, thereby improving the accuracy of calculating the three-dimensional position and orientation of the camera 101a.

[0061] Furthermore, in this embodiment, even if only one marker 102 is included in the image, if the 3D position and orientation of camera 101a have been calculated in the past using the 3D position of that marker 102, the current 3D position and orientation of camera 101a can be calculated. This improves the versatility of calculating the 3D position and orientation of camera 101a.

[0062] Furthermore, in this embodiment, each marker 102 has an independent ID, and the number of IDs in the set can be extremely large, so that a large number of markers 102 with different IDs can be placed over a very wide area. For this reason, within space 500, regardless of the previous state, the absolute distance to markers 102 can be determined immediately upon receiving the IDs of two markers 102.

[0063] It should be noted that the present invention is not limited by the above description of embodiments and drawings, and modifications can be made to the above embodiments and drawings as appropriate.

[0064] In the embodiment described above, only camera 101a was used, but multiple cameras may be used. For example, as shown in Figure 12, if camera 101b is provided in addition to camera 101a, camera 101a will image marker 102a and camera 101b will image marker 102b, and an XY plane vector V3 will be defined in the XY plane from camera 101a to camera 101b, thereby enabling accurate calculation of the three-dimensional position of the forklift 100.

[0065] Furthermore, in the embodiment described above, the marker 102 transmitted an ID by changing its emission color over time, but it may also transmit the 3D position of the marker 102. In this case, the server 200 can detect the 3D position of the marker 102 corresponding to the emission pattern. For this reason, the marker 3D position table shown in Figure 4 is unnecessary.

[0066] Furthermore, the marker 102 is not limited to an LED. For example, a light source may be incorporated into a part of a display device such as an LCD, PDP, or EL display.

[0067] Furthermore, the server 200 can be any device that acquires images from cameras 101a and 101b.

[0068] Furthermore, in the above embodiment, the program to be executed may be stored and distributed on a computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), or MO (Magneto-Optical Disc), and the system that performs the above-mentioned processing may be configured by installing the program.

[0069] Alternatively, the program may be stored on a disk drive or similar device owned by a designated server on a network such as the Internet, and then downloaded, for example, by superimposing it onto a carrier wave.

[0070] Furthermore, if the above-mentioned functions are implemented by the OS (Operating System) or through collaboration between the OS and the application, then only the parts other than the OS may be stored and distributed on a medium, or they may be available for download.

[0071] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and the present invention includes the invention described in the claims and its equivalents. The invention described in the original claims of this application is listed below.

[0072] (Note 1) From among a plurality of light-emitting devices whose coordinate positions in three-dimensional space are known, including two light-emitting devices installed at arbitrary positions with different heights in three-dimensional space, the coordinate positions of at least two light-emitting devices in the image are obtained based on an image including at least two light-emitting devices captured by an imaging device attached to a moving body. Based on the coordinate position in the three-dimensional space and the coordinate position in the image, at least one of the coordinate position of the moving object in the three-dimensional space and the direction the moving object is facing is determined. A mobile object positioning device that performs processing.

[0073] (Note 2) Based on a plurality of images, including the aforementioned image, each with a different acquisition timing, the system identifies at least two light-emitting devices that emit light with different emission patterns. A mobile body position determination device according to claim 1, which performs processing.

[0074] (Note 3) The system acquires a plurality of three-dimensional direction vectors, starting from the focal point of the imaging device and ending at the coordinate positions of the at least two light-emitting devices in the image, and attitude data indicating the direction the imaging device is facing. Based on the acquired plurality of three-dimensional direction vectors and the height difference between the at least two light-emitting devices in the three-dimensional space, the distance in the three-dimensional space from the imaging device to the at least two light-emitting devices and the optical axis orientation of the imaging device are acquired. A mobile body position determination device as described in Appendix 1 or 2, which performs the processing.

[0075] (Note 4) Adjust the longer of the obtained distances. A mobile body position determination device as described in Appendix 3, which performs the processing.

[0076] (Note 5) Each distance is adjusted according to the ratio of the amount of deviation between the coordinate position in the three-dimensional space calculated based on the plurality of three-dimensional direction vectors and each of the acquired distances. A mobile body position determination device as described in Appendix 3, which performs the processing.

[0077] (Note 6) To determine the coordinate position of the moving object, control is performed so as not to use the coordinate position of the light-emitting device located at the outer edge of the image. A mobile body position determination device described in any of the appendices 1 to 5 that performs the processing.

[0078] (Note 7) If, after acquiring a first image showing an image including the first light-emitting device and other light-emitting devices, the acquired image is a second image that includes only the first light-emitting device and does not include any other light-emitting devices, the coordinate position of the moving body of the first light-emitting device is determined based on the calculated movement vector of the moving body of the first light-emitting device. A mobile body positioning device described in any of the appendices 1 to 6 that performs processing.

[0079] (Note 8) The coordinate position of the moving body is determined according to the images acquired from the plurality of imaging devices attached to the moving body. A mobile body positioning device described in any of the appendices 1 to 7 that performs processing.

[0080] (Note 9) A method for causing a computer to perform the processing of a mobile object position determination device as described in any of the appendices 1 to 8.

[0081] (Note 10) A program that causes a computer to perform the processing of a mobile object position determination device as described in any of the appendices 1 to 8. [Explanation of symbols]

[0082] 1…Visible light communication system, 100…Forklift, 101a, 101b…Camera, 102, 102a, 102b, 102c…Marker, 200…Server, 202…Control unit, 203…Wireless communication unit, 204…Image processing unit, 205…Memory, 206…Operation unit, 207…Display unit, 208…Communication unit, 230…Image acquisition unit, 232…Marker identification unit, 234…Image position correction unit, 236…3D position acquisition unit, 50 0...Space, 601a, 601b...Image, 603a, 603b, 603c, 603d...Invalid region, 610a, 610b...Marker image position, 700...Image region, 710...Camera position candidate line, C10...Center, C20...Camera's 3D position, P1...Previous position, P2...Previous position, V1, V2, V3...XY plane vectors, V100...3D direction vector, V110...Movement vector, V120...Extension vector

Claims

1. Among a plurality of light-emitting devices including two light-emitting devices installed at arbitrary positions having different heights in a three-dimensional space and whose coordinate positions in the three-dimensional space are known, based on an image including at least two light-emitting devices captured by an imaging device attached to a moving body, coordinate positions of the at least two light-emitting devices in the image are acquired; obtaining a plurality of three-dimensional direction vectors from the imaging device to coordinate positions in an image of the at least two light-emitting devices; acquiring a plurality of two-dimensional vectors different in dimension from the height difference in the three-dimensional space based on the acquired plurality of three-dimensional direction vectors and a height difference in the three-dimensional space between the imaging device and each of the at least two light-emitting devices; adjusting at least one of a length and a direction of at least one of the plurality of two-dimensional vectors so as to reduce an error between the imaging device and each of the at least two light-emitting devices in the two-dimensional space, and determining at least one of a coordinate position of the moving body and a direction in which the moving body is facing in the three-dimensional space based on the plurality of two-dimensional vectors whose length and / or direction have been adjusted and attitude data of the imaging device; A mobile positioning device that executes the process.

2. identifying the at least two light-emitting devices that emit light in different light-emitting patterns based on a plurality of images including the image and captured at different timings; 2. The mobile positioning device of claim 1, further comprising:

3. acquiring the plurality of three-dimensional direction vectors having a focal point of the imaging device as a starting point and a coordinate position in an image of the at least two light-emitting devices as an ending point, and the attitude data indicating a direction in which the imaging device is facing; acquiring distances in the three-dimensional space from the imaging device to the at least two light-emitting devices and an optical axis direction in which the imaging device is facing, based on the acquired three-dimensional direction vectors and a height difference in the three-dimensional space of the at least two light-emitting devices; 3. A mobile position determination device according to claim 1, which executes a process.

4. adjusting the longer of the obtained distances; 4. The mobile position determining device of claim 3, further comprising:

5. and controlling the image capturing apparatus so as not to use coordinate positions of light emitting devices present at an outer edge of the image in order to determine the coordinate position of the moving object.

5. A mobile position determining apparatus according to claim 1, which performs processing.

6. determining a coordinate position of the moving body based on a calculated movement vector of the moving body when an image acquired after acquiring a first image showing an image including the first light-emitting device and other light-emitting devices is a second image including only the first light-emitting device and not any other light-emitting device; 6. A mobile position determining apparatus according to claim 1, which performs processing.

7. determining a coordinate position of the moving object according to images acquired from the plurality of imaging devices attached to the moving object; 7. A mobile position determining device according to claim 1, which performs processing.

8. A method for causing a computer to execute the process of a mobile positioning device according to any one of claims 1 to 7.

9. A program for causing a computer to execute the processing of the mobile object position determining device according to any one of claims 1 to 7.

Images