Position Information Acquisition Device, Position Information Acquisition Method, and Recording Medium

Abstract

For each pair of cameras, the server (200) obtains reliability information related to the calculation of the installation position of the second marker (102a) etc. that is captured by both cameras (201a) etc. included in the pair of cameras. Further, the server (200) calculates the installation position of the second marker (102a) etc. based on the images captured by the pair of cameras. At this time, when the second marker (102a) etc. can be captured by multiple pairs of cameras and the installation position of the second marker (102a) etc. can be calculated for each pair of cameras, the server (200) selects the pair of cameras with the highest reliability information for the second marker (102a) etc., and calculates the installation position of the second marker (102a) etc. based on the images captured by the pair of cameras.

Description

[0001] This application is a divisional application of the patent application with the application number 201911151905.8, the application date of November 21, 2019, and the title "Position Information Acquisition Device, Position Information Acquisition Method, Recording Medium, and Position Information Acquisition System".

[0002] Regarding this application, a priority is claimed based on Japanese Patent Application No. 2018-224457 filed on November 30, 2018 and Japanese Patent Application No. 2019-105599 filed on June 5, 2019, and the contents of the basic applications are all incorporated into this application. Technical Field

[0003] The present invention relates to a position information acquisition device, a position information acquisition method, a recording medium, and a position information acquisition system. Background Art

[0004] As described in International Publication No. 2005 / 124687, a technique for determining the three-dimensional position of a marker by imaging the marker with multiple cameras is known.

[0005] However, in the technique of the above patent document, it is susceptible to the optical characteristics of the camera, small deviations in the position of the marker, etc., and thus there may be a deviation in the reliability of the determination of the three-dimensional position. Summary of the Invention

[0006] The present invention of this application is proposed in view of such problems, and its object is to provide a solution for generating a deviation in the position of a position information acquisition object.

[0007] The position information acquisition device according to the present invention acquires the position information of a position acquisition object provided in a given space. The position information acquisition device includes a processor that executes: detecting light based on identification information that is commonly included in a plurality of captured images obtained by imaging the given space from different shooting directions, and obtaining, based on the detected positions of the light in the respective captured images and the position information of the imaging device at the time of imaging each of the plurality of images, the three-dimensional position of the position information acquisition object determined by the identification information in the given space, and obtaining reliability information of the three-dimensional position of the acquired position information acquisition object based on information related to the respective imaging conditions of the imaging device when imaging the plurality of captured images, and includes: a memory that stores the acquired reliability information.

[0008] The position information acquisition method according to the present invention acquires the position information of a position acquisition object provided in a given space, and the position information acquisition method includes: a detection step of detecting light based on identification information, the identification information being commonly included in a plurality of captured images obtained by capturing the given space from various different shooting directions; an acquisition step of acquiring the three-dimensional position of the position acquisition object determined by the identification information in the given space according to the respective detection positions of the light detected in the plurality of captured images in the detection step and the position information of the imaging device when capturing the plurality of images; an acquisition step of acquiring reliability information of the three-dimensional position of the acquired position acquisition object based on information related to the respective imaging conditions of the imaging device when capturing the plurality of captured images; and a storage step of storing the acquired reliability information in a given memory.

[0009] A recording medium according to the present invention is a computer-readable recording medium included in a position information acquisition device that acquires the position information of a position acquisition object provided in a given space, and the recording medium records a program that causes the computer to function as the following units: a detection unit that detects light based on identification information, the identification information being commonly included in a plurality of captured images obtained by capturing the given space from various different shooting directions; an acquisition unit that acquires the three-dimensional position of the position acquisition object determined by the identification information in the given space according to the respective detection positions of the light detected by the detection unit in the plurality of captured images and the position information of the imaging device when capturing the plurality of images; an acquisition unit that acquires reliability information of the three-dimensional position of the acquired position acquisition object based on information related to the respective imaging conditions of the imaging device when capturing the plurality of captured images; and a storage unit that stores the acquired reliability information in a given memory.

[0010] The position information acquisition system according to the present invention includes: an imaging device that images a given space from various different imaging directions; and a position information acquisition device that acquires position information of a position acquisition object provided in the given space. The position information acquisition system is characterized in that the position information acquisition device includes a processor, and the processor executes: detecting light based on identification information that is commonly included in a plurality of captured images captured by the imaging device, and acquiring, based on the detection positions of the light in each of the plurality of captured images and the position information of each of the plurality of captured images of the imaging device at the time of imaging, the three-dimensional position of the position acquisition object determined by the identification information in the given space, and acquiring reliability information of the three-dimensional position of the acquired position acquisition object based on information related to each imaging condition when imaging the plurality of captured images, and including: a memory that stores the acquired reliability information.

[0011] Effects of the Invention

[0012] According to the present invention, a solution for generating a deviation in the position of a position acquisition object can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. is a diagram showing an example of a visible light communication system according to an embodiment of the present invention.

[0014] Figure 2 FIG. is a diagram showing an example of the structure of a server according to this embodiment.

[0015] Figure 3 FIG. is a diagram showing an example of parallax obtained from images captured by two cameras according to this embodiment.

[0016] Figure 4 FIG. is a diagram showing an example of calculation of the installation positions and imaging directions of two cameras according to this embodiment.

[0017] Figure 5 FIG. is a diagram showing an example of speed calculation according to this embodiment.

[0018] Figure 6 FIG. is a diagram showing an example of spatial segmentation according to this embodiment.

[0019] Figure 7 FIG. is a flowchart showing an example of reliability information acquisition performed by the server according to this embodiment.

[0020] Figure 8 FIG. is a flowchart showing an example of a process for acquiring the installation position of a second marker performed by the server according to this embodiment.

[0021] Figure 9 It is a flowchart showing an example of generation and maintenance of a location / reliability information table performed by a server related to other embodiments.

[0022] Figure 10 It is a diagram showing an example of a location / reliability information table related to other embodiments.

[0023] Figure 11 It is a flowchart showing another example of generation and maintenance of a location / reliability information table performed by a server related to other embodiments.

[0024] Figure 12 It is a diagram showing an example of a location / reliability information table related to other embodiments.

[0025] Figure 13 It is a diagram showing an example of calculation of an error in a location related to other embodiments. Detailed Embodiments

[0026] The following describes a visible light communication system as a location information acquisition system according to an embodiment of the present invention with reference to the accompanying drawings.

[0027] Figure 1 It is a diagram showing the configuration of the visible light communication system. As Figure 1 shown, the visible light communication system 1 includes devices 100a, 100b, 100c (hereinafter appropriately referred to as "device 100" without separately defining devices 100a, 100b, 100c) provided in a space 500 and a server 200 corresponding to the location information acquisition device.

[0028] Device 100a is installed with a second marker 102a, device 100b is installed with a second marker 102b, and device 100c is installed with a second marker 102c (hereinafter, when not separately defining the second markers 102a, 102b, and 102c, it is appropriately referred to as "second marker 102"). The server 200 is installed with cameras 201a, 201b, 201c, and 201d corresponding to the imaging devices (hereinafter, when not separately defining the cameras 201a, 201b, 201c, and 201d, it is appropriately referred to as "camera 201"). In addition, first markers 300a, 300b, 300c, 300d, and 300e are provided in the space 500 (hereinafter, when not separately defining the first markers 300a, 300b, 300c, 300d, and 300e, it is appropriately referred to as "first marker 300"). The first marker 300 and the second marker 102 include LEDs (Light Emitting Diodes) not shown. The second marker 102 corresponds to the position information acquisition object.

[0029] In the present embodiment, the second marker 102 installed in the device 100 transmits information by emitting light corresponding to information of various transmission objects such as the state of the device 100. On the other hand, the server 200 demodulates the change in the emission color in the image of the light obtained by continuously imaging the camera 201 in time series to acquire the information emitted by the second marker 102.

[0030] In the present embodiment, initially, the positions and imaging directions of the cameras 201a to 201d are unknown. Therefore, before the server 200 acquires the state of the device 100 and the like, first, the server 200 calculates the positions (installation positions) and imaging directions of the cameras 201a to 201d in the three-dimensional space 500 based on the positions (two-dimensional coordinate information) of the images of the first markers 300a, 300b, 300c, 300d, and 300e in the images obtained by imaging with the cameras 201a to 201d. Furthermore, the server 200 generates a transformation matrix for transforming the positions (two-dimensional coordinate information) of the images of the first marker 300 in the image obtained by imaging into the positions (installation positions) in the space 500.

[0031] Figure 2 It is a diagram showing an example of the structure of the server 200. As Figure 2 shown, the server 200 includes a control unit 202, an image input unit 204, a memory 205, an operation unit 206, a display unit 207, and a communication unit 208. In addition, the cameras 201a to 201d are installed in the server 200 via wiring.

[0032] The camera 201a includes a lens 203a, the camera 201b includes a lens 203b, the camera 201c includes a lens 203c, and the camera 201d includes a lens 203d (hereinafter, the lenses 203a, 203b, 203c, and 203d are appropriately referred to as "lens 203" without separately defining them). The lens 203 is composed of a zoom lens or the like. The lens 203 moves by a zoom control operation from the operation unit 206 and a focus control performed by the control unit 202. The imaging angle of view and the optical image of the camera 201 imaging are controlled by the movement of the lens 203.

[0033] The cameras 201a to 201d are composed of a plurality of light receiving elements regularly arranged two-dimensionally on the light receiving surface. The light receiving elements are, for example, imaging devices such as CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor). The cameras 201a to 201d image (receive light) an optical image that enters via the lens 203 within a given range of imaging angles of view based on a control signal from the control unit 202, convert the image signal within the imaging angle of view into digital data, and generate a frame. In addition, the cameras 201a to 201d continuously image and generate frames in time, and output the continuous frames to the image input unit 204 in the server 200.

[0034] In the image input unit 204, the frame (digital data) output by the camera 201 is input based on a control signal from the control unit 202.

[0035] The control unit 202 is, for example, a processor composed of a CPU (Central Processing Unit). The control unit 202 controls various functions of the server 200 by executing software processing according to a program stored in the memory 205 (for example, a program for implementing the operation of the server 200 shown later). Figure 3 The memory 205 stores various information (programs, etc.) used in the control and the like in the server 200.

[0036] The memory 205 is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory).

[0037] The operation unit 206 is composed of a numeric keypad, function keys, etc., and is an interface for inputting the operation content of the user. The display unit 207 is composed of, for example, an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an EL (Electro Luminescence) display, etc. The display unit 207 displays an image according to the image signal output from the control unit 202. The communication unit 208 is, for example, a LAN (Local Area Network) card. The communication unit 208 communicates with an external communication device based on the control of the communication control unit 242.

[0038] The control unit 202 includes an image processing unit 231, a camera position / camera direction calculation unit 232, a matrix generation unit 234, a light emission position acquisition unit 236 corresponding to the calculation unit, a shooting condition acquisition unit 238 corresponding to the information acquisition unit, a reliability information acquisition unit 240 corresponding to the reliability information acquisition unit, and a communication control unit 242.

[0039] The image processing unit 231 performs peripheral dimming correction, distortion correction, adjusts the image quality and image size for the frames (digital data) output from each of the cameras 201 and input to the image input unit 204 so that they can be displayed as a live view image on the display unit 207. In addition, the image processing unit 231 has the following function: if a control signal based on a recording instruction operation from the operation unit 206 is input, the optical image within the shooting angle of view of the camera 201 at the time point of the recording instruction or within the display range displayed on the display unit 207 is encoded and filed in a compression encoding method such as JPEG (Joint Photographic Experts Group). The camera position / camera direction calculation unit 232 detects the position (two-dimensional coordinate information) of the image of the first marker 300 in each image obtained by shooting with the cameras 201a to 201d. Here, the setting positions (three-dimensional coordinate information) of the first markers 300a, 300b, 300c, 300d, 300e in the space 500 are set to be known. In addition, the first markers 300a, 300b, 300c, 300d, 300e emit light with a pattern that cyclically changes in three colors of R (red), G (green), and B (blue) modulated by an ID (Identification) that can uniquely identify itself.

[0040] The camera position / camera direction calculation unit 232 sets a combination of two cameras 201 (camera pair) for the cameras 201a to 201d. The number of patterns of combinations of any two cameras 201 (camera pairs) from the four cameras 201 is six (6 types).

[0041] The camera position / camera direction calculation unit 232 detects the light of the cyclic three-color pattern contained in each image obtained by photographing with the cameras 201a to 201d. Further, the camera position / camera direction calculation unit 232 detects the ID corresponding to the three-color light-emitting pattern, and then attempts to demodulate the ID. In the memory 205, the setting positions and IDs of the first markers 300a, 300b, 300c, 300d, and 300e are stored in correspondence with each other.

[0042] Further, the camera position / camera direction calculation unit 232 attempts to detect the modulated light region (a pixel region having a specific size and shape with a high luminance value equal to or higher than a preset value), which is light modulated according to the same ID, from both images obtained by photographing with the two cameras 201 included in each camera pair. Then, when the detection is possible, it is regarded that the first marker 300 corresponding to the ID can be detected. Further, the camera position / camera direction calculation unit 232 recognizes the number of detections of the first marker 300 for each camera pair.

[0043] Next, the camera position / camera direction calculation unit 232 sets, for each camera pair, an algorithm for calculating the position (setting position) and the camera direction in the space 500 of the two cameras 201 included in the camera pair corresponding to the number of the first markers 300 contained in both images photographed by the two cameras 201 included in the camera pair. The algorithms are stored in the memory 205 according to the different numbers of the first markers 300. For example, a five-point algorithm is prepared when the number of the first markers 300 contained in each image photographed by the camera pair is 5, and an eight-point algorithm is prepared when the number is 8.

[0044] Next, the camera position / camera direction calculation unit 232 calculates the setting position and the camera direction of the two cameras 201 included in the camera pair using the set algorithm for each camera pair.

[0045] The algorithm will be described below. Figure 3 This is a diagram showing an example of the parallax obtained from the image photographed by the camera 201. In addition, Figure 4 This is a diagram showing an example of the calculation of the setting position and the camera direction of the camera 201.

[0046] As Figure 3As shown, when two cameras 201 (here cameras 201a and 201b) included in the camera pair image the same first marker 300c, since the installation positions of camera 201a and camera 201b are different, there is a difference (parallax) S in the positions (two-dimensional coordinate information) of the image 251a of the first marker 300c in the image of the imaging plane 250a obtained by imaging with camera 201a and the position (two-dimensional coordinate information) of the image 251b of the first marker 300c in the image of the imaging plane 250b obtained by imaging with camera 201b.

[0047] In addition, as Figure 4 shown, the distances from the imaging planes 250a of one of the two cameras 201 (here camera 201a) included in the camera pair to the focal positions and the distances from the imaging planes 250b of the other camera 201 (here camera 201b) to the focal positions are set as F (the same value), the distance between the installation positions of camera 201a and camera 201b is set as B, and the shortest distance between the straight line connecting the focal positions of camera 201a and camera 201b and the first marker 300c is set as D. Then, when the parallax between the position of the image 251a of the first marker 300c and the position of the image 251b obtained by virtually overlapping the imaging planes 250a and 250b is set as S, the distance calculation formula D = B × F / S holds. In this formula, F and S are known fixed values.

[0048] In the present embodiment, the camera position / imaging direction calculation unit 232 obtains distance calculation formulas for each of the detected first markers 300 in addition to the distance calculation formula for the first marker 300c. Further, the camera position / imaging direction calculation unit 232 calculates the installation positions and imaging directions of the two cameras 201 included in the camera pair based on the obtained distance calculation formulas and the installation positions of the first markers 300 measured in advance.

[0049] Specifically, the camera position / imaging direction calculation unit 232 obtains the relative installation positions and imaging directions of the two cameras 201 from the combination of the position (Xga1, Yga1) of the image 251a of the first marker 300 included in the image obtained by imaging with one of the two cameras 201 included in the camera pair and the position (Xgb1, Ygb1) of the image 251b of the first marker 300 included in the image obtained by imaging with the other camera 201.

[0050] Next, the camera position / camera direction calculation unit 232 reads out the set positions of the first markers 300a to 300e stored in the memory 205 with reference to the IDs of the first markers 300a to 300e, and calculates the set positions and camera directions of the two cameras 201 included in the camera alignment in the space 500 using the read set positions. Then, the matrix generation unit 234 obtains a transformation matrix based on the set positions and camera directions of one camera 201 and the other camera 201 that are calculated, and this transformation matrix can achieve the transformation from the combination of the positions (two-dimensional coordinate information) of the images of the first marker 300 included in the images obtained by photographing with one camera 201 and the positions (two-dimensional coordinate information) of the images of the first marker 300 included in the images obtained by photographing with the other camera 201 to the set position (position information defined in three-dimensional space coordinates) of the first marker 300 in the space 500. The transformation matrix is obtained for each camera pair.

[0051] The second markers 102a, 102b, and 102c emit light in which a pattern of three colors, R (red), G (green), and B (blue), that cycles is modulated with an ID that can uniquely identify themselves.

[0052] After obtaining the transformation matrix for each camera pair, the light emission position acquisition unit 236 detects the light of the cyclic three-color pattern included in each of the images obtained by photographing with the cameras 201a to 201d. Further, the light emission position acquisition unit 236 detects the ID corresponding to the three-color light emission pattern, and then attempts to demodulate the ID. When the light emission position acquisition unit 236 can detect the same ID from both of the images obtained by photographing with the two cameras 201 included in the camera pair, it is regarded that the second marker 102 corresponding to the ID can be detected.

[0053] Next, the light emission position acquisition unit 236 obtains the position (Xga2, Yga2) of the image of the second marker 102 in the imaging surface of one of the two cameras 201 included in the camera pair and the position (Xgb2, Ygb2) of the image of the second marker 102 in the imaging surface of the other camera 201 for each camera pair. Further, the light emission position acquisition unit 236 uses the combination of the positions (Xga2, Yga2), (Xgb2, Ygb2) of both images and the transformation matrix to obtain the set position (Xk2, Yk2, Zk2) of the second marker 102 in the space 500.

[0054] Through the above processing, there is a case where the setting position is obtained for each of multiple camera pairs corresponding to one second marker 102. Corresponding to such a case, for each camera pair, reliability information (likelihood information) regarding the setting position within the space 500 of the second marker 102 is obtained. The acquisition of the reliability information is described below.

[0055] The imaging condition acquisition unit 238 sets the reliability related to the position of the image of the second marker 102 in the image obtained from the imaging surface of one of the two cameras 201 included in the camera pair to be higher the closer the position of the image of the second marker 102 is to the center of the image, and lower the farther the distance from the center. Similarly, the imaging condition acquisition unit 238 sets the reliability related to the position of the image of the second marker 102 in the image obtained from the imaging surface of the other camera 201 to be higher the closer the position of the image of the second marker 102 is to the center of the image, and lower the farther the distance from the center. This setting process is a process associated with the distortion correction process in the image processing unit 231. In this distortion correction process, the correction intensity of the distortion correction is weakened the closer to the center of the imaging surface, and strengthened as it approaches the periphery. For this reason, the more the position of the image of the second marker 102 exists in the periphery, the more the position deviation caused by the distortion correction occurs, and as a result, the reliability is lower.

[0056] Through the above processing, for one of the two cameras 201 included in the camera pair, reliability information related to the image position (image position reliability information) B1 for one second marker 102 being focused on is obtained, and for the other camera 201, reliability information related to the image position (image position reliability information) B2 for one second marker 102 being focused on is obtained.

[0057] In addition, the imaging condition acquisition unit 238 refers to multiple frames (images) obtained by continuously imaging with one of the two cameras 201 included in the camera pair, and calculates the moving speed of the second marker 102 based on the change in the position of the second marker 102.

[0058] Figure 5 It is a diagram showing an example of speed calculation. For example, among multiple frames F1, F2 to Fn continuously imaged in the time direction t, the position of the image of the second marker 102 in frame F1 is 710a, and the position of the image of the second marker 102 in frame Fn is 710n.

[0059] Next, consider the case where the distance L between the positions 710a and 710n of the images in the frame Fx where the frames F1 and Fn coincide is L. In this case, the imaging condition acquisition unit 238 can calculate the moving speed of the second marker 102 based on the result of comparing the sizes of the images of both second markers 102 with the known size of the second marker 102 pre-stored in the memory 205, the distance L, and the frame rate. Furthermore, the imaging condition acquisition unit 238 sets the reliability related to this moving speed such that the slower the moving speed (the smaller the distance L), the higher the speed reliability.

[0060] Similarly, the imaging condition acquisition unit 238 calculates the moving speed of the second marker 102 based on the distance L between the positions of the images of the second marker 102 in each image obtained by continuous imaging of the other camera 201 among the two cameras 201 included in the camera pair. Furthermore, the imaging condition acquisition unit 238 sets the reliability related to this moving speed such that the slower the moving speed (the smaller the distance L), the higher the speed reliability.

[0061] Even when the second marker 102 is fixed to the installation position for a long time, there is a possibility that its installation position may change due to minute changes in the imaging angle in the camera 201 and / or minute movement of the second marker 102. The above processing is performed for such a case. Specifically, for one of the two cameras 201 included in the camera pair, reliability information related to the moving speed (speed reliability information) C1 for the second marker 102 is obtained, and for the other camera 201, reliability information related to the moving speed (speed reliability information) C2 for the same second marker 102 is obtained.

[0062] In addition, the imaging condition acquisition unit 238 divides the space 500 into a plurality of regions. Figure 6 This is an example of space division. In Figure 6 , the image is divided into three equal parts in the vertical direction at equal intervals and divided into three equal parts in the horizontal direction at the same equal intervals, thereby forming nine regions (division regions) 501a, 501b, 501c, 501d, 501e, 501f, 501g, 501h, 501i (hereinafter, when not separately limiting the division regions 501a to 501i, it is appropriately referred to as "division region 501").

[0063] Furthermore, the imaging condition acquisition unit 238 obtains reliability information (installation position reliability information) D related to the installation position of the one second marker 102 being focused on based on the relative positional relationship between the installation positions of the two cameras 201 included in the camera pair.

[0064] Specifically, since the closer to the set positions of the two cameras 201 in the center of the camera, the larger the image of the second marker 102 can be captured, the imaging condition acquisition unit 238 sets a higher set position reliability. For example, in the case where the cameras 201a and 201d are set as shown in Figure 6 if the second marker 102 exists in the divided regions 501a, 501b, 501c, the set position reliability becomes high; if it exists in the divided regions 501d, 501e, 501f, the set position reliability becomes medium; if it exists in the divided regions 501g, 501h, 501i, the set position reliability becomes low.

[0065] In addition, the reliability information acquisition unit 240, for each pair of cameras, uses the combination of the positions (two-dimensional coordinate information) obtained by imaging with one of the two cameras 201 included in the pair of cameras and the positions (two-dimensional coordinate information) obtained by imaging with the other camera 201, and the transformation matrix corresponding to the pair of cameras to calculate the set position of the first marker 300 in the space 500 (position information defined in three-dimensional space coordinates). Further, the reliability information acquisition unit 240 calculates the error between the calculated set position of the first marker 300 in the space 500 and the set position of the first marker 300 (known information) stored in the memory 205. Further, the reliability information acquisition unit 240 sets the error reliability information A such that the smaller the error, the higher the reliability (error reliability) related to the error.

[0066] Next, the reliability information acquisition unit 240, for each pair of cameras, uses the image position reliability information B1, B2, speed reliability information C1, C2, set position reliability information D, and error reliability information A obtained through the above processing to calculate the reliability information (reliability information of the second marker 102) N related to the set position calculation for one second marker 102 being focused on. For example, it is calculated by N = A × (B1 + B2 + C1 + C2 + D).

[0067] After that, the light emission position acquisition unit 236 calculates the set position of the second marker 102 in the space 500. At this time, for one second marker 102 being focused on, there may be a situation where the set positions can be calculated from each of multiple pairs of cameras. In such a case, the light emission position acquisition unit 236 compares the reliability information N obtained for each pair of cameras for one second marker 102 being focused on. Then, the light emission position acquisition unit 236 selects the pair of cameras corresponding to the highest reliability information N.

[0068] Next, the light emission position acquisition unit 236 acquires the position (two-dimensional coordinate information) of the image of the second marker 102 in the image captured by one of the selected cameras 201 and the position (two-dimensional coordinate information) of the image of the second marker 102 in the image captured by the other camera 201. Further, the light emission position acquisition unit 236 uses the combination of the positions of these two images and the transformation matrix to calculate the installation position of the second marker 102 in the space 500 (position information defined in three-dimensional space coordinates).

[0069] Next, the operation of the server 200 will be described with reference to the flowchart. Figure 7 It is a flowchart showing an example of the acquisition of reliability information performed by the server 200. Figure 7 The operations shown are performed for each camera pair and for each first marker 300 captured by both of the two cameras included in the camera pair.

[0070] The two cameras 201 included in one camera pair capture the same first marker 300, and the first marker 300 is determined by ID acquisition (step S101).

[0071] Next, the imaging condition acquisition unit 238 acquires the image position reliability information B1 such that the closer the position of the image of the second marker 102 in the image captured by one of the two cameras 201 included in the camera pair is to the center of the captured image, the higher the image position reliability, and acquires the image position reliability information B2 such that the closer the position of the image of the second marker 102 in the image captured by the other camera 201 is to the center of the captured image, the higher the image position reliability (step S102).

[0072] Next, the imaging condition acquisition unit 238 calculates the moving speed of the second marker 102 based on each image captured continuously by one of the two cameras 201 included in the camera pair, and acquires the speed reliability information C1 such that the slower the moving speed, the higher the speed reliability. Similarly, the imaging condition acquisition unit 238 calculates the moving speed of the second marker 102 based on each image captured continuously by the other camera 201, and acquires the speed reliability information C2 such that the slower the moving speed, the higher the speed reliability (step S103).

[0073] Next, the imaging state acquisition unit 238 acquires the setting position reliability information D based on the positional relationship between the setting positions of the two cameras 201 included in the camera pair and the setting position of the second marker 102, such that the closer the setting position of the second marker 102 is to the setting positions of the two cameras 201 included in the camera pair, the higher the setting position reliability (step S104).

[0074] Next, for the image of the first marker 300 imaged by both of the two cameras 201 included in the camera pair, the reliability information acquisition unit 240 calculates the setting position of the first marker 300 in the space 500 using the combination of the position obtained by imaging with one of the cameras 201 and the position obtained by imaging with the other camera 201, and the transformation matrix corresponding to the camera pair. Further, the reliability information acquisition unit 240 calculates the error between the calculated setting position of the first marker 300 in the space 500 and the setting position (known information) of the first marker 300 stored in the memory 205, and acquires the error reliability information A such that the smaller the error, the higher the error reliability (step S105).

[0075] Further, the reliability information acquisition unit 240 uses the acquired image position reliability information B1, B2, speed reliability information C1, C2, setting position reliability information D, and error reliability information A to acquire the reliability information N of the second marker 102 (step S106).

[0076] Figure 8 It is a flowchart showing an example of the process of acquiring the setting position of the second marker 102 performed by the server 200. Multiple cameras 201 image the second marker 102 in the space 500 (step S201).

[0077] Next, for one second marker 102, when there are multiple camera pairs in which both of the two cameras 201 are imaging the second marker 102, the light emission position acquisition unit 236 selects the highest reliability information among the reliability information of the second marker 102 acquired for each of the multiple camera pairs. Further, the light emission position acquisition unit 236 selects the camera pair corresponding to the selected reliability information (step S202).

[0078] Next, the light emission position acquisition unit 236 acquires the position of the image of the second marker 102 in the image obtained by imaging with one of the cameras 201 in the selected camera pair, and the position of the image of the second marker 102 in the image obtained by imaging with the other camera 201. Further, the light emission position acquisition unit 236 calculates the setting position of the second marker 102 using the combination of the two acquired positions and the transformation matrix (step S203).

[0079] Next, the light emission position acquisition unit 236 determines whether the installation positions have been calculated for all the second markers 102 imaged in step S201 (step S204). If the installation positions have been calculated for all the second markers 102 (step S204 "Yes"), a series of operations ends. On the other hand, if there is a second marker 102 for which the installation position has not been calculated (step S204 "No"), the operations after step S202 are repeated.

[0080] In this way, in the present embodiment, the server 200 obtains reliability information related to the calculation of the installation position of the second marker 102 from the positions of the images of the second marker 102 in the images obtained by imaging with both of the two cameras 201 included in each camera pair. Further, the server 200 calculates the installation position of the second marker 102 based on the images obtained by imaging with the camera pair. At this time, when the installation position of the second marker 102 can be calculated for each camera pair by imaging the second marker 102 with a plurality of camera pairs, the server 200 selects the camera pair with the highest reliability for the second marker 102 and calculates the installation position of the second marker 102 based on the images obtained by imaging with this camera pair. Thereby, the installation position of the second marker 102 based on the imaging with the camera pair with high reliability of the second marker 102 can be calculated, and the calculation accuracy can be improved.

[0081] Specifically, the server 200 obtains image position reliability information such that the closer the image position of the second marker 102 is to the center of the image in the captured image, the higher the reliability of this image position. Thereby, the farther the image position of the second marker 102 is from the center of the image, the more the reliability of this second marker 102 can be reduced, and appropriate reliability information corresponding to the characteristic of the image with a larger distortion rate as it is farther from the center can be obtained.

[0082] In addition, the server 200 calculates the speed of the second marker 102 based on the captured image and obtains speed reliability information such that the slower the speed, the higher the reliability of this speed. Thereby, appropriate reliability information corresponding to the characteristic that the calculation accuracy of the installation position of the second marker 102 decreases as the moving speed increases can be obtained.

[0083] In addition, the server 200 obtains installation position reliability information such that the closer the installation position of the second marker 102 is to the installation positions of the two cameras 201 included in the camera pair, the higher the installation position reliability. Thereby, in general triangulation, appropriate reliability information corresponding to the characteristic that the calculation accuracy of the installation position decreases as the distance from the camera 201 increases can be obtained.

[0084] In addition, the server 200 calculates the error between the calculated setting position of the first marker 300 within the space 500 and the known information of the setting position of the first marker 300, and obtains error reliability information such that the smaller the error, the higher the error reliability. Thus, cameras with small errors, that is, high calculation accuracy, can be preferentially used in calculating the setting position of the second marker 102.

[0085] Next, other embodiments will be described. In this embodiment, the visible light communication system 1 and Figure 1 Similarly, the server 200 and Figure 2 Similarly. In this embodiment, for one marker, multiple setting positions of the first marker 300 and the second marker 102 are calculated, and reliability information regarding each setting position is set.

[0086] Figure 9 It is a flowchart showing an example of the generation and maintenance of the position / reliability information table performed by the server 200 according to other embodiments. Figure 9 The actions shown are performed for each first marker 300.

[0087] In each pair of cameras, if the two cameras 201 included in the pair of cameras capture the same first marker 300, the captured image is obtained via the image input unit 204, and the control unit 202 attempts to identify the first marker 300 by obtaining the control unit ID (step S301).

[0088] Next, the camera position / camera direction calculation unit 232 selects the pair of cameras that captured the first marker 300 and obtained the ID in step S301 (step S302).

[0089] Next, the light emission position acquisition unit 236 calculates the setting position of the first marker 300 for each pair of cameras selected in step S302 based on the captured images of the two cameras 201 included in the pair of cameras (step S303). Specifically, Figure 8 In the same manner as step S203 of , the light emission position acquisition unit 236 obtains the position of the image of the first marker 300 in the image captured by one of the cameras in the pair of cameras and the position of the image of the first marker 300 in the image captured by the other camera in the pair of cameras. Further, the light emission position acquisition unit 236 calculates the setting position of the first marker 300 using the combination of the two obtained positions and the transformation matrix corresponding to the pair of cameras.

[0090] Next, the reliability information acquisition unit 240 generates a position / reliability information table 2051 for the first marker 300 for which the setting position has been calculated in step S303, and stores it in the memory 205 (step S304).

[0091] Figure 10 FIG. is an example of a position / reliability information table 2051 generated and held in a given storage area of the memory 205 in step S304. Figure 10 The shown position / reliability information table 2051 is composed of, for each first marker 300 as a marker, the ID of the first marker 300, the set position obtained by imaging with the camera pair that images the first marker 300, the information of the camera pair for the image imaging used in the calculation of the set position, the reliability information of the set position, the update date and time indicating the date and time when the set position is calculated, and the error.

[0092] The reliability information is set in three stages A, B, and C in descending order of reliability. The reliability information acquisition unit 240 appropriately selects, for the first marker 300, Figure 7 the image position reliability information obtained in the same manner as in step S102 of this figure, the speed reliability information obtained in the same manner as in step S103 of this figure, the set position reliability information obtained in the same manner as in step S104 of this figure, the error reliability information obtained in the same manner as in step S105 of this figure, etc. to set the reliability information.

[0093] The error is set in three stages R1, R2, and R3 in ascending order of error. The reliability information acquisition unit 240 sets the error, for example, such that the closer the update date and time is to the current, the smaller it is.

[0094] Figure 11 FIG. is a flowchart showing another example of the generation and holding of the position / reliability information table performed by the server 200 according to another embodiment. Figure 11 The shown operation is performed for each second marker 102.

[0095] In each camera pair, the two cameras 201 included in the camera pair image the same second marker 102 and attempt to determine the second marker 102 by ID acquisition (step S401).

[0096] Next, the camera position / camera direction calculation unit 232 selects the camera pair that images the second marker 102 and has acquired the ID in step S401 (step S402).

[0097] Next, the light emission position acquisition unit 236 calculates the set position of the second marker 102 based on the captured images of the two cameras 201 included in each camera pair selected in step S402 (step S403). Specifically, Figure 8Similarly, in step S203, the light emission position acquisition unit 236 acquires the position of the image of the second marker 102 in the image captured by the camera of one of the cameras 201 and the position of the image of the second marker 102 in the image captured by the camera of the other camera 201. Further, the light emission position acquisition unit 236 calculates the installation position of the second marker 102 using the combination of these two image positions and the transformation matrix corresponding to the camera pair.

[0098] Next, the reliability information acquisition unit 240 generates position / reliability information for the second marker 102 whose installation position has been calculated in step 403, and adds and stores it in the position / reliability information table 2051 (step S404).

[0099] Figure 12 FIG. is an example of the position / reliability information table 2052 in step S404. Figure 12 Indicates in Figure 10 The position / reliability information table 2052 obtained by adding the position / reliability information generated for each second marker 102 in step S404 to the position / reliability information table 2051 generated for each first marker 300 as shown.

[0100] The position / reliability information table 2052 generated for each second marker 102 is the same as the position / reliability information table 2051 generated for each first marker 300, and is composed of the ID of the second marker 102, the installation position obtained in the imaging of the camera pair that images the second marker 102, the information of the camera pair that images the image used in the calculation of the installation position, the reliability information of the installation position, the update date and time indicating the date and time when the installation position is calculated, and the error.

[0101] The reliability information is set in three stages of A, B, and C in descending order of reliability. The reliability information acquisition unit 240 appropriately selects, for the second marker 102, the image position reliability information obtained in the same manner as step S102 of Figure 7 , the speed reliability information obtained in the same manner as step S103 of this figure, the installation position reliability information obtained in the same manner as step S104 of this figure, the error reliability information obtained in the same manner as step S105 of this figure, etc. to set the reliability information.

[0102] Further, the reliability information acquisition unit 240 can also acquire the installation position of the first marker 300 for the second marker 102 to obtain the installation position, reliability information, and error of the second marker 102.

[0103] The errors are set in three stages, R1, R2, and R3, in ascending order of the error. The reliability information acquisition unit 240, for example, sets the error such that the closer the update date and time is to the current, the smaller it is.

[0104] Next, the light emission position acquisition unit 236 determines whether the setting positions have been calculated for all the second markers 102 imaged in step S401 (step S405). If the setting positions have been calculated for all the second markers 102 (step S405 “Yes”), the series of operations ends. On the other hand, if there is a second marker 102 for which the setting position has not been calculated (step S405 “No”), the operations after step S402 are repeated.

[0105] In this way, by generating and maintaining the position / reliability information tables for the first marker 300 and the second marker 102, the reliability information of the calculated multiple setting positions of the first marker 300 and the second marker 102 is obtained. For this reason, for the first marker 300 and the second marker 102, the setting position with the highest reliability can be selected, or the most appropriate setting position incorporating both the reliability information and the error can be selected.

[0106] Furthermore, the determination of the setting position can be appropriately performed. For the first marker 300 and the second marker 102 for which only setting positions with low reliability are calculated, it is regarded that the setting position cannot be determined, or for the first marker 300 and the second marker 102 for which the appropriate setting position incorporating both the reliability information and the error has not been calculated, it is regarded that the setting position cannot be determined, etc.

[0107] In addition, when only the position / reliability information table 2051 is generated, the setting position, reliability information, and error of the second marker 102 can be obtained later. The following is described as a specific example: Only the position / reliability information table 2051 is generated, and after removing the first marker 300 from the space 500, the cameras 201a to 201d image the inside of the space 500. As Figure 13 shown, for the space 500, after generating and maintaining the position / reliability information table 2051, the first marker 300 is removed, and a new second marker 102c (normal operation state) is set. Then, the cameras 201a to 201d image the image of the space 500 in this state. After each captured image is input to the image input unit 204, the image processing unit 231 detects the second marker 102c from these images. Furthermore, based on the image and determinant of the second marker 102c in these captured images, the setting position of the second marker 102c is calculated. Then, referring to the position / reliability information table 2051, the first marker 300 that is imaged by a camera with high reliability and is closest to the calculated setting position of the second marker 102c ( Figure 13The information of the first markers 300d and 300e) is read out. In addition, the error E of the second marker 102c is obtained by the following method. As Figure 13 As shown in the figure, if the distance in the X direction between the first marker 300d and the second marker 102c in space is set as Xf, the distance in the X direction between the first marker 300e and the second marker 102c is set as Xg, the error between the position based on the known information and the calculated position of the first marker 300d is set as Ef, and the error between the position based on the known information and the calculated position of the first marker 300e is set as Eg, then the error E of the position of the second marker 102c is calculated by interpolation through E = (Ef * Xg + Eg * Xf) / (Xf + Xg).

[0108] In addition, the present invention is not limited by the description and the drawings of the above embodiments, and changes and the like can be appropriately added to the above embodiments and the drawings.

[0109] For example, in the above embodiment, the reliability information acquisition unit 240 uses the image position reliability information B1, B2, speed reliability information C1, C2, installation position reliability information D, and error reliability information A obtained through the above processing for each pair of cameras, and calculates the reliability information N about one second marker 102 being focused on through N = A × (B1 + B2 + C1 + C2 + D).

[0110] However, the calculation formula is not limited to this. For example, the image position reliability information B1, B2, speed reliability information C1, C2, installation position reliability information D, and error reliability information A can all be multiplied. In addition, the reliability information acquisition unit 240 can also appropriately select and discard the image position reliability information B1, B2, speed reliability information C1, C2, installation position reliability information D, and error reliability information A to calculate the reliability information N about the second marker 102. For example, when the reliability information of the highest second marker 102 is less than the threshold value, the error reliability information A may not be multiplied. Furthermore, for example, the installation position reliability information D can be calculated for the two cameras 201 included in each pair of cameras. Furthermore, the information of the pair of cameras with the most recent update date and time can be read out and adopted.

[0111] In addition, in the above embodiment, the server 200 selects the pair of cameras with the highest reliability of the second marker 102, and calculates the installation position of the second marker 102 based on the image captured by the pair of cameras. However, the method for calculating the installation position is not limited to this.

[0112] For example, the server 200 can also calculate the installation position of the second marker 102 based on the imaging of all pairs of cameras where two cameras 201 are imaging the second marker 102. For a pair of cameras with a higher reliability of the second marker 102, the weight of the installation position of the second marker 102 calculated based on the imaging of this pair of cameras is greater. In addition, the server 200 can also calculate the average value of the installation positions of the second marker 102 calculated based on the imaging of multiple pairs of cameras with a higher reliability ranking of the second marker 102.

[0113] In addition, in the above-described embodiment, as Figure 6 shown, the space 500 is divided into nine divided areas 501a to 501i. If the second marker 102 exists in the divided areas 501a, 501b, 501c, the installation position reliability information becomes high. If it exists in the divided areas 501d, 501e, 501f, the installation position reliability information becomes medium. If it exists in the divided areas 501g, 501h, 501i, the installation position reliability information becomes low. However, the setting of the divided areas 501 and the installation reliability information corresponding to each divided area 501 are not limited to this. The reliability information can also be made different for each divided area 501.

[0114] In addition, regarding the first marker 300, the installation position in the space 500 is stored in the memory 205 in correspondence with each ID. However, it is also possible to emit light modulated corresponding to the installation position in the space 500 through visible light communication.

[0115] In addition, in the above-described embodiment, the reliability information in the position / reliability information table is set in three stages, A, B, and C, in descending order of reliability. However, it is not limited to this. It can also be set in more stages, or can be set numerically. In addition, the reliability information is preferably selected from image position reliability information, speed reliability information, installation position reliability information, error reliability information, etc. for setting. However, it is not limited to this.

[0116] Furthermore, the error in the position / reliability information table is set in three stages, R1, R2, and R3, in ascending order of error. However, it is not limited to this. It can also be set in more stages, or can be set numerically. In addition, the error is set to be smaller as the update date and time is closer to the current. However, it is not limited to this.

[0117] For example, the first marker 300 and the second marker 102 are not limited to LEDs. For example, it is also possible to form part of the markers in an LCD, PDP, EL display, etc. that make up the display device.

[0118] In addition, the server 200 only needs to be equipped with a camera and can be any device.

[0119] In addition, in the above-described embodiment, the executed program can be distributed by being stored in a recording medium readable by a computer, such as a floppy disk, a CD-ROM (Compact Disc - Read Only Memory), a DVD (Digital Versatile Disc), an MO (Magneto - Optical Disc), etc., and a system for executing the above-described processing is configured by installing the program.

[0120] In addition, the program can also be stored in a disk device or the like of a given server on a network such as the Internet, and for example, downloaded by being superimposed on a carrier wave.

[0121] In addition, in the case where the above-described functions are implemented by sharing the OS (Operating System), or in the case where the above-described functions are implemented through the cooperation of the OS and an application, etc., only the part other than the OS can be stored in a medium for distribution, and in addition, downloading or the like can also be performed.

[0122] The preferred embodiments of the present invention have been described above, but the present invention is not limited to the related specific embodiments, and the present invention includes the invention described in the claims and its equivalent scope.

Claims

1. A position information acquisition device, comprising a processor that performs the following processes: An image acquisition process that acquires images of an object captured by each of a first camera, a second camera, and a third camera provided at different positions; A determination process that, based on the position of the object in the images acquired by the image acquisition process, determines, among the three cameras, the pair with the highest reliability represented by reliability information as the camera pair to be used for deriving the installation position of the object; and A derivation process that derives the installation position of the object based on the images captured by the camera pair determined by the determination process, In the image acquisition process, the processor further acquires images of markers different from the object captured by each of the first camera, the second camera, and the third camera, In the determination process, the processor compares the installation positions of the markers derived for each pair of two images out of the three marker images acquired by the image acquisition process in combination, with the installation positions of the markers stored in a memory, and determines, among the three cameras, the pair with the highest reliability represented by the reliability information as the camera pair to be used for deriving the installation position of the object.

2. The position information acquisition device according to claim 1, wherein, The reliability information is an index that is set to be higher for a camera pair in which the object is closer to the center of the viewing angle.

3. The position information acquisition device according to claim 1, wherein, The reliability information is an index that is set to be higher for a camera pair that is closer to the object.

4. A position information acquisition method, comprising the following steps: An image acquisition step that acquires images of an object captured by each of a first camera, a second camera, and a third camera provided at different positions; A determination step that, based on the position of the object in the images acquired by the image acquisition step, determines, among the three cameras, the pair with the highest reliability represented by reliability information as the camera pair to be used for deriving the installation position of the object; and A derivation step that derives the installation position of the object based on the images captured by the camera pair determined by the determination step, In the image acquisition step, images of markers different from the object captured by each of the first camera, the second camera, and the third camera are further acquired, In the determination step, the installation positions of the markers derived for each pair of two images out of the three marker images acquired by the image acquisition step in combination are compared with the installation positions of the markers stored in a memory, and the pair with the highest reliability represented by the reliability information is determined among the three cameras as the camera pair to be used for deriving the installation position of the object.

5. A non-transitory computer-readable recording medium records a program executable by a processor of a position information acquisition device, and the program causes the processor to perform the following steps: An image acquisition process that acquires images of an object captured by each of a first camera, a second camera, and a third camera disposed at different positions; A determination process that, based on the position of the object in the images acquired by the image acquisition process, determines, among the three cameras, a pair with the highest reliability represented by reliability information as the camera pair to be used for deriving the installation position of the object; and A derivation process that derives the installation position of the object based on the images captured by the camera pair determined by the determination process; In the image acquisition process, the processor further acquires images of markers different from the object captured by each of the first camera, the second camera, and the third camera; In the determination process, the processor compares the installation positions of the markers derived for each pair of two of the three images of the markers acquired by the image acquisition process in combination with the installation positions of the markers stored in the memory, and determines, among the three cameras, a pair with the highest reliability represented by the reliability information as the camera pair to be used for deriving the installation position of the object.

Images