Estimation system

Abstract

This estimation system comprises at least one blinking light emitter, an imaging unit capable of generating a captured image, and a control unit. The control unit estimates the position of the imaging unit on the basis of one or more flickers generated by one or more of the light emitters in the captured image.

Description

Estimation System

[0001] The present disclosure relates to estimation systems.

[0002] Conventionally, there are known techniques for estimating the position of a camera indoors from an image captured by the camera capturing an image of a light-emitting object. For example, Non-Patent Document 1 describes calculating the coordinates of a smartphone from an image captured by capturing an existing lighting infrastructure using light-emitting diodes (LEDs). Non-Patent Document 2 describes analyzing the position of a robot indoors from an image captured by a rolling shutter.

[0003] Md. Tanvir Hossan et al., “A Novel Indoor Mobile Localization System Based on Optical Camera Communication,” Wireless Communications and Mobile Computing, vol. 2018, January 2018 Md. Shahjalal et al., “An Implementation Approach and Performance Analysis of Image Sensor Based Multilateral Indoor Localization and Navigation System”, Wireless Communications and Mobile Computing, Volume 2018, October 22, 2018

[0004] With conventional technology, it is not possible to estimate the indoor position of a camera unless a light-emitting object is captured in a captured image.

[0005] In view of the above, an object of the present disclosure is to provide a technology that can estimate the position of a light-emitting object indoors even if the light-emitting object is not captured in a captured image.

[0006] An estimation system according to one embodiment of the present disclosure includes: at least one flickering light-emitting body; an imaging unit capable of generating an image; and a control unit that estimates the position of the imaging unit based on one or more flickers caused by the one or more light-emitting bodies in the image.

[0007] According to an embodiment of the present disclosure, it is possible to provide a technology capable of estimating the position of a light-emitting object indoors even if the light-emitting object is not captured in a captured image.

[0008] FIG. 1 is a block diagram showing an example of an estimation system according to a first embodiment of the present disclosure. FIG. 2 is a diagram showing an example of a light emission pattern. FIG. 3 is a diagram showing an example of a light emission pattern. FIG. 4 is a flowchart showing an example of a position estimation process according to the first embodiment of the present disclosure. FIG. 5 is a diagram showing an example of a captured image after n-valuing. FIG. 6 is a block diagram showing an example of an estimation system according to a second embodiment of the present disclosure. FIG. 7 is a flowchart showing an example of a position estimation process according to the second embodiment of the present disclosure. FIG. 8 is a block diagram showing an example of an estimation system according to a third embodiment of the present disclosure. FIG. 9 is a flowchart showing an example of a position estimation process according to the third embodiment of the present disclosure.

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0010] First Embodiment As shown in FIG. 1, an estimation system 1 according to a first embodiment includes light-emitting devices 10-1, 10-2, . . . , 10-N (N is an integer satisfying 1≦N) and a terminal device 20.

[0011] Hereinafter, when there is no need to particularly distinguish between the light-emitting devices 10-1 to 10-N, they will be referred to as "light-emitting devices 10." The estimation system 1 shown in Fig. 1 includes three or more light-emitting devices 10. However, the number of light-emitting devices 10 included in the estimation system 1 may be one, or two or more.

[0012] The light emitting device 10 may be a lighting device for indoor use or may be a dedicated device.

[0013] The light emitting device 10 includes a light emitter 11 and a control device 12. The light emitters 11 included in each of the light emitting devices 10-1, ..., 10-N are also referred to as light emitters 11-1, ..., 11-N, respectively.

[0014] The light emitting body 11 is disposed on the ceiling 2. The light emitting body 11 blinks based on a control signal received from the control device 12. The blinking refers to the repeating of a state in which the light emitting body 11 emits bright light and a state in which the light emitting body 11 emits dim light or a state in which the light emitting body 11 is turned off. The blinking may include flashing. The light emitting body 11 includes a light emitting diode (LED), a light bulb, a fluorescent lamp, or the like. However, the light emitting body 11 may be configured to include any element as long as it can blink based on a control signal received from the control device 12. The light emitting body 11 can become a light source that generates flicker in a captured image by blinking.

[0015] The control device 12 controls the blinking of the light emitter 11. The control device 12 includes a storage unit 13 and a control unit .

[0016] The storage unit 13 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The storage unit 13 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 13 stores data used in the operation of the control device 12 and data obtained by the operation of the control device 12. The storage unit 13 may also store a program executed by the control unit 14.

[0017] The control unit 14 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 14 controls each part of the control device 12 and executes processes related to the operation of the control device 12.

[0018] The control unit 14 causes the light emitter 11 to emit light in a preset light emission pattern by transmitting a control signal to the light emitter 11. The control unit 14 causes the light emitter 11 to emit light so that the light emission pattern is repeated.

[0019] The light emission pattern may be set so as to cause flicker in the captured image. The light emission pattern includes blinking. The light emission pattern is configured to include, for example, a period in which the light emitter 11 emits light brightly and a period in which the light emitter 11 emits light dimly or is turned off. The light emitter 11 blinks according to the light emission pattern. The light emitter 11 emits light by repeating the light emission pattern. Different light emitters 11 blink according to different light emission patterns.

[0020] In this embodiment, the light emission pattern includes position information of the light emitter 11. The position information of the light emitter 11 in the light emission pattern may be given by a bit. In the light emission pattern, the bit may be given by setting a period in which the light emitter 11 emits light brightly to "1" and a period in which the light emitter 11 emits light dimly or a period in which the light emitter 11 is turned off to "0". The position information of the light emitter 11 may be coordinates such as two-dimensional coordinates or three-dimensional coordinates, or may be any information capable of identifying the position of the light emitter 11. The information capable of identifying the position of the light emitter 11 is, for example, first identification information that identifies an indoor conference room or second identification information that identifies an arbitrary block divided into an indoor space.

[0021] A pilot signal may be included in one or more initial periods of the light emission pattern. The initial periods of the light emission pattern may constitute a pilot signal. In this case, the pilot signal may be a blinking pattern as described below with reference to FIG. 5. By including a pilot signal in the light emission pattern, it is possible to easily identify the position information of the light emitter 11 following the pilot signal in the light emission pattern.

[0022] For example, FIG. 2A shows a light emission pattern 3-1 of the light emitter 11-1. FIG. 2B shows a light emission pattern 3-2 of the light emitter 11-2. The light emission patterns 3-1 and 3-2 each include seven periods in which the light emitter 11 emits light brightly or is turned off. However, a light emission pattern may include any number of periods. In FIGS. 2A and 2B, periods shown in white are periods in which the light emitter 11 emits light brightly. Periods shown in hatching are periods in which the light emitter 11 is turned off.

[0023] The terminal device 20 may be a general-purpose device such as a smartphone or a mobile phone, or may be a dedicated device.

[0024] The terminal device 20 includes an imaging unit 21, an input unit 22, an output unit 23, a storage unit 24, and a control unit 25. Each of these elements may be built into the terminal device 20 or may be externally attached to the terminal device 20.

[0025] The imaging unit 21 includes at least one camera. If the terminal device 20 is a smartphone, the camera may be an internal camera, an external camera, or an external camera. The imaging unit 21 is capable of performing a rolling shutter.

[0026] The input unit 22 can accept input from a user. The input unit 22 includes at least one input interface that can accept input from a user. The input interface is, for example, a physical key, a capacitance key, a pointing device, a touch screen that is integrated with the display of the output unit 23, a microphone, or the like.

[0027] The output unit 23 is capable of outputting data. The output unit 23 includes at least one output interface capable of outputting data. The output interface is, for example, a display or a speaker.

[0028] The storage unit 24 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The storage unit 24 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 24 stores data used in the operation of the terminal device 20 and data obtained by the operation of the terminal device 20. The storage unit 24 may also store a program executed by the control unit 25.

[0029] The control unit 25 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is, for example, a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA or ASIC. The control unit 25 executes processes related to the operation of the terminal device 20 while controlling each part of the terminal device 20.

[0030] FIG. 3 is a flowchart illustrating an example of a position estimation process according to the first embodiment of the present disclosure.

[0031] In the processing of step S1, the control unit 25 receives an input from the user instructing the execution of a position estimation process via the input unit 22. For example, when the user moves indoors while carrying the terminal device 20, the user inputs this input into the input unit 22. After inputting this input into the input unit 22, the user points the imaging unit 21 of the terminal device 20 toward the light-emitting body 11.

[0032] In the processing of step S2, the control unit 25 sets parameters of the imaging unit 21. These parameters are parameters for adjusting various functions of the imaging unit 21. The parameters of the imaging unit 21 include, for example, the shutter speed of the rolling shutter of the imaging unit 21. However, the parameters of the imaging unit 21 may be any parameters as long as they are parameters for adjusting various functions of the imaging unit 21. The parameters of the imaging unit 21 may include, for example, at least one of a gain for adjusting the brightness of the camera of the imaging unit 21, the shutter speed of the rolling shutter of the imaging unit 21, and the aperture value of the camera of the imaging unit 21. In the first processing of step S2, the control unit 25 sets the parameters of the imaging unit 21 to initial values. The initial values ​​of the parameters of the imaging unit 21 may be set depending on the application of the estimation system 1.

[0033] In the process of step S3, the control unit 25 causes the imaging unit 21 to generate a captured image based on the parameters of the imaging unit 21 set in the process of step S2.

[0034] In step S4, the control unit 25 performs image processing on the captured image generated in step S3. This image processing may be any processing as long as it uses an algorithm that can detect flicker. This image processing may be, for example, grayscaling.

[0035] In the process of step S5, the control unit 25 determines whether or not flicker is detected from the captured image after the image processing of step S4.

[0036] If the control unit 25 determines that flicker is not detected (step S5: NO), the control unit 25 returns to the processing of step S2. In the processing of step S2 from the second time onwards, the control unit 25 changes the parameters of the image capturing unit 21. The parameters of the image capturing unit 21 at which flicker is detected may vary depending on the environment. Therefore, by changing the parameters of the image capturing unit 21 in the processing of step S2 from the second time onwards, and then generating a captured image in the processing of step S3 based on the changed parameters, flicker will be detected in the processing of step S5. The degree to which the parameters of the image capturing unit 21 are changed in the processing of step S2 from the second time onwards may be set according to the application of the estimation system 1.

[0037] If the control unit 25 determines that flicker is detected (step S5: YES), the process proceeds to step S6.

[0038] In the process of step S6, the control unit 25 detects the number of flickers. Here, the light emitter 11, which serves as the light source of the flickers, emits light in a predetermined light emission pattern repeatedly. Therefore, the flickers appear as a striped pattern of blinking lights. Therefore, the control unit 25 detects each striped pattern of blinking lights as one flicker.

[0039] As an example of the processing of step S6, the control unit 25 first sets multiple thresholds and converts the captured image generated in the processing of step S3 into n-values ​​(n is an integer satisfying n≧2). Here, if the number of light-emitting bodies 11 serving as the light source of flicker in the captured image is one, binarizing the captured image allows the number of flickers to be detected as one. However, in this embodiment, multiple light-emitting bodies 11 are likely to affect the occurrence of flicker in the captured image. Therefore, when multiple flickers occur in the captured image, binarizing the captured image may not be able to detect the number of flickers. Therefore, the control unit 25 converts the captured image into n-values ​​(n is an integer satisfying n≧3). For example, the control unit 25 converts the captured image into n-values ​​and obtains the n-value-converted captured image 4 shown in FIG. 4 . The control unit 25 detects the number of flickers from the n-value-converted captured image. The n-value may be set depending on the application of the estimation system 1. The number of detected flickers corresponds to the number of light sources of flickers in the captured image, that is, the number of light-emitting bodies 11 that are affecting the occurrence of flickers in the captured image.

[0040] In the process of step S7, the control unit 25 identifies the light emission pattern of the light emitter 11 that serves as the light source of the flicker, based on at least one of the captured image after n-value conversion in the process of step S6 and the captured image generated in step S3. If the number of flickers detected in the process of step S6 is multiple, the control unit 25 identifies the light emission patterns of the multiple light emitters 11 that serve as the light sources of each of the multiple flickers.

[0041] In the process of step S8, the control unit 25 estimates the position of the imaging unit 21. An example of the process of step S8 will be described below.

[0042] First, when the control unit 25 identifies the light emission patterns of the multiple light emitters 11 in the processing of step S7, it acquires position information for each of the multiple light emitters 11 from each of the multiple light emission patterns. If the position information for the light emitters 11 is coordinates, the control unit 25 estimates the distance from the image capture unit 21 to each of the multiple light emitters 11 that serve as the light sources of the multiple flickers, based on the light intensity of each of the multiple flickers in the captured image. Here, as shown in FIG. 1 , light emitted from the light emitters 11 arranged on the ceiling 2 spreads radially from the light emitter 11. Therefore, the closer the light emitter 11 is to the image capture unit 21, the stronger the light intensity of the flicker caused by that light emitter 11 as a light source. For example, in FIG. 1 , the image capture unit 21 is located closer to the light emitter 11-2 than to the light emitter 11-1. Therefore, the light intensity of the flicker caused by the light emitter 11-2 as a light source in the captured image is stronger than the light intensity of the flicker caused by the light emitter 11-1 as a light source. With this configuration, the control unit 25 can estimate the distances to the plurality of light-emitting bodies 11 that serve as the light sources of the plurality of flickers, based on the light intensities of the plurality of flickers. The distances to the plurality of light-emitting bodies 11 may be relative distances or absolute distances.

[0043] Next, the control unit 25 estimates the position of the imaging unit 21 based on the distance from the imaging unit 21 to each of the plurality of light-emitting bodies 11 and the position information of each of the plurality of light-emitting bodies 11. The control unit 25 may estimate the two-dimensional coordinates of the imaging unit 21 in a two-dimensional plane parallel to the ceiling 2 as the position of the imaging unit 21. Regardless of whether the position information of the light-emitting bodies 11 is two-dimensional coordinates or three-dimensional coordinates, the control unit 25 may estimate the two-dimensional coordinates of the imaging unit 21 in the two-dimensional plane parallel to the ceiling 2. When the two-dimensional coordinates of the imaging unit 21 have been estimated, the control unit 25 may estimate the three-dimensional coordinates of the imaging unit 21 based on the estimated two-dimensional coordinates of the imaging unit 21 and the height of the terminal device 20 from the ground. The height of the terminal device 20 from the ground may be set in advance based on the assumed height from the ground to the terminal device 20 when the user is standing while holding the terminal device 20. The height of the terminal device 20 from the ground is, for example, 1 m. However, when the position information of the light-emitting body 11 is three-dimensional coordinates, the control unit 25 may estimate the three-dimensional coordinates of the imaging unit 21 .

[0044] The processing of step S8 is not limited to the above-described processing. As another example of the processing of step S8, the control unit 25 may acquire first identification information or second identification information as position information of the light-emitting body 11 from the light emission pattern of the light-emitting body 11 identified in the processing of step S7. In this case, the control unit 25 estimates the position of the imaging unit 21 by using the acquired position information of the light-emitting body 11 as position information of the imaging unit 21. In this example, the control unit 25 can estimate the position of the imaging unit 21 even when only one flicker occurs in the captured image.

[0045] In the process of step S9, the control unit 25 causes the output unit 23 to output the position of the imaging unit 21 estimated in the process of step S8. The control unit 25 may cause the output unit 23 to display the position of the imaging unit 21 as an image or text data on the display of the output unit 23, or may cause the output unit 23 to output the position of the imaging unit 21 as sound from the speaker of the output unit 23. However, depending on the specifications of the terminal device 20, the control unit 25 may cause the output unit 23 to output the position of the imaging unit 21 as the position of the terminal device 20 or the position of the user.

[0046] As described above, the estimation system 1 according to the first embodiment includes at least one blinking light-emitting body 11, an imaging unit 21 capable of generating a captured image, and a control unit 25. The control unit 25 estimates the position of the imaging unit 21 based on one or more flickers caused by one or more light-emitting bodies 11 in the captured image. By estimating the position of the imaging unit 21 based on the flickers caused by the light-emitting body 11 in the captured image in this manner, the position of the imaging unit 21 can be estimated even if the light-emitting body 11 does not appear in the captured image. Therefore, according to this embodiment, it is possible to provide a technology that can estimate the indoor position of the light-emitting body 11 even if it does not appear in the captured image.

[0047] Furthermore, in the estimation system 1 according to the present embodiment, the position of the imaging unit 21 can be estimated even if the light-emitting body 11 is not captured in the captured image, and therefore the light-emitting body 11 may have any shape or size. With this configuration, the estimation system 1 can reduce the cost of the light-emitting body 11.

[0048] Second Embodiment As shown in FIG. 5, an estimation system 101 according to a second embodiment of the present disclosure includes light emitting devices 10-1, 10-2, . . . , 10-N (N is an integer satisfying 1≦N) and a terminal device 120.

[0049] The terminal device 120 includes an imaging unit 21, an input unit 22, an output unit 23, a storage unit 124, and a control unit 25. The storage unit 124 may be built into the terminal device 120 or may be externally attached to the terminal device 120.

[0050] The storage unit 124 is configured to include at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The storage unit 124 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 124 stores data used in the operation of the terminal device 120 and data obtained by the operation of the terminal device 120. The storage unit 124 may also store a program executed by the control unit 25.

[0051] The storage unit 124 includes a database 124DB. The database 124DB associates the light emission patterns of the multiple light emitters 11 with the position information of the multiple light emitters 11. In this embodiment, the light emission pattern is composed of a pilot signal pattern and an identifier pattern of the light emitter 11. In the light emission pattern, the pilot signal pattern is followed by the identifier pattern of the light emitter. In the database 124DB shown in FIG. 5 , a period during which the light emitter 11 emits bright light is recorded as "1." Furthermore, a period during which the light emitter 11 emits dim light or a period during which the light emitter 11 is turned off is recorded as "0." For example, if the pilot signal is "1010" and the identifier of the light emitter 11 is "101," the light emission pattern of the light emitter 11 is "1010101." Furthermore, if the pilot signal is "1010" and the identifier of the light emitter 11 is "110," the light emission pattern of the light emitter 11 is "1010110." Furthermore, if the pilot signal is "1010" and the identifier of the light emitter 11 is "110," the light emission pattern of the light emitter 11 is "1010110." In Figure 5, the three-dimensional coordinates of the light-emitting body 11 are used as the position information of the light-emitting body 11. For example, the coordinates of the light-emitting body 11 whose identifier is "101" are three-dimensional coordinates (X1, Y1, Z1). Furthermore, the coordinates of the light-emitting body 11 whose identifier is "110" are three-dimensional coordinates (X2, Y2, Z2). However, the two-dimensional coordinates of the light-emitting body 11 may be used as the position information of the light-emitting body 11, or any information capable of identifying the position of the light-emitting body 11 may be used. The information capable of identifying the position of the light-emitting body 11 is, for example, first identification information or second identification information.

[0052] FIG. 6 is a flowchart illustrating an example of a position estimation process according to the second embodiment of the present disclosure.

[0053] The control unit 25 executes the processes of steps S11 to S17 that are the same as or similar to the processes of steps S1 to S7 shown in FIG.

[0054] In the processing of step S18, the control unit 25 searches the database 124DB of the storage unit 124, and acquires the position information of the light-emitting body 11 associated with the light-emitting pattern identified in the processing of step S17. In the configuration shown in Fig. 5, the control unit 25 acquires the three-dimensional coordinates of the light-emitting body 11 as the position information of the light-emitting body 11. The control unit 25 may use a pattern indicating an identifier from the identified light-emitting patterns to search the database 124DB, or may use the entire identified light-emitting pattern to search the database 124DB. When the control unit 25 has identified the light-emitting patterns of multiple light-emitting bodies 11 in the processing of step S7, it acquires the position information of each of the multiple light-emitting bodies 11 from the database 124DB.

[0055] In the process of step S19, the control unit 25 estimates the position of the imaging unit 21. The control unit 25 estimates the position of the imaging unit 21 based on the light intensities of the multiple flickers in the captured image and the acquired position information of the multiple light-emitting bodies 11. As an example of this process, first, as described above in the first embodiment, the control unit 25 estimates the distance from the imaging unit 21 to each of the multiple light-emitting bodies 11 that serve as the light source of each of the multiple flickers, based on the light intensities of the multiple flickers in the captured image. Next, as described above in the first embodiment, the control unit 25 estimates the position of the imaging unit 21 based on the distance from the imaging unit 21 to each of the multiple light-emitting bodies 11 and the position information of each of the multiple light-emitting bodies 11.

[0056] The control unit 25 executes the process of step S20 in a manner that is the same as or similar to the process of step S9 shown in FIG.

[0057] Here, if the database 124DB includes first identification information or second identification information as position information of the light-emitting body 11, in the process of step S18, the control unit 25 may acquire the first identification information or the second identification information of the light-emitting body 11 associated with the light emission pattern identified in the process of step S17. In this case, in the process of step S19, the control unit 25 estimates the position of the imaging unit 21 by using the acquired first identification information or second identification information as position information of the imaging unit 21. In this example, the control unit 25 can estimate the position of the imaging unit 21 even if only one flicker occurs in the captured image.

[0058] Furthermore, in the second embodiment, if the estimation system 101 includes the database 124DB, the database 124DB does not need to be stored in the storage unit 124. As another example, the database 124DB may be included in a cloud server. In this case, the terminal device 120 may further include a communication unit capable of communicating with the cloud server. In the processing of step S18, the control unit 25 may search the database 124DB by communicating with the cloud server via the communication unit, and acquire the position information of the light-emitting body 11 associated with the light emission pattern identified in the processing of step S17.

[0059] Other effects and configurations of the estimation system 101 according to the second embodiment are the same as or similar to those of the estimation system 1 according to the first embodiment.

[0060] Third Embodiment As shown in FIG. 7, an estimation system 201 according to a third embodiment includes light emitting devices 10-1, 10-2, . . . , 10-N (N is an integer satisfying 1≦N) and a terminal device 220.

[0061] The terminal device 220 includes an imaging unit 21, an input unit 22, an output unit 23, a storage unit 124, a control unit 25, and a sensor unit 26. The sensor unit 26 may be built into the terminal device 220 or may be externally attached to the terminal device 220.

[0062] The sensor unit 26 is capable of detecting at least one of the state information of the terminal device 220 and the environmental information around the terminal device 220 .

[0063] The state information of the terminal device 220 may be, for example, information about the tilt of the terminal device 220. In this case, the sensor unit 26 is configured to include at least one of a gyro sensor and an acceleration sensor.

[0064] The environmental information surrounding the terminal device 220 may be, for example, information about electromagnetic waves around the terminal device 220. The information about electromagnetic waves around the terminal device 220 may be information about electromagnetic waves used in short-range wireless communication. Short-range wireless communication can also be used indoors. That is, even indoors, the terminal device 220 can receive electromagnetic waves for short-range wireless communication, unlike electromagnetic waves for long-range wireless communication. The information about electromagnetic waves around the terminal device 220 may be information about at least one of Wi-Fi (registered trademark) and Bluetooth (registered trademark). The electromagnetic wave information may be information that can be used to perform self-location estimation. The Wi-Fi (registered trademark) information includes, for example, information about Wi-Fi access points and information about the electromagnetic wave intensity of Wi-Fi (registered trademark). The sensor unit 26 is configured to include a communication module compatible with a short-range wireless communication standard.

[0065] FIG. 8 is a flowchart showing an example of a position estimation process according to the third embodiment of the present disclosure.

[0066] The control unit 25 executes the processes of steps S21 to S29 in the same manner as or similar to the processes of steps S11 to S19 shown in FIG.

[0067] In the processing of step S30, the control unit 25 acquires, from the sensor unit 26, at least one of state information of the terminal device 220 detected by the sensor unit 26 and environmental information around the terminal device 220. The control unit 25 corrects the position of the imaging unit 21 estimated in the processing of step S29 based on at least one of the state information of the terminal device 220 and the environmental information around the terminal device 220.

[0068] As an example of the processing of step S30, if the status information of the terminal device 220 includes information about the tilt of the terminal device 220, the control unit 25 corrects the light intensity of each of the multiple flickers in the captured image detected in the processing of step S29. If the terminal device 220 is tilted, it is highly likely that the imaging unit 21 is also tilted. If the imaging unit 21 is tilted, the light intensity of the flickers in the captured image may differ from that when the imaging unit 21 is not tilted. Therefore, based on the detection result of the sensor unit 26, the control unit 25 corrects the light intensity of the multiple flickers in the captured image to the light intensity when the imaging unit 21 is not tilted. Furthermore, in the same or similar manner as in step S19, the control unit 25 estimates the position of the imaging unit 21 based on the light intensities of the multiple flickers after correction and the acquired position information of the multiple light-emitting bodies 11. Here, the correction of the light intensity of the flickers can be considered as correction of the position of the imaging unit 21, since the position of the imaging unit 21 is estimated based on the light intensity after correction of the flickers.

[0069] As another example of the process of step S30, if the environmental information surrounding the terminal device 220 includes information about electromagnetic waves surrounding the terminal device 220, the control unit 25 performs self-location estimation using information about electromagnetic waves such as Wi-Fi (registered trademark). The control unit 25 corrects the position of the imaging unit 21 estimated in the process of step S29 based on the result of the self-location estimation. As an example of this process, the control unit 25 corrects the position of the imaging unit 21 by calculating the average of the coordinates acquired by performing the self-location estimation and the coordinates of the imaging unit 21 estimated in the process of step S29. In other words, the control unit 25 sets the average of these two coordinates as the corrected position of the imaging unit 21. For example, suppose that the self-location estimation estimates the position of the imaging unit 21 to be three-dimensional coordinates (1,0,0), and the process of step S29 estimates the position of the imaging unit 21 to be three-dimensional coordinates (0,0,0). In this case, the control unit 25 sets the average of these two three-dimensional coordinates, that is, coordinates (0.5,0,0), as the corrected position of the imaging unit 21.

[0070] Here, when the sensor unit 26 is capable of detecting both the state information of the terminal device 220 and the environmental information surrounding the terminal device 220, the control unit 25 may correct the position of the imaging unit 21 based on a combination of the state information of the terminal device 20 and the environmental information surrounding the terminal device 20. As an example, the control unit 25 may correct the position of the imaging unit 21 by combining information on the tilt of the terminal device 220 and information on electromagnetic waves such as Wi-Fi (registered trademark). In this case, the control unit 25 corrects the light intensity of flicker based on the information on the tilt of the terminal device 220, and estimates the position of the imaging unit 21 based on the light intensity after the flicker correction. Furthermore, the control unit 25 performs self-location estimation using information on electromagnetic waves such as Wi-Fi (registered trademark), and corrects the position of the imaging unit 21 based on the result of the self-location estimation.

[0071] In the process of step S31, the control unit 25 causes the output unit 23 to output the position of the imaging unit 21 after the correction in the process of step S30 in the same or similar manner as in the process of step S20.

[0072] As described above, in the third embodiment, the terminal device 220 further includes a sensor unit 26 that can detect at least one of status information of the terminal device 220 and environmental information around the terminal device 220. Furthermore, the control unit 25 corrects the estimated position of the imaging unit 21 based on the detection result of the sensor unit 26. With this configuration, the position of the imaging unit 21 can be calculated with higher accuracy.

[0073] Other effects and configurations of the estimation system 201 according to the third embodiment are the same as or similar to those of the estimation system 1 according to the first embodiment or the estimation system 101 according to the second embodiment.

[0074] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagram may be integrated, or one block may be divided. Two or more steps shown in the flowchart may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure.

[0075] The terminal device of the present disclosure or the control unit provided in the terminal device can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network.

[0076] For example, an embodiment is also possible in which a general-purpose computer functions as the terminal devices 20, 120, and 220 or the control unit 25 according to the above-described embodiments. Specifically, a program describing the processing content for realizing each function of the terminal devices 20, 120, and 220 according to the above-described embodiments is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor or a non-transitory computer-readable medium storing the program.

[0077] 1, 101, 102: Estimation system, 2: Ceiling, 3-1, 3-2: Light-emitting pattern, 10: Light-emitting device, 11: Light-emitting body, 12: Control device, 13: Memory unit, 14: Control unit, 20, 120, 220: Terminal device, 21: Imaging unit, 22: Input unit, 23: Output unit, 24, 124: Memory unit, 25: Control unit, 26: Sensor unit

Claims

1. An estimation system comprising: at least one flickering light-emitting body; an imaging unit capable of generating an image; and a control unit that estimates the position of the imaging unit based on one or more flickers caused by the one or more light-emitting bodies in the image.

2. The estimation system according to claim 1, wherein the light emitter blinks according to a light emission pattern including position information of the light emitter.

3. The estimation system according to claim 1, further comprising a database associating each of the light emission patterns of the plurality of light emitters with each of the positional information of the plurality of light emitters, and the control unit identifies the light emission pattern of each of the plurality of light emitters that serve as the light source of each of the plurality of flickers, searches the database, and acquires the positional information of each of the plurality of light emitters associated with the identified light emission pattern of each of the plurality of light emitters, and estimates the position of the imaging unit based on the light intensities of the plurality of flickers and the acquired positional information of the plurality of light emitters.

4. The estimation system according to any one of claims 1 to 3, wherein the estimation system includes a terminal device equipped with the imaging unit, the terminal device having a sensor unit capable of detecting at least one of status information of the terminal device and environmental information surrounding the terminal device, and the control unit corrects the estimated position of the imaging unit based on at least one of the status information of the terminal device and the environmental information surrounding the terminal device.

Images