Optical radio communication system

Abstract

To provide an optical wireless communication system for estimating a position of a signal transmission source of an optical wireless communication in an absolute coordinate system.SOLUTION: An optical wireless communication system includes: a light source 10-T that transmits an optical wireless communication signal; one or more receiving devices; and one or more information processing devices. Each of receiving devices 20a, b receives the optical wireless communication signal through an event camera, and a second information processing device 30-2 performs a light source position estimation that estimates a position of the light source in an absolute coordinate system. Each receiving device acquires a second camera estimated arrangement indicating a position and an orientation of the event camera in the absolute coordinate system when a target optical wireless communication signal is received. A target light source image position SPI-T is acquired by specifying a signal region of the target light wireless communication signal in an image plane coordinate system of an image obtained by the event camera, and a light source estimated position SPE-T which is the estimated position of the target light source in the absolute coordinate system is acquired based on a second camera estimated arrangement CP2 and the target light source image position.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to an optical wireless communication system that estimates the absolute position of a light source configured to transmit an optical wireless communication signal. [Background technology]

[0002] Patent Document 1 discloses a communication system that distributes information using visible light. In the communication system, a user acquires the distributed information by pointing a user terminal toward a visible light transmission source (light source). The transmission source also transmits light source identification information, along with the distributed information, that makes it easier to identify the position of the transmission source on the display screen of the user terminal. The user terminal displays the light source identification information on the display screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-155085 Summary of the Invention [Problem to be solved by the invention]

[0004] In optical wireless communications including visible light, the position of a light source, which is a signal transmission source, in an absolute coordinate system may be required. In the communication system described in Patent Document 1, the position of the light source is easy to identify on the display screen of the user terminal, but the position of the light source in the absolute coordinate system is not specified.

[0005] An object of the present disclosure is to provide a technique for estimating the position in an absolute coordinate system of a light source, which is a signal transmission source, in optical wireless communication. [Means for solving the problem]

[0006] The first aspect relates to an optical wireless communication system. Optical wireless communication systems include: a target light source configured to transmit a target light wireless communication signal; one or more receiving devices; One or more information processing devices Equipped with. Each of the one or more receiving devices Equipped with an event camera, The event camera is configured to receive the target optical wireless communication signal through the event camera. A second information processing device included in the one or more information processing devices is configured to perform light source position estimation to estimate the position of the target light source in the absolute coordinate system. Light source position estimation is acquiring a second camera estimated position indicating a position and orientation in an absolute coordinate system of the event camera when each receiving device receives the target optical wireless communication signal; acquiring a target light source image position, which is a position where the target light source is projected onto the image plane coordinate system, by identifying a signal region of the target optical wireless communication signal in an image plane coordinate system of an image acquired by the event camera; acquiring an estimated light source position, which is an estimated position of the target light source in the absolute coordinate system, based on the estimated second camera arrangement and the target light source image position; Includes. [Effects of the Invention]

[0007] In the optical wireless communication system, the second information processing device performs light source position estimation, thereby estimating the absolute position of the target light source. The light source position estimation is performed based on the second camera estimated position and the light source image position. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an overview of an optical wireless communication system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a schematic diagram of a first camera placement estimation. [Figure 3] FIG. 10 is a schematic diagram showing the process of acquiring the light source image position. [Figure 4] 1 is a graph showing data frequencies of an optical signal and noise; [Figure 5] FIG. 10 is a block diagram illustrating an example of light source registration position acquisition. [Figure 6] FIG. 1 is a schematic diagram of light source position estimation. [Figure 7] FIG. 10 is a block diagram showing the relationship between light source position estimation and first camera arrangement estimation. [Figure 8] FIG. 1 is a block diagram showing a first configuration example of an optical wireless communication system. [Figure 9] FIG. 10 is a block diagram showing a second configuration example of the optical wireless communication system. [Figure 10] FIG. 10 is a block diagram showing a third exemplary configuration of the optical wireless communication system. DETAILED DESCRIPTION OF THE INVENTION

[0009] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] 1.Basic configuration 1 is a diagram showing an overview of an optical wireless communication system 1 according to this embodiment. The optical wireless communication system 1 includes a first light source 10-1, one or more receiving devices 20, and an information processing device 30-1. The one or more receiving devices 20 include an event camera 21. Hereinafter, the one or more receiving devices 20 will be simply referred to as "receiving device 20." The information processing device 30-1 can also be called a first information processing device 30-1.

[0011] The first light source 10-1 is fixedly attached in space and installed indoors or outdoors. Examples of the first light source 10-1 include a visible light LED (light emitting diode) and an infrared LED. Visible light LEDs are widely used in street lights, indoor lights, traffic lights, electronic billboards, etc., so using visible light LEDs in the optical wireless communication system 1 leads to effective use of existing facilities. Visible light LEDs repeatedly blink at a speed too fast to be detected by the human eye, so by controlling the blinking, visible light can be used as a communication signal.

[0012] The first light source 10-1 transmits the first wireless optical communication signal S-1 by blinking. Hereinafter, for simplicity, the "wireless optical communication signal" will be simply referred to as the "optical signal."

[0013] The receiving device 20 receives the first optical signal S-1 through the event camera 21. Typical examples of the receiving device 20 include a smartphone, a tablet, and a wearable device for augmented reality. Furthermore, the receiving device 20 is not limited to these examples, and any object equipped with the event camera 21 can function as the receiving device 20. For example, if a vehicle, a robot, a wheelchair, a cane, or the like is equipped with the event camera 21, it can function as the receiving device 20. In this way, the receiving device 20 is typically an object or a terminal that is not fixed in space. When the receiving device 20 moves, the event camera 21 also moves along with it.

[0014] The event camera 21 has a built-in event-based vision sensor (EV sensor). The EV sensor observes changes in the luminance of light received by the pixels (image sensors) within the EV sensor. When the EV sensor observes a luminance change equal to or greater than a preset threshold, it detects the luminance change as an "event." An event is detected when a situation different from the previous situation occurs. For example, when the subject or the event camera 21 moves, the relative position between the subject and the event camera 21 changes, causing the subject to appear in pixels where it had not previously existed. At this time, a significant luminance change occurs in the pixels surrounding the subject, which is detected as an event. In addition, the first optical signal S-1 is detected as an event because it causes a luminance change associated with the blinking of the first light source 10-1.

[0015] The event camera 21 outputs data related to a pixel where an event is detected as event data EVD. The event data EVD includes at least the coordinates of the pixel where the event occurred on the image plane, the time when the event was detected, and the brightness polarity (positive / negative). The threshold value referenced by the EV sensor when detecting a brightness change is set to be changeable. When an event occurs, threshold values ​​are set on the positive side (a change toward brightness) and the negative side (a change toward darkness) of a voltage (reference voltage) based on the brightness level at that time. A voltage change exceeding the positive threshold is detected as a positive event, and a voltage change exceeding the negative threshold is detected as a negative event. In other words, the EV sensor detects the first optical signal S-1 when the first light source 10-1 turns on as a positive event, and when the first light source 10-1 turns off as a negative event.

[0016] The information processing device 30-1 acquires event data EVD from the event camera 21. The information processing device 30-1 may be included in each of the receiving devices 20, or may be an external device to the receiving devices 20.

[0017] The event data EVD acquired by the information processing device 30-1 can be used for various purposes. In the process of using the event data EVD by the information processing device 30-1, the position (absolute position) of the first light source 10-1 in the absolute coordinate system may be required. A typical example of such a case is a process in which the information processing device 30 uses the event data EVD to estimate the absolute position and orientation of the event camera 21 (hereinafter referred to as "first camera arrangement estimation"). In this specification, "arrangement" refers to a state determined by six degrees of freedom, consisting of three axial degrees of freedom and rotational degrees of freedom about each axial direction in the absolute coordinate system. An overview of first camera arrangement estimation will be described below.

[0018] 2. First camera placement estimation 2-1. Overview When the optical wireless communication system 1 includes a plurality of light sources 10-1 to 10-n (n is an integer satisfying n≧2), the information processing device 30-1 can estimate the location of the event camera 21 by performing first camera location estimation. Hereinafter, the location of the event camera 21 calculated by the first camera location estimation will be referred to as "first camera estimated location CP1." Figure 2 is a schematic diagram of the first camera location estimation.

[0019] Each of the plurality of light sources 10-1 to 10-n has the same function and configuration as the first light source 10-1 described above. The plurality of light sources 10-1 to 10-n are preferably simultaneously included within the angle of view of the event camera 21. Each light source 10-i (i = 1 to n) constituting the plurality of light sources 10-1 to 10-n transmits an optical signal Si by blinking.

[0020] The receiving device 20 receives the optical signal Si through the event camera 21. The receiving device 20 preferably simultaneously receives a plurality of optical signals S-1 to S-n transmitted from a plurality of light sources 10-1 to 10-n.

[0021] The information processing device 30-1 acquires event data EVD from the event camera 21. The information processing device 30-1 acquires the two-dimensional position of the optical signal Si on the image plane using the event data EVD. This two-dimensional position indicates the position where each light source 10-i is projected on the image plane. Hereinafter, the position where each light source 10-i is projected on the image plane will be referred to as the "light source image position SPI-i." Furthermore, the process of acquiring the light source image position SPI-i will be referred to as "light source image position acquisition." In the drawings of the present disclosure, the image plane coordinate system is represented by the u axis and the v axis. Furthermore, the light source image position SPI-i is expressed as [u i ,v i A specific example of obtaining the light source image position will be described later.

[0022] The optical signal Si includes position-related information Pi for specifying the absolute position of each light source 10-i that is the source of the signal. In the drawings of the present disclosure, the absolute coordinate system is represented by the X-axis, Y-axis, and Z-axis. The information processing device 30-1 acquires the absolute position of each light source 10-i from the position-related information Pi included in the optical signal Si. Information regarding the absolute position of each light source 10-i is set in advance and registered in the optical wireless communication system 1. Hereinafter, the pre-registered absolute position of each light source 10-i will be referred to as a "light source registration position SPR-i." Furthermore, the process of acquiring the light source registration position SPR-i will be referred to as "light source registration position acquisition." In the drawings of the present disclosure, the light source registration position SPR-i is represented by [X Ri ,Y Ri ,Z Ri Specific examples of the position-related information Pi and the light source registration position acquisition will be described later.

[0023] Through the above-described steps, the information processing device 30-1 determines the light source registration position SPR-i ([X Ri ,Y Ri ,Z Ri ]) and the light source image position SPI-i ([u in Figure 2 i ,v i ]) are acquired. The information processing device 30-1 estimates the location of the event camera 21 from the geometric relationships between the n sets of data. More specifically, a rotation matrix and a translation vector are obtained from the n sets of data. A specific solution is known as the PnP (Perspective n Point) problem, and the number of data sets required to solve this problem (i.e., the value of n) varies depending on the method. For example, a method called the 8-point algorithm is known as one method for solving such a problem. Note that the event camera 21 is provided in the receiving device 20, so estimating the location of the event camera 21 is synonymous with estimating the location of the receiving device 20.

[0024] In summary, the first camera arrangement estimation is a process of calculating the first camera estimated arrangement CP1, which is unknown information, using the absolute position of each light source 10-i (that is, the light source registration position SPR-i) as known information.

[0025] 2-2.Light source image position acquisition FIG. 3 is a schematic diagram showing the process of acquiring the light source image position.

[0026] 3A is a graph showing the spatiotemporal distribution of the event data EVD received by the information processing device 30-1 from the event camera 21. The event data EVD includes not only events caused by the blinking of each light source 10-i (i.e., the optical signals Si), but also noise N, which is an event caused by the movement of the subject or the event camera 21. In other words, the event data EVD is output in a state where the optical signals Si necessary for camera position estimation and the noise N unnecessary for camera position estimation are mixed together. Therefore, in order for the information processing device 30-1 to acquire the light source image position SPI-i, it is necessary to perform signal separation, which is a process of separating the optical signals Si from the noise N.

[0027] One method of signal separation is based on the "data frequency" of the event data EVD for each pixel. As described above, the event data EVD includes the time at which the event was detected. Therefore, the information processing device 30-1 can calculate the number of event data EVD detected per unit time for each pixel. The number of event data EVD detected per unit time can be called the "data frequency."

[0028] As shown in FIG. 4, the characteristics of the optical signal Si and the noise N differ significantly in terms of data frequency. The data frequency of the optical signal Si is linked to the blinking frequency of each light source 10-i, and its value is approximately several 100 Hz to several 100 kHz. On the other hand, the data frequency of the noise N caused by the movement of the subject or the event camera 21 is significantly smaller (approximately several 10 Hz) than the data frequency of the optical signal Si. Therefore, the information processing device 30-1 can separate the optical signal Si from the noise N by using a frequency filter. An example of a frequency filter is a high-pass filter that cuts off signals below a preset frequency. In this case, the information processing device 30-1 determines that a pixel region in which a high data frequency not cut off by the high-pass filter is observed is a signal region occupied by the optical signal Si.

[0029] (B) in FIG. 3 is a graph showing the spatiotemporal distribution of the event data EVD after signal separation has been performed. As described above, the event data EVD includes information on coordinates in the image plane coordinate system, so the information processing device 30-1 can acquire the position of the separated optical signal Si in the image plane coordinate system. The position of the optical signal Si in the image plane coordinate system indicates the position where each light source 10-i is projected onto the image plane coordinate system, i.e., the light source image position SPI-i. In reality, the signal area occupied by the optical signal Si on the image plane extends over multiple pixels, so for example, the center coordinate of the signal area of ​​the optical signal Si may be considered to be the light source image position SPI-i. In this manner, light source image position acquisition is performed.

[0030] 2-3. Obtaining the light source registration position FIG. 5 is a block diagram showing an example of light source registration position acquisition. As described above, the optical signal Si transmitted from each light source 10-i includes position-related information Pi for specifying the light source registration position SPR-i. The position-related information Pi is, for example, identification information SID-i for identifying each light source 10-i. In this case, the optical wireless communication system 1 further includes a storage device 50. The storage device 50 stores the light source registration positions SPR-1 to SPR-n of the multiple light sources 10-1 to 10-n in association with the identification information SID-1 to SPR-n of the multiple light sources 10-1 to 10-n. The information processing device 30-1 accesses the storage device 50 and acquires the light source registration position SPR-i corresponding to the identification information SID-i. The storage device 50 may be built into the information processing device 30-1 or may be an external device different from the information processing device 30-1. Alternatively, the storage device 50 may be managed by a management server, and the information processing device 30-1 may acquire the light source registration position SPR-i through communication with the management server.

[0031] As described above, first camera position estimation is a process of estimating the location of the event camera 21 through light source image position acquisition and light source registration position acquisition. Existing position estimation systems (e.g., satellite positioning systems) may not be able to accurately determine position in places where satellite radio waves are difficult to reach (inside buildings, underground, between high-rise buildings, etc.). On the other hand, first camera position estimation by the optical wireless communication system 1 does not use satellite radio waves, so it can be said that there are fewer limitations on the places where it can be used.

[0032] When the process executed by the information processing device 30-1 uses the light source registration positions SPR-i, such as first camera position estimation, the accuracy of the light source registration positions SPR-i is an important factor. For example, if the accuracy of the light source registration positions SPR-i of each light source 10-i is low, the accuracy of the first camera estimated arrangement CP1 obtained based on the light source registration positions SPR-i also decreases. Therefore, it is important to obtain the accurate absolute position of each light source 10-i and update the light source registration positions SPR-i when necessary. Hereinafter, the process of calculating the absolute position of each light source 10-i using the event camera 21 will be referred to as "light source position estimation."

[0033] 3.Light source position estimation FIG. 6 is a schematic diagram related to light source position estimation. The information processing device 30-2 is the subject of light source position estimation processing. The information processing device 30-2 can also be called the second information processing device 30-2. The target of light source position estimation is denoted as target light source 10-T. For example, the target light source 10-T is one of the multiple light sources 10-1 to 10-n used for first camera placement estimation. Typically, light source position estimation is performed when the target light source 10-T is newly installed or moved and the position of the target light source 10-T needs to be accurately determined. Alternatively, light source position estimation may be performed periodically according to a preset schedule.

[0034] <Get camera location> For light source position estimation, the information processing device 30-2 acquires the position (i.e., absolute position and orientation) of the event camera 21 when the receiving device 20 receives the target light signal ST. Hereinafter, the process by which the information processing device 30-2 acquires the position of the event camera 21 is referred to as "second camera position estimation." The position of the event camera 21 acquired by the second camera position estimation is referred to as "estimated second camera position CP2." The second camera position estimation is based on a different method from the first camera position estimation described in Section 2. One known method for second camera position estimation is SLAM (Simultaneous Localization and Mapping). SLAM is a general term for technologies in which a device equipped with SLAM simultaneously performs "self-localization" to estimate its current location and "environmental mapping" to grasp the surrounding conditions. The information processing device 30-2 may use SLAM for second camera position estimation. For example, the information processing device 30-2 can use images acquired by the event camera 21 for second camera position estimation. In general, SLAM that uses camera images is called Visual SLAM.

[0035] An example of Visual SLAM is a method for estimating self-location and creating an environmental map based on markers (e.g., two-dimensional codes). Multiple markers are installed in advance within a specified area. A map associated with the absolute position information of the markers is also created in advance. In this method, a device equipped with SLAM estimates its own location using the absolute position information of the detected markers and the map.

[0036] Another example of Visual SLAM is a method that uses the scenery itself to estimate absolute position. In this method, images of the scenery in a specified area are captured in advance and stored in association with a map database. Devices equipped with SLAM estimate their own position by comparing the scenery captured by the camera with the map database.

[0037] In this way, the information processing device 30-2 performs second camera arrangement estimation and acquires the second camera estimated arrangement CP2.

[0038] <Light source image position acquisition> In light source position estimation, one or more receiving devices 20 receive the target light signal ST emitted by the target light source 10-T at multiple different locations. In FIG. 6, the receiving device 20a receives the target light signal ST at the second camera estimated location CP2a, and the receiving device 20b receives the target light signal ST at the second camera estimated location CP2b. The receiving device 20a and the receiving device 20b may be the same device or different devices. In other words, a single receiving device 20 may receive the target light signal ST multiple times at different locations, or multiple receiving devices 20 may each receive the target light signal ST at different locations. Furthermore, the timing at which the receiving device 20 receives the target light signal ST may differ for each location.

[0039] The information processing device 30-2 acquires the target light source image position SPI-Ta of the target light source 10-T at the second camera estimated arrangement CP2a and the target light source image position SPI-Tb of the target light source 10-T at the second camera estimated arrangement CP2b. The method for acquiring the target light source image position SPI-Ta and the target light source image position SPI-Tb is the same as the method described in Section 2-2. That is, the target light source image position SPI-Ta corresponds to the light source image position SPI-i when the receiving device 20a receives the target light signal ST at the second camera estimated arrangement CP2a. Also, the target light source image position SPI-Tb corresponds to the light source image position SPI-i when the receiving device 20b receives the target light signal ST at the second camera estimated arrangement CP2b.

[0040] <Light source position estimation> The information processing device 30-2 performs light source position estimation based on the geometric relationship between the second camera estimated position CP2 and the target light source image position SPI-T. The absolute position of the target light source 10-T obtained by light source position estimation is called the "light source estimated position SPE-T." The same method as in Section 2-1 is used for light source position estimation. In the case of light source position estimation, the known information is the position of the event camera 21 (second camera estimated position CP2), and the unknown information, the light source estimated position SPE-T, is calculated. That is, contrary to the first camera position estimation, light source position estimation estimates the unknown absolute position of the target light source 10-T using the position of the event camera 21 as known information. Note that for simplicity, only two second camera estimated position CP2a and CP2b are shown in FIG. 6, but three or more second camera estimated position CP2s may be used to acquire the light source image position.

[0041] 3-1.Light source position update Consider a case where the optical wireless communication system 1 includes a storage device 50, and the storage device 50 stores the target light source registration position SPR-T of the target light source 10-T in association with identification information SID-T. In this case, the target light source 10-T transmits the identification information SID-T as position relation information PT. The information processing device 30-2 can update the target light source registration position SPR-T of the target light source 10-T using the estimated light source position SPE-T obtained through the above steps. Hereinafter, updating the target light source registration position SPR-T is referred to as "light source position update." For example, as shown in FIG. 6, the information processing device 30-2 performs the light source position update by rewriting the target light source registration position SPR-T corresponding to the identification information SID-T with the estimated light source position SPE-T. That is, the information processing device 30-2 performs the rewrite by regarding the estimated light source position SPE-T obtained by light source position estimation as the true absolute position of the target light source 10-T. The information processing device 30-2 may determine whether to update the light source position based on the difference between the target light source registration position SPR-T and the estimated light source position SPE-T. For example, if the distance between the target light source registration position SPR-T and the estimated light source position SPE-T is equal to or greater than a threshold, the information processing device 30-2 updates the light source position. The information processing device 30-2 rewrites the target light source registration position SPR-T stored in the storage device 50 to the estimated light source position SPE-T.

[0042] The information processing device 30-2 can use the first camera estimated position CP1 obtained through the first camera position estimation as a trigger to start light source position estimation. FIG. 7 is a block diagram showing the relationship between light source position estimation and first camera position estimation. For example, if the difference between the first camera estimated position CP1 and the second camera estimated position CP2 is equal to or greater than a threshold, the difference is considered to be caused by low accuracy of the target light source registration position SPR-T. In such a case, the information processing device 30-2 may start light source position estimation. The information processing device 30-2 updates the information on the light source registration position SPR-i. That is, the information processing device 30-2 performs light source position estimation by regarding each of the multiple light sources 10-1 to 10-n used in the first camera estimation as the target light source 10-T, and also performs light source position update as necessary.

[0043] In addition to the above-described examples, the information processing device 30-2 may consider one of the multiple light sources 10-1 to 10-n, for which a predetermined time or more has passed since the previous light source position update, as the target light source 10-T, and perform light source position estimation and light source position update.

[0044] 3-2.Effects In the optical wireless communication system 1 according to this embodiment, the information processing device 30-2 executes light source position estimation, thereby making it possible to estimate the absolute position of the target light source 10-T. The absolute position of the target light source 10-T is used, for example, for the above-mentioned first camera placement estimation. Note that the technology disclosed in Patent Document 1 makes it easier to identify the position of the transmission source on the display screen of the user terminal, but does not estimate the absolute position of the transmission source.

[0045] The light source position estimation involves estimating the second camera position using SLAM or the like. The optical wireless communication system 1 uses the estimated light source position SPE-T obtained through the light source position estimation as the target light source registered position SPR-T for estimating the first camera position. In other words, the optical wireless communication system 1 can employ a configuration in which two types of processing for estimating the position of the event camera 21 are used in combination. The effects of this are described below.

[0046] Because the SLAM used for the second camera position estimation imposes a heavy processing load, always using the second camera position estimation is undesirable from the viewpoint of saving computing resources. Furthermore, the information processing device 30-2 with the high processing performance required for SLAM is not necessarily provided in commonly available devices. Meanwhile, the first camera position estimation detects changes in brightness due to the blinking of each light source 10-i and uses this as event data EVD. Therefore, the data volume of the event data EVD can be said to be smaller than the amount of image data output by a normal camera. Therefore, the first camera arrangement estimation based on the event data EVD has a smaller processing load than the second camera arrangement estimation using SLAM, etc. As a result, it can be said that the information processing device 30-1 that executes the first camera arrangement estimation does not need to have as high a processing performance as the information processing device 30-2 that executes the second camera arrangement estimation.

[0047] The accuracy of the first camera position estimation is largely dependent on the accuracy of the light source registration position SPR-i. Therefore, in the second camera position estimation, a device equipped with a high-performance information processing device 30-2 is used and the light source registration position SPR-i is updated as appropriate, thereby maintaining the accuracy of the first camera position estimation.

[0048] The following are examples of actual operational modes of the first camera position estimation and the second camera position estimation. Most users of the optical wireless communication system 1 obtain the first camera estimated position CP1 using the first camera position estimation. On the other hand, some users with high-performance devices perform light source position estimation and light source position update using the second camera position estimation via their own information terminals. Alternatively, the light source position estimation and light source position update may be performed by a dedicated terminal, vehicle, robot, or the like used by an administrator who manages the optical wireless communication system 1. In other words, light source position estimation and light source position update using the second camera position estimation are necessary for users who do not have devices compatible with the second camera position estimation to accurately obtain the first camera estimated position CP1.

[0049] 5. Optical communication system configuration example FIG. 8 is a block diagram showing a first configuration example of the optical wireless communication system 1. As shown in FIG.

[0050] The blinking control device 60 controls the blinking pattern of the target light source 10-T. The blinking control device 60 may be built into each facility (street light, indoor light, etc.) that includes the target light source 10-T. Alternatively, the blinking control device 60 may be included in an external facility (such as a management server) and the blinking of the target light source 10-T may be controlled externally.

[0051] The information generating unit 61 generates a digital signal D. The digital signal D is a signal that represents the positional relation information PT using two values, “0” and “1.” The generated digital signal D is output to the modulation unit 62.

[0052] The modulator 62 generates a modulated signal M by modulating the digital signal D into a signal suitable for optical wireless communication. As a modulation method, a pulse width modulation (PWM) method, a pulse position modulation (PPM) method, or the like is used. The PWM method is a method in which the ratio of the ON (lighting) and OFF (extinguishing) time of the target light source 10-T is changed by an input signal. The PPM method is a modulation method in which the position of the carrier pulse on the time axis is changed by an input signal.

[0053] The blinking control device 60 passes the modulation signal M to the target light source 10-T. The target light source 10-T blinks in accordance with the modulation signal M. The target light signal ST is a signal represented by the blinking pattern of the target light source 10-T represented by the modulation signal M.

[0054] The receiving device 20 receives the target light signal ST through the event camera 21. Specifically, an EV sensor built into the event camera 21 detects a change in brightness due to the target light signal ST as an event. The receiving device 20 transmits the event data EVD to the signal separation unit 31 in the information processing device 30-2.

[0055] The information processing device 30-2 is responsible for the processing of second camera position estimation, light source position estimation, and light source position update. In Fig. 8, the information processing device 30-2 is a device (e.g., an external server) different from the receiving device 20. That is, the receiving device 20 transmits event data EVD to the external information processing device 30-2.

[0056] The signal separation unit 31 acquires the target light source image position SPI-T by performing the process (the signal separation described above) of separating the target light signal ST and noise N contained in the event data EVD. The target light source image position SPI-T is output to the light source position estimation unit 34.

[0057] The demodulator 32 demodulates the target optical signal ST separated by the signal separator 31 to obtain position related information PT (for example, identification information SID-T). The demodulator 32 passes the position related information PT to the light source position updater 35.

[0058] The second camera arrangement estimation unit 33-2 executes second camera arrangement estimation to calculate the second camera estimated arrangement CP2, and passes the second camera estimated arrangement CP2 to the light source position estimation unit .

[0059] The light source position estimation unit 34 performs light source position estimation to obtain an estimated light source position SPE-T. The light source position estimation is performed based on the target light source image position SPI-T and the second camera estimated position CP2. The light source position estimation unit 34 passes the estimated light source position SPE-T to the light source position update unit 35.

[0060] The light source position update unit 35 executes light source position update. The light source position update unit 35 first acquires the target light source registration position SPR-T based on the position relation information PT. Specifically, as shown in the example of FIG. 5, the information processing device 30-2 accesses the storage device 50 and acquires the target light source registration position SPR-T corresponding to the identification information SID-T. Next, the light source position update unit 35 compares the target light source registration position SPR-T with the estimated light source position SPE-T, and rewrites the target light source registration position SPR-T to the estimated light source position SPE-T based on the difference. As a result, the accurate absolute position of the target light source 10-T obtained by light source position estimation is reflected in the storage area of ​​the storage device 50.

[0061] Fig. 9 is a block diagram showing a second configuration example of the optical wireless communication system 1. The example of Fig. 9 shows a case where first camera arrangement estimation using a plurality of light sources 10-1 to 10-n is linked with light source position estimation and light source position update for the target light source 10-T. Description of the configuration overlapping with Fig. 8 will be omitted where appropriate.

[0062] The blinking control device 60 controls the blinking patterns of the multiple light sources 10-1 to 10-n. The blinking control device 60 may be built into each facility (street light, indoor light, etc.) that includes each light source 10-i. Alternatively, the blinking control device 60 may be included in an external facility (such as a management server) and externally control the blinking of each light source 10-i. Furthermore, when the blinking control device 60 is provided in an external facility, it may collectively control the blinking of the multiple light sources 10-1 to 10-n.

[0063] The information processing device 30-1 includes a signal separation unit 31, a demodulation unit 32, and a first camera arrangement estimation unit 33-1. The functions of the signal separation unit 31 and the demodulation unit 32 are the same as those in FIG.

[0064] The first camera arrangement estimation unit 33-1 acquires the light source image position SPI-i from the signal separation unit 31. It also acquires the position relation information Pi from the demodulation unit 32. The first camera arrangement estimation unit 33-1 acquires the light source registration position SPR-i based on the position relation information Pi. As a result, the first camera arrangement estimation unit 33-1 acquires n sets of data sets of the light source registration position SPR-i and the light source image position SPI-i for each light source 10-i. The first camera arrangement estimation unit 33-1 calculates the estimated first camera arrangement CP1 from the geometric relationship between these n sets of data sets.

[0065] The first camera arrangement estimation unit 33-1 transmits information relating to the first camera estimated arrangement CP1 to the light source position estimation unit 34 in the information processing device 30-2. The information relating to the first camera estimated arrangement CP1 may be transmitted directly from the information processing device 30-1 or may be transmitted via an external device such as a management server.

[0066] As described above, the light source position estimation unit 34 receives the second camera estimated position CP2 from the second camera position estimation unit 33-2, and also receives the first camera estimated position CP1 from the first camera position estimation unit 33-1. The light source position estimation unit 34 compares the first camera estimated position CP1 with the second camera estimated position CP2, estimates the light source position according to the difference between them, and passes the light source estimated position SPE-T to the light source position update unit 35. In this case, the light source position estimation and light source position update are performed by regarding any one of the multiple light sources 10-1 to 10-n as the target light source 10-T.

[0067] Fig. 10 is a block diagram showing a third example configuration of the optical wireless communication system 1. The example in Fig. 10 shows a case where the various processes described so far are executed within one receiving device 20. That is, the functions of the information processing devices 30-1 and 30-2 described so far are integrated into the information processing device 30, which is provided in the receiving device 20. In this case, the series of processes described so far are executed within the receiving device 20 (smartphone, augmented reality terminal, robot, etc.). [Explanation of symbols]

[0068] 1: Optical wireless communication system 10-1: 1st light source 10-T: Target light source 20: Receiving device 21: Event Camera 30: Information processing device 50: Storage device CP1: Estimated location of the first camera CP2: Estimated placement of the second camera EVD: Event Data N: Noise S-1: First optical wireless communication signal SID-i: Identification information SPE-T: Estimated light source position SPI-T: Target light source image position SPI-i: Light source image position SPR-T: Target light source registration position SPR-i: Light source registration position

Claims

1. a target light source configured to transmit a target light wireless communication signal; one or more receiving devices; one or more information processing devices; Equipped with Each of the one or more receiving devices: Equipped with an event camera, configured to receive the target optical wireless communication signal through the event camera; a second information processing device included in the one or more information processing devices is configured to perform light source position estimation to estimate a position of the target light source in an absolute coordinate system; The light source position estimation is acquiring a second camera estimated position indicating a position and an orientation of the event camera in the absolute coordinate system when each of the receiving devices receives the target optical wireless communication signal; acquiring a target light source image position, which is a position where the target light source is projected onto the image plane coordinate system, by identifying a signal region of the target light wireless communication signal in an image plane coordinate system of an image acquired by the event camera; acquiring a light source estimated position, which is an estimated position of the target light source in the absolute coordinate system, based on the second camera estimated position and the target light source image position; Contains Optical wireless communication system.

2. 2. The optical wireless communication system according to claim 1, The one or more receiving devices receive the target optical wireless communication signal at a plurality of different locations. Optical wireless communication system.

3. 3. The optical wireless communication system according to claim 1, a storage device for storing a target light source registration position indicating the position of the target light source in the absolute coordinate system; The second information processing device is configured to update the target light source registered position with the light source estimated position when a difference between the light source estimated position and the target light source registered position is equal to or greater than a threshold. Optical wireless communication system.

4. 2. The optical wireless communication system according to claim 1, further comprising a plurality of light sources including the target light source; each of the plurality of light sources transmits an optical wireless communication signal; the optical wireless communication signal includes position-related information for identifying a position of a source of the optical wireless communication signal in the absolute coordinate system; a first information processing device included in the one or more information processing devices is configured to perform a first camera placement estimation; The first camera placement estimation includes: acquiring light source image positions indicating positions where each of the light sources is projected onto the image plane coordinate system by identifying a signal region of the optical wireless communication signal in the image plane coordinate system of the image acquired by the event camera; acquiring a position of each of the light sources in the absolute coordinate system based on the position related information included in the optical wireless communication signal; obtaining a first camera estimated position indicating a position and orientation of the event camera in the absolute coordinate system based on the light source image position and the position of each of the light sources in the absolute coordinate system; Contains Optical wireless communication system.

5. 5. The optical wireless communication system according to claim 4, In the light source position estimation, the second camera estimated arrangement is acquired by a method different from that of the first camera arrangement estimation, When an error between the first camera estimated arrangement and the second camera estimated arrangement exceeds a threshold, the second information processing device is configured to perform the light source position estimation. Optical wireless communication system.

Images