Optical wireless communication system

Abstract

The present application relates to an optical wireless communication system, and provides a technique for coping with a case where the number of light sources appearing in a captured image is insufficient in an optical wireless communication system using a plurality of light sources and a camera. The optical wireless communication system has a plurality of light sources including a first light source and a second light source, and a reception device having a camera. The reception device is configured to receive an optical wireless communication signal by the camera. The optical wireless communication system executes a light source guidance process in a case where a first image captured by the camera includes the first light source and does not include the second light source. The light source guidance process includes acquiring an absolute position of the first light source, i.e., a first light source absolute position, and an absolute position of the second light source, i.e., a second light source absolute position. The light source guidance process also includes determining a guidance direction for capturing the second light source into a field of view of the camera based on at least the first light source absolute position and the second light source absolute position. The light source guidance process also includes a process of notifying the guidance direction via the reception device.

Description

Technical Field

[0001] This disclosure relates to an optical wireless communication system utilizing multiple light sources and a camera. Background Technology

[0002] Patent Document 1 discloses a communication system that performs self-position inference and communication in parallel. The communication system includes multiple light sources and a camera. The communication system infers the position of the camera based on the position information of each light source and the coordinates of the light sources in the captured image.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2024-111470

[0004] To infer camera position using multiple light sources, the camera needs to capture multiple light sources (which are reflected in the captured image). However, depending on the camera angle, there may be situations where the number of light sources required for camera position inference is not reflected in the captured image. In such scenarios, camera position inference cannot be performed properly. Summary of the Invention

[0005] One object of this disclosure is to provide a technique for dealing with insufficient number of light sources reflected in captured images in an optical wireless communication system utilizing multiple light sources and a camera.

[0006] The first point relates to optical wireless communication systems.

[0007] The optical wireless communication system includes: multiple light sources including a first light source and a second light source, a receiving device with a camera, and one or more processors.

[0008] Multiple light sources transmit optical wireless communication signals respectively.

[0009] The receiving device is configured to receive optical wireless communication signals via a camera.

[0010] One or more processors are configured to perform light source guidance processing when a first image acquired by a camera contains a first light source but does not contain a second light source.

[0011] Light source guidance processing includes:

[0012] The absolute position of the first light source (i.e., the absolute position of the first light source) and the absolute position of the second light source (i.e., the absolute position of the second light source) are obtained in response to the reception of the first optical wireless communication signal transmitted by the first light source.

[0013] The guiding direction for capturing the second light source into the camera's field of view is determined at least based on the absolute positions of the first and second light sources; and

[0014] The direction of guidance is communicated via a receiving device.

[0015] The light source guidance processing notification disclosed herein is used to guide the direction in which a second light source, not reflected in the first image captured by the camera, is captured into the camera's field of view. A user with a camera receiving device can move or tilt the camera in the direction of the second light source according to the guidance, enabling the camera to capture more light sources. Attached Figure Description

[0016] Figure 1 This is a diagram showing an overview of the optical wireless communication system according to this embodiment.

[0017] Figure 2 It is a three-dimensional diagram that represents the geometric relationship between the absolute position of the light source and the position of the light source image.

[0018] Figure 3 This is a schematic diagram illustrating the process of obtaining the position of the light source image.

[0019] Figure 4 It is a graph representing the characteristics of the data frequency of optical signals and noise.

[0020] Figure 5 It is a diagram representing the image plane before and after the acquisition of the light source image position and the acquisition of the absolute position of the light source.

[0021] Figure 6 This is a schematic diagram illustrating the first example of light source guidance processing.

[0022] Figure 7 This is a schematic diagram illustrating the second example of light source guidance processing.

[0023] Figure 8 This is a block diagram representing the third example of light source guidance processing.

[0024] Figure 9 This is the block diagram corresponding to the fourth example of light source guidance processing.

[0025] Figure 10 This is a block diagram illustrating an example of the configuration of an optical wireless communication system.

[0026] Explanation of reference numerals in the attached figures

[0027] 1… Optical wireless communication system, 10-1… First light source, 10-2… Second light source, 20… Receiver, 21… Event camera, 30… Information processing device, 31… Signal separation unit, 32… Demodulation unit, 33… Position inference unit, 40… Guiding direction, 50… Server, 60… Flash control device, 61… Information generation unit, 62… Modulation unit, AP-1… Absolute position of the first light source, AP-2… Absolute position of the second light source, IMG-1… First image, R-1… First communication range, R-2… Second communication range, S-i… Optical wireless communication signal. Detailed Implementation

[0028] The embodiments of this disclosure will be described with reference to the accompanying drawings.

[0029] 1. Basic Components

[0030] Figure 1 This is a diagram showing an outline of the optical wireless communication system 1 according to this embodiment. The optical wireless communication system 1 includes a plurality of light sources 10-1 to n (n is an integer satisfying n≥2), a receiving device 20 equipped with an event camera 21, and an information processing device 30.

[0031] Multiple light sources 10⁻¹ to n are fixedly installed in the space and located indoors or outdoors. Examples of multiple light sources 10⁻¹ to n include visible light LEDs (light emitting diodes) and infrared LEDs. Since visible light LEDs are widely used in streetlights, indoor lights, signal lights, electronic billboards, etc., utilizing visible light LEDs in the optical wireless communication system 1 enables the effective and flexible use of existing equipment. Because visible light LEDs can flash at high speeds imperceptible to the human eye, their flashing can be controlled to utilize visible light as a communication signal. Preferably, multiple light sources 10⁻¹ to n are simultaneously reflected within the field of view of the event camera 21.

[0032] Multiple light sources 10-i (i = 1 to n) constituting light sources 10-1 to n transmit optical wireless communication signals S-i by flashing. The optical wireless communication signal S-i contains object information TI-i to be transmitted. The object information TI-i includes, for example, the absolute position information of the light source 10-i that serves as the transmitting source. In the accompanying drawings of this disclosure, the absolute coordinate system is represented by the X-axis, Y-axis, and Z-axis. Here, the X-axis and Y-axis are orthogonal to each other and define the horizontal plane, while the Z-axis represents the vertical direction. Furthermore, the absolute position AP-i of the light source 10-i is represented as [X...]. i Y i Z i For simplicity, “optical wireless communication signal S-i” will be referred to as “optical signal S-i” below.

[0033] The receiving device 20 receives the light signal S-i via the event camera 21. Preferably, the receiving device 20 simultaneously receives light signals S-1 to S-n transmitted from multiple light sources 10-1 to 10-n. Typical examples of the receiving device 20 include smartphones, tablets, and wearable terminals used for extended reality. However, it is not limited to these examples; any object equipped with the event camera 21 can function as the receiving device 20. For example, vehicles, robots, wheelchairs, canes, etc., equipped with the event camera 21, can function as the receiving device 20. Thus, the receiving device 20 is typically configured as a movable object or terminal. When the receiving device 20 moves, the event camera 21 also moves accordingly.

[0034] The event camera 21 incorporates an event-based vision sensor (EV sensor). The EV sensor observes changes in the brightness of light received by the pixels (imaging elements) within it. When the EV sensor observes a brightness change exceeding a preset threshold, it detects that brightness change as an "event." Events are detected when a situation arises that differs from previous conditions. For example, if the subject or the event camera 21 moves, the subject may appear in pixels that were not previously present due to a change in their relative positions. In this case, a significant brightness change occurs in the pixels surrounding the subject, and thus an event is detected. Additionally, an event is detected when the light signal S-i also produces a brightness change accompanied by the flickering of the light source 10-i.

[0035] The event camera 21 outputs data related to the pixels of the detected event as event data EVD. The event data EVD includes at least the coordinates of the pixel where the event occurred on the image plane, the time the event was detected, and the brightness polarity (positive / negative). The threshold referenced by the EV sensor when detecting brightness changes is set to be changeable. If an event occurs, thresholds for the positive side (change in bright direction) and the negative side (change in dark direction) are set based on a voltage (reference voltage) based on the brightness level at that time. Voltage changes exceeding the positive side threshold are detected as positive polarity events, and voltage changes exceeding the negative side threshold are detected as negative polarity events. That is, for the light signal S-i, the moment when the light source 10-i is lit is detected by the EV sensor as a positive polarity event, and the moment when the light source 10-i is turned off is detected by the EV sensor as a negative polarity event.

[0036] The information processing device 30 acquires event data EVD from the event camera 21. The information processing device 30 uses the event data EVD to acquire the two-dimensional position of the light signal S-i on the image plane. This two-dimensional position represents the position where the light source 10-i is projected onto the image plane. Hereinafter, the position where the light source 10-i is projected onto the image plane is referred to as the "light source image position IP-i", and the process of acquiring the light source image position IP-i is referred to as "light source image position acquisition". In this disclosure, the image plane coordinate system is represented by the u-axis and v-axis. Furthermore, the light source image position IP-i of the light source 10-i is represented as [u...]. i v i The information processing device 30 may be included in the receiving device 20, or it may be an external device to the receiving device 20.

[0037] Furthermore, the information processing device 30 obtains the three-dimensional position (absolute position AP-i) of the light source 10-i in the absolute coordinate system based on the object information TI-i contained in the optical signal S-i. Hereinafter, the process of obtaining the absolute position AP-i of the light source will be referred to as "absolute position acquisition of the light source". Specific examples of object information TI-i and absolute position acquisition of the light source will be described later.

[0038] After the above-described process, the information processing device 30 obtains the absolute position AP-i of the light source 10-i. Figure 1 [X] i Y i Z i ]) and the image position of the light source IP-i ( Figure 1 [u] i v i n sets of datasets. Figure 2 This is a stereo diagram representing the geometric relationship between the absolute position AP-i of the light source and the image position IP-i of the light source. The information processing device 30 infers the position and orientation of the event camera 21 in the absolute coordinate system based on the geometric relationships between n sets of datasets. More specifically, the rotation matrix and translation vector are calculated based on the n sets of datasets. Specific solutions are known as the PnP (Perspective n Point) problem, and the number of datasets (i.e., n) required to solve this problem varies depending on the method. For example, the method known as the eight-point algorithm (i.e., n = 8) is known as one solution to such a problem. Since the receiving device 20 includes the event camera 21, inferring the absolute position and orientation of the event camera 21 is synonymous with inferring the absolute position and orientation of the receiving device 20. Furthermore, in the case of inferring only the position of the event camera 21, theoretically, n = 3 is sufficient to determine its position. To improve inference accuracy, more light sources 10-i than the minimum required number can be used.

[0039] As explained above, the information processing device 30 infers the absolute position and orientation of the event camera 21 by acquiring the image position of the light source and the absolute position of the light source. This series of processes will be referred to as "camera position inference" below. Existing position inference systems (e.g., satellite positioning systems) may fail to accurately locate in places where satellite waves cannot reach (inside buildings, underground, between high-rise buildings, etc.). On the other hand, the camera position inference involved in the optical wireless communication system 1, since it does not use satellite waves, can be said to have fewer limitations on the location it utilizes.

[0040] 2. Camera position inference

[0041] The following section details the series of processes involved in camera position inference.

[0042] Figure 3 This is a schematic diagram illustrating the process of obtaining the position of the light source image. Figure 3 (A) in the diagram represents the spatiotemporal distribution of the event data EVD received by the information processing device 30 from the event camera 21. In addition to events caused by the flickering of the light source 10-i (i.e., light signal S-i), the event data EVD also includes events caused by the movement of the subject and the event camera 21, i.e., noise N. That is, the event data EVD is output as a mixture of the light signal S-i required for camera position inference and the noise N not required for camera position inference. Therefore, in order to obtain the image position IP-i of the light source, the information processing device 30 needs to perform a process that separates the light signal S-i from the noise N, i.e., signal separation.

[0043] One method for signal separation is based on the "data frequency" of event data EVD for each pixel. As described above, the event data EVD contains the time at which an event was detected. Therefore, the information processing device 30 can calculate the number of event data EVDs detected per pixel per unit time. The number of event data EVDs detected per unit time can be referred to as the "data frequency".

[0044] like Figure 4As shown, the characteristics of the light signal S-i and the noise N are significantly different in the data frequency. The data frequency of the light signal S-i is linked to the flickering frequency of the light source 10-i, and its value is approximately several hundred Hz to several hundred kHz. On the other hand, the data frequency of the noise N caused by the movement of the subject and the event camera 21 is significantly lower than the data frequency involved in the light signal S-i (approximately tens of Hz). Therefore, the information processing device 30 can separate the light signal S-i from the noise N by using a frequency filter. As an example of a frequency filter, a high-pass filter that filters out signals below a predetermined frequency can be cited. In this case, the information processing device 30 determines the pixel region with a high data frequency that is not filtered out by the high-pass filter as the signal region occupied by the light signal S-i.

[0045] Figure 3 (B) in the diagram represents the spatiotemporal distribution of the event data EVD after signal separation. As described above, since the event data EVD contains coordinate information in the image plane coordinate system, the information processing device 30 can obtain the position of the separated light signal S-i in the image plane coordinate system. The position of the light signal S-i in the image plane coordinate system represents the position where the light source 10-i is projected onto the image plane coordinate system, i.e., the light source image position IP-i. Since the signal area occupied by the light signal S-i on the image plane actually extends to multiple pixels, for example, the center coordinates of the signal area of ​​the light signal S-i can be considered as the light source image position IP-i. In this way, the light source image position can be obtained.

[0046] The light signal S-i transmitted from the light source 10-i contains object information TI-i used to determine the absolute position AP-i of the light source. The information processing device 30 obtains the absolute position AP-i of the light source based on the object information TI-i.

[0047] Figure 5 It is a diagram representing the image plane before and after the acquisition of the light source image position and the acquisition of the absolute position of the light source. Figure 5 Image (A) shows the image plane before the acquisition of the light source image position and the acquisition of the light source absolute position, i.e., at the moment when the information processing device 30 acquires the event data EVD. At this moment, the event data EVD contains noise N along with the multiple light signals S-1 to n transmitted from the multiple light sources 10-1 to n.

[0048] The information processing device 30 performs signal separation between the optical signal S-i and the noise N, and obtains the position of the optical signal S-i in the image plane coordinate system, that is, the light source image position IP-i of the light source 10-i. Furthermore, the information processing device 30 obtains the absolute position AP-i of the light source 10-i by acquiring the absolute position of the light source. As a result of obtaining the light source image position and the absolute position of the light source, the information processing device 30... Figure 5 As shown in (B), a dataset of the absolute position AP-i of the light source 10-i and the image position IP-i of the light source is obtained. The information processing device 30 is able to infer the position and orientation of the event camera 21 in the absolute coordinate system based on this dataset and the geometric relationship based on the focal length of the event camera 21.

[0049] 3. Light source guidance processing

[0050] As previously described, in order for the camera position inference of the optical wireless communication system 1 to function properly, it is necessary to capture multiple light sources 10-1 to n within the field of view of the event camera 21. However, the event camera 21 is not always able to capture a sufficient number of light sources for camera position inference. The number of light sources that can be captured varies depending on the camera angle of the event camera 21. Therefore, when the number of light sources captured in the field of view of the event camera 21 is insufficient, a mechanism is effective in guiding the user of the receiving device 20 to capture light sources present in the surrounding area within the field of view of the event camera 21. In this section, consider the situation where the first light source 10-1 is reflected in the first image IMG-1 captured by the event camera 21, but the second light source 10-2 is not reflected. The optical wireless communication system 1 provides the user via the receiving device 20 with a notification to guide the inclusion of the second light source 10-2 in the field of view of the event camera 21. This series of processes is referred to as the "light source guidance process". The light source guidance process includes obtaining the absolute position of the first light source 10-1 (i.e., the first light source absolute position AP-1) and the absolute position of the second light source 10-2 (i.e., the second light source absolute position AP-2) in response to receiving the first light signal S-1 transmitted by the first light source 10-1. The light source guidance process also includes determining a guidance direction 40 for capturing the second light source 10-2 within the field of view of the event camera 21, based at least on the first light source absolute position AP-1 and the second light source absolute position AP-2. Several embodiments of the light source guidance process will be described below.

[0051] 3-1. The first example

[0052] Figure 6 This is a schematic diagram illustrating a first example of light source guidance processing. In this first example, the first optical signal S-1 transmitted from the first light source 10-1 contains information about the absolute position AP-1 of the first light source and the absolute position AP-2 of the second light source. Figure 6 In (A), the first light source 10-1 and the second light source 10-2 are positioned on the same pole, with the second light source 10-2 positioned below the first light source 10-1. The first light source 10-1 transmits the absolute position of the second light source 10-2 (i.e., the absolute position AP-2 of the second light source) together with its own absolute position (i.e., the absolute position AP-1 of the first light source) in the first optical signal S-1 (i.e., the first optical wireless communication signal). The receiving device 20, which receives the first optical signal S-1, determines the guiding direction 40 based on the absolute positions AP-1 and AP-2 of the first and second light sources. Since the second light source 10-2 is positioned below the first light source 10-1, the X and Y coordinates of the absolute positions AP-1 and AP-2 of the first and second light sources obtained by the receiving device 20 are approximately the same, while the Z coordinate is different. Therefore, the information processing device 30 determines that the guiding direction 40 is downward. After determining the guiding direction 40, the information processing device 30 notifies the receiving device 20 of the guiding direction 40. The notification can be shown as an arrow in the first image IMG-1 as illustrated, or it can be delivered by sound to prompt the user to head in the guidance direction 40. Alternatively, the user can be notified of the presence of a second light source 10-2 in the vicinity of the current event camera 21's field of view by simultaneously vibrating the receiving device 20 with the notification of the guidance direction 40. Thus, with the positional relationship between the first light source 10-1 and the second light source 10-2 defined, the optical wireless communication system 1 can determine the guidance direction 40 based on the absolute positions AP-1 and AP-2 of the first and second light sources. Specifically, the first example works effectively when the two light sources are positioned on the same object (e.g., a pole) and are close to each other.

[0053] Figure 6 (B) shows a block diagram corresponding to the first example of light source guidance processing. The first light source 10-1 transmits the first light source absolute position AP-1 and the second light source absolute position AP-2 contained in the first optical signal S-1. The receiving device 20 obtains the first light source absolute position AP-1 and the second light source absolute position AP-2 by receiving the first optical signal S-1.

[0054] 3-2. Second case

[0055] In the second example of light source guidance processing, the optical wireless communication system 1 utilizes information that defines the position of the receiving device 20, namely "receiving position definition information". Thus, the information processing device 30 can determine the guidance direction 40 in a wider variety of situations. Figure 7 This is a schematic diagram illustrating the second example of light source guidance processing.

[0056] Figure 7(A) in the figure is a schematic diagram (top view viewed from the Z direction) showing the positional relationship of the first light source 10-1, the second light source 10-2, and the receiving device 20. The first light source 10-1 and the second light source 10-2 are arranged far apart in the XY plane. In the example shown in the figure, the first optical signal S-1 is transmitted to a range defined in a specific direction (first communication range R-1). That is, the receiving device 20 can receive the first optical signal S-1 when it is located within the first communication range R-1. The information processing device 30 uses the first communication range information RI-1, which represents the first communication range R-1, as receiving position defining information. The first communication range information RI-1, for example, indicates, as coordinates, the direction in which the first optical signal S-1 is transmitted centered on the first light source 10-1. The first optical signal S-1 includes the first communication range information RI-1 as receiving position defining information. That is, the receiving device 20, which receives the first optical signal S-1, can obtain its approximate position by referring to the first communication range information RI-1. Specifically, the direction in which the receiving device 20 is positioned relative to the first light source 10-1 is determined. In addition to the absolute positions of the first light source AP-1 and the second light source AP-2, the receiving device 20 also receives the first communication range information RI-1, enabling the information processing device 30 to determine the guidance direction 40. In the example shown, in order to capture the second light source 10-2, which is not reflected in the first image IMG-1, within the field of view of the event camera 21, a right-pointing arrow is displayed in the first image IMG-1 as the guidance direction 40.

[0057] The receiving device 20 can also utilize an external system to obtain receiving location limiting information. For example, a Global Navigation Satellite System (GNSS) is one such external system. The information processing device 30 can use the location information of the receiving device 20 obtained from the GNSS as receiving location limiting information. In the second example, the information processing device 30 has already obtained the location of the receiving device 20 (i.e., the event camera 21) before using the camera location inference from the optical wireless communication system 1. Nevertheless, it is also effective to determine a more accurate location and orientation of the event camera 21 by utilizing the camera location inference from the optical wireless communication system 1.

[0058] Figure 7 (B) shows a block diagram of a second example of light source guidance processing. The first light source 10-1 transmits first communication range information RI-1 in addition to the first light source absolute position AP-1 and the second light source absolute position AP-2, within the first optical signal S-1. The receiving device 20 obtains the first communication range information RI-1 by receiving the first optical signal S-1.

[0059] Comparing the first and second examples, the first example is particularly effective when the guiding direction 40 is vertical, as it works by determining the guiding direction 40 based on the absolute positions of the two light sources (absolute position AP-1 of the first light source and absolute position AP-2 of the second light source). On the other hand, the second example is suitable not only for vertical directions but also for horizontal directions. If the positional relationship between the first light source 10-1 and the receiving device 20 cannot be determined solely based on the information of the absolute positions of the two light sources, then the guiding direction 40 cannot be determined. In the second example, the information processing device 30 can use the received position limiting information to determine the positional relationship between the first light source 10-1 and the receiving device 20.

[0060] 3-3. The third case

[0061] The third example of light source guidance processing utilizes a database DB. The optical wireless communication system 1 obtains the information required for light source guidance processing (absolute position AP-1 of the first light source, absolute position AP-2 of the second light source, first communication range R-1, etc.) from the database DB. In this case, the first light source 10-1 transmits the first identification information ID-1 corresponding to the first light source 10-1 in the first optical signal S-1.

[0062] Figure 8 This is a block diagram illustrating the third example of light source guiding processing. In Figure 8 In (A), the receiving device 20 has a database DB. The database DB stores information corresponding to each light source 10-i. Figure 8 In this example, the database DB records information such that the absolute position AP-1 and the first communication range R-1 of the first light source correspond to the first identification information ID-1, and the absolute position AP-2 of the second light source and the second communication range R-2 correspond to the second identification information ID-2. The second identification information ID-2 and the second communication range R-2 represent the identification information of the second light source 10-2 and the communication range of the second light source 10-2, respectively. The receiving device 20, having obtained the first identification information ID-1, refers to the database DB to obtain the absolute position AP-1 and the first communication range R-1 of the first light source corresponding to the first identification information ID-1. The receiving device 20 also obtains the absolute position of the second light source 10-2, i.e., the absolute position AP-2 of the second light source, based on the absolute position AP-1 of the first light source. For example, the receiving device 20 selects a light source 10-i within a predetermined distance from the first light source 10-1 as the second light source 10-2 to be the object of light source guidance processing. The receiving device 20 may also preferentially select a light source 10-i that is closer to the first light source 10-1 as the second light source 10-2. The information processing device 30 obtains the information required for light source guidance processing in this way.

[0063] The database DB can also be contained on a server 50 that is different from the receiving device 20 (see reference). Figure 8 (B)). In this case, the receiving device 20 sends the first identification information ID-1 to the server 50 and obtains the information required for the light source guidance process from the server 50 via communication.

[0064] 3-4. Fourth case

[0065] In the fourth example of light source guidance processing, the first light source 10-1 transmits the guidance direction 40 itself within the first optical signal S-1. This fourth example is effective when the guidance direction 40 is predetermined based on the positional relationship between the first light source 10-1 and the second light source 10-2. For example, in... Figure 6 (A) Figure 7 In (A), if the positional relationship between the first light source 10-1 and the second light source 10-2 has not been changed for a long time, the guiding direction 40 can be directly included in the first optical signal S-1. Figure 9 This is a block diagram corresponding to the fourth example of light source guidance processing. The first light source 10-1 transmits the guidance direction 40, which is included in the first optical signal S-1.

[0066] 3-5. Effects

[0067] The light source guidance processing is effective when the information processing device 30 performs processing using light signals S-1 to n transmitted from multiple light sources 10-1 to n. The camera position estimation described above is a typical example of such processing. When a sufficient number of light sources cannot be reflected at a certain camera angle, the light source guidance processing notifies the guidance direction 40, allowing the user to move the receiving device 20 so that the event camera 21 can capture more light sources 10-i. In other words, the light source guidance processing makes camera position estimation easier to utilize.

[0068] When both the first light source 10-1 and the second light source 10-2 are simultaneously reflected, the information processing device 30 does not need to perform light source guidance processing. Specifically, when both the first light source 10-1 and the second light source 10-2 are reflected in the first image IMG-1 at the initial camera angle, or when the result of the light source guidance processing is that the event camera 21 captures both light sources, the light source guidance processing is not performed. In this case, there is no need to notify the user of the guidance direction 40. It is possible to determine whether a certain light source 10-i is reflected in the first image IMG-1 based on whether the receiving device 20 receives the corresponding light signal S-i. For example, the state where the first light source 10-1 is reflected in the first image IMG-1 and the second light source 10-2 is not reflected in the first image IMG-1 means that the receiving device 20 receives the first light signal S-1 but does not receive the second light signal S-2. When both the first light source 10-1 and the second light source 10-2 are reflected in the first image IMG-1, the optical wireless communication system 1 can also notify the user of this situation. Therefore, if the event camera 21 captures two light sources as a result of the light source guidance process, the user can be aware of the situation. This notification can be a message displayed on the first image IMG-1, or it can be a sound-based notification.

[0069] 4. Example of an optical communication system configuration

[0070] Figure 10 This is a block diagram illustrating an example of the configuration of an optical wireless communication system 1.

[0071] The flashing control device 60 controls the flashing mode of the light source 10-i. The flashing control device 60 can be individually built into devices equipped with light sources 10-i (streetlights, traffic lights, indoor lights, etc.). Alternatively, the flashing control device 60 can be included in an external device (management server, etc.) and the flashing of the light source 10-i can be controlled from the outside. Furthermore, when the flashing control device 60 is installed in an external device, it can centrally control the flashing of multiple light sources 10-1 to n.

[0072] The information generation unit 61 generates a digital signal D. The digital signal D is a signal that represents the object information TI-i using two values, "0" and "1". The generated digital signal D is output to the modulation unit 62.

[0073] The modulation unit 62 generates a modulation signal M by modulating the digital signal D into a signal suitable for optical wireless communication. As a modulation method, pulse width modulation (PWM) and pulse position modulation (PPM) can be used. PWM modulates the ratio of the on (light-up) to the off (light-down) time of the light source 10-i according to the input signal. PPM modulates the position of the carrier pulse on the time axis according to the input signal.

[0074] The flicker control device 60 sends the modulation signal M to the light source 10-i. The light source 10-i flickers according to the modulation signal M. The optical signal S-i refers to the signal represented by the flicker pattern of the light source 10-i as expressed by the modulation signal M.

[0075] The receiving device 20 receives the light signal S-i via the event camera 21. Specifically, the brightness change associated with the light signal S-i is detected by the EV sensor built into the event camera 21 as an event. The receiving device 20 transmits the event data EVD to the signal separation unit 31 within the information processing device 30.

[0076] exist Figure 10 In this embodiment, the information processing device 30 is included within the receiving device 20. The information processing device 30 performs a series of subsequent processes based on the received event data EVD. The information processing device 30 can also be a different device from the receiving device 20 (e.g., an external server). In this case, the receiving device 20 and the information processing device 30 exchange information via communication.

[0077] The signal separation unit 31 obtains the light source image position IP-i by performing a process that separates the light signal S-i contained in the event data EVD from the noise N (the above-described signal separation). The light source image position IP-i is output to the position inference unit 33.

[0078] The demodulation unit 32 demodulates the optical signal S-i separated by the signal separation unit 31 to obtain object information TI-i. The demodulation unit 32 then passes the object information TI-i to the position inference unit 33.

[0079] The position inference unit 33 obtains the absolute position AP-i of the light source based on the object information TI-i. For example, in a situation where... Figure 6 , Figure 7 In the case where the absolute position AP-i of the light source is directly sent as object information TI-i, the position inference unit 33 directly obtains the absolute position AP-i of the light source. Furthermore, in cases such as... Figure 8When the identification information is sent as object information TI-i, the information processing device 30 accesses the database DB to obtain the absolute position AP-i of the light source corresponding to the identification information. The information processing device 30 obtains n sets of data sets of the absolute position AP-i of the light source 10-i and the image position IP-i of the light source. The information processing device 30 infers the position and orientation of the event camera 21 in the absolute coordinate system based on the geometric relationship between these n sets of data sets.

[0080] The optical wireless communication system 1 includes one or more processors (hereinafter referred to simply as "processors"). The processors are included in the flicker control device 60 and the information processing device 30, and perform various processes required by the optical wireless communication system 1. The processors perform processes such as obtaining the absolute position AP-i of the light source, the image position IP-i of the light source, light source guidance processing, and camera position inference. Examples of processors include general-purpose processors, special-purpose processors, CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), integrated circuits, conventional circuits, and / or combinations thereof. A processor can also be referred to as a circuitry or processing circuitry.

Claims

1. An optical wireless communication system, wherein, have: Multiple light sources, including a first light source and a second light source; The receiving device includes a camera; and One or more processors The aforementioned multiple light sources each transmit optical wireless communication signals. The receiving device described above is configured to receive the optical wireless communication signal via the camera described above. The aforementioned one or more processors are configured to perform light source guidance processing when the first image acquired by the camera includes the first light source but does not include the second light source. The above light source guiding process includes: In response to receiving the first optical wireless communication signal transmitted by the first light source, the absolute position of the first light source and the absolute position of the second light source are obtained. The guiding direction for capturing the second light source into the field of view of the camera is determined at least based on the absolute positions of the first and second light sources; and The aforementioned guidance direction is communicated via the aforementioned receiving device.

2. The optical wireless communication system according to claim 1, wherein, The aforementioned first optical wireless communication signal includes information indicating the absolute position of the first light source and information indicating the absolute position of the second light source.

3. The optical wireless communication system according to claim 1, wherein, The above-mentioned light source guiding process also includes: Obtain receiving location limiting information that limits the location of the aforementioned receiving device; and The guiding direction is determined at least based on the absolute position of the first light source, the absolute position of the second light source, and the receiving position limitation information.

4. The optical wireless communication system according to claim 3, wherein, The aforementioned first optical wireless communication signal is transmitted to a first communication range defined in a specific direction. The aforementioned receiving location limitation information includes information indicating the aforementioned first communication range, i.e., first communication range information.

5. An optical wireless communication system, wherein, have: Multiple light sources, including a first light source and a second light source; The receiving device includes a camera; and One or more processors The aforementioned first light source transmits a first optical wireless communication signal. The aforementioned first optical wireless communication signal is transmitted to a first communication range defined in a specific direction. The receiving device described above is configured to receive the first optical wireless communication signal via the camera described above. The aforementioned first optical wireless communication signal includes information indicating a guiding direction for capturing the aforementioned second light source within the field of view of the camera. The aforementioned one or more processors are configured to perform the light source guidance process. The above light source guiding process includes: The guiding direction is determined by referring to the first communication range and the absolute position of the second light source, i.e., the absolute position of the second light source; and The aforementioned guidance direction is communicated via the aforementioned receiving device.

Images