Visible light communication system

Abstract

A visible light communication system 100 comprises an apparatus 10 and a portable terminal 30. The apparatus 10 includes: an indicator unit 11 which is an indicator provided on a surface of a housing of the apparatus 10; a first control unit 12 which generates encoded information obtained by encoding output information which is arbitrary information; and a light emission unit 14 which is provided on a surface on the side of the apparatus where the indicator unit is provided and flashes on the basis of the encoded information. The portable terminal 30 includes: an image recognition unit 31 that recognizes an image of the indicator unit 111 in an image obtained by imaging the surface on the side of the apparatus 10 where the light emission unit 14 is provided, and recognizes an image of the light emission unit 14 in the image on the basis of the position of the image of the indicator unit 11; and a second control unit 33 that generates light emission information on the basis of the image of the light emission unit 14 and generates decoded information corresponding to the output information by decoding the light emission information.

Description

Visible Light Communication System CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2023-127351, filed on August 3, 2023, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to visible light communication systems.

[0003] When trying to obtain information about an information device that does not have a display, it is possible to connect an external device, such as a laptop computer, that has a display to the information device and display the information on the display of the external device. In this case, peripheral devices, specialized knowledge, time, etc. are required to connect the external device to the information device.

[0004] Therefore, if an information device is equipped with an LED (light-emitting diode) as an indicator, information about the information device, such as whether the power supply of the information device is on or off, can be obtained by the blinking of the LED. Furthermore, each of Patent Documents 1 and 2 describes that a camera generates an image by capturing the blinking of an LED equipped in the information device, and a terminal obtains more detailed information about the information device based on the blinking of the LED shown in the image.

[0005] JP 2008-165453 A JP 2020-156131 A

[0006] However, in the above-mentioned technology, a specific method for recognizing the image of the LED in the image generated by the camera is not clear, and the relative positions of the camera and the LED must be fixed in order to recognize the image of the LED in the image. Therefore, the relative positions of the light receiving device such as the camera and the light emitting unit such as the LED must be fixed, which sometimes makes it difficult to easily obtain information about the device.

[0007] The present disclosure has been made in consideration of the above-described problems, and an object of the present disclosure is to provide a visible light communication system that can easily acquire information about devices.

[0008] In order to solve the above problem, one embodiment of a visible light communication system is a visible light communication system comprising an apparatus and a mobile terminal, wherein the apparatus has a sign unit which is a sign provided on the surface of a housing of the apparatus, a first control unit which generates encoded information obtained by encoding output information which is arbitrary information, and a light-emitting unit which is provided on the surface of the apparatus on the side on which the sign unit is provided and which blinks based on the encoded information, and the mobile terminal has an image recognition unit which recognizes an image of the sign unit in an image captured of the surface of the apparatus on the side on which the light-emitting unit is provided, and recognizes the image of the light-emitting unit in the image based on the position of the image of the sign unit, and a second control unit which generates light-emitting information based on the image of the light-emitting unit and decodes the light-emitting information to generate decoded information corresponding to the output information.

[0009] In addition, the visible light communication system of one embodiment further includes a light receiving device that generates the image and a display device that displays the decoded information, and the mobile terminal, the light receiving device, and the light receiving device may be constituted by a smartphone.

[0010] In addition, the visible light communication system according to one embodiment further includes a light receiving device that generates the image and a display device that displays the decoded information, and the light receiving device and the display device may be configured as augmented reality glasses.

[0011] In addition, in one embodiment of the visible light communication system, the sign unit may indicate identification information for identifying the device, and the second control unit may identify the device based on the sign unit.

[0012] According to the visible light communication system according to the present disclosure, information about a device can be easily obtained.

[0013] Fig. 2 is a block diagram showing an example of a configuration of a visible light communication system according to an embodiment of the present disclosure. Fig. 3 is a diagram showing an external appearance of the device shown in Fig. 1. Fig. 4 is a diagram showing an example of an image captured of the device shown in Fig. 2. Fig. 5 is a diagram showing another example of an image captured of the device shown in Fig. 2. Fig. 6 is a sequence diagram showing an example of an operation of the visible light communication system shown in Fig. 1. Fig. 7 is a hardware block diagram of a model learning device.

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0015] <Configuration of Visible Light Communication System> As shown in FIG. 1 , a visible light communication system 100 according to this embodiment of the present disclosure includes a device 10, a light receiving device 20, a mobile terminal 30, and a display device 40.

[0016] The device 10 includes a sign unit 11 , a first control unit 12 , a first storage unit 13 , and a light-emitting unit 14 .

[0017] The first control unit 12 is configured by a controller (first controller). The controller may be configured by dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or may be configured by a processor, or may be configured by including both. The first storage unit 13 is configured by a memory. The memory may be a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), or the like. The light-emitting unit 14 may be configured by a member having a light-emitting function. The member having a light-emitting function may be an LED, or the like.

[0018] As shown in Fig. 2, the sign 11 is a sign provided on the surface of the housing of the device 10. The sign 11 is an image IM (see Fig. 3) of the sign 11 included in the image IM generated by capturing the sign 11 by the light receiving device 20. １１ For example, the sign 11 has an arbitrary pattern that can be identified by the image IM of the sign 11 included in the image IM. １１The marking unit 11 may have any pattern that allows an image at a predetermined relative position in the image to be identified from the marking unit 11. The predetermined relative position in the image is a relative position in the image space that corresponds to a predetermined relative position in the real space, which will be described later. The marking unit 11 may also be a mark that indicates identification information for identifying the device 10.

[0019] Returning to FIG. 1 , the first control unit 12 generates coded information by encoding the output information. The output information may be information pre-stored in the first storage unit 13, or may be information input by an input interface provided in the device 10. The first control unit 12 can generate coded information by encoding the output information according to any coding rule. For example, in a configuration in which the output information is a number, the first control unit 12 can generate coded information by encoding the number into a binary number. For example, the output information may be an IP address, and the IP address may be "192.168.0.98".

[0020] The first storage unit 13 stores a first control program. The first control program causes a computer to execute the operations of the first control unit 12, thereby causing the computer to function as the first control unit 12. In other words, the computer functions as the first control unit 12 by executing operations in accordance with the first control program.

[0021] 2, the light-emitting unit 14 is provided on the surface of the device 10 on the side where the sign unit 11 is provided. Specifically, the light-emitting unit 14 is provided on the surface at a predetermined relative position with respect to the sign unit 11 in real space.

[0022] The light-emitting unit 14 blinks based on the coded information generated by the first control unit 12. For example, the light-emitting unit 14 can be turned on at a predetermined time interval when the coded information expressed in binary is "1" and turned off when the coded information is "0." The predetermined time interval can be, for example, 100 milliseconds. In this case, the inventors' experiments confirmed that it takes about four seconds for the light-emitting unit 14 to turn on and off based on the coded information indicating the binary number obtained by coding the above-mentioned IP address "192.168.0.98."

[0023] The light receiving device 20 is configured with a camera. The light receiving device 20 generates an image IM (see FIG. 3 ) by capturing an image of the surface of the device 10 on which the light emitting unit 14 is provided. The image IM may be a moving image. The moving image may show the blinking of the light emitting unit 14 described above. The light receiving device 20 may also output the image IM to the mobile terminal 30.

[0024] The mobile terminal 30 includes an image recognition unit 31 , a second control unit 32 , a second storage unit 33 , and an output unit 34 .

[0025] The image recognition unit 31 and the second control unit 32 are configured by a controller (second controller). The second storage unit 33 is configured by a memory. The output unit 34 is configured by an output interface. The output interface may be, for example, an interface for outputting information to a display, a speaker, or the like. The output interface may include a communication interface.

[0026] The image recognition unit 31 recognizes an image IM of the sign 11 in the image IM, which is generated by the light receiving device 20 and is an image of the surface of the device 10 on which the light emitting unit 14 is provided. １１ At this time, the image recognition unit 31 can recognize the image of the sign unit 11 by a known method.

[0027] The image recognition unit 31 also recognizes the image IM of the sign 11 in the image IM. １１ Based on the position of the light emitting unit 14 in the image IM, １４Specifically, the image recognition unit 31 recognizes the image IM of the sign 11 in the image IM. １１ The image IM of the light emitting unit 14 is projected from the position of the light emitting unit 14 at a predetermined relative position within the image. １４ Recognize as such.

[0028] The second control unit 32 detects the image IM of the light emitting unit 14 recognized by the image recognition unit 31. １４ The second control unit 32 generates light emission information based on the image IM of the light emitting unit 14. １４ For example, the second control unit 32 generates light emission information based on the luminance of the image IM of the light emitting unit 14 at predetermined time intervals. １４ The second control unit 32 can determine whether the brightness of the image IM of the light-emitting unit 14 is equal to or greater than a threshold value (i.e., whether the light-emitting unit 14 is turned on). １４ is determined to be equal to or greater than the threshold value (i.e., the light-emitting unit 14 is turned on), the light-emitting information is generated as "1" and the image IM of the light-emitting unit 14 is １４ If it is determined that the luminance of the light emitting unit 14 is less than the threshold value (i.e., the light emitting unit 14 is turned off), the light emitting unit 14 generates light emitting information as "0".

[0029] Furthermore, the second control unit 32 generates decoded information by decoding the light-emitting information. Specifically, the second control unit 32 generates decoded information by decoding the light-emitting information according to a decoding rule corresponding to the encoding rule used when the first control unit 12 encoded the output information. Therefore, the decoded information is information equivalent to the output information described above. For example, as described above, in a configuration in which the output information is numbers and the first control unit 12 generates encoded information by encoding the numbers into binary numbers, the second control information can generate decoded information by decoding the light-emitting information into decimal numbers. More specifically, as described above, if the first control unit 12 encodes the IP address "192.168.0.98" into binary numbers as output information, the second control unit 32 generates the binary number of the IP address "192.168.0.98" as light-emitting information and decodes the light-emitting information into decimal numbers to obtain the IP address "192.168.0.98" as decoded information.

[0030] Furthermore, as described above, in a configuration in which the indicator 11 indicates identification information for identifying the device 10 , the second control unit 32 can identify the device 10 based on the indicator 11 .

[0031] The second storage unit 33 is configured with a memory. The second storage unit 33 stores a second control program. The second control program causes a computer to execute the operations of the second storage unit 33, thereby causing the computer to function as the second storage unit 33. In other words, the computer functions as the second storage unit 33 by executing operations in accordance with the second control program.

[0032] The output unit 34 outputs the decoded information. The output unit 34 may output the decoded information to the display device 40, or may output the decoded information to an arbitrary device via a communication network. Furthermore, as described above, in a configuration in which the label unit 11 indicates identification information for identifying the device 10, the output unit 34 can output information indicating the device 10 identified by the second control unit 32. The output unit 34 may output the information indicating the device 10 to the display device 40, or may output the information indicating the device 10 to an arbitrary device via a communication network.

[0033] The display device 40 is configured by a display interface. The display interface can be, for example, an organic EL (Electro Luminescence) or liquid crystal panel. The display device 40 displays the decoded information output by the output unit 34.

[0034] The light receiving device 20, the mobile terminal 30, and the display device 40 described above may be configured as a single unit, for example, as a smartphone. The light receiving device 20 and the display device 40 may also be configured as a single unit, for example, as AR (Augmented Reality) glasses. In such a configuration, when the light receiving device 20 in the AR glasses generates an image IM, the image IM is transmitted to the mobile terminal 30 via a communication network, and the display device 40 in the AR glasses can receive and display the output information output by the mobile terminal 30 as described above using the image IM.

[0035] <Operation of Visible Light Communication System> Here, the operation of the visible light communication system 100 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the operation of the visible light communication system 100 according to this embodiment. The operation of the visible light communication system 100 described with reference to Fig. 5 corresponds to an example of a visible light communication method executed by the visible light communication system 100 according to this embodiment.

[0036] In step S11, the first control unit 12 generates coded information by coding output information, which is any information.

[0037] In step S12, the light emitting section 14 blinks based on the coded information.

[0038] In step S13, the light receiving device 20 generates an image IM by capturing an image of the surface of the device 10 on which the light emitting unit 14 is provided.

[0039] In step S14, the light receiving device 20 outputs the image IM generated in step S13 to the mobile terminal 30.

[0040] In step S15, the image recognition unit 31 recognizes the image of the sign unit 11 in the image IM captured from the surface of the device 10 on which the light-emitting unit 14 is provided.

[0041] In step S16 , the image recognition unit 31 recognizes the image of the light-emitting unit 14 in the image IM based on the position of the image of the sign unit 11 .

[0042] In step S17 , the second control unit 32 generates light emission information based on the image of the light emitting unit 14 .

[0043] In step S18, the second control unit 32 decodes the light emission information to generate decoded information corresponding to the above-mentioned output information.

[0044] In step S198, the output unit 34 outputs the output information. For example, the output unit 34 can output the decoded information to the display device 40.

[0045] In step S20, the display device 40 displays the decoded information.

[0046] As described above, the visible light communication system 100 according to this embodiment comprises a device 10 and a mobile terminal 30. The device 10 comprises a sign unit 11, which is a sign provided on the surface of the housing of the device 10, a first control unit 12 that generates encoded information obtained by encoding output information, which is arbitrary information, and a light-emitting unit 14 that is provided on the surface of the device 10 on the side where the sign unit 11 is provided and that flashes based on the encoded information. The mobile terminal 30 comprises an image recognition unit 31 that recognizes an image of the sign unit 11 in an image IM captured of the surface of the device 10 on the side where the light-emitting unit 14 is provided, and recognizes an image of the light-emitting unit 14 in the image IM based on the position of the image of the sign unit 11, and a second control unit 32 that generates light-emitting information based on the image of the light-emitting unit 14 and decodes the light-emitting information to generate decoded information corresponding to the output information.

[0047] This allows the visible light communication system 100 to easily obtain information about the device. Specifically, the visible light communication system 100 allows the light-emitting unit 14 of the device 10 to blink according to a predetermined encoding rule, so that the device 10 can output information to the mobile terminal 30 even if the device 10 and the mobile terminal 30 are not connected via a communication network. At this time, the sign unit 11 is used to detect the image IM of the light-emitting unit 14 in the image IM generated by the light-receiving device 20. １４Therefore, the portable terminal 30 can recognize the blinking of the light-emitting unit 14 with high accuracy, and accordingly, can recognize information about the device 10 with high accuracy. In particular, even when the device 10 includes a light-emitting member A as shown in FIG. 2 in addition to the light-emitting unit 14, the portable terminal 30 can detect the image IM of the light-emitting unit 14 with high accuracy. １４ The light-emitting member A is a light-emitting member that is different from the light-emitting section 14.

[0048] As described above, the mobile terminal 30 uses the marker 11 to display the image IM of the light-emitting unit 14 in the image IM. １４ Therefore, even if the light-receiving device 20 is not provided at a fixed position relative to the light-emitting unit 14 (for example, even if the direction of the optical axis of the light-receiving device 20 is shifted from the normal direction to the surface of the device 10 including the light-emitting unit 14 as shown in FIG. 4 ), the mobile terminal 30 can recognize the blinking of the light-emitting unit 14 with high accuracy, and therefore can recognize information about the device 10 with high accuracy.

[0049] <Program> The above-described device 10 and mobile terminal 30 can be realized by a computer. Furthermore, a program for causing a computer to function as each of the device 10 and the mobile terminal 30 may be provided. Furthermore, the program may be stored in a storage medium or provided via a network. FIG. 6 is a block diagram showing a schematic configuration of a computer 501 functioning as the device 10. A computer functioning as the mobile terminal 30 may have a similar configuration to the computer 501. Here, the computer 501 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, or the like. The program instructions may be program code, code segments, or the like for executing necessary tasks.

[0050] 6 , the computer 501 includes a processor 510, a read-only memory (ROM) 520, a random access memory (RAM) 530, a storage 540, an input unit 550, an output unit 560, and a communication interface (I / F) 570. Each component is communicably connected to one another via a bus 580. The processor 510 is specifically a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), a digital signal processor (DSP), a system on a chip (SoC), or the like, and may be configured by multiple processors of the same type or different types.

[0051] The processor 510 controls each component and performs various arithmetic processing. That is, the processor 510 reads a program from the ROM 520 or the storage 540 and executes the program using the RAM 530 as a work area. The processor 510 controls each component and performs various arithmetic processing in accordance with the program stored in the ROM 520 or the storage 540. In the above-described embodiment, the program according to the present disclosure is stored in the ROM 520 or the storage 540.

[0052] The program may be stored in a storage medium readable by the computer 501. Using such a storage medium, the program can be installed in the computer 501. Here, the storage medium on which the program is stored may be a non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. Furthermore, the program may be downloaded from an external device via a network.

[0053] The ROM 520 stores various programs and various data. The RAM 530 temporarily stores programs or data as a working area. The storage 540 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0054] The input unit 550 includes one or more input interfaces that receive input operations from a user and acquire information based on the user operations. For example, the input unit 550 may be, but is not limited to, a pointing device, a keyboard, a mouse, etc.

[0055] The output unit 560 includes one or more output interfaces that output information. For example, the output unit 560 is a display that outputs information as a video or a speaker that outputs information as an audio, but is not limited to these. Note that if the output unit 560 is a touch panel display, it also functions as the input unit 550.

[0056] The communication interface 570 is an interface for communicating with an external device.

[0057] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-described embodiments, and various modifications or alterations are possible without departing from the scope of the claims. For example, multiple building blocks shown in the block diagrams of the embodiments can be combined into one, or one building block can be divided.

[0058] REFERENCE SIGNS LIST 10 Device 11 Sign unit 12 First control unit 13 First memory unit 14 Light emitting unit 20 Light receiving device 30 Portable terminal 31 Image recognition unit 32 Second control unit 33 Second memory unit 34 Output unit 40 Display device 100 Visible light communication system 501 Computer 510 Processor 520 ROM 530 RAM 540 Storage 550 Input unit 560 Output unit 570 Communication interface 580 Bus

Claims

1. A visible light communications system comprising a device and a mobile terminal, wherein the device has: a sign unit which is a sign provided on the surface of a housing of the device; a first control unit which generates encoded information obtained by encoding output information which is any information; and a light-emitting unit which is provided on the surface of the device on which the sign unit is provided and flashes based on the encoded information; and the mobile terminal has: an image recognition unit which recognizes an image of the sign unit in an image captured of the surface of the device on which the light-emitting unit is provided, and recognizes the image of the light-emitting unit in the image based on the position of the image of the sign unit; and a second control unit which generates light-emitting information based on the image of the light-emitting unit and generates decoded information corresponding to the output information by decoding the light-emitting information.

2. The visible light communication system of claim 1, further comprising: a light receiving device that generates the image; and a display device that displays the decoded information, wherein the mobile terminal, the light receiving device, and the light receiving device are constituted by a smartphone.

3. The visible light communication system of claim 1, further comprising: a light receiving device that generates the image; and a display device that displays the decoded information, wherein the light receiving device and the display device are constituted by augmented reality glasses.

4. A visible light communication system as described in any one of claims 1 to 3, wherein the sign unit indicates identification information for identifying the device, and the second control unit identifies the device based on the sign unit.

Images