Image processing apparatus, optical communication system, program product, and image processing method

Abstract

Provided is an image processing device, an optical communication device, an optical communication system, a recording medium, and an image processing method. An image processing device (5) includes a processing section that acquires images (44) that are continuous in time, acquires the position of a visible light source in an on state among the continuous images (44) as the position of a mobile body (3), and acquires identification information of the mobile body based on the flicker pattern of invisible light of an invisible light source provided to the mobile body (3).

Description

[0001] Cross-referencing of related applications

[0002] This application claims priority and interest in Japanese Patent Application No. 2021-207493, filed on December 21, 2021. The entire description, claims, and drawings of Japanese Patent Application No. 2021-207493 are incorporated herein by reference. Technical Field

[0003] This invention relates to an image processing apparatus, an optical communication apparatus, an optical communication system, a recording medium, and an image processing method. Background Technology

[0004] The following technique is known: a light source, which acts as a light transmitting device, changes its illumination pattern according to a predetermined pattern; a camera, which acts as a light receiving device, captures and analyzes the illumination pattern, thereby transmitting information.

[0005] As a document describing this technology, there is Japanese Patent Publication No. 2020-523752. Japanese Patent Publication No. 2020-523752 relates to a light transmission device that uses visible light LEDs (Light Emission Devices) and infrared light LEDs as light sources.

[0006] [The technical problem the invention aims to solve]

[0007] However, when a moving object is equipped with a flickering visible light source, and the trajectory of the moving object is analyzed by tracking the movement of this visible light source, there are concerns that the flickering of the visible light source may affect the human eye and cause discomfort. Using an invisible infrared light source has also been considered, but if the infrared light source flickers, its position cannot be determined when it is off, thus hindering high-precision image processing-based light source tracking. There is room for improvement in the existing technology regarding tracking performance. Summary of the Invention

[0008] The present invention was made in view of this situation, and its purpose is to achieve optical communication with excellent tracking performance and without the unpleasant flicker of visible light sources.

[0009] [Methods used to solve technical problems]

[0010] To achieve the above objectives, an image processing apparatus processing unit according to one aspect of the present invention acquires a plurality of images that are sequentially continuous in time, acquires the position of a visible light source in the plurality of images that is set in the lit state of a moving object as the position of the moving object, and acquires identification information of the moving object based on the flickering pattern of an invisible light source in the plurality of images.

[0011] [The effects of the invention]

[0012] According to the optical communication device, optical communication system, recording medium, and image processing method of the present invention, optical communication can be achieved without the unpleasant sensation of light flicker and with excellent light source tracking. Attached Figure Description

[0013] Figure 1 This is a schematic diagram illustrating the structure of an optical communication system according to an embodiment of the present invention.

[0014] Figure 2 It means Figure 1 A block diagram of the hardware structure of the LED transmitting device.

[0015] Figure 3 It means Figure 1 A block diagram illustrating the hardware structure of the camera device and the image processing device.

[0016] Figure 4 It means Figure 1 The functional block diagram of the CPU of the image processing device for performing control processing.

[0017] Figure 5 It indicates that the camera device is shooting. Figure 1 A schematic diagram showing how a moving object moves.

[0018] Figure 6 It means to reflect Figure 5 The image shows an example of a moving object with bright LEDs.

[0019] Figure 7 It means Figure 6 The image shows the color pattern of the highlights, especially the green highlights.

[0020] Figure 8 It means Figure 6 The image shows the colors of the bright spots, especially the infrared bright spots.

[0021] Figure 9 It means Figure 6 The image shows the colors in the highlight image.

[0022] Figure 10 This is a flowchart that explains the outline of the image processing performed by the image processing apparatus of this embodiment.

[0023] Figure 11 This is a flowchart illustrating in detail the decoding process performed by the image processing apparatus of this embodiment. Detailed Implementation

[0024] Hereinafter, with reference to the accompanying drawings, the optical communication system 1 according to the embodiments of the present invention will be described.

[0025] Figure 1 This is a schematic diagram illustrating the structure of an optical communication system 1 according to an embodiment of the present invention. For example... Figure 1 As shown, the optical communication system 1 includes an LED transmitting device 2, a camera device 4, an image processing device 5, and a service management device 6. Furthermore, in this specification, the term "system" means an overall apparatus comprising multiple devices, multiple units, etc.

[0026] LED transmitting device 2 is an example of a light transmitting device utilizing a light source. An LED is an example of a light source; any light source that emits light is acceptable, and its type is not limited. For example, a light bulb or a laser can be used. As a light source, it can be visible light (as seen by the eye) or invisible light (as seen by the eye). Invisible light includes, for example, infrared light or ultraviolet light.

[0027] LED transmitting device 2 is a light transmitting device mounted on a mobile body 3. The mobile body 3 is, for example, a forklift. Figure 1 The three mobile units 3a~3c are represented in the middle.

[0028] The LED transmitting device 2 in this embodiment is configured to control the continuous illumination of visible light for tracking and the flashing or extinguishing of infrared light for information transmission such as identification numbers.

[0029] The LED transmitting device 2 keeps visible light constantly lit and, in parallel, converts information transmitted via optical communication into variations of infrared light and emits light. The position of the visible light source is obtained as the position of the moving body 3, and information based on invisible light, i.e., infrared light, is also acquired. The information transmitted via optical communication is, for example, the identification information of the moving body 3. Figure 1 The LED transmitting devices 2a~2c are respectively configured on the mobile bodies 3a~3c, and transmit their respective identification information through optical communication.

[0030] The camera device 4 is positioned to capture the movement of the moving body 3. For example, the camera device 4 is a digital camera. The camera device 4 captures images 44 (moving images) in a continuous sequence over time. Figure 1In the example shown, the camera device 4 has an FHD (Full High Definition) pixel count (1920 pixels in the X direction and 1080 pixels in the Y direction). The identification numbers (IDs) of the three moving objects 3 (forklifts) and their positions in the image captured by the camera device 4 are represented by the position of the pixels. For example, the moving object 3a in the upper left of the figure is identified as having an identification number (ID) of 101, being the 280th pixel in the 1920 pixels in the X direction, and the 590th pixel in the 1080 pixels in the Y direction.

[0031] Image processing device 5 is an information processing device that tracks the position of the always-lit visible light LED 29 on an image and analyzes the flashing or on / off pattern of the infrared light LED 30 that flashes or turns on / off at the same position as the visible light LED 29. Camera device 4 is used as a light receiving device because it can... Figure 1 That way, information from the LED transmitting device 2 can be obtained from multiple locations simultaneously, and the positions on the image of the LED transmitting device 2 can be replaced with actual position coordinates and the position of the moving body 3 can be analyzed.

[0032] Visible light LED29 is an example of a visible light source that emits visible light. There are no limitations on the type of light source that emits visible light. For example, a light bulb or a laser can be used. Infrared light LED30 is an example of an infrared light source that emits infrared light. There are no limitations on the type of light source that emits infrared light. For example, a light bulb with an infrared filter or an infrared laser can be used.

[0033] The business management device 6 is a host computer that analyzes the operation status and the handling of goods transported by the mobile body 3 based on the analysis results of the movement of the mobile body 3. Figure 1 In this configuration, the business management device 6 and the image processing device 5 are configured separately. However, the business management device 6 and the image processing device 5 can also be configured by a single computer or by three or more computers.

[0034] Next, refer to Figure 2 Here is an example illustrating the hardware structure of the LED transmitting device 2.

[0035] Figure 2 It means Figure 1 A block diagram of the hardware structure of the LED transmitting device 2. (See diagram for example.) Figure 2 As shown, the LED transmitting device 2 includes: CPU 21, ROM 22, RAM 23, accelerometer 24, switch 25, dry cell battery 26, power controller 27, LED driver 28, visible light LED 29, and infrared light LED 30.

[0036] CPU 21 performs various processes based on the program recorded in ROM 22 or the program loaded into RAM 13. CPU 21 is implemented by a processor that performs arithmetic processing. The processor includes, for example, a structure composed of various processing devices such as single processors, multi-processors, and multi-core processors, as well as a structure obtained by combining these various processing devices with processing circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field-Programmable Gate Array).

[0037] RAM23 also stores data required by CPU21 for various processing tasks.

[0038] Accelerometer 24 senses the acceleration of the moving body 3. Switch 25 switches the CPU 21 on / off and resets it. Dry cell battery 26 provides power. Power controller 27 controls the power supply to the LED driver, etc.

[0039] The LED driver 28, based on instructions from the CPU 21, provides power to the visible light LED 29 and the infrared light LED 30, and controls the lighting, blinking, or on / off state of the visible light LED 29 and the infrared light LED 30.

[0040] Visible light LED 29 and infrared light LED 30 are arranged close to each other. The visible light LED 29 and infrared light LED 30, for example due to the curved surface or unevenness of the glass cover, cause light to refract and scatter outwards. When reflected in the camera device 4, the visible light LED 29 and infrared light LED 30 are identified as information indicating a single location.

[0041] Visible light LED 29 is always lit, while information is transmitted by the blinking or on / off state of infrared light LED 30. For example, based on instructions from CPU 21, infrared light LED 30 emits a corresponding lighting mode, i.e., a blinking or on / off mode, that is, establishes with the identification number of the mobile body 3 stored in ROM 22. In addition, by making the blinking or on / off mode of infrared light LED 30 redundant, it is easier to distinguish from noise, and more reliable information (such as identification number) can be extracted.

[0042] Reference Figure 3 This illustrates an example of the hardware structure of the image processing device 5. For example... Figure 3 As shown, the image processing device 5 includes: CPU 51, ROM 52, RAM 53, input unit 54, output unit 55, storage unit 56, and communication unit 57.

[0043] CPU 51 performs various processes based on programs recorded in ROM 52 or programs loaded from storage unit 56 into RAM 53.

[0044] The RAM53 also stores data required by the CPU51 to perform various processes.

[0045] CPU 51, ROM 52, and RAM 53 are interconnected via a bus. Input / output interfaces are also connected to this bus. A camera device 4, an input unit 54, an output unit 55, a storage unit 56, and a communication unit 57 are connected to the input / output interfaces.

[0046] The input unit 54 includes various buttons, a microphone, etc., to input various information according to user instructions. The output unit 55 includes a display, a speaker, etc., to output images 44 and sound. The storage unit 56 includes a hard disk or flash memory, etc., to store various data of the images 44. The communication unit 57 is a network interface that controls communication between the service management device 6 and other devices via a network including the Internet.

[0047] The camera device 4 includes: a color filter 41, an optical lens 42, and an image sensor 43. The image sensor 43 is a single sensor that can receive light from both visible and invisible light sources and output a captured image 44.

[0048] Color filter 41 is a filter that allows red, green, and blue light to pass through, formed one by one in each pixel of the image sensor 43. The combination of red, green, and blue light forms a unit.

[0049] The optical lens unit 42 includes lenses that focus light, such as a focusing lens and a zoom lens, for capturing images of a subject. The focusing lens is a lens that forms an image of the subject on the light-receiving surface of the image sensor 43. The zoom lens is a lens that allows the focal length to change freely within a certain range. The optical lens unit 42 also includes peripheral circuitry for adjusting setting parameters such as focus point, exposure, and white balance as needed.

[0050] Image sensor 43 includes photoelectric conversion elements and AFE (Analog Front End). The photoelectric conversion elements may include, for example, CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion elements. An image of the subject is incident on the photoelectric conversion element through the optical lens section 42. For this purpose, the photoelectric conversion element performs photoelectric conversion (imaging) on ​​the subject image and accumulates the image signal for a certain period of time, then sequentially provides the accumulated image signal as an analog signal to the AFE. The AFE performs various signal processing operations, such as A / D (Analog / Digital) conversion, on this analog image signal.

[0051] Digital signals are generated through various signal processing methods and output as output signals of the camera device 4. These output signals of the camera device 4 are hereinafter referred to as "camera images". The data of the camera images is appropriately provided to the CPU 51, etc.

[0052] Next, refer to Figure 4 The processing unit 60, implemented by the CPU 51 of the image processing apparatus 5, will be described below. The processing unit 60 may also be referred to as a processor. The processing unit 60 includes an image acquisition unit 61, a tracking processing unit 62, and a parsing processing unit 63.

[0053] The image acquisition unit 61 acquires images 44 captured by the camera device 4. The camera device 4, for example, has an FHD (Full High Definition, 1920x1080 pixels) imaging element and captures, for example, 30 images 44 per second. The image acquisition unit 61 acquires this digital information from the camera device 4.

[0054] The tracking processing unit 62 identifies consistently lit bright spots 45, particularly green bright spots 45G, that are consistently lit in green, within the image 44 acquired by the image acquisition unit 61. For example, the tracking processing unit 62 extracts signals only from the green sub-pixels in the image 44 and identifies pixels whose green signals consistently exceed a predetermined threshold. Furthermore, the tracking processing unit 62 tracks the green bright spots 45G that continuously change position in the image 44 and determines their positions within the image 44. For example, the tracking processing unit 62 determines which pixel in the image 44 is the location of the green bright spot. Figure 1 In the example, for instance, the position of the bright spot 45 of the forklift with ID101 in the upper left corner is determined to be the position of the 280th pixel in the X direction and the 590th pixel in the Y direction.

[0055] The analysis processing unit 63 analyzes the flashing or on / off pattern of the infrared LED 30 that flashes or turns on at the same location as the green dot 45G extracted by the tracking processing unit 62, and determines, for example, the identification number (ID, Idendification) of the LED transmitting device 2.

[0056] Here, using Figures 5 to 9 This illustrates an example of the movement of the moving body 3, the imaging device 4, and the light signal received by the image processing device 5. For example... Figure 5 As shown, the camera device 4 captures the moving body 3 moving from left to right. The LED transmitting device 2 of the moving body 3 keeps the visible light LED 29 constantly lit and makes the infrared light LED 30 flash or turn on / off.

[0057] By using a green LED for the always-on visible light LED 29, changes in infrared light are easily captured. In image sensors 43 using conventional color filters 41, infrared cutoff filters are used. One embodiment of the present invention involves an image sensor 43 that does not utilize an infrared cutoff filter, or uses a filter with weak infrared cutoff capability. In this case, any image sensor 43 that receives light passing through the red, green, and blue color filters 41 will also respond to infrared light. Typically, to achieve white balance, the outputs of blue and red are amplified compared to the higher light intensity of green.

[0058] Since the intensity of infrared light received is the same across red, green, and blue, the amplified blue-red signal is relatively strong, resulting in an output signal resembling violet. Therefore, for example, when infrared and green light are simultaneously received, the overall effect is close to white. When green is illuminated but infrared light is absent, it remains green. Furthermore, when green is always illuminated and infrared light flickers or fades, it results in a flickering or fading effect between white and green. Thus, the combination of green and infrared light enables decoding of simple color shifts (white and green), exhibiting excellent robustness.

[0059] In this state, since the transmitter does not emit red or blue light, the human eye can only see the "green" light that is always on without flashing, thus reducing the discomfort.

[0060] Visible light LED 29 that remains constantly lit and infrared light LED 30 that flashes or turns on / off, as shown. Figure 6 As shown, the trajectory of bright spot 45 is projected into image 44. Visible light LED 29, for example, is green and is identified as green bright spot 45G, as... Figure 7 As shown, it remains lit throughout the movement. To indicate a green dot 45G, Figure 7 The green dot 45G in the image depicts a diagonal line to the right. Infrared LED 30, for example... Figure 8 The light flashes or dims as shown, and as described above, it is perceived as purple by camera device 4. To represent the infrared light spot 45P, in... Figure 8 The infrared bright spot 45P in the image depicts a diagonal line to the left. Combine them, as... Figure 9 As shown, since the green visible light LED 29 is always lit when the green visible light LED 29 and the infrared light LED 30 are lit, and the white bright spot 45W and the infrared light LED 30 are off, the green visible light LED 29 is identified as the green bright spot 45G. To represent the white bright spot 45W, Figure 9 The middle line depicts both the right-hand and left-hand diagonal lines. For example... Figure 9 As shown, bright spot 45 is identified as the bright spot 45 that flashes or turns on between green bright spot 45G and white bright spot 45W.

[0061] Next, refer to Figure 10 as well as Figure 11 The process from acquiring the image 44 from the camera device 4 to recognizing the moving body 3 and determining its position on the image will be explained. Figure 10 Summary of the description Figure 11 The details are explained in the section on the decoding process of image 44.

[0062] like Figure 10 As shown, if the optical communication system 1 is started, the image acquisition unit 61 of the image processing device 5 acquires the image 44 captured by the camera device 4 (step S101).

[0063] Next, the tracking processing unit 62 extracts the always-lit visible light bright spot 45, more specifically the green bright spot 45G, from the image 44 and performs image analysis processing to determine its position in the image 44 (step S102). In parallel, the analysis processing unit 63 performs image analysis processing to determine the identification number of the moving body 3 associated with the bright spot 45, based on the flashing or fading pattern of the infrared bright spot 45P that flashes or fades at the same position as the green bright spot 45G (step S102). Wherein, once the identification number of the bright spot 45 is determined, it is not necessary to re-obtain the identification number from the flashing or fading pattern of the infrared bright spot 45P, as long as the always-lit green bright spot 45G is tracked.

[0064] Next, the parsing processing unit 63 performs parsing result output processing (step S103) through the output unit 55 of the image processing device 5, and then ends the processing (step end).

[0065] Reference Figure 11 The image acquisition unit 61 of the image processing apparatus 5 performs image acquisition processing to acquire one frame of image 44 from continuously captured images (step S201). Next, the tracking processing unit 62 performs bright spot 45 extraction processing to extract bright spots 45 with green components (step S202), and performs direction inference processing to infer the movement direction of the moving body 3 based on the movement history of bright spots 45 (step S203).

[0066] After step S203, the tracking processing unit 62 performs the following candidate point extraction process: weighting the movement direction, extracting the bright spots 45 of the adjacent green components that include the movement range within the specified range as candidate points, and tracking the movement of the green bright spots 45G (step S204). Since the light from the visible light LED 29 becomes a relatively bright bright spot 45 in the imaging element, the bright spots 45 can be screened to a considerable extent simply by shooting in a darker environment.

[0067] After step S204, the tracking processing unit 62 uses shape filtering to select candidate points that are considered to perform optical communication, and completes the candidate point extraction process (step S204).

[0068] Next, the analysis processing unit 63 performs decoding processing using the change history of infrared light at the bright spot location (step S205). Infrared light is identified as violet in the image sensor 43 as described above. Therefore, the change history of infrared light in image 44 becomes the change history between green and white, a combination of green and violet. When identified as white, infrared light emits light.

[0069] The parsing processing unit 63 performs mode detection processing based on the decoding result to attempt to detect a valid light emission mode (step S206). If a valid light emission mode is detected (step S206: Yes), the parsing processing unit 63 performs recording processing to store the ID of the moving body 3 and its position in the frame from the valid light emission mode in the storage unit 56 (step S207).

[0070] If no valid emission mode is detected based on the decoding result during the detection process (step S206: No), and after storage processing (step S207), a verification process is performed to check whether the mode check of all bright spots 45 is complete (step S208). If it is not yet complete (step S208: No), the process returns to the bright spot extraction process for extracting bright spots 45 with green components (step S202). If the mode check of all bright spots 45 is complete, the processing flow ends (step end).

[0071] As explained above, the image processing apparatus 5 includes: a processing unit 60, which acquires images 44 that are sequential in time, acquires the position of the visible light LED 29 in the sequential images 44 as the position of the moving body 3, acquires the infrared light of the infrared light LED 30 provided on the moving body 3, and acquires information based on the infrared light.

[0072] Therefore, an image processing apparatus 5 can be obtained for optical communication that achieves excellent tracking performance and eliminates the unpleasant feeling of flickering visible light LED 29. In optical communication using a conventional camera, information is transmitted by flashing white LEDs, but if the timing of LED extinguishing increases, it becomes difficult to track the LED. By keeping visible light LED 29 constantly lit, and setting the format such that infrared light LED 30 can freely flash or turn on / off while ensuring tracking performance of the movement of LED transmitting device 2, tracking performance can be ensured while increasing the coding rate. Furthermore, since the infrared light emitted by the flashing or turning on / off infrared light LED 30 is invisible to humans, it will not be an unpleasant cause.

[0073] The image processing device 5 has a single image sensor 43 that acquires the visible light from the visible light LED 29 and the infrared light from the infrared light LED 30.

[0074] Thus, images of the visible light LED 29 and the infrared light LED 30 are projected onto a single image sensor 43. The images of the visible light LED 29 and the infrared light LED 30 can be arranged in the same image 44. A correlation can be established between the visible light LED 29 and the infrared light LED 30.

[0075] The image processing device 5 has a processing unit 60 that tracks the position of the green visible light LED 29G, which is either always lit or on / off, as the position of the moving body 3, and obtains the illumination mode of the infrared light in the infrared region of the infrared light LED 30.

[0076] Therefore, excellent visual recognition is achieved by using the highly sensitive green light of camera device 4. Furthermore, when light is received by camera device 4 along with infrared light, the green signal is suppressed by the white balance adjustment circuit and is perceived as white as a whole. This allows for the flickering or fading of both white and green light, making resolution easy and highly reliable.

[0077] In the image processing device 5, the visible light LED 29 and the infrared light LED 30 are arranged close to each other.

[0078] Visible light from visible light LED 29 and infrared light from infrared light LED 30 can be received by the same pixel in image sensor 43. Infrared light from infrared light LED 30 is received by each sub-pixel of red, green, and blue light, and outputs a signal. Infrared light from infrared light LED 30 provides colors similar to visible light.

[0079] A color signal containing red, green, and blue pixels is created by combining infrared and visible light signals.

[0080] When the processing unit 60 of the image processing apparatus 5 detects a monochromatic color represented by green at the object position, it determines that only the visible light LED 29 and the infrared light source are lit. When white is detected at the object position, it determines that both the visible light LED 29 and the infrared light source are lit.

[0081] Therefore, the visible light LED29 is determined to be always lit, making position tracking easy.

[0082] In addition, the blinking or on / off state of the infrared LED30 is extracted, and high-precision information can be obtained by analyzing the blinking or on / off pattern.

[0083] The LED transmitting device 2 is an LED transmitting device for optical communication mounted on the mobile body 3, and includes: a visible light LED 29, which is controlled to be lit in order to obtain the position of the mobile body 3; and an infrared light LED 30, which is disposed on the mobile body 3 and controlled to be lit so as to emit information corresponding to the mobile body 3.

[0084] Therefore, by receiving light from the visible light LED29 and infrared light LED30 of the camera device 4, the identification number and position of the moving body 3 equipped with the LED transmitting device 2 can be determined. Furthermore, optical communication can be achieved without the unpleasant feeling of flickering and with excellent tracking performance of moving light sources. In conventional camera-based optical communication, information is transmitted using the flickering of a white light source; however, if the timing of the light source's extinguishing increases, it becomes difficult to track the light source. By keeping the visible light LED29 constantly lit, and by allowing the infrared light source to flicker or extinguish freely while ensuring tracking of the moving body 3, tracking performance can be ensured while increasing the coding rate. Moreover, since the infrared light from a flickering or extinguishing infrared light source is invisible to humans, it will not cause any unpleasantness.

[0085] The optical communication system 1 includes: an LED transmitting device 2 having a visible light LED 29 controlled to be lit for obtaining the position of a moving body 3, and an infrared light LED 30 disposed on the moving body 3 and controlled to be lit so as to emit information corresponding to the moving body 3; and an image processing device 5 having a processing unit 60, which acquires images 44 that are sequential in time, acquires the position of the visible light LED 29 in the sequential images 44 as the position of the moving body 3, acquires the infrared light of the infrared light LED 30, and acquires information based on the infrared light.

[0086] Thus, an optical communication system 1 is realized that enables optical communication with excellent tracking performance and without the unpleasant flickering of visible light LED 29.

[0087] The program causes the image processing device 5 to perform the following functions: an image acquisition function, which acquires images 44 that are continuous in time; an acquisition processing function, which acquires the position of the visible light LED 29 in the continuous images 44 as the position of the moving body 3; and an analysis processing function, which acquires the infrared light of the infrared light LED 30 set on the moving body 3 and acquires information based on the infrared light.

[0088] Thus, a program can be implemented that achieves optical communication with excellent tracking performance and without the unpleasant flickering of visible light LED29.

[0089] The image processing method is an image processing method executed by the image processing device 5, comprising: an image acquisition step, acquiring images 44 that are sequential in time; an acquisition processing step, acquiring the position of the visible light LED 29 in the sequential images 44 as the position of the moving body 3; and a parsing processing step, acquiring the infrared light of the infrared light LED 30 set on the moving body 3, and acquiring information based on the infrared light.

[0090] Thus, an image processing method is achieved that enables optical communication with excellent tracking performance and without the unpleasant flickering of visible light LED29.

[0091] Furthermore, the present invention is not limited to the embodiments described above, and variations and improvements that can achieve the purpose of the present invention are included in the present invention.

[0092] For example, a visible light LED 29 emitting a color other than green can also be used. For example, a red visible light LED 29 can also be used as the visible light LED 29, including to attract attention. In this case, in the imaging device 4, the signal can be adjusted by the amplification circuit of the imaging device 4 or the arithmetic circuit of the image processing device 5, as described above, so that when both the infrared light LED 30 and the red visible light LED 29 are lit, they can be recognized as white. In this case, the flickering or on / off mode is white and red flickering or on / off during the resolution processing stage based on the image processing device 5.

[0093] As invisible light, ultraviolet light can be used instead of infrared light. Since ultraviolet light is also invisible to humans, its flickering will not cause discomfort, and information can be obtained from the flickering pattern.

[0094] In this case, the signal can be adjusted in the camera device 4 through the amplification circuit of the camera device 4 or the arithmetic circuit of the image processing device 5, so that when both the green LED 29G and the ultraviolet light are lit, it can be identified as white. In this case, the flickering or on / off mode becomes white and green flickering or on / off during the resolution processing stage based on the image processing device 5.

[0095] In information transmission, the flashing of infrared light can be substituted for the change in the wavelength of infrared light. Since the sensitivity of the camera device 4 depends on the wavelength of infrared light, the change in the wavelength of infrared light can be set as a change in the intensity of the received light signal, or the difference in the intensity of the received light signal in each sub-pixel of red, green and blue, i.e., a color difference, so that the change in the wavelength of infrared light can be recognized.

[0096] The above describes an implementation method for keeping the visible light LED 29 always on. The visible light LED 29 can also have a state where it can be tracked without causing discomfort, i.e., it is normally lit and appropriately turned off. In this case, the same effect as the visible light LED 29 in a constantly lit state can be achieved, allowing for stable tracking without causing discomfort.

[0097] Furthermore, in the above embodiments, the imaging device 4 used in this invention has been described using a digital camera as an example, but it is not particularly limited to this. Additionally, the imaging device 4 and the image processing device 5 have been described as independent devices, but the image processing device 5 may also be integrated into the imaging device 4.

[0098] Furthermore, in each diagram, a functional module can be composed of a single hardware unit, a single software unit, or a combination of both. In other words, Figure 4 The functional structure described is merely illustrative and not particularly limited. That is, as long as the image processing device 5 possesses the function to perform the aforementioned series of processes as a whole, the specific functional module used to achieve this function is not particularly limited. Figure 4 Examples.

[0099] Furthermore, the aforementioned series of processes can be executed either through hardware or software. When the processes are executed through software, the program constituting the software is installed onto the computer from a network or recording medium.

[0100] A computer can be a computer assembled with dedicated hardware. Alternatively, a computer can be a general-purpose personal computer, which can perform various functions by installing various programs.

[0101] Recording media containing such programs include not only removable media arranged separately from the main unit of the device for providing programs to users, but also recording media provided to users in a pre-assembled state with the main unit of the device. Removable media include, for example, magnetic disks (including floppy disks), optical disks, or optical discs. Optical disks include, for example, CD-ROMs (Compact Disk-Read Only Memory), DVDs (Digital Versatile Disk), and Blu-ray Discs. Optical discs include MDs (Mini-Disk).

[0102] Furthermore, recording media provided to the user in a pre-assembled state with the main body of the device, such as those containing a program... Figure 2 ROM22 Figure 3 The storage unit 56 includes hard drives, etc.

[0103] Furthermore, the steps of a program recorded in a recording medium as described in this specification include, of course, processing performed sequentially in time, as well as processing that may not be performed sequentially but is executed in parallel or separately.

Claims

1. An image processing apparatus comprising a processing unit, The processing unit, To acquire multiple images that are consecutive in time, The position of the moving object is obtained by identifying the position of the visible light source in the lit state of the moving object from among the multiple images. Based on the flickering patterns of invisible light sources in the multiple images, the identification information of the moving object is obtained. The visible light source and the invisible light source are respectively positioned near the moving body. The processing unit, If green light is detected, it is determined that only the visible light source is lit, among the visible and invisible light sources. If white light is detected, it is determined that both the visible light source and the invisible light source are illuminated.

2. The image processing apparatus according to claim 1, wherein, The image processing device includes: a single image sensor for acquiring the visible light from the visible light source and the invisible light from the invisible light source.

3. The image processing apparatus according to claim 1, wherein, The processing unit, The position of the green visible light source, which is either always lit or lit for a specified period, is used as the position of the moving body for tracking. The identification information of the moving object is obtained by acquiring the flashing pattern of the invisible light in the infrared region of the invisible light source.

4. An optical communication system comprising an optical transmitting device and an image processing device, The optical transmitting device has: Visible light sources are used to illuminate objects so that image processing devices can determine the position of moving objects; and An invisible light source is illuminated so that the image processing device can acquire the identification information of the moving object. The image processing device has a processing unit. The processing unit, Acquire multiple images that are consecutive in time, and determine the position of the moving object by identifying the position of the visible light source that is lit up in the consecutive images. The identification information of the moving object is obtained based on the flashing pattern of invisible light. The visible light source and the invisible light source are respectively positioned near the moving body. The processing unit, If green light is detected, it is determined that only the visible light source is lit, among the visible and invisible light sources. If white light is detected, it is determined that both the visible light source and the invisible light source are illuminated.

5. A program product comprising a program that causes an image processing apparatus to perform the following functions: Image acquisition function, which acquires multiple images that are consecutive in time; The acquisition and processing function obtains the positions of the visible light sources in the lit states among the multiple images as the positions of the moving objects; and The analysis and processing function obtains the identification information of the moving object based on the flickering pattern of invisible light sources in the multiple images. The visible light source and the invisible light source are respectively positioned near the moving body. The program causes the image processing device to perform the following functions: If green light is detected, it is determined that only the visible light source is lit, among the visible and invisible light sources. If white light is detected, it is determined that both the visible light source and the invisible light source are illuminated.

6. An image processing method, which is a computer-executed image processing method. The image acquisition step involves acquiring multiple images that are consecutive in time. The processing step involves obtaining the position of the visible light source in the lit state among the multiple images as the position of the moving body. and The analysis and processing steps involve obtaining the identification information of the moving object based on the flickering patterns of invisible light sources in the multiple images. The visible light source and the invisible light source are respectively positioned near the moving body. The image processing method further includes the following steps: If green light is detected, it is determined that only the visible light source is lit, among the visible and invisible light sources. If white light is detected, it is determined that both the visible light source and the invisible light source are illuminated.

Images