Information processing device, information processing method, and program

The information processing device controls infrared LEDs of electronic pens to blink at different timings, addressing the challenge of distinguishing multiple pens' positions, ensuring accurate tracking and preventing ID swapping, especially during collaborative writing or drawing.

JP7586098B2Active Publication Date: 2024-11-19SONY GROUP CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2021569828
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-01-09
Filing Date
2020-12-23
Publication Date
2024-11-19
Estimated Expiration
2040-12-23

AI Technical Summary

Technical Problem

Existing systems struggle to accurately distinguish and detect the positions of multiple electronic pens when their infrared lights are close to each other, as the camera recognizes them as a single bright spot.

Method used

An information processing device that controls the infrared LEDs of electronic pens to blink at different timings when they approach within a threshold distance, allowing the system to reliably identify and track each pen's position using a light emission control unit, detection unit, and display control unit.

Benefits of technology

The solution enables reliable capture and separation of positions corresponding to each electronic pen, even when they are close together, preventing ID swapping and maintaining a high frame rate during tasks like writing or drawing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007586098000001
    Figure 0007586098000001
  • Figure 0007586098000002
    Figure 0007586098000002
  • Figure 0007586098000003
    Figure 0007586098000003
Patent Text Reader

Abstract

A light-emission control unit (34a) of an information processing device (10a) causes an IR LED (13) (light emitting unit) provided in an electronic pen (12) (operating body) that has established communication with the information processing device (10a) to be lit, and when a different electronic pen (12) approaches, causes the IR LEDs (13) of the electronic pens to flash with mutually different timings. A light emission recognizing unit (32) (detecting unit) then detects a position corresponding to the IR LED (13) provided in each electronic pen (12), in association with the electronic pen (12). If one of the electronic pens (12) is in an information input state, an input / output control unit (36) (display control unit) displays information associated with the electronic pen (12) in a position corresponding to the IR LED (13) detected by the light emission recognizing unit.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program, and in particular to an information processing device, an information processing method, and a program that can reliably capture positions corresponding to each electronic pen even when multiple electronic pens are in close proximity. [Background technology]

[0002] Conventionally, interactive projectors have been commercialized that use electronic pens to write figures and characters on a display screen or to operate a UI (User Interface). In such systems, light emitted from the electronic pen, mainly infrared light (IR light), is captured by a camera and the captured image is analyzed to detect the position of the electronic pen (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 130696 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, when IR lights emitted by multiple electronic pens are close to each other, it is difficult to separate and detect these IR lights. This is because a camera that captures IR lights recognizes IR lights that are close to each other as the same bright spot.

[0005] The present disclosure proposes an information processing device, an information processing method, and a program that are capable of reliably capturing positions corresponding to each electronic pen even when multiple electronic pens are in close proximity to each other. [Means for solving the problem]

[0006] In order to solve the above problems, one form of information processing device according to the present disclosure is an information processing device that includes: a light-emitting control unit that turns on a light-emitting unit provided on an operating object that has established communication with the information processing device, and, when a different operating object approaches, causes the light-emitting unit provided on the operating object to blink at different timings; a detection unit that detects a position corresponding to the light-emitting unit provided on each operating object in association with the operating object; and a display control unit that, when the operating object is in an information input state, displays information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection unit. [Brief description of the drawings]

[0007] [Figure 1] 1 is a system configuration diagram of an interactive projector according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing a schematic structure of an electronic pen. [Diagram 3] FIG. 2 is a functional block diagram showing an example of a functional configuration of an interactive projector. [Figure 4] 5 is a flowchart showing an example of a flow of processing performed by the information processing apparatus and the electronic pen according to the first embodiment. [Diagram 5] 13 is a flowchart showing an example of the flow of an input position detection process and an ID identification process of the electronic pen. [Figure 6] 4 is a time chart showing the flow of a process in which the interactive projector according to the first embodiment identifies an electronic pen. [Figure 7] FIG. 13 is a diagram illustrating a bright spot detection method. [Figure 8] 11A and 11B are diagrams for explaining an overview of an ID exchange determination process based on the distance of a bright point. [Figure 9] 13 is a diagram illustrating an IR LED control command issued by a light emission control unit when the distance of a bright spot is within a threshold distance. FIG. [Figure 10] 13 is a time chart showing an example of preventing ID swapping when two electronic pens are used. [Figure 11] 13 is a time chart showing an example of preventing ID swapping when three electronic pens are used. [Figure 12] 13 is a flowchart showing an example of the flow of a process for determining whether or not an ID of an electronic pen has been swapped. [Figure 13] FIG. 4 is a state transition diagram showing an example of state transition of the electronic pen. [Figure 14] 11 is a diagram illustrating a process flow for converting a detected bright spot position from the camera coordinate system to the projector screen coordinate system. FIG. [Figure 15] FIG. 13 is a diagram showing an example of a state in which groups including a plurality of electronic pens that are close to each other within a threshold distance partially overlap each other. [Figure 16] 13 is a time chart showing an example of time-division emission of IR LEDs of all electronic pens belonging to a union; [Figure 17] 11 is a time chart showing an example of blinking the IR LED of the electronic pen so that the light emission patterns do not overlap within each group. [Figure 18] FIG. 1 is a schematic diagram of a shooting system which is an application example of the present disclosure. [Figure 19] FIG. 1 is a first diagram showing an overview of a miniature car racing system that is an application example of the present disclosure. [Figure 20] FIG. 2 is a second diagram showing an overview of a miniature car racing system that is an application example of the present disclosure. [Figure 21] FIG. 1 is a diagram showing an overview of a spinning top playing system which is an application example of the present disclosure. [Figure 22] FIG. 1 is a diagram showing an overview of an air hockey system which is an application example of the present disclosure. [Figure 23] FIG. 1 is a diagram showing an overview of a bouldering system which is an application example of the present disclosure. [Figure 24] FIG. 1 is a diagram showing an overview of a swarm robot control system which is an application example of the present disclosure. [Diagram 25] FIG. 1 is a diagram showing an overview of an AR / VR application system which is an application example of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0009] The present disclosure will be described in the following order. 1. First embodiment 1-1. Description of the interactive projector according to the first embodiment 1-2.Explanation of the structure of the electronic pen 1-3. Explanation of the functional configuration of the interactive projector 1-4. Description of the process flow performed by the information processing device and electronic pen 1-5. Explanation of the electronic pen input position detection process and ID identification process 1-6.Explanation of the bright spot detection method 1-7. Explanation of the process to prevent the exchange of electronic pens 1-8.Explanation of electronic pen state transition 1-9. Explanation of how to reflect electronic pen input results 1-10. Effects of the First Embodiment 2. Second embodiment 3. Third embodiment 4. Fourth embodiment 5. Fifth embodiment 6. Application Example of the Present Disclosure (1) 7. Application Example of This Disclosure (2) 8. Application Example of This Disclosure (3) 9. Application Example of This Disclosure (4) 10. Application Example of This Disclosure (5) 11. Application Example of This Disclosure (6) 12. Application Example of This Disclosure (7)

[0010] 1. First Embodiment First, an information processing device 10a according to a first embodiment of the present disclosure will be described.

[0011] [1-1. Description of the interactive projector according to the first embodiment] The first embodiment is an example in which an information processing device 10a of the present disclosure is applied to an interactive projector 5a that acquires position information drawn on a whiteboard 26 with multiple electronic pens and projects information by a projector at a position corresponding to the acquired position information. FIG. 1 is a system configuration diagram of the interactive projector according to the first embodiment. The interactive projector 5a includes an information processing device 10a, multiple electronic pens 12 (12a, 12b, 12c, ...), a camera 20, a projector 22, and a wireless communication module 24.

[0012] The electronic pen 12 is an information input device that inputs information such as characters and figures by moving the electronic pen 12 in contact with a whiteboard or the like. Each electronic pen 12 has a different identification number (ID) and can be used simultaneously by multiple users. The electronic pen 12 is an example of an operating object in this disclosure.

[0013] The camera 20 is a camera capable of observing infrared light irradiated by the electronic pen 12. The camera 20 may be equipped with an optical filter that passes only light in the wavelength band of infrared light irradiated by the electronic pen 12. The camera 20 is installed near the projector 22 and captures an image of an area that covers at least the projection range of the projector 22. The optical axis of the camera 20 and the optical axis of the projector 22 are preferably installed approximately parallel to each other.

[0014] The projector 22 projects information corresponding to the trajectory of the electronic pen 12 onto the position where information is input by the electronic pen 12 onto the whiteboard 26. The position corresponding to the trajectory of the electronic pen 12 is, for example, the contact position between the tip of the electronic pen 12 and the whiteboard 26. The information corresponding to the electronic pen 12 is, for example, the color, thickness, type, etc. of the line set by the electronic pen 12.

[0015] The wireless communication module 24 wirelessly connects the information processing device 10a and the electronic pen 12. The wireless connection may be, for example, a Bluetooth (registered trademark) connection, but is not limited thereto and may be, for example, a wireless LAN (Local Area Network) or ZigBee (registered trademark), etc.

[0016] The information processing device 10a detects the contact position between the whiteboard 26 and the electronic pen 12 from the image captured by the camera 20. Specifically, the contact between the whiteboard 26 and the electronic pen 12 is determined according to the ON / OFF state of a switch 14 (see FIG. 2) mounted on the electronic pen 12. The contact position between the whiteboard 26 and the electronic pen 12 is detected based on the position of a bright spot caused by an IR LED 13 (see FIG. 2) mounted on the electronic pen 12, which is included in the image captured by the camera 20. The information processing device 10a also causes the projector 22 to project information corresponding to the electronic pen 12 at a position corresponding to the detected contact position.

[0017] 1, a display may be used as a substitute for the whiteboard 26, and information may be input onto the surface of the display using the electronic pen 12. Alternatively, instead of the projector 22, a screen may be placed at the position of the whiteboard 26, and rear projection may be used to project information from the rear side of the screen. Furthermore, a sound output device such as a speaker may be combined. The projector 22 may be a device having, for example, a drive mechanism and capable of projecting in any direction. By having such a mechanism, it becomes possible to display images not only in one place but also in various places, and to realize interactive functions.

[0018] [1-2. Explanation of the structure of the electronic pen] The structure of the electronic pen 12 will be described below with reference to Fig. 2. Fig. 2 is a schematic diagram showing the schematic structure of the electronic pen.

[0019] The electronic pen 12 includes an IR LED 13 , a switch 14 , a CPU (Central Processing Unit) 15 , and a wireless communication module 16 .

[0020] The IR LED 13 is an LED that is built into the pen tip of the electronic pen 12 and emits infrared light of a wavelength that is invisible to humans and to which the imaging element of the camera 20 (see FIG. 1) is sensitive. The IR LED 13 has a shape in which the front surface of the LED chip is sealed in a bullet-shaped resin mold 17 that faces the pen tip. Since the resin mold 17 plays the role of a lens, the light emitted from the IR LED 13 forms a bright spot of a size corresponding to the shape of the bullet near the pen tip. Therefore, when the IR LED 13 is turned on while the pen tip of the electronic pen 12 is pressed against the whiteboard 26, the camera 20 observes a bright spot on the surface of the whiteboard 26 due to the IR light emitted from the IR LED 13 (see FIG. 8). The IR LED 13 is an example of a light-emitting unit in the present disclosure.

[0021] It should be noted that electronic pen 12 can achieve the same effect as described below even if it is equipped with an LED that emits visible light. However, when an LED that emits visible light is used, the imaging element equipped in camera 20 needs to have sensitivity to light of the emission wavelength of the LED. Furthermore, if an object with the same color as the emission wavelength of the LED is present in the image observed by camera 20, it will become noise, and in some cases it may be necessary to perform signal processing to remove such noise.

[0022] In addition, the side surface of the resin mold 17 is a diffusing surface, and when the IR LED 13 is turned on, the pen tip of the electronic pen 12 itself also emits diffuse light. Therefore, when the electronic pen 12 is removed from the surface of the whiteboard 26 with the IR LED 13 turned on, the camera 20 observes the state in which the pen tip of the electronic pen 12 is emitting and shining IR light. This allows the information processing device 10a to track the position of the pen tip of the electronic pen 12 that has been removed from the surface of the whiteboard 26, as described later (see state Sg in FIG. 13). Note that the IR LED 13 is an example of a light-emitting unit in the present disclosure.

[0023] It is assumed that the IR LEDs 13 included in the multiple electronic pens 12 (12a, 12b, 12c, . . . ) all emit infrared light of the same wavelength.

[0024] The switch 14 is a switch that operates in response to a pressure acting on the pen tip of the electronic pen 12. That is, when the pen tip of the electronic pen 12 is pressed against a fixed object such as the whiteboard 26 or a wall, the switch 14 is pressed and turns ON, and when the pen tip is released, the switch 14 returns to its original state and turns OFF. Based on the operating state of the switch 14, the information processing device 10a determines whether the electronic pen 12 is pressed against the whiteboard 26, i.e., whether the electronic pen 12 is in an input state.

[0025] Based on the control command received from the information processing device 10a, the CPU 15 controls the lighting state of the IR LED 13. The CPU 15 also detects the state of the switch 14. Furthermore, the CPU 15 receives a control command related to the lighting state of the IR LED 13 from the information processing device 10a, and transmits the lighting state of the IR LED 13 and the state of the switch 14 to the information processing device 10a.

[0026] The wireless communication module 16 performs wireless communication with the information processing device 10a, and transmits the states of the switch 14 and the IR LED 13 to the information processing device 10a.

[0027] In this embodiment, the electronic pen 12 is assumed to be a pen-shaped device as shown in Fig. 2. However, the electronic pen 12 may be constructed of a device of a different shape. For example, the electronic pen 12 may be a device that is operated by the user to spray something like a spray can, or may be a device worn on the hand or foot.

[0028] [1-3. Explanation of the functional configuration of the interactive projector] Next, the functional configuration of interactive projector 5a will be described with reference to Fig. 3. Fig. 3 is a functional block diagram showing an example of the functional configuration of the interactive projector.

[0029] First, the functional configuration of the information processing device 10a will be described. The information processing device 10a receives images captured by the camera 20 and wireless information from the electronic pens 12, performs internal processing, and then generates information to be projected by the projector 22 and transmits it to the projector 22. In addition, the information processing device 10a transmits control commands to each electronic pen 12 for controlling the lighting state of the IR LED 13.

[0030] The information processing device 10a includes an ID management unit 30, a communication management unit 31, a light emission recognition unit 32, a coordinate conversion unit 33, an electronic pen control unit , a storage unit 35, and an input / output control unit .

[0031] The ID management unit 30 manages the identification number (ID) of the electronic pen 12 and the light emission state of the IR LED 13 provided in the electronic pen 12.

[0032] The communication management unit 31 manages communication device information of each electronic pen 12, for example, information such as the identification number of the electronic pen 12a being "AA" and the identification number of the electronic pen 12b being "BB."

[0033] The light emission recognition unit 32 analyzes the image captured by the camera 20 and detects the position (coordinate value in the camera coordinate system) of the bright point formed by the electronic pen 12. The light emission recognition unit 32 is an example of a detection unit in this disclosure.

[0034] Coordinate conversion unit 33 converts the positions of bright points detected in the image captured by camera 20 into coordinate values ​​in the projector screen coordinate system onto which projector 22 projects. Specifically, the coordinate conversion is performed using a projection matrix determined based on the positional relationship between camera 20, projector 22, and whiteboard 26.

[0035] The electronic pen control unit 34 controls the light emission state of the IR LED 13 included in the electronic pen 12, and performs ID swap prevention determination when multiple electronic pens 12 are in close proximity. The electronic pen control unit 34 further includes a light emission control unit 34a, an ID swap prevention determination unit 34b, a communication control unit 34c, and a linking processing unit 34d.

[0036] The light emission control unit 34a turns on the IR LED 13 (light emitting unit) of the electronic pen 12 (operation object) that has established communication with the information processing device 10a. When different electronic pens 12 approach within a predetermined threshold distance, the light emission control unit 34a blinks the IR LED 13 (light emitting unit) of the electronic pen 12 at different timings.

[0037] When multiple IR LEDs 13 approach within a predetermined distance, the ID swap prevention determination unit 34b performs a swap prevention determination process to prevent a bright spot caused by one electronic pen 12 from being mistakenly recognized as a bright spot caused by another electronic pen 12, and to prevent the IDs of the electronic pens 12 associated with the positions of the bright spots from being swapped. The specific contents of the process will be described later.

[0038] The communication control unit 34c receives the state of the switch 14 transmitted from the electronic pen 12 to the information processing device 10a, and transmits a light emission command for the IR LED 13 from the information processing device 10a to the electronic pen 12.

[0039] The association processing unit 34d associates the position of the bright point of the IR LED 13 detected by the camera 20 with the ID of the electronic pen 12, and stores the associated information in the storage unit 35.

[0040] The storage unit 35 stores information linking the position of the bright spot of the IR LED 13 with the ID of the electronic pen 12. The storage unit 35 is configured with, for example, a flash ROM.

[0041] The input / output control unit 36 ​​determines whether the electronic pen 12 is in an information input state based on the state of the switch 14 transmitted from the electronic pen 12. The input / output control unit 36 ​​also controls the input and output of information between the information processing device 10a and the camera 20, projector 22, and wireless communication module 24 connected to the information processing device 10a. The input / output control unit 36 ​​is an example of a display control unit in the present disclosure.

[0042] The electronic pen 12 includes a communication control unit 40, a light emission state switching processing unit 41, and an input / output control unit .

[0043] The communication control unit 40 transmits the state of the switch 14 of the electronic pen 12 to the information processing device 10a, and receives a light emission command for the IR LED 13 transmitted from the information processing device 10a.

[0044] The light emission state switching processing unit 41 switches the lighting state of the IR LED 13 (off, on, blinking) in response to a light emission command for the IR LED 13 received from the information processing device 10a.

[0045] The input / output control unit 42 controls the input and output of information between the switch 14 , the IR LED 13 , the wireless communication module 16 , and the CPU 15 .

[0046] [1-4. Description of the process flow performed by the information processing device and electronic pen] Next, a description will be given of the flow of a series of processes performed by the information processing device 10a and the electronic pen 12. Fig. 4 is a flowchart showing an example of the flow of processes performed by the information processing device and the electronic pen of the first embodiment.

[0047] First, the flow of the process performed by the electronic pen 12 will be described. The wireless communication module 16 communicates with the wireless communication module 24 and determines whether a Bluetooth connection (hereinafter referred to as a BT connection) has been completed (step S11). If it is determined that the BT connection has been completed, the process proceeds to step S12. On the other hand, if it is not determined that the BT connection has been completed, step S11 is repeated.

[0048] The communication control unit 40 transmits the ID of the electronic pen 12 to the information processing device 10a (step S12).

[0049] The input / output control unit 42 determines whether an IR LED control command has been received from the information processing device 10a (step S13). If it is determined that an IR LED control command has been received (step S13: Yes), the process proceeds to step S14. On the other hand, if it is determined that an IR LED control command has not been received (step S13: No), the process proceeds to step S15.

[0050] If the result of the determination in step S13 is Yes, the light emission state switching processing unit 41 switches the illumination state (off, on, blinking) of the IR LED 13 in response to the IR LED control command received from the information processing device 10a (step S14). Then, the process proceeds to step S15.

[0051] If it is determined as No in step S13 or following the execution of step S14, the input / output control unit 42 determines whether or not a change in the switch state of the electronic pen 12 has been detected (step S15). If it is determined that a change in the switch state of the electronic pen 12 has been detected (step S15: Yes), the process proceeds to step S16. On the other hand, if it is not determined that a change in the switch state of the electronic pen 12 has been detected (step S15: No), the process proceeds to step S17.

[0052] If the result of the determination in step S15 is Yes, the communication control unit 40 transmits a switch state change event of the electronic pen 12, i.e., information indicating whether the switch 14 has changed from OFF to ON or from ON to OFF, to the information processing device 10a (step S16). Then, the process proceeds to step S17.

[0053] If it is determined that the result of step S15 is No, or following the execution of step S16, the input / output control unit 42 determines whether the application being executed by the information processing device 10a has been terminated or the power supply of the information processing device 10a has been turned off based on the communication result with the information processing device 10a (step S17). If it is determined that the application being executed by the information processing device 10a has been terminated or the power supply of the information processing device 10a has been turned off (step S17: Yes), the electronic pen 12 ends the process of Fig. 4. On the other hand, if it is not determined that the application being executed by the information processing device 10a has been terminated or the power supply of the information processing device 10a has been turned off (step S17: No), the process returns to step S13 and repeats the above-mentioned processes.

[0054] Next, the flow of the process performed by the information processing device 10a will be described. First, the ID management unit 30 determines whether a new electronic pen 12 has been connected (step S21). If it is determined that a new electronic pen 12 has been connected (step S21: Yes), the process proceeds to step S22. On the other hand, if it is not determined that a new electronic pen 12 has been connected (step S21: No), step S21 is repeated.

[0055] If the result of the determination in step S21 is Yes, the light emission recognition unit 32 and the association processing unit 34d perform an input position detection process and an ID identification process of the electronic pen 12 (step S22). The content of the process performed in step S22 will be described in detail later (see FIG. 5).

[0056] The ID interchange prevention determination unit 34b performs an ID interchange prevention determination process (step S23). Note that the content of the process performed in step S23 will be described in detail later (see FIG. 12).

[0057] The communication control unit 34c transmits a control command for the IR LED 13 according to the determination result to the electronic pen 12 (step S24).

[0058] The coordinate conversion unit 33 performs a coordinate conversion process to convert the position of the bright spot in the camera coordinate system input by the electronic pen 12 into the projector screen coordinate system of the projected image generated by the projector 22 (step S25).

[0059] The input / output control unit 36 ​​causes the projector 22 to project the image that has been subjected to the coordinate conversion in step S25 (step S26). After that, the process proceeds to step S27.

[0060] Following step S26, the input / output control unit 36 ​​determines whether the application has ended or the information processing device 10a has been powered off (step S27). If it is determined that the application has ended or the information processing device 10a has been powered off (step S27: Yes), the information processing device 10a ends the process in Fig. 4. On the other hand, if it is not determined that the application has ended or the information processing device 10a has been powered off (step S27: No), the process returns to step S21.

[0061] [1-5. Explanation of electronic pen input position detection process and ID identification process] Next, the flow of the input position detection process and the ID identification process of the electronic pen will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of the input position detection process and the ID identification process of the electronic pen.

[0062] The ID management unit 30 determines whether an undetected electronic pen 12 exists (step S31). If it is determined that an undetected electronic pen 12 exists (step S31: Yes), the process proceeds to step S32. On the other hand, if it is not determined that an undetected electronic pen 12 exists (step S31: No), the process proceeds to step S33. Whether an undetected electronic pen 12 exists is determined based on whether an electronic pen 12 exists whose ID and bright point position are not associated with each other among the BT-connected electronic pens 12.

[0063] If the result of the determination in step S31 is Yes, the light emission control unit 34a transmits a control command to turn on the IR LED 13 to the undetected electronic pen 12 (step S32).

[0064] Next, the light emission recognition unit 32 detects the position of the bright spot caused by the IR LED 13 turned on by the process of step S32 (step S33).

[0065] The linking processing unit 34d sets a variable i representing the currently focused electronic pen 12 to i=0 (step S34). For example, if there are three electronic pens 12, the variable i takes the values ​​i=0, 1, 2.

[0066] The linking processing unit 34d determines whether the variable i is smaller than the total number n of bright spots detected by image analysis (step S35). If it is determined that the variable i is smaller than the total number n of bright spots (step S35: Yes), the process proceeds to step S36. On the other hand, if it is not determined that the variable i is smaller than the total number n of bright spots (step S35: No), the process of FIG. 5 is terminated and the process returns to the main routine (step S23 in FIG. 4).

[0067] If the result of step S35 is Yes, the ID management unit 30 determines the state of the IR LED 13 (step S36). If the result of step S35 is Yes, the ID management unit 30 determines the state of the IR LED 13. If the result of step S36 is No, the ID management unit 30 proceeds to step S37. On the other hand, if the result of step S35 is No, the ID management unit 30 proceeds to step S40.

[0068] If it is determined in step S36 that the IR LED 13 is in an on state, the association processing unit 34d determines whether an ID has been assigned to the IR LED 13 in the previous frame (step S37). If it is determined that an ID has been assigned to the IR LED 13 in the previous frame (step S37: Yes), the process proceeds to step S38. On the other hand, if it is not determined that an ID has been assigned to the IR LED 13 in the previous frame (step S37: No), the process proceeds to step S42.

[0069] Returning to step S36, if it is determined that the IR LED 13 is in a blinking state, the ID management unit 30 analyzes the light emission pattern of the IR LED 13 (step S40).

[0070] Next, the ID management unit 30 assigns an electronic pen ID corresponding to the light emission pattern (step S41), and then proceeds to step S39.

[0071] Returning to step S37, if the determination in step S37 is Yes, the linking processing unit 34d assigns the ID of the previous frame (step S38), and then proceeds to step S39.

[0072] On the other hand, if the result of step S37 is No, the association processing unit 34d assigns the electronic pen ID that last transmitted a lighting command to the IR LED 13 (step S42), and then proceeds to step S39.

[0073] Following steps S38, S41, and S42, the linking processing unit 34d increments the variable i (step S39), and then returns to step S35.

[0074] In this way, by executing the flowchart of Figure 5, a new ID is assigned to the electronic pen 12 each time a new bright spot is detected, and once an ID has been assigned to a bright spot, it is tracked with the same ID assigned to it.

[0075] Next, a process flow in which interactive projector 5a identifies electronic pen 12 will be described with reference to a time chart using Fig. 6. Fig. 6 is a time chart showing a process flow in which the interactive projector according to the first embodiment identifies an electronic pen.

[0076] As shown in Fig. 6, it is assumed that there are two electronic pens (electronic pen 12a and electronic pen 12b). An identification number (ID) is stored in each of the two pens. As an example of the identification number, it is assumed that the identification number of the electronic pen 12a is "AA" and the identification number of the electronic pen 12b is "BB".

[0077] At time t1, the electronic pen 12a and the information processing device 10a are connected via BT.

[0078] At time t2, the information processing device 10a transmits a control command to the BT-connected electronic pen 12a to turn on the IR LED 13. The electronic pen 12a receives this control command and turns on the IR LED 13. At this time, a bright spot caused by the emission of light from the IR LED 13 of the electronic pen 12a is observed in the camera image captured by the camera 20.

[0079] At time t3, the electronic pen 12a is pressed against the whiteboard 26, and the switch 14 at the pen tip is turned ON. At this time, the electronic pen 12a sends to the information processing device 10a the identification number "AA" of the electronic pen 12a and information indicating that the switch 14 has been turned ON. At this time, a bright spot caused by the emission of light from the IR LED 13 of the electronic pen 12a is observed in the camera image captured by the camera 20. Thereafter, the electronic pen 12a draws on the whiteboard 26.

[0080] At time t4, the electronic pen 12b and the information processing device 10a are connected via BT.

[0081] At time t5, the information processing device 10a transmits a control command to the BT-connected electronic pen 12b to turn on the IR LED 13. The electronic pen 12b receives this control command and turns on the IR LED 13. At this time, a bright spot caused by the emission of light from the IR LED 13 of the electronic pen 12a and a bright spot caused by the emission of light from the IR LED 13 of the electronic pen 12b are observed in the camera image captured by the camera 20.

[0082] At time t6, the electronic pen 12b is pressed against the whiteboard 26, and the switch 14 at the pen tip is turned ON. At this time, the electronic pen 12b sends to the information processing device 10a the identification number "BB" of the electronic pen 12b and information indicating that the switch 14 has been turned ON. At this time, a bright spot caused by the emission of light from the IR LED 13 of the electronic pen 12a and a bright spot caused by the emission of light from the IR LED 13 of the electronic pen 12b are observed in the camera image captured by the camera 20. Thereafter, the electronic pens 12a and 12b perform drawing on the whiteboard 26.

[0083] The information processing device 10a accepts drawing input from the electronic pen 12 and reflects the drawing input results in the application only when the bright spot of the IR LED 13 of the electronic pen 12 is detected, the ID of the electronic pen 12 is identified, and the switch 14 of the electronic pen 12 is turned ON.

[0084] [1-6.Explanation of the bright spot detection method] Next, a method in which the light emission recognition unit 32 detects a bright spot caused by the IR LED 13 will be described with reference to Fig. 7. Fig. 7 is a diagram for explaining the bright spot detection method.

[0085] The light emission recognition unit 32 binarizes the image Ia captured by the camera 20 using a predetermined brightness threshold value, thereby generating a binary image Ib.

[0086] Next, the light emission recognition unit 32 extracts the contour of the binary image Ib (image Ic).

[0087] Then, the light emission recognition unit 32 obtains the center of gravity G of the contour in the image Ic (image Id). The center of gravity G thus obtained is regarded as the position of the bright point.

[0088] The method for calculating the position of the bright spot is not limited to the method shown in Fig. 7. For example, the center of gravity of the area of ​​the bright spot in the binary image Ib may be obtained and used as the position of the bright spot.

[0089] Furthermore, when multiple electronic pens 12 are used simultaneously, multiple bright points are observed in the image Ia. Therefore, the light emission recognition unit 32 performs a labeling process on the binary image Ib to assign a number to each area. Then, the position of each bright point is obtained by applying the image processing shown in FIG. 7 to each labeled area.

[0090] [1-7. Explanation of the process for determining whether or not to swap electronic pen IDs] Next, the ID swap prevention determination process of the electronic pen 12 will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining an overview of the ID swap determination process based on the distance of a bright spot.

[0091] When drawing is performed simultaneously with a plurality of electronic pens 12 (for example, electronic pen 12a (ID: AA) and electronic pen 12b (ID: BB)), as shown in FIG. 8, electronic pen 12a and electronic pen 12b may come close to each other. In such a case, the information processing device 10a may erroneously recognize the position of a bright spot generated by electronic pen 12a as the position of a bright spot generated by electronic pen 12b. For example, in FIG. 8, it is assumed that electronic pen 12a and electronic pen 12b each make a U-turn just before coming into contact with each other. In this case, the information processing device 10a may determine that electronic pen 12a and electronic pen 12b have crossed, and may erroneously recognize the bright spot generated by electronic pen 12a as the bright spot generated by electronic pen 12b. Similarly, the information processing device 10a may erroneously recognize the bright spot generated by electronic pen 12b as the bright spot generated by electronic pen 12a. When such an erroneous recognition occurs, ID: BB is assigned to the bright spot generated by electronic pen 12a, and ID: AA is assigned to the bright spot generated by electronic pen 12b. That is, ID switching occurs.

[0092] In order to prevent such ID swapping, the information processing device 10a of the present embodiment issues an IR LED control command according to the state of each electronic pen 12 when the positions of the bright spots generated by the multiple electronic pens 12 are within a threshold distance d. Specifically, for example, when the two electronic pens 12a and 12b are both in an input state (drawing state) and approach each other within the threshold distance d, the electronic pens 12a and 12b are blinked in a time-division manner to ensure that the positions of the bright spots generated by each electronic pen can be detected. Note that the positions of the bright spots that approach within the threshold distance d are the positions of the bright spots generated by the emission of the IR LEDs 13 of the multiple electronic pens 12, but for simplicity, the following expression will be used to refer to "when the multiple electronic pens 12 approach within the threshold distance d."

[0093] It is preferable that the threshold distance d is determined based on a value obtained by converting the coordinates to the actual size projected by the projector 22, rather than the difference between the detected positions in the image captured by the camera 20. As described later, the information processing device 10a converts the detected bright spot position into the projector screen coordinate system when the input result of the electronic pen 12 is projected onto the whiteboard 26 by the projector 22. At this time, by measuring the actual size of the whiteboard 26, it is possible to scale the distance of the bright spot position in the image captured by the camera 20 to the actual size, and it is possible to perform a threshold determination as to whether the converted coordinate value approaches within the threshold distance d. It is possible that the threshold distance d is a parameter that can be freely set by the user, or a fixed threshold distance d is set at the time of manufacture. In addition, as described in the fourth embodiment, the threshold distance d may be dynamically changed according to the drawing state of the electronic pen 12.

[0094] This shows an example in which the threshold distance d is set by converting the size of the bright spot of the IR LED 13 observed by the camera 20 into actual size. For example, if the diameter of the bright spot caused by the IR LED 13 converted into actual size is 5 cm, the threshold distance d is set to d=20 cm (diameter of the bright spot + 10 cm). This is an example in which the minimum distance of the bright spot area when the IR LED 13 is observed is 10 cm, and an example in which a buffer of 10 cm is added when the light emission pattern of the IR LED 13 is changed and detected by the camera 20, until ID identification can be performed from the light emission pattern.

[0095] Next, the content of the IR LED control command issued by the light emission control unit 34a (see FIG. 3) when the positions of the bright spots generated by the multiple electronic pens 12 are within the threshold distance d will be described with reference to FIG. 9. FIG. 9 is a diagram for explaining the IR LED control command issued by the light emission control unit when the distance of the bright spots is within the threshold distance.

[0096] 9, the state of the electronic pen 12 "inputting" indicates a state in which a bright spot by the IR LED 13 is detected, the identification number of the electronic pen 12 emitting the IR LED 13 that formed the bright spot is identified, and the switch 14 of the electronic pen 12 is ON. Meanwhile, "not inputting" indicates a state in which a bright spot by the IR LED 13 is detected, the identification number of the electronic pen 12 emitting the IR LED 13 that formed the bright spot is identified, and the switch 14 of the electronic pen 12 is OFF.

[0097] Hereinafter, four cases (a) to (d) shown in Fig. 9 will be specifically described. For the sake of simplicity, it is assumed that another electronic pen 12 approaches within a threshold distance d from a focused electronic pen 12 (electronic pen 12 to be controlled).

[0098] State (a) is a case where the electronic pen 12 to be controlled and the other electronic pen 12 are both “inputting.” In this case, the information processing device 10a transmits a control command to both electronic pens 12 to blink alternately in a time-division manner.

[0099] In state (b), the electronic pen 12 to be controlled is "inputting" and the other electronic pens 12 are "not inputting". In this case, the information processing device 10a transmits a control command to the electronic pen 12 to be controlled to turn on the IR LED 13. Also, the information processing device 10a transmits a control command to the other electronic pens 12 to turn off the IR LED 13.

[0100] In state (c), the electronic pen 12 to be controlled is "not inputting data" and the other electronic pens 12 are "inputting data". In this case, the information processing device 10a transmits a control command to the electronic pen 12 to be controlled to turn off the IR LED 13. Also, the information processing device 10a transmits a control command to the other electronic pens 12 to turn on the IR LED 13.

[0101] State (d) is a case where the electronic pen 12 to be controlled and the other electronic pen 12 are both “not inputting.” In this case, the information processing device 10a transmits a control command to both electronic pens 12 to blink alternately in a time-division manner.

[0102] When the IR LED 13 is alternately blinked in a time-division manner as in state (a), the occurrence time of the bright spot related to a specific electronic pen 12 becomes shorter, and the frame rate when detecting the bright spot position decreases. Therefore, when there is only one electronic pen 12 "inputting data", as in state (b) and state (c), the IR LED 13 of the electronic pen 12 "not inputting data" is turned off to put it into a lost state. In this way, by creating a state where there is only one electronic pen 12 "inputting data", it is possible to suppress a decrease in the frame rate.

[0103] Also, when both electronic pens 12 are "not inputting" as in state (d), the IR LED 13 is blinked in a time-division manner to continue ID identification. In this case, the frame rate decreases, but tracking of the bright spot position of each electronic pen 12 can be continued.

[0104] Next, a specific example of preventing ID swapping will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a time chart showing an example of preventing ID swapping when two electronic pens are used. Fig. 11 is a time chart showing an example of preventing ID swapping when three electronic pens are used.

[0105] The time chart in FIG. 10 shows the light emission state of the IR LED 13 when two of the three electronic pens 12a (ID: AA), 12b (ID: BB), and 12c (ID: CC) are approaching each other within the threshold distance d while all of the three electronic pens 12a (ID: AA), 12b (ID: BB), and 12c (ID: CC) are in the input state.

[0106] At time ta in Fig. 10, the electronic pens 12a and 12b approach each other within the threshold distance d, which satisfies the condition of state (a) described in Fig. 9. After time ta, the IR LED 13 of the electronic pen 12a and the IR LED 13 of the electronic pen 12b alternately blink in a time-division manner. More specifically, the IR LED 13 of the electronic pen 12a and the IR LED 13 of the electronic pen 12b alternately blink at a time interval obtained by dividing a predetermined recognition period t0 in half. Note that the recognition period t0 is set to a time during which the camera 20 captures an image at least once while the IR LED 13 is turned on when the IR LED 13 is blinked, and the light emission recognition unit 32 can recognize the bright point position. In other words, while the flashing is repeated, the information processing device 10a can grasp the timing at which the electronic pens 12a and 12b emit light, and therefore the information processing device 10a can continue tracking the bright spot positions of the electronic pens 12a and 12b based on the bright spot positions recognized by the light emission recognition unit 32.

[0107] 10, when the electronic pens 12a and 12b are separated from each other by a distance greater than the threshold distance d, the information processing device 10a transmits a turn-on command to the electronic pens 12a and 12b, so that the IR LED 13 of the electronic pen 12a and the IR LED 13 of the electronic pen 12b are both turned on after the time tb.

[0108] The time chart in Figure 11 shows the light emission state of the IR LED 13 when the three electronic pens 12a (ID: AA), 12b (ID: BB), and 12c (ID: CC) are all in an input state and approach each other within a threshold distance d.

[0109] At time tc in FIG. 11, the electronic pens 12a, 12b, and 12c approach within the threshold distance d, and the condition of state (a) described in FIG. 9 is met. After time tc, the IR LED 13 of the electronic pen 12a, the IR LED 13 of the electronic pen 12b, and the IR LED 13 of the electronic pen 12c alternately blink in a time division manner. More specifically, the IR LED 13 of the electronic pen 12a, the IR LED 13 of the electronic pen 12b, and the IR LED 13 of the electronic pen 12c alternately blink at a time interval obtained by dividing the recognition period t0 into three. While the IR LED 13 alternately blinks, the information processing device 10a can grasp the timing at which the electronic pens 12a, 12b, and 12c emit light. Therefore, the information processing device 10a can continue tracking the bright spot positions of the electronic pens 12a, 12b, and 12c based on the bright spot positions recognized by the light emission recognition unit 32.

[0110] Then, at time td in FIG. 11, when the electronic pen 12a is separated from the electronic pens 12b and 12c by more than the threshold distance d, the information processing device 10a transmits a turn-on command to the electronic pen 12a. As a result, the IR LED 13 of the electronic pen 12a is turned on after time td. Since the electronic pens 12b and 12c remain close to each other within the threshold distance d, the IR LED 13 of the electronic pen 12b and the IR LED 13 of the electronic pen 12c alternately blink at a time interval obtained by dividing the recognition period t0 in half. In the example of FIG. 11, the blinking time of the IR LED 13 is set to be constant. Therefore, the recognition period t0 between time tc and time td when the three electronic pens 12 approach each other is set to be longer than the recognition period t0 after time td when the two electronic pens 12 approach each other.

[0111] Next, the flow of the process of determining whether or not the ID of the electronic pen has been swapped will be described with reference to Fig. 12. Fig. 12 is a flow chart showing an example of the flow of the process of determining whether or not the ID of the electronic pen has been swapped.

[0112] The light emission recognition unit 32 judges whether there is another bright spot within the threshold distance d from the bright spot of interest (step S51). If it is judged that there is another bright spot within the threshold distance d from the bright spot of interest (step S51: Yes), the process proceeds to step S52. On the other hand, if it is not judged that there is another bright spot within the threshold distance d from the bright spot of interest (step S51: No), the process proceeds to step S58.

[0113] If the answer to step S51 is Yes, the ID swapping prevention determination unit 34b determines whether the focused bright spot (the associated electronic pen 12) is in an input state (step S52). If the focused bright spot (the associated electronic pen 12) is determined to be in an input state (step S52: Yes), the process proceeds to step S53. On the other hand, if the focused bright spot (the associated electronic pen 12) is not determined to be in an input state (step S52: No), the process proceeds to step S54.

[0114] If the answer to step S52 is Yes, the ID swapping prevention determination unit 34b determines whether the other bright spot (the electronic pen 12 associated with it) is in an input state (step S53). If it is determined that the other bright spot (the electronic pen 12 associated with it) is in an input state (step S53: Yes), the process proceeds to step S55. On the other hand, if it is determined that the other bright spot (the electronic pen 12 associated with it) is not in an input state (step S53: No), the process proceeds to step S56.

[0115] Returning to step S52, if the result of the determination in step S52 is No, the ID swapping prevention determination unit 34b determines whether the other bright spot (the electronic pen 12 associated with it) is in an input state (step S54). If it is determined that the other bright spot (the electronic pen 12 associated with it) is in an input state (step S54: Yes), the process proceeds to step S57. On the other hand, if it is determined that the other bright spot (the electronic pen 12 associated with it) is not in an input state (step S54: No), the process proceeds to step S55.

[0116] If the result of the determination in step S53 is Yes or the result of the determination in step S54 is No, the light emission control unit 34a sets a control command to blink the electronic pen 12 related to the bright spot of interest and the electronic pen 12 related to another bright spot in a time-division manner (step S55). Then, the process in FIG. 12 ends and the process returns to the main routine (step S24 in FIG. 4).

[0117] If the result of the determination in step S53 is No, the light emission control unit 34a sets a control command to turn on the electronic pen 12 associated with the bright spot of interest and turn off the electronic pen 12 associated with another bright spot (step S56). Then, the process of FIG. 12 ends and the process returns to the main routine (step S24 in FIG. 4).

[0118] If the result of the determination in step S54 is Yes, the light emission control unit 34a sets a control command to turn off the electronic pen 12 associated with the bright spot of interest and turn on the electronic pen 12 associated with another bright spot (step S57). Then, the process in FIG. 12 ends and the process returns to the main routine (step S24 in FIG. 4).

[0119] If the result of the determination in step S51 is No, the light emission control unit 34a sets a control command to turn on the electronic pen 12 associated with the bright spot of interest (step S58). After that, the process in FIG. 12 ends and the process returns to the main routine (step S24 in FIG. 4).

[0120] [1-8. Explanation of electronic pen state transitions] Next, the state transition of the electronic pen 12 will be described with reference to Fig. 13. Fig. 13 is a state transition diagram showing an example of the state transition of the electronic pen.

[0121] 13, the interactive projector 5a is in an initial state with the power turned off. When the interactive projector 5a is turned on in the state Sa, the BT connection between the information processing device 10a and the electronic pen 12 is turned on. As a result, the interactive projector 5a transitions to the state Sb. In the state Sb, the IR LED 13 is turned off, and the identification number of the electronic pen 12 is also undefined.

[0122] When the input / output control unit 36 ​​of the information processing device 10a receives a BT event for turning on the IR LED 13 from the electronic pen 12, the light emission control unit 34a transmits a light emission instruction for the IR LED 13 to the electronic pen 12. This causes the IR LED 13 of the electronic pen 12 to turn on. However, the identification number of the electronic pen 12 remains indefinite (state Sc).

[0123] The light emission recognition unit 32 of the information processing device 10a acquires the position of a bright spot that occurs when the IR LED 13 of the electronic pen 12 lights up or blinks. When the light emission recognition unit 32 detects the position of the bright spot, the association processing unit 34d associates the position of the bright spot with the identification number of the electronic pen 12. As a result, the IR LED 13 continues to light up or blink, and the identification number of the electronic pen 12 is determined (state Sd).

[0124] On the other hand, if the light emission recognition unit 32 cannot acquire the position of the bright point within the predetermined time, the identification number of the electronic pen 12 remains indefinite (state Se).

[0125] In state Sd, when the pen tip of the electronic pen 12 touches the whiteboard 26 and the switch 14 is turned ON, the electronic pen 12 transmits a switch-ON BT event. After that, the IR LED 13 continues to light or blink, and the positions of the bright points that are successively recognized are linked to the identification number of the electronic pen 12, and the projector 22 projects an image (the handwriting of the electronic pen 12) onto the positions of the detected bright points, thereby drawing (state Sf).

[0126] In state Sf, when the pen tip of the electronic pen 12 is removed from the whiteboard 26, the switch 14 is turned OFF. Then, the electronic pen 12 transmits a switch-OFF BT event. After that, the IR LED 13 continues to light up or blink, and the positions of the bright points that are successively recognized are linked to the identification number of the electronic pen 12, and the position of the electronic pen 12 is tracked (state Sg).

[0127] In addition, when in states Sd, Sf, or Sg, if the light emission recognition unit 32 of the information processing device 10a cannot detect the position of the bright spot (if the IR LED 13 is lost), the link between the position of the bright spot and the identification number of the electronic pen 12 is released, and the state transitions to Se.

[0128] [1-9. Explanation of how to reflect input results from the electronic pen] Next, a method for reflecting the input result of the electronic pen, in which the input result by the electronic pen 12 is projected on the whiteboard 26 by the projector 22, will be described with reference to Fig. 14. Fig. 14 is a diagram for explaining the flow of the process for converting the detected bright point position from the camera coordinate system to the projector screen coordinate system.

[0129] Information written by the electronic pen 12 is reflected when the bright spot position is detected, the ID is identified, and the switch 14 is ON, and is projected from the projector 22 onto the whiteboard 26. This corresponds to the state Sf of FIG.

[0130] Since the coordinates (x, y) of the detected bright spot position are in the camera coordinate system, as shown in Fig. 14, the coordinates detected by the camera are multiplied by a two-dimensional to two-dimensional projection transformation matrix (homography matrix) to convert from the camera coordinate system to the screen coordinate system projected by the projector 22. The converted coordinates (x', y') are reflected in the application as input coordinates.

[0131] [1-10. Effects of the first embodiment] As described above, according to the information processing device 10a of the first embodiment, the light emission control unit 34a lights up the IR LED 13 (light emission unit) of the electronic pen 12 (operation body) that has established communication with the information processing device 10a, and when a different electronic pen 12 (operation body) approaches, causes the IR LED 13 of the electronic pen 12 to blink at different timings. Then, the light emission recognition unit 32 (detection unit) detects a position corresponding to the IR LED 13 of each electronic pen 12 in association with the electronic pen 12. When the electronic pen 12 is in an information input state, the input / output control unit 36 ​​(display control unit) displays information associated with the electronic pen 12 at a position corresponding to the IR LED 13 detected by the light emission recognition unit 32.

[0132] As a result, even if a plurality of electronic pens 12 are located close to each other, the positions corresponding to the respective electronic pens 12 can be identified and captured reliably.

[0133] Furthermore, according to the information processing device 10a of the first embodiment, the light emission control unit 34a determines that multiple electronic pens 12 (operating objects) have approached when the position corresponding to the IR LED 13 (light emission unit) detected by the light emission recognition unit 32 (detection unit) approaches within a threshold distance d.

[0134] This makes it possible to recognize that a plurality of electronic pens 12 are approaching each other through simple processing.

[0135] Furthermore, according to the information processing device 10a of the first embodiment, the light emission control unit 34a blinks the IR LED 13 (light emission unit) of the electronic pen 12 at different timings every time the predetermined time is divided by the number of electronic pens 12 (operating objects) approaching within the threshold distance d, which is at least the time required for image input.

[0136] As a result, the IR LEDs 13 of the electronic pens 12 that are close to each other within the threshold distance d blink in a time-division manner, so that it is possible to identify the position of the bright spot generated by the IR LEDs 13 of each electronic pen 12 at the timing when the electronic pen 12 is turned on. Therefore, it is possible to prevent the identification numbers (IDs) given to the electronic pens 12 from being interchanged.

[0137] Furthermore, according to the information processing device 10a of the first embodiment, when a bright spot position (position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 of another electronic pen 12 is detected within a threshold distance d from a bright spot position (position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body), and both the electronic pen 12 and the other electronic pen 12 are in an information input state, the light-emitting control unit 34a causes the IR LED 13 of the electronic pen 12 and the IR LED 13 of the other electronic pen 12 to blink at different timings.

[0138] This makes it possible to prevent the IDs of multiple electronic pens 12 (operating objects) from being swapped.

[0139] Furthermore, according to the information processing device 10a of the first embodiment, when a bright spot position (position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 of another electronic pen 12 is detected within a threshold distance d from a bright spot position (position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body), and the electronic pen 12 is in an information input state and the other electronic pen 12 is not in an information input state, the light-emitting control unit 34a turns on the IR LED 13 of the electronic pen 12 and turns off the IR LED 13 of the other electronic pen 12.

[0140] This makes it possible to reliably track the position of the bright spot caused by the IR LED 13 of the electronic pen 12 in an information input state without decreasing the frame rate.

[0141] Furthermore, according to the information processing device 10a of the first embodiment, when a bright spot position (position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 of another electronic pen 12 is detected within a threshold distance d from a bright spot position (position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body), and the electronic pen 12 is not in an information input state but the other electronic pen 12 is in an information input state, the light-emitting control unit 34a turns off the IR LED 13 of the electronic pen 12 and turns on the IR LED 13 of the other electronic pen 12.

[0142] This makes it possible to reliably track the position of the bright spot caused by the IR LED 13 of the electronic pen 12 in a non-information input state without decreasing the frame rate.

[0143] Furthermore, according to the information processing device 10a of the first embodiment, when a position corresponding to the IR LED 13 of another electronic pen 12 is detected within a threshold distance d from a position corresponding to the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body), and both the electronic pen 12 and the other electronic pen 12 are not in an information input state, the light-emitting control unit 34a causes the IR LED 13 of the electronic pen 12 and the IR LED 13 of the other electronic pen 12 to blink at different timings.

[0144] This makes it possible to continue tracking the position of the electronic pen 12 when it is not inputting information.

[0145] 2. Second Embodiment When the task performed by the electronic pen 12 is a simple one such as button operation, as described in the first embodiment, the decrease in frame rate when the IR LEDs 13 are blinked in a time-division manner when multiple electronic pens 12 approach each other is not a significant problem. On the other hand, when the electronic pen 12 is writing characters or drawing pictures, it is desirable to suppress the decrease in frame rate as much as possible.

[0146] In order to address such issues, an information processing device 10b (not shown) according to a second embodiment of the present disclosure is an example that adds a function of varying the method of controlling the light emission of the IR LED 13 when multiple electronic pens 12 approach within a threshold distance d depending on the content of the task being performed by the electronic pen 12.

[0147] In other words, when the information processing device 10b blinks the IR LEDs 13 of multiple electronic pens 12 in an information input state in a time-division manner, it changes the time-division ratio for each electronic pen 12 depending on the type of task that each electronic pen 12 is executing.

[0148] Specifically, for electronic pen 12 that is writing characters or drawing pictures, it is important to capture changes in drawing position over time, so by lighting IR LED 13 for a longer period of time, the decrease in frame rate is kept small. On the other hand, for electronic pen 12 that is used for button operations, etc., it is important to capture the instantaneous position, so a low frame rate is not a big problem. Therefore, it is possible to set the lighting time of IR LED 13 to be short.

[0149] Therefore, the information processing device 10b has a function of determining the content of the task being performed by the electronic pen 12. The content of the task being executed can be determined, for example, by referring to the operation content of an operation menu. Although not shown, the information processing device 10b performs the selected operation after, for example, selecting an operation menu projected on the whiteboard 26 with the electronic pen 12. At that time, when an operation menu such as "drawing" or "writing" is selected, information indicating the task type corresponding to "drawing" or "writing" is assigned to the identification number (ID) of the electronic pen 12 that selected the menu. On the other hand, when an operation menu such as "button operation" is selected, information indicating the task type corresponding to "button operation" is assigned to the identification number (ID) of the electronic pen 12 that selected the menu.

[0150] Then, when the above-mentioned ID swap prevention determination process (FIG. 12) determines that multiple electronic pens 12 are approaching within the threshold distance d, the light emission control unit 34a time-divides the recognition period t0 (predetermined time) shown in FIG. 10 and FIG. 11 according to the task type information assigned to the approaching electronic pen 12.

[0151] As described above, in the information processing device 10b of the second embodiment, the light emission control unit 34a controls the proportion of the lighting time of the IR LED 13 (light emission unit) of each electronic pen 12 within the recognition period t0 (predetermined time) in accordance with the content of the task being executed by the electronic pen 12 (operation object).

[0152] This allows the lighting time of the IR LED 13 to be set long when writing characters or drawing pictures, and the lighting time of the IR LED 13 to be set short when operating instructions such as button operations are being performed. Therefore, when performing a task that requires tracking of the drawing position, the decrease in frame rate can be kept small.

[0153] 3. Third embodiment 1, when multiple operators hold electronic pens 12 and write on the same whiteboard 26, the multiple electronic pens 12 may approach each other within a threshold distance d without the operators realizing it. In such a case, it is desirable to alert the operators that the electronic pens 12 are approaching each other before performing the above-mentioned ID swap prevention determination process.

[0154] In order to address such issues, the information processing device 10c (not shown), which is a third embodiment of the present disclosure, is an example in which, in contrast to the information processing device 10a described in the first embodiment, a function is added to warn the operator of the electronic pen 12 to move away when multiple electronic pens 12 approach within a threshold distance d.

[0155] Specifically, when the light emission control unit 34a determines that a plurality of electronic pens 12 have approached within the threshold distance d, the input / output control unit 36 ​​may instruct the projector 22 to project information in the form of blinking handwriting drawn by the approaching electronic pen 12. Also, the electronic pen 12 may notify the approach of another electronic pen 12 by vibrating vibration means such as a vibrator built in the electronic pen 12.

[0156] As described above, according to the information processing device 10c of the third embodiment, when the bright spot positions (positions corresponding to the light-emitting parts of the operating bodies) of the IR LEDs 13 (light-emitting parts) of different electronic pens 12 (operating bodies) approach within a threshold distance d, the input / output control unit 36 ​​(display control unit) causes the projector 22 to display information encouraging the user to move the electronic pen 12 away.

[0157] This makes it possible to intuitively inform the user that another electronic pen 12 is approaching.

[0158] 4. Fourth Embodiment In each of the above-described embodiments, a threshold distance d, which is a preset fixed value, is used when determining whether or not multiple electronic pens 12 are approaching each other. Therefore, when the electronic pens 12 move quickly, it may be difficult to detect a state in which multiple electronic pens 12 are approaching each other within the threshold distance d.

[0159] In order to address such issues, an information processing device 10d (not shown) according to a fourth embodiment of the present disclosure is an example that has a function for making the threshold distance d used when determining whether multiple electronic pens 12 are close to each other variable depending on the movement speed and movement direction of the electronic pens 12.

[0160] Specifically, the information processing device 10d continuously monitors the moving speed of each electronic pen 12, and increases the threshold distance d when the moving speed of the electronic pen 12 is high or the acceleration is increasing. That is, a circular area of ​​radius d / 2 centered on the position of the bright spot detected by the light emission recognition unit 32 moves along the handwriting of the electronic pen 12 while constantly changing size. Then, the light emission control unit 34a determines that another electronic pen 12 has approached when the position of the bright spot by the other electronic pen 12 enters the circular area of ​​radius d / 2.

[0161] The threshold distance d may be changed in consideration of the movement of the electronic pen 12 near the electronic pen 12. That is, when the two electronic pens 12 are moving in directions toward each other, the threshold distance d may be changed to be larger. When the two electronic pens 12 are moving in directions away from each other, the threshold distance d may be changed to be smaller.

[0162] That is, the information processing device 10d continuously monitors the moving speed and moving direction of each electronic pen 12, and each time, sets a circular area with a radius of d / 2 centered on the position of each bright point.

[0163] As described above, according to the information processing device 10d of the fourth embodiment, the threshold distance d is set based on the moving speed or moving direction of the electronic pen 12 (operating object).

[0164] This makes it possible to set an appropriate threshold distance d depending on the situation in which the electronic pen 12 is being used, thereby further improving the robustness of the ID swap prevention determination process.

[0165] 5. Fifth Embodiment An information processing device 10e (not shown) according to a fifth embodiment of the present disclosure is an example having a function for dealing with a case in which multiple groups are formed, each group including multiple electronic pens 12 that are close to each other within a threshold distance d, and each group includes the same electronic pen 12.

[0166] Fig. 15 is a diagram showing an example of a state in which a part of a group including a plurality of electronic pens approaching each other within a threshold distance overlaps. As shown in Fig. 15, it is assumed that a bright spot P1 by an electronic pen 12 having an identification number "AA" and a bright spot P2 by an electronic pen 12 having an identification number "BB" approach each other within a threshold distance d. It is further assumed that a bright spot P2 by an electronic pen 12 having an identification number "BB", a bright spot P3 by an electronic pen 12 having an identification number "CC", and a bright spot P4 by an electronic pen 12 having an identification number "DD" approach each other within a threshold distance d. That is, it is assumed that the bright spots P1 and P2 by the electronic pens 12 having the identification numbers "AA" and "BB" belong to a group U1, and the bright spots P2, P3, and P4 by the electronic pens 12 having the identification numbers "BB", "CC", and "DD" belong to a group U2.

[0167] In such a case, the light emission control unit 34a may blink the IR LEDs 13 of the four electronic pens 12 belonging to the union (U1 ∪ U2) of the group U1 and the group U2 in a time-division manner (see FIG. 16). The light emission control unit 34a may also blink the IR LEDs 13 of the electronic pens 12 belonging to the group U1 so that their lighting periods do not overlap, and may also blink the IR LEDs 13 of the electronic pens 12 belonging to the group U2 so that their lighting periods do not overlap. That is, the IR LEDs 13 of the electronic pens 12 belonging to different groups, for example, the IR LEDs 13 of the electronic pens 12 belonging to the group U1 and having the identification number "AA" and the IR LEDs 13 of the electronic pens 12 belonging to the group U2 and having the identification number "DD" may be blinked at the same timing (see FIG. 17). Alternatively, the IR LEDs 13 of the electronic pens 12 belonging to the group U1 and having the identification number "AA" and the IR LEDs 13 of the electronic pens 12 belonging to the group U2 and having the identification number "CC" may be blinked at the same timing.

[0168] FIG. 16 is a time chart showing an example of time-division emission of the IR LEDs of all electronic pens belonging to a union.

[0169] As shown in FIG. 16, by blinking the IR LEDs 13 of the four electronic pens 12 belonging to the union of group U1 and group U2 (U1∪U2) at timings that divide the recognition period t0 into four, the positions of all the electronic pens 12 belonging to the union can be reliably identified.

[0170] FIG. 17 is a time chart showing an example of blinking the IR LED of the electronic pen so that the light emission patterns do not overlap within each group.

[0171] 17, the IR LED 13 of the electronic pen 12 having the identification number "AA" belonging to group U1 and the IR LED 13 of the electronic pen 12 having the identification number "DD" belonging to group U2 are caused to blink at the same timing. As a result, the IR LED 13 of the two electronic pens 12 belonging to group U1 blink at different timings, so that the information processing device 10e can reliably identify the bright spot positions of the IR LED 13 of each electronic pen 12 belonging to group U1. Also, the IR LED 13 of the three electronic pens 12 belonging to group U2 blink at different timings, so that the information processing device 10e can reliably identify the bright spot positions of the IR LED 13 of each electronic pen 12 belonging to group U2.

[0172] As described above, according to the information processing device 10e of the fifth embodiment, when the bright spot position (position corresponding to the light-emitting part of the operating body) caused by the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body) and the bright spot positions caused by the IR LED 13 of multiple electronic pens 12 other than the electronic pen 12 are close to each other within the threshold distance d, and further, among the positions corresponding to the IR LED 13 of the multiple electronic pens 12 other than the electronic pen 12, there is a position that is farther away than the threshold distance d, the light-emitting control unit 34a causes the IR LEDs 13 of all electronic pens 12 that are close to each other within the threshold distance d from the position corresponding to the IR LED 13 of the electronic pen 12 to blink at different timings.

[0173] As a result, even if bright spot positions by multiple other electronic pens 12 are observed within the threshold distance d from the bright spot position by the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body) and even if there is a bright spot position among the bright spot positions by the multiple other electronic pens 12 that is farther away than the threshold distance d, the bright spot position by the IR LED 13 of each electronic pen 12 can be reliably identified.

[0174] Furthermore, according to the information processing device 10e of the fifth embodiment, when a bright spot position (a position corresponding to the light-emitting unit of the operating body) caused by the IR LED 13 (light-emitting unit) of the electronic pen 12 (operating body) and a bright spot position caused by the IR LED 13 of a plurality of electronic pens 12 other than the electronic pen 12 are close to each other within a threshold distance d, and furthermore, among the positions corresponding to the IR LED 13 of the plurality of electronic pens 12 other than the electronic pen 12, there is a position that is farther away than the threshold distance d, the light-emitting control unit 34a blinks at least the IR LED 13 associated with the bright spot positions that are close to each other within the threshold distance d at different timings.

[0175] As a result, even if bright spot positions by a plurality of other electronic pens 12 are observed within the threshold distance d from the bright spot position by the IR LED 13 (light emitting unit) of the electronic pen 12 (operation body), and even if a bright spot position by the plurality of other electronic pens 12 is located farther away than the threshold distance d, it is possible to reliably identify the bright spot position by the IR LED 13 of each electronic pen 12. Also, it is possible to minimize a decrease in the frame rate.

[0176] (6. Application Example (1) of the Present Disclosure) Here, application examples of the present disclosure will be described. The present disclosure can also be applied to information input devices other than the electronic pen 12. That is, if a controller-type device having both a function of detecting a position from a sensor such as a camera and a communication function in an arbitrary space is available, the device can be treated as an input device similar to the electronic pen 12 described in each of the above-mentioned embodiments. Note that, in each of the above-mentioned embodiments, the switch 14 is incorporated to detect the input state of the electronic pen 12, but the switch 14 is not essential as long as it is possible to identify the position and ID of the target device.

[0177] FIG. 18 is a schematic diagram of a shooting system which is an application example of the present disclosure. In the shooting system 50, a user can shoot an object (e.g., an enemy character) displayed on a wall surface using a gun-shaped controller 52 (52a, 52b) which is an example of an operating body in the present disclosure held by the user. In such a game application, the contents of the present disclosure can be applied by mounting a highly directional IR LED on the gun-shaped controller 52 and observing the wall surface illuminated by the IR LED with a camera. Even if multiple users aim their guns at nearby locations or the same location, that is, even if the bright points 54a and 54b observed by the camera are observed in the same location, it is possible to identify which gun illuminated the bright point.

[0178] (7. Application Example (2) of the Present Disclosure) FIG. 19 is a first diagram showing an outline of a miniature car race system which is an application example of the present disclosure. FIG. 20 is a second diagram showing an outline of a miniature car race system which is an application example of the present disclosure. This example is an example in which the present disclosure is applied to a racing system 60 in which a race is held with miniature cars such as mini 4WD (registered trademark) and radio-controlled cars. By mounting an IR LED and a communication module on a car 62 (62a, 62b) which is an example of an operating body in the present disclosure, and installing a camera for detecting the IR LED (for example, a camera for capturing an overhead image of the road on which the cars are running), it is possible to identify multiple cars 62. In such a case, the present invention can identify cars even in a scene where the cars are close to each other. By applying the present disclosure, it is possible to perform a performance with visual effects by projecting information in real time at the position of the car 62. In addition, since the positions of the multiple cars 62 are known, by mounting a front projection projector on the car 62, it is possible to perform effects such as an attack effect or a power-up effect on the other car by the display image by the front projection projector, as shown in FIG. 20.

[0179] (8. Application Example (3) of the Present Disclosure) FIG. 21 is a diagram showing an overview of a top-shaped spinning top competition system, which is an application example of the present disclosure. This example is an example in which the present disclosure is applied to a top-shaped spinning top competition system 70. By incorporating an IR LED and a communication module inside the tops 72 (72a, 72b), which are an example of an operating body in the present disclosure, the camera 20 can identify multiple tops. In addition, the tops 72 can be identified even in cases where they collide or come close to each other during a match on stage. By applying the present disclosure, it is also possible to perform a performance that adds visual effects, such as projection mapping, based on the position of the detected tops 72 or the timing when the tops 72 collide with each other.

[0180] (9. Application Example (4) of the Present Disclosure) FIG. 22 is a diagram showing an outline of an air hockey system which is an application example of the present disclosure. This example is an example in which the present disclosure is applied to an air hockey system 80. In this application example, an IR LED and a communication module are incorporated into a mallet 82 (82a, 82b) which is an example of an operating body in the present disclosure held by a user, and the mallet 82 of the user can be identified. When users play cooperatively, different mallets 82 may be close to each other, but by applying the present disclosure, it is possible to identify each mallet 82 even in such a case. It is also possible to incorporate a similar mechanism into a puck to identify multiple pucks. Furthermore, by detecting the positions of the puck and the mallet, it is possible to perform a performance that adds visual effects, a collision judgment between the puck and the mallet, and a performance according to the event. It is also possible to apply the present invention to a game in which players hit projected virtual pucks against each other.

[0181] (10. Application Example (5) of the Present Disclosure) FIG. 23 is a diagram showing an outline of a bouldering system which is an application example of the present disclosure. This is an example of a bouldering system 90 applied so that a user can detect the positions of his / her hands and feet by wearing devices 91 and 92 which are an example of an operating body in the present disclosure on his / her hands and feet. The devices 91 and 92 are equipped with an IR LED and a communication module. Specifically, one user wears devices 91a, 91b, 91c, and 91d on his / her right hand, left hand, right foot, and left foot, respectively. Another user wears devices 92a, 92b, 92c, and 92d on his / her right hand, left hand, right foot, and left foot, respectively.

[0182] In FIG. 23, a user wearing devices 91 and 92 is playing bouldering. At this time, the positions of the user's hands and feet can be determined from the positions of the devices 91 and 92 detected by a camera (not shown), so the bouldering system 90 can recognize the holds used by the user and perform effects according to the results. According to the present invention, even if the IR LEDs attached to the user's hands and feet overlap when viewed from the camera due to the user's movement, each device 91 and 92 can be identified and detected. In addition, multiple users may be identified and the places where each user climbed may be identified like a territory capture game. In this case, by grouping the devices used by each user in advance (for example, grouping the IDs of the four devices worn on the right hand, left hand, right foot, and left foot), the user information and the detected part can be identified.

[0183] (11. Application Example of This Disclosure (6)) 24 is a diagram showing an overview of a group robot control system which is an application example of the present disclosure. The group robot control system 94 uses multiple cameras 20a, 20b installed upward facing downward to detect the positions of multiple robots 95 (95a, 95b, 95c, ...) of the same form, which is an example of an operating body in the present disclosure, and controls the behavior of each robot. Each robot 95 has a built-in IR LED that emits light upward, and the cameras 20a, 20b detect the position of the bright point of the IR LED to detect the current position of each robot 95.

[0184] In the example of Fig. 24, the contents of the present disclosure are used to identify the position and ID of each robot 95, which are robots of the same form and are called swarm robots. Therefore, even in a case where multiple robots 95 are crowded together, that is, even in a state where the IR LEDs installed on each robot 95 are close to each other, it is possible to identify the position of each robot 95 and track them. Such a swarm robot control system 94 may be applied to other devices such as drones, or may be applied to living things such as humans.

[0185] (12. Application Example of This Disclosure (7)) FIG. 25 is a diagram showing an outline of an AR / VR application system which is an application example of the present disclosure. The contents of the present disclosure may be applied to position detection of a device worn by a user, such as a head mounted display (HMD). Specifically, as shown in FIG. 25, by mounting an IR LED and a communication module on an HMD 97 (97a, 97b, 97c, ...), which is an example of an operating body in the present disclosure, in a scene in which multiple users wearing the HMD 97 are in the same space, it is possible to identify the position of each user and the ID (user ID) given to the HMD 97. This makes it possible to reflect, for example, the position of a user experiencing a virtual space in a position in the real world. In addition, in order to prevent collisions between users, it is possible to alert nearby users that there is a possibility of contact with another user.

[0186] In addition, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may be achieved. In addition, the embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit and scope of the present disclosure.

[0187] For example, the present disclosure can also be configured as follows.

[0188] (1) a light emission control unit that turns on a light emitting unit included in an operating object that has established communication with the information processing device, and when a different operating object approaches, causes the light emitting unit included in the operating object to blink at different timings; a detection unit that detects a position corresponding to the light-emitting unit included in each of the operating objects in association with the operating object; a display control unit that displays information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection unit when the operating object is in an information input state; An information processing device comprising: (2) The light emission control unit determines that the plurality of operating objects have approached when the positions corresponding to the light emitting units detected by the detection unit have approached within a threshold distance. The information processing device according to (1). (3) The light emission control unit blinking a light-emitting unit of the operating object at different timings every time the predetermined time is exceeded by a time required for image input, the time being calculated by dividing the predetermined time by the number of the operating objects approaching within the threshold distance; The information processing device according to (2). (4) The light emission control unit When a position corresponding to a light-emitting unit of another operating body is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating body, and both the operating body and the other operating body are in an information input state, blinking a light emitting unit of the operating body and a light emitting unit of the other operating body at different timings; The information processing device according to (2) or (3). (5) The light emission control unit When a position corresponding to a light-emitting unit of another operating object is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating object, and the operating object is in an information input state and the other operating object is not in an information input state, turning on a light-emitting unit of the operating body and turning off a light-emitting unit of the other operating body; The information processing device according to any one of (2) to (4). (6) The light emission control unit When a position corresponding to a light-emitting unit of another operating object is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating object, and the operating object is not in an information input state and the other operating object is in an information input state, turning off the light-emitting unit of the operating object and turning on the light-emitting unit of the other operating object; 13. The information processing device according to claim 12, wherein the first and second information processing devices are connected to each other. (7) The light emission control unit When a position corresponding to a light-emitting unit of another operating body is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating body, and both the operating body and the other operating body are not in an information input state, blinking a light emitting unit of the operating body and a light emitting unit of the other operating body at different timings; The information processing device according to any one of (2) to (6). (8) The light emission control unit controlling a ratio of a lighting time of the light-emitting unit of each of the operating objects within the predetermined time period according to the content of the task being executed by the operating object; The information processing device according to (3). (9) The display control unit is When the positions of the different operating objects corresponding to the light-emitting units approach each other within the threshold distance, information is displayed to prompt the user to move the operating objects away from each other. The information processing device according to any one of (2) to (8). (10) The threshold distance is: is set based on the moving speed or moving direction of the operation object, The information processing device according to any one of (2) to (9). (11) When a position corresponding to the light-emitting unit of the operating body and positions corresponding to the light-emitting units of a plurality of operating bodies other than the operating body are close to each other within the threshold distance, and among the positions corresponding to the light-emitting units of the other plurality of operating bodies, there is a position that is farther away than the threshold distance, the light emission control unit blinks the light emitting units of all of the operating bodies that are approaching within the threshold distance from a position corresponding to the light emitting unit of the operating body at different timings. The information processing device according to any one of (2) to (10). (12) When a position corresponding to the light-emitting unit of the operating body and positions corresponding to the light-emitting units of a plurality of operating bodies other than the operating body are close to each other within the threshold distance, and among the positions corresponding to the light-emitting units of the other plurality of operating bodies, there is a position that is farther away than the threshold distance, The light emission control unit blinks at least the light emitting units associated with the positions that are approaching within the threshold distance at different timings. The information processing device according to any one of (2) to (10). (13) a light emission control process for turning on a light emitting unit included in an operating object that has established communication with the information processing device, and blinking the light emitting unit included in a different operating object at different timings when the operating object approaches another operating object; a detection process for detecting a position of the light-emitting unit included in each of the operating objects in association with the operating object; a display control process for displaying information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection process when the operating object is in an information input state; An information processing method comprising: (14) Computer, a light emission control unit that turns on a light emitting unit included in an operating object that has established communication with the information processing device, and when a different operating object approaches, causes the light emitting unit included in the operating object to blink at different timings; A detection unit that detects a position of the light-emitting unit included in each of the operating objects in association with the operating object; a display control unit that displays information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection unit when the operating object is in an information input state; A program that makes it work. [Explanation of symbols]

[0189] 5a... interactive projector, 10a, 10b, 10c, 10d, 10e... information processing device, 12, 12a, 12b, 12c... electronic pen (operation body), 13... IR LED (light emitting unit), 14... switch, 20... camera, 22... projector, 26... whiteboard, 32... light emission recognition unit (detection unit), 34a... light emission control unit, 36... input / output control unit (display control unit), d... threshold distance, t0... recognition period (predetermined time)

Claims

1. a light emission control unit that turns on a light emitting unit included in an operating object that has established communication with the information processing device, and when a different operating object approaches, blinks the light emitting unit included in the operating object at different timings; a detection unit that detects a position corresponding to the light-emitting unit included in each of the operating objects in association with the operating object; a display control unit that displays information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection unit when the operating object is in an information input state; Equipped with The light emission control unit is When the positions corresponding to the light-emitting units detected by the detection unit approach within a threshold distance, it is determined that the plurality of operating objects have approached each other; blinking a light-emitting unit of the operating object at different timings every time the predetermined time is exceeded by a time required for image input, the time being calculated by dividing the predetermined time by the number of the operating objects approaching within the threshold distance; controlling a ratio of a lighting time of the light-emitting unit of each of the operating objects within the predetermined time period according to the content of the task being executed by the operating object; Information processing device.

2. The light emission control unit is When a position corresponding to a light-emitting unit of another operating body is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating body, and both the operating body and the other operating body are in an information input state, blinking a light emitting unit of the operating body and a light emitting unit of the other operating body at different timings; The information processing device according to claim 1 .

3. The light emission control unit is When a position corresponding to a light-emitting unit of another operating object is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating object, and the operating object is in an information input state and the other operating object is not in an information input state, turning on a light-emitting unit of the operating body and turning off a light-emitting unit of the other operating body; The information processing device according to claim 1 .

4. The light emission control unit is When a position corresponding to a light-emitting unit of another operating object is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating object, and the operating object is not in an information input state and the other operating object is in an information input state, turning off the light-emitting unit of the operating object and turning on the light-emitting unit of the other operating object; The information processing device according to claim 1 .

5. The light emission control unit is When a position corresponding to a light-emitting unit of another operating body is detected within the threshold distance from a position corresponding to a light-emitting unit of the operating body, and both the operating body and the other operating body are not in an information input state, blinking a light emitting unit of the operating body and a light emitting unit of the other operating body at different timings; The information processing device according to claim 1 .

6. The display control unit is When the positions of the different operating objects corresponding to the light-emitting units approach each other within the threshold distance, information is displayed to prompt the user to move the operating objects away from each other. The information processing device according to claim 1 .

7. The threshold distance is: is set based on the moving speed or moving direction of the operation object, The information processing device according to claim 1 .

8. When a position corresponding to the light-emitting unit of the operating body and positions corresponding to the light-emitting units of a plurality of operating bodies other than the operating body are close to each other within the threshold distance, and among the positions corresponding to the light-emitting units of the other plurality of operating bodies, there is a position that is farther away than the threshold distance, the light emission control unit blinks the light emitting units of all of the operating bodies that are approaching within the threshold distance from a position corresponding to the light emitting unit of the operating body at different timings. The information processing device according to claim 1 .

9. When a position corresponding to the light-emitting unit of the operating body and positions corresponding to the light-emitting units of a plurality of operating bodies other than the operating body are close to each other within the threshold distance, and among the positions corresponding to the light-emitting units of the other plurality of operating bodies, there is a position that is farther away than the threshold distance, The light emission control unit blinks at least the light emitting units associated with the positions that are approaching within the threshold distance at different timings. The information processing device according to claim 1 .

10. a light emission control process for turning on a light emitting unit included in an operating object that has established communication with the information processing device, and blinking the light emitting unit included in the operating object at different timings when a different operating object approaches the operating object; a detection process for detecting a position corresponding to the light-emitting unit included in each of the manipulation objects in association with the manipulation object; a display control process for displaying information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection process when the operating object is in an information input state; Equipped with The light emission control process includes: When the positions corresponding to the light emitting units detected by the detection process approach within a threshold distance, it is determined that the plurality of operating objects have approached each other; blinking a light-emitting unit of the operating object at different timings every time the predetermined time is exceeded by a time required for image input, the time being calculated by dividing the predetermined time by the number of the operating objects approaching within the threshold distance; controlling a ratio of a lighting time of the light-emitting unit of each of the operating objects within the predetermined time period according to the content of the task being executed by the operating object; Information processing methods.

11. Computer, a light emission control unit that turns on a light emitting unit included in an operating object that has established communication with the information processing device, and when a different operating object approaches, blinks the light emitting unit included in the operating object at different timings; a detection unit that detects a position corresponding to the light-emitting unit included in each of the operating objects in association with the operating object; a display control unit that displays information associated with the operating object at a position corresponding to the light-emitting unit detected by the detection unit when the operating object is in an information input state; and make it work. The light emission control unit is When the positions corresponding to the light-emitting units detected by the detection unit approach within a threshold distance, it is determined that the plurality of operating objects have approached each other; blinking a light-emitting unit of the operating object at different timings every time the predetermined time is exceeded by a time required for image input, the time being calculated by dividing the predetermined time by the number of the operating objects approaching within the threshold distance; controlling a ratio of a lighting time of the light-emitting unit of each of the operating objects within the predetermined time period according to the content of the task being executed by the operating object; program.

Citation Information

Patent Citations

  • Image processor, method, and program

    JP2010152717A

  • Indication input system and optical indication input device

    JP2011123580A

  • Display device, display system, and control method for display device

    JP2016184268A

  • Multiplex information display system and lighting device for use therein

    JP2017157916A

  • Display control device, display control method, display control program, and display control system

    JP2018156466A