Information processing system and information processing method

Abstract

This enables the detection of the orientation of a moving object, such as a controller, within any given space. [Solution] The information processing system 1 comprises a station 10, a mobile body 30 that can move relative to the station, and one or more processors 34. The station has a first light source 12 and a two-dimensionally driven MEMS mirror 15, and while the MEMS mirror is driven, it reflects a light ray from the first light source with the MEMS mirror and radiates it into a predetermined space around the station, thereby scanning the space two-dimensionally with the light ray. The mobile body has a first optical sensor 31 that detects the reception of the light ray. The processor has a direction determination unit 341 that determines the value of a direction parameter representing the relative direction of the mobile body to the station based on the timing at which the first optical sensor detects the reception of the light ray during scanning of the space at the station.

Description

Technical Field

[0001] The present disclosure relates to an information processing system and an information processing method.

Background Art

[0002] Conventionally, it has been known to provide an acceleration sensor and an angular velocity sensor in a controller held by a user (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the controller described in Patent Document 1, the posture and operation of the controller can be detected by the acceleration sensor and the angular velocity sensor. However, it has been difficult to accurately detect in which direction the controller is located in an arbitrary space.

[0005] In view of the above problems, an object of the present disclosure is to enable detection of the direction of a moving body such as a controller in an arbitrary space.

Means for Solving the Problems

[0006] The gist of the present disclosure is as follows.

[0007] (1) An information processing system including a station, a moving body movable with respect to the station, and one or more processors, The station comprises a first light source and a two-dimensionally driven MEMS mirror, and while the MEMS mirror is being driven, it reflects a ray from the first light source and radiates it into a predetermined space around the station, thereby scanning the space two-dimensionally with the ray. The moving body has a first light sensor that detects the reception of the light ray, The processor is an information processing system having a direction determination unit that determines the value of a direction parameter representing the relative direction of the moving body to the station based on the timing at which the first optical sensor detects the reception of the light ray during scanning of the space in the station. (2) The processor is A position determination unit that determines the position of a virtual object in the virtual space based on the value of the aforementioned direction parameter, An image generation unit that generates image data based on the position of the virtual object, The information processing system according to (1) above, further comprising: an image output unit that outputs the generated image data. (3) The information processing system described in (2) above, wherein the image output unit outputs the image data to a monitor installed in the real world. (4) The information processing system according to (3) above, wherein the station is positioned near the monitor such that the light rays are emitted toward the space in front of the display screen of the monitor. (5) The information processing system according to (3) above, wherein the processor further includes a guide unit that guides the user to position the station near the monitor such that the light rays are emitted toward the space in front of the display screen of the monitor. (6) The station further comprises a second light source and a second light sensor for detecting the reception of reflected light from the light rays emitted from the second light source, The information processing system according to any one of (1) to (5) above, further comprising a distance determination unit that determines a value of a distance parameter representing the distance between the station and an object around the station based on reflected light detected by the second optical sensor. (7) The information processing system according to (6) above, wherein the second light source emits a light ray such that it is incident on the MEMS mirror and reflected by the MEMS mirror. (8) The information processing system described in (7) above, wherein the second light source and the first light source are the same shared light source. (9) The information processing system according to (8) above, wherein the station alternately performs a scanning process for positioning, which scans the rays from the shared light source in two dimensions to determine the value of the direction parameter, and a scanning process for distance measurement, which scans the rays from the shared light source in two dimensions to determine the value of the distance parameter, and the shared light source emits rays in different manner for the scanning for positioning and the scanning for distance measurement. (10) The information processing system according to (8) above, wherein the shared light source emits a ray in which a continuous ray and a pulsed ray are superimposed while scanning the ray from the shared light source in two dimensions. (11) The information processing system described in (7) above, wherein the information processing system is a separate light source that emits different light rays from the first light source and the second light source. (12) The information processing system described in (11) above, wherein the first light source and the second light source emit light rays of different wavelengths. (13) The information processing system according to any one of (6) to (12) above, wherein the processor further includes a positioning unit that identifies a value of a position parameter representing the three-dimensional position of the moving body relative to the station based on the value of the direction parameter and the value of the distance parameter. (14) The station performs a raster scan by performing a linear scan in a first direction with light rays from the first light source multiple times in parallel, while shifting in a second direction perpendicular to the first direction. The information processing system according to any one of (1) to (13) above, wherein the direction determination unit determines the value of a direction parameter relating to the second direction of the moving body, corresponding to the direction of the light ray from the first light source when the light reception time by the first light sensor is longest during a single scan in the first direction. (15) The station performs a raster scan by performing a linear scan in a first direction with light rays from the first light source multiple times in parallel, while shifting in a second direction perpendicular to the first direction. The information processing system according to any one of (1) to (14) above, wherein the direction determination unit determines the value of the direction parameter relating to the first direction of the moving body based on the timing at which the light intensity received by the first light sensor changes to increase. (16) The information processing system according to any one of claims (1) to (15), wherein the processor having the direction-determining unit is provided on the moving body. (17) While the MEMS mirror of the station is being driven, a ray from the first light source of the station is reflected by the MEMS mirror and radiated into a predetermined space around the station, thereby scanning the space two-dimensionally with the ray. The reception of the light ray is detected by a first optical sensor of a mobile body that is movable relative to the station, An information processing method comprising: determining a value of a direction parameter representing the relative direction of the moving object to the station based on the timing at which the first light sensor detects the reception of the light ray during scanning of the space at the station. [Effects of the Invention]

[0008] According to this disclosure, it becomes possible to detect the orientation of a moving object, such as a controller, within any given space. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing the overall configuration of the information processing system. [Figure 2] Figure 2 is a schematic cross-sectional side view showing part of the station's configuration. [Figure 3] Figure 3 is a block diagram showing the configuration of the station's electronic components. [Figure 4] Figure 4 is a block diagram showing the configuration of the mobile unit. [Figure 5] Figure 5 is a block diagram showing the configuration of the main body device. [Figure 6] Figure 6 is a diagram showing how light rays are emitted from the station. [Figure 7] Figure 7 is a time chart of the horizontal angle of the MEMS mirror, etc. when scanning the space around the station. [Figure 8] Figure 8 is an operation sequence diagram showing the flow of the process for specifying the value of the direction parameter representing the direction of the moving body with respect to the station. [Figure 9] Figure 9 is a flowchart showing the flow of information processing when a program is executed in the processor of the main body device. [Figure 10] Figure 10 is a cross-sectional side view similar to FIG. 2, schematically showing a part of the configuration of the station according to the second embodiment. [Figure 11] Figure 11 is a block diagram similar to FIG. 3, showing the configuration related to the electronic components of the station according to the second embodiment. [Figure 12] Figure 12 is a block diagram similar to FIG. 5, showing the configuration related to the electronic components of the main body device according to the second embodiment. [Figure 13] Figure 13 is a time chart of the emission intensity of the light emitting element of the station and the received light intensity detected by the distance measuring optical sensor 19. [Figure 14] Figure 14 is a diagram schematically showing how the space around the station is scanned by the light rays emitted from the station. [Figure 15] Figure 15 is an operation sequence diagram showing the flow of the process for specifying the value of the position parameter representing the three-dimensional position of the moving body with respect to the station. [Figure 16] Figure 16 is a diagram similar to FIG. 14, schematically showing how the space around the station is scanned by the light rays emitted from the station. [Figure 17] Figure 17 is a cross-sectional side view similar to FIG. 10, schematically showing a part of the configuration of the station according to the second modification of the second embodiment. [Figure 18] Figure 18 is a schematic diagram similar to Figure 12, illustrating how a predetermined space around a station is scanned by light rays emitted from the station. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numeral.

[0011] First Embodiment <Configuration of the information processing system> The configuration of the information processing system 1 according to the first embodiment will be described with reference to Figures 1 to 5. Figure 1 is a schematic diagram showing the overall configuration of the information processing system 1.

[0012] As shown in Figure 1, the information processing system 1 comprises a station 10, a mobile unit 30, and a main unit 40. The station 10 and the main unit 40 are interconnected by a cable, thus enabling wired communication between them. The mobile unit 30 and the main unit 40 are also interconnected wirelessly. For communication between the mobile unit 30 and the main unit 40, a communication standard compliant with any standard established by IEEE, ISO, IEC, etc. (e.g., Bluetooth®, Wi-Fi®) is used. The station 10 and the main unit 40 may also be configured to communicate wirelessly. Alternatively, the mobile unit 30 and the main unit 40 may also be configured to communicate via a wired connection.

[0013] Furthermore, the information processing system 1 is connected to a monitor 5 having a display screen 6 for displaying images. In this embodiment, the main unit 40 is connected to the monitor 5 by a cable and is configured to communicate with each other via a wired connection. The information processing system 1 (in particular, the main unit 40 in this embodiment) outputs image data representing the image to be displayed on the monitor 5 to the monitor 5. The main unit 40 may be configured to communicate with the monitor 5 wirelessly. Also, the main unit 40 may be provided with a display screen 6. In other words, the main unit 40 and the monitor 5 may be an integrated unit.

[0014] Monitor 5 is installed in real space. Monitor 5 is installed in a way that prevents movement, for example, on the floor, a table, or a shelf, or it is installed in a way that it is fixed to a wall. Monitor 5 may also be a portable monitor installed in real space.

[0015] In the information processing system 1, the value of a direction parameter representing the relative direction of the moving object 30 with respect to the station 10 is identified, and image data is generated to be output to the monitor 5 based on the identified value of the direction parameter.

[0016] In this embodiment, the station 10 is a separate device from the main unit 40, but it may be an integrated device with the main unit 40. In this case, the information processing system 1 will determine the value of the direction parameter that represents the relative direction of the mobile body 30 with respect to the main unit 40.

[0017] ≪Station≫ The configuration of station 10 will be described with reference to Figures 1 to 3. Station 10 emits a continuous beam of light into the space around it, scanning a predetermined space around station 10 with this continuous beam of light. Station 10 is also positioned near monitor 5. In particular, in the example shown in Figure 1, station 10 is positioned above monitor 5. However, station 10 may be positioned in locations other than above monitor 5, such as in front of and below the display screen 6 of monitor 5, or to the side of monitor 5. In any case, station 10 may be positioned in any location as long as it is positioned so that a beam of light is emitted from station 10 toward the space in front of the display screen 6 of monitor 5.

[0018] Figure 2 is a schematic cross-sectional side view showing part of the configuration of station 10. As shown in Figure 2, station 10 includes a housing 11, a light-emitting element 12, a focusing lens 13, an optical mirror 14, a MEMS mirror 15, and a concave lens 16.

[0019] The housing 11 houses the light-emitting element 12, the focusing lens 13, the optical mirror 14, the MEMS mirror 15, and the concave lens 16. In addition to these components, the housing 11 may also house a communication interface (communication I / F) 21, a storage device 22, and a processor 23, which will be described later.

[0020] The light-emitting element 12 functions as a first light source that emits positioning rays used to determine the direction of the moving object 30 relative to the station 10. In this embodiment, the light-emitting element 12 is a laser diode that emits infrared rays of a certain wavelength with a dot-like projection shape. The light-emitting element 12 is fixed to the housing 11 so that the emitted rays are incident on the reflective surface of the optical mirror 14. Note that a light source other than a light-emitting element may be used as long as it can emit highly linear (highly directional) rays.

[0021] The focusing lens 13 is a lens that focuses light rays emitted from a light source to increase the linearity of the light rays. The focusing lens 13 is placed on the light-emitting surface of the light-emitting element 12. After passing through the focusing lens 13, the light rays emitted from the light-emitting element 12 travel in a straight line. The focusing lens 13 may be a wafer-level lens that has been processed at the wafer level.

[0022] The optical mirror 14 is a mirror that reflects light rays. The optical mirror 14 is fixed to the housing 11 so as to be located in the optical path of the light rays emitted from the light-emitting element 12 through the focusing lens 13. The optical mirror 14 reflects the light rays from the light-emitting element 12 so as to enter the reflective surface of the MEMS mirror 15.

[0023] In this embodiment, the light rays from the light-emitting element 12 are reflected by the optical mirror 14 before entering the MEMS mirror 15. However, the light-emitting element 12 may be positioned so that the light rays from the light-emitting element 12 enter the MEMS mirror 15 directly.

[0024] The MEMS mirror 15 is a small mirror that utilizes MEMS (Micro Electro Mechanical Systems) technology. The MEMS mirror 15 is mounted on the housing 11 such that its reflective surface is located in the optical path of the light reflected by the optical mirror 14.

[0025] The MEMS mirror 15 is configured to rotate around multiple axes. In particular, in this embodiment, it is configured to rotate around a first axis and a second axis perpendicular to the first axis. That is, in this embodiment, the MEMS mirror 15 is driven to rotate around two axes and is therefore configured to be driven in two dimensions.

[0026] When the MEMS mirror 15 rotates and the orientation of its reflective surface changes, the reflection direction of the light rays incident on the MEMS mirror 15 changes. Therefore, when the MEMS mirror 15 rotates around the first axis X1, the direction of the light rays reflected by the MEMS mirror 15 changes in the direction perpendicular to the first axis X1 (first direction). Similarly, when the MEMS mirror 15 rotates around the second axis X2, the direction of the light rays reflected by the MEMS mirror 15 changes in the direction perpendicular to the second axis X2 (second direction). Therefore, in this embodiment, the direction of the light rays emitted from the station 10 can be changed in two dimensions by the MEMS mirror 15. Note that the MEMS mirror 15 may be configured to rotate around two axes that are not perpendicular and parallel to each other.

[0027] In particular, in this embodiment, the first axis X1 extends perpendicular to the second axis X1 and parallel to the surface on which the MEMS mirror 15 is mounted within the housing 11. The second axis X2 extends parallel to the mounting surface on which the station 10 is mounted (the top surface of the monitor 5 in the example shown in Figure 1) and parallel to the surface on which the MEMS mirror 15 is mounted within the housing 11. Therefore, when the MEMS mirror 15 rotates around the first axis X1, the horizontal orientation of the MEMS mirror 15 (first direction orientation) changes, and the horizontal orientation of the light rays reflected by the MEMS mirror 15 changes. Also, when the MEMS mirror 15 rotates around the second axis X2 between the two positions shown by the dashed lines in the figure, the vertical orientation of the MEMS mirror 15 (second direction orientation) changes, and the vertical orientation of the light rays reflected by the MEMS mirror 15 changes between the two directions shown by the dashed lines in the figure. In Figure 2, the solid line indicates the position of the MEMS mirror 15 and the direction of the reflected light rays (direction of the center in the second direction) when the position of the MEMS mirror 15 around the second axis X2 is at the midpoint between the two positions shown by the dashed line.

[0028] The concave lens 16 is a lens that spreads light. The concave lens 16 is fixed to the housing 11 such that its central axis is located on the point where the light rays emitted from the light-emitting element 12 enter the MEMS mirror 15. Therefore, when the reflected light from the MEMS mirror 15 enters the concave lens 16, the angle of this reflected light with respect to the central axis becomes large. As a result, even if the rotation angle of the MEMS mirror 15 is small, the radiation angle of the light rays from the station 10 can be made larger compared to this rotation angle.

[0029] In the station 10 configured in this way, when a light ray is emitted from the light-emitting element 12, the emitted light ray passes through the focusing lens 13, increasing its directivity. The light ray that has passed through the focusing lens 13 is reflected by the optical mirror 14 and incident near the center of the reflective surface of the MEMS mirror 15. The light ray that has incident on the reflective surface of the MEMS mirror 15 is reflected by the reflective surface of the MEMS mirror 15. At this time, when the MEMS mirror 15 is rotated, the direction of the reflected light from the MEMS mirror 15 changes. As described above, since the MEMS mirror 15 is driven in two dimensions, the direction of the reflected light from the MEMS mirror 15 also changes in two dimensions. Then, the reflected light from the MEMS mirror 15 is spread out by the concave lens 16, and a light ray with a dot-like projection shape is emitted from the station 10.

[0030] Figure 3 is a block diagram showing the configuration of the electronic components of station 10. As shown in Figure 3, in addition to the light-emitting element 12 and MEMS mirror 15 described above, station 10 has a communication interface 21, a storage device 22, and a processor 23. The light-emitting element 12, MEMS mirror 15, communication interface 21, and storage device 22 are communicated with the processor 23 via signal lines.

[0031] The communication interface 21 of station 10 is an interface for communicating with external devices of station 10. In this embodiment, so that station 10 can communicate with the main unit 40 via a wired connection, the communication interface 21 has a connector for connecting the wiring connector of a cable (not shown) connected to the main unit 40. If station 10 communicates with the main unit 40 wirelessly or if station 10 communicates with the mobile unit 30 wirelessly, the communication interface 21 may have a wireless communication module.

[0032] The storage device 22 of station 10 includes, for example, volatile semiconductor memory (e.g., RAM), non-volatile semiconductor memory (e.g., ROM), etc. The storage device 22 of station 10 stores computer programs for executing various processes in the processor 23 of station 10, and various data used when the processor 23 executes various processes. The storage device 22 of station 10 may be built into the processor 23 of station 10.

[0033] The processor 23 of station 10 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 of station 10 may further have arithmetic circuits such as a logic unit or a numerical unit. The processor 23 of station 10 executes various processes based on computer programs stored in the storage device 22 of station 10. In particular, in this embodiment, the processor 23 of station 10 has a control unit 231 that controls the light-emitting element 12 and the MEMS mirror 15.

[0034] In this embodiment, the station 10 is provided with a storage device 22 and a processor 23, but the station 10 does not necessarily need to be provided with a storage device 22 and a processor 23. In this case, for example, the processor 43 of the main unit 40 may have a control unit 231, and therefore the light-emitting element 12 and the MEMS mirror 15 may be controlled by the processor 43 of the main unit 40.

[0035] ≪Mobile≫ Next, the configuration of the mobile unit 30 will be described with reference to Figures 1 and 4. The mobile unit 30 is a device that can move relative to the station 10. In particular, in this embodiment, the mobile unit 30 is a device that moves relative to the station 10 as the user's hand position moves. The mobile unit 30 can be any device that is grasped by the user or attached to a device grasped by the user and moves as the user's hand position moves. Therefore, the mobile unit 30 may be a controller to which user operations are input. Alternatively, the mobile unit 30 may be a peripheral device attached to such a controller by a mounting device or the like. The mobile unit 30 may also be attached to any part of the user's body. Furthermore, the mobile unit 30 may move autonomously.

[0036] Figure 4 is a block diagram showing the configuration of the mobile unit 30. As shown in Figure 4, the mobile unit 30 includes a positioning optical sensor 31, a communication interface (communication I / F) 32, a storage device 33, and a processor 34. The positioning optical sensor 31, the communication interface 32, and the storage device 33 are communicated with the processor 34 via signal lines. The mobile unit 30 may be equipped with a battery if power is not supplied from an external source.

[0037] The positioning optical sensor 31 functions as a first optical sensor that detects the reception of light rays from the first light source of the station 10. The positioning optical sensor 31 is, for example, a photodiode. The positioning optical sensor 31 detects the reception of light rays emitted from the light-emitting element 12 of the station 10, that is, infrared rays of a specific wavelength in this embodiment. The positioning optical sensor 31 may be an optical sensor that responds only to infrared rays of such a specific wavelength, or it may be a sensor that responds to infrared rays of a somewhat wide range of wavelengths, including infrared rays of such a specific wavelength. When the positioning optical sensor 31 detects the reception of light rays emitted from the light-emitting element 12, it outputs a reception signal corresponding to the reception intensity to the processor 34.

[0038] The communication interface 32 of the mobile unit 30 is an interface for communicating with external devices of the mobile unit 30. In this embodiment, the communication interface 32 has a wireless communication module for performing communication compliant with the above-described communication standard, so that the mobile unit 30 can communicate wirelessly with the main unit 40, or with the main unit 40 and the station 10. If the mobile unit 30 communicates with the main unit 40 via a wired connection, the communication interface 32 may have, for example, a connector for connecting to a wiring connector of the wiring connected to the main unit 40.

[0039] The storage device 33 of the mobile device 30 includes, for example, a volatile semiconductor memory (e.g., RAM) or a non-volatile semiconductor memory (e.g., ROM). The storage device 33 of the mobile device 30 stores computer programs for executing various processes in the processor 34 of the mobile device 30, and various data used when various processes are executed by the processor 34 of the mobile device 30. The storage device 33 of the mobile device 30 may be built into the processor 34 of the mobile device 30.

[0040] The processor 34 of the mobile unit 30 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 34 of the mobile unit 30 may further have arithmetic circuits such as a logical operation unit or a numerical operation unit. The processor 34 of the mobile unit 30 executes various processes based on a computer program stored in the storage device 33 of the mobile unit 30. In particular, in this embodiment, the processor 34 of the mobile unit 30 has a direction-determining unit 341 that determines the direction in which the mobile unit 30 is located relative to the station 10.

[0041] Furthermore, if the mobile body 30 is a peripheral device attached to the controller, the mobile body 30 may be attached, for example, so as to be fixed to the controller in a predetermined position and orientation. In this case, the controller may have a mounting portion for attaching to the mounting portion of the main unit 40. In addition, the mobile body 30 may have a mounting portion to which the above-mentioned mounting portion of the controller is attached when the controller is detached from the main unit 40. The structure of the mounting portion and the mounting portion is not limited and may be, for example, a rail structure, a fitting structure, a hook structure, or a suction structure.

[0042] Furthermore, if the mobile unit 30 is a peripheral device attached to the controller, the mobile unit 30 may be configured to communicate with the controller via a wired connection once attached, and the controller may be configured to communicate wirelessly with the main unit 40, etc. In addition, in this case, the controller may be equipped with a battery, and the mobile unit 30 may be powered by the controller's battery. Furthermore, in this case, the controller may have a storage device and a processor, and the controller's processor may have at least a part of the direction-determining unit 341. Note that in this case, the mobile unit 30 does not have to have a storage device 33 and / or a processor 34.

[0043] ≪Main Unit≫ Next, the configuration of the main unit 40 will be described with reference to Figures 1 and 5. The main unit 40 executes various programs such as OS programs and application programs (for example, game programs), and outputs image data of images to be displayed on the display screen 6 of the monitor 5 as a result of executing these programs.

[0044] Figure 5 is a block diagram showing the configuration of the main unit 40. As shown in Figure 5, the main unit 40 has a communication interface (communication I / F) 41, a storage device 42, and a processor 43. The communication interface 41 and the storage device 42 are connected to the processor 43 via signal lines so as to be communicative.

[0045] The communication interface 41 of the main unit 40 is an interface for communicating with external devices of the main unit 40. In this embodiment, the communication interface 41 has a wireless communication module for performing communication compliant with the above-described communication standard, so that the main unit 40 can communicate wirelessly with the mobile unit 30. Furthermore, the communication interface 32 may have a connector for connecting to a wiring connector of the wiring connected to the station 10, so that the main unit 40 can communicate with the station 10 via a wired connection. The wireless communication module may be used to communicate wirelessly with the station 10, and the connector may be used to connect to a wiring connector of the wiring connected to the mobile unit 30.

[0046] The storage device 42 of the main unit 40 includes, for example, volatile semiconductor memory (e.g., RAM), non-volatile semiconductor memory (e.g., ROM), etc. Furthermore, the storage device 42 of the main unit 40 may also include a hard disk drive (HDD), a solid-state drive (SSD), or an optical recording medium. In addition, a part of the storage device 42 of the main unit 40 may be removable. The storage device 42 of the main unit 40 stores computer programs for executing various processes in the processor 43 of the main unit 40, and various data used when various processes are executed by the processor 43. Computer programs include OS programs and application programs (e.g., game programs), etc.

[0047] The processor 43 of the main unit 40 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 43 of the main unit 40 may further have arithmetic circuits such as a logical operation unit or a numerical operation unit. The processor 43 of the main unit 40 executes various processes based on computer programs stored in the storage device 42 of the main unit 40. In particular, in this embodiment, the processor 43 of the main unit 40 has a position determination unit 431 that determines the position of virtual objects in the virtual space, an image generation unit 432 that generates image data based on the position of the virtual objects, an image output unit 433 that outputs the generated image data, and a guide unit 434 that guides the placement position of the monitor 5.

[0048] <Direction Identification> Next, with reference to Figures 6 and 7, a method for determining the relative direction of the mobile body 30 to the station 10 will be described. Figure 6 shows how a light ray 51 is emitted from the station 10. In particular, Figure 6 shows the trajectory of the light ray 51 on a virtual plane 52 in a predetermined space around the station 10. In the example shown in Figure 6, the virtual plane 52 is a plane perpendicular to the optical path of the light ray 51 emitted from the station 10 and represents the plane on which the positioning optical sensor 31 of the mobile body 30 is located. The trajectory of the light ray 51 shown as a solid line in the figure 53 represents the trajectory of the light ray 51 when the light ray 51 is continuously emitted from the station 10. On the other hand, the trajectory shown as a dashed line in the figure 54 shows the trajectory that the light ray 51 would have taken if it were emitted from the light-emitting element 12, even though no light ray is being emitted from the light-emitting element 12, considering the operation of the MEMS mirror 15.

[0049] As can be seen from Figure 6, in this embodiment, scanning by the light ray 51 in the first direction (horizontal direction in this embodiment) is performed repeatedly. In other words, in this embodiment, a raster scan is performed in which linear scanning in the first direction by the light ray 51 is performed multiple times in parallel while shifting in the second direction. In particular, in this embodiment, in the first direction, scanning in the forward direction and scanning in the reverse direction in the first direction (horizontal direction) are performed repeatedly while shifting the position in the vertical direction. As a result, in a single scanning process, a predetermined space around the station 10 is scanned by the light ray 51 emitted from the station 10.

[0050] Figure 7 is a time chart of the horizontal and vertical angles, radiant intensity, and received light intensity of the MEMS mirror 15 as it scans the space around station 10. The horizontal angle of the MEMS mirror 15 (angle in the first direction) in the figure represents the angle with respect to the orientation of the center of the MEMS mirror 15 in the horizontal direction. Similarly, the vertical angle of the MEMS mirror 15 (angle in the second direction) represents the angle with respect to the orientation of the center of the MEMS mirror 15 in the vertical direction (the orientation of the MEMS mirror 15 shown by the solid line in Figure 2). The radiant intensity represents the intensity of the light rays 51 emitted from station 10, i.e., the light emission intensity of the light-emitting element 12. The received light intensity represents the intensity of the light rays detected by the positioning optical sensor 31 of the mobile body 30.

[0051] As shown in Figure 7, in this embodiment, the horizontal angle of the MEMS mirror 15 is changed from its maximum value to its minimum value while the vertical angle of the MEMS mirror 15 remains constant, and light rays continue to be emitted from the light-emitting element 12 during this time (horizontal scanning period R1 in the figure). When the horizontal angle of the MEMS mirror 15 reaches its minimum value, the emission of light rays from the light-emitting element 12 stops, and the vertical angle of the MEMS mirror 15 is changed to a slightly smaller value while the horizontal angle remains constant. Subsequently, the horizontal angle of the MEMS mirror 15 is changed from its minimum value to its maximum value while the vertical angle of the MEMS mirror 15 remains constant, and light rays continue to be emitted from the light-emitting element 12 during this time (horizontal scanning period R2 in the figure). When the horizontal angle of the MEMS mirror 15 reaches its maximum value, the emission of light rays from the light-emitting element 12 stops, and the vertical angle of the MEMS mirror 15 is changed to a slightly smaller value while the horizontal angle remains constant. By repeating this operation, a raster scan of a predetermined space around the station 10 is performed by the light rays 51.

[0052] In the example shown in Figure 7, when the horizontal angle of the MEMS mirror 15 changes, light rays are emitted from the light-emitting element 12, and when the vertical angle of the MEMS mirror 15 changes, the emission of light rays from the light-emitting element 12 is stopped. However, the scanning mode is not limited to this mode, and scanning in various modes is possible. For example, even when the horizontal angle of the MEMS mirror 15 changes, the emission of light rays from the light-emitting element 12 may be stopped near the maximum and minimum values. In this case, the period during which light rays are emitted from the light-emitting element 12 when the horizontal angle of the MEMS mirror 15 changes is longer than the period during which such light rays are emitted. Also, light rays may be emitted from the light-emitting element 12 even when the vertical angle of the MEMS mirror 15 changes. Furthermore, the vertical angle of the MEMS mirror 15 may be changed from its maximum to minimum value, at which point light rays are emitted from the light-emitting element 12. Then, the horizontal angle of the MEMS mirror 15 may be slightly changed, at which point the emission of light rays from the light-emitting element 12 is stopped. This operation may be repeated to perform a raster scan.

[0053] Furthermore, in the example shown in Figure 7, the horizontal and vertical angles of the MEMS mirror 15 change linearly at a constant speed (they change linearly in Figure 7). However, the horizontal and vertical angles of the MEMS mirror 15 may also change along a sine wave, for example. In this case, for example, the horizontal and vertical angles of the MEMS mirror 15 may be changed along a sine wave by vibrating the MEMS mirror 15 at its resonant frequency.

[0054] Furthermore, in this embodiment, a predetermined space around station 10 is raster-scanned by the light ray 51. However, scanning may be performed by other methods as long as the predetermined space around station 10 can be scanned thoroughly. Also, scanning may be performed so that the scanning area in the virtual plane, as shown in Figure 6, is a shape other than a rectangle, such as a circle or an ellipse.

[0055] Furthermore, in this embodiment, while horizontal scanning is being performed by the light ray 51, a continuous light ray is emitted from the light-emitting element 12. However, pulsed light rays may also be emitted from the light-emitting element 12.

[0056] As the space around station 10 is scanned by the light ray 51, the positioning optical sensor 31 of the mobile body 30 located within the scanned space detects the reception of the light ray 51. The timing of the reception of the light ray 51 by the positioning optical sensor 31 represents the direction of the mobile body 30 relative to station 10.

[0057] In the example shown in Figure 7, the positioning optical sensor 31 detects the reception of light rays 51 during horizontal scanning periods R4, R5, and R6. In particular, in the example shown in Figure 7, the positioning optical sensor 31 detects reception from time t1 to reception time Δt1 within horizontal scanning period R4, from time t2 to reception time Δt2 within horizontal scanning period R5, and from time t3 to reception time Δt3 within horizontal scanning period R6. Furthermore, in the illustrated example, the reception time Δt2 is longer than the reception times Δt1 and Δt3.

[0058] In this embodiment, the vertical orientation of the mobile body 30 relative to the station 10 is determined by the vertical angle of the MEMS mirror 15 when the light reception time is longest, i.e., the vertical direction of the light ray 51 emitted from the station 10. In the illustrated example, the vertical orientation of the mobile body 30 is determined based on the vertical angle of the MEMS mirror 15 during the horizontal scanning period R5 when the positioning optical sensor 31 detects light reception for the longest light reception time Δt2.

[0059] Furthermore, in this embodiment, the horizontal orientation of the moving body 30 relative to the station 10 is determined based on the timing at which the detection of the longest light reception time begins, that is, the timing at which the light reception intensity changes to increase with the detection of the longest light reception time. More specifically, the horizontal orientation of the moving body 30 relative to the station 10 is determined in accordance with the horizontal angle of the MEMS mirror 15 at such timing, that is, the horizontal orientation of the light rays 51 emitted from the station 10. Therefore, in the illustrated example, the horizontal orientation of the moving body 30 relative to the station 10 is determined in accordance with the horizontal orientation of the light rays 51 emitted from the station 10 at time t2, when the detection of the longest light reception time Δt2 by the positioning optical sensor 31 begins.

[0060] According to this embodiment, the station 10 scans a predetermined space around it in two dimensions using light rays, and the relative direction of the mobile body 30 to the station 10 is determined based on the timing at which the positioning optical sensor 31 of the mobile body 30 detects light reception with this configuration. In particular, in this embodiment, determination is mainly performed by the light-emitting element 12, the MEMS mirror 15, and the positioning optical sensor 31, so that the relative direction of a mobile body in any space (in this embodiment, the direction of the mobile body 30 to the station 10) can be detected with a simple configuration.

[0061] Furthermore, in this embodiment, a light ray having a dot-shaped projection is emitted from the station 10. As a result, in this embodiment, since the shape of the light ray that is ultimately projected is dot-shaped, the energy required for emission of the light ray is smaller compared to, for example, when the shape of the light ray that is ultimately projected is linear, and therefore the relative direction of the moving body 30 can be determined using a relatively low-output light-emitting element 12. In addition, in this embodiment, scanning is performed using the MEMS mirror 15, so a wide area of ​​space can be scanned in a short time even when using a light ray with a dot-shaped projection. Also, since the shape of the light ray that is ultimately projected is dot-shaped, if the shape of the light ray emitted from the light-emitting element 12 is dot-shaped, the light ray emitted from the light-emitting element 12 can be projected without changing its shape. Note that "dot-shaped" does not necessarily mean a perfect circle, but may be an ellipse or a polygon. Furthermore, a light ray with a shape other than dot-shaped may be emitted from the light-emitting element 12, in which case the shape of the light ray until it is finally projected is converted to a dot shape.

[0062] Figure 8 is an operation sequence diagram showing the process flow for determining the value of the direction parameter, which represents the direction of the mobile body 30 relative to station 10. The illustrated process is performed at regular time intervals (for example, every 17 ms).

[0063] First, the processor 43 of the main unit 40 transmits a scan start signal to the station 10 to initiate scanning of space using the light ray 51 (step S11). At the same time, the processor 43 transmits a synchronization signal to the mobile unit 30 indicating the timing when scanning has started (step S12). When the processor 34 of the mobile unit 30 receives the synchronization signal, the scanning start time by the station 10 is synchronized between the processor 34 of the mobile unit 30 and the processor 43 of the main unit 40.

[0064] In step S11, when the processor 23 of station 10 receives a scan start signal, the control unit 231 of the processor 23 controls the light-emitting element 12 and the MEMS mirror 15 to scan a predetermined space around station 10 using a light ray 51 emitted from station 10 (step S13). Specifically, the light-emitting element 12 and the MEMS mirror 15 are controlled as shown in Figure 7. As a result, one scanning process is performed, which includes a series of scans using the light ray 51 emitted from station 10.

[0065] While a single scanning process is being performed at station 10, the positioning optical sensor 31 of the mobile body 30 detects the intensity of infrared light received (step S14). At this time, as shown in Figures 6 and 7, the intensity of infrared light received by the positioning optical sensor 31 of the mobile body 30 increases at a timing corresponding to the position of the mobile body 30 in the predetermined space where the scanning is performed.

[0066] Once a scan by the light ray 51 at station 10 is completed, the direction determination unit 341 of the mobile body 30's processor 34 identifies the light reception with the longest reception time from the light reception waveform of the positioning light sensor 31 during the scan (step S15). For example, in the example shown in Figure 7, the direction determination unit 341 identifies the light reception starting at time t2 as the light reception with the longest reception time.

[0067] Subsequently, the processor 34 of the mobile unit 30 determines the value of a direction parameter representing the direction of the mobile unit 30 relative to the station 10, based on the timing at which the longest light reception time began to be detected during a single scanning process by the synchronized station 10 (step S16). The direction parameter is, for example, the horizontal angle and vertical angle of the position of the mobile unit 30 relative to the station 10. Alternatively, since the timing at which the longest light reception time is detected during a single scanning process effectively represents the direction of the mobile unit 30 relative to the station 10, the direction parameter may also be the timing at which the longest light reception time is detected during a single scanning process. Once the value of the direction parameter is determined, the processor 34 of the mobile unit 30 transmits the determined value of the direction parameter to the main unit 40 (step S17).

[0068] <Processing in the main unit> Next, referring to Figure 9, the processing in the main unit 40 using the direction parameter value will be described. The processor 43 of the main unit 40 executes an OS program or an application program. In particular, in the processor 43 of the main unit 40 of this embodiment, a two-dimensional virtual space is formed in the OS program or application program, and the position of virtual objects in this virtual space is controlled by input from the user. Such user input includes the relative direction of the mobile body 30 held by the user with respect to the station 10.

[0069] Figure 9 is a flowchart showing the information processing flow when an OS program or application program is executed in the processor 43 of the main unit 40. The processes shown in Figure 9 are executed at regular time intervals.

[0070] As shown in Figure 9, first, the position determination unit 431 of the processor 43 of the main unit 40 determines the two-dimensional position of the virtual object in the virtual space (step S51) based on the value of the direction parameter received from the mobile body 30 in step S17 of Figure 8. In particular, in this embodiment, the position determination unit 431 determines the position of the virtual object so as to correspond to the relative direction of the mobile body 30 with respect to the station 10, which is represented by the value of the direction parameter. Therefore, for example, if the mobile body 30 moves upward with respect to the station 10, the position of the virtual object in the virtual space also moves upward.

[0071] Next, the image generation unit 432 of the processor 43 generates image data for the image to be displayed on the display screen 6 of the monitor 5, based on the position of the virtual object identified in step S51 (step S52). Therefore, for example, if the position of the virtual object in the virtual space moves upward, the image data is generated so that the position of the virtual object in the image moves upward.

[0072] Next, the image output unit 433 of the processor 43 outputs the image data generated in step S52 to the monitor 5 (step S53). As a result, the display screen 6 of the monitor 5 displays an image represented by the image data generated in step S52. Therefore, according to this embodiment, the position of the virtual object displayed on the display screen 6 of the monitor 5 changes based on the orientation of the mobile body 30 relative to the station 10, thereby enabling the user to intuitively manipulate the virtual object.

[0073] In this embodiment, the image output unit 433 outputs image data to the monitor 5 connected to the main unit 40. However, the image output unit 433 may output image data to other devices, as long as they have a display screen capable of displaying video. For example, if the main unit 40 has a display screen, the image data may be output to the screen control unit that controls the display screen of the main unit 40.

[0074] Furthermore, in this embodiment, the position determination unit 431 determines the two-dimensional position of the virtual object based on the value of the direction parameter in step S51. However, based on the value of the direction parameter, the orientation (orientation) of the virtual object may be determined in addition to the position of the virtual object, or instead of the two-dimensional position of the virtual object.

[0075] As mentioned above, the station 10 needs to be positioned near the monitor 5 so that light rays are emitted toward the space in front of the display screen 6 of the monitor 5. Therefore, in this embodiment, the guide unit 434 of the processor 43 of the main unit 40 guides the user to position the station 10 near the monitor 5 so that light rays from the station 10 are emitted toward the space in front of the display screen 6 of the monitor 5.

[0076] Specifically, the guidance unit 434 displays a guidance screen on the display screen 6 of the monitor 5 instructing the user to set up the station 10 in a predetermined location near the monitor 5. The guidance unit 434 may also output voice guidance from the monitor 5 or the speaker of the main unit 40 to encourage the user to set up the station 10. This guidance from the guidance unit 434 is provided, for example, when the main unit 40 is started up, or when the processor 43 of the main unit 40 starts executing the OS program or application program.

[0077] According to this embodiment, the output is displayed on a monitor 5 installed in the real space outside the information processing system 1. Therefore, since the user operates within the range in which the monitor 5 can be seen, it is expected that the movement of the user and the moving object held by the user will be smaller compared to cases where the display screen 6 moves with the user, such as with a head-mounted display. Therefore, by limiting the scanning area of ​​the station 10, the time required for one scanning process can be shortened, and the scanning frequency can be increased.

[0078] Furthermore, in this embodiment, the user is likely to be in the space in front of the display screen 6 so that they can see the display screen 6 of the monitor 5. And, in this embodiment, since the station 10 is placed near the monitor 5 and scans the space in front of the monitor 5, the detectability of the moving object 30 can be increased.

[0079] Second Embodiment Next, the information processing system 1 according to the second embodiment will be described with reference to Figures 10 to 15. The configuration and operation of the information processing system 1 according to the second embodiment are basically the same as those of the information processing system 1 according to the first embodiment. The following describes the parts that differ from the information processing system 1 according to the first embodiment.

[0080] The information processing system 1 according to the second embodiment determines the three-dimensional position of the mobile body 30 relative to the station 10. In particular, the information processing system 1 according to the second embodiment determines the three-dimensional position of the mobile body 30 relative to the station 10 by determining the distance between the station 10 and the mobile body 30, in addition to the direction of the mobile body 30 relative to the station 10.

[0081] Figure 10 is a schematic cross-sectional side view similar to Figure 2, showing a part of the configuration of the station 10 according to the second embodiment. As shown in Figure 10, in addition to the components shown in Figure 2, the station 10 includes a convex lens 17, an IR filter 18, and a distance measuring optical sensor 19.

[0082] In this embodiment, the light-emitting element 12 also functions as a second light source that emits a rangefinder ray used to determine the distance between the station 10 and the mobile body 30. Therefore, in this embodiment, the light-emitting element 12 is a common light source that functions as both a first light source that emits a positioning ray and a second light source that emits a rangefinder ray. The rangefinder ray, like the positioning ray, is emitted from the light-emitting element 12 so that it is incident on the MEMS mirror 15 via the optical mirror 14 and reflected by the MEMS mirror 15.

[0083] The convex lens 17 is a lens that collects incident light rays. In this embodiment, the convex lens 17 collects the incident light rays into the light-receiving part of the distance-measuring light sensor 19. This increases the intensity of the incident light into the light-receiving part of the distance-measuring light sensor 19.

[0084] The IR filter 18 is a filter that transmits only infrared light, particularly infrared light of a specific wavelength emitted from the light-emitting element 12. This IR filter 18 ensures that only the reflected light emitted from the light-emitting element 12 and reflected at any given location reaches the distance-measuring optical sensor 19.

[0085] The distance measuring light sensor 19 functions as a second light sensor that detects the reception of light rays from the second light source of the station 10. The distance measuring light sensor 19 is a photodiode, similar to the positioning light sensor 31 of the mobile body 30, for example. When the distance measuring light sensor 19 detects the reception of light rays emitted from the light-emitting element 12, it transmits a light reception signal corresponding to the light reception intensity to the processor 23.

[0086] Figure 11 is a block diagram similar to Figure 3, showing the configuration of the electronic components of station 10 according to the second embodiment. As shown in Figure 11, the processor 23 of station 10 has a control unit 231, as well as a distance determination unit 232 that determines the value of a distance parameter representing the distance between station 10 and the object.

[0087] Figure 12 is a block diagram similar to Figure 5, showing the configuration of the electronic components of the main unit 40 according to the second embodiment. As shown in Figure 13, the processor 43 of the main unit 40 further includes a positioning unit 435 that determines the three-dimensional position of the mobile body 30 relative to the station 10.

[0088] <Identifying a three-dimensional position> Next, with reference to Figures 10 and 13-15, a method for determining the three-dimensional position of the moving object 30 relative to the station 10 will be described. As shown in Figure 10, when a pulsed light ray 56 is emitted from the station 10 toward an object around the station 10, such as the moving object 30, the light ray incident on the object is diffusely reflected, and a portion of it is incident on the distance measuring light sensor 19 via the convex lens 17.

[0089] Figure 13 is a time chart showing the radiation intensity from the light-emitting element 12 of station 10 and the received light intensity detected by the distance-measuring optical sensor 19. As shown in Figure 13, when the light-emitting element 12 emits a pulsed light ray, the reflected light of this pulsed light ray is detected by the distance-measuring optical sensor 19 with a time delay t. This time t increases as the distance from station 10 to the object located in the direction from which the pulsed light ray was emitted increases. Therefore, by measuring the time t from when station 10 emits a pulsed light ray in any direction until the reflected light is received, the distance between station 10 and the object located in that direction can be determined.

[0090] Furthermore, by emitting such pulsed light rays while changing the orientation of the MEMS mirror 15, the distance to objects located in various directions relative to the station 10 can be determined. For this reason, in this embodiment, the pulsed light rays emitted from the station 10 scan a predetermined space around the station 10 (the same space as the space scanned by the continuous light rays 51). This makes it possible to determine the value of a distance parameter that represents the distance between the station 10 and objects around the station 10 in the predetermined space around the station 10.

[0091] The distance parameter values ​​generated in this way represent the distance between station 10 and objects located in various directions from station 10. Of these, the distance in the direction of the moving body 30 relative to station 10, determined using the positioning optical sensor 31, represents the distance between station 10 and the moving body 30. Therefore, in this embodiment, the three-dimensional position of the moving body 30 relative to station 10 can be determined based on the distance parameter value representing the distance to objects around station 10 and the direction parameter value representing the direction of the moving body 30 relative to station 10. Thus, in this embodiment, the three-dimensional position of the moving body 30 is determined mainly by the light-emitting element 12, MEMS mirror 15, and positioning optical sensor 31, in addition to the distance measuring optical sensor 19, and the three-dimensional position of the moving body 30 can be determined with a simple configuration.

[0092] Figure 14 schematically shows how light rays emitted from station 10 scan the space around station 10. Figures 14(A) and 14(B) show the trajectory of the light source in the virtual plane 52. Figure 14(C) shows the time progression of the radiation intensity from the light-emitting element 12 when scanning a portion of the area in Figure 14(A), and Figure 14(D) shows the time progression of the radiation intensity from the light-emitting element 12 when scanning a portion of the area in Figure 14(B).

[0093] As can be seen from Figures 14(A) and 14(C), in this embodiment, first, a single two-dimensional scanning process for positioning is performed using a continuous beam of light. This scanning process for positioning is performed in the same manner as the scanning process described using Figures 6 and 7. This scanning process for positioning determines the direction of the moving object 30 relative to the station 10.

[0094] After the positioning scan is completed, a two-dimensional distance measurement scan is performed using pulsed light rays, as shown in Figures 14(B) and 14(D). The emission interval of the pulsed light rays is set to be longer than the time required for the emitted pulsed light rays to reflect off an object and return. For example, the emission interval of the pulsed light rays is set to the longest time required for them to reflect off an object located within the expected range of use. In the distance measurement scan, scanning is performed in the same scanning order as in the positioning scan. This distance measurement scan determines the distance to objects around station 10. In this embodiment, the positioning scan and the distance measurement scan are performed alternately. In addition, light rays are emitted from the light-emitting element 12 in different ways for the positioning scan and the distance measurement scan. Specifically, the positioning scan is performed with continuous light rays, while the distance measurement scan is performed with pulsed light rays. Furthermore, in this embodiment, the light emission intensity of the light-emitting element 12 during the scanning process for positioning may be weaker than the light emission intensity of the light-emitting element 12 during the scanning process for distance measurement.

[0095] Figure 15 is an operation sequence diagram showing the process of determining the value of a position parameter representing the three-dimensional position of the mobile body 30 relative to station 10. The illustrated process is performed at regular time intervals (for example, every 33 ms). Steps S11 to S17 are the same as in Figure 8, so their explanation is omitted.

[0096] In step S11, when the processor 23 of station 10 receives a scan start signal, the control unit 231 of the processor 23 controls the light-emitting element 12 and the MEMS mirror 15 to perform a positioning scan of a predetermined space around station 10 using a continuous beam of light 51, as shown in Figures 14(A) and 14(C) (step S13). As a result, one positioning scan process is performed using a continuous beam of light 51.

[0097] Once one positioning scan is completed, the control unit 231 of the processor 23 controls the light-emitting element 12 and the MEMS mirror 15 to perform a distance-measuring scan of a predetermined space around the station 10 using pulsed light rays 56, as shown in Figures 14(B) and 14(D) (step S18). As a result, one distance-measuring scan is performed using pulsed light rays 56. During this distance-measuring scan, the distance-measuring optical sensor 19 of the station 10 detects the intensity of infrared light received (step S18).

[0098] When the light intensity is detected by the distance measuring optical sensor 19, the distance determination unit 232 of the processor 23 of station 10 determines the value of a distance parameter representing the distance to the object in each direction based on the emission timing of the pulsed light rays 56 and the timing when the light intensity increases (step S19). That is, the distance determination unit 232 determines the value of a distance parameter representing the distance between station 10 and the object around station 10 based on the reflected light detected by the distance measuring optical sensor 19. The distance parameter is, for example, a depth map representing the distance between station 10 and the object around station 10 in a predetermined space around station 10. Once the value of the distance parameter is determined, the processor 34 of station 10 transmits the determined value of the distance parameter to the main unit 40 (step S20).

[0099] The positioning unit 435 of the processor 43 of the main unit 40 receives the value of the direction parameter from the mobile body 30 and the value of the distance parameter from the station 10, and based on these parameter values, it identifies the value of the position parameter that represents the three-dimensional position of the mobile body 30 (step S21). Specifically, in the distance parameter (e.g., depth map), the parameter relating to the distance between the mobile body 30 and the station 10 can be identified from the value of the distance parameter in the direction corresponding to or approximating the value of the direction parameter. As a result, the value of the position parameter that represents the three-dimensional position of the mobile body 30 can be identified from the direction parameter and the parameter relating to the distance between the mobile body 30 and the station 10.

[0100] The position parameter values ​​identified in this way are used in the processor 43 of the main unit 40 to generate and output image data, as explained with reference to Figure 9. In particular, in this embodiment, a three-dimensional virtual space is formed in the OS program or application program in the processor 43, and the three-dimensional position within the virtual space is determined based on the position parameter values.

[0101] <Variation> Next, with reference to Figure 16, a first modified example of the information processing system 1 according to the second embodiment will be described. In the second embodiment described above, a scanning process for positioning using a continuous light ray and a scanning process for distance measurement using a pulsed light ray are performed alternately. The position of the moving object 30 is determined each time both the scanning process using a continuous light ray and the scanning process using a pulsed light ray are completed. Therefore, in the second embodiment, two scanning processes must be performed to determine the position of the moving object 30, and the number of times the position of the moving object 30 is determined per unit time is small.

[0102] In contrast, in the first modified example, during a single scanning process, the light-emitting element 12 emits a ray that is a superposition of a continuous ray used to determine the value of the direction parameter and a pulsed ray used to determine the value of the distance parameter. Then, a predetermined space around the station 10 is scanned by this ray, which is a superposition of a continuous ray and a pulsed ray.

[0103] Figure 16 is a schematic diagram similar to Figure 14, illustrating how the space around station 10 is scanned by light rays emitted from station 10. As can be seen from Figure 16, in this embodiment, pulsed light rays are emitted from the light-emitting element 12 during the period when the orientation of the MEMS mirror 15 is changing in the horizontal direction. In addition, during this period, even when pulsed light rays are not being emitted from the light-emitting element 12, light rays are emitted at an intensity weaker than the emission intensity of pulsed light rays.

[0104] When a ray is emitted in which a continuous ray and a pulsed ray are superimposed in this way, the resulting emission is a continuous ray. Therefore, when the moving object 30 is scanned by this ray, the positioning optical sensor 31 can detect the ray emitted from the station 10. As a result, the direction of the moving object 30 relative to the station 10 can be determined.

[0105] Furthermore, because the intensity of the light ray is weak between pulses, even if this light ray is reflected by objects around station 10, the intensity of the reflected light is small. For this reason, the high-intensity light above a predetermined intensity detected by the distance measuring light sensor 19 is considered to be reflected light from the pulsed light ray. Therefore, the distance between station 10 and the mobile object 30 can be determined based on the high-intensity light above a predetermined intensity detected by the distance measuring light sensor 19.

[0106] As mentioned above, the positioning ray may be a pulsed ray instead of a continuous ray. In this case, the continuous ray and the pulsed ray may or may not be superimposed. In either case, the intensity of the pulsed ray for positioning is weaker than the intensity of the pulsed ray for distance measurement. Also, the emission frequency of the pulsed ray for positioning may be higher than the emission frequency of the pulsed ray for distance measurement. Furthermore, the emission time per pulse of the pulsed ray for positioning may be longer than the emission time per pulse of the pulsed ray for distance measurement. Alternatively, the same pulsed ray used for both positioning and distance measurement may be emitted. In these cases, only the pulsed ray is emitted from station 10, and the direction parameter and distance parameter are determined based on this pulsed ray.

[0107] Next, a second modification of the information processing system 1 according to the second embodiment will be described with reference to Figures 17 and 18. In the second embodiment and its first modification, a single light-emitting element 12 was used to function as a first light source and a second light source. In contrast, in the second modification, a continuous ray used to determine the value of the direction parameter and a pulsed ray used to determine the value of the distance parameter are emitted from separate light-emitting elements 12a and 12b.

[0108] Figure 17 is a schematic cross-sectional side view similar to Figure 10, showing a part of the configuration of station 10 according to a second modified example of the second embodiment. As shown in Figure 17, station 10 according to the second modified example has components similar to those of station 10 shown in Figure 10. However, station 10 in this embodiment has two light-emitting elements: a first light-emitting element 12a (first light source) and a second light-emitting element 12b (second light source). The first light-emitting element 12a and the second light-emitting element 12b emit different light rays, particularly light rays with different wavelengths, in substantially the same direction.

[0109] Figure 18 is a schematic diagram similar to Figure 12, illustrating how a predetermined space around station 10 is scanned by a ray emitted from station 10. As shown in Figure 18, the first light-emitting element 12a emits a continuous ray, and the second light-emitting element 12b emits a pulsed ray.

[0110] Furthermore, the positioning optical sensor 31 is configured to detect the reception of light rays of a specific wavelength. Specifically, it is configured to be able to detect the reception of light rays emitted from the first light-emitting element 12a, but not to detect the reception of light rays emitted from the second light-emitting element 12b. On the other hand, the distance measuring optical sensor 19 is configured to detect the reception of light rays of a specific wavelength different from the wavelength detected by the positioning optical sensor 31. Specifically, the distance measuring optical sensor 19 is configured not to detect the reception of light rays emitted from the first light-emitting element 12a, but to be able to detect the reception of light rays emitted from the second light-emitting element 12b.

[0111] In the information processing system 1 according to this second modified example, only the continuous light rays emitted from the first light-emitting element 12a are detected by the positioning optical sensor 31. Therefore, the direction of the moving object 30 relative to the station 10 can be determined based on the detection result by the positioning optical sensor 31. In addition, only the pulsed light rays emitted from the second light-emitting element 12b are detected by the distance measuring optical sensor 19. Therefore, the distance between the station 10 and the moving object 30 can be determined based on the detection result by the distance measuring optical sensor 19.

[0112] While preferred embodiments of the present disclosure have been described above, the present disclosure is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims.

[0113] For example, in the above embodiment, the processor 34 of the mobile unit 30 has a direction-determining unit 341, the processor 23 of the station 10 has a distance-determining unit 232, and the processor 43 of the main unit 40 has a position-determining unit 435. However, the processors 23 and 43 of the station 10 or the main unit 40 may have a direction-determining unit 341, the processors 34 and 43 of the mobile unit 30 or the main unit 40 may have a distance-determining unit 232, and the processors 23 and 34 of the station 10 or the mobile unit 30 may have a position-determining unit 435.

[0114] In the above embodiment, the image data generated by the main unit 40 is output to a monitor 5 installed in real space. From another perspective, the monitor 5 does not move even if the moving object 30 or the user moves. However, such image data may also be output to a monitor that moves with the user, such as a head-mounted display.

[0115] Furthermore, in the second embodiment described above, a MEMS mirror 15 is used for scanning to measure distance. However, scanning to measure distance may be performed without using a MEMS mirror 15, such as with a mechanically rotating LiDAR.

[0116] Furthermore, in the above embodiment, the position of the virtual object is determined based on the value of the direction parameter or position parameter of the moving body 30, and an image is output based on the position of this virtual object. However, different control or measurement may be performed based on the value of the direction parameter or position parameter of the moving body 30, in addition to the output of the image.

[0117] Furthermore, in the above embodiment, the station 10 performs scanning of the space in front of the monitor 5. However, the station 10 does not have to be positioned to scan the space in front of the monitor 5, and it may perform scanning of real space independently of the monitor 5.

[0118] In addition, in the above embodiment, the vertical orientation of the mobile body 30 relative to the station 10 is determined by the vertical angle of the MEMS mirror 15 when the light reception time is longest. However, the vertical orientation of the mobile body 30 relative to the station 10 may be determined based on the vertical angle of the MEMS mirror 15 at times other than when the light reception time is longest. Specifically, for example, an intermediate angle between the vertical angles of the MEMS mirror 15 at the time when light reception is first detected and at the time when light reception is last detected may be determined as the vertical orientation of the mobile body 30 relative to the station 10.

[0119] Furthermore, in the above embodiment, the horizontal orientation of the mobile body 30 relative to the station 10 is determined based on the timing at which light reception begins to be detected. However, the horizontal orientation of the mobile body 30 relative to the station 10 may also be determined based on other timings, such as the timing in the middle of the light reception period or the timing when the light reception intensity is maximum.

[0120] Furthermore, the mobile body 30 may be equipped with multiple positioning optical sensors 31 that detect light rays from different directions relative to the mobile body 30. In this case, the attitude of the mobile body 30 may be detected based on the orientation of the positioning optical sensors 31 that detected the light rays.

[0121] In addition, two or more stations 10 may be connected to a single main unit 40. In this case, each station 10 may be configured to detect only directional parameters without detecting distance parameters, and the three-dimensional position of the moving object 30 may be determined by triangulation based on the values ​​of the directional parameters detected by the two stations 10. [Explanation of Symbols]

[0122] 1. Information Processing System 5 monitors 6 Display screen 10 stations 11 Housing 12 Light-emitting elements 15 MEMS Mirror 30 Mobile Units 40 Main unit

Claims

1. An information processing system comprising a station, a mobile body movable relative to the station, and one or more processors, The station comprises a first light source and a two-dimensionally driven MEMS mirror, and while the MEMS mirror is being driven, it reflects a ray from the first light source by the MEMS mirror and radiates it into a predetermined space around the station, thereby scanning the space two-dimensionally with the ray. The moving body has a first light sensor that detects the reception of the light ray, The processor has a direction determination unit that determines the value of a direction parameter representing the relative direction of the moving body to the station based on the timing at which the first optical sensor detects the reception of the light ray during scanning of the space in the station, The station performs a raster scan by performing a linear scan in a first direction with light rays from the first light source multiple times in parallel, while shifting in a second direction perpendicular to the first direction. The direction determination unit is an information processing system that determines the value of a direction parameter relating to the second direction of the moving body, corresponding to the direction of the second direction of the light ray from the first light source when the light reception time by the first light sensor is longest during a single scan in the first direction.

2. The information processing system according to claim 1, wherein the direction-determining unit determines the value of a direction parameter relating to the first direction of the moving body based on the timing at which the light intensity received by the first light sensor changes to increase.

3. The information processing system according to claim 1 or 2, wherein the processor having the direction-determining unit is provided on the moving body.

4. During the operation of the station's MEMS mirror, a ray from the station's first light source is reflected by the MEMS mirror and radiated into a predetermined space around the station, thereby scanning the space two-dimensionally with the ray. The reception of the light ray is detected by a first optical sensor of a mobile body that is movable relative to the station, This includes determining a value of a direction parameter representing the relative direction of the moving object to the station based on the timing at which the first light sensor detects the reception of the light ray during scanning of the space at the station, The two-dimensional operation described above includes performing a raster scan, in which a linear scan in a first direction by a ray from the first light source is performed multiple times in parallel while shifting in a second direction perpendicular to the first direction. An information processing method for determining the value of the directional parameter, which includes determining the value of the directional parameter for the second direction of the moving body in accordance with the orientation of the second direction of the light ray from the first light source when the light reception time by the first light sensor is longest during a single scan in the first direction.

5. The information processing method according to claim 4, wherein determining the value of the directional parameter includes determining the value of the directional parameter relating to the first direction of the moving body based on the timing at which the light intensity received by the first light sensor changes to increase.

6. The information processing method according to claim 4 or 5, wherein the value of the directional parameter is determined by a processor provided on the moving body.

Images